Discussion 1: The maturity of the computer medium

[Hannes Vilhjalmsson]

When you look at the history of various media for human expression, you notice that whenever some new technique appears, it gets used in a fairly unimaginative manner that reflects existing trends. Film is a good example where it started off being mere theater on a screen.

As a medium "matures", its potential is more fully explored and it develops a unique form of expression that exploits the possibilities. This gives rise to new disciplines and new industries. For example, it was considered a major leap for the medium of film when "cuts" were "discovered". Showing two things happening at the same time (by cutting between them) was a visual language that did not exist in the beginning (and people were skeptical of it). In film the whole language and discipline of cinematography has developed.

Computer can provide interactive experiences that challenge most of the techniques for expression that existed earlier.

Is the medium of interactive experiences provided by computer technology mature already? Where do you think we are in terms of developing the language of expression with this medium? How do you think it will develop?

Post your thoughts here before Monday the 26th and we'll review some of your findings in class that day.

[David H. Brandt]

While the interactive experiences provided by today's computers are quite amazing, my opinion on this subject is that the medium is still in it's infancy when compared to it's vast potential.

Computers have the ability to create infinitely complex and detailed simulations, while humans are highly evolved to interact with and comprehend a 3D reality. One would thus assume that combining the two would be a match made in heaven. Yet the interaction of humans with most consumer versions of virtual environments is typically constrained by the physical limitations of a keyboard, mouse and a miniscule 2D monitor. Even the high-end devices are clunky, nerdy, costly and inaccurate at best. The usability and usefulness of 3D virtual environments is thus severely hampered by the lack of common availability of truly plausible 3D I/O devices. Furthermore, the cost of developing a useful, commercially competitive virtual environment remains staggering for most purposes.

These problems are however obviously quite solvable, thus it is simply a question of when commercially viable (and socially acceptable) solutions become commonplace, not if. It may take years or even decades, but one day mankind will interface seamlessly with the computer and the net. True maturity will not be achieved by this medium until then, when the full magical power of virtual environments will be unleashed, explored and exploited.

Personally, I expect that these obstacles will be overcome within the next decade or so, and then the usual suspects will spearhead the technology among consumer applications. Probably we'll first see something like a "Wii 3D", providing a new generation of gaming solutions, as well as a stylish "Wearable Computing iPhone". With the abundance of 3D consoles available, the cinema and television industries will soon follow. There will be a boom in the entertainment industry as the new media once again makes content piracy unpractical (for a while) and all the old 2D content is rendered as obsolete, paving the way for remakes of previous content. The porn industry will boom further, achieving hitherto unfathomable heights in depravity. And then the true innovations will start to appear. Everyday data in all sectors of society will start to go 3D, including classical office applications such as the spread sheet and word processor as well as more abstract data such as server status information and road traffic data. If AI has achieved respectable levels at this point in time for consumer usage, we'll soon after see virtual secretaries and servants replace traditional applications.

And then one day, much sooner than we can ever imagine, we will find that virtual/augmented reality has become an integral part of our lives that we can no longer imagine living without, joining the ranks of a long list of everyday technologies that were once considered high fiction or even unthinkable: wireless phones, email, the internet, computers, microwave ovens, dishwashers, television, airplanes, automobiles, refridgerators, electricity, indoor plumbing, canned foods, mass printed books, ...

[gudleifur05]

Predicting the future has always been a tricky thing. In regards to answering wether interactive experiences provided by computer technology is mature or not the question can rather be answered in the future when we look back and compare the experiences nowadays to our current experiences then.

We always want more so in that sense interactive experiences haven´t matured enough for us. In fact I think they will never be fully matured.

Regarding the language of expression there could be a lot of possibilities and I think we haven´t reached very far in that sense. I can think of many ways to stimulate our sense, thus creating an interactive experience, such as highly developed body implants.

In the distant future I see that as a possibility, that is the stimuli of the experiences coming within the body itself instead of coming from the environment to our senses to grasp. In the nearer future I can see more advanced communication with computer through a specialized and unambiguous verbal language for instance.

The possibilities are endless, but deciding wether we really are satisfied or not is a rather hard task. Or is it...?

[Hlynur Sigurrsson]

With current computer technology we can create complex simulation that mimics real world objects or create one that could never exist in the real world. We can interact with these computer programs with input devices such as mouse, keyboards, light pens, etc.
The experience of these programs from the graphical point of view is mature but how we interact with them is what hinders us from being able to fully use these tools.

I agree with Davið that this IO problem can easily be solved by creating socially acceptable gadgets to interact with virtual environments, but how long that will take is another question. Personally, few years ago I thought that we would have much more powerful tools to interact with virtual environments by now. But we’ve come far and many of the programs that exist today are great but I think that we aren’t even half way there.

I think that this invention (interactive virtual environments) will fall into the same “lazy-category” as other successful invention in the computing area, that is “You can solve your problems in the comfort of your own home, without ever leaving the house”. In the future we will probably extend the idea of shopping online into a virtual store where you can see the objects and try them in a virtual environment.

It’s always hard to be the person after the first spokesman and it’s even harder when they they’re talk covered the whole subject, so again I agree with Davið that when the medium of interactive experiences provided by computer technology will hit maturity then virtual environments will be integrated part of our daily lives.

[hlynurs06]

Note to self: log-in before posting replies.

[thors]

The medium isn't mature, by far. For every new disruptive technology there's an acceptance period that may span decades. In computer graphics in the field of visualisation, most failures have been because the technology was cumbersome, unwieldy and expensive. The 3-D-helmets weren't something people were to use in general, due to size, weight, requirements and price. They should be considered a proof of concept only. The 3-D glasses (with switching LCDs) were uncomfortable, required specialised hardware and only worked with a very small subset of the software available. They were also rather expensive. New technology is coming along, such as micro-projectors, retinal projectors and HUDs. As with most technology, this new technology, if accepted by consumers, will be accepted by the younger generation first. It will only be commonplace once the generation that originally accepted the technology will be raising their own children, with the new technology as a „household item“, the technology will be commonplace and room for new grounds with a new generation.

As with the cinema, 3-D didn't catch on in the 80's. Partly because it was a hugely expensive technique, but also because the consumers weren't ready to accept it. I recall seeing a couple of 3-D movies way back when, and I also remember that the audience wasn't in any great numbers. The concept of immersion and dimensionality has gotten a wider acceptance now, and so customers are better suited for this medium in the „new wave“ of 3-D movies.

We have already proven (with Final Fantasy: The spirits within[2001]) that we can make full feature-films, completely computer-generated. There are already ideas airing about making the films customer-centric, basically scanning the facial features of the watcher, and replacing an actors face with the customers.

Several experiments have been made in the cinema industry. In the late 80's, there was a cinema in Kópavogur which lead the cinema industry in Iceland for a while. They exhibited all the latest „greatest“ technology like 3-D movies, „smelly“ movies (basically the movie stopped briefly, asking the viewers to scratch a numbered blot on a card, releasing aroma), and interactive movies (the viewers voted on how the movie should progress).

The 3-D part is gradually becoming more popular again, partly due to IMAX, and partly due to video games.

The „smelly“ part never caught on, probably because the lack of smell is a welcome thing during many movie scenes (Imagine walking through a rotting swamp, a slaughterhouse or being in the same room as someone breaking wind).

The interactive movies however did catch on, albeit in a disguise. Many of the DVDs available today have optional story lines. You press a button on your remote when an icon appears on-screen, and you have chosen an alternate path for the movie. This is also used in the interactive DVD-games which generally accompany children's DVDs. It's still a slow and cumbersome technology due to the relatively slow response rate of DVD players, but it will improve.

Retinal technology has yet to catch on. People are rather fond of their eyes, and thus rather sceptical when it comes to illuminating the eye with a laser-beam. If this technology is proven 100% safe, then it might catch on in a few years with the younger generation. It may then become commonplace about the time we hit the homes for the elderly...

There is a lot of room for improvement in every aspect. Graphics cards can barely create a believable reality in real-time. Currently there are ongoing experiments with real-time ray-tracing which will provide believable lights and shadows (something we still can't create with the common methods, like OpenGL). The number of cores in the GPUs is on the rise - the latest nVidia Quadro has a whoppin' 192 parallel processor cores. In only a few years, GPUs with over 200 cores will be the norm for desktop computers, and thus adding the capability of near-true realism in computer graphics.

Additionally, in visualisation, there are both a new line of 3-D plasma displays and the hologram projectors, both which have grounds to break in the market. The hologram projector was put to its first use at the introduction of the new Ford Mondeo a couple of years back.
www.youtube.com/watch?v=zG1RXsim9Ns

There are already smaller versions in use, like:

www.metacafe.com/watch/1065682/3d_hologram_projector/

And of course (although this one is for fun), this is the reality we'll have to suffer...

www.metacafe.com/watch/1283451/hologram_the_computer_of_the_future/



I think there's plenty of room for excitement, and plenty of room for discussion. I'll let those be my closing words.

[Birna Íris]

I agree with previous posts, that the medium of interactive experiences provided by computer technology is not matured at all. We are probably just entering this world but it is of course impossible to say. Generations to come could take a totally different path in these matters than we imagine - for example go back to being in very close relation with nature again. However it is much more likely that these techniques will evolve from here on.

There are a lot of bits and bytes flying around us already and my idea is that some day there will be equipment to "grab" some date flying around in the air and display the environment in thin air so that people will be able to interact with the whole body. As stated before there are a lot of limitations with todays IO devices.

Brain computer interfaces (en.wikipedia.org/wiki/Brain-computer_interface) will probably evolve more, first within the prosthesis industry. But this will open a lot of opportunities in other computer driven industries.

[david]

I would hope that the medium isnt mature. It's of course hard to say without knowing happens in the future though
We are still using mouses and keyboards to interact with the computer mostly although we have other means of input available they simply havent proven to be better.
Sure you can give the computer commands in speech or by touching the monitor (if you have that kind of a monitor) but atleast to me that is almost always slower than just using the mouse and keyboard.

In the future it would be interesting to see if there will be more developement of head-tracking like seen in this video: www.youtube.com/watch?v=Jd3-eiid-Uw

There you will the usage of a wii-remote to change where you are looking in a 3D enviroment. It looks pretty cool and the 3D seems to almost come out the TV. I wouldnt be surprised if some device would come out in the next years to put this in effect. Although 3D monitors do of course exist today and in some ways they imitate this effect.

But I guess I'll have to agree with alot of people here saying the most of the changes will be with the I/O devices. With the developement going on now it seems that multi-touch screens will be the new way to interact with computers. I would like to see more gadgets like the wiimote since I thought that was an interesting way to interact with the console but kinda poorly executed.

[Hrafn Þorri]

There are several things that play a role in the development of the computer medium and interactive experiences. The most obvious is the availability of processing power and how that power is distributed amongst developers. This is a factor that many here will know is progressing exponentially (see Ray Kurzweil's predictions, law of accelerating returns).

I believe computing is only moving out of an embryo-stage. As processing power increases so will the possibilities of interactive experiences—especially due to advances in artificial intelligence. Since I write a lot about that subject, I'm covering two interesting technologies instead that will bring a myriad of possibilities within 2 years.

Augmented reality (AR) which we are already starting to see crawl out from the success of cellphones-that-really-are-computers-with-a-phone-in-them. There are incredible interactive-experience opportunities in AR technologies—it could transform the gaming market, for example. The technology also has monstrous marketing opportunities although companies haven't realized but a fraction of their potential yet. As commercial- or open source efforts (e.g. Google Android) roll the ball faster there will be an inevitable race to develop cheap, augmented reality glasses/displays.

Wikitude debuted late 2008, the first AR application for general consumers.



Brain-computer interfaces (BCIs) are also on the verge of entering the market. Two companies showed consumer BCI systems at the Game Developers Conference in February '08 and of those, Emotiv Systems is the company that in all likelihood will market the first complete BCI for general consumers, called EPOC. The system is comprised of a headset and three software components; each of which measures a special range of activities: the emotional state of users (fear, anger, frustration), the user's intent to physically control objects on-screen (actions can be mapped to the keyboard!) and the third measures the users facial expressions (smiling, frowning, etc).

Emotiv EPOC brain-computer interface for general consumers

To briefly mention other things that I think will play a large role in the near future:

• Virtual reality systems—especially when combined with neural interfaces. Like augmented reality, this will make a second coming (first was around 1990) due to increased demand in the portable/wearable computer market
  
• Creative systems; systems that can invent and help us deal with solutions to problems
  
• World Wide Web moving to what's being called a web wide world; everyday objects becoming computerized and encompassing a server (I believe the classroom projectors at RU have a primitive server, for example).
  

To rehash the first paragraph: we are only at the beginning of computer developments. As processing power becomes cheaper and connectivity increases, we'll both be able to create more powerful simulations and have better techniques to utilize the power (a lot of today's power is idle a lot of the time). Lastly, intelligent systems won't only make interactivity more interesting and powerful, it'll help us create better machines & simulations faster; propelling the acceleration of development in a feedback loop.

[Stefán Freyr]

Hi, sorry for the late reply.

The question posed was: "Is the medium of interactive experiences provided by computer technology mature already? Where do you think we are in terms of developing the language of expression with this medium? How do you think it will develop?"

It's not a simple question to answer, first we have to define what we mean by "interactive experiences".

In some areas we've come a long way. For some applications, all you need is the mouse and keyboard. People have set up Christmas light shows, connected them to a web page and allowed people to control the lights through a set of switches on the page while observing the results on a webcam. Audience voting on a TV show is another example. These examples are most certainly interactive but of course they are hopelessly simple... but sometimes all you need is a pen and paper... I don't see myself writing an essay in a 3D environment... it just wouldn't make any sense. I assume most people have watched the movie Minority Report. I almost wet myself laughing when I watched Tom Cruise waving his hands like an idiot trying to scroll through some documents. I feel that people are at the point where they are struggling to find the applications for this kind of technology. We have a hint of what is to come, but our attempts of putting this not-yet-available technology to good use are, well, to be honest, somewhat ridiculous at times. Of course, we do have some good applications such as the 3D car dealership we saw in class the other day. That actually made a lot of sense. You could see exactly what you were ordering without it having to be manufactured.

In my opinion the adoption of an interactive virtual environment depends mainly on the following:

1) Does it make sense for the application at hand? Does it increase user performance? As I said, for writing a text file it makes no sense to use a 3D interactive environment. Interacting with your operating system in 3D doesn't make much sense either, at least not with the current ideology. There may be some chance of a paradigm shift with the current computer experience where a 3D environment would increase the ease of use but I don't see it myself. Examples of where 3D interaction could help might be many CAD applications (although CAD experts might disagree) such as architectural design and even mechanical design (although I think designers would prefer doing the actual design in 2D but the 3D might help with finding "bugs" and fine tuning the result).

2) Does it impose limitations or inconveniences to the user? This is a major problem currently. All the (true) 3D interactive experiences currently impose an impossible constraint on the user. Having to wear a cumbersome 3D helmet or glasses, wearing specialized gloves (another thing I found odd with the Minority Report solution, all this technology and they couldn't get rid of the frigging' glove?). Another type of inconvenience is the fact that people are lazy. Having to walk 100 meters to get your file is just not going to happen when I could sit at my desk and only have to move the mouse to do the same thing. It might be fun for a while in some games to have to run around with your gun but I think for the long run, lazy teenagers would prefer the point and click style games. Another major inconvenience with current technology is also portability. Packing a head mounted 3D helmet into your laptop bag is a major pain.

3) The portability issue brings up another point. What if you forget your 3D virtual interaction gizmo and you necessarily need to use that application on the road? There will need to be an acceptable bridging between the new and the old ways of interacting if 3D interaction is to be used for critical applications (not just games).

I would say that BCIs (as mentioned above) are an interesting development in this field but I'm not going to hold my breath for a properly working version of these things to appear. They also have a major drawback in being only input devices, you control things with them but the representation of your virtual world is still stuck in the same old 2D screen or cumbersome 3D helmet. Maybe the time will come some day where BCIs will inject senses into the user (ala Strange Days www.imdb.com/title/tt0114558/) but I think we'll have to wait quite a while for that and finding beta testers will be a pregnant dog!

Well, this contemplation has become way too long winded so I think I'll stop now. But the short answer is "no". We haven't but scratched the surface of interactive experiences with computers yet but people need to be careful not to go too far saying that this is the thing to come. It makes sense for some applications but not nearly all. Those who will mainly benefit from maturing this form of interaction is the entertainment industry so I have full confidence in money being spent on the problem and I think we'll see continuous major development in times to come.

[Eirikur Ari Petursson]

Hi
I think that we have gone very far with the interactive experiences but not clearly far enough if we compare it to hmmm... lets say the movie 2001, compare to the movie we have a very long way to go but what we already have i think is pretty cool stuff.

I think that everything is possible but don't see anything mature happening in the developing of my awesome holodeck i'm always waiting for, still it would be pretty cool if i could just go on vacation in my living room and could feel the sun's heat and such. Maybe in 1000 years or so.

[peter]

First of all, it is, of course, hard to judge what level of "maturity" a medium has achieved, since such a label can only be properly attributed when the whole picture of the historic development is evident. The medium computer, in general, can be said to have achieved some maturity simply because there is now a generation that is fully "computer-literate" and can transfer knowledge from one device or system to another one with relative ease. This means that there must be basic rules and concepts that are consistent throughout the various platforms.

Since the computer is unique in that it can be put to use for a virtually unlimited variety of purposes, I think that the maturity of its applications depends on the purpose and context. Some thoughts:

All applications the primary purpose of which is to support the user in creating, finding and organising information (office suites, specialised design tools, web browsers, media organisation tools, ...) have, in terms of their basic metaphors and paradigms, remained in a rather stable state for a while. They can "mature", I think, by improving their capability to understand ambiguity in human input and by providing easier-to-use interfaces. With the intense development of (multi-)touch-responsive interfaces for the consumer market, a big step is being taken; another big step would be reliable speech recognition that can be put to efficient use. Still, the more such applications are focussed on productivity, the less likely they are to stray from the classic 2d desktop metaphor. While I know of a brand of CAD-software (targeted at concrete and steel constructions design) that is mainly operated in a 3d view, the ease with which we have always used two-dimensional representations will prevent this category of applications to change radically.

A second category would be applications that are used in an art/design context. Here I can see a big potential for maturity in the sense that ever-new forms of taking advantage of what the computer does best are being explored. The tools for creating music, visuals and digital performances and installations for various arts often differ fundamentally in their approach. There is software that emulates traditional tools (such as the Adobe CS), there are applications such as puredata and vvvv that use a dataflow aproach to generating and processing sound and video, there are devices such as the tenori-on, a kind of programmable, experimental instrument... These tools usually take advantage of the (pseudo-)randomness a computer can generate and of the possibility to rapidly process and recombine large amounts of mediadata.

When we regard games, the situation is different again. Early games focussed on quick reaction and/or strategic thinking and by that just translated the elements of sports and board/card games to a new medium. Current-generation games often borrow heavily from the medium movie, in their narrative structure, their visual design, their soundtrack and their photorealism. Although there are games that are advertised to have an "open" world (Oblivion, Fallout 3, GTA 4), the basic structure usually still follows a script, just like a movie would, with the player taking over many of the protagonist's actions. MMOs, on the other hand, are often caught in the dilemma of providing scores for everything and of being too "game"-like while they could maybe offer more of a "play"-environment, without going all the way to the strangely uncaptivating character of Second Life.

I think that games and virtual environments, especially if they employ some kind of VR technology, have still not quite managed to fully use the potential that the technology can provide (or will soon be able to provide). Especially if they are to be used in a context that is not related to art or entertainment, these applications have to become more natural to use.

[haukurhaf]

The mouse and keyboard method is certainly really mature. It has to be - it's been around for what, 20 years? ;D

That being said, I believe that we've yet to see different input devices, but if you think about it, what could substitute a mouse and a keyboard for general everyday use? For gaming and such, it's a completely different story. Someone mentioned the head-tracking hack using a Wii-mote, I believe that will soon be more popular.

[haukurk]

When you ask “Is the medium of interactive experiences provided by computer technology mature already?” I would say no, not at all. Actually, I would have thought that mediums to experience virtual environments would have been more advanced already. The idea of the 3D goggles, with movement sensor gloves and 3D holograms has existed for decades but hasn’t been constructed fully for the general population. It’s kind of cool to see the prototype of the hologram that Þór posted earlier, I would definitely think that this is what we will see in the future (of course more advanced and more useful).

The general population uses only keyboard, mouse (etc..) to interact with the environment but not some high budget gadget and that is of course never the same experience that we get in the real life environment. I would say that it is very hard to predict when we could see more evolvement (mature) in medium for experiencing virtual environment, but it is a very exciting field to research and the possibilities are endless when it will be, as you can see in all of the ideas that everyone have been posting.

[eirikurn]

Everybody seems to be writing heavy papers on this subject here so it seems to be getting quite covered. Personally, I prefer thinking and predicting about software development than hardware development. In my opinion most hardware being developed to further the interactive experiences are really inconvenient and unpractical for daily applications.

• In recent future Multi-Touch will likely make a fair consumer splash as it can be made cheap and in some application it is plenty practical and user-friendly.
  
• From there, we'll maybe see augmented reality with visors to assist in daily routines.
  
• The biggest technological breakthrough will be with Brain-computer interface and especially sense injection, as that can provide the ultimate interactive experience and entertainment possibilities. However, it will take a lot of time for it to be socially trusted and acceptable.
  

But of course, Apple will lead the way in all this development, as they are always on the forefront on new inputs and technology, f.ex. with their latest announcement: