I just spent the last month collaborating with NMBU (Norwegian University of Life sciences), prototyping an educational app for biology students and then I finished it off by going to VRDC, a VR developer conference. A month long immersion into the immersive technology of VR. Here’s my report. But first of all:
Why I am stoked about VR (and you should be too)
VR and all its sibling technologies, AR (augmented reality), MR (mixed reality), is by some being fronted as “the last medium”. That is quite a statement to live up to. I have friends who are amazed by what is currently on offer with today’s technology, and friends who compare VR to a slight improvement to the much-hyped 3DTV’s that TV manufacturers desperately wanted us to buy in the early 2010’s. The arguments against VR is that we have abstractions that work very well on a 2D surface, and that bringing them into a virtual world adds very little in terms of effectiveness to accomplish the goal of the experience. What added benefit is there to ordering a plane ticket inside of VR compared to pressing buttons in a native mobile app? I believe that is the wrong way of approaching a new medium. There is no need to start moving experiences that we are solving effectively in old channels, just for the sake of doing so. We should rather identify what it is that works exceptionally well in VR and shape our ideas around the strengths of the medium.
If you’ve seen someone with a fear of heights don on VR gear and face their fears, you can easily see the effectiveness of the medium. Interestingly, the human brain easily understands that this is a make-believe world. The body, however, is fooled. Big-time fooled, to the point where ethics come into play: the medium is so effective at simulating the world that the reptile brain kicks into action. This means we need to work hard and test often to ensure we create comfortable experiences. It also means the medium has the power to create very powerful emotions. With great power comes great responsibility. Psychologists are doing great research to ensure that we can use the medium in order to better control this effect. PTSD is just one of the actively researched fields that have seen great success with using VR for exposure therapy.
Another strength of the medium is how intuitive it makes exploration of 3D design and the mechanics of motion within 3D-simulated environments. If you ever had the fortune of learning linear algebra in 3D-space with unit vectors and similar, you know how confusing it can be. Even IKEA-instructions can become a spatial puzzle when you have to translate from a two-dimensional instruction to a real world object. It seems like trivial examples, but imagine how costly a spatial design error becomes if it is a part of a civic engineering project. Allowing users that design objects for real world usage to enter virtual spaces allows for an intuitive study of form and function before real-world prototyping and manufacturing take place.
So as stated, I just came back from a ten day “VR tour” in San Francisco. Even though I’ve dubbed it a VR tour, and even though Donald Trump ended up winning the election, this Virtual Reality tour was in fact not virtual. The plan was simple, travel to the Virtual Reality Developers Conference (VRDC) and spend some extra days in San Francisco to get immersed in the VR scene. My first stop was at IBM’s innovation center. They held a meet-up in their cognitive computing group and by chance, this month’s meet-up was “using Watson in Virtual Reality”. The case was using Watson’s speech generation/recognition API’s together with the HTC Vive to create a voice-controlled version of Scribblenauts. In effect, you could just point anywhere and say “Make me a red car “, and that wish was your command. If you wish to experiment with this yourself, you can find the example here. The case for voice-interaction in VR is that your hands are free to interact with the world and that typing in VR is currently a bit cumbersome.
The conference itself was a curious mix of game developers , traditional entertainment, industry, healthcare, education and more. Most in common between all was being curious. Everyone at the conference shared the impression that what we are seeing, is the early days of a new medium. A lot of high-five encouragements and genuine interest in each other’s experiments was shared. There was a guy from Tesla interested in doing pre-manufacturing tests of the ergonomics of their new vehicles, people from healthcare wanting to use VR in end-of-life anxiety treatment for cancer patients. A provider of a smart helmet that uses AR to assist civic engineers in construction work. Experts in education that presented what they learned when doing pilot projects of using VR in schools. Game developers sharing their knowledge about how to minimise motion sickness in games that feature flying. Creators of hardware that have started collaborating with startups that are creating VR arcades/centres. Training tool providers that are creating a real version of the comedy medical game Surgeon Simulator to actually train surgeons. And many more things.
Showcased as well, was some interesting hardware. Currently, to get a simulated reality that gives you the feeling of actually being there, one needs to aim for something called “room-scale”. In essence, it gives your the freedom to move around in a virtual world, as if it were real. Turn around, crouch, jump, walk from one point to the other, grab and interact. The equipment needed for this type of VR is currently expensive and requires you to be plugged in with a cable to a beast of a computer. This is about to change. Qualcomm, the company that has provided the internals for almost all Android phones of the last eight years, showcased a reference design for a new headset. This headset had no cables and was not connected to a monster computer, but did offer the “room-scale” experience. From my experience with working on early mobile graphics (showcase software for GLES2.0 in 2008), it could be two years until we see reference hardware turn into actual products. Then again, since then, mobile innovation has accelerated, so it could be sooner. It is hard to predict the actual time slot.
When true room-scale VR is something that comes for free as a part of your mobile phone, or if the technology can be fitted in a glasses form factor, then it will be accessible to most.
Another trend that I noticed in San Francisco’s VR scene, is that investors are opening up their wallets. People are eager to invest somewhat broadly in order to build portfolios for the future. There is a feeling among the VR scene of optimism and dreams of discovering great strengths of a new medium. There is a belief that those who are starting early to develop skills and insight, will be the best at transitioning them to the improved VR technologies five, ten years further down the road. Much like the internet bubble, it’s quite obvious to see the strength of the technology and the potential impact it will have on the future, and it is at the same time quite obvious that not every VR company will make it big. The investors are planting a seed in every garden, hoping for at least one of them to turn into an orchard.
After the conference, I was lucky to find another VR-event to attend. This time, Samsung was the host. The event named “Hack Reality”, was a two-day hackathon at Samsung’s innovation lab. The first price was 10 000 dollars in startup capital as well as a bundle of Samsung equipment, 3D-cameras, phones , VR-googles and similar. One of the teams made a virtual school-bus ride that traveled throughout our solar system. Participants could ask questions to the teacher and fly away to the moon and beyond. Another team worked on exposure therapy for spiders. The most impressive , and the winner , was maybe a sign of what’s to come in the next generation of kids. A fourteen-year-old programmer created a virtual conference, multiuser , with room for keynote speakers and an expo hall. The icing on the cake was how you could go to a companies booth and pick up their flyers and info, and these documents would automatically get added to your dropbox for later study.
I myself worked on an open source “reference” implementation of a technique called VR motion blinders. It is a technique that reduces the peripheral vision according to the motion of nearby objects. If you are interested in checking it out, the unity project can be found here.
But what should one do with VR today, as a digital agency, as a company, when does it make sense to invest hours into VR? The answer to that I believe is now. Here’s what I think you should do. Brainstorm on how the technology could be used in your business/company. Spend one or two days to figure out if there are any interesting cases where it would make sense to use VR. Set aside a month to explore one of the ideas. You will gain understanding in a new medium, you will see the limitations of what’s possible today and you will be much better equipped to predict the future.
We at Agens just finished a month of VR experimentation in collaboration with NMBU (Norwegian University of Life Sciences). We worked on a prototype of new ways of doing education. The case we worked on was adapting the curriculum of protein synthesis, from biology textbook to room-scale VR.
If you have something you want to prototype in VR, or if you have use-cases that can greatly improve with VR technologies, we welcome you to get in touch. We would love to help you accomplish your goals in the virtual world as well as the real world!
And to finish it off, here are some pictures from my VR tour.
Using an external handheld monitor, tracked with a Vive controller to record gameplay footage. Quite ingenious.
Google showcasing their new Daydream platform.
A smart helmet for civil engineers, by DAQRI.
The fourteen-year-old winner of Samsung’s hackathon, who blew us all away.
Qualcomm’s reference hardware for a standalone untethered headset with room-scale tracking. A peek at the future.
From Samsung’s Hackathon. This team was working on spatial audio in VR.
Stickers and hackathons. Silicon Valley style.
Yes, it does look a bit goofy.
A prototype for exposure therapy created for the Samsung hackathon.