DREAMSCAPES
Virtual Reality, Photogrammetry, Motion Capture
YEAR: 2019
LENGTH: 2.5 month academic project
TEAM: Salisa Jatuweerapong, Yasmine Liang, Mahnoor Shahid, and Nik Szafranek
My role included concept and narrative development (storyboards), visual development, Unity VR integration, 3D animation (assets and cameras), VR filming, and project management
HOW DO WE CREATE THE FEELING OF DREAMING?
DREAMSCAPES is an experimental virtual reality (VR) experience created by Salisa Jatuweerapong, Yasmin Liang, Mahnoor Shahid, and Nik Szafranek. This time-based film is centered around the concept of the collective dream and explores the conditions required to create a believable dreamscape. Traditional film editing conventions are explored within a VR space of photo-realistic environments to emulate the sensation of dreaming. Within this dialectic, virtual avatars and the connection between VR and dreams are also explored.
Topics: collective dream theory, film editing conventions, perspectives as a narrative element, digital double
Created with Blender, Unity using the Oculus SDK. Boards done in Procreate.
THE BEDROOM
Within a photo-realistic environment, how do we signal to the player that they are dreaming?
Solution: Instead of manipulating the visual realism, we play with the viewer’s perception of the environment in order to recreate the feeling of dreaming. One of the ways we achieve this is creating an "out-of-body" experience in the first scene.
We use perspective switches through the dissociation of the camera to the player, allowing the camera to moveindependent of the player at points. While the work starts with the viewer watching the scene from their avatar sitting on the bed (see header image), in a story beat the camera teleports to a the side of the room, leaving the virtual avatar of theplayer still on the bed (see image to right).
Composition in a frame that is indiscriminate—
While I originally started storyboarding like I would for film (boxes), I realized it was inadequate for conveying our VR scene. I switched to a set schematic paired with a net frame from the player view. This guide was important in making sure our POIs remained consistent in cuts (I diagrammed head angles) and acted as a roadmap for how we wanted the viewer to experience our narrative.
However, there's always a rebel: someone who ignores all our narrative cues (a guiding orb as a POI, flashes of light and bursts of movement) to look at the speck of dust in the corner.
Some VR films (ex Disney) compromise decide to sacrifice the illusion of choice altogether in favour of assuring the narrative delivery—in their film Cycles, areas outside of the intended FOV are greyed out.
In early tests of DREAMSCAPE, we found rebellion to be a persistent challenge. Our cuts were sudden, so the viewer would miss the action while they spent time re-orienting themselves to their space. Since we did not want to compromise on full agency, we experimented with staggering the action. With every big cut, the viewer would get 45 seconds or more of time to look around, satisfying their curiosity, before a visual cue would direct them to the POI and the action would resume.
The orb also acts as the connecting narrative thread throughout the 3 scenes. At the start of the film, the glowing orb leaves the player avatar, and the avatar loses it glow. As the player chases the orb throughout the different environments, the orb grows in size. In the final scene, it has coalesced into a fiery supernova with objects in its orbit, and while the player seems to be reaching for it still, in hindsight the gravitational force would not have let them drift away. The two reunite.
THE ENDLESS HALLWAY
To be or not to be? Discussions on triggers for scene transitions.
A piece of feedback we got was a discussion on ourchoice to not include controls. We had decided to cap the level of agency at field of view, but not any movement. Some viewers would have preferred moreagency in the narrative, such as deciding when to transition between scenes.They suggested control may reduce disorientation.
However, if we were to insist on the passiveness of the experience (as a characteristic of dreams), we could explore using sight targets and ray-casting to inform the film when to progress scenes. This would remove the issue of viewers missing the action or looking in the wrong place for a match cut, as the scene would only proceed once the view was correct.
THE VOID
The Digital Double — a real lucid dreaming experience
Real-time Motion Capture was the original plan for this project that we were not able to implement due to technical issues. The idea was for the audience to be wearing a motion capture suit while experiencing the film—and their motion data would be fed to the virtual avatar in real-time. Only, we would be able to manipulate the flow of time (and along with that their movements) within the dream world—this would allow us to simulate dreams temporally as well.
We also believe that this would have strengthened our exploration of the virtual self. While our current avatar is purposefully generic as to allow viewers to ascribe their own self to it, research shows that customized avatar increase player immersion. The inclusion of the viewer’s motion data would have personalized the experience for eachviewer. Exploring it in tandem with the out-of-body experience, we would also be able to selectively choose when the player could control their avatar—likely only in a third person view. In first person, the avatar would not be in their control—in fact, in a future iteration the avatar may be removed entirely while in the first person view.
Conclusions
Here are some of our key findings and how they influenced the final experience.
1. Hardcuts: These were the cornerstone of the out-of-body experience, but were, as warned, rather disorienting. In order to reduce the disorientation, we tried making sure the viewer would have know where they were in relation to the previous cut. For example, in the transition between Scene 2 and Scene 3, the stone staircase remains to give you context. The cut teleports you to the top of the staircase—also the end POI of the scene—but it is identifiably the sameone.
2. Matchcuts: These were also intended to lessen the disorientation of hard cuts. Itis a way of preserving visual continuity while breaking spatial continuity.
3. Slowpans and tracking shots: while these were somewhat effective, their symbolic movement was not as apparent as it were on film. Without an accompanying transition, they are rendered somewhat insignificant for our purposes. With careful control on the speed, and keeping steady level, we areable to create a successful pan shot.
(page still under construction)
All projects