Making a virtual space of interaction with your face motion would be a great experience of virtual reality. Imagine that your computer screen became a space with depth that you can actually feel and move around in. There is a specific type of artistic feel within the video; a hybrid of movement and color that looks two-dimensional but feels three-dimensional.
LineWave 3D is an interactive computer application that transforms the user into a 2D representation of themselves within a virtual 3D space. This is caused by the neon sequence of lines that are continuously moving with the user while trying to wrap around objects.
The major way for users to interact with this program is by moving their faces. In addition, the user interface of the program in the presentation mode makes the program becomes easy to learn and use.
UI: Presentation Mode in the Max Patcher
The patcher will be automatically opened in the presentation mode that shows the entire interface for users.
Figure 1. A simple UI in the presentation mode.
There are only a few buttons, including a power toggle, a toggle to switch the display to full screen, and some buttons for changing the background music. In addition, it has labels next to these buttons, and also a paragraph of introductions that make user easier to understand how to use.
Behind the code: Programming in Max
Some elements that for developers to test and debug is hidden behind the presentation mode, which is the code that shows how this program actually works.
The platform is extended from our mid-term assignment, including Cody Bischoff’s “Sound Trip” and Junshu Liu’s “Positi-On”.
This application is based on the face-tracking technology. The depth perception is taken from my project “Positi-On” that used the face-tracking technology provided by the Max package cv.jit (computer vision for jitter).
The visuals and sound sequencing were taken from “Sound Trip”, with an addition for the artist, Saydi Driggers, with her song “Lavender Ghost”.
Including “Lavender Ghost”, there are 4 songs in the playlist that matches the style of our graphic, which enhances the overall experience with both visuals and sounds. Besides pressing the button, we also designed the keyboard input that user can press the number key 1-4 for changing the songs.
In addition, the more you move, the louder the background music becomes.
Construction Process
The 3-dimensional effect is made by using the value of face tracking. The x, y and z value represents user’s face position and the distance from the camera.
For making a 3D effect, our design for the interaction is, if the user moves their face to the left, the graphic will also move left, to make an effect that you can look the hidden part on the right. Plus, by using the face size value, if the user moves closer to the camera, the graphic will actually zoom out that looks like you are approaching the virtual space on the screen closer and closer.
The most important step is finding the relation between the face tracking value and the offset_x, offset_y and zoom_in/out value for jit.rota that actually making the “3D effect” for output graphic. I made some calculation and get the primary function that represents this relation.
Tips
- When using this patch, make sure that you have a camera connected to your Mac or PC, and the better light condition in your environment will enhance the face-tracking accuracy.
- While interacting with your face and the application, make sure to move slow enough to let the visuals and music process your coordinates. Also, remember to keep your face within the borders of the screen so that the camera can track your facial movement.
We believe that LineWave 3D is the best interactive art piece we’ve ever made. From the psychedelic nature of the line art to the intuitive interaction of the motion capture, we are taking you to the next dimension of virtual reality. Enjoy this patch and have a fantastic time in your interactive LineWave virtual space!
Skills
- Max/MSP/Jitter
- Final Cut Pro X
Special thanks
- Cody Bischoff is a designer and artist. He was a Digital Culture major with a focus on design and graduated in December 2017. You can check his latest projects from his design firm, Dream Weaver Design Co. Here are his company’s website and Instagram.
- We used the song “Lavender Ghost” by Saydi Driggers, an Arizona-based musician, singer, and songwriter. She is also our friend at Arizona State University graduated in 2017 with a Digital Culture major.
- Saydi is now part of TYCHE, a multi-genre duo band. You can follow TYCHE’s pages to check her latest album and events.
- You can also take a look at her original works under the artist name SAYDI from Apple Music, Spotify, and SoundCloud.
Copyright © 2016 Junshu Liu & Cody Bischoff. All rights reserved.
Notes
- We finished this article together as the transcript of the instruction video. The final article was partially modified.
- The instruction video was originally posted in December 2016. While it has editing flaws, I remade this video in February 2018, without any major changes and made it perfect.
POSTED February 20, 2018
UPDATED March 30, 2022
Ted (Junshu) Liu 
Whooooah
LikeLiked by 1 person