Muse
“Muse” is a digital artwork / experimental study, exploring interactive 3D space and motion, combined with generative digital audio. I created “Muse 001” using a combination of tools and Javascript libraries, including Three.js, Anime.js, Tone.js, Blender, and Mixamo.
This project is best viewed in its own browser window and with headphones or decent speakers. You can interact with the 3D space using standard touch / mouse controls.
In a nutshell, the audio is composed with code and generated on the fly. I also included probability in the code so the audio has no set form as well as a very low file size. I then used the timing of the audio events to trigger effects in the 3D environment.
Unfortunately, one thing that didn’t work so well is that the motion capture sequence that I used doesn’t loop in time with the rhythm of the dancing. I’m looking forward to trying this again at some point and getting the dancing in perfect time with the audio.
Otherwise, this is the first example I have published along these lines, and I feel like I have only just touched the surface of what is possible here. One of the next things I would like to try is making the audio ‘spatial’, as in emitted from objects and positions within the 3D environment.
I’m also planning to develop tutorials and resources for things like combining Tone.js with Three.js and Anime.js, and integrating motion captures with Three.js. So if you’re interested in this kind of stuff please stay tuned. 🙂