Trip The Light Fantastic
This project uses a combination of different methodologies and software. Initially, the code is written and compiled in C++ using Open Frameworks and the OfxMaxim add-on. Low-poly meshes have been downloaded from the internet, altered in Blender, and then uploaded to Open Frameworks.
Once the mesh is in Open Frameworks, the fun begins. I have used FFT analysis and beat onset detection to alter the appearance of the 3D mesh. Both of these procedures are computationally expensive. This fact influenced decisions further down the development chain. I used onset/beat detection to change the size and movement of the mesh I worked with for each song. The amount of response can be adjusted in a GUI for easy access- then hide the GUI with a key press when ready to screen capture. The main issue was that I wanted to change the colour according to a range of frequencies using FFT analysis.
This is actually fairly straightforward. If you want to know the details, message me, and I will talk you through it. The problem was that if I used vertex shading on the mesh, the code ran so slow as to be useless. It was simply too expensive. I chose instead to change the colour of the lights. This has the effect of changing the appearance of the paint on the mesh. The mesh is not changing colour. The lights are changing colour. This is much faster, and it was a successful solution.
I wanted to move the work into a 3D environment. I used the easy cam feature of Open Frameworks to film the activated mesh from different angles. The other videos were loaded into the game engine Unity. At present, the work is running on Google Cardboard. This is very much a work in progress-lots of potential for more development as time allows.
Once the mesh is in Open Frameworks, the fun begins. I have used FFT analysis and beat onset detection to alter the appearance of the 3D mesh. Both of these procedures are computationally expensive. This fact influenced decisions further down the development chain. I used onset/beat detection to change the size and movement of the mesh I worked with for each song. The amount of response can be adjusted in a GUI for easy access- then hide the GUI with a key press when ready to screen capture. The main issue was that I wanted to change the colour according to a range of frequencies using FFT analysis.
This is actually fairly straightforward. If you want to know the details, message me, and I will talk you through it. The problem was that if I used vertex shading on the mesh, the code ran so slow as to be useless. It was simply too expensive. I chose instead to change the colour of the lights. This has the effect of changing the appearance of the paint on the mesh. The mesh is not changing colour. The lights are changing colour. This is much faster, and it was a successful solution.
I wanted to move the work into a 3D environment. I used the easy cam feature of Open Frameworks to film the activated mesh from different angles. The other videos were loaded into the game engine Unity. At present, the work is running on Google Cardboard. This is very much a work in progress-lots of potential for more development as time allows.