{"id":156,"date":"2006-07-19T23:55:40","date_gmt":"2006-07-19T21:55:40","guid":{"rendered":"http:\/\/blogs.bl0rg.net\/netzstaub\/?p=156"},"modified":"2006-07-19T23:55:40","modified_gmt":"2006-07-19T21:55:40","slug":"processing-vj-application","status":"publish","type":"post","link":"https:\/\/blogs.bl0rg.net\/netzstaub\/2006\/07\/19\/processing-vj-application\/","title":{"rendered":"Processing VJ application"},"content":{"rendered":"
\"Alt<\/div>\n

I wanted to have some sort of visuals going on for my next gig (which will be detroit-like techno improvisation
\nand maybe chillout-dub stuff at this party<\/a>), searched a bit on the net and found all those
\nbeautiful processing sketches, which I could modify to sync to the MIDI coming out of my sequencer. This blog
\nposting will be a bit chaotic like all my posts, and sprinkled with code, but the main point is: I can load multiple
\nPApplets, sync them to the incoming MIDI in a simple way, and control everything out of the Ableton Live applicaiton
\nI use to play my music. You can basically take most processing sketches off the web, modify them slightly, and
\nrun them in the VJ application. You can get the code at svn co svn:\/\/bl0rg.net\/processing , or the (debug version) at
\nhere<\/a> (run with java -jar Polymaniac.jar –standalone). This is a kind of alpha
\nrelease, it has not been tested really, but I will
\nin the next few days, and keep updating blog postings, and it’ll have to work until the 5th august anyway. The posting is
\nnow about how the application’s internals, and a kind of reference for me as I keep forgetting everything.
\nWhen you start the application using the –standalone switch, you can use the keyboard keys (qwerty layout) to trigger
\nmidi events. Top row is DRUM notes, middle row is BASS notes, lower row is EVENT notes). A few test applets are loaded, but
\npeek inside the setup() method of Polymaniac.java to load your own. I’ll add a dynamic ClassLoader in the next few days.
\nYou can switch to fullscreen using “f”.<\/p>\n

<\/p>\n

I first started to modify simple sketches, adding promidi support, but this quickly turned into a bigger endeavour
\nas I started to refactor code and make abstractions and stuff. The application is now called Polymaniac and is a standalone
\napplication (not started from the PDE). I quickly switched
\nover from PDE to eclipse for my development. The goal of Polymaniac is to be able to load existing sketches, and run
\nmultiple instances of them (either on top of each other, or as background, etc…). Furthermore, specific methods
\nout of those sketches can be controlled by MIDI (either using MIDI notes or MIDI controller data).<\/p>\n

\nSo for example you can write:\n<\/p>\n

\nMidiApp baumApp = new BaumApp(this, midiHandler, baumReflection.getDefaultMappings());\naddForegroundApp(baumApp);\nmidiHandler.registerMidiAppNote(baumApp, \"addNode\", \"removeNode\", MidiHandler.MIDI_SYNTH_CHANNEL);\n<\/pre>\n
\nto create a sketch from the class BaumApp (a straight version of traer’s physics examples), add it as a foreground
\napplication (drawn on top of background applications), and map the method “addNode” of the newly created app
\nto any NOTE ON event on the midi synth channel (hardwired for my needs), and the “removeNode” to corresponding
\nNOTE OFF method. So playing a note will create a new node, and remove a node when the note stops.<\/p>\n
\"Alt<\/div>\n
I started by hardwiring all the MIDImethods, but now I have a central hub (called MidiHandler) distributing incoming
\n events to registered
\nsketches. You can use java reflection to set up all this dynamically, and my going goal for the next version of
\nPolymaniac is to add a GUI with MIDI learn functionality (tweak a knob, click a sketch parameter, the parameter is now
\nassigned).<\/p>\n
The most complicated thing about Polymaniac is actually how the usual order of events of a processing sketch is modified.
\nFor now only applets using draw() are supported, and there may still be weirdo things going with certain renderings
\n(I have just tested a few sketches). The main application class Polymaniac is actually an instance of PApplet. It creates
\nan instance of MidiHandler to handle incoming midi notes, and sets up just like a normal sketch by calling setup(). However,
\nit keeps a list of foreground and background apps, which are instances of MidiApp and kept in the package bl0rg.vj.apps.
\nMidiApp is a special version of a PApplet. It draws offscreen. Actually, a MidiApp uses the PGraphics context of the polymaniac
\napplication to draw. However, this may not be desired in certain cases in which the sketch depends on the screen not being
\nredrawn (for example particles leaving trails). In this case, we must create a special PGraphics for the class, and copy the frame
\nof the MidiApp on top of the main frame of Polymaniac. This is done by calling setBuffered(true). This is used in the demo
\nLineApp for example, taking from processing.unlekker.net . setBuffered(true, RENDERER) may also be used to switch the renderer.
\nThe main Polymaniac renderer has to be JAVA2D for now, but you can add P3D applets (like for examples CirclesApp).<\/p>\n
Setting up parameters works by either modifiying the default mappings of classes at their instantiation, like for example<\/p>\n
MidiApp baumApp = new BaumApp(this, midiHandler, new MidiAppMapping[] { \n\t\t\t\tnew MidiAppNoteMapping(\"blabla\", \"\", MidiHandler.MIDI_DRUM_CHANNEL, -1),\n\t\t\t\tnew MidiAppNoteMapping(\"blabla\", \"\", MidiHandler.MIDI_BASS_CHANNEL, -1) });\n\t\t\n<\/pre>\n
\nwhich will create baumApp with mapping blabla() to notes on the drum channel and the bass channel. Or later on by calling
\none of MidiHandler.midiRegister methods. I defined 4 channels on the midi device, but you can pretty easily change this (I’ll
\nadd switches and configuration parameters for that for the first real release). This way, you can use midi clips in ableton to
\nspecifically trigger visual stuff (using the MIDI_EVENT_CHANNEL), or by using the running midi clips (drums, etc…) to modify
\nspecial parameters. For example, each bass drum kick can switch the visible MidiApp, so you can have short subliminal animation
\nframes on the beat. A lot of cool things are possible this way, I haven’t began to really think about it, but the first impressions
\nwere awesome.<\/p>\n
\"Alt<\/div>\n
You may have noticed that this comes with its own version of processing.core, that’s because I had to modify the existing
\nrenderers to be able to render a transparent backgorund, in case a buffered MidiApp wants to only add foreground elements. I
\nhad to modify the colorspace of the P3D model, and to add real transparent backgorund drawing in JAVA2D. I also added a quick
\nblit function for offscreen MidiApps in the JAVA2D renderer. Didn’t implement it in P3D yet.<\/p>\n
Another thing which doesn’t work yet is looping processing apps (not frame based), I’d have to start a new thread for each added
\napplet, so that will be added in the next few days too.<\/p>\n","protected":false},"excerpt":{"rendered":"
I wanted to have some sort of visuals going on for my next gig (which will be detroit-like techno improvisation and maybe chillout-dub stuff at this party), searched a bit on the net and found all those beautiful processing sketches, … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17,19],"tags":[],"class_list":["post-156","post","type-post","status-publish","format-standard","hentry","category-computerstaub","category-english"],"_links":{"self":[{"href":"https:\/\/blogs.bl0rg.net\/netzstaub\/wp-json\/wp\/v2\/posts\/156","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.bl0rg.net\/netzstaub\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.bl0rg.net\/netzstaub\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.bl0rg.net\/netzstaub\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.bl0rg.net\/netzstaub\/wp-json\/wp\/v2\/comments?post=156"}],"version-history":[{"count":0,"href":"https:\/\/blogs.bl0rg.net\/netzstaub\/wp-json\/wp\/v2\/posts\/156\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.bl0rg.net\/netzstaub\/wp-json\/wp\/v2\/media?parent=156"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.bl0rg.net\/netzstaub\/wp-json\/wp\/v2\/categories?post=156"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.bl0rg.net\/netzstaub\/wp-json\/wp\/v2\/tags?post=156"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}