"Intermediary midi mapping" software *discussion thread* (Traktor mapping is a pain)

[zestoi]

Hey mate,
How's the new version working out 4 ya?

hopefully releasing a new version today with improved slicer-goodness

[zestoi]

You're biggest flaw is probably your runloop. You should try to find a way to use an even queue with no polling and async events. Try googling for realtime runloops, as most of the types you'll find are usually game oriented with UI runloops synced to FPS.

latest version to be released still runs in a main loop but an alarm is triggered from RtMidi that causes the loop to bail from it's 1ms sleep and process any messages. the only other way afaik with RtMidi is to process each incoming event in it's own thread with copious mutex locking. ofc another option is to not use RtMidi or make it work differently - tho RtMidi atm works by having one new thread to listen to each incoming midi port.

ideally i'd like to just process each RtMidi event as it comes in, multi threaded. the only reason for me not doing that now is that any calls to (and from) lua causes all worlds of pain and crashes. execution can ping ping between the C core and LUA several times of depending on whether the LUA code is creating events or sending out midi and/or calling extra funcs from C.

Secondly, if you're not in love with LUA, I'd recommend running away while you still can. I have to use it for my job, but it's a horrible language really. There's not enough cross platform frameworks and the OO frameworks always seems hacky to me. Since a lot of controllers share similiar concepts (pots, pads, grids, faders, leds) it's just easier to use a scripting language with better OO support.

personally i quite like lua - i don't see anything wrong with it. it's a very simple language to get into compared to most others and lets you define functions onn the fly ala javascript. i use jquery for most of my website coding - and lua lets you code in a very similar way - assigning dynamic functions to callbacks.

edit: i wouldn't say having some kind of OO framework would be an advantage for what midimasher wants the LUA config to do. it would certainly raise the bar for who could even use it. OO is fine in it's place - but that place isn't everywhere.

Also for your host app, you might want to look into C# over C++ if you plan to support OS X. The addition of interfaces and the fact that you can run the same binaries (if you're careful to split OS specific functions in a sane way) using Mono and .Net on Windows make help in the long run. MonoMacs C# support for the CoreMIDI/Audio apis is very thorough (I help port those as well ) and you'll get all sort of bonus in async thread handling that can be a nightmare to do cross platform. My current app for my employer is a cross platform realtime audio app and we've had a lot of success plugging in different audio modules for CoreAudio, Windows sound, alsa and port audio on linux. Interfaces really save the day with cross platform.

interesting - not likely anytime soon to stray away from C/C++. what i've seen of C# tho is that it looks quite good, similarish to java without all the annoyingness of java (damn... i hate java! had to code in java/jsp for about 3 years and if they ever wanted me to again they'd have to pay me a lot more )

there again maybe a midimasher2 could see the light of day in C# if i fancied it - just not had any real experience of C# so not an option for me at this point. i don't have enough free time from my work clients as it is to work on midimasher as much as i'd like to.

Then again, you could also do what I'm doing and use Max since it already handels MIDI in realtime across platform, and you can distribute the compiled patches for free. The lua interprator libs are really basic and compile without hassle on all platforms and you could probably easily write a Max C/Java interface to pass midi along, if someone hasn't already written this.

if MAX was free then sure but i don't fancy shelling out that kind of cash just so i can code a MAX patch etc.

i'm not worried about cross platform issues atm - my code has been mostly written so it should compile anywhere and the only libs i am using (LUA, RtMidi and hidapi) all support osx and windows. i should be getting a mac in a month or so will look at osx support then.

cheers for the info i'll be sticking with C+LUA most likely but am open to other ways of handling the main run loop - tho atm this latest version i have here doesn't seem to add any extra delays in.

some more complicated callbacks, like the one to enabled/setup various effects unit parameters can take around 0.4ms to complete. doesn't sound like a lot but i guess it all adds up.

the biggest issue (i think) i was having before this latest version was with with my slicer implementation enabled - since that had to add delays in between messages which then delayed the whole process (kind of like the audio glitches that used to come from traktor pro1 when sending lots of led updates to a launchpad)

it's not the absolute latency i'm so worried about, it's the jitter (the variance) and the actual "feel" of using it. if it "feels" laggy then ofc people won't want to use it.

having native virtual midi ports instead of using an external loopback device would help too - as then processing wouldn't be going via the windows midi stack twice.

[zestoi]

Secondly, if you're not in love with LUA, I'd recommend running away while you still can.

last bit of waffle - and my last post was already too big one thing i like about lua is that it's really a language for non-coders, which i think is important.

anyone, if they tried hard enough, should be able to add in some simple logic using lua as it's constructs are just so simple.

also - i don't see lua being the only/main front end to midimasher long term as most things people want to do should be possible with an even simpler config format, that would then in turn call internal functions or lua.

most of the time people only need to be able to do stuff like:

* route midi event X to output target Y with value Z

* toggle the input midi event

* when midi event X comes in send back event Y to light up the pads led

* any of the above but only when some other pad is held down or toggled

so that's what i'll be looking at sometime - simplifying custom configs for users while allowing them to use lua, or define custom lua funcs when what they need to do doesn't quite fall into that set

cheers for your feedback - much appreciated

[muffintop]

I'm about to build a serious ableton layout... Is there any possibility of a MM step sequencer anytime? I would love to have one ( or 8 )on a page with some basic controls at least.

I would also be interested in helping getting it to work if need be. Just would need a push or 2 in the right direction.

Thanks again for all of your hard work. Looking forward to making a logical layout for the LP in ableton

[zestoi]

I'm about to build a serious ableton layout... Is there any possibility of a MM step sequencer anytime? I would love to have one ( or 8 )on a page with some basic controls at least.

interesting idea, i had thought before it should be possible once this message queuing is in. just in the process of a major change in the way events are processed which should remove some latency - and then will get back to implementing that. what i had before for message queuing won't quite work now due to the other changes i've made, but it should be in for the next version which i want to release tomorrow.

we'd need midimasher to process midi timeclock, but apart from that the standard queuing stuff i'm adding, with some custom code should be enough.

I would also be interested in helping getting it to work if need be. Just would need a push or 2 in the right direction.

Thanks again for all of your hard work. Looking forward to making a logical layout for the LP in ableton

cheers i've been pretty busy the last couple of months - this weekend is the first time i've really had to do some decent work on it - been a while since i pushed out a release.

also will be fixing the lpd8 issue that someone pointed out the other week

[zestoi]

You're biggest flaw is probably your runloop. You should try to find a way to use an even queue with no polling and async events. Try googling for realtime runloops, as most of the types you'll find are usually game oriented with UI runloops synced to FPS.

cheers for the pointer which made me rethink the processing loop and i've just tested some code that goes back to how i originally wanted it to work.

for some reason i had no luck mutex locking before. either i was just being stupid (quite possible) or putting them in the wrong place and/or it was my use of pthread_mutex_lock() which i assumed would work under cygwin. i've changed to using native windows calls and it seems fine...

now my callback function gets called by RtMidi as soon as a message comes in, i lock/wait on a mutex and release when all processing has been done - nice and simple and none of that processing loop shenanigans or any extra delays...

does mean i need an extra thread now to handle queued up messages to be sent out in the future (stacked traktor messages needed a delay inbetween and/or arpeggios etc) but that's no biggie.

[zestoi]

Hey Zestoi,

I hope this *bump* will make u consider to do a new version, whereas the LPD8 will function properly

I've tried out an earlier version, and in that, I don't get the suddenly shut-down of Midimasher after loading the LPD8-config, so yeah, I guess indeed, there's a bug when it comes down to the LPD8.

lpd8 code is fixed, will def release it today, just need to do at least a bit of testing as so much has changed in mm this time round.

i forgot i had managed to cram in so many controls in that lpd8 mapping... i *think* it still makes sense?

pots: eq+levels for decks A+B
PAD mode: 7 transport type controls, 1 button toggles between decks A+B
CC mode: itch style slicer (if activated) else hot cues
PC mode: beat jumps (are limitted to what can go here)

maybe i'll do some graphics for the layout and create a new thread for this one.

the slicer "loop mode toggle" isn't uber important and could then be used for something else - and the PC mode buttons could be change, tho since the lpd8 doesn't send out a message on pad release not many things can use that.

maybe a 4beat slicer would make more sense for an lpd8? would free up some pads - tho would only be half as fun

[zestoi]

I'm about to build a serious ableton layout... Is there any possibility of a MM step sequencer anytime? I would love to have one ( or 8 )on a page with some basic controls at least.

as the delayed message queuing seems to be working so well i started some sequencer code...

messages can be queued at fractions of millisecs in the future but they're all checked approx once every millisec.

sequencer code needs to be slightly different as we'd want to be able to say something like "send X to ableton on beat Y each time". i guess we're only talking about a 8beat sequence here or 16 or 24 maybe? if it was 16 and u had specified something to trigger on the 1st beat of a 8bar loop - then ofc it would actually have to trigger on beats 1 and 9 of a 16 one. just want to work out how complicated this needs to be so it's flexible enough.

it now decodes the midi 'start playing' message which resets the current beat number and the midi 'timing ticks' to work out the current tempo. we get 24 of those ticks per beat which equates to something like 30ms between each 16th note for a tempo around 120bpm

so the new sequencer, once we've worked out what it needs (and doesn't need) to be able to do, will use all that stuff.

not sure whether i can just fire off the midi when we need to, or whether we would also need to send out midi clock too as then u can specify ahead of time that a midi message should be sent "on the next click" etc. will only know if it sounds tight enough when trying i guess...

[zestoi]

the calculated bpm (from the midi clock ticks) seems to fluctuate a bit from about 140.6 to 139.7 when i have it set to 140.0 in traktor. be interesting to see how it works anyway once we have some actual sequencer code in

this bit of code prints out the current bpm once a second. we need to reset the print_bpm value to OFF else the next ON event won't fire, so we queue up a value of OFF (0) 1 millisecond after the ON (127) one. as well as set() the functions send(), send_midi(), send_midi_raw() and create() all take an optional final parameter which is how many milliseconds in the future to execute it.

capture("_varchange", "print_bpm", ON, 0, function()
    print("bpm=" .. get_bpm())
    set("print_bpm", ON, 1000)
    set("print_bpm", OFF, 1001)
end)

set("print_bpm", ON, 1000)

good little test of the event queuing stuff anyway and is how auto-repeat controls could be implemented (tho will remove the need for needing to send an OFF event as that's a pain - just makes sense for variables, as if the value hasn't changed we don't want any code depending on that variable to be run)

[zestoi]

hopefully i haven't broken anything too royally but there's a new version if you want it: http://midimasher.djism.com/downloads/midimasher-20120202.zip

i think there's been some tweaks to the tsi since last time so you'll need to (re)load the tsi from the settings dir

this has a completely rewritten core for processing incoming midi messages, that should be faster, more efficient with hopefully less latency.

i've tested the launchpad and lpd8 configs - both have twitch style slicers, which now doesn't hold up the processing of other messages when it needs to stack messages for traktor that need a delay in between.

install info etc as usual is here and api docs here tho i haven't updated those docs yet with the extra 'delay' param that send()/create() etc can use

midi clock stuff is in - it's just not used by anything yet. but i suspect some kind of sequencer will be appearing this weekend... i'm really dying to know what will be in traktor 2.5 tho to really know what would be useful.

edit: first update in over 2 months... just took me a while to work out what/how to improve on latency etc