The Beginning of the end. Pt 2
2 weeks ago we wrote a controversial article on the effect of HID on MIDI in the digital dj world. There are a lot of great points in the comments but most people missed the most important point of the article. To help clarify, here is part 2 of the discussion which explains what HID, MIDI and OSC are and goes into more detail about what is really at stake in this important discussion.
A Controller Dictionary
MIDI
“universal”
a standardized communication format that anyone software or hardware can use. If your software understands midi and I build a controller that sends midi then they will work perfectly together out of the box. Anyone else can customize the way they work with each other and it will always work well with future versions of any software.
OSC
“faster and more powerful but harder to implement”
a newer, faster communication standard that was meant to replace MIDI. OSC works the same way as MIDI and if adopted by 2 devices, anyone can customize the way they work together. The problem with OSC is that very few controllers send it and even fewer pieces of software support it, therefore it has not really caught on.
HID
“easy, powerful and very proprietary”
HID is a device type that allows any computer to recognize a HID USB device when its plugged in. Common HID devices include mice, joysticks, guitar hero guitars and now dj controllers. The HID standard is very open and only insures that any HID compliant device will work very well with any computer out there. Because its so open, only the software designed for that hardware will work out of the box. HID is easy to implement and capable of doing very complicated things but it eliminates most customization for the end user given its non-standard formats. Its also very difficult for companies like NI to support customizable HID control over their software.
Dj TT asked Friedemann Becker, the creator of Traktor and a digital dj pioneer why that is. He told us:
“HID is a non standardized protocal so each company sends out their own messages in a proprietary way specific to the device. Therefore the software developer must program an interface that understands the data coming from the device and splits it into info that can be used by the software” …”Midi on the other hand is fully standardized so the manufacturer is constrained to sending out data that Traktor can understand and support in the future”
So basically, unless Numark provides Traktor with the secret HID decoder ring for the NS7 then you can only configure it yourself using MIDI which is vastly inferior to what ITCH can do with HID. This goes beyond hi resolution controllers but complex ways that the software and hardware can work together to make your experience as a dj more fluid.
Party Like its 1999
This is not the future we are talking about folks. There are already 2 controllers on the market that tell the story better than I do; The VCI-300 and the NS7. Bottom line is that Serato could not get the performance they needed out of midi so they went to the more robust HID system. To actually use the HID data, other software companies need to spend time and money to program and support each controller manually. Even then, it would would never like the original software and you most likely would not be able to customize the controls. Instead, controllers will also send out midi for the custom programmers. Sure you can do the job that way, but trying to program midi is like building a boat out of scraps from the lumber yard instead of custom ordering the right timber from the start.
Whats to be Done?
- Push software makers to fully support hi-res midi
- Get the MIDI guys to get off their asses and finish the MIDI 2 spec NOW and not in 3 years as planned
- come up with a new standard that can manage to succeed where OSC failed
- Contribute to this poll and tell the manufactures what you really want
To clarify, the original article was not about HID replacing midi. It was about the emergence of HID controllers killing the (dj) midi controller market before it gets the chance to really flourish. Look at it this way, in a few years you will have the following options:
1) a bunch of controller systems that work really well out of the box exclusively with one software but are not customizable.
2) a few boutique controllers which are fully customizable but cant compete with the others in terms of performance features.
Its not hard to see which option will win the majority of customers over time. Eventually, some software manufactures may just decide to discontinue midi support entirely (Itch does not have it now). There is clearly a good reason to go with the fist option but its in everyone’s best interest if we find the right balance between both.
May 3rd, 2009 at 8:22 pm Quote
Contributed to the poll and think that OSC is still being greatly overlooked by both the software and hardware producers. HID is a step up, but still is inferior to OSC in the amount of information vs. speed. I think that the software should support as many standards as possible, and the controller designers should get together and either come up with a standard that will last for years, or adopt a standard that is in the works and use it exclusively. I love certain ends of propriety, but only when it works well with others and is still highly customizable.
May 3rd, 2009 at 8:42 pm Quote
Fully agree with article, contributed to the poll.
May 3rd, 2009 at 8:49 pm Quote
Contributed as well, only eleven votes so far.
People, let the industry hear your voice.
May 3rd, 2009 at 9:24 pm Quote
Contributed and agreed! If MIDI 3 will include downward compatibility!
By the way, Ean do you buy all those Lego-Figures :D? Great visualisations in my opinon :).
May 3rd, 2009 at 9:26 pm Quote
I like being able to create my own controls, but I honestly wish I didn’t have to.. I wish I could give Itch a try, but I don’t want to have to buy a VCI-300 to do so.
Basically, I want the MIDI supporting programmers to make better mappings for all the hardware out there.
I know that is a lofty request (because the possibility of all the hardware) but still its a dream I have:
Where all hardware (Vestax, Numark, Pioneer) will work with all software (Serato, Traktor), and switching is as simple of switching applications. And keep iTunes intergration too…
May 3rd, 2009 at 10:28 pm Quote
I agree with Nvidia.
We realy need open standarts.
If you say that your hardware works only with its software and vice versa – this pair MUST be very very good, the best! But usualy when it comes to locking, the product suks)
I trusted such a pair once. It was awful Torq. Usualy i had about 3 dropouts per gig, waiting a year for an update.
Its much better when you have an option. It stimulates the progress: manufactureres make better product, not better advertisement.
May 4th, 2009 at 7:00 am Quote
Took the poll. Amazing article as usual, Ean.
Well, I agree with everthing except the bit about Torq, NZ.
I had it fail me once, and that was user error within the first 72 hours of owning it. I now use it 3-4 times a week. Beat juggling. Sampling on the fly. Effects. No crashes. I use the Xponent and/or my VCI-100, depending how tweaked a set I want.
I have had more problems with my Traktor set-up, but I still love them both, and know that s#$t happens with each dj’s personal setup. So someone else probably had Traktor working right away with out a glitch. Or hated it and went to Live. You get the idea, yea?
Which brings me back to why we need to be able to customize our own. That’s the only way we’ll all stay fresh and be able to push our game beyond the boundaries of what the bar has been set to.
May 4th, 2009 at 7:20 am Quote
Thanks! Now I just need to find more excuses to use amazing photos of little lego men. They are just so cool!
May 4th, 2009 at 7:55 am Quote
Took the poll, thank you Ean for a follow up on a very controversial article.
May 4th, 2009 at 8:55 am Quote
I took the poll too.
1 question though.. I have a 4D and TSP, and all is working dandy.
I cannot imagine what more i need, beside a few program tweaks.
What will the new systems improve in practice?
May 4th, 2009 at 10:10 am Quote
I think the most appropriate solution would be (“thinking loud”, as we say in Italy, so I might tell crap or non-sense, did not reflect too much on this issue):
- on one side the software houses create a HID protocol to be used with their software for as many controllers as possible… so they will have to work
1. to prepare a HID protocol for a lot of controllers and provide their manifacturers with it
2. to know and learn what the possibilities and the limits of each controller are
- on the other side the manifacturers will need to implement on their controllers the protocol which was created by the various software houses
I think the job is pretty split in this way. The software houses’ part might be a bit more though…
May 4th, 2009 at 11:54 am Quote
Nice write up, that this pretty much covers the market at the moment as it stands.
It is really up for us the users to put pressure on both the hardware and software manufacturers to express just how much that we want open environments. Ideally that allow not only for the art form of djing music but also the art form of customising out own features, effects, audio routing’s – ala controllerism .
Building public awareness about this subject is also very important, the more people that know about this will help to put the pressure on software and hardware manufacturers by drilling into them what we want.
By the time HID would ever be supported by programs across the board the HD midi specification will be out – even if the slow progress on the midi specification continued! Then we will be pushing for the next god knows how many years for software and hardware vendors to push for this new much more superior and flexible standard. I personally think what is best for us is to make our push for this open HD Midi standard to help focus the resources on the most practical solution for everyone instead of this expensive stepping stone.
Those looking for short term instant gratification while the HD Midi specification makes its way to us should be pushing for OSC support and not HID support – OSC is customisable, fast, reliable and already has more support in programs than HID currently does. Most importantly it doesn’t require anyone to get together to form any kind of standard of communication – the money spent getting manufacturers together and putting together a standard would be much much much better spent poured into HD Midi !
May 4th, 2009 at 12:46 pm Quote
Non-cutomizable controllers will be one-hit wonders. People will all be doing the same styles and tricks and get tired of them quickly. They will sell OK at first, but they will be outshown by performers able to tap into the vastness of individual creativity. Either follow the path to customization or suffer the backlash of a giant watered-down DJ scene/culture. Haven’t we seen enough bubbles in this country? Manufacturers, please think in the long-term and keep this culture vibrant.
May 4th, 2009 at 3:06 pm Quote
…
I took the poll too.
HID is for one point terminal users, not truly interface network.
OSC with midi downgrade compatibility is the key.
This is the reason for supporting OSC.
Why to purchase an APC40 instead a Monome?
I prefer freedom to choice/play than closed solutions.
…
May 4th, 2009 at 4:16 pm Quote
of course, a lack of consistent sets of standards would be rather predictable in the world of dj equipment …
May 4th, 2009 at 6:22 pm Quote
I don’t like when everything works out of the box. Than you get people who don’t understand midi and/or dj program and ask stupid questions about it. Fully customizable controllers are the way to go!
May 4th, 2009 at 7:47 pm Quote
Thanks for the great feedback guys. I just want to mention someone had a great Idea in the previous thread to create polls like this that readers from all other sites can also contribute to. Many, Many industry cats read this blog so this poll IS a great chance for your voice to be heard. So help spread the word and share the post/poll with other blogs and sites.
May 4th, 2009 at 9:00 pm Quote
This will eventually be the best outcome… but it’s a question of when. Unfortunately, people aren’t going to wait for the MMA, so MIDI 2.0 may become obsolete before it’s released.
And THAT would be a tragic waste.
-KLH
May 5th, 2009 at 4:26 pm Quote
The reason I took the jump from DVS to fully digital is just that…the ability to program my controller, my way, to suit my workflow, and help mix the music in my style.
The possibilities created by being able to have a custom hardware interface, to me, will trump any sort of performance gains given by a locked in HID system. As the producer/DJ/performance artist lines are slowly blurred out of existence, I think the ability for each artist to mould his equipment and software to help them express themselves is the key to the continued growth of DJing.
May 6th, 2009 at 12:50 am Quote
The way I see it, between Midi, OSC and HID, OSC seem to be the coolest.
See, Midi wears a cape and a mask. That’s sooo 70’s. Pointing ears mask with a red and black suit! pleeeease! No wonder they are working on a makeover. Very old fashion, customizable, but old fashion.
HID and OSC are more modern, they have their space suits and guns. It will be hard to tell which one is better, but if you look closely, OSC has Big Ass lasers, while HID only has machine guns. Furthermore, OSC has a white suit vs HID green suit. We all know the good guys always wear white suits.
OSC will blast the crap out HID and Midi with those lasers any giving day!
May 6th, 2009 at 6:42 am Quote
well that pretty much sums it up! why didnt we notice that the OSC lasers are clearly more powerful?
May 6th, 2009 at 11:07 am Quote
One up for OSC.
As interfacing protocol, it has AUTOCONFIGURATION methods. It means that for lightweight customisation there will be autoconfigure button. Interestingly, good OSC controller(there is alwas cheap crap) can learn FROM software instead vice versa like it was with MIDI.
Traktor partially implemented OSC for 2 versions (v2 & v3) and now in “PRO” there is no mention of it. “The PROs” dont use OSC?
We have next gen. communication already at our hands and still there are ideas like “ow, we gotta make a run for hdMIDI”. That is insane because after hdMIDI is obsolete something like OSC will take over.
Why not just skip that unnecessary step!
May 6th, 2009 at 12:06 pm Quote
ok as mainly a dj and not as much into tinkering as most on this forum. I have a different look. I personally own the vci-300. It think is there is going to be One to one straight out the box then the software needs to step up and provide some customization inside that
May 9th, 2009 at 1:36 am Quote
well, I am able to get the best of all worlds by using Bidule, where you can use both HID and OSC (via LAN) to control any MIDI signal
and if you’re using Wormhole, you can even send audio between a mac and a pc, you can eventually plug everything you want in every way you want :D
May 10th, 2009 at 10:47 am Quote
I’ve been researching DJ controllers for weeks and have come down to two choices. Either the Vestax VCI 300 because of the hi res jog wheels (it’s pulse resolution is 4x higher than the VCi-100). My other choice is Reloop’s Digital Jockey 2 – Contraktor.
The Reloop controller is made to work with Traktor like Itch works with NS7 and VCI-300. So, I’m thinking for Traktor users looking for decent jog wheels, the Reloop Contraktor is the way to go. Go with the VCI-300 if the midi setup doesnt bug you. Before I make my purchase I’d like to make sure that the Reloop jog wheels are on par with the Vestax variety if anyone knows that answer.
May 12th, 2009 at 11:04 am Quote
Personally, I think there might be room for another option here. People like HID because it works out of the box. People like Midi because it’s customizable. What I’d like to see is a protocol specifically targetted at DJ software that both hardware and software manufacturers adhered to (so things would work out of the box), but which was also extensible for future proofing and customizable by the end user.
There’s a basic set of controls and feedback methods that exist in all DJ hardware/software solutions, and unless our paradigm of using decks and a mixer (whether real or virtual) changes dramatically in the foreseeable future, I think we’ll be stuck with that paradigm for a while. Here’s the basic set of controls we have and need across the board as DJs currently:
Deck controls: play/pause, cue, pitch bent, jog wheel, pitch, loop/cue controls etc.
Mixer controls: EQ, upfaders, gain, crossfader etc.
Browser controls: track select, track load, playlist select etc.
FX controls: FX select, paramater changes, wet/dry etc.
Output display: time remaining, stripe display, sticker position, track name, Vu meters, and button feedback.
So, what I’m saying is that if we could have all controllers send/recieve standard messages for things like stop/start/cue/jog dial/pitch bend/track name display/sticker position display etc. etc. etc. so things worked out of the box with any software that supported those messages. Each unit (input method from the user, eg. crossfader or stop button or output from the computer eg. track name display) would have it’s own unique ID, so that you could easily reconfigure each one to interact with the software differently from it’s original factory settings if you so desired.
I have a feeling a lot of this can be done by OSC already (I don’t know much about the protocol), but from what I’ve heard OSC can become very complex for the end user setup point of view. Perhaps if manufacturers could agree on a standard set of OSC messages for the basic controls listed above, everyone would be more willing to accept it:
The software vendors would be much more willing to implement OSC if it meant not having to create a HID interpreter for each controller released onto the market, and know that controllers released in the future would work just as well with their software as ones currently already supported.
End users would feel much more confident about their purchases knowing that they can plug it straight into their computer and that it will work out of the box without having to go through hours/days getting a logically and intuitive setup working, but with the ability to customize individual elements as and when they see fit if need be.
Manufacturers can spend less on R&D costs for proprietary protocols, whilst knowing that if they adhere to the standard, their hardware will work exactly as they intended regardless of the software being used with it.
Of course, all this is only taking into account what I see as the ‘ideal’ situation – none of this takes into account inter-company politics and the fact that companies form partnerships and make their hardware only work properly with specific software intentionally (to retain a degree of exclusivity not achievable with other hardware/software combos). In the long run, I think this behavior is only hindering the progress of digital DJ technology – customers should be free to use any controller with any software and be confident that it will work as expected and without issue right out of the box, but allow a high degree of customization should the user want that also.
Fus
May 12th, 2009 at 3:21 pm Quote
Rufus definitely has the right idea.
The software should be designed to work with open standards and the controller should supply a standard set of drivers that can operate with any software with an additional layer that allows us to customise.
If some new DJ software release is launched I don’t want to have to pay for a new controller in order to use this it and vice versa. It’s definitely in the software and hardware manufacturers best interests to do this as well because their success won’t be tied to another firm.
This problem is the same as any other software/protocol standards issue in its infancy like the Internet was back in the day. Eventually things will settle down, a standard will be adopted and you will be able to use controllers with different pieces of software, it’s as certain as Betamax.
May 25th, 2009 at 7:59 am Quote
The sort of proprietary solution providers are not standing still here:
http://createdigitalmusic.com/2009/05/24/novation-automap-ableton-live-clip-control-coming-to-the-iphone/