Speach recognition will never totaly replace the need for hand operated controls anymore than software totaly replaced paper, or robots totaly replaced human manual labor. People will always need hand operated controls to take notes in meetings and enter confidential data while in public. It is also just faster and easier to hit an email button rather than tell your computer to check your email. The need for mobility hampers the adoption of voice recognition. Voice recognition drains batteries quickly which destroys mobility. That's why voice recognition is done in the cloud but being connected to the cloud also drains batteries. The thing that is going to replace the keyboard can be found at singlehandtextentry.com and abolishthetinykeyboard.com

Cool video, Brad. I'm waiting for visual overlay in my heads up display. It would make annotation of the real world and interaction with the real world data rich.

I think what you're looking at in twenty years is a re-imagining of how we interact with technology in the first place. We're currently constraining ourselves to figuring out how to better interact with today's technology rather than stepping back and asking how we'd really like to interact with our experiences and data. I'll coin the phrase here, "naturalized interfaces", in which technology simply augments the more natural way of looking at data. Companies like Oblong and the new interfaces that you're seeing this year and last at CES are still exploring the potential of what those different technologies can deliver, but in twenty years we should be moving past that. Your ultimate interface is a *blend* of all these different technologies in a way that feels incredibly seamless and natural.

So, maybe instead of doing spreadsheet analysis on a screen, I have a virtual display in front of me (it doesn't matter if it's projected or augmented reality onto my eyeballs – the question is whether it's for a shared or personal experience) and I can physically raise and lower inputs into my analysis either through a tangible object or haptic feedback – the point is that I *feel* the physical manipulation of the input. If I need more granular control, maybe I use the pinch or inverse-pinch popular on early multitouch devices like the iPhone. If I'm reviewing a couple of companies, I toss another business card onto my table which begins to pull the financial data from the cloud to add into my scenario… you get the idea.

I want an interface world where I'm thinking about my interactions in the most natural ways, not conforming to another technology out there. I want my interaction verbs to be the ones I learned in kindergarten — I want more of this product by adding to a pile of 'stuff' not product++ or sum(a13:a47).

My virtual world and physical world shouldn't be separate worlds to my perception.

I've been playing with Dragon Dictation for the iPhone the past several days. While the reco quality is higher than I was expecting (things have gotten a lot better since my last use over a decade ago), the impact it had on the content (words) I created was surprising. Typing this blog comment gives me parallel processing abilities and context that I don't have with speech (serial). The content I create when speaking is interestingly different from the stuff that comes out of my fingers on a keyboard. Keyboard is more concise and structured… voice is more verbose and loose. I've been trained over the decades, via email, to produce one way on the keyboard, and another w/ my voice.

My latest gripe about modern HCI is display (you touch on one dimension of it in this post). Right now, my wife and I are sitting in front of a large TV screen (passively watching a show), each w/ laptops in our laps, and iphones at our sides. If something pops up on my iphone that I want her to see, I have to hand her the phone (despite the fact she has a better screen sitting right on her lap). If I want to talk to her about our schedules tomorrow, we have to share a laptop screen, as opposed to just looking at the big screen 10' from us and having a communal experience around shared content. Display is horribly broken (as is input (keyboard/mouse)). Need a fix. Unfortunately, while we can all dream up solutions, some replete with technical architecture within today's state of the art, I fear the deployment of said solutions is going to be akin to auto-industry investment. So much capital is required to change the physical interaction world around us (as opposed to software alone) that relatively few swings at bat can be made.

Longing for highly iterative hardware I suppose.

Another MIT project is doing something like this- SixthSense at the MIT Media lab (http://www.pranavmistry.com/projects/sixthsense/index.ht...

I used to crew AWACS planes for USAF (as an aside, if you want to see a crappy UI, go fool with pretty much anything operated by DOD). There was an old story floating around that the first AWACS crew stations had keyboards with the keys arranged in alphabetical order so that folks who were whizzes on QWERTY boards wouldn't jam up the computer memory. I always thought that sounded oddly familiar, and then someone reminded me of the origin of the QWERTY arrangement.

The problem with any sort of gestural interface for the mainstream consumer: people are lazy. While the keyboard and mouse are outdated, but you can do a heck of a lot with them, with very little energy or movement. (By gestural, I'm excluding touch-based interfaces.)

I agree. These are just interim. Computer-Brain interfaces will be the true paradigm. This will change everything: the human condition won't be the same.

Once I have a little cash and credibility under my belt, I'll spend all my days working on this. Something will surely have gone awry if we can't achieve this in the next 60 years.

Imagine an early 80s vintage computer whose main job is to process a bazillion radar returns every 10 seconds while also dealing with 20 crew members frantically pounding in keyboard commands (you can't accurately describe the way a person uses a keyboard in a high-stress environment as "typing"), and the story is plausible.