I Want More Information, Not Less

I spent the last two days at the Defrag Conference.  It was awesome on so many levels including the content, the venue, seeing a bunch of great friends, and meeting a bunch of new people.

The conference originated out of an email exchange that Eric Norlin (the amazing guy who puts on the Defrag and Glue Conferences with his even more amazing wife Kim) and I had as a result of a series of blog posts that I wrote in 2006 starting with There Is A Major Software Innovation Wave Coming and Intelligence Amplification.

Over the past three years there has been an incredible amount of innovation around this theme (which we originally called the Implicit Web.)  While lots of it is still messy, sloppy, or ineffective, that’s just part of the innovation cycle.  A consistent discussion point at Defrag was “how to deal with this overwhelming amount of information” – there is no debate about the (a) amount, (b) need to deal with it, or (c) value of dealing with it.  However, a lot of the subtext was that there was too much information and we needed better ways to deal with it.

I agree with the conclusion, but not the premise.  I don’t think there is too much information.  I want more.  More, more, more, more, more.  MORE.  I don’t want to stop until I have all the information.  MORE!  You can’t give me too much information!

I don’t believe the issue is too much information.  This is an independent variable that we can’t control.  For the foreseeable future, there will be a continuous and rapid increase of information as more of the world gets digitized, more individuals become content creators, more systems open up and provide access to their data, and more infrastructure for creating, storing, and transmitting information (and data) gets built.

Yeah – I know – that’s obvious.  But there are a few ways to approach it.  My desired way is to accept the thing you can’t control (more information) and drastically improve the methods for consuming it.  I spent the better part of two days having this thought over and over again. 

By the end of the second day, I’d decided that my original premise was correct – there continues to be a huge innovation wave in software that addresses this.  And we are just starting to deal with it.  And while software is at the core of it, we’ve learned an enormous amount in the past few years about the power of people to help curate it, both directly (by doing things to it) and indirectly (by software interpreting the broad signals of what a large number of people are doing to it.)

The user interfaces – and user interaction model – for all of this stuff still sucks rocks.  And I love things that suck, because that creates huge opportunities for innovation.

  • very true. I think handling, storing and accessing large quantities of data faster is gonna be key. But I agree, we need more information, only more relevant information. Tools are not there yet to bring relevancy out of the information quickly enough, but the information itself is lacking.

  • It's not really an argument about wanting more or less. I want all the information that I would find interesting and nothing else. It's an Information Retrieval problem, which means there are always 2 criteria that go hand in hand – precision and recall.

  • Yup – succinctly said.  Totally consistent with my notion of “don’t focus on the thing you can’t control”.  However, I don’t think it’s just information retrieval – that’s a tiny part of it.

    • Agree it is more than Information Retrieval. The other side of the equation of finding relevance information is the User. The next move would go beyond the search queries or any other kind of explicit user intent indication, but to implicitly understand user's interests and deliver information that is relevant to this particular user. Relevance should be defined per user, not just by query terms. Explicitly personalization such as subscribing to twitter feed, create an iGoogle page, etc, is time consuming and results are often non-satisfactory. A ton of work has been done around finding relevant information around query terms via Search. There is yet to be a broad solution to deliver relevant information to users automatically. Would that be the next big thing?

  • I would agree that I want all of the relevant information I am looking for but their is plenty of opportunities for innovation around:

    Finding relevant information
    in a timely manner
    and in a format that is usable

    Too often I know the information I need is available but I cannot locate in a timely fashion using current tools or the information is not in a retrievable or easily digested format.

  • This is a stuff problem. You have lots of stuff – information – and a finite container (imagine a closet) – time & attention – into which you can shove it.

    As you talked about the user interaction model sucking yesterday, I wrote a quick note & showed it to Seth. On it was two words: Temporal cognition.

    This sums it up for me – I want to be able to grab stuff from my closet whenever I need it. I don't want to have to stand near my closet all the time waiting for people to deliver new stuff.

    I just want it to be there, already stored. Labeled & in chronological order, so that I may come back as frequently as I wish & find my stuff. I have very specific ideas on how this both looks as a user interaction model, as well as how it would work technically.

    • The discipline of Library Science is concerned with how best to organize information. I suspect that there's a way to apply the already-known best practices for Library Science to the information collection.

  • I saw the note – great phrase. I expect you will see the phrase temporal cognition appear in future blog posts.

  • Pingback: Tweets that mention I Want More Information, Not Less -- Topsy.com()

  • Pingback: Tweets that mention I Want More Information, Not Less -- Topsy.com()

  • The world is as it is filled with infinite information and due to easy digitization and web it's becoming easily available to us. It will be only foolish to say that we need to stop or mitigate that. The only problem is how to surface the information which is valuable to us at a given time or reach to the information which interests us improving our day-today life.

  • There's also a bit of an arms race going on, as exemplified by Google and SEO. When a new search algorithm comes out, it can lead to a step-function increase in information relevance and ease of retrieval. Over time, though, the entire world starts to work on gaming the system, and my highly subjective view is that vanilla search results are slowly getting worse again.

    This issue applies across all media. I want what I want, but marketers and publishers want me to see their stuff, even if it's not always a good fit. Part of this can be solved by better targeting and matching, but there will always be somewhat of a disconnect in motives. I suppose it'll be fodder for an ongoing cycle of innovation on both the user and marketer sides of the equation.

  • The dynamic between push and pull also plays into this.  When I go to Google and “Search” I’m behaving one way.  When stuff comes into my inbox (or RSS reader, or Twitter client) I’m behaving another way.  It gets really interesting when you merge the two.

  • There's also a bit of an arms race going on, as exemplified by Google and SEO. When a new search algorithm comes out, it can lead to a step-function increase in information relevance and ease of retrieval. Over time, though, the entire world starts to work on gaming the system, and my highly subjective view is that vanilla search results are slowly getting worse again.

    The same thing happens in media in general. I want specific information that I'm interested in, but marketers and publishers want me to look at their stuff. This problem can partially be solved with better targeting and matching, but there will always be a bit of a disconnect in motives. On the bright side, I think it will fuel a longer-term cycle of innovation on the consumer and marketer sides of the equation.

  • rockford

    and this makes money how?

    • The companies that create solutions for the problem have a very wide variety of ways to make money.  They are – after all – just software companies when you reduce them to their bare essence.

  • Shyam Kapur

    To put it quite bluntly, you are a genius. This is the best post about this subject I have seen in all my life. Yes, all my life and I am not that young anymore. I have worked for over two decades on the problem of how to get the most out of information, the more the better. My latest creation TipTop, a semantic, social, real-time search engine on tweets, is now available in a beta version at http://FeelTipTop.com Please give it a try. Also, check out our special, dynamically changing promotion for Thanksgiving that we launched today at http://ftt.nu/thanksgiving

    • Thanks for the kind words.  I’ll play around.

  • I think there are software solutions but also editorial solutions – huge value to come from perspective and view point. Eg. Drudgereport – sifts and aggregates OTHER information around a view point. Techmeme does this automatically – algorithm with view point.

    In addition to tech innovation there is huge value in editorial innovation. The small town newspaper could be the sifter/aggregate for helping me make more sense of more info.

  • There is always a tension between 'give me more' and 'give me less' in any information service. At the beginning you want to expand your realm; later you want to constrain it – to find the interesting bits.
    People make the best filters – a key thing I keep going back to is how good our brains are at associating trust with people, and having their photos next to the information calls that trust code in our brains. We all look at a lot of thinsg and pass on a few; we all see different subsets of the web; together we sieve it for meaning and interest and pass it on to each other. We've done this with blogs for years, and now we do it with twitter and all the other services shown off at defrag.

    The point of the open social web standards I was advocating at Defrag is that by putting this information in common forms, and having clear ways to share it both publicly and selectively, we can apply our existing social filters, and make it easier for new applications that make more sense of these flows of information to be developed without having to be bound to multiple companies different formats and policies.

    • Having photos next to the people to help with the trust association is huge – I wrote about it a while back but don’t feel like searching for the post <g>).  But – that’s right on the money.  The work you’ve been doing around open social web standards is great.  We’re living in a world of too many custom chunks of code to translate and combine common data elements and it seems to be getting worse, not better.

  • hey brad- tks for the comments.

    I'm struck by the idea that you raise (and that i've been chewing on for a few days) that when we "started" digging in at defrag we thought that the technology would be the filter, but as we get further into this, we see that the technology is creating platforms of connected/networked people that are currently a much more effective filter.

    That's probably one of my big takeaways from the first 3 years of defrag (yea, it only took me three years to get to that ;-)…..that and what I mentioned at defrag about "open follow" architectures seem like hugely important philosophical underpinnings for whatever it is we're calling this "defrag problem set" (implicit web, intel amp, etc) these days.

    I'm kinda figuring that in another 3 years we'll learn another 2 or 3 big things that we didn't anticipate….if i had to pick the big areas that are really untapped opportunities: BI, analytics, UI/interface.


    • I think curation, UI, and analytics are biggies.  I think next year should be organized more deeply around specific forward looking concepts, rather than “the now.”

  • People are discovering polymaths…and discovering that they are polymaths themselves.

    RSS has allowed me to read blogs on Frank Zappa, QRcodes, and Chinese domain names…..go figure….

    balance between the Real time web firehose……archives…..search…and personal curation …….fused with mobile access…

    That is the zen I am seeking…

  • That is a neat Zen!

  • Edie Frederick

    Speaking of information retrieval from this relevant thread, where is the RSS icon?

  • Edie Frederick

    Now I see it, but it follows the overall blog rather than this thread. Paul Kedrosky's Infectious Greed has both RSS services. Nice.

    • You should be able to subscribe to all comments at the bottom under the “Post a new comment” box.  Simply choose “Subscribe to” and then select “All new comments” in the dropdown box.

  • There is a good (and long) piece on social interaction design from Gentry Underwood at IDEO at

    The interaction models are awful now, for sure. (The FB parody YouTube video is great, if you haven't seen it.) CSCW research has shown that context is so important in human-human interactions that, as a practical matter, design rules in these systems are often filled with contingencies. It makes sense to me that rich, long persona and scenario development would go a long way in raising the social elements (trust, power…) in the design process.

  • Defrag was super…can't wait for Glue…Larry @ w3w3.com

  • Pingback: Finance Geek » Opportunities created by the growth in data()

  • Yup – you are definitely on track.

  • Amit

    Temporal Cognition? you either know something or you don’t. if you do then there’s no problem, if you don’t then you need to search for the solution which is exectly what we do nowadays.

    Please be more specific otherways it simply sounds like a buzz creation attempt…

  • Like I said in my talk, I think the problem is we need more information and less data. It's only a matter of time before we see a flood of companies taking the next step, from simply aggregating data to creating meaning from it.

  • sara

    Nice blog..


  • Pingback: software for scheduling()

  • Pingback: software for scheduling()

  • So, some tip about this issue are welcome and really sorry if my question is very simple. Thanks in advance
    Thanks again for your help. Your site contain a many useful information.

  • Craig Lachman

    Don't know if you still check comments on these old posts, but this Frazz cartoon reminded me of your Lifestream interest: http://comics.com/frazz/2010-10-10/

  • Pingback: โหลดหนังโป๊()

  • Pingback: Women of()

  • Pingback: Google()

  • Pingback: auto car insurance quotes()