Social Search

Last week, Me.dium released its Alpha version of Me.dium Social Search.  This coincided with Yahoo!’s launch of BOSS – Me.dium was one of the initial launch partners.  This was picked up by Techmeme and prominently talked about throughout the blogosphere, tech media, and even the mainstream media.

I’ve been involved with Me.dium since its first financing and the launch of Me.dium Social Search is a key pivot point as it starts to capitalize on the initial vision of the company.  If you are familiar with Me.dium, you may know of it as a company that has a browser sidebar that enables real time browsing with friends.  This concept started out as a "recommendation service" where the algorithms suggested alternative web sites and people based on how your browsing patterns matched the browsing patterns of your friends and the overall community.

This was – and is – a pretty neat idea, but it’s really hard to do effectively in a browser sidebar. Me.dium built out a lot of backend infrastructure to process a large amount of information in real time, which is necessary to make the algorithms useful across a large user base.  In the process of doing this, it occurred to the team that the sidebar might not be the best way to surface the information and that search might be a better way to deliver its value to the user since Me.dium’s collaborative filtering algorithms are an entirely different search algorithm than the PageRank type ones we’ve gotten used to.

A group of folks at Me.dium went heads down and starting working on using the stream of data they were getting to turn out a real time social search engine.  Along the way, Yahoo! decided to open up their search engine infrastructure through Yahoo! BOSS and a natural collaboration was born.

Rather than hold on tight, create a "closed beta", and limit the use and exploration of Me.dium Social Search, they did what I wish more companies would do and went "straight to Alpha".  Even though it’s alpha and still evolving rapidly, Me.dium Social Search produces some really interesting search results that correspond to the web pages that people are looking at right now about specific topics. 

The notion of the Me.dium Sidebar has morphed into a Social Toolbar which, in addition to providing a bevy of social features, also starts including your clickstream in the corpus of data that Me.dium is using for their social search algorithms. Me.dium is fanatical about your privacy and includes a simple one click way in your toolbar to turn Me.dium tracking on and off and never tracks information on secure (via HTTPS) sites.

Yahoo! has stirred the search pot in an interesting way with BOSS.  Me.dium’s going after one particular vector – that of social search – by building on a lot of work they’ve been doing over the past eighteen months.  I expect you’ll hear a lot about The Future of Search in the coming year – I think Me.dium will be one of the companies regularly mentioned in the mix of folks trying new approaches.

Give Me.dium Social Search a try (simply type your search term into the little box and hit the "I Feel Social" button) and tell us what you think.  And – if you want to go deeper on the ideas, take a look at Robert Reich’s (one of Me.dium’s co-founders) blog titled Why to hear him riff on search.

  • skywalker

    What is the advantage of doing this “real time social search”? I am still trying to figure why I should use this service..I use delicious and it works just fine. I can see what my friends have bookmarked although it is not real time. hmm..but interesting concept though!

    • http://www.feld.com Brad Feld

      Delicious is obviously very cool also, but it’s very different.  First, search on Delicious isn’t real time based – it’s based on what people tagged and ordered by the number of tags.  Next, the hotlist is about what is getting tagged right now so it has a real time component, but once again requires a tag.  In both cases, the user has to do something (namely tag a URL); in Me.dium’s case there is no tagging required.  So – the fundamental difference is (a) no tags required on Me.dium and (b) algorithm on Me.dium is driven off of real time behavior.  There are plenty of other differences, but these two generate meaningful different types of search results.

  • http://menro.blogspot.com Robert Reich

    I am big fan of delicious and have been building my personal Folksonomy of the web for years. I probably check or search my delicious sidebar 2 to 3 times a day, but I also use a search engine 20 plus times a day.

    What we are doing differently at Me.dium is leveraging the implicit actions of our community to determine which pages to index and in what order we should show them to the user. If you test the Alpha product (http://me.dium.com/search) you will see the advantage with current event type queries. I would recommend trying Bill Clinton today. The publisher driven search engines Google, Yahoo, Ask, Live and even my folksonomy tend to be historical and produce a very different result set then Me.dium's Crowd Powered Search.

    Try it and let me know how it performs.

    • matteofabiano

      Robert,
      When people talk about social search they fundamentally talk about 3 types of apps:
      1. Search on a user-contributed index (e.g. delicious)
      2. Search on content my social network has produced (e.g. Facebook, Friendfeed)
      3. Search with collaborative filters (e.g. Amazon)
      Which category does me.dium fit in? Or is it something else?

      • http://menro.blogspot.com Robert Reich

        Me.dium's Crowd Powered Search is a hybrid. We enable you to search via the user-contributed index. We have the ability, but have not turned on filtering via your social network, and we will be launching a directory of popular topics shortly.

        What also makes us different is the relationship with Yahoo. Me.dium can be used as a replacement for current search engine's, blending the results into a single user experience.

  • http://www.burnertrouble.com Martin Edic

    I have to weigh in here since we also do something related. This engine appears to add in social functionality to what is basically a web search. Searching social media involves far more than this model embraces- you need to search microblogs, social networks, comment threads, user-generated content, etc. and it is almost imperative to get quite specific with your array of keyword phrases or you'll simply return too many irrelevant results.

    • http://www.feld.com Brad Feld

      Martin – good comments.  I’m surprised the results from a search came back the same as Google.  Can you tell me the search terms you were using?

      Also – can you confirm that you were clicking on the “I Feel Social” button?  If you simply were clicking “Search”, you will get the same results that you’ll get on Yahoo since it’s merely using Yahoo BOSS.  However, “I Feel Social” is the Me.dium Social Search algorithm.

      • http://www.burnertrouble.com Martin Edic

        I tried both for several terms including our company name (because I'm intimately aware of what should be coming back. The results with the Search query were predictable, a combination of site hits and a few blogs. With 'I'm feeling Social' I got only one result even though it said there were 2000. That one result was our home page. Either I'm missing something or it has not truly indexed social media because our system, SM2 returns thousands of legitimate social media results on the same keyword. It does not do so in real time, in fact a search can initially take hours as we are pulling data into our social media

        • http://www.feld.com Brad Feld

          It’s is an “alpha” after all…  I tried “Techrigy” and you are correct – there is only one result (your home page) which comes back from the Social Search.  So – the “Results 1 – 1 of about 2,000” looks like a bug.  The result, however, makes sense – it appears there is nothing in their search corpus from real time browsing of folks using Me.dium’s sensors.   

          Based on my read of your comments, I think you are actually talking about something I refer to as “Network Search”.  Look for a post on that next week.

          • http://www.burnertrouble.com Martin Edic

            I'll watch for that one- obviously we are in new territory here.

    • http://menro.blogspot.com Robert Reich

      First I echo Brad – good comments.

      I wanted to address each of your comments and or opinions.

      Item 1) This engine appears to add in social functionality to what is basically a web search.
      A) We are not crawling the web looking for social content, our users do this implicitly based on their surfing patterns.

      Item 2) Searching social media involves far more than this model embraces- you need to search microblogs, social networks, comment threads, user-generated content, etc. and it is almost imperative to get quite specific with your array of keyword phrases or you'll simply return too many irrelevant results.
      A) I do not agree that these are required sources for an engine to be considered social. In Me.diums case our users decide what is interesting, we do not filter, except porn, the content type. If our users visit a microblog, specific pages in a social network, comment threads, or some other type of user-generated content we should be indexing and making it available.

      Item 3) The results on searches I got in Me.diem were exactly the same as those I found on Google. Doing true social media discovery brings back a very different set of results and requires a number of filters to help you understand those results: demographics, location, authority, context, etc.
      A) As brad says if you click i feel social you are only receiving pages that have been vetted by Me.dium users. The additional metadata: demographics, location, authority, context, ext… can be used as filters, but I do not view these as requirements for being social search engine.

      • http://www.burnertrouble.com Martin Edic

        Thanks Robert, I think we're talking about two very different objectives. we monitor for the purpose of tracking what people are saying about brands and reputations in social media.
        BTW, I think I'm seeing some bugs in Intense Debate. Can't scroll in the box when you write a long post and want to check spelling, etc and my last comment text seems to remain in the box when I return.
        Off topic..could be Safari/Mac

  • http://barfieldmanagement.com Chase Barfield

    Interesting. I joined and will play around with the features. My results were different than Google, depends on your search terms I suppose and whether they are social based or genral in nature. I absolutely love to contribute to new worthwhile services. They should consider adopting an OpenID login model like Intense Debate and many other sites are migrating to.

  • Jud Valeski

    The importance of “social search” is high; _very_ high. Today's crawlers/indexers are obviously useful (we all use Google a thousand times a day), as they've provided a fundamental navigation paradigm that has shaped data consumption for years to come. However, they're lame; we're suckers. They're based on very old techniques for building indexes of relatively static information, and generally have no context other than the average 2.5 keywords we feed the “search box.” They've been distilled, long ago, into pure math and statistics, and therefore won't get much better than they are today, without some dramatic changes.

    There have been many disambiguation models/techniques built over the years, but they generally require end-user interaction (e.g. “did you mean Eagles the band, or Eagles the bird…..?”) and end-users don't like to do work. “Sensing” our social networks, and our behaviors allows context to be gathered automagically. Augmenting search facilities with “real-time”, sensed, social context and attention will yield amazing results in the end. I'm stoked to see me.dium's initial crack at the mix, and can't wait for more.

    Today's crawlers follow links on pages. It doesn't get any more rudimentary than this, and yet we all seem content with basing our web navigation on it; unfortunate. Tomorrow's maps and indexes will be constructed using the actions that end-users take while navigating content and resources. Humans establish the best connections between various chunks of content; not links on pages.

    me.dium gets this. Google gets this too; http://venturebeat.com/2008/01/31/googles-marissa

    I can't wait to be a part of this [r]evolution. I'm weary of walking graphs generated by machines. Where's the People Dimension of the Internet?

  • Robert Reich

    I am big fan of delicious and have been building my personal Folksonomy of the web for years. I probably check or search my delicious sidebar 2 to 3 times a day, but I also use a search engine 20 plus times a day.

    What we are doing differently at Me.dium is leveraging the implicit actions of our community to determine which pages to index and in what order we should show them to the user. If you test the Alpha product (http://me.dium.com/search) you will see the advantage with current event type queries. I would recommend trying Bill Clinton today. The publisher driven search engines Google, Yahoo, Ask, Live and even my folksonomy tend to be historical and produce a very different result set then Me.dium's Crowd Powered Search.

    Try it and let me know how it performs.

  • http://intensedebate.com/people/bfeld bfeld

    Delicious is obviously very cool also, but it’s very different.  First, search on Delicious isn’t real time based – it’s based on what people tagged and ordered by the number of tags.  Next, the hotlist is about what is getting tagged right now so it has a real time component, but once again requires a tag.  In both cases, the user has to do something (namely tag a URL); in Me.dium’s case there is no tagging required.  So – the fundamental difference is (a) no tags required on Me.dium and (b) algorithm on Me.dium is driven off of real time behavior.  There are plenty of other differences, but these two generate meaningful different types of search results.

  • http://intensedebate.com/people/bfeld bfeld

    It’s is an “alpha” after all…  I tried “Techrigy” and you are correct – there is only one result (your home page) which comes back from the Social Search.  So – the “Results 1 – 1 of about 2,000” looks like a bug.  The result, however, makes sense – it appears there is nothing in their search corpus from real time browsing of folks using Me.dium’s sensors.   

    Based on my read of your comments, I think you are actually talking about something I refer to as “Network Search”.  Look for a post on that next week.

  • http://intensedebate.com/people/bfeld bfeld

    Martin – good comments.  I’m surprised the results from a search came back the same as Google.  Can you tell me the search terms you were using?

    Also – can you confirm that you were clicking on the “I Feel Social” button?  If you simply were clicking “Search”, you will get the same results that you’ll get on Yahoo since it’s merely using Yahoo BOSS.  However, “I Feel Social” is the Me.dium Social Search algorithm.

  • Robert Reich

    First I echo Brad – good comments.

    I wanted to address each of your comments and or opinions.

    Item 1) This engine appears to add in social functionality to what is basically a web search.
    A) We are not crawling the web looking for social content, our users do this implicitly based on their surfing patterns.

    Item 2) Searching social media involves far more than this model embraces- you need to search microblogs, social networks, comment threads, user-generated content, etc. and it is almost imperative to get quite specific with your array of keyword phrases or you'll simply return too many irrelevant results.
    A) I do not agree that these are required sources for an engine to be considered social. In Me.diums case our users decide what is interesting, we do not filter, except porn, the content type. If our users visit a microblog, specific pages in a social network, comment threads, or some other type of user-generated content we should be indexing and making it available.

    Item 3) The results on searches I got in Me.diem were exactly the same as those I found on Google. Doing true social media discovery brings back a very different set of results and requires a number of filters to help you understand those results: demographics, location, authority, context, etc.
    A) As brad says if you click i feel social you are only receiving pages that have been vetted by Me.dium users. The additional metadata: demographics, location, authority, context, ext… can be used as filters, but I do not view these as requirements for being social search engine.

  • matteofabiano

    Robert,

    When people talk about social search they fundamentally talk about 3 types of apps:

    1. Search on a user-contributed index (e.g. delicious)

    2. Search on content my social network has produced (e.g. Facebook, Friendfeed)

    3. Search with collaborative filters (e.g. Amazon)

    Which category does me.dium fit in? Or is it something else?

  • http://intensedebate.com/people/chase_barf55451 chase_barf55451

    Interesting. I joined and will play around with the features. My results were different than Google, depends on your search terms I suppose and whether they are social based or genral in nature. I absolutely love to contribute to new worthwhile services. They should consider adopting an OpenID login model like Intense Debate and many other sites are migrating to.

  • http://intensedebate.com/people/martin_edi18731 martin_edi18731

    I'll watch for that one- obviously we are in new territory here.

  • Robert Reich

    Me.dium's Crowd Powered Search is a hybrid. We enable you to search via the user-contributed index. We have the ability, but have not turned on filtering via your social network, and we will be launching a directory of popular topics shortly.

    What also makes us different is the relationship with Yahoo. Me.dium can be used as a replacement for current search engine's, blending the results into a single user experience.

  • http://intensedebate.com/people/martin_edi18731 martin_edi18731

    Thanks Robert, I think we're talking about two very different objectives. we monitor for the purpose of tracking what people are saying about brands and reputations in social media.
    BTW, I think I'm seeing some bugs in Intense Debate. Can't scroll in the box when you write a long post and want to check spelling, etc and my last comment text seems to remain in the box when I return.
    Off topic..could be Safari/Mac

  • http://intensedebate.com/people/jud_valeski3968 jud_valeski3968

    The importance of "social search" is high; _very_ high. Today's crawlers/indexers are obviously useful (we all use Google a thousand times a day), as they've provided a fundamental navigation paradigm that has shaped data consumption for years to come. However, they're lame; we're suckers. They're based on very old techniques for building indexes of relatively static information, and generally have no context other than the average 2.5 keywords we feed the "search box." They've been distilled, long ago, into pure math and statistics, and therefore won't get much better than they are today, without some dramatic changes.

    There have been many disambiguation models/techniques built over the years, but they generally require end-user interaction (e.g. "did you mean Eagles the band, or Eagles the bird…..?") and end-users don't like to do work. "Sensing" our social networks, and our behaviors allows context to be gathered automagically. Augmenting search facilities with "real-time", sensed, social context and attention will yield amazing results in the end. I'm stoked to see me.dium's initial crack at the mix, and can't wait for more.

    Today's crawlers follow links on pages. It doesn't get any more rudimentary than this, and yet we all seem content with basing our web navigation on it; unfortunate. Tomorrow's maps and indexes will be constructed using the actions that end-users take while navigating content and resources. Humans establish the best connections between various chunks of content; not links on pages.

    me.dium gets this. Google gets this too; http://venturebeat.com/2008/01/31/googles-marissa

    I can't wait to be a part of this [r]evolution. I'm weary of walking graphs generated by machines. Where's the People Dimension of the Internet?

  • http://intensedebate.com/people/skywalker skywalker

    What is the advantage of doing this "real time social search"? I am still trying to figure why I should use this service..I use delicious and it works just fine. I can see what my friends have bookmarked although it is not real time. hmm..but interesting concept though!

  • http://intensedebate.com/people/martin_edi18731 martin_edi18731

    I tried both for several terms including our company name (because I'm intimately aware of what should be coming back. The results with the Search query were predictable, a combination of site hits and a few blogs. With 'I'm feeling Social' I got only one result even though it said there were 2000. That one result was our home page. Either I'm missing something or it has not truly indexed social media because our system, SM2 returns thousands of legitimate social media results on the same keyword. It does not do so in real time, in fact a search can initially take hours as we are pulling data into our social media warehouse constantly. Each search updates the entire data warehouse which will gradually speed up the system. Once you've pulled a search the first time the system can go back on a periodic basis and send you any new results.
    Because we developed SM2 as a tool it was never designed to emulate traditional web search- however they only achieve real time by building a huge index which isn't truly real time either because not all up to date info gets indexed in real time- there's a lag.
    The nice thing about being a tool is that there's a revenue model. I suspect that attempting to add contextual advertising to social search is going to backfire as a revenue model. We'll see.

  • http://intensedebate.com/people/martin_edi18731 martin_edi18731

    I have to weigh in here since we also do something related. This engine appears to add in social functionality to what is basically a web search. Searching social media involves far more than this model embraces- you need to search microblogs, social networks, comment threads, user-generated content, etc. and it is almost imperative to get quite specific with your array of keyword phrases or you'll simply return too many irrelevant results. The results on searches I got in Me.diem were exactly the same as those I found on Google.
    Doing true social media discovery brings back a very different set of results and requires a number of filters to help you understand those results: demographics, location, authority, context, etc. This is because you are not searching for information, you are sifting through conversations. That's why the term 'discovery' is different from 'search'.
    It's a fascinating thing to watch unfold.