Brad's Books and Organizations

Books

Books

Organizations

Organizations

Hi, I’m Brad Feld, a managing director at the Foundry Group who lives in Boulder, Colorado. I invest in software and Internet companies around the US, run marathons and read a lot.

« swipe left for tags/categories

swipe right to go back »

Renormalizing Denormalized Data

Comments (44)

Yummy – that’s a fun tongue twister.  It doesn’t quite mean “synchronizing data”, but it’s in the same family.  I don’t have a better phrase yet for “renormalizing denormalized data”, but there is probably a construct for it that someone reading this blog can tell me (or invent – here’s your choice to replace the TLA RDD with something else.)

Two companies – Gnip and Brightkite – that I’m involved with (there – full disclosure!) made announcements today about data integrations.

Here’s a bizarre use case.

  1. I register my location with Brightkite, take a picture, and write a note.
  2. Brightkite saves this data to Brightkite and these starts putting this data out to services I’ve integrated with.
    1. Service 1: Twitter
    2. Service 2: Flickr
    3. Service 3: Facebook
  3. I’m running a Twitter / Facebook status synchronizer.  So – when I twitter something it shows up on Facebook.  When I put a status on Facebook, it shows up on Twitter.

Theoretically, Brightkite, Twitter, and Facebook should know enough about each other not to repost the same thing in cases 2.1, 2.3, and 3 (recursive).  But I don’t think it’s going to do the right thing.  Let’s try and see how many tweets we get!  I’m guessing three.

Nope – it only showed up once.  I’m guess this is either because (a) Brightkite takes care of the issue (smart people at Brightkite), (b) something isn’t set up correctly (just tried twice with two different configurations), or (c) something unknown and mysterious is going on somewhere.

I’m going to bet on (a).  However, not everyone is going to be a tuned in to the issue as the guys at Brightkite and as this dynamic proliferates, the RDD problem will get worse.  Remember that this is bad:

10 Print “Hello”
20 Goto 10

I can’t wait until integration between three services get stuck in an infinite loop, bring down the entire Internet, and suck us all into a black hole.

  • http://blog.thylmann.net/ Oliver Thylmann

    I fully agree with you. I still hate it that I don't really want flickr posts tagged "moblog" included in my Tumblr style blog via Chyrp because they will turn up duplicate in friendfeed. But it goes further than that. When you think about the fact that this is all your content you really want it integrated.

    http://blog.thylmann.net/2008/12/28/self-generate

    E.g. if today I send an update to twitter view brightkite, twitter doesn't know my location, but through the time the tweet happened, should be able to know my location. The same is true for the photos I upload at a later date to flickr which know my location through the EXIF data about when they were taken and should then also know the location they were taken from. All could be tagged for the location. These tags need to be available over everything and need to be possibly integrated into one platform, so I can say "What did I do at home between Christmas and New Years?"… twitter, flickr, blog, tumblr, brightkite, facebook, emails, … all there.

    Now that is where this all should be going.

  • http://intensedebate.com/people/eric_marcou4248 Eric Marcoullier

    >I can't wait until integration between three services get stuck in an infinite loop, bring down the entire Internet, and suck us all into a black hole.<

    That's the dreaded "black goo" scenario, yes?

  • http://sean808080.com sean808080

    i have one word for you PING.FM

    seriously…i turned off all synchronization between services and use ping.fm to post updates to them directly. my life is golden now.

    sean808080
    http://sean808080.com

    • http://sophisticatedfinance.typepad.com rhhfla

      PING.FM looks terrific, but those guys have the slowest site response I have seen since 1993. Hopefully they will invest in infrastructure or use AWS.

    • http://www.telavenir.com voyant

      There is no great tool as twitter even facebook is fighting with them, this also a similar tool as twitter

  • http://www.aadjemonkeyrock.com Aad 't Hart

    I was struggling with the same.. I didn't set up my BrightKite to tweet my posts, although I would like that.. Since I aggregate everything in FriendFeed… I don't want too much to show up double.. ;-)
    I sort of decided to use FriendFeed and Facebook as output channels.. and not to update there (isn't really true, but anyhow).. BrightKite, Twitter.. are activity generating apps… but still too much shows up double everywhere..

  • http://twitter.com/chrischerrett chrischerrett

    The Flickr API prevents duplicate tweets. So if you Post on Brightkite (for example), any services which also gets that Post (such as Twitter, Facebook, Flickr) which themselves also push to Twitter (excluding Twitter itself) which find Twitter 'blocking' the Tweet. Does that make sence?

    Follow me @ http://twitter.com/chrischerrett

    • http://intensedebate.com/people/bfeld Brad Feld

      Yup – makes sense and seems to be the way it works (as it should!)

  • http://twitter.com/chrischerrett chrischerrett

    … so it's essentially neither (a),(b), or (c).

    Follow me @ http://twitter.com/chrischerrett

    • http://intensedebate.com/people/bfeld bfeld

      Correct. It sounds like (d) the various services are handling idempotence correctly! (I learned a new word today).

  • http://www.standingmobile.com kevin cawley

    that's awesome – sounds like a fun weekend project to find just the right sequence… and you don't even need to hack dns :)

  • Pingback: Ben Barren - Confessions of a Mad Man » renormalizing a denormalized new years eve day

  • Pingback: 10 Print “Hello” 20 Goto … | The Hotlist

  • http://intensedebate.com/people/krista51951 Krista

    Thanks for the tip, Brad. I didn't know Brightkite integrated with Facebook. Just enabled mine too!

  • http://www.mooladays.com Albert

    Brightkite I think it just like Twitter, however, it is used to find people locally where in Twitter you can find anyone around the world without classifying where you live locally.

  • http://www.new.us Dot US

    It seems same as twitter, But any thing to highlight this tool. So far twitter is the best way to communicate fast with friends. It's really simple.

  • http://horoscopes.proastro.com tarot cards

    There is no great tool as twitter even facebook is fighting with them, this also a similar tool as twitter.

    • http://www.telavenir.com voyance

      I actually approve it or any similar tool like twitter facebook.

  • http://golfjoe.wordpress.com/2008/09/24/warrior-custom-golf warrior

    As a communication tool, I prefer facebook social networking. Even i'm not using twitter. So this also looks as twitter.

  • RobKelley

    I heard this called Social Media Workflow. It's been driving me batty, particularly the brightkite integration with Facebook and Twitter. For example, Brightkite has a nice Facebook app that thumbnails the picture on your status. It's better than the blurb Brightkite will shoot to Twitter. My friends respond more quickly to that than to the Twitter blurb. But Twitter is feeding facebook as well, so there are always two entries there. Argh.

    And don't get me started about location-updating between brightkite, fireeagle, and foursquare. Well, it's not as bad–the problem there is that foursquare doesn't update anything or get updated and it's my favorite location app.

    i think this workflow will get harder and harder as more apps come on to the scene unless we start talking about roles like you did:

    * generating app (brightkite?)

    * displaying app (Facebook?)

    * interstitial app (twitter?)

    Help appreciated.

  • http://intensedebate.com/people/RohitK RohitK

    Hi,
    Some days before in digg u can register through ur facebook profile. They came together but the dream u r watching to bring all these 3 together without any problem is difficult. Now any how u have to update everywhere..

  • http://www.intensedebate.com/people/RohitK RohitK

    Hi,
    Some days before in digg u can register through ur facebook profile. They came together but the dream u r watching ..to bring all these 3 together without any problem is difficult. Now any how u have to update everywhere..

  • http://www.intensedebate.com/people/RohitK RohitK

    Hi,
    Some days before in digg u can register through ur facebook profile. They came together but the dream u r watching ..to bring all these 3 together without any problem is difficult. Now any how u have to update everywhere..

  • http://www.webergrillaccessories.org paul

    i love this tool. no matter what, there is nothing perfect so oh well..

  • http://www.respectablereviews.com Respectable Reviews

    It sounds like a challenge, but I'm up for it :-).

  • http://www.righttobuymortgagesuk.co.uk Right to buy

    I’m running a Twitter / Facebook status synchronizer. So – when I twitter something it shows up on Facebook. When I put a status on Facebook, it shows up on Twitter. I think this is a good idea.

  • http://www.freevacationtravels.com/ Free Vacation

    there is nothing perfect so oh well..

  • http://www.beritasemasa.com Khai

    @paul: me too..thnx for sharing :)

  • http://www.telavenir.com voyant

    I prefer to use facebook rather than twitter this seems more interresting

  • http://www.bestlowcostwebhosting.net Top

    It stops duplicate posts. I think it's the best app out there for that.

  • http://www.hypnosisblacksecrets.com Hypnosis

    It is good time saver and helps to stay organized in scoail media marketing.

  • http://trentbrownrigg.com Trent

    Try saying renormalizing denormalized data three times fast! I'm not even sure I know what you just said but it sounds like a good thing to do. I guess it keeps things from being updated duplicates on social sites while keeping them organized? Cool enough.

  • http://www.besthomehumidifier.com Humidifier

    I didn't know this stops duplicate content. I think this is very important.

  • http://natural-fibromyalgia-remedies.info john

    Any thing that will help me communicate faster is welcome

  • http://www.natural-gout-remedies.info/ gout

    Twitter should increase the 140 characters.

  • http://www.massageceresource.com/ rowena

    How does it prevent duplicate posts?

  • http://www.squidoo.com/conversationalstreethypnosis hypnoguy

    brightkite is a useful tool.

  • http://pellgranteligibility.net pell grant eligibile

    I agree, I've used it in the past with success.

  • http://www.freelegitmoney.com money online

    Nice post informative as well.

  • http://www.replica-china.net replcia handbags

    So, some tip about this issue are welcome and really sorry if my question is very simple. Thanks in advance
    Thanks again for your help. Your site contain a many useful information.

  • http://teachyourselfspanisheasy.org speaking spanish

    Aw man, I don't like these social networking sites… Though this could be beneficial for those running said services I certainly won't be getting any use out of it. Perhaps one day…

  • Pingback: compare car insurance

  • Pingback: bonsai gallery

  • Pingback: cheap online auto insurance

Build something great with me