Brad's Books and Organizations

Books

Books

Organizations

Organizations

Hi, I’m Brad Feld, a managing director at the Foundry Group who lives in Boulder, Colorado. I invest in software and Internet companies around the US, run marathons and read a lot.

« swipe left for tags/categories

swipe right to go back »

Manually Automate Your API

Comments (4)

When 369 of something suddenly appears, I get excited about the opportunities to do something with it.  When I noticed the 370th new ad network, I realized there were probably a few opportunities to do some interesting things across them.  AdReady – a company I made an investment in a year ago – is seriously kicking ass (if you do any display advertising on Google, Yahoo/RightMedia, or AOL go take a look at them) and their early success reinforced this idea to me.

While I’ll talk more about a new investment we’ve made in this area in a few weeks (yeah – that’s what real journalists call "the tease" or something like that), when I was at Clickable a few weeks ago (not an investment of mine, but a really interesting one by Union Square Ventures and Pequot with the tagline "online advertising made simple") we were discussing the challenges of integrating with some of the existing ad networks through their weak to non-existent APIs.

As a horizontal thinker, the API is my friend.  APIs are hard and often take a long time to get right.  Anyone that has integrated with someone else’s web service knows there are a whole series of things that "should" be part of the next version of the API.  Frustration abounds and the rationalization of "we’ll do this thing that will massively drive user adoption and our services utility when company X finally puts it in their API."

To that, I say "manually automate your API."  Figure out what you want the workflow to be, hire a person, and have them do whatever the API should do, but do it by simply running the other system.  Way back in the land of DOS there used to be neat little script programs (anyone remember the Peter Norton, Paul Mace, or Dan Bricklin tools that did this) that automated your keystrokes.  Your manual API person can do the same thing today with the contemporary versions of these keystroke automation programs

By manually automating your API – or creating a manual version of the API you wished the other guy’s web service had – you can immediately drive more value to your users while prototyping – with precision – what the integration points between the two systems need to be.  Yeah – you could get a developer to write a bunch of screen scraping code, but you don’t have any extra developers laying around, plus it’ll break in six days anyway when the other service does a new release.  Just hire a young smart person and give him the mission of figuring out how the puzzle pieces fit together.

  • David Ulevitch

    There is a person out there, his name is HTTP::Recorder. Hiring a person when you already have one that works 24×7 and for “near free” (some disk and cpu and network) is a bad idea.

    Check out: http://search.cpan.org/~leira/HTTP-Recorder-0.05/

    Most languages have their version of WWW::Mechanize. PHP, Ruby, etc. It's easy these days when there is existing code that will record what you do manually, and then add a module around it. It's not as ugly as scraping used to be.

    side note, this is the worst comment box I've ever used. It keeps shrinking and growing.

    • http://www.feld.com Brad Feld

      Yup – that's definitely an approach to getting the script process built more quickly. However, it still requires someone to pay attention to it on the programming side which quickly gets complex if you are shoving data into someone else's system.

      This is all above getting things rolling ahead of the time when you are going to automate it. Sometimes it's easy, but when it isn't, I find that it usually becomes a lower priority project, which isn't always the right way (e.g. we'll get to it someday.)

      If there's real leverage from this, just having a junior person “do the work” – while costing a little money – can have a pretty powerful payoff.

  • http://venturecyclist.blogspot.com Richard Dale

    Check out OpenSpan (full disclosure: Sigma Partners portfolio company) which can mix and match a modern version of “screen scraping” with SOA/Webservices.

  • http://www.adready.com Aaron Finn

    Great post and thanks for the AdReady mention. It's always appreciated.

    I want to second your comments. Over the last year we have been integrating with several online advertising API's. I would recommend to everyone that they work with the systems in a manual way before starting the integration process. When your people understand the system they are able to make quicker more stable choices when developing the automation. I estimate that we cut our development time in half by manually working with the system for 3 months. The best part is that we were doing business during this learning phase.

    We even found that because our people were familiar with the features of the system, they were able to understand the API better than some of the partner's personnel. We have become so familiar that a few have asked us to help them develop additional documentation and features.

  • Aaron Finn

    Great post and thanks for the AdReady mention. It's always appreciated.

    I want to second your comments. Over the last year we have been integrating with several online advertising API's. I would recommend to everyone that they work with the systems in a manual way before starting the integration process. When your people understand the system they are able to make quicker more stable choices when developing the automation. I estimate that we cut our development time in half by manually working with the system for 3 months. The best part is that we were doing business during this learning phase.

    We even found that because our people were familiar with the features of the system, they were able to understand the API better than some of the partner's personnel. We have become so familiar that a few have asked us to help them develop additional documentation and features.

  • http://intensedebate.com/people/david_ulevi3244 david_ulevi3244

    There is a person out there, his name is HTTP::Recorder. Hiring a person when you already have one that works 24×7 and for "near free" (some disk and cpu and network) is a bad idea.

    Check out: http://search.cpan.org/~leira/HTTP-Recorder-0.05/

    Most languages have their version of WWW::Mechanize. PHP, Ruby, etc. It's easy these days when there is existing code that will record what you do manually, and then add a module around it. It's not as ugly as scraping used to be.

    side note, this is the worst comment box I've ever used. It keeps shrinking and growing.

  • Richard Dale

    Check out OpenSpan (full disclosure: Sigma Partners portfolio company) which can mix and match a modern version of "screen scraping" with SOA/Webservices.

  • http://intensedebate.com/people/bfeld bfeld

    Yup – that's definitely an approach to getting the script process built more quickly. However, it still requires someone to pay attention to it on the programming side which quickly gets complex if you are shoving data into someone else's system.

    This is all above getting things rolling ahead of the time when you are going to automate it. Sometimes it's easy, but when it isn't, I find that it usually becomes a lower priority project, which isn't always the right way (e.g. we'll get to it someday.)

    If there's real leverage from this, just having a junior person "do the work" – while costing a little money – can have a pretty powerful payoff.

Build something great with me