Comments on: Manually Automate Your API

By: Aaron Finn

Aaron Finn — Wed, 07 Jan 2009 01:55:49 +0000

Great post and thanks for the AdReady mention. It's always appreciated.

I want to second your comments. Over the last year we have been integrating with several online advertising API's. I would recommend to everyone that they work with the systems in a manual way before starting the integration process. When your people understand the system they are able to make quicker more stable choices when developing the automation. I estimate that we cut our development time in half by manually working with the system for 3 months. The best part is that we were doing business during this learning phase.

We even found that because our people were familiar with the features of the system, they were able to understand the API better than some of the partner's personnel. We have become so familiar that a few have asked us to help them develop additional documentation and features.

By: david_ulevi3244

david_ulevi3244 — Wed, 07 Jan 2009 01:55:49 +0000

There is a person out there, his name is HTTP::Recorder. Hiring a person when you already have one that works 24x7 and for "near free" (some disk and cpu and network) is a bad idea.

Check out: http://search.cpan.org/~leira/HTTP-Recorder-0.05/...

Most languages have their version of WWW::Mechanize. PHP, Ruby, etc. It's easy these days when there is existing code that will record what you do manually, and then add a module around it. It's not as ugly as scraping used to be.

side note, this is the worst comment box I've ever used. It keeps shrinking and growing.

By: Richard Dale

Richard Dale — Wed, 07 Jan 2009 01:55:49 +0000

Check out OpenSpan (full disclosure: Sigma Partners portfolio company) which can mix and match a modern version of "screen scraping" with SOA/Webservices.

By: bfeld

bfeld — Wed, 07 Jan 2009 01:55:49 +0000

Yup - that's definitely an approach to getting the script process built more quickly. However, it still requires someone to pay attention to it on the programming side which quickly gets complex if you are shoving data into someone else's system.

This is all above getting things rolling ahead of the time when you are going to automate it. Sometimes it's easy, but when it isn't, I find that it usually becomes a lower priority project, which isn't always the right way (e.g. we'll get to it someday.)

If there's real leverage from this, just having a junior person "do the work" - while costing a little money - can have a pretty powerful payoff.