« swipe left for tags/categories
swipe right to go back »
As a user, how often have you thought “I wish this web service was faster.” As a CEO, how often have you said “just make it faster.” Or, more simply, “why is this damn thing so slow?”
This is a not a new question. I’ve been thinking about this since I first started writing code (APL) when I was 12 (ahem – 33 years ago) on a computer in the basement of a Frito-Lay data center in Dallas.
This morning, as part of my daily information routine, I came across a brilliant article by Carlos Bueno, an engineer at Facebook, titled “The Full Stack, Part 1.” In it, he starts by defining a “full-stack programmer“:
“A “full-stack programmer” is a generalist, someone who can create a non-trivial application by themselves. People who develop broad skills also tend to develop a good mental model of how different layers of a system behave. This turns out to be especially valuable for performance & optimization work.”
He then dissects a simple SQL query (DELETE FROM some_table WHERE id = 1234;) and gives several quick reasons why performance could vary widely when this query is executed.
It reminded me of a client situation from my first company, Feld Technologies. We were working on a logistics project with a management consulting firm for one of the largest retail companies in the world. The folks from the management consulting firm did all the design and analysis; we wrote the code to work with the massive databases that supported this. This was in the early 1990′s and we were working with Oracle on the PC (not a pretty thing, but required by this project for some reason.) The database was coming from a mainframe and by PC-standards was enormous (although it would probably be considered tiny today.)
At this point Feld Technologies was about ten people and, while I still wrote some code, I wasn’t doing anything on this particular project other than helping at the management consulting level (e.g. I’d dress up in a suit and go with the management consultants to the client and participate in meetings.) One of our software engineers wrote all the code. He did a nice job of synthesizing the requirements, wrestling Oracle for the PC to the ground (on a Novell network), and getting all the PL/SQL stuff working.
We had one big problem. It took 24 hours to run a single analysis. Now, there was no real time requirement for this project – we might have gotten away with it if it took eight hours as we could just run them over night. But it didn’t work for the management consultants or the client to hear “ok – we just pressed go – call us at this time tomorrow and we’ll tell you what happened.” This was especially painful once we gave the system to the end client whose internal analyst would run the system, wait 24 hours, tell us the analysis didn’t look right, and bitch loudly to his boss who was a senior VP at the retailer and paid our bills.
I recall having a very stressful month. After a week of this (where we probably got two analyses done because of the time it took to iterate on the changes requested by the client for the app) I decided to spend some time with our engineer who was working on it. I didn’t know anything about Oracle as I’d never done anything with it as a developer, but I understood relational databases extremely well from my previous work with Btrieve and Dataflex. And, looking back, I met the definition of a full-stack programmer all the way down to the hardware level (at the time I was the guy in our company that fixed the file servers when they crashed with our friendly neighborhood parity error or Netware device driver fail to load errors.)
Over the course of a few days, we managed to cut the run time down to under ten minutes. My partner Dave Jilk, also a full-stack programmer (and a much better one than me), helped immensely as he completely grokked relational database theory. When all was said and done, a faster hard drive, more memory, a few indexes that were missing, restructuring of several of the SELECT statements buried deep in the application, and a minor restructure of the database was all that was required to boost the performance by 100x.
When I reflect on all of this, I realize how important it is to have a few full-stack programmers on the team. Sometimes it’s the CTO, sometimes it the VP of Engineering, sometimes it’s just someone in the guts of the engineering organization. When I think of the companies I’ve worked with recently that are dealing with massive scale and have to be obsessed with performance, such as Zynga, Gist, Cloud Engines, and SendGrid I can identify the person early in the life of the company that played the key role. And, when I think of companies that did magic stuff like Postini and FeedBurner at massive scale, I know exactly who that full system programmer was.
If you are a CEO of a startup, do you know who the full-stack programmer on your team is?
More from our friends Nate and Natty at Everlater – this time on debugging.
One of the most important techniques we used when learning how to code is debugging. It allowed us to do two things: fix our own code when it was broken and parse through others’ code to better understand how they were doing things.
From a backend perspective, debugging is essential. When I first started writing code, I would write what I *thought* was the correct way to do it. Ninty-nine times out of a hundred I was wrong and the code would blow up with some ugly exception that I had no clue about. Copying and pasting the exception into google got me decent mileage, but the real silver bullet I discovered was just start from the beginning of the method and step through it using the ruby debugger and figure out where I had gone wrong. Almost always I had forgotten to assign some variable or I was calling a method that did something different than I thought.
The other huge thing that is related is that I read and still read copious amounts of open source code written by other people. I’m better at understanding it now, but at the beginning I would take a debugger and walk through the code myself to try to figure out exactly how their code was working so I could make my own code better.
This is super important on the front end as well. Using tools like firebug or web inspector to take apart existing sites and figure out how they’re doing something or making something beautiful is a great way to learn better techniques for front end development. They’re also essential for figuring out how to fix a layout when it’s broken, or how to figure out why an ajax request didn’t respond the way you would expect.
This is a great, free resource to use when learning how to program and really helped us to bridge the gap from a basic programming book to what current philosophies on development were. A huge hat tip goes out to Jeffrey Kalmikoff for posting a comment on a previous post in this series that made me remember how important this is.
Somewhat related are tools that help you keep your code/team in order. By far the most important is a good version control system. This is something that they don’t ever teach even in college classes, but it’s hugely critical to building a project in the real world. We use git at Everlater and it’s been an amazing choice. When coupled with GitHub, it makes working even in a team of two seamless even if you’re working on almost the same code.
Also important is some way of keeping track of the quality of your code as your project grows. This includes a good issue tracker (we simply use the to-do list feature of the free version of Basecamp), and a good way to know when there’s something wrong in your application (Hoptoad is great for Rails devs.)
I know it’s been a few weeks since my last Nate and Natty / Everlater post on learning to program. I’ve gotten a few notes asking for more – expect a couple of posts over the next few days. In the mean time, here’s Nate’s view on how to divide tasks between partners – in this case him and Natty.
Having good systems in place around your coding is just as important as the coding itself. Natty and I spent a huge chunk of our time figuring out a great workflow that would allow us to program more effectively, and we think it’s paid huge dividends over the lifespan of Everlater.
- Mock up the pages super roughly with pen and paper and figure out the different database models
- Do Photoshop mockups and build the routes and models
- Actually build the pages in HTML/CSS and build out the controller actions associated with each view
At each point in the process Natty and I would sit down and talk about how it was going, and implement our side of the task. It bears repeating that this is also a highly personal part of learning how to program. The following worked very well for Natty and I but other people might be better off pair programming the whole thing, work at completely different speeds, etc. The most important thing is thinking about the workflow in general and making sure it’s a conscious decision rather than something that just happens.
As Fred Wilson likes to say, often the best content for blogs is in the comments. In this case, it was in an email I got from Boaz Fletcher in response to my post Web Sites and Books for Novice Programmers. Boaz made a very interesting observation:
“As for learning how to code, I think good storytellers make the best programmers. I used to freak prospective employees out by having them write a story for me instead of the “what’s wrong with this code?” tests. But it showed me who was able to think well, organized, creatively, and filled in the details.”
He also had an insightful comment about teaching kids to program.
“I had an exchange with someone in the industry about teaching kids how to program – or, more appropriately, how little there actually is to start kids off (think Alice or Scratch). Considering the ubiquity of computers in our lives, I think it’s untenable that most people are just passive users of the things. It should be mandatory to teach kids how to program. They don’t all need to become software engineers (never mind that I think most software engineers today, aren’t) but a basic understanding of how to build something simple and useful to them. Think about “shop” in junior high – hands-on manipulation of the physical world. So you may never need to lathe out a wooden bowl again, but at least you can hang a picture straight. Kids can browse the net, but don’t have a clue why their computer gets stuck when they’re trying to print a webpage.“
I’ve been thinking and talking about this particular construct a lot lately, especially in the context of NCWIT. A person younger than 15 years old has never experienced life without the existence of the web. Their view of the world, especially 29 years from now when they’ll be as old as I am today, will be radically different because of how the computers and the web are integrated with their life.
I never took shop in high school. I’m not mechanically inclined (or skilled) at all. Not only can I not hang a picture straight, I’m not sure I know what to do with a power tool. And forget changing the oil in my car. When I reflect on things I wish I had done more as a kid, it’s tinker with mechanical things so I’d be more comfortable with them. In contrast, I’m completely comfortable with anything that’s “not physical” – I like to say "I’m only interested in it if I can’t touch it.”
We are definitely living in a world where both are important, but the not-physical is becoming increasingly pervasive. Making sure that young people are tuned into this seems critical. When I think hard about this, there’s real insight in Boaz’s comment about the power of storytellers.
Thanks for all the feedback and comments on the Learning to Program series with Nate Abbott and Natty Zola from Everlater. In the last post, titled Web Sites and Books for Novice Programmers, I foreshadowed some of the tools that Nate and Natty chose to build Everlater. Now that you know how they got started, here’s what they ended up choosing.
The technology stack that they’ve ended up with has evolved over time. The very first decision – which web framework/backend language to use – was the toughest. Once again, our friend Google appeared – this time for the phrase “web framework comparison.” A few days later, the exploration shifted from simply finding and poking around in the various languages (most notably Ruby/Rails, PHP/CakePHP/CodeIgniter, Python/Django, ColdFusion, .net, and Java), to figuring out the salient points in the debate: speed, ease of use, active development of the platform, security, and cost.
Over beers, Nate and Natty put on blindfolds and threw darts at a board. After incorporating these results into their decision matrix, they chose Ruby/Rails mostly because they felt that it had an active community developing it and seemed to be the easiest to learn the quickest. It took roughly a week to come do a decision, start to finish.
After choosing Ruby as the main language they would be working with, they immediately began searching out every possible Ruby coding Meetup. Through those meetings they became connected with Boulder’s Ruby community which is an amazing group of incredibly smart people. They also found two great people, Charlie and Ryan who began working with Everlater for equity early on and helped make some of the key early decisions.