Do engineering things that don't scale

One of the most popular blog posts from Paul Graham (founder of YCombinator) tells founders to do things that don’t scale. While good founders do a decent job of internalizing this for all sorts of parts of the business, I still find that engineers at startups are horrible at making the right tradeoff when it comes to software and code. Part of it is that it feels bad to do things manually; it’s almost the antithesis of being an engineer. The other part is that scaling and automating are often more exciting engineering problems and make people think they’re better engineers for doing them.

However, people rarely use these reasons as justification for working on scaling/automation. Instead, they go off about how much time they’re wasting and claim that they’ll save themselves and the team so much by spending a week on it. The XKCD chart below shows the actual tradeoff of automating a repetitive manual task across the two dimensions of time and frequency. Note that this is over a 5 year window, which is way more than a typical early-stage startup wants to optimize for. I’d argue that saving time over a one year window is about as much as is worthwhile, so we can divide everything by 5.

What’s the first takeaway? Well, anything that you do daily or less often that you would save 30 minutes or less on is not worth spending a week on. That right there eliminates the vast majority of things that people try to do; if you are rigorous about making this tradeoff, you’ll find that so many things don’t meet that bar. One fundamental reason is that humans are incredibly adaptable to situations, and so even if your repetitive task is a slight variation each day (e.g. a job fails and you have to re-run it), you probably won’t have too much trouble dealing with it. But automating to handle all of the variations can be extremely challenging, as you have to both understand and encode the logic of the entire space of the problem.

Engineers who have an objective function that aligns well with the company’s success will naturally make the right tradeoffs, whereas those with other (not necessarily wrong, but different) motives will not. Part of being a great engineer at a startup is ruthlessly prioritizing the right problems to work on, and often that looks like leaving a bunch of manual things the way they are because there are more important things to spend time on, like building your product and helping customers.

I’ll end with a final note which is that there are tons of situations in which it is correct to automate things as well, and inexperienced engineers/teams will make the mistake in the other direction. If something is waking people up at night, fix or automate that immediately. But by far the more common way that “good” engineers mess up is over-indexing on how automating some manual job will save time, when in practice it doesn’t save that much and takes longer than they expected to do it. So make sure your objective function is aligned, evaluate tradeoffs rigorously, and don’t worry if it feels like you’re doing a bunch of engineering things that don’t scale. If you make it to the point where it matters, then you’re in good shape.