Holding a Program in One's Head (2007)

> You never understand other people's code as well as your own. No matter how thoroughly you've read it, you've only read it, not written it.

There is certainly some truth to this. On the other hand, it's possible to become blinded to defects in code you've written yourself. You see what you intended for the code to do rather than what it actually does. Reading someone else's code, it can be easier to see what's really going on, since you just see what's there.

IMO this is one of the most important skills for a developer to have. In an age of Github Copilot and similar systems, it's both far more viable (because, per OP's #1, you won't get distracted by the "side quests" of implementing utilities you need, when you can just tab-complete to get them) and far more vital (because with that productivity increase comes an increase in the complexity of problems you'll be asked to tackle at any given level of seniority).

My advice on this would be: never be afraid, and even force yourself to, follow the chain of how a certain function is implemented in third-party libraries you are using. Set up your IDE to be able to click into functions, make that muscle memory, and perhaps even have a dedicated project you can bring up for your site-packages or node_modules. Don't just rely on documentation, see how things actually work a few levels deep. You'll gain a deeper understanding of the code you're using, as well as incredible practice for when you need to do this on an existing first-party codebase.

Oh, and if you can, get one or more large 4k monitors, and split your IDE into 4 quadrants! It's certainly possible to hold a codebase in your head on a small laptop screen, but being able to see the code you're working on alongside its dependencies and dependents makes this far easier!

> […] bottom-up programming, where you write programs in multiple layers, the lower ones acting as programming languages for those above

I like to explain this as “hide the bad parts behind a good API”. Anything interesting is going to require “bad parts”, which just means the low-level, difficult work. From there, compose up and up until a high level orchestration is achieved. It works so much better than the bad parts being distributed everywhere! That’s what you’d also call a “leaky abstraction”

A good way to keep a program in your head:

Break it up into a few or several smaller programs that interact through clean interfaces. Then you can keep one smaller, simpler program in your head at a time, then integrate them at the end once all the smaller programs are working.

Brilliant article. I read a brilliant book written by Jessica Livingston (Paul Graham's wife) called Founders at Work. In the book Steve Wozniak talked about how he got designs done fast by keeping it all in his head. Ever since I read that I have tried to do the same. I would recommend this to anyone working on their own project.

Of course in a corporate setting it is much harder to do as you have code written in multiple languages using multiple patterns stored in different places, so often there is not underlying architectural thread to hold it all together.

I once watched a presentation by Dan North where he said that a microservice should never be bigger than your head. What he meant was that all the code for the microservice should fit on your screen and you should be able to put your head against the screen and it should cover the code.

Yes, this was in the microservices-heyday.

I'll call out

   7. Don't have multiple people editing the same piece of code. 
   You never understand other people's code as well as your own. No matter
   how thoroughly you've read it, you've only read it, not written it. So
   if a piece of code is written by multiple authors, none of them understand
   it as well as a single author would.

On some level it's true but it is also true that most of the world's code is in some sort of maintenance mode and the original developer is not always available. When I work on code that I think would be difficult to maintain I write a lot of comments to explain invariants that should be followed or anything strange. (Hmm, there are three different kinds of "row" talked about in this source file)

If you have a front end-back end system and you want to do something simple like add a field to a form there's a strong case for a single developer or maybe paired developers to make all the changes to make that change happens as opposed to assign two developers (not paired) to communicate with each other about doing the task, or worse yet, assign two teams. You might have had two people build out the front-end and back-end systems but for sustainable maintenance one person should be able to ship a feature.

I more or less agree with the main thesis although I would point out that typical software development practices in industry make this very difficult. Fred Brooks told us to plan to throw out the first draft. Industry says ship it.

Another PG classic masterpiece - thanks for re-posting, I agree with all eight points, and this should be mandatory reading for people that manage developers particularly if the new manager does not herself/himself have a development background.

Just to elaborate on "start with a simpler subcase": you can even start with a single example and do what your code will be supposed to be doing automatically by hand. After your draft architecture stands, you can even hardwire what each component does for that one example to see if things fit together before implementing the general method for each component, which provides an opportunity for early integration testing that is priceless.

The current essay recommends building bottom-up primitives that serve as primitives or languages (DSLs) for the next levels. This is true, but you can do this upside down. For example, you can write code that looks like pseudo-code because implementations of functions you call do not (or not yet) exist. The code directly reflects how you think, and you worry about the implementation later. A special instance of that is API design, where you design interfaces without implementing them yet; you can then write client code and judge if the hypothetical API would be flexible enough, be easy enough to use, and be feature complete and orthogonal. When designing an architecture in a team with multiple people, you can use paper cards with an example piece of data that you pass from person to person, with each person "performing" a component. They can then check that they have what they need to carry our their task (otherwise there may be bad surprises when integrating components later).

I found that some people are more leaning "top down" and others more "bottom up"; I like to mix both styles, with system architectures designed top down and the core algortihms inside their components often being designed bottom-up, sometimes also top-down.

Ironically, looking only at the headline, one could say that abstraction enables one to solve problems without getting the whole problem in one's head, or problems bigger than what anyone can hold in their head, at least not at the same time, which kind of is the whole point made by SICP.

Take a look at the list of people who read the draft of this post.

> Thanks to Sam Altman, David Greenspan, Aaron Iba, Jessica Livingston, Robert Morris, Peter Norvig, Lisa Randall, Emmett Shear, Sergei Tsarev, and Stephen Wolfram for reading drafts of this.

Past comments: http://news.ycombinator.com/item?id=45698, http://news.ycombinator.com/item?id=2988835

I don't know how far this can go but it's a feeling that I've had which is relevant and some people might resonate with.

I used some tiny devices (e.g. Casio PB80 http://oldcomputermuseum.com/casio_pb80.html) as a kid to write small programs on in BASIC. It has only a single line of display and you had to sort of remember where things were to make your GOTOs work properly etc. This means that most of the program had to be in your head. You couldn't easily jump between things and see the whole thing as a single screen of text. I also remember reading that people like Ken Thompson wrote significant parts of the original UNIX on a line editor (ed). That would also mean that most of it had to be in his head in a semi parsed format.

Sure, with modern editors and IDEs, you can outsource that to the computer and focus on the more valuable stuff but I still wonder how much the grunt work of remembering things in detail is under appreciated in creative work.

I'd add "Focus on the Nouns, not the Verbs", or more prosaically, the data and the changes that occur to data, not the process of changing it.

A great article that accurately describes the challenges I've faced over the past 12 years while creating the best project for programmers and the various attempts to find work and funding, observing all the madness in the organizations It seems like the best of it was left in 2007, from which we still draw inspiration and knowledge, while since 2008 we've been living in a new world of marketing and wars. The insatiable interests of faceless systems are destroying everything alive in this great science of "programming".

As a developer who has aphantasia (can't visualize at all), I'm curious how much visualization is used by others when "holding a program in your head". I can hold a program in my head just fine without visualizing anything, I'm just curious how much of it involves visualization for others.

"Use succinct languages" is somewhat at odds with "Write rereadable code." There's a point beyond which making your code more succinct makes it more difficult for a human to parse. This can be somewhat mitigated by comments but I'd rather just read more readable code than more succinct code.

> You never understand other people's code as well as your own.

Not only is that often false, but there are times when you can understand another author's code better than that author does. Like to see through it and why it cannot possibly work, while they are laboring toward that.

>Probably the best we'll do is some kind of hack, like making the programming parts of an organization work differently from the rest. Perhaps the optimal solution is for big companies not even to try to develop ideas in house, but simply to buy them.

I remember the CTO of a big American bank telling me they didn't want to develop any software in-house. Their plan was to buy everything in. At the time I thought it was rather strange but having dealt with a lot of enterprise software since then I can see why.

> You can magnify the effect of a powerful language by using a style called bottom-up programming where you write programs in multiple layers, the lower ones acting as programming languages for those above. If you do this right, you only have to keep the topmost layer in your head.

And if you do it wrong (overengineering), you would need to learn not just one, but as many programming languages as there are layers...

Reminds me of when I spent some time learning functional programming. I did a Scala course and I still remember solving some of the programming challenges. The slightest lapse in concentration - I would immediately lose understanding of what I was writing. It would turn from something amazingly simple and elegant into a thing that was incomprehensible.

Which is why working memory performance is so tightly correlated with so many factors (maybe not all) involving programming

How relevant is this when AI has the ability to both write and debug massive code bases?

I imagine a future world where 100% of coding and debugging is done by prompt not by editing code.

I call this program in one's head "suspended comprehension" as it has a degree of transience and fragility that requires effort to resume where you left off.

It's (mostly) not about the code. It's about the data structures and their relations.

As someone who has written programs professionally his whole career, I find the idea that any programmer holds an entire non-trivial program in their head to be laughable. Good software design means writing programs in such a way that you absolutely do not need to hold the whole thing in your head at one time.

> And of course you can't safely redesign something other people are working on.

Unless, you know, it's automatically tested to hell and back.