Project vs Product

I've worked on production software. I've also worked on projects with no intention of making money with them, purely to satisfy my own curiousity. And I'm working on a project that started off as a hobby project, but now I want to make money with it.

The kinds of things I focus on when I want others to pay for something vs when I do it for myself are very different.

When I make something for myself, I focus almost exclusively on novelty. I don't care if it doesn't work on someone else's machine, or if people can't login, or if it doesn't look good, or if it's hard for someone who is seeing it for the first time to figure out how to use it. The most important thing for me is: is it useful. I disappear into my cave for ages, and after many false steps and complications, emerge with a jewel, the likes of which have never been seen before, as long as you don't try to run it on your machine. Actually my track record for that isn't too bad.

On the other hand, when users are involved, I focus a lot more on things that are almost the same for every single product, but not so similar that they can be automated. Database integration, user sign in, working on multiple devices and form factors. I can't have the app be in a broken state for a significant amount of time. I have to be very careful messing with user data, and money. It needs to be easy to use for people who have never seen it before. Something that was a simple change when I was the only user becomes a delicate complex operation when users are involved. I need to perform a database migration, without downtime, whilst keeping old and new clients working.

All of that stuff is important in a product. If you don't have it your product won't succeed. Not having it is bad for your users. And you'll be flayed alive or forgotten. But, if that's all you do, you haven't made a worthwhile product - one that is not just easy to use but actually useful.

I'm going to refer to this as Project Mode and Product Mode. Project Mode is when you're working on something without users, and focusing on making it good, and Product Mode is when you have users and you're more worried about reducing defects than innovation.

Why Project Mode Is important

According to Modern Software Wisdom, we need to "shift left" all the things, and get our product in front of users before we even figure out what it is. There is no Project Mode. As a consequence, Modern Software is impotent. It's like a burger with the patty thrown out. A movie where they got rid of the plot, character development, and world building, and left in the kiss and punchlines, and credits. A book with a fancy blurb and author photo, available in all bookstores, perfectly grammatically correct, with no story.

Think about how you live your day to day at home, and what you do when a guest is coming over. There are conditions that you can tolerate living in that you wouldn't want others to see. If you had to be perpetually ready to host guests, there are certain things you'd miss out on. You wouldn't do spring cleaning. If your desk is falling apart but a guest is coming in a few hours, you wouldn't buy another one and put it together. Maybe you wouldn't read as much or experiment with new recipes, and actually be a less entertaining host.

When I'm in Project Mode, I do things that are almost impossible to do in Product Mode. I make significant rewrites, the kinds that can't really be done through a series of behaviour preserving refactors, because I'm changing the fundamental metaphors of my system, and specifying everything in terms of that. I'm getting rid of things, which is a huge no-no when users are involved. I have a much shorter feedback loops for making changes than if I had to preserve user data. I end up with a sort of Newtonian physics version of my system, that has fewer base parts, but is able to compose into more things than less compact systems that basically have a 1-1 mapping between the base parts and observable behaviours. I can do really weird, never been done before things, that at various points seem like it won't work until it is, and the system spends a significant amount of time in a broken state until it all works. I'll elaborate more on the properties of a good system in other essays, but the important point is that the constraint of having live users makes it much harder to significantly change the system to achieve these properties.

Now that I'm in Product Mode, things are different. My product is fairly niche, and I can only see people who would be able to use it once a month. So, for the last few months I would try to get releases ready for then, and I can see how the kind of work I've done has shifted. In the two weeks coming up to an event, I focus much more on reducing defects, and making things easier to use, rather than ambitious, novel, or useful things. For example, signing in, analytics, working on multiple devices. Because of the good base foundation I've made, I can still compose and create interesting new things, that I would not have been able to do without the initial Project Mode phase.

It's not just my projects that benefit from this, though. Some of my favourite pieces of software have spent considerable time in Project Mode.

Rich Hickey wrote Clojure whilst he was on sabbatical. He famously described his process as "hammock driven development". As a result, Clojure is extremely powerful, it's a lot simpler than most other programming languages, but able to accomplish much more.

The same is true of Elm, which originally started as a thesis about Functional Reactive Programming. Evan Czaplicki, against conventional open source wisdom, prefers to let issues pile up before thinking of a solution that address all the issues, rather than hacking on and immediately releasing a fix for each issue. Releases in Elm take a long time, and he does a lot of work in the background. He prefers not to make promises, and only when something works he talks about it, and the dead ends he encountered. Of course both of these projects, after they've reached some level of success, had to switch to Product Mode. Although they aren't selling their languages, they are being used by companies so money is on the line. They had to focus on tooling, community, and backwards compatibility. But the initial work they did in Project Mode, has paid off by producing something significantly better than other languages.

Xerox Parc also operated in Project Mode quite a lot. They made significant contributions to laser printing, the Internet, GUIs, object Oriented Programming, the computer mouse, tablets - Steve Jobs actually visisted them in the 80s and was highly inspired. And, of course, they made Smalltalk.

Alan Kay talks a lot about what they did and it's clear they operated in Project Mode. In on of his talks, he addresses a previous speaker talking about how we need to make and ship things in 3 months, and calls that laughable. The Xerox Parc people operated in years. They would write things twice, first to try out different things and get it to work, and then to write a simpler, more elegant and performant version.

Why Project Mode Is Rare

Depending on how you're politically oriented, you might view this as an example of Capitalism scoring an own goal, or as evidence that all this talk about "novelty" and "innovation" is nothing more than starry eyed idealism that falls out of the window when you try to fund it.

I don't think there is anything inevitable about the disproportionate focus on Product Mode. I think it's a consequence of Modern Software Methodology, and that's a consequence of ideas that have gained prominence by historical accident.

Case in point, in a previous job there were times when I was in Project Mode, and despite not using Clojure, I look back at that fondly. They were a small company, without lots of spare resources, so when they wanted something done they were often in Product Mode. But, things ebb and flow. Especially when releasing a mobile app, there'd be heavy periods of Product Mode trying to get things released to the store and making sure things don't crash because updating the store is a pain, then times when the app isn't their top priority and we had more autonomy to work on it.

This company wasn't clued up about "best practices" or methodology at the time, which I saw as a downside back then, so they weren't actively sabotaging themselves to the same extent as other companies.

How Can We Have More Project Mode

There are things that companies can do to spend more time in Project Mode, and therefore spend less time lying about how innovative they are.

Don't Rush Into Product Mode.

In entrepentuership circle-jerks, not launching immediately is procrastination and fear and waste or whatever. In reality, the time my current project spent in the cave before releasing it to production was incredibly valuable, and I'm looking forward to doing more of that before getting the next big part ready for the public. This isn't just my opinion. When people use my project, the parts that they like the most, are parts I made during Project Mode.

Don't break everything down

The second most basic thing is to get rid of the round hole that rejects all the square project pegs, the one we call "sprint planning". By carefully breaking everything into itty bitty chunks on Jira where everything has a "user story", and takes "3 story points" to complete, we basically guarantee that we only do things that aren't novel. Far from sounding an alarm when something has "8 story points" because we haven't done it before, we should just do things. Technically, this takes us to a sort of hybrid mode, but it does swing the ratio to a better place.

Get Rid of Scrum

The third thing, similar to the second, is to remove the time-wasting we call scrum, with meetings everytime someone needs to take a dump, so that we have time to actually work, and focus on difficult things. There's a double benefit to this. The first benefit is, whenever we have users and we have less time to work, the first thing that gets cut off is the projecty stuff. When the hot air balloon gets too heavy, first people throw off the stuff they don't really care about, then they start stripping. The other benefit is that the projecty stuff especially tends to require the deep focus more.

Go On Project Mode Retreats

Finally, go on Project Mode retreats even after releasing a product. Obviously do it on a separate branch so you don't introduce defects into the live product. And test it well before merging. See if you can introduce and work on something locally but comment it out in production until you're ready, but that's not always an option.

What Doesn't Work

Lots of people either don't believe in the distinction between Project Mode and Product mode, or think it's possible to launch something to users and then mitigate the issues with that.

They bring up things like:

Third party tools and automation

This doesn't work, and almost always backfires. For a few reasons:

• External tools are harder to override than something you have control over, and you need to override them if you're making anything significant.
• These tools have overlapping concerns, which leads to synchronisation issues. At work we spent ages trying to make it possible for users to change their names, because the identity service, the payment service, the CRM, and the applications users column each had different validation requirements that meant the same name can be out of sync, and we couldn't tell which to treat as the source of truth.
• You end up spending a lot of time translating between multiple made up things, restricted by any bad decisions made by the third party.
• You don't just want generic "sign in" or whatever, you want a specific kind for your application, and everything else is affected by that.
• No automation can deal with the fact that there are people with the old data, and new people. And when you transition your data, both clients need to work because the client and server, no matter how closely you sync them, are not the same thing. So you have to do things like add a new field that no one uses, and make sure it has the right value, then when everyone's data has the new field in there, make the application use that instead. Instead of just changing the field once and having it work.

The fact is, you can't beat 0. Any tool that purports to get you there is lying. In Project Mode you don't have to deal with that stuff at all, in Product Mode you do, and it will take a significant amount of time.

Hiring More People

• The data synchronisation problem is not solved by having more people. No more than 9 people can delivery a baby faster than 1.
• Actually, by having more potential merge conflicts, and more things edited simulataneously at once, there is even more pressure to do things in tiny chunks to reduce the chance of merge conflicts, and spend even more time in Product Mode.
• Companies don't typically allocate different types of task at the same time, with some falling under the bracket of product mode and others falling under the bracket of project mode. Typically everyone has to move to the same beat, but that doesn't work.

Refactoring

• There are some changes you can eventually make by refactoring whilst every change goes to production, but it takes ages and that's important. It also requires more planning. So it's not done very often. Which means most of what you do is Product Mode rather than Project Mode.
• Refactoring gets you to local maxima, but it doesn't jump accross peaks. You can sometimes get there by having a bastard hybrid system, in production, for a while, until you can get rid of ancestral cruft, but that's a risky thing to do, and doesn't get done very often.

Conclusion

Important parts of software are best developed before putting it in front of users. Hopefully, by identifying and understanding this, people can be less surprised by what they're working on, and can be more deliberate about when to release to production.