Software disenchantment

Leadership


Back in university, the professor asked the class what the definition of leadership was. People proceeded to give a lot of touchy feely kumbaya responses about how a leader is nice and understanding and inspiring.

What horrified me wasn't the kumbaya answers, it's that for them that was the definition of leadership - the question wasn't asking about the most effective way to be a leader or what the best leaders did. No one had thought to answer the question "what is all this for?"

Leadership has a moral aura around it. Leadership is something people often mention in their CVs. It's something companies claim they want. People attend leadership workshops and courses.

At this point I usually tune out when people talk about leadership, it triggers my bullshit radar.

But, the fact is, although I think that individual technical skills are vastly underrated, I've started my own company and I need other people to make it work, so I might as well think about how to effectively lead them and for me that involves figuring out what leadership even is.

Defining Leadership

As I see it, leadership is the ability to make a group people act in a way that achieves a goal.

This says nothing about how to achieve it, just what it is. Maybe you make a group of people achieve a goal by being cruel to them, or maybe by being nice and considerate, the point is that the test is whether the goal is achieved. If it isn't, it does not matter how cruel or nice you were as far as leadership is concerned.

Getting people to achieve a goal has two aspects to it - two almost definitional sub-components.

The first is that the steps you make people follow lead to that goal.

You can evaluate this without making reference to other people. This is the ability to decompose a goal into subgoals and steps, and effectively prioritise and come up with a plan of action that leads to your goal. Without this ability, even if you are good at getting people to do what you tell them, the ship will still sink.

The other part of leadership is getting people to do what you want.

Great Leadership

Ok this is leadership defined. But what is great leadership?

I loosely defined leadership as getting people to achieve a goal, but that's too vague. Both having a picnic and landing on the moon are goals, but landing on the moon is far more grand and novel, and thus requires greater leadership.

A regular leader achieves things, and a great leader achieves great things.

When I say the word great, I mean like Alexander The Great. A level of accomplishment so high that it seems like it could not have been achieved by mere mortals. Not just something that was new at the time but commonplace since, but rather something that even since people are still in awe, and would not believe it was possible if it hadn't been done. Tony Stark making a suit in a cave that Obadiah Stane struggled to replicate in the lab after. Greatness is not about being in the news, but in the history books.

Let's keep this in mind as I analyse leadership. This is the limit we'll be approaching, and as we get there many crutches that are enough for adaquate or good leadership don't work if you want to be great.

The other thing to keep in mind is that it's not just about achieving a goal, regardless of the means. Whilst I want to analyse the ability to achieve a goal from a morally neutral perspective, putting aside whether I think a leader is treating people well and instead focusing on the outcome, I don't want to completely decouple means from ends.

Imagine 3 different restaurants each hiring a cook to apprentice under their respective head chefs. In the first restaurant, as long as the cook produces something that resembles the order, the chef is satisfied. The cook quickly whips up something that looks like spaghetti bolognese, and the chef doesn't care about the ingredients or how they put it together. In the second restaurant, the chef is far more strict on the cook. The chef needs to know every ingredient and the proportions, the techniques used, and how the final result tastes. For any given order, it is more difficult to produce in the second restaurant than the first. And in the third restaurant, the head chef is even more strict, about things that don't seem to have any causal connection to the quality of the dish. From which hand to use, to how to speak, and what food to eat in your own time. I think we can agree that the third restaurant is excessive. This is the micromanager archetype. But when choosing between the first and second restaurants, which one produces better food? The second one. I haven't even said what their standards are, just that they have standards. The first restaurant moves fast and breaks appetites, it's the npm of cuisine. Menu mercenaries. The second restaurant is Lisp. And the last restaurant is Haskell.

Leadership isn't about petty control freaks satisfying their lust for power. But it's also not about people that want money doing whatever. It's about turning a specific vision into reality through other people.

The contribution of a leader is best understood counterfactually - imagine you selected a random other person in charge, how well would they fare? Would it make no difference or a huge difference?

I have defined leadership in a way that says little about how to do it - no implementation details. That puts us in a much better place to discuss what a good/great leader does.

Unpacking The Definition

Let's look at both properties in turn.

Effectiveness

The first is some combination of productivity, effectiveness, pragmatism, engineering skill, tactics, strategy, and vision. As I said before, we don't need to look at leaders for this. Great novelists, artists, scientists, inventors, chess players, and athletes all exemplify this aspect. And, as far as I can tell, within all these fields there are various kinds of approaches that succeed.

Some well known novelists are "outliners" - they have a plan for every book, whilst others are "pantsers" - they follow the story and see where it leads. They have different sorts of productivity routines and tendencies.

If I had to pick one underlying trait that was relevent to leadership - especially the great kind - it would be "practical imagination".

If you're doing something that hasn't been done before, you can't rely on recall. But you also can't rely on ordinary imagination. You need to come up with different things, but also things that work. This is what I call mental simulation. Or practical imagination. The venn diagram overlap of things we don't have now, that are physically relialisable with our resources.

If I had to pick another underlying trait, or cross between two traits, it would be "smart and gets things done". There are plenty of people with one but not the other. People who are really smart but don't do much. And people with a strong action bias who aren't all that bright but gosh darn it they're always doing stuff. The intersection is far less common, but among the highest performers it's quite prevalent.

E.g, there are people with all sorts of interesting fiction ideas who don't write anything, but can talk about it for hours. And then there's plenty of uninspired fiction but at least they wrote something. But it's the people who have both put in the time actually writing and honing their craft, but also have such a brilliant world-changing way of looking at things that write the best fiction.

Getting People To Do What You Want

As for getting people to do what you want, this is something I understand far less. That said, I have made some observations.

It is way easier to get people to do what you want if it involves doing things they've done before many times. They basically just go into autopilot. But, since I defined great leadership to include novelty, a great leader has to be able to get people to do what they want even if they haven't done it before, and therefore has to be more involved.

It's also easier to get people to do what you want if you have some title bestowed upon you by some respected authority. Like being a teacher or a manager. If you were shipwrecked on an island with a bunch of people, and the captain was dead, it would be much more difficult to lead them than if you were given some manager title at a software company. The project manager can afford to be far more affable than the island guy.

The downside of bestowed authority is that the origin of this authority places limits on what you can do with it. If you're a priest you have this bestowed authority, and you have some scriptural interpretive latitude, but you have to tiptoe carefully around blasphemy. If you're a project manager at some company that does scrum, erm, you get to decide what time to do standup and what days to do retro, but you'll never be allowed to not do Agile. Even royalty find that they have plenty of duties, and very little opportunity to make a difference, without a war.

Absent bestowed authority, leaders exude authoritativeness, basically by being confident and assertive ... even when they're wrong.

There's a tension there. To be good at the first aspect - making good decisions - a certain level of doubt and open-mindedness is needed to correctly change your mind with new evidence and approach being more correct. But, people won't listen to uncertain commands, they want confident commands. I am not sure how this is solved. Do great leaders have dual personas? Do they think privately until they're confident? Or are they just reckless and lucky?

I suspect that this also varies by who you're leading. If you're leading a random assortment of people then maybe the chest-beating gorilla is the way to go, but if you're leading a bunch of engineers on the spectrum, they won't respect you if they percieve you as not tracking reality. Maybe they'd appreciate quantifications of uncertainty, with a rationale for why the chosen course of action maximises expected return.

That said, whilst you may not be certain, you can't act uncertainly. You can't put a chess piece on two different squares. You think hard, then commit, and only course correct if you have evidence. Especially when you are coordinating multiple people, you want an aligned strategy.

Talking confidently is not enough, or at least, not when you're trying to do something great. If you want results that are well above average (or well below average), you can't do/command normal things. Ordinary actions yield ordinary results, and weird actions yield different results (mostly failures but the occasional wild success). If you look at the top performers in any field, they aren't doing the same things as everyone else but somehow getting different results, they're doing different things that might seem nuts, but gets them abnormal results. If you want to be a great leader, you need to confidently tell people to do things that seem nuts, and stand firm in the face of many incredulous scoffs. It helps if they respect you, or if you have a track record. Or if you have a reality distortion field.

The more people you have to lead, the more your influence gets diluted. It's mathematically not possible to micromanage a large number of people. So as the number of people grows, you end up with this wave that you can only influence indirectly, perhaps by introducing a hierarchy of managers that report to you, but it's still not the same as if you had a small team you can directly lead.

If the number of people you're leading is 0, then you're just working by yourself and leadership does not apply here. If you're leading 1 person, then unless you're also an individual contributor - if you spend all your time micro-managing then it's like texting with gloves on. You're better off just doing it yourself. And if you are leading 2 or more people, unless you have them work 8 hours each and not at the same time, you can't be directly observing, controlling, and intervening every minute.

So micromanagement does not work, and leadership is hard to scale. If you just let a bunch of people do their thing, however, you have problems. They will be uncoordinated, stepping on each others toes, and not moving in one coherent direction. And also, it won't matter who the leader is, they will do the same things - so the influence gets diluted.

To deal with this problem, you keep the team size managable, and look for strategic ways to influence rather than directly control. One concept that is useful here is the game theoretic idea of focal points, also known as Schelling points. Imagine you need to meet with someone in New York, but haven't agreed on a time or exact location. What is a good default that they will likely think of? In experiments, most people answer "Grand Central Station at noon". A focal point is a solution people choose in the abscense of communication. Applying this to leadership means setting the context and information, and letting the team fill in the blanks. Rather than direct control over everything, set the key constraints that propogate across the team.

I've described how to get people to do what you want if they aren't already doing it. Somtimes you're in a position where they already are doing what you want, and you better not interrupt them. It's amazing how often people fuck this up. The classic example is when programmers are constanly interrupted by meetings. A programmer that is actively working on fixing bugs or implementing new features is frequently told to stop for some inane ceremony. Just as a programmer is expected to produce code, and a designer is expected to produce designs, and a copywriter is expected to write copy, managers are expected to schedule meetings and write emails. If a manager has an empty calendar, it looks like they aren't doing anything, and they get the axe. When companies get large enough that they hire middle managers, they are basically productivity foot-shooters. This is why so many people fail the simple task of doing nothing.

If you don't have to engage in productivity theatre by interrupting your team all the time, it's easy enough to let them do things they are already doing. Especially if they suggested it themselves, and are very excited about it.

Aristotle thought that movement requires a mover. For something to move, it needs something to actively push it, and once it stops being pushed, it will stop moving, and head back to its "natural place" - his explanation for why things fall down without being pushed. But Newton showed that movement does not require constant pushing, that things can keep moving forever. It's only because of friction and air resistance that things on Earth slow down and eventually stop, but in space it takes no energy to keep going at a constant velocity forever. Motivation is a bit like this. People focus a lot on moving others, through carrots and sticks, but for every step it requires a push. When people are intrinsically motivated, however, you don't need to keep pushing them. One push and they're gliding. Introduce enough friction and you need to resort to the Aristotelian model.

This is all well and good when people are doing exactly what you want them to do by themselves, but what about if they're doing something adjacent to it? There are two scenarios. The first scenario is where the thing they're doing is just as good as the thing you want them to do, and can be done in any order. In that case you're better off letting them do the thing they're doing, because it will take less management on your part. The second scenario is where they're doing something you want eventually, but the thing you want to do is actually much more important and urgent. They're sorting out your accounts, but you want them to deflect the asteroid coming at you, so to speak. They're very excited to work on your social media, but right now the biggest issue is onboarding, and until you fix onboarding any effort in social media is wasted. Here is when you put your foot down, and interrupt them. You should feel a little bad at doing this, but still do it. And, if you're transparent enough about the state of things, and give your team a sense of ownership as well as knowledge, this will happen less often because they will independently come to the same conclusions regarding what is more urgent or important. Like with the restaurant example earlier, it's important to distinguish which things causally impact the goal, and which things are personal preference. Intervene when things deviate from your goal or core principiles, but not when they're inconsequential differences in preference. In fact, different preferences can be a good thing - I don't care about social media, but someone excited about it will do a much better job there then someone more instrumental about the whole thing.

Not interrupting motivated people is enough to be good, but to be great you need to foster that motivation. You can't create it from nothing, but you can grow the spark, and transmute it. One person who was great at this was Steve Jobs. He got John Sculley to leave Pepsi and work for him by asking "Do you want to sell sugar water for the rest of your life, or do you want to come with me and change the world?". So it turns out that cheerleading is an important aspect of leadership after all.

Finally, another seemingly obvious one, you can't get people to do something that they aren't able to do. If you want to achieve great things, you want people with the capacity to do them. In different fields that will involve different qualities. If you want to win a world cup, you want world class athletes, not couch potatoes or even active middle aged people. If you want to make the first nuclear bomb, you need the brightest minds on the planet. These people literally built some of the first computers to write monte carlo simulations to simulate nuclear diffusion in the bomb.

There are two parts to this: innate talent and learning capacity - potential, and growing to their potential through practice. If you're doing something great you can't wait for people to already reach their potential, you have to make them. It won't do much good to wait for the Manhattan Project to be completed before getting those scientists to make the second bomb.

Getting people to do things they can never do is a waste of time and effort. So you need to be able to recognise ability but also potential. Maybe potential that people don't see in themselves, in areas they weren't given a chance to develop before. Sometimes it's easier to recognise potential in others than yourself because of the fact that anything you do seems normal to yourself. You don't inhabit other people's minds to know how yours compares. If I can do something, it seems normal to me, until I'm exposed to lots of evidence that it is not normal in others. But if I see someone's mind do something I can't, maybe learn the same thing much faster than I did, that stands out. And if they spent their life focusing on whatever it is they're good at, they might not know how they compare to most people.

My Qualities

How do I fare?

I think I'm pretty good at that first aspect. I'm relatively disorganised, but I'm generally quite effective. I have been doing solo projects for years that blow peoples minds.

As far as the second aspect is concerned, well, that's my weakness. Maybe I'll never be a great leader. I'm really not a people person. I also don't like conflict. What I do have going for me is that I can be quite stubborn. I can also get quite excited about certain things, and that can be infectious. And I'm not a foot shooter, for the most part.

Can Leadership Be Taught?

The underlying assumption when reading an article like this is if you just do the things I say, you'll be a good/great leader. That's certainly the idea behind so-called leadership workshops. But I think most of the time trying to be instrumental about these things is ineffective. Putting on a persona is is often disingenuous, and noticable. What is possible is observing your own deficits in practice, and resolving to do better. Not just act in a different way, but work on becoming the sort of person who does it right.

The Varieties of Leadership

I've outlined the two definitional components of leadership, and described a number of strategies for achieving them. The point is not to say that there is one way to be a leader, nor is it to say that anything goes. When a computer evaluates a chess position, it often finds a handful of moves that are almost as good as each other, people with different styles might rate differently. And most moves are terrible. Often the top move is a tiny bit better than the second best move, but feels far less natural, it's harder to find what to play next. In those cases the best thing to do is to play the second best move. And so it is with leadership. Achieving these properties is a filter over normal behaviour, but you should pick the strategies that are most accessible.

Online you often see two leadership archetypes:

  • The macho chest-beating gorilla whose word is final and shoves people around
  • The sappy heart warming baby-sitter who eats last and makes everyone happy

The truth is that although each of these archetypes is satisfying to certain groups of people, they are not very effective. If you picked the nicest person you went to school with, and the meanest, none of them are the best leaders. It's clear that you can be an effective leader and an asshole - e.g Steve Jobs. But although he was cruel, there are far more cruel people.

I suspect that if there was a chess engine for leadership, the top "move" would be "kind of an asshole", the slightly worse second best would be "fair", the third best would be "kind of good", and then "shuts down people with withering glare", "people pleaser", and "chest-beating gorilla". As the number of people grows "kind of an asshole" reduces deviation from the plan more than "fair", but doesn't miss out on good ideas as much as "chest-beating gorilla".

Leadership Is Overrated

Leadership is often viewed as "the next step" once you reach a certain level as in individual contributor. People often view it as higher / more important. It makes people warm and fuzzy. I think it should be removed from this pedestal.

Leadership is a means to an end. What's important is the end. If you command a medieval army, you have no chance against someone that can develop and launch missiles.

Who is greater, Alexandre Dumas (author of the three Musketeers and the Count of Monte Cristo) or some scrum leader? Picasso or the dozens of US Presidents no one can name? Isaac Newton or the thousands of kings even most historians don't care about? History shows that you don't need to be a leader to achieve great things, and that many leaders did not achieve great things.

Leadership is relevant when the thing you want to do can't be achieved by one person, and is specific and targeted enough that you don't want alternate methods of organisation like self-organisation or committees involved.

For example, in a sports team where there is low variance between individuals, and teamwork and strategy are more important. Even then, there are outliers. Take football (soccer to the Yanks reading this). Football is a sport where there are relatively few goals compared to the number of players (as opposed to basketball). There are 11 players on each side, and on average maybe 1 or 2 goals. Any one player is unlikely to significantly increase goal contributions by themselves. It's more important that the whole team coordinates to stop the other team from scoring, and correctly transfer the ball to each other so that they maintain possession and increase their chances of scoring. Teamwork and leadership/strategy. They also have to adjust to the kind of team they're playing, and what the current score is. And yet, despite this, Lionel Messi exists. He scored 91 goals in a calendar year, which is more than many of the top teams. He also contributed towards 400 assists, which is almost 100 goals more than second place. The best players in the world, in the best teams, with great chemistry and managers and teamwork, couldn't give humans a chance to compete with an alien.

In software, which is the main thing I talk about in this blog, I think that the variance between individual contributors is even greater, and there are two reasons for that:

  • The sheer power of modern computers
  • Turing Completeness

Turning Completeness means that anything that computers can do can be done, basically. And that encompasses a lot of things. If you can find a way to express it, and it falls under the subset of doable things, then it can be computed. And with a bajillion operations a second, it's not just theoretical it's practical. Now, there are things which mathematically grow even faster than even the best possible hardware, problems where the number of possible states exceed the the number of atoms in the universe, but there are plenty of good heuristics, and in many cases these problems can be solved in reasonable time complexity if you know how. When you combine these things together, someone that has truly mastered programming is the closest thing we have to omnipotent. And you really have to master it for this to be possible. The less able someone is to express something in a way that is computable, or in a way that doesn't squander resources, the less they can express. Just like the books and movies, one wizard who has mastered magic is a world threat that mountains of mediocore magicians can't even touch.

Thinking that you can improve effectivenss by "leading" multiple ordinary programmers is like thinking you can get to your destination faster by hiring several ordinary cyclists rather than one Olympic cyclist.

Leadership shouldn't be viewed as the "next step" in every domain after a few years of skill as an individual contributor. It makes more sense in some domains where the variance in skill is lower, but many fields could benefit from much more focus on becoming even more skilled as individual contributors.

Conclusion

Hopefully I've provided a level-headed and analytic take on leadership that leaves you knowing what to look for and how to assess how effective something is, better than the drivel that is out there at least. I've also tried to put leadership in the proper context with regards to different kinds of activities. Whether something is an effective leadership trait is a question that should be analysed functionally, not by whether it makes you feel warm and fuzzy.

Back to home