The mindset towards hierarchy has been changing fast in the last decades. If you saw my last article, Your team at work and the american civil war, it will be easier to understand why I think we should change our mental model about how to organize the teams at work. In this article, I will attack specifically the mental model regarding hierarchy.

I start by giving context on the changes that have been happening in the last decades, then discussing about what is the direction the software market is taking and whether it’s the right one, to finally give you my opinion on what we should change in our work environments.

A little bit of history

Back in the day, when I left college, working as a software engineer was much more problematic, in average, in any organization, than what I see in the market today (as of 2023). The big enemy at that time was bureaucracy - software and internet were new things yet and nobody knew exactly how to manage software engineers.

That’s probably because long before the internet boom and the explosion in the number of software companies in the market, the management theories that dominated universities and companies were all based in the reality of car manufacturing factories. No surprise, the concept of “Software Factory” was really popular at that time. That caused many management theories to be based on designing everything first (like designing a car), then implementing it (manufacturing).

The idea never worked for software development, as usually the problem is completely different, you won’t produce the same code 1000s of times, each software project is different and it’s meant to be done just once. Also, if you create a car which requires much maintenance, nobody will buy it any more and it will get a bad fame in the market. That wouldn’t happen with software, as every software is custom, and I saw many companies profiting from doing a bad job - the more maintenance software required, more your customer would depend on you.

Then it came Agile. A completely different mental model that was very welcome in the market where you give more value to the humans in your team than to the processes you follow, designing the whole software upfront (waterfall model) was replaced by interactive processes, and architects and managers which defined everything and that thought that the main job of a software engineer was to right code were slowly replaced in the market by managers which would abdicate control of the team in favour of delegating more to the team members.

The concept of architect was slowly replaced by the concept of team lead, with Y career progression paths to staff / principal software engineer, where the role wouldn’t be defining every piece of the software, but to motivate and lead the team by turning them into more independent team members, who could make decisions by themselves.

All those changes in software development world came together with a set of political and lifestyle changes world wide, where thanks to internet, people could collaborate more to each other online, more participative methods were created where people who could never participate on decisions now could have a voice, through social media, influencing skills and much easier access to information.

I remember being on a software team a little more than a decade ago where we were discussing About big data tools and comparing Cassandra and HBase, and the main arguments in the team defending the first was that it was fully distributed, while the second had leader election based in Zookeeper where there would still be a central point. Although we were talking about technical data tool, it was embedded in everyone’s mind at that time that “centralized is bad, distributed is good”. You didn’t have to argue much, just defending de-centralization was usually enough to convince people whatever was the subject, being software, politics, or whatever.

I must mention the crypto currency fever as well - the whole idea of crypto was based on the idea that centralization is bad and everything will be solved if we decentralize stuff. Bitcoin was based on the idea that all economic problems were due to governments and central banks controlling economy and the idea of smart contracts was to automatize the law so no central judge would be needed for agreements, that could be logically programmed.

This makes sense if you understand that moment - we were coming from a world with little or no decentralization, there was a cultural movement for decades saying “centralization is evil” and our minds became biased towards it.

Motto in the last decades:

All decentralization is good, all centralization is evil

That was the dominant mindset for the last decades all around the world.

But what happened later?

The bad side of decentralization

Right after the positive feeling towards anarchy dominated both the world and also work environments, decentralization started showing some of its flaws.

Relying completely on the “majority” to decide what was right or wrong seemed like a good idea in the beginning, it seems the world is more free is people on the bottom have more power to make decisions.

Reality, though, is showing us a different story, unfortunately. The majority can go wrong and the fact something is incredibly popular not necessarily is an indication of being right.

We all want to live in a better world for everyone, but the assumption that a world where people in the bottom make all the decisions seems to have been a false one. Here is a small summary I can give of the general consequences of this in the last decades:

  • UK decided for BREXIT by a referendum and now everyone regrets it
  • Millions of people died during COVID pandemic due to misinformation
  • Epidemic of fake news on internet
  • Far right governments taking power all over the world, in many different countries with no recent history of that
  • Historically reliable source of news losing popularity, slowly being replaced by social media and “conspiracy theory” influencers which were often put in the same level of reliability than specialists in many areas.

And these consequences are just the easier ones to see, but many smaller consequences have been happening in all levels of our society, including the workplace.

In Software Engineering world, I could testify countless teams where everyone would have the right of making a decision, regardless of experience, knowledge and specially results. Daring to make a decision on your own in the workplace became often interpreted as oppression, as trying to rule over others, as going back to the “architect model”, where some decide and think and some do.

Scrum teams and Agile in many companies became a synonym of “everything is decided democratically”. If you want a team to follow a single direction, you have to lead them into this direction, by convincing people your idea is good, otherwise they might just not follow.

There is a bright side on that - we definitely see less of that arrogant behaviour we use to see before agile, where people up in the hierarchy decide crap stuff and everyone else has to swallow, killing innovation and not allowing people to shine, to contribute with their own ideas. A DIVERSE environment is something great and I would argue it’s the only way to go, some problems are just to complex to be solved by a single mindset… If you hired a team to do some task and you want it done the best way, you have to give people the chance to make their own mistakes and also the chance of showing how good they are.

But unfortunately, in many workplace environments the full decentralization bring as consequences:

  • Lack of direction - a member of the team is trying to create an elephant while the other is trying to create a duck. Both a duck and an elephant solve the problem, so the members fail to convince each other of following a single direction and you end up with an animal which is part duck, part elephant, part any other animals other members imagined
  • Higher cost - even if you have a team member which knows which mistakes not to make, you have to let everyone learn that by failing themselves. While being allowed to try and to fail is important for your development, doing that for everything can be a problem.
  • Demotivation - surprisingly, an unexpected outcome of everyone deciding everything is that very often nothing is decided and team moves very slowly, without the feeling of progress. Also, some ideas have to be implemented by multiple people, so if you’re unable to convince people to follow your idea, that idea might end up being dead, even if it was a good idea in the end. Lack of results is often demotivating specially to people who actually love what they do. Also, if you’re not working to drive results, why are you working? It would probably be better to do something you like instead and work less time, but more effectively.
  • Non meritocratic environments - in the spirit of having a diverse environment where everyone has the same voice on decisions, you might not be acknowledging how better a specific team member is. And how do you define merit when you part from the assumption anyone has the same voice becomes non-obvious.

Again, I want to freeze how much freedom is something important to me and I recommend again taking a look at my previous article if you’re starting to think I am here to defend dictatorships. Really, it’s not. Please read this text with an open mind, as if you want people to be more free, it’s even more important to acknowledge the problems that come with decentralization - the first step to solve these problems is to accept the facts and resist the temptation to deny them.

Decentralization also brought us bad consequences. Fact.

Please take a moment to think on whether you agree or not with that - is it clear that this is a fact? A fact that many people I know will hate and to be honest one I am not super happy to state either, but it’s important to see it - reality is knocking at our door showing that some things went wrong, and trying to ignore this and just insisting in the mistake seems like a really bad idea.

But once you accept the fact, you might start asking: what was our mistake? And how to fix this?

What was our mistake

IMHO, the main mistake was to think that anarchy has only good sides and ignoring the trade offs that exist when you compare anarchy vs hierarchy. We can’t say we want to go back to what we had before, that is also bad or maybe even worse, but solving that problem might just be a little harder than we, as a society, initially thought.

Decentralization is not always the answer, at least not initially. Let’s take a moment to analyse these 2 forms of society organization, advantages and disadvantages of each one.

Also, we must have clarity on what is the problem we want to solve here. IMO, decentralization, ultimately, was trying to solve the problem of modern slavery. The reason why we didn’t like hierarchy in the first place was not how bad it was as a society organization method, the real reason is probably a mix of the following:

  • We don’t want to live in a world where some rule and some obey
  • We don’t want to live in a world where people in the top of the hierarchy can make crap decisions, damaging the lives of everyone, without having a mechanism to fix that effectively when it happens
  • We don’t want poor decision making by the boss, where the top of hierarchy makes decisions based on things he or she has absolutely no visibility about. If you’re more than 40 years old and lived that age, you probably lived the situation at work where you knew what to do but you had to convince your boss that you had to do that. Why??

Looking at the disadvantages of hierarchy, anarchy seemed like a good solution. But when we experimented it, we learned even more about the problem:

  • We don’t want poor decision making by the bottom. A news channel could be bad as information is in the control of few that can manipulate it, but sharing peer 2 peer news on whatsapp doesn’t make it any more reliable
  • We don’t want to live in a world where a group of people can’t follow the same direction because no decision is made
  • We want high productivity, driving results with minimum costs and motivating people to do their jobs
  • We want at the same time give people the chance of giving their opinions, but at the same time acknowledge individual talents, as meritocracy contributes to improve the results of the whole group of people and that matters!

So, now we see it, it’s possible to see clearly that anarchy was just the obvious attempt to move forward regarding the evolution we want - a more free society. It might not have been the solution, but it was a step forward.

Decentralization is not the goal, it’s a mean. The goal is a more free society.

Hierarchy vs Anarchy comparison table

I tried to create a table below with a minimum set of differences between these 2 organization forms. The table is definitely incomplete, but hopefully it transmits the mental model I want to show here - we should constantly be thinking on both as 2 solutions with trade-offs between them, instead of religiously defending one of them for any situation.

  Hierarchy Anarchy Why
Speed to change decision making process Slow Fast To change decision making process in a hierarchy you have to change people in the top and that takes time. In an anarchy, like the market, everyone is constantly trying to decide things differently to achieve the results
Risk of high impact mistakes (like misinformation) low high Hierarchy has better control, which can be good if in good hands, but horrible if the top of the hierarchy doesn’t bring good results. Anarchy has less control, but many heads can think better than just one.
Innovation hard easy Hierarchy is good to keep things controlled and working the way they are. Things can change, but they can take time and probably require people in the top to be replaced, some times you have to replace the whole org and start from scratch. Anarchy allows innovation easily, good if you need prototypes, but it’s harder to scale.

What’s better for your use case? Would a decentralized organization bring better results? Or would a centralized one? Can you imagine forms of combining both, applying each one in the proper situation?

How to get to a more free society

Given anarchy is not the solution we thought it was, what is then?

To answer that question, let’s refine a little our concept of freedom. What is a more free society? One could argue that being free is being able to make decisions independently. However, we are free to decide, but we are prisoners of the consequences - if you have the freedom to decide which design to use for a software you’re creating, and that design decision results in bugs, you might lose your freedom to work on any task you want, as you will probably have to fix the bugs you generated with your decision.

In that example - if you were on a team, what would you decide to be free on, if you had the choice between:

  1. To be free to not spend any time studying how to cook
  2. To be free to get a tasty meal in less than 30 min whenever you want

Of course it’s an hypothetical example, but hopefully it’s enough to show that some times there are conflicts on what to be free on. Freedom is rarely absolute - nature limits the amount of freedom we can have, even if nobody is imprisoning us. One could argue that a more free society is a society where people can make all the choices, in a decentralized way - but looking at the options above, even if you have the freedom to decide, you will lose freedom for either 1 or 2 regardless of what you choose, as nature imposes to us we can’t have both.

So, in a group, how to define freedom and how to define slavery?

I have a special view on what being free means and I hereby propose the following rough definitions for this:

  • Autonomous - You’re free when you don’t need orders to perform actions. Like in the case of cooking, mastery and skills make you more autonomous - the more knowledge you acquire and the more you can move alone without depending on others, the more free you are.

Mastery, skills, knowledge and discipline - lead to freedom.

  • Accountable - If you’re not ready to take the consequences of your own acts, you’re not free. If you rely on others for everything, that makes you dependable on them, even though it might look like the opposite some times. Never taking accountability on anything moves you towards parasite relationships, where you cannot leave without others doing things for you.

The best leaders are not afraid of assuming their own mistakes, they do mistakes and learn lessons from them

  • Have common goals - humans do not survive alone - we only reached the stage we are as humanity by helping each other. This is different from being dependent on each other, this regards being part of a group, that moves in the same direction, because the individuals want to get to a common place.

The more social and soft skills you have, the more free you are.

  • Organized - some people think anarchy is the same as chaos, which is the absence of organization - but it’s not, the name of this is anomie. Hierarchy is just another valid form of organization, so people are not necessarily non-free for organizing themselves using a hierarchy. Organization is essential to freedom, as without it you are slave of the waste of time and productivity disorganization implies.

Societies are only free when they can organize themselves to split and sync the work across individuals

This last point is very tricky, as hierarchy can be there to organize, but it can be misused to the opposite - and here we arrive at the main point of this article: what is the best way of achieving organization?

Anarchy can apply in some situations - whenever you have established mechanisms inside a team, or centralized decision making based on votes, if this leads to organized groups, then good. As we saw above, it’s not always the case, as anarchy requires a high level of autonomy and mastery from the individuals - the more free the individuals are, the more you can use anarchy as a form of organization.

But hierarchy is also valid to organize, as long as you ensure it won’t drift to just oppression. And how can you do that? How can you make sure hierarchy on a groups is kept under control and only serves to bring organization, not random oppression?

The answer is not immediately obvious, but IMO the solution is clarity of delegation.

How to use hierarchy the right way

The secret for hierarchy to work fine and understanding that management is just another job, like any other. The hierarchy is just there to bring organization to the group, not to make sure that every decision in a company with 1000s of employees have to be decided by the CEO.

Companies hire employees so they can make decisions better than the bosses. The boss job is just to bring organization to the chaos, as well as forming more leaders - good leaders will form more leaders and they will want to drive the team members to improve in the qualities that will make them more free - more accountable, more autonomous, working more towards common goals and in the most organized way possible.

And what about bad leaders? Bad leaders will fail to do that and what happens in most companies is they blame others for their failures, because they are in a position of privilege due to the hierarchy. That is the thing we must fight against - leaders must be accountable for what they do and if they are being bad leaders, there must be some mechanism (anarchy again, how odd!) to drive them out.

Good leaders create more leaders and turn people into more free.

Bad leaders blame others for their mistakes and get credit for things they didn’t do.

I believe the best mechanism I have found to the date for that is Management 3.0 delegation levels:

Management 3.0 delegation levels

In this mechanism, managers are in control of delegation and they can decide whether they will do something themselves, or whether they will delegate something. As things are not always black and white, they can some times delegate most of the decision but still be involved to veto if needed, or delegate but be involved to just keep informed about it, for instance.

The fact is - when these managers use delegation boards and publish them to their teams and to other parts of the organization, it’s much much harder to run away from accountability. Given a situation where something went wrong, you can just look at the delegation board and see whether the team member was to blame, or whether the manager failed on his/her role.

Good managers are accountable and even if you have hierarchical environments, if you have crystal clear delegation and accountable leaders, who admit their mistakes and who step down as leaders if they are failing, the system works fine.

Clear delegation and accountability makes hierarchy work well enough for a free society, decentralization is an option, but not a must have

Bringing this to practical terms - that’s what we should be moving towards to. Sure, some things can be decentralized, but instead of assuming everything will be fine with flat hierarchies and anarchy like environments, we should be demanding clear delegation in the work environment. That’s what you should be asking questions about in your next interview.

TLDR Summary

Pragmatically, here are the takeaways I hope you got with this reading:

  • Focus on results - making decisions as a team can be nice some times, but it’s not what you were hired for - do not let yourself to deviate from the team goals thinking the manager is forbidden to make any decisions - hierarchy is not the real evil.
  • Earn your freedom - become more autonomous and independent, be accountable for the responsibilities you pull or are assigned, and pay attention to where the whole group is moving to - you’re not an island. Do that for yourself, not because your manager told you to do it.
  • Be accountable - assume mistakes if you make them, be humble, the best leaders are not the most arrogant, the best leaders make mistakes and learn a lesson from them.
  • Lack of accountability is the real evil - demand clear delegation from your managers and when you’re the manager, provide clear delegation. Create delegation boards, use delegation levels mechanism. When you interview for joining a company, do not ask them whether they make decisions as a team - try to find out how accountable they are.