Success, failure, company's bureaucracies and other frustrations.
Something is broken in Tech...
I was reading the other day a book written by Rachel, from https://rachelbythebay.com/w/, called “The Bozo Loop”. I could talk or explain why I consider a must read blog if you are in the tech business (more even if you have a truly technical profile as programmer, sysadmin, devops…), but I want to actually talk about some of the chapters of the book in which she talks about the decay of a well functioning company like Google (or others). Decay in the sense of becoming a bureaucratic company more than a technological company, which actually means the company stops being a great place for technology oriented people where you can play, design, discover… Well, let’s go one step at a time.
I would say that almost everything I read by Rachel is interesting. And she is the kind of engineer that, when explaining something, either a story or some technical detail or idea, makes you want to become better engineer. But in particular, when I was reading some chapters about her experiences in Google, Facebook and other big tech companies, it felt too familiar. I realised that I had been living those same things in other companies and the frustration I felt was totally justified. Maybe it is because I think I am right, or I have some kind of pretentiousness just because I can punch keys and make some software works, but every time I was in a corporation where non technical people were calling the shots, having “ideas” and treating developers as if they didn’t know anything, I got upset. I mean, a company that crafts software to sell or serve their customers, should put the decision power about technical things in hands of the technical people. Those who know better. And I think this is the key point of the problem.
Tech people (devs, admins, etc) tend to be not very good with people. This is a bit cliche, but true to some degree. Tech people tend to forget that other people not so much interested in computing may consider difficult a lot of things that for us are just like drinking water: something simple you have to do every day. And considering this, I agree that for a business it is not always a good idea to have these people managing customers, deals and all. But, and this is a big but, that doesn’t mean that business people have to make decisions that affects the tech stack without discussing and agreeing with the techs. Why, may you ask? Because you can kill a good software product with a bad idea, followed by a bad schedule and wrong expectations.
One trend I have been noticing during the last few years is managers, bosses, new-agile-flavour master (I’m not going to enter that rabbit hole today)… in general those positions surrounding the development team, start considering themselves strongly knowledgeable in tech just because they can tell you that this system is written in Scala, with AWS containers for data storage and running in Airflow as a DAG scheduler. Essentially they believe that because they can enter the web UI of Airflow and see the boxes representing jobs, and being able to repeat what is in some document or something a sw engineer mentioned in a meeting, makes them good enough for making decisions regarding that stack. So they decide that Airflow is no good and we have to move to a new stack they found, that is the latest product you have to use, or you’re a loser.
The problem then appears when the new amazing product/system/language is something too new, with some rough edges that, surprise surprise, are what we need. So you get bitten in the ass. And they will be stressed and all, but the ones cracking the code, breaking their heads to find a workaround and having to put code below their own standards in Prod, are the developers. Things start going downhill from there, with more mistakes, hacks and problems appearing in the way, more decisions made by the same people who started it that leads to even worse issues. And you end up with engineers feeling frustrated and burnt, because they realised nobody will actually listen. And because promised were made, deals were sealed and ads were thrown to customers about how the new shiny technology would improve the product to levels never imagined,the company can not stop the plans, that were a mistake since the start. So they have to keep plowing, doing something they believe is a wrong solution or a bad approach or both. It is really difficult and depressing to have to do some work that you yourself think is BS.
That means you turned your engineers in code monkeys. And they know it.
Is there a worst way to waste talent than that? I don’t think so. But the fact is that the companies that went this way, which is almost any one above 300 employees in my experience, keep making money for a long way before they’re merged or acquired. And regarding the big ones, they keep functioning. And one wonders how that’s possible?
Well, I think I have some non-excluding theories about that. First one is from Rachel herself: due to the time that happens between a design plan is in motion to the moment it ends up in production, there are months at least. So the software that was designed by engineers is still the running horse. It can also happen that the changes are introduced in steps, so the engineers have time to break their brains fixing the problems as they come. This is a horrible situation, and I’ll explain: they are working in a design or technology that they deem wrong or inadequate for the problem at hand, and they think it should go down in flames. But because they have to deal with it, some patches and workarounds are put in place so their future Me will not suffer so much. The worst part is that, for the business or management point of view, it looks like it was a great idea because everything works and customers are happy and praising the new solution. But the tech is the one fighting the fires and making it “work”.
Then we have the situation in which a company can keep working as the software it produces is getting rotten but in incremental steps, so it takes time for the company to go down. Customers get used to some tool or solution and it is hard to move to a new one, so it takes long time. If the company is lucky, before it start failing hard, it is acquired by some big tech corporation that start pouring human bodies there. No matter what happens with that software, once the original company is acquired, it is considered that those decisions were totally successful. So the cycle will repeat in the future, because the people who made those decisions now have the aura of success.
And the last possibility is the company is a behemoth with strong purchasing power. So once their software or solution start showing the decadence, customers start looking for options and small startups or companies produce alternative solutions that are better… well, the big corp buys them, kills the decayed product and substitutes it with the new one. But the cycle starts again with that product.
It doesn’t help that promoting a technical person to such decision-making positions is difficult because that same people dislike the job itself, as it is more “political” than technical. So it is difficult to cover such positions with someone with a strong technical background. Hell, it is even difficult to cover the senior positions for developers, so no surprise here.
It sounds really bad and anyone might think that we are set for failure. Well, it is not exactly like that. Most big companies have some specialized teams that are doing the research and coming with new ideas and solutions. Those teams usually work on the key parts of the systems and I think they have more freedom to make decisions. I am not saying that no one that is not coding and just making decisions doesn’t know the craft. But the general landscape is like that. My point is that bureaucracy can kill a good project or even a company. And bureaucracy comes with a lot of redundant people with fancy titles calling the shots, even if they don’t really get the implications of such decisions. Usually they would listen to the developers, but the reality of such bureaucracy is that they have the hands tied. Someone else made a decision based on trendy fashions in software (like the obsesion to make everything a microservice, even if the monolith works great for the problem at hand) or based on some agreement with another company that forces everyone to use a specific solution or technology no matter what.
It is a problem, not a rant. But it is a big problem that we have no solution in sight yet. Actually, it doesn’t seem like the industry is actually trying to fix it or even recognize it as such. Extremely high demand for developers and engineers due to the increasingly high demand for software solutions and not enough supply of people with knowledge, capability or the interest to get the knowledge. I want to be some day like Rachel, having a similar level of knowledge like she has. And more people would have to have that aspiration if we wanted to solve the problem. But people have a life and not everyone in the industry wants (or need) that. Or have that passion.
In the meantime, tell your friends!