Part 1 As we have said before, the software industry has a decades-long history of cost-cutting, commoditization, and a successful sales model of "pile 'em high, sell 'em cheap." This has worrying consequences if your skill set is the next one to be commoditized. But there may be ways out of this narrowing commercial bottleneck.

The stages of commoditization

Commoditization leads to consolidation - fewer choices and less differentiation in design. All that's left is branding and pricing. That can be simplified to "cheap and interchangeable" versus "fancy and premium-branded." A couple of months ago, we described how commoditization had simplified both the hardware and software markets.

In hardware, a huge variety of general-purpose computing kit has been simplified down to essentially just two alternatives. One is a profusion of x86 PCs, still largely interchangeable because of hardware compatibility standards. The other is a smaller array of pocket and battery-powered devices, which mostly use RISC processors in the Arm family - themselves a descendant of Acorn hardware. Acorn Computers always prioritized the less price-sensitive educational market over the home and games markets. This proved to be a winning tactic in the medium term. Acorn was still going over a decade after Sinclair Research was sold to Amstrad, and after Commodore and Atari battled one another to mutual destruction.

In software, there are roughly three mainstream choices. Whereas in the 16-bit era there were dozens of OSes, now we have Windows and a few flavors of Unix (and Windows is increasingly Unix-like). There's one main paid-for commercial OS left: Microsoft's Windows NT. There is one successful family of partly proprietary Unixes: Apple's macOS, iOS and iPadOS (alongside the company's non-general-purpose ones, such as watchOS and tvOS, based on the same code). And, of course, there is the large, sprawling, chaotic community of really quite similar FOSS Unix-like OSes: Linux in a thousand forms, from Android and ChromeOS to legions of server and desktop distros... and the BSD family, which remains significant partly because of licensing issues, which seem trivial to folks from the proprietary world but are very important in FOSS - and cultural issues, which in FOSS can be even more influential.

There is loads more diversity in the worlds of embedded hardware, RTOSes, and hobby OSes, but they have little presence in general-purpose computing. Much of this is industrial stuff, and the customers often don't get to choose. (Even those strongholds are under threat from real-time Linux, now that kit able to usefully run it can be cheap enough to be disposable.)

A key point to take away from this is that niche markets can be safer. It's better to be a big fish in a small pond than in the ocean.

What will get commoditized into oblivion next?

The process of commoditization is inexorable, and even now is moving on to important new phases. What comes next is at a different scale, making it less apparent - but just as important.

First is network-scale commoditization. The market for whole interconnected computer systems, the infrastructure that holds together businesses and governments.

Here, OSes matter much less, and hardware is barely significant. Once an organization buys into one set of vendors' products, it's very likely to stay with them - and things marketed as compatible with them - than switch. Not from laziness. Switching is hard, and expensive, and risky. Worse still, with software increasingly commoditized, you don't stand to gain significant benefits from switching.

Sometimes, the perceived differences are cultural rather than technical or financial. To technical-level folks, sometimes the reasons for picking a standard and staying with it are not obvious.

The big example here is people. As ever, Sir Terry Pratchett nailed it:

The personnel an organization needs to run its tech are just as much a cost as the kit and the programs on it. Hiring staff, training them (if they're lucky, or requiring certain certifications if they aren't), and - ideally - retaining them, or failing that, replacing them. This is a less obvious difference between the proprietary and open source job markets. We looked for ways to be diplomatic about this, but maybe simply being rudely direct is more efficient.

Windows is still mostly point-and-click. Windows is an axiom, a given. It's the baseline of much business and governmental computing. That is why ransomware pays, even when the crooks are less competent than the executive decision-makers. This is despite Microsoft pushing hard for over a decade to insert scriptable layers in between the GUI front-ends and the server back-ends. This is the main reason PowerShell exists. A sysadmin can point-and-click their way through server provisioning, take the resultant script, tweak it slightly, and use it to deploy countless cloud instances in Azure.

But the cultural background of the people matters a lot here. Windows is also very strong in home computing and gaming, which means predominantly non-techie folks. This means millions of moderately tech-literate folks grew up with Windows and have basic familiarity with it. So if they drift into tech, they are quite likely to stay with what they knew first: Windows.

We are absolutely not knocking this. The Reg FOSS desk's degree was in biology, and this vulture drifted sideways into tech support and sysadmin work because it paid much better than counting fossil spores and pollen for oil companies, or counting blood cells for healthcare companies.

These staffing factors historically made Windows more attractive for corporates and large organizations. The hardware is interchangeable, and the software predetermined. It wasn't chosen but inherited, the purchasing decisions made by long-gone management decades ago. What matters now is that the people who install it and maintain it are cheap.

Lumping all the more mainstream FOSS OSes together as Unix, it appeals more to programmers and scientist types, rather than gamers and leisure users. In real life, Windows is not more expensive. As far as non-specialists are concerned, Windows comes free with your computer, and 100 percent of games and work stuff run on it. Price doesn't matter, kids have been happily pirating software for more than 40 years now.

Why swap one free thing for a different free thing that can't run all the games?

The upshot of this is that for those who are interested and motivated, FOSS Unix has a lot to offer and tempt them, while for the many that drift into computing accidentally, Windows is the path of least resistance.

A lot of minor tech problems are trivial. That's why "have you tried turning it off and back on again?" is a joke. Junior, largely untrained and unskilled technical staff can fix the majority of tech problems, without having to understand them - especially if they're provided with a knowledgebase.

The point is that low-level junior techies are cheap and they are fungible. If some level 1 or level 2 support operators quit - or they burn out, or change to a more rewarding career - it doesn't matter to their employer because they can be easily replaced with other modestly skilled Windows techies who are just as cheap.

Which makes them very different from Unix techies, who used to hand-build their systems and the interconnections, meaning that nobody else could run them.

For the Unix vendors, this massively increases the appeal of automatic deployment and management tools, and portability aids such as containers, Infrastructure-as-Code, cloud servers, and so on.

If what your expensive Unix techies do is simplifying deployment down to some easy-to-read config files for software management tools, replacing thousands of lines of scripts, then it's less of a disaster if those techies quit - or price themselves out of their jobs. You can replace them with other techies familiar with the same automation tools.

In summary, Unix staff tend to be more knowledgeable and more skilled, meaning scarcer and more expensive. Unix techies tend to cost more money than Windows techies.

Cheaper staff makes Windows-based IT infrastructure cheaper than Unix-based infrastructure, even when the Unix-based stuff is free of charge or cheap.

But automation and cloud-scale deployment adjusts that balance, so watch out. Software that makes your job easier makes you easier to replace.

So, step 1: Commoditize the hardware.

Step 2: Commoditize the software.

Step 3: Commoditize the people.

Step 4: Profit!

Windows started to get built-in large-scale management tooling from Windows 2000 Server onward. Over the next decade or so, it became centrally managed and administered, and the support jobs got deskilled, outsourced, and offshored.

Around the same time, Linux matured and started to rival - or outdo - what the Windows software market offered.

In the next decade, thanks to AWS and Docker, to Puppet, Chef, Ansible, Salt, and so on, that homogenization started happening to Unix techies.

If you're not already alarmed by step 3, step 4 will be worse.

Large-scale FOSS Unix deployment is increasingly automated. When the customers don't have to pay and the instances don't have to be licensed, you can deploy at scale, then purge back down again at will. The creation and destruction of these uncountable host machines is managed by software - that was the key technological innovation that made AWS rich. Once it is all automated by software deployment management tools, the next step is to automate the automated deployment.

This is why Red Hat acquired Ansible and later started laying off middleware folks while investing in "open source AI," meaning large language models.

A couple of years ago it was so-called AIOps. If LLM bots can do an acceptable job of deploying and managing software-defined infrastructure, then they will drive down the price of skilled FOSS practitioners.

Remember the era of "my job went to India and all I got was this lousy T-shirt"? Multiple companies are working hard to bring the processes of commoditization and rationalization in order to eliminate junior jobs. That will be followed by the more highly paid specialist sectors, and then to the developers building this stuff.

FOSS enables this in a way that licensed code doesn't. If your infrastructure is built from free components, it's massively redundant, and offers easy rollback to known working configs, then minimally viable LLM-powered deployment tools may prove to be "just barely good enough" - and a lot of skilled people will face the chop.

Of course, this destroys the career path of gradually building your skills up from rookie to seasoned pro, but senior management doesn't care about that. It cares about cutting costs and increasing profits now, and it's not concerned whether there'll be anyone to hire in a decade.

The bottom line

In a way, this is all the result of mass-scale deployment of the cheapest, simplest available tech: of Unix, the original fungible OS, and C, the lowest-common-denominator programming language. There used to be alternatives, but they cost more. Now code is so plentiful, we are drowning in it.

First the hardware got commoditized and driven down to cheap and fungible - a process aided and abetted first by virtualization, making it easy to take complete, complex server software deployments, and move them onto other hardware. Then by reducing the need for hardware at all, by offloading it all into "the cloud."

Then the price of that software was driven down to effectively zero, when cloud-scale deployments of FOSS could do the job of carefully constructed and maintained proprietary software server farms.

Now comes the next stage, when most of the relatively expensive, skilled people needed to build and run those clouds of FOSS boxes can be replaced by a tiny crew aided by LLM bots.

"AI" can't think - but it's coming for your jobs anyway. ®

https://www.theregister.com//2024/09/16/the_future_of_software_part_one/