Devconf.cz This year, along with all the usual in-depth technical talks about Linux at Red Hat’s Devconf.cz developer conference, there were also several people there to promote AI-linked projects and the tech bros’ previous favorites – blockchain projects.
The promise of said projects is that they will change the world and relieve us of tedious burdens such as financial regulation, government supervision, and taxation and so on – leading to the undefined and empty (but surely immensely lucrative) web3.
Although this stuff gets everywhere now, its connections with open source Unix isn’t obvious, and it’s not directly related to FOSS. Of course, the algorithms commonly used to generate and update blockchains are mostly open source. So too are many of the tools that generate the gigantic statistical text-prediction models these marketing folks are absolutely desperate for you to believe are functioning artificial intelligence. (They’re not, and the proof is that they’ve moved the target by inventing the new term “AGI” – artificial general intelligence – for that.)
But while the tools may be FOSS, what they’re doing with them definitely is not – and it’s evident that the companies building large language models don’t own the data that’s being used to generate the models.
The presence of some of the more predatory companies in the space is an unfortunate but almost inevitable consequence of the market forces that have driven open source software to the multibillion-dollar industry it is today.
We thought it might be interesting to try to trace how we got here, look at where some folks would like to steer the software industry next, and drill into why that is. That in turn will take us to how it’s being done, and the question of whether there are any alternative directions for the software business.
The commoditization of software
Starting roughly in the 1970s, the software industry is undergoing a similar developmental pattern to that which other manufacturing industries went through after the Second World War. Expansion of markets leads to commoditization, reducing scarcity and driving down prices and perceived value. Market growth is accompanied by simplification, reduction of choice, and the elimination of less competitive suppliers.
Now very similar market forces to those that drove the evolution of FOSS are now driving AI in a sort of race to the bottom.
Are there realistic alternatives? Yes, there are, and they have worked before … but they will need changes in methods and attitudes. However, there’s an added benefit: The tech bros will hate it.
First came hardware commoditization
The necessary precondition for the commoditization of software was the earlier commoditization of hardware. Programmable digital electronic computers have existed for about 75 years now, but for the first third of that time they were rare and expensive. It wasn’t until the invention of microprocessors in the 1970s that they started to become cheap and plentiful.
Microprocessors started a pattern of hardware development that’s now being recapitulated. In the very early days, there were only a trickle of early microprocessors and a few companies selling the components to build your own computers from these parts.
Then in the 1980s there was an explosion of varieties: Hundreds of manufacturers, thousands of models, mostly incompatible with each other. Each successive model was faster, with more storage and better capabilities.
More and better computers led to more and better software. When programmers had to hand-toggle bits into the front panel, everyone kept everything as short as humanly possible. When professionals could afford a decent editor and a compiler, and floppy disks to keep stuff on, programs got more ambitious. When developers had GUIs and hard disks, software started to get fancy, and when the customers could safely be assumed to have those things too, applications started to compete on style as much as functionality: Easy and, importantly, attractive UIs; not just ease of use, but ease of customization and integration with other tools.
Then commoditization of commercial software
Sales, however, leads to competition, and competition leads to the elimination of weaker rivals. The 1990s saw an era of consolidation, when the majority of makers of both components and whole ranges of computers collapsed. Only the strongest survived: x86 compatible processors (and a few RISC chips, themselves consolidating down to Arm); multiple ports and interfaces disappeared, replaced by USB and SATA; and so on.
The same happened to software.
Early commercial programs – all different types, from applications to games to OSes and tools – were hand-written, often by lone individuals. As specs grew, and software grew to fill them, that led to ever larger teams of developers, and specialist roles, and whole management methods.
Dozens of weird and wonderful OSes and their apps, and the programming languages and complete toolchains they were built with, quietly went extinct. What was left were a few varieties of Unix, and Microsoft Windows.
Mass-market commercial software followed the same pattern. When once dozens of word processors and spreadsheets competed on their unique text-only UIs, features and performance and speed, standardized GUIs eliminated much of their distinctiveness. Few apps, and few app vendors, survived the transition from console apps to GUI apps. As apps got bigger and more complex, almost paradoxically, they got cheaper – partly as OSes provided ever more supporting components, partly as development tools provided more and more assistance to developers, and partly due to the ever-diminishing need for efficiency.
Just as hardware standardization on the x86 PC drove multiple hardware vendors out of business, so software standardization on Windows drove multiple software vendors out of business. OSes gradually incorporated what had been third-party tools, killing off those vendors too. Software vendors resorted to bundling, acquiring one another, combining separate products into suites sold for little more than the price of originally independent products.
Once there was just a small choice of very similar suites with very similar features, then other steps became necessary to maintain sales, such as vendor lock-in: Changing file formats, so even happy customers must pay to upgrade so that they can still read others’ documents.
Worse still, today ubiquitously online computers enable even tougher restrictions, such as subscription schemes where you must keep paying just to access your own existing files, even if you don’t receive any useful upgrades for your fees.
Commoditization of free software too
Some of the pressures driving down the prices of commercial end user software don’t apply so much to server software. When tools by their nature work by communicating over the network using standardized protocols, vendor lock-in becomes more difficult. Not impossible, but harder.
Software efficiency is at an all-time low, and bloat is worse than ever, but sales can be kept trickling in thanks to customers’ need for a constant stream of security patches.
We find ourselves put in mind of the great Douglas Adams:
“It is very easy to be blinded to the essential uselessness of them by the sense of achievement you get from getting them to work at all. In other words – and this is the rock solid principle on which the whole of the Corporation’s Galaxy-wide success is founded – their fundamental design flaws are completely hidden by their superficial design flaws.” – So Long, and Thanks for all the Fish
This only works for so long. Software can keep getting bigger and slower only for as long as computers keep getting faster, and the rate of improvement there has fallen off a cliff and shows no sign of recovery.
With FOSS, though, the pattern has been a little different because of the nature of the consumers. Open source OSes and server software have not out-competed each other on the strengths of their user friendliness, for instance.
Desktop FOSS OSes and the accompanying apps have been good enough for well over a decade. They’re quicker and cheaper than their commercial rivals, just as safe and just as powerful. But they’re not identical, and many people don’t want to relearn or take the time to switch. Much like one T-shirt is very much like any other T-shirt, some people are happy to pay over the odds for brands that they like, or that they want others to know they can afford.
Free software that works well enough but needs a bit more effort from the user appeals to a certain sort of person, and that’s often the sort of person who works with computers for a living.
FOSS hasn’t succeeded because it made hard stuff easy. It didn’t. It still isn’t easy. It succeeded because it made expensive stuff cheap.
(One proprietary computer vendor has successfully hung on to its market segment, because it worked out that it could use a lot of FOSS to build shiny friendly consumer kit with pretty apps, without the substantial cost of building the whole software stack from scratch on its own.)
On the server side, though, the pricing race to the bottom has further to run. There’s little customer loyalty because the sort of techie folks who work with this stuff enjoy learning it. With current open source licenses, if desperate developers or vendors attempt to impose pricing, users can simply fork the last all-free version and keep using it for nothing.
The result is that the price can fall all the way to zero.
Commoditization has extended all the way down to the network protocols. Now almost everything runs over not merely TCP/IP, the lowest common denominator of network protocols, but specifically over web protocols.
In a market of free tools on free OSes, this offers one last revenue stream: The lure of Software-as-a-Service, so seductive to many a mildly technophobic financial directory. No need to do any of that tedious mucking around with infrastructure, with expensive hardware and complicated software. That all becomes someone else’s problem, leaving you only with trivial implementation details such as where your data is, who else can get at it, and if there’s any way to get it back again.
As capacity in the big datacenters climbs to effective infinity, and costs of storage become tiny, and costs of software shrink to basically zero, executives must look elsewhere for ways to cut costs, or increase margins, or gain some other edge. That is what we will look at next. ®