Yesterday I applied for a job at Bird, since it seemed like a nice Dutch company working on API specs and schema translation. Then I read a bit more about their founder's views on efficiency and quickly retracted my application. The "Business in a Box" they plan to build is exactly the kind of autonomous artificial proprietor, seeking profit without moral in an automated way, that I was already scared of a decade ago. They were in the process of moving their offices out of the Netherlands anyway because it's too regulated for them.
In society's reaction to the rise of AI, a lot of focus goes to safety research (AI alignment) and avoiding bias in human selection procedures. While those are of course very important topics in themselves, I think we should view AI as part of a larger phenomenon (call it simply "technology") that makes processes more efficient.
From an engineering perspective, efficiency is often a goal in itself. And the natural barriers that the pursuit of efficiency has met have often been enough to balance it out, for instance in the expansion of the railway system, that has led to more well-being at a reasonable environmental cost.
But AI is now hyperboosting the efficiency of engineering itself, leading to a self-accelerating explosion of efficiency in many areas, not all of which are harmless.
Obviously, like nuclear weapons before, the rise of AI weapons has brought with it a potential for larger catastrophes. Even if there is also evidence that the overall per capita risk of dying in conflict has decreased over the centuries, when weapons suddenly get a lot more powerful, it's obviously scary.
AI also increases the efficiency, and therefore the power, of industry and consumerism as a political force in the world. And at this point, there is a melting together of the power of capital, the power of persuasion (political lobbying as well as populist media and clickbait), and the power of software. Money plus software builds new software, earning more money and gaining more political power, without humans in the middle to stop and think whether all this efficiency is actually nice.
I use the term "nice" to mean a lot of things here: not harming anyone's social position or access to resources, not harming life and climate on the planet, preserving equality, staying in touch with nature, etcetera.
A lot of efficient processes, such as the payments industry for instance, are amoral. It's nice that we can buy cheap toys but the efficiency of it makes it to easy to forget about the working conditions in the factories where they were manufactured. This is the core problem of capitalism, and it's what I've been wanting to help fix by working on degrowth and collaborative finance. Maybe we can invent an alternative economy without legal persons bullying the physical persons.
In the meantime, what we as engineers contribute to the world is still essentially just efficiency. The World Wide Web is a good example of a system that I think is both efficient and nice at the same time. But Web 2.0 corrupted its original vision by building silos on top of it. My work on Unhosted and other projects has been aimed at making the web nicer again. Efficient but nice.
Niceness often stands in the way of efficiency, for instance when industry has to comply with regulation. But efficiency also often stands in the way of niceness, and with the current acceleration of AI, I'm afraid that efficiency is going to win, and humanity will lose. I'm afraid that AI-managed companies will own more and more land, obtain more and more access to electrical power, and more and more power to influence politicians, to the point where they will be unstoppable. In fact, they already are unstoppable, but we can stand our ground. Let's use regulation to fence their rage, and build open source alternatives to make them partially replaceable.
So instead of focusing only on the safety of AI and the alignment of AI, let's focus on efficiency as the broader concept of which AI forms a part, and on niceness as a more long-term goal than just safety or alignment, and make sure AI-powered efficient companies operate inside a state of law that we humans control, not the other way around.