One of the topics that comes up often with AI and real world automation is the concept of “human in the loop” – not just as an oversight layer (or canary in the coal mine) but as an integral part of these systems.
At one level the human becomes the eyes and ears of the machine, a sensor layer for anomalies. This is the same function that IoT performs in real-time (at machine speed). At another level of understanding, human inputs serve to graduate response patterns, to calibrate a system of data points.
But there is another layer beyond the conceptual here. Humans are still the only entities that carry consequences. The machine can turn left on a stop light and fail to predict a traffic cop’s hand signal. That is just a correctible parsing error with fatal consequences.
AI systems can optimize, predict, synthesize and adapt. They can also make a “bad” call by perfectly addressing the wrong context without “personal” consequences. In some settings that is why the AI is preferred, no social fallout, no loss, no carrying of legitimacy. Societies are governed by legitimized responsibilities. “It is human to err, to err at scale is AI”…they will say.
There are constructs outside the tech bubble, just as there are constructs outside the veil of science. Systems can model behavior but they do not inhabit meaning. A system can model a zero-trust society based on tracking card transactions, and develop an incentive structure formulated on that basis. Banks have been doing that for centuries, but they still need the leverage, the collateral, the bond.
If you completely remove humans from the loop, what happens is systems become optimization proxies. But decisions sit within a set of parameters; reality. Once you let a disengaged intellect run your world, you’re pretty much asking for the pursuit of flawed goals. Humans rightly distrust concentrated opaque power. They know that legitimacy requires negotiability. They know that systems remain contestable. Skepticism is an adaptive trait within a competitive system.
Fully autonomous centralized systems are very hackable. That is why they hide them underground and surround them with multiple layers of “security.” Every time there is a war, the first thing to go boom is that previously impenetrable obstacle.
These bluff historical artefacts are a magnet for gaming, deception and adversarial behavior. They nicely illuminate the law of unintended consequences. Humans are still important within the domain because they are “native” to the environment. The issue is not whether machines can think for themselves, it is who gets blamed when the thing’s think goes wrong.
There are standard toolkits to renegotiate a fresh future:
- reversibility
- honest brokerage
- modular systems – decentralization
- preserving optionality
- a kill switch
