We need to think about second-order consequences.
A capacitive touchscreen is able to detect a finger by picking up changes in the electric field; this technology was one of the elements that made the iPhone possible. The widespread use of smartphones, together with GPS and mobile connectivity, led to the development of services such as Uber. A later innovation in electronics eventually caused a change in the way people move around.
It is difficult to predict those developments since basic technologies interact with markets, behavioral patterns, and institutions, and their most significant effects usually appear several stages later.
The capacity of artificial intelligence leads to a variety of other changes. Research, invention, decision-making, and coordination are all part of nearly every human activity. A substantial expansion of these abilities could affect the speed at which all the other changes take place.
What happens if the cognition available to us increases a thousandfold? A millionfold? What changes if everyone has access?
I want to explore those questions through one example, the company; one difficulty, defining what improvement means; and one limit, delegating deliberation about our own lives.
The shifting scarcity
By cognition, I mean the capacity to investigate, interpret, reason, plan, and solve problems.
Until now, that capacity has been constrained by our training, time, and attention. We have organized much of society around those constraints, training specialists and building institutions that bring together knowledge no individual could fully command.
As access to those abilities becomes cheaper, other constraints carry more weight: the quality of information, the goals we pursue, the authority to act, and our ability to verify the consequences.
In software, we will be able to research markets, analyze behavior, and build solutions at a previously impossible scale. We will also be able to execute extraordinarily well on something nobody needs. Delivery requires testing evidence, resolving contradictions, and verifying results against explicit success criteria.
Evaluating evidence, setting aims, and exercising judgment are all cognitive activities themselves. There needs to be an explanation as to why the idea that human decision-making will always be a scarce resource would remain outside the scope of that expansion.
Every technological advance will require a reexamination of which decisions are delegated and of the criteria which make such delegation acceptable.
Companies learning to work differently
Take a company as an example. It is a useful unit of collaboration for thinking about this change: it brings together people, knowledge, and resources, distributes authority, and coordinates action around a set of goals.
Its ways of working reflect its members' limitations. Each person can process a certain amount of information and oversee a limited range of decisions. Roles, meetings, reporting, and approval processes help organize that effort. They also shape our assumptions about what work requires.
Since cognition becomes more accessible, these assumptions should be examined once again. A person who is used to carrying out a task can then make several attempts, compare the results of these attempts, and investigate why there is a difference between them. A team could keep a shared context, make the dependencies clear, and constantly check on their progress. This would give people the opportunity to take responsibility for a broader outcome.
The need for such changes means that one's way of thinking must change; it is important to learn how to formulate problems, make knowledge easy for others to understand, assign tasks with clearly stated boundaries, and evaluate work that one has not done personally. In this situation, experience is vital since specialists in a field can include exceptions, historical context, and functional limitations in how the system operates.
I expect the same teams to become capable of pursuing substantially more ambitious goals. Projects repeatedly postponed for lack of time or specialist capacity become realistic candidates for action. The opportunity is large enough to change what a company believes it can attempt.
The harder adjustment may be organizational. A company can give everyone access to AI and still require every idea to travel through the same meetings, handoffs, and approval queues. Plenty of businesses will call that transformation. Their employees will recognize the same workweek in another tool to manage.
Leaders should provide opportunities for experimentation, for the development of judgment, and for the exercise of greater agency. This requires rethinking the distribution of decision-making power and the way responsibility is assigned. The degree of benefit will depend on whether individuals have the necessary context, trust, and authority to use their improved capabilities.
The advantage compounds. A team that researches, builds, and tests in shorter cycles learns faster. Each project expands its knowledge and execution capabilities.
Wide access to intelligence will still go hand in hand with inequalities in the areas of data, capital, distribution, and authority, and these disparities will determine who ends up benefiting. The same can be said of laboratories, universities, and public institutions as they adjust to the new collaborative settings.
What improvement means
A greater capacity to act makes an earlier question more urgent: what counts as an improvement?
For a narrow task, we can agree on a criterion: reducing energy use while maintaining output and quality. A life or a society involves multiple goals and values that can pull in different directions.
For instance, an education system could boost its test scores while at the same time reducing people's curiosity, or a company might increase its profitability by sacrificing operational resilience. In each case, important questions arise as to what has improved, for whom, for how long, and at what cost.
Chosing a metric means expressing a certain idea of what is desirable, just as the choice to omit some consequences from consideration does. Even if people are given the same facts, they may place different values on freedom, security, or growth.
Desires also change as a result of experience. The technologies which shape our attention can affect our preferences; and if these technologies then use those preferences as a measure of their success, they end up fostering the very desires by means of which they are later assessed.
To evaluate such technology, we need to analyze the habits, relationships, and way of life that it promotes, and it is important to keep the ability to critically examine its fundamental aims.
Artificial intelligence will play a role in this evaluative process; it could point out inconsistencies and help in setting goals which are not yet fully defined. Nevertheless, doubts will remain about who decides the criteria, who has the power to challenge them, and who will be held responsible for the consequences.
The more capable we become in carrying out tasks, the more important it becomes how we define improvement.
When everyone can think a thousand times more
Let us examine this hypothesis in its most extreme form; a increase in intelligence by a factor of a thousand or a million acts as a thought experiment, since intelligence is multidimensional and does not have a single scale upon which such a multiplication can be based.
Imagine that we have universal access to systems which are capable of investigating, planning, and solving problems over a scope that is much broader than is currently the case. At first, such systems would be put to use on well-known goals like curing diseases, improving housing, and reducing waste. Later on, completely new projects—those which are at present beyond our capacity to conceive—would appear, turning previously impossible ideas into practical experiments.
Physical limits, scarce resources, and uncertainty would remain. In a negotiation or political dispute, the other participants would have expanded capabilities. That could make agreements easier and conflicts more intense. The rules governing how we live together would matter as much as each participant’s power.
There is also a more personal difficulty. Suppose there is a system which knows our situation and is able to predict the effects of our decisions more accurately than we can; even if sticking to its advice appears reasonable, we may eventually agree to decisions without having any understanding of the reason for them.
What, then, does it mean to exercise autonomy in regard to our own lives? When we approve of decisions, how much independence remains if the logic behind them is no longer accessible to us?
Some activities derive part of their value from doing them: learning, creating, caring, exploring. Delegating them changes the experience we seek, even when measurable results improve. We also discover what we want through our own attempts. The room we preserve for that experience will be part of our definition of a better life.
Where we are heading
The reason for my optimism about technology is that a great deal of suffering and potential that goes unrealized arises from our inability to understand and take action; research into disease or the coordination of solutions usually requires resources that are not available. When these barriers are reduced, a wider range of problems will be within reach.
I expect there to be a substantial rise in productivity, in our knowledge, and in our ability to take action. This increase will place a great deal of pressure on the institutions and power structures which control how it is applied.
We could be stepping into an age when an ever-greater share of our imaginings will come true. As a result, the question of which ones should exist will become all the more important. The ability to define, put into action, and allow challenges to these decisions will then be a major factor in shaping the future.
