Sergey Markov: ''Superintelligence seems to be about to appear. Actually, this road is by far thornier''

About automation and reduction in staff

Sergey, we all see that automation and concurrent processes inevitably repeat. For instance, Sberbank has just recently announced a big reduction in staff. Will there be space for the human in this system in the future? How to amuse him when job opportunities are given to robots?

I have a big lecture on this topic called E-Sheep Ate People. If you're interested, you can then watch it on YouTube. The case here is the following. We have already had waves of technological unemployment. In the past, the appearance of new technologies brought to such problems many times. And probably the most famous story is about how sheep ''ate'' people. With the appearance of looms and the textile industry in England, it turned out that it's more profitable to breed sheep, not to give peasants land to cultivate. Of course, a big number of people lost their job.

Of course, there is a serious problem of technological unemployment. But if we look at it in the long term, we'll see the unemployment level now isn't much higher than that of in the 19 ^th, 20^th centuries. The problem is solved in the long term because, for instance, new professions appear, old ones transform, working day reduces. For instance, an accountant. If a hundred years ago it was a person who manually did calculations with the help of some primitive mechanical devices at best, and sometimes simply on paper and in mind. Today we use calculators, computers to calculate, but accountants haven't disappeared. In this respect, applied artificial intelligence systems we're creating today are just very complicated tools that are able to rather enhance the human's possibilities that allow him not to do the routine, a primitive job by delegating it to machines.

As for modern-day trends, undoubtedly, there are professions that are under serious pressure of technological unemployment. For instance, the same call centre operators. It's clear we can substitute a person for the machine in generic communication, it's 4-7 times cheaper to use, and nowadays it doesn't lose to it. People don't do this job anymore, but there is a huge number of tasks they can do. For instance, tasks of processing and marking data files for the same machine learning systems. Talking about specifically our subdivision, people who used to be online now dedicate more and more time to other tasks – listening to robots' talks, marking speech files, files for speech recognition, its synthesis and so on. Secondly, working time is appearing now where it used to be in deficit. For instance, you could automate standard cases and can dedicate more time to non-standard situations. That's to say, in general, this can improve the quality of service. If earlier call centre operators didn't go into details of a client's problems – just because there was a huge load on them, very strict rules – now they can dedicate more time to it. It's possible to reduce a client's waiting time.

The situation is absolutely unsatisfactory, and everyone who faces medicine knows about it. So it's the sphere that could accommodate a huge number of people if needed

If we look at the problem more globally, society has plenty of spheres that lack people very much. Let's say we've been working on optimising the health system in Russia in the last years, which, let's be honest, means a reduction in the availability of medical care – doctors are imposed strict rules regarding service and so on. The situation is absolutely unsatisfactory, and everyone who faces medicine knows about it. So it's the sphere that could accommodate a huge number of people if needed, moreover, even relatively unqualified like the junior medical staff. Or, for instance, the task of caring for children or old people. In some developed countries, for instance, in France, the state provides some amount of hours of babysitting to every woman who gave birth to a child. Why not introduce such services and not help young mothers to have free time? In other words, we actually find a huge number of spheres of application of human workforce. In addition, there are many spheres where machines are unlikely to drive people out in the short run for different reasons.

We see the problem becomes not technological but social, political and economic. Here economy and politics are closely connected. How are social resources distributed? Nowadays certain companies make a surplus profit by introducing high technologies. A part of the profit, undoubtedly, should be distributed to retrain people to enter new professions, to expand, for example, the social sphere. Maybe simply to reduce the working day, introduce a scheme similar to the universal basic income, increase benefits and so on. That's to say, a part of the surplus profit made by big technology companies should be also used to control the social consequences of these changes. But this question, of course, should be given not to people from technologies like me. This is a question for political elites anyway.

You said there are spheres where robots won't be able to substitute a person in the short run. Which ones, for example?

Babysitting.

Won't a robot be able to perform this function?

Firstly, at the moment we are far from creating Androids that will accomplish tasks as precisely and effectively as people. Secondly, it's simply a psychological barrier. This means there will probably be a huge number of people who just won't want a robot to take care of their child.

At a moment, technologies will exceed the complexity of the human body's organisation, our biology, and the difference between machines and people will start to fade away. The mobile phone has already become today a kind of part of the human body

It seems to me to be a matter of time.

Generally speaking, there is such a thing as conspicuous consumption. The food industry has progressive technologies that allow to produce quite a big quantity of foodstuffs with low costs. But there are people who say: ''We just don't want to consume industrial products, we want them to be grown by a farmer in his garden, we like it more.'' Actually, it's a kind of minimum balance in the sector. The organic food industry settled in on the threshold of several per cent of the volume of the food market, and this share neither grows nor reduces significantly, though technologies make further progress. Anyway, there are people who want to show their status by consuming something handmade. People will probably say the same thing: ''I don't want to have my hair cut by a robot hairdresser, I want an organic hairdresser and I am ready to overpay for it. Actually, this prejudice of people isn't so weak and it doesn't disappear so fast. We live in the era of technological progress, but somebody still spits over his shoulder, touches wood and so on.

I think in a longer term the problem will be solved in a different way. At a moment, technologies will exceed the complexity of the human body's organisation, our biology, and the difference between machines and people will start to fade away. Today mobile phone has already become a kind of part of the human body. If the smartphone is taken away, we will see something like amputation syndrome. All our species, if we look at it, do nothing but constantly changes itself, interferes in its own biology in the most dramatic way. Fire was invented many millions of years ago, and people started to cook on it. As a result, people with weak jaws began to survive. As we stopped needing strong jaws, there was room for the brain, and it probably became one of the factors of development of intellect.

In a word, we genetically modified ourselves through technology. Another thing is that this modification wasn't that direct – we just changed the environment we live in, and our biology changed under its influence. As we got tools in our hands enabling to interfere in our organism, it didn't take much time to start using them. You take an aspirin in and interfere in your organism's biochemical ways and dramatically change it. Clothes that substitute the skin and allow to exist in a cold climate, the tools we use and that are a continuation of our body. Due to the primitiveness of technologies we use, at the moment we have a clearly distinctive barrier. We understand our arm starts here, the hammer is here, the ear ends here, the smartphone starts here. But not the imperfection of technologies is the problem. Roughly speaking, if it's possible to install a smartphone inside the head without pain and side effects, you can get any picture in front of your eyes at any time. This is why the problem will likely be solved in a way that some time later we won't have separately cars and people, everything will unite into some post-human essence. But this, of course, is a matter of quite a remote future.

Perhaps, it's not a remote future considering the speed of development of technologies?

Yes, technological progress has been very fast in the last hundred years. But, by the way, there are different opinions here. For instance, Neal Stephenson and Peter Thiel consider that the technological progress slowed down significantly, that there were more innovations in the 60s than today. Then people went to space, but people have been still milling around on the border of the Solar system and no more. Though in the 60s it seemed that we were about to go the stars. Any technology provokes optimistic expectations. When a technological revolution takes place, everyone wants to continue the curve further to the sky in mind. In fact, there are certain limits we face when technologies develop, but they aren't felt at the dawn of this development. I think the same thing will happen to artificial intelligence technologies because there are certain fundamental physical restrictions linked with the calculation speed, there are certain barriers it won't be possible to overcome in general. Now it seems to us a superintelligence is about to appear. Actually, this road can be by far thornier and longer. Nevertheless, I think what I am talking about will happen sooner or later.

''Neal Stephenson and Peter Thiel consider that the technological progress slowed down significantly, that there were more innovations in the 60s than today. Then people went to space, but people have been still milling around on the border of the Solar system and no more. Though in the 60s it seemed that we were about to the stars.'' Photo: mil.ru

About role of government and its substitution by artificial intelligence

It's clear everyone talks about progress. Can there be on the contrary, regression?

As people say, you can't be sure the progress became so steady. We know the same Hellenistic culture demonstrates impressive paces of technological progress once, then the Middle Ages came when regression in several technologies took place. Certain social mechanisms that will bring us to the new Middle Ages might work. For this reason, those people who support technological progress are responsible for the fight for the support of progressive tendencies in society. Further development depends on how public resources are distributed, how much money is given to science, education, enlightening. This money can be easily spent on something different – palaces, yachts, casino. In this respect, a lot turns on what tendencies will win, what will be more important – progress or bling. We'll see. Sometimes there is no contradiction, new bling can be made thanks to progress. Probably it's what has always saved it.

We can just keep our fingers crossed. But I anyway think regression is unlikely because most people realised the value of the progress. But who knows, fashion can change, and tomorrow everyone will say that it's bad to read books, progress is bad, let's live in caves.

As for resource management, do you think artificial intelligence might substitute a government in the future? Perhaps, it's too futuristic, but does this idea have any prospects?

In my opinion, there is nothing futuristic here. It became possible to do many things a long time ago. Just technically, we can hold direct voting of all citizens of the country or even the world on some important issues. We have the Internet, steady cryptographic algorithms to protect the integrity of this voting. But all this isn't done, and not for technical reasons. It's clear that powerful elites aren't interested in solving some problems in a way that's unprofitable for them. On the other hand, such an argument as people aren't ready for democracy, that we actually can't hold voting on some issues because the people will vote wrongly because it doesn't know something and so on is often used.

In my opinion, it's a bit crafty position. Why don't many people understand and don't know how the world works and which decisions are correct and which not? Because there is unequal access to education. People get bad upbringing, bad education and they can't make decisions society needs. Why does it happen so that society spends huge money on defence? The US military budget is over $600bn. Every year this money is spent, in fact, to protect developed countries from the angry majority of the third world. Why not spend them on education, enlightening to transform this third world into blossoming regions where people rightly making responsible decisions will live in several decades?

Here there is a question: how smarter are people than these bacteria? Each of us seems to be smarter individually. But altogether we are complete idiots, and it's well seen in such problems as global warming

Actually, the question is very uneasy because society, unfortunately, often goes by plans scheduled for a very short period of time. People who own a business today, who rule in politics don't make plans longer than 4, 5, 10 years. It's important for large corporations to get a high income in the next several years. And nobody cares that it might bring to a global ecological catastrophe in a decade. Quite much money is needed to provide normal conditions in the third world, to bring up a generation of sensible, smart, educated people. But in 20-30 years, these investments will turn into a huge effect for the whole humankind. But nobody cares about it because either someone's donkey like that of Hodja Nasreddin or padishah will die in 20-30 years.

This is why a very big number of problems we face come from the absence of a long-term strategy. Humankind, in fact, isn't the first species in the history of the Earth that make a global contribution to the whole biosphere. Bacterial mats that lived in the first atmosphere of the Earth joyfully ate whole methane, produced a big amount of oxygen and joyfully died in this oxygen. We exist with you thanks to their global win, which turned into a failure. But here there is a question: how smarter are people than these bacteria? Each of us seems to be smarter individually. But altogether we are complete idiots, and it's well seen in such problems as global warming.

Or waste.

Yes, a very characteristic example. Perhaps, we will also shower ourselves with waste, CO2 emissions and die like these bacteria because they didn't have any plan and development strategy either, they just jealously grabbed and cared about nothing. This is why we have a very serious challenge. The stronger we become, the higher the danger is.

At least the problem of ecology started to be raised, discussed.

It's not enough to simply talk. It's necessary to make decisions that seriously affect all world economy. As soon as such initiatives are taken, they find a big number of opponents. And business easily blocks them through a system of lobbyists, through the media. Even such brave Band-Aid solutions as the Kyoto Protocol ended with nothing, in general.

''I think the reality will be much less entertaining because robotised military technologies are actively used and tested now. Remotely controlled drones, in general, became a common practice at war of developed countries against underdeveloped.'' Photo@ fpi.gov.ru

About management of the masses and robot militaries

You've mentioned the topic of democracy. But the same USA, the presidential election, leaked data of millions of Facebook users. What about management of the masses with the help of new technologies?

It's no secret that masses can be manipulated. Why is it possible? Because there are those who own the media, and there are those who don't. For this reason, it turns out that we have those who can manipulate, things to manipulate and people to manipulate. But even if the same media didn't belong to tens of magnates but were managed democratically, it would turn out that the tool that can potentially be used for manipulation isn't used for it any more, in fact, as those few people who gain from manipulation control it.

The same thing happens here – modern machine learning technologies also can potentially augment the power of few people over many people. Some advanced journalists, big-mouthed orators were taken to manipulate the masses yesterday, and they persuaded the public into choosing the decision that was unprofitable for them. Today we can make this car stronger using algorithms accomplishing this task more effectively than people. We can analyse people's behaviour in social networks. Again, both the state and corporations have got a bigger amount of information about every person. Of course, this gives greater opportunities. Potentially, there is a problem, of course. And it's not completely clear what to do with it. Here the thing can bring to a catastrophic scenario. And now it's our ability to enable technologies to use everything, not few things, for the better mainly determines the future of all our species, I won't be afraid to use such bombastic words.

Your opinion about militaries is interesting. Can robots substitute people in the army?Can military actions reach a new level?

Stanislav Lem has a novel Peace on Earth where robots are sent to the Moon. But I think the reality will be much less entertaining because robotised military technologies are actively used and tested now. Remotely controlled drones, in general, became a common practice at war of developed countries against underdeveloped. These tendencies will likely to grow, and again, tomorrow developed countries will get an effective means to subdue those who are poorer and who don't like the existing situation very much to their will. I don't think it will bring to a reduction in the aggression level, it's rather outsourcing of violence when stratification in society becomes quite strong, and representatives of elites already can hire somebody to beat the protestors with batons instead of doing it themselves. Here machines can also play a role of such employees.

Actually, developed countries are very vulnerable in terms of human losses. The United States lost the war in Vietnam mainly because coffins started to come to the homeland, and this caused a powerful protest and, as a result, made this military operation to stop. And let's imagine such an operation today – but not citizens of developed countries but robots will fight now. The public will say: ''So what? We killed some indigenous people, ours didn't die. It means OK, fine, we can keep fighting.'' Unfortunately, we tend to have such scenarios, not those where we will hold robot tournaments and decide according to their results who won and who lost.

To be continued