‘It is great to have publications, but is a scientific contribution limited only to them?’

Konstantin Fursov on breakthroughs in science and measuring the effectiveness of scientists

Science is a sphere that’s quite hidden from society, for instance, due to its secrecy, plenty of special terminology or impossibility of briefly explaining the gist of discoveries. This is why the state has to introduce an institution of scientific managers who would understand scientists’ language and could control who of them is effective and who isn’t. Vice Director of the Institute for Statistical Studies and Economics of Knowledge at HSE Konstantin Fursov talked about it in an interview with Realnoe Vremya.

“Something important happens in science every week, but, first of all, this is very important for the scientific community”

Mr Fursov, could you give us a few prime examples of scientific breakthroughs that Russian scientists have made in the last years?

Here the question is what is considered as breakthrough areas. We all know about Russian Nobel prize laureates such as Zhores Alfyorov, Andrey Geim and Konstantin Novosyolov, or about Grigory Perelman, who refused his award (Fields Medal). Something important happens in science every week. But, first of all, this is very important for the scientific community. For instance, Russia as well as other countries actively researches new materials, looks for treatment for cancer or Alzheimer’s, but to a certain extent, this is a routine that gives some new clues and allows moving forward. A discovery becomes a scientific breakthrough when it is recognised in the scientific community.

For instance, the Russian president’s award is given to young scientists every year in Russia. If we carefully have a look at the list of laureates, it is always mentioned what for the award is given. And one can understand what achievements laymen can see. There are several examples that are close to me. In 2014, Aleksandra Kalashnikova received an award for developing superfast magnetic phenomena. Maksim Nikitin who develops smart nanomaterials for biomedical applications received it in 2017. Here, first of all, we are talking about targeted delivery of medication to certain organs. Magnetic components are introduced to the molecules of medications that are delivered to the part of the body that needs to be treated through inductance coil.

At last year’s presidential award ceremony that I had a chance go to, one of the laureates presented results of the development of methods of detecting explosive materials. It is already on the edge with military science. Interestingly, it was a real issue of secrecy. Should one tell the public about this discovery? Because every laureate has to make a public statement, say something about his discovery. It is hard to say what a particular discovery gives us today. One thing is when you discover a material with new characteristics or the same method of detection of explosive substances. And the other thing is when this discovery is used in a solution, starts to be applied in practice. It can take years for new knowledge to be applied at least because technologically the economy hasn’t reached the necessary level.

So there is a lot of discoveries, and they happen every day. The question is which of them reach our ears and in what form.

We all know about Russian Nobel prize laureates such as Zhores Alfyorov, Andrey Geim and Konstantin Novosyolov, or about Grigory Perelman, who refused his award (Fields Medal). Something important happens in science every week. But, first of all, this is very important for the scientific community

“Americans are very proud of their military science as well as citizen science and try to popularise it by all means”

Does it turn out that military discoveries are hidden from us in general?

Of course, because it is an issue of global competition and national security. The secrecy of science is quite an interesting phenomenon in general. It contradicts the idea of the ethos of science to a certain extent. If we consider science as a social institution that moves and develops and is regulated by certain social norms, according to Robert King Merton’s idea of the ethos of science (that’s to say, a set of values and norms that are considered compulsory for a scientist), the scientist must openly share his knowledge and achievements. It is called the principle of communism. But science doesn’t exist in a vacuum. Society and the state give it some autonomy in decision-making in exchange for an independent professional appraisal.

In this respect, it is reasonable to ask if secrecy in science is possible. Merton replies by placing the scientist himself who demonstrates ambivalence, that’s to say, duality and contradiction of his motives and behaviour at the centre. On the one hand, the norms of the scientific community regulate and support his activity. On the other hand, there is an aspiration for success, competition for the right to be the first. Moreover, as it has already been said, science exists in the structure of other institutions (authorities, economy, religion) dictating their norms one should respect. As a result, the scientist can simultaneously go by different norms.

Science remaining a social institution also can be fully immersed in the system of secrecy when it is possible to freely exchange opinions and findings inside a group, but this can’t go outside. It is such a paradox.

And isn’t the fact that modern scientists are encouraged to publish all their research in magazines from the Scopus list a violation of secrecy?

I have a colleague who researches the future and long-term projections. He has several articles in which he analyses the disciplinary structure of military science based on open sources and even considers development trends of some of its areas. In other words, it isn’t completely closed, of course. Some solutions, which are already used in technologies, are quite visible. Moreover, some areas of their further development are clear as well. If you watch an American TV channel, you will see that Americans are very proud of their military science as well as citizen science and try to popularise it by all means. The Japanese love to save the world in mass culture by demonstrating technologies of the future. We show our accomplishments in a bit different way, mainly on parades. It deeply depends on culture.

Obviously, you will never be shown what is happening at the forefront of science on TV or in the cinema. But you can understand some vector of development. This includes to, for instance, the same unmanned vehicles, automation and robotisation of military machinery or soldiers’ new uniform — all this is quite open and understandable trends. Another thing is that nobody will tell you how to obtain that material the super armour is made of.

If we consider science as a social institution that moves and develops and is regulated by certain social norms, according to Robert King Merton’s idea of the ethos of science (that’s to say, a set of values and norms that are considered compulsory for a scientist), the scientist must openly share his knowledge and achievements. It is called the principle of communism

“In Russia, this indicator is quite low and accounts for about a third of total costs on science, while two-third are public money. Everything is the other way round in most of the developed countries”

What are today’s parameters used to measure the effectiveness of the scientist and the organisation he works in?

This depends on what we consider the effectiveness and the scale of our actions. The case is that there is plenty of indicators that allow us to talk about the intensity of scientific development. And these yardsticks were born in smart people’s minds quite a long time ago. The first attempts at systematically measuring the total budget of science, for instance, date back to the first half of the 20th century. British physicist John Desmond Bernal became the founder of scientific statistics in some respect having evaluated its national budget in England.

Specialists gradually switched from measuring public costs to the category of internal costs on research and developments. The difference is that all money that was really spent by organisations on doing corresponding works regardless of their origin was taken into account. In other words, it is the money (one’s own money or money received from any client) that was really spent by the executor on salaries, equipment, trips, experiments, organisation of conferences or other things that are needed to do research. In English, it is often called performance.

New indicators appeared as accounting systems developed and became available. First of all, I mean the indicator of publication activities, which is very popular today, that’s to say, the number of scientific publications. This and related indicators were registered in a system approximately in the 60s of the 20th century when specialised scientific literature databases appeared. Firstly, they appeared as libraries so that it could be easier for scientists to look for necessary literature. Later, when a considerable pool of works accumulated, there was an interest in systematising them by authors, fields of science, demand. Hence the next yardstick — the number of citations. In other words, you were cited (no matter positively or negatively), it means your work is interesting, it got feedback in the system of scientific production. This system of measures remained neutral for quite a long while. It was rather a way of finding information about what was going on in different fields of science and a way of verifying specific hypotheses about its social order than an aspiration to evaluate somebody.

However, with the improvement of measuring tools and reduction in financial resources, a question arose: how effectively did some groups work? How taxpayers’ money was spent? As a result, more complex models appeared when costs stopped being considered as an indicator of the result but started to be considered as characteristic of its component of resources. But what can be meant as an indicator of result? It is great to have publications, but is a scientific contribution limited only to them? There are technologies, but it is very hard to measure them. For instance, it is possible to calculate sets of design documentation because any technology has to be documented in a certain way before being used in production to be able to hand it over and put into operation. It was calculated, but somebody created one big and complex technology and collected a box of documents, while somebody created small solutions and collected tens of thin folders. Both organisations spent the same amount of money. Which of them was more effective?

Let’s say, we calculated everything that could be calculated, and then the next question arises: how do scientific knowledge and technical solutions get to the economy? There are some indirect signs of such a transition. For instance, it is possible to evaluate the share of a business’s money in costs on research and developments. Roughly speaking, how much a business is ready to pay for science. It is quite a good indicator of quality that suggests potential demand for results of scientific activity in the economy.

By the way, in our country, this indicator is quite low and accounts for about a third of total costs on science, while two-third are public money. Everything is the other way round in most of the developed countries. And it is a question for not only science but also the market where scientific accomplishments must transform into new products.

For instance, it is possible to evaluate the share of a business’s money in costs on research and developments. Roughly speaking, how much a business is ready to pay for science. It is quite a good indicator of quality that suggests potential demand for results of scientific activity in the economy

Can a scientist’s number of patents be an indicator of the effectiveness of a scientific organisation?

Yes, there are indicators of patenting activity. For instance, the number of patent applications for inventions, useful models, industrial patterns. Of course, mainly inventions are patented because it is the most valuable result. Why patent? Because it is a title certifying the right to freely use the result of intellectual activity on a certain territory for some time. And this result is protected on the territory of one country or several countries, it means that it has commercial potential there and can influence the market structure. Patent protection costs money, it means that there is sense in patenting a development in case it is really valuable. The patent intensity depends on the economy of a country and culture. For instance, every nut is patented in Japan, more complex and system-forming solutions are patented in Germany or the USA. It is mainly determined also by how the invention can be included in the production system later.

The fact that the country’s biggest competition and management training programme Leaders of Russia has had a separate track this year dedicated to science at least suggests that a lot of attention is paid to this issue

It is important to remember that any evaluation must have a goal. The so-called Leiden Manifesto was published recently in Nature, in 2015. Several authors, including founders of the scientometric system of evaluation of research presented 10 principles that one should go by when using qualitative indicators of scientific development. It became a reply to the use of many indicators point-blank, without a due understanding of goals of measurements and the essence of measured indicators.

During one of the projects, one of my colleagues imagined how these principles were applied in the national system evaluating the effectiveness of science in Russia. Let’s say that they aren’t applied to the full. It is quite hard to say why it happens. Probably we are so keen on accounting instead of answering long-term questions.

In one of your interviews, you said about the manager’s art who has to pick out a scientific talent and invite it to the university. How developed is this system of scientific scouts in Russia and around the world?

Such a system is developed quite well in the world, and it works through a big number of different competitions: to occupy the post of candidates, doctors, postdocs. The competition for talented scientists is quite high, but, moreover, the best of the best are looked for. Considering quite a notable problem of lack of scientists, we obviously detected this problem a long time ago and have been actively solving it. The fact that the country’s biggest competition and management training programme Leaders of Russia has had a separate track this year dedicated to science at least suggests that a lot of attention is paid to this issue.

By Matvey Antropov