Internationally acclaimed expert Tomáš Mikolov coming from Facebook AI to join CIIRC CTU and focus on developing strong artificial intelligence.

The Czech Institute of Informatics, Robotics and Cybernetics, part of the Czech Technical University in Prague (CIIRC CTU), is boosting its research team. Tomáš Mikolov, an internationally respected authority in artificial intelligence research, who also helped improve Google Translate, will join CIIRC in April as the head of a research group. His arrival at CIIRC CTU has been made possible thanks to the new RICAIP centre project, through which CIIRC CTU, together with its partners in the Czech Republic and Germany, plans to strengthen its role in artificial intelligence and robotics research for advanced industrial production in Europe.

 Tomáš Mikolov is primarily known to the scientific community and the general public for his dramatic improvements to the function of language recognition and processing applications, such as Google’s machine translator. He did this by creating new neural network models that considerably surpassed previous approaches to language modelling. The improvements that followed in natural language processing were the greatest seen in recent decades. He was named Neuron Award laureate for major scientific discovery in artificial intelligence in computer science at the start of 2019.

After successfully completing a doctoral programme at Brno University of Technology eight years ago, he left the Czech Republic to work at Google Brain and, later, Microsoft Research. He comes to CIIRC CTU from Facebook AI Research, where he had been developing intelligent algorithms to achieve the most natural machine/human communication since 2014.

Mikolov is returning to basic research at CIIRC CTU, where he will focus, inter alia, on complex systems that may lead to further dynamic improvements in artificial intelligence. As the leader of a new research group at the RICAIP centre, he will build his own team to seek to develop a system to gradually develop strong artificial intelligence. Just as complex forms of life on Earth developed through natural evolution, complex artificial intelligence should also be created through gradual evolution. At the same time, he sees basic research as a fundamental prerequisite for innovation.

“A combination of factors contributed to my decision to continue my work at CIIRC: CIIRC CTU impressed me as one of the most progressive Czech research institutes, and there are also a number of interesting scientists here who, like me, worked in leading research abroad for several years. It’s important for me to do things that are meaningful to me, and with people I enjoy working with,” Tomáš Mikolov said about working at CIIRC CTU

“We are very happy that we managed to fill one of the three newly created research positions with such a leading figure in the field,” said prof. Vladimír Mařík, Scientific Director at CIIRC CTU and chief coordinator of the RICAIP project. “Mr. Mikolov will bring further much-needed innovative approaches to CIIRC. Our long-term goals at both CIIRC and RICAIP are to support structural changes to the academic environment, combat stereotypes, and be able to best respond to the needs of industry and society. I believe we can succeed thanks to leading talents like Tomáš Mikolov.”

We took this opportunity to ask Tomáš Mikolov what made him decide in favour of CIIRC CTU and what areas of research he will focus on going forward.

 What led you to the idea of returning to the Czech Republic?

I had been abroad with the odd break since 2010. I was already working full time in California in 2012, followed by New York and then a year in Paris. That meant I had been away throughout the past decade. Yet I felt I wanted to come back to the Czech Republic. I had formed the impression I had tried everything I needed to try and seen everything there was to see in my field. I had managed to work with a number of scientists who are the most cited experts in their fields. For example, I had the opportunity at various times to work with scientists who have won the Turing Award, something like the Nobel Prize in IT (note: the ACM A.M. Turing Award is annually awarded to individuals for contributions of lasting and major technical importance to the computer field by the Association of Computing Machinery). So remaining abroad was no longer the main motivation for me. There are smart people everywhere in the world, including in the Czech Republic. A scientist’s success is often dependent on a combination of different factors, including chance – some can be influenced, others cannot. Anyway, great research can be done more or less anywhere, especially basic research. You mainly need inspiration and good ideas for basic research in our field.

But why come back to the Czech Republic?

The Czech Republic is generally greatly underestimated, but when you look around the world, you realise that a lot of things work well here. We don’t have earthquakes or major natural disasters. People often only realise it is a good place to live after spending a few years abroad and coming back. I honestly don’t understand questions like why would you move to Prague from New York. New York is a nice city but, for example, public transport is much better in Prague. What’s more, people are less aggressive here. In terms of tourist numbers, it is probably a draw. When I started comparing the two, I realised I could do the same research here. I was never interested in going to a prestigious university like MIT or Stanford. It would probably look better on my CV, but I was never that bothered. I have never sought to be a rich and famous scientist. Things just turned out like that on their own (laughs).

What was decisive about the work at CIIRC CTU?

It wasn’t a clear decision at the start – it was more a combination of factors. I also considered other workplaces. However, CIIRC CTU impressed me as one of the most progressive and attractive in terms of location and the new building. What’s more, I found the ecosystem of people already working at CIIRC interesting. There are a number of people here who, like me, have worked abroad for several years in leading research and, again like me, have returned to the Czech Republic. They have similar experience and quite possibly similar views on research or education. The three most prominent scientists I’ve been communicating with since the beginning are Jan Šedivý, who initially invited me here, Josef Urban and Josef Šivic. It’s important for me to do things that are meaningful to me, and with people I enjoy working with.

You mentioned you will continue to do basic research. What do you see as its main purpose?

The concept of basic research is often difficult to grasp. Scientists work on ideas that often appear abstract. Outsiders feel it’s something that has no concrete results. It can take a long time for an idea to be put into practice. Yet in the end, it might all happen faster than people can imagine: a hundred years ago who would have thought people would land on the Moon a few decades later? Basic research is a bit like roulette – you don’t know where the ball will drop. Sometimes it’s chance – you try twenty or thirty ideas, it turns out that four work, but only one of them is ultimately put into practice. The probability of making a fundamental discovery is quite low, yet its effect and impact can be enormous. This is what drives us forward – we are inventing completely new things, new machines, medicines. That’s what basic research is about – we have to test things and move forward. Innovation wouldn’t be possible without basic research. That’s something often forgotten. We usually only see the cherry on the cake when the result has already been put into practice. Yet when we look at successful technology companies today, whether Google, Facebook, Microsoft or Apple, they are often based around discoveries made by scientists some 20, 30 or more years ago. These companies got rich on the internet, but neither Google nor Facebook invented any internet. In order to have new things and for the system to work, we need the support of basic research and subsequently to support putting the results of basic research into practice. No further support is required if the result is commercially successful.

How do you see current research into artificial intelligence?

In the past, research into artificial intelligence was primarily scientists trying to copy some part of the human decision-making process. This can be well demonstrated, for example, using chess. They said to themselves that for a computer to play chess like a human, it must be at least as clever and intelligent as a human to understand such a complex game. It’s a relatively complex intellectual activity. Gradually, however, it was realised that it doesn’t actually have to be such a complex intellectual activity. A computer can go through millions of combinations of moves, test them out and judge which combinations will give the best result. It can thus play chess well without having to understand the game at all.

In the past, artificial intelligence was about scientists looking for shortcuts to achieve a particular result. We define one concrete, narrowly specified task. If a computer can decently solve it, it may be possible to come up with techniques that can be generalised and applied to other tasks. That was the idea, but one that could not be entirely accomplished. Personally, I don’t think it will happen in the foreseeable future either.

Couldn’t the research into neural networks you’ve already been doing contribute?

Although neural networks can be used for a number of tasks, the principle always remains supervised learning. This means you always have to tell the computer what to do. If you show it a million examples of what a cat and dog look like, the computer will be able to distinguish a cat from a dog perhaps 99.9% of the time. It appears to be working. Yet if you then show it a kitten or a puppy, where the pictures look a little different, you find you have to start again from the very beginning. You need to teach it again. That’s inefficient. Transfer between tasks is negligible compared to human capabilities. It’s the same with chess and the game Go. If you change one rule in the game or place the pieces differently, the algorithms completely fall apart and it doesn’t work. You have to start building the solution from scratch again.

So, how are complex systems different?

Complex systems could lead to general or strong artificial intelligence, as the scientific community calls it. Algorithms should have the potential for independent intelligence, like humans. Yet no one currently knows how to do that, no one knows how to build such a thing. Even world-famous scientists like Geoffrey Hinton, Yoshua Bengio and Yann Lecun, with whom I have spoken about it, have no idea how to create truly strong artificial intelligence. When I was at Google Brain or Facebook AI Research, I visited dozens of other research groups. I don’t know anyone who knows how to move forward today.

This is why we have to try new ideas. A lot of people are doing applied research, where they use what already works. For example, they can teach that classifier that can recognise cats and dogs to recognise something else. They basically take another lot of data from another domain. Yet that’s still just a different application of something already invented, perhaps ten or twenty years ago.

So is there another solution?

Complex systems are systems with lots of simple elements that can interact with each other. Intelligence in complex systems is not something we define explicitly. We don’t put it in the system directly. On the contrary, we try to create systems where a kind of gradual artificial evolution creates the complexity – intelligence. It’s similar to natural evolution on Earth. Here, too, the complexity of life gradually increased over millions of years. Millions or billions of years ago, life would simply have been unable to solve the tasks that we can and must solve now as humans.

Is there something to build on in the field of complex systems or is it a completely new field?

Ideas about complex adaptive systems are nothing new. The scientists who built the first computers were already thinking about how to make smart computers as early as in the 1950s. Evolution was one of the main interesting ideas. The situation now is similar to that of neural networks. Mathematical models of neurons and the brain can be traced back to the 1940s. A significant number of scientists were already working on neural networks in the 1980s, when there was a period of about ten years that artificial neural networks were relatively popular among scientists. However, they fell out of favour in the 1990s, as it became clear that simpler models could handle similar tasks just as well as neural networks. Yet no one thought they could work better. I saw the potential of neural networks – I believed they could be used in more areas. That we could use the same model to solve language, translating, speech recognition and image recognition. The idea of ​​neural networks was definitely not new – the question was rather how to get it across the finish line, how to get it going, to make it work – and it was only my generation that succeeded.

Does that mean you now want to also get complex systems off the ground?

You could say that. We can compare it to the idea of building a plane. People had already had the idea 500 years ago. Yet it took a few more centuries to learn how to make it work – how to get a plane off the ground. My colleagues and I managed to get neural networks off the ground about ten years ago.

I now think we could also succeed with complex systems. Again, this is not a new idea that no one’s had before. Yet complex systems are not used where they could actually work today. They are used in areas that don’t make sense. They are often replaced by simpler models.

I think there are currently no good ideas about how to get complex systems off the proverbial ground. It may be something of a lottery because no one knows what will and won’t work and how long such research will take. Yet it’s very creative activity.

Is anyone else helping you in your efforts?

I’m already building a small team of students and postgraduate students at CIIRC. I have a student from France and one from Charles University in my team. I think our group will have around five people in a year or two. My goal is to have several people around me with similar scientific interests and who are not afraid to try to invent something new, basically from scratch.