Pathways will handle thousands or millions of tasks, understand multiple types of data, and do so with remarkable efficiency. It will move us forward from the era of single-purpose models to one in which more general-purpose intelligent systems reflect a deeper understanding of our world and adapt to new needs.
What is Google Pathways?
Pathways is a new way of thinking about AI that addresses many of the weaknesses of existing systems and synthesizes their strengths.
Google Pathways is multi-tasking
Jaff Dean says in his article:Today’s AI models are typically trained to do only one thing. Pathways will enable us to train a single model to do thousands or millions of things.
Today’s machine learning systems are often trained from scratch for each new problem. This means that each model is trained to solve only one problem, and each time you introduce a model, you start from scratch. It is as if every time we learn a new skill, we forget the ones we knew before. The result is thousands of single models for single tasks that take more time and more data to learn.
The idea with Pathways is to train a model that can handle multiple tasks together and learn new tasks faster because it draws on prior knowledge. So the idea is to create a model that combines already known capabilities and puts them together to perform further more complex tasks.
Google Pathways is multi-modal
Jeff Dean says in his article:Today’s models mostly focus on one sense. Pathways will enable multiple senses.
The world is generally perceived through different senses. Current AI systems process only one mode of information at a time: they can take in text, images, or speech but not all three at once.
The idea of Google Pathways is to enable multi-modal models that include vision, hearing, and speech understanding simultaneously. The result would be models that are more insightful and less prone to error and bias.
Google Pathways is more efficient
In the same article, Jeff Dean says:Today’s models are dense and inefficient. Pathways will make them sparse and efficient.
The third problem is that most of today’s models are “dense,” meaning that the entire neural network is activated to accomplish a task, regardless of whether it is very simple or very complicated.
The idea is to make Google Pathways work like the human brain, that is, to build a single model that is activated “sparsely,” meaning that only small paths through the network are called into action when needed.
The model learns which parts of the network to activate in which situations and for which tasks. In this way, not only is the model able to learn multiple tasks together, but it can do so faster and more efficiently because it does not have to activate the entire network for each task.
To find out more about Google Pathways, check out our Web Stories here.