What Is Transfer Learning?

Transfer learning repurposes existing AI algorithms for new tasks

By Robert Earl Wells III
Published on June 29, 2023

In machine learning, transfer learning occurs when an existing algorithm takes on a different (but similar) job. Here's what transfer learning is all about, its benefits, and its applications.

Transfer Learning Definition

In transfer learning, developers reuse an algorithm designed for a specific purpose for a different task. The new algorithm applies what it already knows to perform the new work.

For example, if you had an algorithm that could identify pictures of dogs, you could easily tweak it so that it can also identify cats. You could further build upon the algorithm to create one that can identify any animal.

Artificial intelligence (AI) programs rely on different machine learning algorithms to get better and faster at performing their intended task. Transfer learning is not really a type of machine learning but rather a method used within the field. Transfer learning also has applications outside of machine learning.

Machine learning represented by brain

 

imaginima / istock / Getty Images

Applications of Transfer Learning

One of the most exciting prospects of transfer learning is deep learning. As part of neural networks, transfer learning can be used to train AI how to solve new problems even when very little data is available.

Real-world situations can't always be broken down into data points, so strong AI must be able to make inferences based on past experiences. Transfer learning has made the possibility of artificial general intelligence (AGI), or AI that can perform any task a human can do, closer to reality.

Benefits of Transfer Learning

Transfer learning is more appealing than creating a new algorithm from scratch; it can take a lot of time, data, and computing power to train a neural network. Aside from saving time, you can improve results by building upon a pre-trained model.

Thanks to transfer learning, AI programs can be trained to perform tasks they wouldn't otherwise be able to do. If there isn't enough data to train a neural network for the desired job, you can train it to perform a similar task for which there is an abundance of data. Then, you can build upon that model to successfully train your network for the new task using the limited data you have.

Types of Transfer Learning Models

There are dozens of pre-trained algorithms that are publically available for AI engineers to build upon. You can use only part of the existing model or the whole thing. Alternatively, you can build your own algorithms and repurpose them.

Image classification, object recognition, and computer vision are popular applications of transfer learning. Transfer learning models used for image recognition include Google Inception and Microsoft ResNet. These models are open-source and available for anyone to use.

Another promising application of transfer learning is natural language processing, specifically translating text to speech (and vice versa). Google’s Word2vec and Stanford’s GloVe are two publically available models that can be adapted for deep-learning language projects.

Caffe Model Zoo is a community-run website with a repository of pre-trained models for transfer learning projects as well as tutorials on how to use them.

FAQ
  • What is transfer learning in a CNN?

    Transfer learning in a convoluted neural network (CNN) is the same process as in other areas of machine learning. Existing algorithms move over to apply their "knowledge" to new work. Although a CNN may have a different structure from other machine-learning systems, this process works the same.

  • When should you use transfer learning?

    You should use transfer learning at a variety of points in a machine-learning project. One is when you're confident your current model has "learned" everything it can and would like to expand its functionality. Another good use is if the scope or purpose of your project changes; transfer learning can help you pivot without having to start over completely.

Was this page helpful?