Machine Learning News Hubb
Advertisement Banner
  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us
  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us
Machine Learning News Hubb
No Result
View All Result
Home Machine Learning

Introduction to Transfer Learning with NLP | by Ravidu Perera | Sep, 2022

admin by admin
September 10, 2022
in Machine Learning


A brief look at Transfer Learning for NLP tasks

A model created for one task is used as the basis for another using the machine learning technique known as transfer learning.

Pre-trained models are frequently utilized as the foundation for Deep learning tasks in computer vision and natural language processing because they save time and money compared to developing neural network models from scratch and perform vastly better on related tasks. On the other hand, transfer learning is the use of skills acquired while completing one activity to assist in the solution of another related challenge. To make machine learning as human-like as possible, machine learning specialists are working to create algorithms that facilitate transfer learning processes.

Algorithms for machine learning are often created to handle discrete tasks. Methods to transfer information from one or more of these source tasks to enhance learning in a related target task are developed through transfer learning. This exchange of learning techniques aims to advance machine learning and make it as effective as human learning.

Transfer learning uses three different methods to resolve deep learning issues.

1. More straightforward training requirements using a pre-trained dataset

2. Using small memory requirement

3. Reduce the model training target

Image from Top bots

In Natural Language Processing, Transfer learning can be divided into three categories. The order in which the tasks are learned, the nature of the source and target domains, and whether the source and target settings deal with the same task are all factors.

Transfer learning in NLP has limitations when working with several languages and cultural requirements. For instance, most models are trained for the English language, so using the same model in a different language will be challenging due to differences in grammatical construction. It can also be challenging for NER tasks that require extracting unique, non-general entities.

Transfer learning refers to a process or task where knowledge obtained from unlabelled data can be applied to similar tasks with a small, annotated dataset. And with earlier data, that little labelled dataset reaches excellent accuracy. Compared to ML and DL approaches, NLP transformers have achieved good accuracy in every application.

The main principle of TL is to gather data from related domains to aid machine learning-based systems in obtaining more accuracy in the targeted domain. Transfer learning can also attain excellent performance with less human supervision than active and supervised learning.

In Transfer learning, there are two most popular architectures, BERT and Elmo. The BERT model can turn words into numbers. This step is essential because machine learning models require numerical inputs — not text — and you can only train these models on text-based data by using an algorithm that turns text into numbers.

Transfer learning can use as a mechanism to account for the distinction between upstream and downstream activities in NLP.

The lack of training data is one of the main problems facing NLP. Most task-specific datasets only contain a few thousand or a few hundred thousand human-labelled training samples because NLP is a diverse area with numerous separate jobs. On the other hand, modern transfer learning-based NLP models benefit from far more significant quantities of data. They improve when trained on millions or even billions of annotated training instances. Researchers have created several methods for training general-purpose language representation models using the vast volume of unannotated content on the web to close this data gap. When used for small-data NLP tasks like sentiment analysis and question answering, the pre-trained model can be adjusted, yielding significant accuracy gains over training on these datasets from scratch. Therefore this BERT architecture gives excellent help to NLP tasks.

There are numerous real-world instances that we can relate to transfer learning. For example, if a system has already been taught to identify apples, it can be used to identify pears with some minor adjustments. Less training time and data will be required for this. Attention is the central concept behind transformers.

The suggested methodology has addressed specific issues. However, it has a drawback in that it might not be able to learn well because it lacks information about the label. Furthermore, the labelling’s learning direction may change if the original labelling is declared erroneously. The methodology can be improved in future investigations by creating a label embedding initialization technique that gets over the drawbacks. Additionally, although the gap between the features needed for pre-training and fine-tuning was filled in this study, additional solutions could be suggested.

Image from machine learning mastery

As you can see, the model with transfer learning performs better and reaches saturation more quickly than the other model.

Like any technique, transfer learning has drawbacks and restrictions of its own. The issue of negative transfer is one of transfer learning’s main drawbacks.

Transfer learning can have several significant drawbacks, such as social prejudices, distributional biases, potentially revealing training samples and other potential damages. One specific form of damage caused by NLP language pre-trained models is the production of toxic language, which includes threats, profanities, insults, and hate speech.

There are no free meals in life. Transfer learning can repurpose models for new problems with less training data, saving time and resources despite several drawbacks.



Source link

Previous Post

CUDA by Numba Examples Part 1 | by Carlos Costa | Medium

Next Post

How to Heapify a Tree in C++. A beginner’s guide to building a heap… | by Sadrach Pierre, Ph.D. | Sep, 2022

Next Post

How to Heapify a Tree in C++. A beginner’s guide to building a heap… | by Sadrach Pierre, Ph.D. | Sep, 2022

Best Python Libraries for Machine Learning | by Akriti Srivastava | Sep, 2022

Deploying Machine Learning Models with Heroku | by Avi Chawla | Sep, 2022

Related Post

Artificial Intelligence

Exploring TensorFlow Model Prediction Issues | by Adam Brownell | Feb, 2023

by admin
February 2, 2023
Machine Learning

Different Loss Functions used in Regression | by Iqra Bismi | Feb, 2023

by admin
February 2, 2023
Machine Learning

How to organize bills? – 3 ways to track bills

by admin
February 2, 2023
Artificial Intelligence

How to decide between Amazon Rekognition image and video API for video moderation

by admin
February 2, 2023
Artificial Intelligence

The Future of AI: GPT-3 vs GPT-4: A Comparative Analysis | by Mohd Saqib | Jan, 2023

by admin
February 2, 2023
Deep Learning

6 Ways To Streamline Tech Hiring With A Recruitment Automation Platform

by admin
February 2, 2023

© 2023 Machine Learning News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us

Newsletter Sign Up.

No Result
View All Result
  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us

© 2023 JNews - Premium WordPress news & magazine theme by Jegtheme.