How to Train Your Own Language Model

OpenAI recently released GPT-4o – their flagship multimodal artificial intelligence (AI) model that can process text, audio, and vision in real-time, i.e., it can respond to an input query with any combination of text, audio, and vision within 320 milliseconds.

That’s human-level response time in a conversation.

It took OpenAI 6 years to achieve this performance.

They started developing the GPT family back in June 2018.

Compared to present-day GPT-4o, the first GPT was a “humble” language understanding model.

It was one of the first large language models (LLMs) that could perform language modeling tasks like textual entailment, semantic similarity, reading comprehension, commonsense reasoning, and sentiment analysis with significant accuracy.

Besides the GPT family, many model families started as LLMs and morphed into powerful multimodal AI models after years of improvement.

Hence, LLMs are considered the cornerstone of modern AI advancements.

TL;DR:

• OpenAI recently launched GPT-4o, a state-of-the-art multimodal AI model with human-level conversational speed.
• Large language models (LLMs) like GPT have revolutionized AI, enabling tasks from chatbots to medical diagnostics. You can build your LLM by:
  • Gathering high-quality training data.
  • Tokenizing the data into numerical sequences.
  • Choosing a transformer-based model architecture and tuning hyperparameters.
  • Using external vector databases for enhanced accuracy.
  • Implementing guardrails to mitigate bias and hallucination.
  • Evaluating and fine-tuning the model for optimal performance.

• LLMs face challenges like reliability and privacy concerns but offer vast potential in text generation, summarization, translation, and more.
• Future advancements aim for better multimodal capabilities and ethical AI practices.

Jump to Section

Importance of LLM in Modern AI
How Does a Language Model Work?
Steps to Train Your LLM
Using Pre-trained Models
Applications of LLMs
Limitations and Challenges in Training LLMs
Future of LLMs
FAQ
Conclusion

Importance of LLM in Modern AI

They serve as a powerful bridge for human-computer interaction.

Most importantly, they can process the biggest data modality on the internet, which is textual data.

They provide humans with a flexible interface to query large datasets and find answers quickly, can hold realistic conversations, write creative text, and break down complex ideas easily.

In addition, they are an integral part of modern-day chatbots, powering the customer support services of many organizations as well as engaging content for social media marketing campaigns.

Moreover, they assist doctors and patients with clinical diagnosis and can help with translating and preserving many low-resource languages.

In conclusion, LLMs have played an integral part in democratizing modern AI to the masses. And, any robust AI system would be incomplete or inefficient without LLM capabilities.

How Does a Language Model Work?

Today, large language models are much more advanced and complex.

However, their core architecture and training methodology remains the same.

This is why anyone with a little bit of programming knowledge can build and train their own language model.

This is what we are going to learn today.

But for that, we need to understand how a large language model works.

We’ll briefly discuss the major components of a language model.

 1. Training Data

The training data is the soul of a language model.

High-quality training data results in more accurate LLMs while low-quality data adversely affects the model’s performance.

This is why AI researchers spend days or months collecting task-specific high-quality data from multiple sources.

For instance, for a finance-based LLM, researchers would have to curate training data from financial blogs, newsletters, organizational reports, surveys, and social media trends.

Then this raw data will be cleaned up in a process known as data preprocessing.

They would perform several tasks such as removing redundant values, filling up missing values, and anonymizing critical information like email addresses, IP addresses, passwords, or Personally Identifiable Information (PII) to improve the quality of data and make it suitable for training.

Once preprocessing is complete, the data is ready for the next step, i.e., tokenization.

2.  Tokenization

Language models cannot understand sentences or words written in a language directly.

Like any other computer program, they need a numerical sequence to understand the input and perform complex mathematical operations (like matrix multiplication).

Before obtaining the numerical sequence, we perform tokenization on the training dataset.

Tokenization is the process of converting LLM’s training data into smaller discrete subword units known as tokens. One token typically corresponds to ~4 text characters for common English text, but it can be smaller or larger.

Before model training, the preprocessed data is passed through a tokenizer that prepares a domain-specific vocabulary of tokens.

Researchers use different tokenizer algorithms best to convert the training data into a tokenized vocabulary.

These tokens are then assigned a unique numerical representation through a process called embedding (discussed below) and passed on to the model for training.

3. Model Architecture

Before feeding the tokenized data to your LLM, you need to decide what kind of model architecture you would like to use.

Most LLMs are built on top of a transformer-based architecture – introduced by Google in 2017.

For instance, model families like GPT and BERT are built using the transformers.

As an AI practitioner, you need to decide your model layers and hyperparameters.

You also need to decide which attention mechanism your transformer will use.

For example, multi-head attention, self-attention, and flash attention are common attention mechanisms used by transformer models.

In simple words, the attention mechanism allows the transformer model to focus on the most important portions of the input sequence, i.e., the portion most suitable for making predictions.

Now that we are familiar with the fundamentals of a language model, let’s look at the steps you need to perform to build your custom model.

Steps to Train Your LLM

Suppose you want to create a question-answering LLM to process and query your company’s internal documents and communication workflows.

A good example of this use case is PwC’s ChatPwC LLM which is improving PwC’s employees’ productivity in service delivery.

If you want to build such a model, you have two options:

1. Choose a proprietary LLM API like OpenAI or Amazon Bedrock and customize their pre-trained model to your specific case by fine-tuning it on your custom dataset.
2. Build your own LLM applications from scratch using frameworks like Langchain and LlamaIndex.

We’ll discuss the second option today so you can understand the LLM training process in detail.

(In production, we recommend using managed LLM APIs like OpenAI, Bedrock, etc., directly. Use Langchain or LlamaIndex just for experimentation.)

Step 1: Gathering and Preparing Your Data

First, identify your company’s internal data sources, such as emails, planning/projection documents, project/product/technical documentation, policy/HR documentation, reference documentation, budget tracking, etc.

Then, create a pipeline to curate all the data in a data store or data warehouse. This process would include transforming, cleaning, and standardizing all data.

Documents and emails could be labeled under their relevant project name or they can be categorized based on project phase, such as planning, design, implementation, etc.

Outdated documents and emails would be discarded.

You can also use data versioning tools to manage your datasets effectively.

After multiple iterations of data cleaning and preparation, it is ready for tokenization.

Step 2: Tokenization

During tokenization, the preprocessed dataset is converted into a vocabulary of tokens.

The tokens can be characters, words, parts of words, punctuations, phrases, regular expressions, special characters, etc.

Once the tokens are obtained, you can remove stop words and apply two linguistic techniques: stemming (remove common prefixes or suffixes from tokens) and lemmatization (find the base word).

They can simplify the tokens and build a more accurate token dictionary.

Python-based natural language processing libraries like NLTK and spaCy provide multiple variations of tokenization methods that can tokenize almost any type of raw textual data.

OpenAI provides a tokenizer called tiktoken that works well with GPT models.

Choose a method based on your requirements or write your custom tokenization method.

Step 3: Building Your Model Architecture

The model architecture is the brain of your LLM application.

We can use the transformer model as the foundation of our architecture.

Since we are building our language model, we need to decide what kind of transformer model we want.

For instance, we use an encoder-only transformer (such as the BERT family), a decoder-only transformer (such as the GPT family), or an encoder-decoder transformer as the base model.

Each configuration is good for specific use cases.

We need to decide which configuration to use and how many layers of encoder/decoder blocks would be required to process our training data.

This process requires multiple iterations of experimentation before reaching an optimal model architecture.

Moreover, the model architecture also requires an embedding layer to convert the tokens into their numerical representation.

This is a critical step because our LLM will perform mathematical calculations on these embedding values to learn language patterns and nuances.

Hence, the embedding layer needs to capture and represent important text features accurately.

There are many embedding models available.

You can choose the one that represents your task. A good place to look for the best-performing embedding models is the Hugging Face MTEB leaderboard.

As of today, Nvidia’s NV-Embed-v1 is leading the leaderboard.

We can use it for our internal company LLM use case or compare the results of different embedding models and choose the best one.

Once converted, the embedded tokens can be passed onto our LLM model for training.

Moreover, you have to define a prompt template (such as using LangChain).

It is a set of instructions for your LLM to generate a response according to the guidelines set in the template.

Here, you can tell the model what kind of input prompts it should expect from the user and what should be the model’s response to such user prompts.

As a result, the model’s outcomes can be improved.

Step 4: Using an External Vector Database

A vector database stores vector embeddings – high-dimensional numerical representations of tokens.

Keep in mind that these vector embeddings are different from the embedding layer we have discussed above.

The embedding layer is typically used during LLM training while a vector database is used during LLM inference (making predictions).

Prominent vector databases like pgvector, Pinecone, MongoDB Atlas, and Qdrant are essential for retrieval-augmented generation (RAG) in LLMs.

They enable researchers to quantify linguistic relationships and capture detailed contextual information in textual datasets.

Acting as external data sources, these databases contain domain-specific factual data, allowing LLMs to quickly access accurate information.

As a result, the user gets a fact-checked and more contextually accurate response from the LLM (most of the time), minimizing the model’s hallucination and bias (discussed below).

For our internal company LLM, we can create an RAG pipeline containing the company’s entire knowledge base.

Our custom LLM can query the RAG store to fetch highly accurate answers in response to a user input prompt.

Step 5: Implementing Guardrails

Once the model is trained, we must consider its limitations, particularly bias and hallucination.

If our LLM generates an inaccurate response to a user query, i.e., abuse, racial slur, or stereotypical phrase, we need to stop it in its tracks, i.e., before the response is displayed to the user.

Hence, you need a watchdog mechanism to monitor your LLM’s responses before they can damage your company’s reputation.

Aporia Guardrails offers a complete set of tools that can mitigate brand-damaging RAG hallucinations and prompt injection attacks.

It allows you to set custom AI policies and guidelines related to user interactions with your LLM.

You can set a list of restricted topics to avoid answering irrelevant questions or you can present data leakage.

For instance, your company’s documents can contain details about employees’ salary packages or client contracts.

Despite your best efforts during data preprocessing, if some of this information is learned by the model during training, Guardrails would stop the LLM from displaying it to the user.

Step 6: Evaluating and Fine-Tuning Your Model

Once the model is trained, it must be evaluated to ensure high-quality performance.

You need to figure out which evaluation metrics are closely aligned with your use case.

For instance, we are building a question-answering LLM for our internal company documents.

Evaluation metrics like ROUGE and MRR are suitable for such tasks.

However, you should try multiple evaluation schemes to see which ones represent your task more effectively.

Now, test your trained model thoroughly and evaluate it based on your evaluation scheme.

If desired results are not achieved you can either retrain the model, which is a laborious task or you can fine-tune it using a high-quality domain-specific dataset.

For our company’s internal LLM, we have a good chance of getting optimal performance during model training because we are curating the training dataset ourselves which minimizes the need for a fine-tuning phase.

Using Pre-trained Models

Pre-trained language models are trained on a large and diverse corpus of web-scale data, capable of performing a wide range of language tasks.

During pre-training, the model learns to recognize generalized language rules, grammar, word usage, and contextual information to predict the next word or sequence of words in a sentence or text passage (this is known as language modeling).

Generally, pre-training is performed using unsupervised learning, leading to less reliance on expensive annotated data required by supervised learning approaches.

As mentioned in the beginning, Generative Pre-training (GPT) by Open AI was one of the first major pre-trained language models that used the transformer architecture to achieve state-of-the-art results on numerous language modeling tasks at the time.

Besides pre-training, an important step in their pipeline was fine-tuning their pre-trained model on a smaller task-specific supervised dataset.

This step is performed to improve the quality of outcomes for different downstream tasks.

More on fine-tuning later, let’s first discuss the various benefits of pre-trained language models.

Advantages of Using Pre-trained Models

Pre-trained models have transformed the AI ecosystem.

They are one of the main reasons for the rapid adoption of AI across domains and industries.

In addition, they enable practitioners across fields to utilize the same core models, as well as develop better tools and applications that improve business productivity and efficiency.

Pre-trained models offer numerous advantages, such as:

• Faster training: Typically, pre-trained models don’t require extensive fine-tuning cycles. Hence, the overall training time of the downstream model is reduced.
• Reduced data requirements: Pre-training datasets contain trillions of tokens. Comparatively, fine-tuning datasets only contain information related to the downstream task. As a result, data volume is reduced significantly.
• Better performance on downstream tasks: Since pre-trained models are already trained on internet-scale data, they can adapt efficiently to most downstream tasks, resulting in state-of-the-art model outcomes.
• Knowledge distillation: Besides fine-tuning, pre-trained models can be used for knowledge distillation – an AI technique used to train smaller models that mimic the performance of larger models, resulting in reduced computational requirements and memory footprint.
• Transfer learning: The information learned by a pre-trained model is highly valuable. It can be transferred to other AI models that solve different but related tasks. This process is called transfer learning.
• Faster deployment: Pre-training takes days, at times, months. For instance, GPT-4 took around five to six months of training time on some of the most advanced Nvidia GPUs. Hence, using a pre-trained model and fine-tuning it for your task can reduce the requirement of computational resources and cut the training time significantly, resulting in quicker time to market for your AI application.

How to Fine-Tune Pre-trained Models

Continuing our example of question-answering LLM for internal company documents, let’s briefly discuss how we can create a fine-tuned LLM using a pre-trained model.

First, we need to curate documents and prepare a fine-tuning dataset.

Then, we’ll tokenize this data so that our selected pre-trained model can understand it.

After that comes an important choice: selecting a suitable pre-trained model.

There are multiple factors that every practitioner must consider before selecting a pre-trained model for fine-tuning.

This includes determining the similarity between the pre-trained model and the type of problem you are trying to solve.

You need to develop a sufficient understanding of the model’s architecture and complexity to be able to interpret its behavior and performance on your fine-tuning dataset.

It may be good to consider if the pre-trained model provides customization options, such as adding your layers or features.

For fine-tuning our company LLM, we have numerous proprietary and open-source pre-trained model options.

If we want more flexibility, we can choose an open-source model like Llama-3 or similar open models.

But if you want better performance, we can utilize the GPT-3.5 model (since GPT-4 fine-tuning is currently in the experimental phase) using OpenAI API.

Once the decision is final, fine-tune the model on your curated dataset.

Try different hyperparameter configurations to achieve good results.

Then, analyze your fine-tuned model performance using evaluation metrics.

Once satisfied, your model is ready for deployment.

Applications of LLMs

In the last few years, large language models have evolved into sophisticated language understanding and generation models.

Not only have their outcomes improved, but their range and scope of task-performing capabilities have also expanded. Some of the major applications of LLMs are discussed below.

Text Generation

ChatGPT disrupted the text generation capabilities of LLMs when it was released in November 2022.

Since then, numerous text generation models have been released in the market, each claiming to surpass the others in textual accuracy and performance.

Currently, models like GPT-4, Google Gemini, Meta Llama 3, and Anthropic Claude 3 are leading models in this domain.

Besides models, there are numerous tools like Jasper, Surfer, Copy.ai, Frase, etc that consumers are using to generate text for different use cases.

Underneath, most of these tools use a transformer-based LLM architecture to achieve good text generation performance.

In the real world, LLM’s text generation capabilities are being used to facilitate critical applications.

For instance, LLMs are powering healthcare applications to facilitate patient diagnosis and care.

They are facilitating the drug discovery process. Similarly, LLMs are playing a critical role in revolutionizing education as AI tutors for underprivileged children.

Another, important application of AI-based text generation is the ability to write code in any language and build complete programming applications.

Text Summarization

Language models are powering text summarization applications for numerous use cases.

For instance, LLMs can summarize legal contacts and documents, financial reviews, newsletters, emails, academic research, literature, etc, improving the workflow productivity of knowledge workers.

LLM-based text summarization can also serve as a handy tool for media monitoring amidst information overload and fake news.

Recently, there has been a surge in the usage of AI-powered search tools like Perplexity, Microsoft Copilot, Google AI Overviews, etc.

These search engines can quickly summarize multiple search results and present condensed information to improve the user’s web search experience.

Machine Translation

For humans, it is important to understand each other.

But, the language barrier is often the biggest obstacle in communication. Modern LLMs are capable of translating text or speech into multiple languages.

They offer businesses an unprecedented opportunity to enter new markets and engage a diverse consumer base.

LLM-based machine translation can also facilitate customer support teams that deal with a demographically diverse customer base.

The customer’s speech can be translated into the language of the customer support agent and vice versa. Moreover, automated chatbots can be used to answer queries in different languages.

Speech Recognition

LLM-powered speech recognition systems enable users to write emails, compose documents, and send text messages without typing out any text.

A common usage of voice recognition is voice search using digital assistants like Siri.

These tools can also be used for voice-enabled biometric security systems.

In healthcare, LLM speech systems are being used for medical note-taking by physicians.
Such tools can also record patient voices to identify patient sentiments.

These applications are handy for therapists to identify signs of depression and anxiety.

Moreover, LLM-based speech recognition can power media captioning for videos to serve a global audience.

It can also be used during online work meetings for note-taking and real-time speech captioning.

Limitations and Challenges in Training LLMs

LLMs or AI in general face major criticism in terms of how safe it is to interact with an artificially intelligent system.

And most of the concerns raised by the AI community and users at large are valid.
Let’s discuss the two main challenges of LLMs below.

Bias & Hallucinations

Just as humans can be biased, AI can generate biased outcomes.

After all, they are trained by humans on training data collected by humans.

Hence, bias can creep into an AI model in several ways, such as historical and social inequities, and the personal bias or preferences of the AI researchers.

Sometimes, if the training data is incomplete, or the model’s prediction capabilities are not monitored or restricted, they hallucinate, i.e., make stuff up without any factual backing.

During the training process, the LLM learns these underlying biases in the data to generate faulty outcomes like racial and gender stereotypes or inaccurate ”facts”.

This can damage user sentiment and lead to a significant distrust in AI.

For instance, AI-powered hiring algorithms can discriminate job applicants based on their gender or demography.

AI-powered recommendation systems can promote stereotypical content to different population groups.
In the case of LLMs, they tend to generate abusive and bigoted text outcomes.

They can generate fake news regarding an event or generate factually inaccurate statements by merging two sources of information.

Handling of Sensitive Information

The protection of confidential data is one of the biggest challenges in the digital ecosystem.

Numerous data and AI regulations standards such as GDPR, CCPA, HIPPA, EU AI Act, etc are in effect worldwide to protect users from digital frauds and thefts.

Companies developing AI technologies must comply with these regulations and many do.

However, numerous cases have been reported where companies have been investigated and fined for not complying with the regulations properly.

For instance, recently, Garante – an Italian data protection watchdog, released a statement about the potential GDPR data privacy violations by OpenAI’s ChatGPT.

In May 2024, OpenAI was also found in another controversy where they allegedly used Hollywood celebrity Scarlet Johansson’s voice to design their ChatGPT voice mode called Sky.

They have denied these allegations and retracted Sky from ChatGPT.

Also, last year, South Korea fined both OpenAI and Meta for exposing and collecting user information without consent.

Hence, a lot of work needs to be done on a global scale to ensure the safety of AI users.

Future of LLMs

Despite known limitations and challenges of language models, LLM development and advancement are accelerating every year.

If the AI community can surpass or minimize these limitations, there can be several exciting possibilities and improvements across domains.

Shortly, AI researchers are going to focus on improving the following key areas:

Multimodal AI

The Internet or digital ecosystem is a combination of text, image, audio, and vision modalities.

Any AI system that can process all four modalities and integrate them cohesively will have a massive impact on the progress of AI.

Hence, future LLMs will have better multimodal capabilities, like text-to-image, text-to-audio, text-to-video, etc.

Multilingual AI

Conversational AI tools are limited by the language they are trained on which is mostly English.

However, researchers are now making significant advancements in training language models on low-resource languages and dialects to engage a diverse global population and preserve at-risk languages.

RAG-powered LLMs

Though retrieval augmented generation is a seasoned LLM improvement technique, researchers are innovating new RAG architectures to further improve the accuracy of LLM outcomes.

Recently released RAG-based techniques like Corrective Retrieval Augmented Generation (CRAG), MultiHop-RAG, and T-RAG are all efforts to improve RAG-enabled LLM implementations.

Ethical AI

As AI becomes more democratized, LLM researchers will focus on ensuring that their models do not discriminate during human interactions.

Significant work is being done by AI labs to develop new baseline datasets and evaluation techniques that can minimize ethical issues like abuse, violence, and hate speech in LLM outcomes.

Moreover, specialized tools are being developed that are capable of detecting and mitigating AI risks.

For instance, Aporia Guardrails can mitigate off-topic detections, AI hallucinations, data leakage, prompt leakage, profanity, and abuse in LLM outcomes in real time, enforcing your company’s custom AI policy regarding risks.

FAQ

Conclusion

We have seen how LLMs work and discussed a step-by-step process of how you can build and train your LLM.

An important question remains to be addressed: Why would you want to train your language model?

Especially, when you have numerous proprietary and open-source LLMs available in the Generative AI ecosystem.

Well, there can be many reasons, such as if you want to:

• Tweak a model’s architecture to obtain better results.
• Build a custom LLM model that is trained and fine-tuned on your proprietary training data.
• Ensure data privacy and security for your customers by not sharing their data with external systems.
• Control how and when your LLM application is updated or improved.
• Ensure your AI system is independent of third-party technologies.
• Own the intellectual property of your LLM.
• You don’t want to pay the API usage fee of a proprietary LLM model.

Overall, it comes down to your specific needs and use cases.

Mitigate AI risks for any data modality, Try out Aporia Guardrails today!