Tom Aarsen’s Post

Marqo published a short course (free, no login) on finetuning embedding models for Semantic Search, covering the foundations of embeddings, vector databases all the way to finetuning embedding models with Sentence Transformers, Vision Transformers, and CLIP/Multimodal models. Check it out here: https://lnkd.in/eAhQkhNC It is a very complete overview, and I'd recommend reading it & running the corresponding code to learn how to do all of this yourself. Here are the current chapters: 1. Introduction to Vector Embeddings 2. Foundations of Embedding Models 3. Introduction to Vector Databases 4. Build Your First Vector Search Application 5. Introduction to Sentence Transformers 6. Training and Fine-Tuning Sentence Transformers 7. Introduction to Vision Transformers 8 Training and Fine-Tuning Vision Transformers 9 Introduction to CLIP and Multimodal Models 10 Fine-Tuning CLIP Models Great work on this by Ellie Sleightholm. Semantic Search/Retrieval models are oh so important, and finetuning them remains one of the best ways to get the most out of the existing models.

There's a new #1 Arabic embedding model for semantic similarity according to the MTEB leaderboard. OMER NACAR - M.Sc. released 6 models, 4 of which outperform all previous models on the STS17 Arabic-Arabic task. Details: 📚 Trained on a large dataset of 558k Arabic triplets translated from the AllNLI triplet dataset 6️⃣ 6 different base models: AraBERT, MarBERT, LaBSE, MiniLM, paraphrase-multilingual-mpnet-base, mpnet-base, ranging from 109M to 471M parameters (see underlined models in the picture). 🪆 Trained with a Matryoshka loss, allowing you to truncate embeddings with minimal performance loss: smaller embeddings are faster to compare. 📈 Outperforms all commonly used multilingual models like multilingual-e5-large, paraphrase-multilingual-mpnet-base, and LaBSE. Check out the 6 models here: https://lnkd.in/ecGrcZdr My personal favourite is likely Arabert-all-nli-triplet-Matryoshka: a very efficient 135M parameters & scores #1 on MTEB. Great work OMER NACAR - M.Sc.!

Akshita Sukhlecha just released a state-of-the-art (Romanized) Hindi embedding model. It performs excellently on search/information retrieval/RAG. Based on my experience, I'd argue that it follows a very solid training strategy as well. Great work!

This'll be very useful for e.g. finding question-context datasets for training SOTA RAG embedding models!