Aaqib Ali

How Does a Retrieval-Augmented Generation (RAG) System Work? 1. Set up the Retrieval Component 2. Adding Relevant Information to the Query 3. Generating Response Using LLM For a more detailed version, see my blog: https://lnkd.in/dccdp3GJ Challenges of Scaling RAG: https://lnkd.in/ggUqECwC Discover how MyScale, a vectorDB designed for scalar RAG applications, can streamline and optimize your RAG workflows. 🚀 #llms #vectordatabase #openai #chatgpt #rag

15

🌟 𝗙𝗲𝗹𝗹𝗼𝘄𝘀𝗵𝗶𝗽 𝗝𝗼𝘂𝗿𝗻𝗲𝘆: 𝗗𝗶𝘃𝗶𝗻𝗴 𝗗𝗲𝗲𝗽𝗲𝗿 𝗶𝗻𝘁𝗼 𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝗰𝗲! 🌟 𝗗𝗮𝘆 𝟯 & 𝟰 of the Bytewise Limited Fellowship have been a whirlwind of learning and growth! 🚀 𝗗𝗮𝘆 𝟯: 𝗝𝘂𝗽𝘆𝘁𝗲𝗿 ���𝗼𝘁𝗲𝗯𝗼𝗼𝗸 - 𝗔 𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁’𝘀 𝗕𝗲𝘀𝘁 𝗙𝗿𝗶𝗲𝗻𝗱 📓✨ 𝗣𝗿𝗼𝗴𝗿𝗲𝘀𝘀 𝗨𝗽𝗱𝗮𝘁𝗲: Successfully installed and set up Jupyter Notebook. Delved into its interactive environment, experimenting with Markdown for rich text elements. Created my first notebook with annotated code snippets and visualizations. Reflections: Jupyter Notebook is proving to be an invaluable tool for documenting my learning process, allowing me to blend code, visuals, and narrative in one place. 𝗗𝗮𝘆 𝟰: 𝗣𝘆𝘁𝗵𝗼𝗻 𝗣𝗿𝗼𝗴𝗿𝗮𝗺𝗺𝗶𝗻𝗴 - 𝗧𝗵𝗲 𝗖𝗼𝗿𝗲 𝗼𝗳 𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝗰𝗲 🐍💡 𝗣𝗿𝗼𝗴𝗿𝗲𝘀𝘀 𝗨𝗽𝗱𝗮𝘁𝗲: Took my first steps into Python, exploring variables, data types, and operators. Practiced writing Python code for calculations and data manipulation. Grasped the concept of dynamic typing and the simplicity of Python syntax. Reflections: Python’s versatility is impressive! It’s clear why it’s the go-to language for data science. Each line of code feels like a building block towards a solid foundation in programming. Stay tuned for more updates as I continue to code, analyze, and grow with each passing day of the #100DaysofByte challenge! 🌐📈 𝗚𝗶𝘁𝗛𝘂𝗯: https://lnkd.in/dfTiE38E #Bytewise #100daysofcode #100daysofbytewise #100daysofnerdflow #DataScience #Python #JupyterNotebook

4

1 Kommentar

I believe most of you have used scikit-learn to train your machine learning models. And recently, scikit-learn released version 1.5.0, bringing some useful new features to the table! Here are some highlights: 1. Tuned Threshold Classifier (TunedThresholdClassifierCV): Automatically finds the best threshold for binary classification tasks. 2. GridSearchCV with Pipeline: Inspect hyperparameters found by GridSearchCV inside a pipeline. 3. Metadata Routing: More components in scikit-learn now support metadata routing. Metadata routing allows data to be passed along pipelines. 4. Dataset API improvements (Sklearn.datasets: n_retries and delay): Allows specifying retries and delay for dataset downloads. 5. TF-IDF Transformer (TfidfTransformer): New transformer for featurizing text data where we don't need to import nltk library. 6. HistGradientBoostClassifier improvements (ensemble.HistGradientBoostingClassifier): Faster predict call even it's an ensemble model that normally takes longer time to run. Curious to see these features in action? Check out this short 13-minute video demo! #MachineLearning #scikitlearn1_5 #NewFeatures

32

#ExtractingSubstrings A substring is a string within a string. For example, tab and able are both substrings of table. string[ [ start, ] length ] string is the string containing the substring. start is a number specifying where the substring starts. The first character of string counts as 1. If start is 0 or a negative number, the starting position is assumed to be 1. If start is omitted, the starting position is calculated according to the following formula: string.length - substring.length + 1 Trailing Substrings. You can specify a trailing substring by omitting start from the syntax. For example, this specification: "1234567890" [5] returns the substring: 67890 Delimited Substrings. You can extract a delimited substring using this syntax: string [ delimiter, instance, field) e.g: in fulldate "2024-04-17T10:06:55" In case you want to trim off time. #StataCommand gen idate = substr(fulldate, 1,10) Result: "2024-04-17" See #SALL

2

🚀 Enhancing MLOps for Seamless Machine Learning Deployments! 🌐 Exploring MLOps, where precision meets scalability in deploying Machine Learning models. Here’s how a systematic MLOps pipeline ensures a robust journey from data to deployment, using an illustrative example: A. MLOps Project Initiation: - Business Problem Analysis: Define goals based on data insights, such as predicting customer churn in a telecom dataset. - Design Architecture: Choose tools and technologies for building predictive models, ensuring scalability. - Derive ML Problem: Frame the problem, e.g., predicting customer churn as a binary classification task. - Understand Required Data: Identify essential features (e.g., usage patterns, demographics). - Connect to Raw Data: Access dataset (e.g., telecom customer records). B. Feature Engineering Pipeline: - Define Transformation Rules: Handle missing data, engineer features like usage trends and customer demographics. - Define Engineering Rules: Normalize data, encode categorical variables for model readiness. - Data Extraction & Cleaning: Extract data from sources, cleanse for consistency. - Feature Engineering: Create new features, such as derived metrics or customer segments. - Data Ingestion: Store prepared data for model training. C. Experimentation: - Data Analysis: Explore relationships between customer attributes and churn rates. - Data Preparation & Validation: Split data for training/testing, validate model inputs. - Model Training: Experiment with algorithms (e.g., decision trees, neural networks) to find optimal solutions. - Model Validation: Assess model performance through metrics like accuracy and precision. - Export Model: Save best model for deployment. D. Automated ML Workflow Pipeline: - Automated Experimentation: Implement automated pipelines to test multiple algorithms and hyperparameters. - Feedback Loop System: Incorporate user feedback and real-time data to continuously improve models. - Drift Detection: Monitor both data and model drift to ensure model performance aligns with changing real-world conditions. - Model Retraining: Schedule regular retraining cycles based on drift detection and performance metrics. - Monitoring Component: Track deployed model performance, ensuring consistency and reliability in predictions. - Model Serving Component: Deploy models for real-time predictions, integrating seamlessly with operational systems. This comprehensive MLOps workflow ensures efficiency and reliability throughout the development-to-deployment cycle, critical for successful AI integration and business impact. Let’s innovate with data-driven solutions! 🌟💡 #MLOps #MachineLearning #DataScience #AI #Deployment #TelecomData #Innovation

4

From D Hwang [Google LLC] (2024) FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information https://lnkd.in/ecCv_Ccs - Adam optimizer can be considered as an approximation of natural gradient descent through the use of diagonal empirical Fisher information matrix (FIM). - The diagonal FIM results in loss of covariance information but retains variance. This approximation seems to work well in practice. - The second momentum term in Adam is actually the diagonal empirical FIM. However, Adam uses the square root of FIM instead of the direct inverse. This is to get a more stable estimate. - The FIM relies on empirical approximations using training data rather than model distribution. This works better for discrete distributions (like text) than continuous (like images). - Some flaws in original Adam algorithm are uncovered based on the analysis. Proposed fixes are: use invariant natural gradient for momentum, adjust bias corrections, clip gradients. - Weight decay should be applied as a natural gradient too based on FIM. This explains the success of AdamW which decouples weight decay. - Overall modified algorithm is named Fisher Adam (FAdam) which shows strong performance on text, speech and image tasks. abs: This paper establishes a mathematical foundation for the Adam optimizer, elucidating its connection to natural gradient descent through Riemannian and information geometry. We rigorously analyze the diagonal empirical Fisher information matrix (FIM) in Adam, clarifying all detailed approximations and advocating for the use of log probability functions as loss, which should be based on discrete distributions, due to the limitations of empirical FIM. Our analysis uncovers flaws in the original Adam algorithm, leading to proposed corrections such as enhanced momentum calculations, adjusted bias corrections, and gradient clipping. We refine the weight decay term based on our theoretical framework. Our modified algorithm, Fisher Adam (FAdam), demonstrates superior performance across diverse domains including LLM, ASR, and VQ-VAE, achieving state-of-the-art results in ASR. #AI #LLM

24

🚀 Bytewise Limited 𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝗰𝗲 𝗙𝗲𝗹𝗹𝗼𝘄 - 𝗗𝗮𝘆𝘀 𝟳 𝘁𝗼 𝟵 𝗨𝗽𝗱𝗮𝘁𝗲 📚 Sharing my progress on Days 7 to 9 of the coding challenge! Let’s dive into the tasks and concepts covered. 🌟 𝗗𝗮𝘆 𝟳: Student Data and Library Book Management Applications: Developed two separate applications: Student Data Application: Features: Add new students, search by ID, delete students, and display all student data. Data persistence: Records saved to text files and reloaded on program start. Library Book Management Application: Features: Add new books, search by ID, borrow/return books, and display book data. Data persistence: Similar to student data, records saved to text files. Whether you’re an educator or a book lover, these apps simplify data management! 🎓📚 𝗗𝗮𝘆 𝟴: Inventory Management System: Introduction: Python app to manage products, categories, and stock levels. Goals: Use data structures (lists, dictionaries, sets, tuples), implement functions for adding/updating products, and define Product/Inventory objects. Demonstrate Python data structures and OOP understanding. Requirements: Python 3.8+, basic understanding of data structures and OOP. Deliverables: inventory.py script, intro.md overview, and sample data (data.txt). 𝗗𝗮𝘆 𝟵: Introduction to Statistics: Overview: Key concepts: mean, median, mode, and common distributions (normal, binomial, Poisson, uniform). Basic probability understanding. Solve problems manually and document your work. 🔗 GitHub Repository: https://lnkd.in/dkBaWKAU Let’s keep coding and learning together! Feel free to share your thoughts or ask questions in the comments below. Happy coding! 💻🚀 #100DaysOfCode #CodingChallenge #DataScience #Statistics #Python #Bytewise

10

2 Kommentare

- MobileCLIP-S0 achieves comparable zero-shot performance to OpenAI's ViT-B/16, with a 4.8x speed improvement and 2.8x smaller size. - MobileCLIP-S2 outperforms SigLIP's ViT-B/16 in average zero-shot performance, boasting 2.3x faster speed, 2.1x smaller size, and trained with 3x fewer seen samples. - MobileCLIP-B(LT) achieves a remarkable zero-shot ImageNet performance of 77.2%, surpassing recent benchmarks like DFN and SigLIP with similar architectures, as well as OpenAI's ViT-L/14@336. Code: https://lnkd.in/gQfUpcQT Paper: https://lnkd.in/gHGux7MS

9

🚀 Breaking Down the Attention Mechanism in Neural Machine Translation 🚀 In 2015, Bahdanau et al. introduced an incredible innovation in the world of Neural Machine Translation: the Attention mechanism. 🧠✨ It's a cornerstone referenced in the "Attention is All You Need" (Vaswani et al., 2017) paper that pioneered the Transformer model! #AttentionMechanism #Seq2Seq #MachineTranslation #NeuralNetworks #DeepLearning #Transformers #Alignment #NLP

11

RAGAS: For Evaluating RAG Systems We have many evaluation metrics in NLP, but how do we evaluate Retrieval-Augmented Generation (RAG) systems? Here comes RAGAS (Automated Evaluation of Retrieval Augmented Generation). RAGAS is an automated evaluation metric designed to assess the quality of outputs from RAG systems. Effectively evaluating these systems necessitates metrics that consider both the relevance of retrieved documents and the quality of the generated text. Key Evaluation Metrics in NLP Evaluating NLP models requires a range of metrics tailored to different tasks. Here's a quick overview of the key metrics used across various NLP applications: Classification Metrics: 1. Accuracy: Measures overall correctness. 2. Precision & Recall: Balances false positives and false negatives. 3. F1 Score: Harmonic mean of precision and recall. 4. ROC-AUC: Performance across classification thresholds. Translation & Summarization Metrics: 1. BLEU: N-gram overlap for translations. 2. ROUGE: N-gram overlap for summaries. 3. METEOR: Precision, recall, synonymy, and word order. 4. CIDEr: Consensus-based evaluation for image descriptions. Language Generation Metrics: 1. Perplexity: Model uncertainty. 2. Human Evaluation: Subjective quality assessments. Question Answering Metrics: 1. Exact Match (EM): Perfect prediction match. 2. F1 Score: Token overlap accuracy. 3. BLEU: Prediction similarity. #NLP #MachineLearning #AI #Metrics #DataScience #genai #evalutaion #metric

15

1 Kommentar

Learn how to fine-tune Seq2Seq models using Kaggle's free GPUs and flaunt your work on Hugging Face Hub. Our ML engineer Abdul Rehman Raja's latest article is the ultimate walkthrough. Link: https://lnkd.in/d_qHm_3x

41

5 Kommentare

🚀 Excited to share a recent implementation for learning purpose. 🚀 I successfully implemented a Variational Autoencoder (VAE) in PyTorch for image segmentation, achieving impressive results. The project incorporated several powerful tools to ensure efficiency and scalability: - **DVC (Data Version Control)** for managing datasets and machine learning experiments. - **MLflow** for tracking the machine learning lifecycle, including experiments, reproducibility, and deployment. - **DagsHub** to streamline data discovery, observability. - **Docker** for containerization, ensuring consistency across different development and production environments. This project was a fantastic opportunity to integrate and leverage these cutting-edge tools, resulting in a robust and scalable image segmentation solution. Github link of the project is given below: https://lnkd.in/ev99ZD9z #MachineLearning #DeepLearning #PyTorch #VAE #ImageSegmentation #DataScience #MLOps #DVC #MLflow #DataHub #Docker #AI

1

Two of the biggest feedback questions we get with Gradient and Databricks optimization are: - Where do I even start with optimization? - Where's my single pane of glass where I can see the ROI across all jobs? Well ask no more! We've just GA'd some major new dashboards and insights for our users over the past few months. We've even re-worked our entire user onboarding experience, and boiled it down into 3 major steps: 1). Discover - Find new jobs to optimize in 1 minute 2). Monitor - Track job metrics and costs for total ROI 3). Automate - Save time, money, and hit SLAs with automation The best part is step 1, Discover, can be completed from scratch in less than a minute. No complex integration required here, just your Databricks credentials. Read our product release blog to learn more! Also - give Discover a shot now! https://lnkd.in/eVK5e_ke https://lnkd.in/eNUu8aNt Sync Computing #dataaisummit2024 #databricks

17

In an article from the Sequence on Substack, I came across interesting techniques related to agent reflection and refinement: 1)𝐒𝐞𝐥𝐟-𝐑𝐞𝐟𝐢𝐧𝐞:  Technique involves a cycle of creating a plan, getting feedback, and making necessary adjustments.  Paper -> https://lnkd.in/dgNqitYJ 2)𝐑𝐞𝐟𝐥𝐞𝐱𝐭𝐢𝐨𝐧: Building upon self-refinement, it adds an evaluation step. The AI assesses its actions and corrects itself when it identifies errors. Paper -> https://lnkd.in/diAnmEsZ 3)𝐂𝐑𝐈𝐓𝐈𝐂: Taking it further, CRITIC uses external resources to verify suggested actions by the LLM which could increase the accuracy of factual info. Paper -> https://lnkd.in/dwjCq-CU 4)𝐈𝐧𝐭𝐞𝐑𝐞𝐜𝐀𝐠𝐞𝐧𝐭: This process, called ReChain, allows the AI to evaluate its responses and decide whether to start over. Paper -> https://lnkd.in/dZegJi8y 5)𝐋𝐄𝐌𝐀: LEMA focuses on identifying planning errors and uses an advanced AI version for corrections, enhancing overall performance. Paper -> https://lnkd.in/dbsEtTPv 6)𝐀𝐠𝐞𝐧𝐭𝐕𝐞𝐫𝐬𝐞: When groups tackle problems, they dynamically adjust team composition during task execution. The task-solving process involves four essential phases: Expert Recruitment, Collaborative Decision-Making, Action Execution, and Evaluation. Paper -> https://lnkd.in/d_-ShswF #agent #artificialintelligence #llm #reflection #reasoning

5

Time Series Algorithms #timeseries #datascience #machinelearning

10

While exploring various transformer-based architectures, I stumbled upon a great blog titled: "Understanding Large Language Models" by Sebastian Raschka, PhD The blog provides a concise overview of the key papers that have shaped the field of large language models (LLMs) in recent years. Sebastian mentioned the papers to read in chronological order to understand the evolution of LLM architectures and training techniques. The post starts with the original transformer paper from 2017 and covers influential works on attention mechanisms, layer normalization, and the bifurcation of LLMs into encoder-style models for predictive tasks and decoder-style models for generative tasks. It also discusses the ULMFiT approach for finetuning language models on specific tasks. The blog concludes by covering the topic of alignment, which aims to steer LLMs towards intended goals and interests. Key papers discussed include the InstructGPT approach that led to ChatGPT and a survey on parameter-efficient finetuning methods. Overall, the blog provides a well-curated reading list for researchers and practitioners looking to understand the rapid progress in LLMs over the past few years. Here's the link: https://lnkd.in/dEuq4e8p Happy Learning! #LLMs #LargeLanguageModels #GenAI #NLP #NaturalLanguageProcessing #TransformerArchitecture #ScalingLaws #ParameterEfficientFinetuning #LanguageModelAlignment #AIResearch #MachineLearning #ComputerScience

15

Offset pagination works OK for small datasets. For large datasets, use this instead: It's called 𝗞𝗲𝘆𝘀𝗲𝘁 𝗽𝗮𝗴𝗶𝗻𝗮𝘁𝗶𝗼𝗻 or 𝘀𝗲𝗲𝗸-𝗯𝗮𝘀𝗲𝗱 𝗽𝗮𝗴𝗶𝗻𝗮𝘁𝗶𝗼𝗻 and is a much more efficient way to paginate through a large dataset. The problem with Offset pagination (skip and take) is that the DB must still process the skipped entries, even if they aren’t used. But with Keyset pagination, you don’t need to skip any rows. You just remember the last key you fetched and use it to fetch the next rows. As long as your ID column is indexed, queries that use Keyset pagination will be very efficient, even for large datasets. Now, writing the LINQ query is not that hard, but how to create a full-stack ASP.NET Core application that can keep track of that last ID? Tomorrow, my newsletter will present a complete ASP.NET Core implementation of Keyset pagination. You will learn: - Which DTOs to implement for requests and responses - How to implement the backend API that keeps track of that last ID - How to implement a modern frontend that uses Keyset pagination Join 14,342 .NET developers here to not miss the issue 👇 https://lnkd.in/e-XkNptn

1.003

56 Kommentare

[Call for Participation: 3 ICML paper presentations and 1 KDD Tutorial presentation] We are happy to share that our group is associated with the three ICML papers in the upcoming conference and is presenting a KDD Tutorial on Reasoning and Planning with Large Language Models in Code Development. ICML papers: [1] Collage: Light-Weight Low-Precision Strategy for LLM Training: We develop the training recipe of leveraging low-precision floating point units (FPUs) effectively provided the accounted error is properly compensated at the critical locations in the training process. https://lnkd.in/gVT8TS-x. [2] Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models: we leverage (forward pass only) zero-order methods with variance reduction techniques to enhance stability and convergence for LLM training, in need of smaller memory requirement than backprop-based methods. https://lnkd.in/gnwwanV8. [3] Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation. The team proposed a new method to measure the task-specific accuracy of Retrieval-Augmented Large Language Models (RAG). https://lnkd.in/gV_paB8c KDD Tutorial: Multi-Hop Reasoning is an emergent capability of LLM. Such reasoning capability is critical for AI assisted code development. In this tutorial we explore the multifaceted impact of LLMs on code development, surveying techniques for generating high-quality codes, automatically documentation creation, code debugging, code decomposition, creating comprehensive test cases, among others.

112

3 Kommentare