Lingfeng Shen’s Post

🌟 Excited to share a new blog post on SFDDM: Single-fold Distillation for Diffusion models. This paper proposes a single-fold distillation algorithm, SFDDM, which effectively compresses the teacher diffusion model into a student model of any desired step. The student model, trained by SFDDM, demonstrates remarkable performance by sampling high-quality data with steps reduced to as little as approximately 1%, thus trading off inference time. Check out the full post here: https://bit.ly/3Kkmkko #MachineLearning #ArtificialIntelligence #DataScience

🚀 Launch Announcement: Unleash the Power of Linear Algebra with libHSL! 🚀 Exciting news for all researchers and industry professionals! I'm delighted to introduce libHSL, a game-changing solution for linear algebra computations with a focus on high-performance linear solvers. Get ready to experience a new level of efficiency and accuracy in your projects. 🌟 Key Highlights: 🌟 💪 Unparalleled Performance: libHSL boasts lightning-fast linear solvers, accelerating your computations and reducing resource usage. 🔧 Optimization Compatibility: Seamlessly integrates with optimization software like Ipopt, enhancing your tools' capabilities. 🌍 Global Accessibility: Accessible to researchers, academics, and professionals worldwide, fostering collaboration and innovation. 🔗 Get Started with libHSL: Explore the power of libHSL by visiting the libHSL website: https://lnkd.in/gt6F4xCq. #libHSL #LinearSolvers #Performance #LinearAlgebra #Optimization #ScientificComputing #ResearchTools

NEW: Congratulations to HumRRO's Harold Doran, Tetsuhiro Yamada, Ted Diaz, Emre Gonulates, and Vanessa Culver on publishing their paper this week in the National Council on Measurement in Education (NCME)'s Journal of Educational Measurement! Their paper, "A Generalized Objective Function for Computer Adaptive Item Selection," presents a new item selection algorithm for computer adaptive testing (CAT) based on a generalized objective function to support multiple types of testing conditions and principled assessment design. Lead author, Dr. Doran, explains the motivation behind the work: "We had a few goals that included building a flexible algorithm to support personalized assessments, building an algorithm that scales with high testing volumes to minimize latency and cost containment for industry-grade applications, and providing a consolidated resource so this work can also stand alone as a specification document with all computational details needed to build an instance of this CAT. We also learned a few interesting things along the way to help reduce test length and improve item exposure rates." Access the full paper here: https://lnkd.in/ead_cRFp Read about the web-based CAT simulation tool here: https://lnkd.in/gPZVyZyF #JEM #JournalofEducationalMeasurement #NCME #Research #CAT #ItemSelection #Algorithm #Assessments

Just finished my talk at ICIAM 2023. I talked about our recent work on Riemannian ADMM. Existing ADMM for manifold optimization with nonsmooth obj do not have convergence analysis. Ours is the 1st one with convergence guarantee. Paper here: https://lnkd.in/gvrVd2n2

While the research on RAG is expanding, it predominantly revolves around systematic reviews and comparisons of new state-of-the-art (SoTA) techniques versus older ones. The paper, in the link below, (written by ML guys from Predli and University of California, Berkeley) aims to bridge a gap by conducting extensive experimental comparisons where they evaluated various RAG methods and analyzed their impacts on retrieval precision and answer similarity. The Sentence Window Retrieval (SWR) emerged as the most effective method for retrieval precision, despite variable performance on answer similarity. They revealed that Hypothetical Document Embedding (HyDE) and LLM reranking, combined with SWR notably improve retrieval precision. However, Maximal Marginal Relevance (MMR) and Cohere rerank did not show significant advantages over a baseline Naive RAG system. Multi-query approaches underperformed in their assessments. According to the paper, Document Summary Index could be a competent retrieval approach. All resources related to this research are publicly accessible for further investigation in this repo: https://lnkd.in/db5Ecn29 paper:https://lnkd.in/dq6XRJcN #RAG #LLM

Excited to share our latest blog post on "Supervised Contrastive Learning based Dual-Mixer Model for Remaining Useful Life Prediction"! This paper addresses the challenges in RUL prediction by introducing the Dual-Mixer model, which employs flexible layer-wise progressive feature fusion and the Feature Space Global Relationship Invariance (FSGRI) training method. The proposed method outperforms latest research works on the C-MAPSS dataset, showing significant improvements in RMSE and MAPE. Click here to read the full paper: https://bit.ly/3ue8Gee

📢 Late Advertising of Our Previous AISTATS Work! 📢 Excited to share that our paper, "Optimal Estimation of Gaussian (Poly)trees," presented at AISTATS 2024, is now available online! Dive into the details here: https://lnkd.in/gd5WTsmH. In this work, we develop optimal algorithms for learning Gaussian trees from data. We tackle both distribution learning (in KL distance) and structure learning (exact recovery). Key highlights: - The first approach utilizes the Chow-Liu algorithm for efficient learning of an optimal tree-structured distribution. - The second approach modifies the PC algorithm for polytrees, employing partial correlation as a conditional independence tester for constraint-based structure learning. - We provide explicit finite-sample guarantees for both methods and demonstrate their optimality by deriving matching lower bounds. Our findings establish the optimal sample complexity for learning Gaussian trees. Additionally, we achieved nearly optimal testers for mutual information and conditional mutual information between Gaussian variables. Check out our work and explore the exciting advancements in this area! #AISTATS2024 #GaussianTrees #DataScience #MachineLearning #Research #OptimalSampleComplexity

Join us next week, April 3, for another exciting talk in our Spring 2024 Center for Targeted Machine Learning and Causal Inference (CTML) Seminar Series. We will host Noel Pimentel and their talk "Score Preserving Targeted Maximum Likelihood Estimation." A TMLE targets an initial estimator of the data density towards a multivariate target estimand so that the updated density estimator solves the efficient influence function of the target estimand. However, if the initial estimator already solves a large collection of score equations, TMLE will not generally preserve scores at the cost of solving the efficient influence function. Solving extra scores beyond the targeted efficient influence function reduces the exact remainder in the first order expansion of the TMLE of the target estimand which can help with finite-sample performance. Because data is sometimes limited and we want estimates as close to the truth as possible, we present a TMLE that preserves the score equations solved by the initial estimator, so that the TMLE solves both the efficient influence function and the score equations solved by the initial estimator. Where: Berkeley Way West | 5th Floor, Room 5401 When: Wednesday, April 3 | 12:00 PM #ctml #ucberkeley #machinelearning #causalinference #berkeleyCTML #tmle #ctmlseminarseries

I am glad to share that I successfully completed the course "Fundamentals of Statistics" offered by MITx (18.6501x). It is an excellent in-depth statistics course that covers both theoretical and practical subjects! Many thanks to MITx on edX and the OL family for this excellent learning experience! Special thanks to Jacob Frias Koehler, PhD for teaching and helping to develop new industry skills! #MITx #micromaster #statistics #datascience