Kolena

At the inaugural AI Quality conference in San Francisco, transformative leaders shared profound industry and government insights. Critical topics such as AI Quality Standards, data integrity, lineage, data decay and drift and data ownership were explored. Strategies that eliminate hallucinations were analyzed alongside cutting-edge design and coding approaches for Retrieval Augmented Generation (RAG), Generative AI (GenAI), and Large Language Models (LLMs). California is proposing test recommendations for LLMs, development strategies that ensure safety protocols, and plans for a public cloud cluster, CalCompute, designed for researchers and startups without the resources to access such expensive systems. Senate Bill 1047, introduced by Senator Scott Wiener (D-San Francisco) and passed in May 2024, aims to ensure safe AI development by establishing clear safety standards and a testing framework for large-scale AI systems while avoiding stifling or burdening innovation. Case studies underscored benefits of the "shift-left" mindset are still applicable. Rigorous testing and newer frameworks define current AI/ML systems maturity levels by targeting trust, enhanced security, privacy protections and safety considerations that meet or exceed self-prescribed and regulatory compliance standards. Advocating for guardrails and continuous monitoring of enterprise AI/ML systems with added controls, lessons-learned from these case studies, underscored the effective value of production readiness checkpoints using high-quality evaluation signals, metrics tracking and non-compliance alerts. These methodologies enhance AI/ML governance, seamlessly aligning with more successful test automation in development and observability in production. Ensuring the consistent retrieval of the latest data is paramount for making well-informed decisions based on the most current data available. Maintaining data integrity, AI/ML systems reviews, and data privacy ensure accuracy, consistency, and reliability, enhancing decision-making and upholding the highest standards of quality, trust, and accountability—a win-win for all stakeholders. Thank you for the awesome hosting Mohamed Elgendy, Phillip Knorr, AIQCon, MLOps Community Nitin Aggarwal, Sr. Director Gen AI at Microsoft Rama Akkiraju, VP Enterprise AI/ML at NVIDIA Steven Eliuk, VP AI & Governance at IBM Hira Dangol, VP AI/ML and Automation at Bank of America Ian Eisenberg, Head of Governance Research at Credo AI Mohamed Elgendy, CEO & Co-Founder at Kolena Salma Mayorquin, CEO at Remyx AI Olga Beregovaya, VP, AI at Smartling Ashley Antonides, PhD, Research Director, AI/ML at Two Six Technologies Faizaan Charania, Senior PM ML at LinkedIn Scott Wiener, California State Senator Chip Huyen "Designing ML Systems" Best Selling Author at Voltron Data Shailvi Wakhlu, Founder at Shailvi Ventures LLC #GenAI #ISO42001 #AIIA #IAAct Encora Inc. Provectus Gina Acosta Gutiérrez Asad Khan Armand Ruiz The Human Ai Institute®

We had a great time at the Auto AI conference-the intimate setting with high quality attendees makes for a productive 2 days!

The AI Quality conference by MLOps Community and Kolena was such an informative experience! Lots of different talks and panel discussions from experts at leading organizations, each packed with a wealth of insights. To highlight a few: - Gordon Hart (Kolena) on using 'quality signals': light-weight classifiers to identify failure modes and add meaningful metadata for images - Chip Huyen (Voltron Data) on debugging AI applications: information-based failures, behavior-based failures, and latency, and how to address those - Mohamed El-Geish (Monta AI) on evaluating ML systems in the real world: shared pitfalls like type III error, streetlight effect, and perverse incentives, with a shoutout to Goodhart's Law - 'when a measure becomes a target, it ceases to be a good measure' - Linus Lee (Notion) on evaluating evaluations: evaluations as co-ordination, as communication, and as value - Emmanuel Ameisen (Anthropic) on integrating LLMs into products: sharing different strategies for doing so Highly recommend checking out the conference recording when available. Thank you to Demetrios Brinkmann and the organizing team! #DataDesigntist #BayAreaBytes #MLOps #AIQCon2024 #ArtificialIntelligence #SanFrancisco

It was my pleasure to host a fireside chat this week on "Attracting & Driving AI Talents in High Growth", at the AI Quality Conference. Addressing bias and quality issues in AI isn't optional! The talent you hire plays a huge role in ensuring intentionality in creating thoughtful solutions. Olga Beregovaya, Ashley Antonides, PhD and I had a great 30-minute discussion packed with strategies for leaders looking to attract, grow, and retain exceptional talent in the AI space. Thanks to Kolena and MLOps Community for creating a space for this discussion, and for organizing the city's first-ever forum for AI Quality! #AIQCon #WomenInAI

Our founders, Mohamed Elgendy, Andrew Shi, and Gordon Hart at AIQCon 👏🏼👏🏼👏🏼

The day is finally here! Doors are open at AIQCon!

Danielle D., Skip Everling, Jacob Rothenberg, and Sam Ott had a great time at CVPR last week in Seattle! We're here to continue the computer vision conversation-schedule a demo to see how Kolena can quickly how you how your ML systems will perform in the real world! https://lnkd.in/gnYRv6xB

This is going to be amazing! Make sure to get your tickets now for this epic event (we only have a few left!) https://lnkd.in/gxTzZSXR

🇺🇸 We are delighted to announce that PREM AI will be participating in the inaugural AI Quality Conference (AIQCon) in San Francisco on June 25th. This pioneering event, hosted by Kolena and the MLOps Community, marks a significant step forward in establishing new gold standards for AI quality. AIQCon aims to bring together esteemed leaders and professionals from across the AI industry to address the paramount challenge of developing robust and reliable AI and autonomous systems. The mission is to leverage the collective knowledge and experience of the community to create a global movement towards reliable #AI solutions and standards. We are eager to contribute and learn from the innovative discussions and insights that will shape the future of AI quality. PREM AI is honored to be part of this groundbreaking conference, and we look forward to collaborating with industry experts to drive the advancement of trustworthy AI systems. Join us in #SanFrancisco as we work together to define the future of AI quality.

“A human can’t read and evaluate things 24/7 and take actions in milliseconds. That’s why the Turing test doesn’t apply anymore because now we’re talking about the same capability as a human but at a 100,000X improvement in scale, speed and accuracy because it’s retrieving much more information,” -Mohamed Elgendy, CEO & Co-Founder, Kolena Check out InformationWeek's article featuring our CEO/Co-Founder's thoughts on using AI to monitor AI. https://lnkd.in/gGBUwGji