Anu Jain

#GenAI models necessitates training on extensive datasets, which in turn requires robust #cloud storage solutions. As Gen AI continues to advance, demand for cloud services would surge correspondingly, creating a symbiotic growth relationship b/w the two. https://t.ly/9Rg-C

9

Mastering the art of impactful communication is key to making a difference, especially when leading a team that is in the driving seat to understand and guide others through a phase of unprecedented buzz and hype around data and AI. I just did my part and am glad that I have something to show for it. There is already so much to catch up on, but taking a step back and reflecting on what is truly required made me realize this. This is a course that I recommend to everyone who wants to understand the right framework of communication or validate the assumptions that have helped them shape their communication style. Thanks to INSEAD and Ian Woodward for an incredibly well-structured program. #Leadership #Communication #ContinuousLearning #INSEAD #ProfessionalGrowth

20

2 Comments

GenAI has emerged as a disruptive force, altering processes and driving innovation across industries. Even industries historically hesitant about adopting new technologies are recognizing the transformative potential of GenAI, embracing it one use case at a time.  At Fractal, we leverage the power of AI, engineering, and design expertise to deliver data-powered solutions customized to your business needs. Anindya Sengupta, Client Partner, Consulting Insurance, illuminates GenAI's potent capabilities across different industries—  Enhanced Adaptability and Growth--GenAI's integration is rising across sectors, especially engineering, lifted by its scalability and operational efficiency potential.  GenAI for Pharma and Insurance Transformation—From use cases like underwriting to drug development, GenAI is redefining operations in process-intensive industries like Pharma and Insurance.  Improved Trust and Ethical Considerations—Addressing concerns about data security, open-source models boost trust and credibility in GenAI usage.  GenAI-driven Differentiation—Transcending automation, GenAI enables optimized customer experiences and personalized product offerings, effecting brand differentiation.  Augmenting Human Input: Disproving fears of job displacement, GenAI enhances human capabilities by enhancing knowledge, thereby facilitating workforce upskilling.  GenAI sets the path for unprecedented efficiency, innovation, client-centricity, and human-centered value. Adopt GenAI to augment your business intelligence and bolster possibilities for growth amidst technological disruption.  Read the article here: https://lnkd.in/e99AK5Ww #GenAI #ArtificialIntelligence #Innovation #BusinessTransformation   

10

An interesting summary of the Gemini 1.5- technical report from Gradientflow. Report: https://lnkd.in/g_ne9gEy #llms #genai #generatieveai #capgemini #llm #opensourceai #capgeminiindia #ai #artificialintelligence #software  #leaderboard #benchmark #benchmarking #gemini #google

13

Ola Krutrim , India’s first AI unicorn, recently launched its LLM, Krutrim-7B-chat. This new model is trained in 10 Indian languages and is available on Databricks Marketplace. It can generate text in Hindi, English, Tamil, Telugu, Marathi, Malayalam, Gujarati, Bengali, and Kannada. This is the first of several India-specific models trained by Ola Krutrim. It has been trained on a massive dataset of Indic languages— “Krutrim delivers text infused with Indian sensibilities and cultural awareness.” Check out the model here. The announcement came to light at the Data Intelligence Day in Mumbai on May 10, 2024, by Gautam Bhargava, VP & Head of AI Engineering at Krutrim. “We’re excited to be at the forefront of building foundational LLMs trained on Indian languages to better serve our customers in India. We have been working very closely with the Databricks team to pre-train and finetune our foundational LLM,” said Bhargava. He added, “We are confident that Krutrim, our base model, will be able to provide high-quality responses in Indian languages and dialects.” According to Bhargava, Krutrim was trained using Databricks Mosaic AI Training’s GPU orchestration and system optimisations, enabling state-of-the-art generative AI solutions for customers globally. The model is now available on the Databricks Marketplace.

49

1 Comment

A preview or our upcoming article on Integrating #ResponsibleAI practices into #LLMOps (in collaboration with Dipta Chakraborty, Soumya Talukder, Dipojjwal Ghosh, Bhargav Mitra, Anindito De) While we see growing adoption of both LLMOps & Responsible AI practices in our #GenAI implementations, the discussions are often occurring in different communities with a big disconnect between the checklists and their downstream implementation / tooling. In this article, we aim to bring the two frameworks together with a unified approach to deploying #LLM use-cases in a scalable & responsible fashion. Extended Abstract. We see growing excitement about #LargeLanguageModels (LLMs) and their potential to disrupt enterprise use-cases. However, this exploration is mostly limited to Proof-of-Concepts (#POCs) at this stage. To truly scale Gen AI use-cases in the enterprise, there is a need to establish a scalable LLM platform with LLMOps capabilities and the right level of LLM #Governance. In this era, we see heightened emphasis are given to use AI component responsibly as it has been witnessed that LLMs can provide #toxic, biased, sexual and violent content if it’s not used responsibly. Sometimes it has been also seen that LLMs are providing #biased and misguided responses. To mitigate the issues, it is utmost important  that we use LLMs responsibly. In this article, we introduce LLMOps and deep dive into the relevant LLMOps #architecturalpatterns. We consider the different Responsible AI dimensions in terms of Data Quality, Data #Privacy, Reproducibility and #Explainability to enable a well governed LLM Platform. We highlight LLM specific issues and propose solution approaches to establish an integrated LLMOps platform governed by Responsible AI practices – leading to the establishment of a Gen AI #CoE delivering strategic enterprise use-cases at scale. For details of the architecture patterns referenced in the attached table, please refer to the below articles (on LinkedIn): 1. #GenerativeAI Architecture Patterns, https://lnkd.in/e2M6AS5S 2. #AIAgents Reference Architecture, https://lnkd.in/exsc8P3u 3. Responsible Generative AI #Design Patterns, https://lnkd.in/e-MCiuBM

304

17 Comments

AI of Databricks AI for #enterprises AI by Celebal Technologies To learn more about our offerings on the leading data intelligence platform, meet me @ #DAIS 2024. Vijay Balasubramaniam Nikhil Gupta Adam Hughes Itamar Ankorion Jayanta (Jay) Banerjee Udayan Saxena chAI - your AI partner of choice

47

Killing Creations over Killing Creativity Driving Enterprise Success with Gen/AI Why is BigTech so good at cranking out innovative AI and tech despite being large, friction-laden enterprises themselves just like FinServ giants? From the looks of it, this observation comes up in *every* CXO and Board forums in one way or another. A partial explanation is that the innovation center of gravity has moved from large institutions to individuals in the internet era where individuals have easy access to knowledge (arxiv?), platforms (Colab?), and collaborators (Github?). BigTech's DNA leverages this reality by allowing experiments and prototypes (on the employees' own time usually) and shark tank prototypes to bubble up, putting the firm's resources only behind the awesome (viable, valuable, and valent) ones. Enterprises take a much riskier approach (a delicious irony that!). They put bets (with real money) on ideas/prototypes (mostly bad) based on instincts. So enterprises buy efforts for prototypes (expensive) versus BigTechs that buy just the awesome hits! The terrible byproduct is that enterprises end-up co-opting their workforce's passion and creativity with a dramatic impact on engagement. Enterprises that hold hackathons, &c., are on the right track as they become BigTechs for a day, ... or week . . . or whatever! The real hack is to create a permanent safe platform for employees to build (prototypes) on their passions. The alternative is to continue like we do today: get behind a tiny number of ideas to prototype and spend oodles of time and money on rescuing and telling tall stories about mostly terrible prototypes! #ml #ai #GenAI4FS #inc81starch

37

13 Comments

As we quickly adopt the positive disruptions caused by #GenAI in the world of #dataanalytics, let's also understand the kind of impact AI & ML are having on the larger #moderndatastack. Join me, Atmesh M. and Manisha Jaiswal at Powai Social on May 2nd as we talk through the opportunities & challenges for the Developers, Architects & Business personas kipi.bi + Snowflake + Chalo + Hevo Data

39

In my view, a data product is a concept that involves structuring and managing your company’s data in a manner that promotes usage and actions. It’s akin to adopting a product-oriented approach to your datasets, viewing them as valuable assets: 1. It explicitly identifies the personas it caters to, with a strong emphasis on the end users! 2. It tackles business use cases pertinent to the selected domain. 3. It features a highly user-friendly application interface to cater to the typical needs of the personas. 4. It distinctly prompts actions rather than merely offering insights. 5. Finally, the data product should encompass a unified data layer that includes modelled data and accompanying metadata. It was great conversing with Brian T. O'Neill! Do listen to the podcast. #TGIF

49

4 Comments

Minimizing hallucinations in Large Language Models (LLMs) is crucial for dependable enterprise applications. Addressing inaccuracies in AI integration is key to upholding trust and operational efficiency. Analytics India Magazine shares early strategies to mitigate these issues for consistent, reliable outputs. Enterprise leaders must prioritize reducing hallucinations to fully harness AI's potential. Read more: https://lnkd.in/gsG-agk8

3

I am speaking at #DES2024. If you guys are interested please attend my session. It would be about how we can use the Embeddings for creating semantic search, custom #RAGs and how do we store them in #VectorDbs for faster Retrieval. I will be giving some practical examples of how we are using them in our products at #Bizom.

63

Did further deep-dive research with George Gilbert on Metadata & the next Intelligent Data Platform: 🔍 Data Management Evolution: Data management has shifted toward centralized consolidation, separating computational power from storage, leading to more efficient data handling. 🌐 AI-Powered Integration: AI and metadata integration face challenges in creating a coherent knowledge graph, but it streamlines complex processes like ingestion, transformation, and pipeline creation, ensuring consistency. 📊 Network Effects: Unifying metadata throughout the data lifecycle enhances its quality, delivering effective data integration and management. 🧩 360-Degree Views: Advanced entity resolution capabilities provide accurate 360-degree views of customer data, critical for maintaining quality and consistency. 💼 Stakeholder Empowerment: AI-driven metadata management supports diverse users, from engineers to analysts, helping them make informed decisions and automate tasks. 📈 Future Innovations: The continued integration of AI into metadata management drives innovation, aiming to build intelligent platforms, data products, and apps, empowering organizations to maximize the value of their data. 🎥 Check out the insights from Gaurav Pathak VP, Product Management AI & Metadata, at Informatica https://lnkd.in/gNzy_2QR Stay tuned as we evolve the reference architecture for the next Intelligent Data Platform together! #DataManagement #Metadata #AI #DataIntegration #DataPlatform #ETL #NoETL #PushDown #Cubes SiliconANGLE & theCUBE John Furrier David Vellante https://lnkd.in/gP6AM5Md

12

1 Comment

As organizations are preparing for AI driven data insights hoping a modern compsable architecture and data mesh approach is critical. In this article at The CDO TIMES (in the comments section you will find the link) I dive deep into data strategy, key terminology and the history of data mesh and composable data architecture. Towards the end you will find actionable steps on how you can get there and take your business intelligence platforms to the next level. How are you preparing to transform your business for advanced AI driven data insights? #datadriven #datastrategy #datamesh #composabledataarchitecture #cdoinsights #cdotimes

3

1 Comment

Hello Manish, Thank you, for “seeing us” and taking time to highlight this unfortunate market trend. The attention you are giving to this as a CEO, Founder & Chairmen of the Board means a lot, speaking on behalf of the entire Global CDO community! This same pattern unfolded when the role of Chief Information Officer (CIO) was introduced to the C-Suite, and overtime we witnessed reduction in CIO turn-over as organizational adoption normalized. There are many contributing factors for CDO turn-over as you so eloquently highlighted. The question we all should be asking is Why, and How do we fix it? As a two-time Chief Data & AI Officer, with a creditiable track record, I encourage/challenge all Chief People Officers, Chief Human Resource Officers, and members of the C-Suite to lean-in on this topic, and consider instituting After Action Reviews and Assessments (AAR’s) to better understand what organizational dynamics maybe contributing to this trend, and for those organizations embarking on CDO, CDAO, CIAO placement take note. Given the CDO’s critical role in driving business impact and value, I recommend organizations proactively engage with sitting CDO’s by creating an open dialogue to better understand, empower, and support these leaders to avoid contributing to this market trend. #ChangeAgents History will repeat itself if left unaddressed and we will see this same pattern unfold with current and future Chief AI Officers! .#ChiefPeopleOfficer,#ChiefHumanReaourceOfficer,#CDO,#CDAO, #ChiefAIOfficer, #CEO, #Corporateboards,#AI, #Csuite, #TakeAction, #AAR, #ExecutiveDevelopment

21

3 Comments

A great summary of my talk at #DES2024

22

This whitepaper is a must read for anyone thinking about the future of data. Raghu Ramakrishnan and Josh Caplan lay out Microsoft’s vision and the design principles behind OneLake & #Fabric.   I appreciate the open approach, based on a desire to: ✅ minimize data movement and eliminate multiple copies of data, and ✅ get a consolidated enterprise view – the “entire data estate”!   Are we “entering the golden age of analytics”? 💯 Here’s why: 1️⃣ Elastic cloud compute and storage, and 2️⃣ The best tools that can take advantage of it.   What does this mean for data?     That we can: 🚂 bring the most appropriate tools (compute engines) to bear on the task at hand (workloads) 💾 using data stored in open efficient accessible formats   So what are these workloads and engines? #Analytics – #SQL & BI tools like Tableau, Power BI, Qlik, Sigma, Trino Data integration engineering for #ETL or #ELT, using SQL, Python, Spark Data Streaming – KQL or Kafka with ksql and/or Flink AI/ML – R, #Python, #GenAI using open source models Statistical analysis using tools like SAS   Open Parquet-based table formats Delta Lake, Icerbeg, and Hudi dominate the open storage discussions. Microsoft (& Databricks) use Delta in Fabric while actively supporting the Apache Xtable project for interoperability with Iceberg and Hudi. It is worth noting that Snowflake's native format is proprietary but they now supports Iceberg Tables.   🔐 Security and governance are top of mind for enterprises and it only makes sense for this to be designed and built into the data layer from the ground up. The Purview integration with OneLake in Fabric addresses a big challenge for enterprises. If you are interested in the future of data & analytics this will be worth your time - better than any webinar, conference, or sales briefing. https://lnkd.in/gTQtczX9

31

1 Comment

⏰ Great interview: Sridhar Ramaswamy, CEO, Snowflake joins #theCUBE hosts David Vellante for Data Cloud Summit 2024. ✳️ They are wearing up for the AI war. The question is: will Snowflake current strategy make them successful with AI? Or will they need to reinvent their straregy to make it more appealing to customers? #data #ai #genai #ainews #dataapplications #datastorage

1