The document summarizes an upcoming virtual meetup on integrating Snowflake and Apache Kafka with Mulesoft. The meetup agenda includes an overview of event-driven architecture, introductions to Snowflake and its benefits as a cloud-based data warehouse, an introduction to Apache Kafka as an open-source stream processing platform, and steps for integrating Snowflake and Kafka with Mulesoft. The document provides details on Snowflake and Kafka APIs and applications, and guidelines for participants to ask questions and provide feedback.
2. 2
Guidelines for our Virtual Meetup
1. You can write down your questions in the chat during the session.
2. Once we are done with the day, don’t forget to provide your
valuable feedback.
3. 3
●About the organizers:
○ Suresh Rathore
○ Lalit Panwar
○ Ajay Rana
Introductions
A SHOW OF HANDS:
Who is new to this Meetup?
●About the sponsor:
○ MuleSoft
5. 5
● Event driven architecture overview
● What is Snowflake ?
● Feature and benefits of Snowflake
● Snowflake Integration with Mulesoft
● Apache Kafka introduction
● Integration with Mulesoft
● Lets see how they fits together in Mulesoft.
Agenda
8. 8
● It’s a 100% cloud-based database and data
warehouse
● Benefits of Snowflake:
○ There is no hardware (virtual or
physical) to select, install, configure, or
manage.
○ There is virtually no software to install,
configure, or manage.
○ Ongoing maintenance, management,
upgrades, and tuning are handled by
Snowflake.
Introduction
11. Integrate Snowflake with mulesoft
● One can sign up on Snowflake for trial using below link
○ https://trial.snowflake.com/
● Select Snowflake edition as Enterprise and Cloud Provider like Amazon Web Services
(AWS).
11
12. APIs
● For activation an email will sent to
user account.
● After successfully logged in, User can
create their own databases and
schemas.
● In Anypoint studio search Snowflake
in exchange and add Snowflake
Connector.
12
15. Introduction
● Its an open-source stream-processing software
platform
● It can handle about trillions of data events in a day.
● Apache Kafka creates real-time streaming apps and
data pipelines.
● It is scalable, fault-tolerant, and incredibly fast.
15
16. Core API’s
16
● Producer API: This API allows/permits an
application to publish streams of records to
one or more topics.
● Consumer API: This API allows an
application to subscribe one or more topics
and process the stream of records produced
to them..
17. Core API’s
17
● Streams API: This API allows an application to effectively transform the input
streams to the output streams. It permits an application to act as a stream
processor which consumes an input stream from one or more topics, and produce
an output stream to one or more output topics.
● Connector API: This API executes the reusable producer and consumer APIs
with the existing data systems or applications.
21. Steps for Apache Kafka
1. Download Apache Kafka binary file from https://kafka.apache.org/downloads
2. Set environment variable for kafka in path as : “Kafka folder>bin>windows”
3. Check “kafka--” in command prompt to check if its installed or not.
4. Set Kafka-logs path in ZooKeeper and Kafka config files respectively.
5. Following commands are used to start and create topics in Kafka :
○ zookeeper-server-start.bat configzookeeper.properties
○ kafka-server-start.bat configserver.properties
○ kafka-topics --zookeeper localhost:2181 --topic myfirst --create --partitions 3 --replication-factor
1
○ kafka-topics.bat --zookeeper localhost:2181 --list
○ kafka-topics.bat --zookeeper localhost:2181 --topic myfirst --delete
21
22. Integration steps with Mulesoft
● In Studio, Click the Exchange (X) icon in the task
bar.
● Select All assets and search for Apache Kafka.
● Select the connector and click Add to project.
22
23. Why Kafka uses Zookeeper ?
● ZooKeeper is used in distributed systems for service synchronization
● In Apache Kafka, ZooKeeper is primarily used to track the status of nodes in the Kafka cluster
and maintain a list of Kafka topics and messages.
23
29. 29
● Share:
○ Tweet your pictures or posts using the hashtag #MuleSoftMeetups
○ Invite your network to join: https://meetups.mulesoft.com/chandigarh/
● Feedback:
○ Fill out the survey feedback and suggest topics for upcoming events
○ Contact MuleSoft at meetups@mulesoft.com for ways to improve the program
It always seems impossible until it is done - NELSON MANDELA
What’s next?