Astronomer’s Post

Modern AI architecture is revolutionizing how data is managed and utilized in baseball. In this episode, Alexander Booth of the Texas Rangers Baseball Club explains the importance of having a defined AI architecture, whether it's using SageMaker, BigQuery ML, Azure ML or Databricks. Alexander highlights how Airflow serves as the main orchestration piece, enabling efficient data routing and prediction modeling. Discover how the Texas Rangers use dynamic tasks and conditional logic in Airflow to handle vast amounts of data, ensuring accurate and timely insights for better decision-making. To learn more, follow the link in the comments. #AI #Automation #Airflow #MachineLearning

Efficient data orchestration is critical in today's data-driven sports environment. In this episode, Alexander Booth discusses how Airflow has been pivotal for the Texas Rangers Baseball Club in organizing and managing their complex data systems. From tracking player performance to integrating weather data, Airflow helps streamline vast datasets into actionable insights. Explore how the Rangers leverage this technology to stay ahead in the game and ensure their players receive timely and accurate data. To learn more, follow the link in the comments to the new episode. #AI #Automation #Airflow #MachineLearning

Using #Databricks in combination with #ApacheAirflow is becoming more common, with many #datateams running heavy workloads (training #ML models, #datatransformations, and #dataanalysis). Airflow’s functionality makes it easy to orchestrate #DatabricksJobs, while integrating with other tools in your #datastack. Join us for this upcoming webinar where we'll be covering how to: 🪄 Orchestrate Databricks Jobs with the Airflow Databricks Provider 🧱 Monitor Databricks Jobs directly from #Airflow ✍️ Write tasks that run Databricks Jobs using the familiar DAG API ↪️ Send repair requests to Databricks when tasks fail Save your spot 👉 https://bit.ly/4cuHPLZ

Join Astronomer for the 10th anniversary of #ApacheAirflow at this year's #AirflowSummit in San Francisco! 🌁 This event will be packed with practical and insightful sessions around everything #dataorchestration, #GenAI, #ETLOrchestration, the future of #Airflow and more — don't miss out on hearing from expert speakers + committers, and networking with the #AirflowCommunity 🎉 Check out the sessions and speakers here, and use discount code 30DISC_ASTRONOMER for 30% off your ticket! 🎟️ https://bit.ly/4eYAZjg

Get a behind the scenes look at how NerdWallet cut #Snowflake costs in half by upgrading to #Airflow2 during this talk at #AirflowSummit. Hear how their multi-tenant #Airflow setup faced challenges like slow DAG processing, resulting in underutilized Snowflake warehouses and elevated costs 📈 If you're looking for buy-in for migration efforts, this session around unlocking the scheduling capabilities of Airflow 2 is for you! Have yet to register? Use code 30DISC_ASTRONOMER and save 30% on your ticket 🎟️ https://bit.ly/45UKYC8 #AirflowSummit2024 #dataconference #Airflow #DataOrchestration #DataPipelines #NerdWallet

In today’s complex #dataecosystem, it can be challenging to unlock the true power of #GenerativeAI. From data quality issues to #APIoutages, adopting AI requires complex and resource-intensive tasks. Astronomer's Sr. Dir of Eng, Kaxil Naik shares his thoughts in NextTech Today on how to overcome common #datachallenges and his advice for achieving maximum #ROI in #AIadoption with modern #dataorchestration:

With 165+ million downloads in 2023, Apache Airflow is the gold standard for data orchestration. See why it's trusted by data pros worldwide 🌏. Dive into our #Airflow101 learning path—your perfect guide to mastering core concepts, the #Airflow UI, and best practices for creating your first data pipeline. 🌟 🎓 Start learning 👉 https://lnkd.in/gQfAf3nE #dataengineering #dataorchestration #datapipelines

Explore how Astronomer transforms #Airflow management with our interactive product tour! From orchestrating tasks and data workflows to ensuring secure 🔒 connections and complete pipeline #observability, we simplify every step and help you understand performance metrics, manage costs, and ensure deployment health. Start the tour now 💻 https://bit.ly/4eHPyYD #DataWorkflows #DataPipelines #DataOps #DataEngineering #Astro

🛠️ Struggling with failed tasks or scheduling issues in #Airflow? Learn best practices for configuring automatic task retries, manually rerunning tasks or #DAGs, triggering historical DAG runs, and mastering catchup and backfill. Check out our detailed guide 👉 https://bit.ly/3VInHi2 #DataEngineering #Automation #DataPipeline

🏆⚾ Step up to the major leagues of data orchestration at Airflow Summit! Don't miss the keynote featuring the 2023 World Series Champions, Texas Rangers Baseball Club. Discover firsthand how they clinched victory with Apache Airflow powering their strategic plays. 🎤 Hear from industry leaders Alexander Booth and Oliver Dykstra as they share their winning data pipeline playbook that drove agile, data-driven decisions both on and off the field. Don’t let this opportunity slide by! Register with code 30DISC_ASTRONOMER and save 30% on your tickets. https://bit.ly/3VV487I #AirflowSummit2024 #dataconference #Airflow #DataOrchestration #DataPipelines #TexasRangers