Apache StreamPipes

Apache StreamPipes

Non-profit Organizations

An open-source self-service Industrial IoT toolbox to connect, analyze and explore IIoT data streams

About us

Apache StreamPipes makes industrial data analytics easy! StreamPipes is an end-to-end toolbox for the industrial IoT. It comes with a rich graphical user interface targeted at non-technical users and provides the following features: * Quickly connect >20 industrial protocols such as OPC-UA, PLCs, MQTT, REST, Pulsar, Kafka and others. * Create data harmonization and analytics pipelines using > 100 algorithms and data sinks to forward data to third-party systems. * Use the data explorer to visually explore historical data with many widgets tailored for time-series data. * A live dashboard to display real-time data from data sources and pipelines, e.g., for shopfloor monitoring. StreamPipes is highly extensible and includes a Java SDK to create new pipeline elements and adapters. Python support is available in an early development stage - stay tuned! Pipeline elements are standalone microservices that can run anywhere - centrally on your server or close at the edge. You want to employ your own machine learning model on live data? Just write your own data processor and make it reusable as a pipeline element. Besides that, StreamPipes includes features for production deployments: * Assign resources such as pipelines, data streams and dashboards to assets for better organization * Monitoring & metrics of pipelines and adapters * Built-in user and access rights management * Export and import resources

Website
https://streampipes.apache.org
Industry
Non-profit Organizations
Company size
11-50 employees
Type
Nonprofit
Specialties
IIoT, IoT, analytics, stream-analytics, stream-processing, self-service, and opensource

Updates

  • View organization page for Apache StreamPipes, graphic

    103 followers

    We highly recommend Alessandro Albani's article on a vulnerability we successfully resolved in our latest release, version 0.95.0. While it's never easy to be highlighted as a negative example, we're grateful for the opportunity this has provided to strengthen StreamPipes, making it more robust and secure. In the end, this experience underscores one of the greatest strengths of open source: fostering collaboration among diverse experts, leading to improvements that wouldn't be possible otherwise. Thank you very much for the great collaboration, Alessandro! #OpenSource #CyberSecurity #Collaboration #SoftwareDevelopment #StreamPipes #ASFWay

    View organization page for Yarix, graphic

    6,495 followers

    🚨 Avviso Critico di Sicurezza per gli Utenti di Apache StreamPipes! Lo sapevi che un algoritmo debole per la generazione dei token utilizzati per il recupero delle password potrebbe consentire agli attaccanti di impossessarsi facilmente degli account di altri utenti?   Per fortuna è possibile: 🔍 Rilevare questa vulnerabilità 🛠️ Sfruttarla a scopo di test 🛡️ Proteggere i tuoi sistemi   Come? Te lo spieghiamo sul nuovo articolo scritto da Alessandro Albani sul nostro blog, YLabs, in cui sveliamo i dettagli di CVE-2024-29868: https://lnkd.in/dTJUXefn   --------------------------------- 🚨 Critical Security Warning for Apache StreamPipes Users! Did you know that a weak algorithm used to generate tokens for password recovery could allow attackers to easily take over other users' accounts? Thankfully, it is possible to: 🔍 Detect this vulnerability 🛠️ Exploit it for testing purposes 🛡️ Protect your systems How? We explain in the new article written by Alessandro Albani on our blog, YLabs, in which we reveal the details of CVE-2024-2986

    • No alternative text description for this image
  • View organization page for Apache StreamPipes, graphic

    103 followers

    We are thrilled to announce the immediate availability of Apache StreamPipes 0.95.0! 🎉 This release addresses over 200 issues, introducing numerous improvements and bug fixes to enhance the user experience significantly. 🚀 Here are some highlights of the latest release: 🔌 Enhanced Integration with Apache PLC4X StreamPipes 0.95.0 now features integration with PLC4X using protocol metadata, expanding the range of adapters for industrial protocols such as Allen Bradley, BACNet, Beckhoff TwinCat ADS, and Modbus. These new adapters also support advanced configuration options at both protocol and transport levels. 💬 New Data Processors and Sinks Version 0.95.0 introduces several new pipeline elements, including a Datetime from String processor and a Microsoft Teams sink. 🐹 Introducing the Go Client A notable new feature is the Go client for Apache StreamPipes, available as an early preview. This client enables interaction with the StreamPipes API from Go applications. We are actively working to expand the feature set supported by the Go client. 📊 Data Explorer Enhancements We've harmonized the technical framework of all data explorer widgets in recent releases. StreamPipes 0.95.0 now utilizes the powerful Apache ECharts library for rendering widgets, enhancing the look and feel while laying the groundwork for feature extensions such as advanced configuration options for data explorer widgets, including axis configuration and data zoom. 👍 Improved User Experience Nearly every aspect of the StreamPipes user interface has been refined. Modifying existing adapters is now more intuitive, several bugs in the pipeline editor have been resolved, and the installation process for extensions has been enhanced for better usability and performance. The Apache StreamPipes community has already begun working on the next release, 0.97.0, since April. Stay tuned for exciting new features that we'll be announcing soon! 🌟 And as usual, the new release is available for download on our website, and also find the blog post with the most important feature highlights! The Apache Software Foundation #IIoT #OpenSource #Analytics

    • No alternative text description for this image
  • View organization page for Apache StreamPipes, graphic

    103 followers

    StreamPipes is part of the OSPP 2024 🧑💻 We are pleased to announce that we are participating in this year's OSPP Challenge for the second year in a row. The Open Source Promotion Plan (OSPP) is a summer program organized by the Institute of Software Chinese Academy of Sciences and aims to encourage college students to actively participate in the maintenance and development of open source software. We've submitted two exciting projects and can't wait for the students to get involved: 1️⃣ Enriching Go Client Features For the last few months we have been working on a client that supports interaction with StreamPipes via Golang. Within this OSPP project we would like to enrich the functionality, implement an end-to-end testing framework and improve the documentation. 2️⃣ Integrating Apache IoTDB with Apache StreamPipes StreamPipes already provides options to read from and write to an Apache IoTDB. In this project, we want to explore a deeper and more advanced integration of StreamPipes with IoTDB, e.g. taking into account #TSFiles. After a successful collaboration last year, we're excited to see the results of this year's OSPP submissions. Interested in participating in our OSPP challenges? Check out the project details and application information in the comments below! 👇 #ospp #opensource #community #studentengagement #golang #iotdb #iiot

  • View organization page for Apache StreamPipes, graphic

    103 followers

    📢 New Blog Post Alert! 📢 Our latest article explores practical applications of StreamPipes in machine learning workflows, focusing on seamless IIoT data extraction and model interoperability via ONNX. Discover how StreamPipes simplifies IIoT data extraction with our Python client, and learn how to apply your ML model to streaming data effortlessly. Join us as we delve into an anomaly detection scenario: detecting leakages in a water pipe system. Read more: https://lnkd.in/eG9sr9h2 #ML #ONNX #AnomalyDetection #IIoT

    Anomaly Detection with StreamPipes Functions in Python and ONNX | Apache StreamPipes

    Anomaly Detection with StreamPipes Functions in Python and ONNX | Apache StreamPipes

    streampipes.apache.org

  • Apache StreamPipes reposted this

    View organization page for Apache StreamPipes, graphic

    103 followers

    🚀 Exciting News! StreamPipes 0.93.0 is here with major enhancements and bug fixes, fortifying the platform's technical foundations. Dive into the key updates: 1. Adapter Updates: - Seamlessly edit running adapters, eliminating the need to stop and recreate. 2. OPC-UA Adapter Improvements: - Intuitive and asynchronous browsing interface for enhanced user experience. 3. S7 PLC Adapter Upgrade: - Now supports array-based data with an advanced register input mode. 4. New Processor: Numeric Rounding: - Perform rounding operations on numeric values within an event stream. 5. Deployment Updates: - Kubernetes deployment improvements with probes for enhanced health monitoring. - Monitoring integration (Grafana & Prometheus) for better system insights. - New deployment mode for first-time users for a quick start. 6. StreamPipes Python Support: - Now supports Python 3.11 and other protocols for interacting with live data. 7. Administrative & Development Features: - Configurable email templates for customized notifications. - Removal of legacy resources and upcoming elimination of Consul dependency. 8. Technical Foundation & Website Redesign: - Enable migrations for pipeline elements for easier updates. - StreamPipes website has a fresh new look! For a more detailed introduction and thorough release notes, please check the comments section. Your feedback is invaluable to us! Let us know your thoughts and stay tuned for more updates 💡🗞️ #StreamPipes #IIoT #ReleaseAnnouncement #ApacheWay #opcua #plc

  • Apache StreamPipes reposted this

    View organization page for Apache StreamPipes, graphic

    103 followers

    🚀 Exciting News! StreamPipes 0.93.0 is here with major enhancements and bug fixes, fortifying the platform's technical foundations. Dive into the key updates: 1. Adapter Updates: - Seamlessly edit running adapters, eliminating the need to stop and recreate. 2. OPC-UA Adapter Improvements: - Intuitive and asynchronous browsing interface for enhanced user experience. 3. S7 PLC Adapter Upgrade: - Now supports array-based data with an advanced register input mode. 4. New Processor: Numeric Rounding: - Perform rounding operations on numeric values within an event stream. 5. Deployment Updates: - Kubernetes deployment improvements with probes for enhanced health monitoring. - Monitoring integration (Grafana & Prometheus) for better system insights. - New deployment mode for first-time users for a quick start. 6. StreamPipes Python Support: - Now supports Python 3.11 and other protocols for interacting with live data. 7. Administrative & Development Features: - Configurable email templates for customized notifications. - Removal of legacy resources and upcoming elimination of Consul dependency. 8. Technical Foundation & Website Redesign: - Enable migrations for pipeline elements for easier updates. - StreamPipes website has a fresh new look! For a more detailed introduction and thorough release notes, please check the comments section. Your feedback is invaluable to us! Let us know your thoughts and stay tuned for more updates 💡🗞️ #StreamPipes #IIoT #ReleaseAnnouncement #ApacheWay #opcua #plc

  • View organization page for Apache StreamPipes, graphic

    103 followers

    🚀 Exciting News! StreamPipes 0.93.0 is here with major enhancements and bug fixes, fortifying the platform's technical foundations. Dive into the key updates: 1. Adapter Updates: - Seamlessly edit running adapters, eliminating the need to stop and recreate. 2. OPC-UA Adapter Improvements: - Intuitive and asynchronous browsing interface for enhanced user experience. 3. S7 PLC Adapter Upgrade: - Now supports array-based data with an advanced register input mode. 4. New Processor: Numeric Rounding: - Perform rounding operations on numeric values within an event stream. 5. Deployment Updates: - Kubernetes deployment improvements with probes for enhanced health monitoring. - Monitoring integration (Grafana & Prometheus) for better system insights. - New deployment mode for first-time users for a quick start. 6. StreamPipes Python Support: - Now supports Python 3.11 and other protocols for interacting with live data. 7. Administrative & Development Features: - Configurable email templates for customized notifications. - Removal of legacy resources and upcoming elimination of Consul dependency. 8. Technical Foundation & Website Redesign: - Enable migrations for pipeline elements for easier updates. - StreamPipes website has a fresh new look! For a more detailed introduction and thorough release notes, please check the comments section. Your feedback is invaluable to us! Let us know your thoughts and stay tuned for more updates 💡🗞️ #StreamPipes #IIoT #ReleaseAnnouncement #ApacheWay #opcua #plc

  • View organization page for Apache StreamPipes, graphic

    103 followers

    🔊 Dear StreamPipes fellows, We owe you a sincere apology for our recent silence, and we truly appreciate your patience. 🙏 The reason behind our temporary quietness? We've been preparing for a release that promises to bring plenty of enhancements, bug fixes, and exciting new features to Apache StreamPipes! We're thrilled about the progress we've made and can't wait to share it with you. Stay tuned for the reveal! 🎉✨ #StreamPipes #OpenSource #IIoT #ReleasePrep #ApacheWay #ASF

  • View organization page for Apache StreamPipes, graphic

    103 followers

    🚀 Feature Preview! 🚀 Since our last release in June, the Apache StreamPipes community has worked on several exciting features which improve user experience and maintenance of IIoT data analytics applications. A very nice upcoming feature is monitoring support for #Prometheus. The endpoint is tightly integrated with our existing health monitoring endpoints. With this feature, it will be much easier to monitor pipeline and adapter health from external applications and to get notifications in case of system failures. 💡📊 Stay tuned for the next release! ✨📆 #ApacheStreamPipes #IIoT #DataAnalytics

    • No alternative text description for this image

Similar pages