The era of continuous actionable intelligence
This is a guest blogpost by Seamus Keating, CEO KX
The frontiers of data management technologies are being extended as organisations across every industry sector seek to extract greater insights from real-time data, whether to improve operational performance, outpace their competition or change business models entirely.
This pace of change is being forced through by two key factors, the exponential growth in data volumes and the speed at which that data is coming into organisations, in particular real-time data from devices and sensors often at the edge of networks. Industry analyst firm IDC estimates that enterprise-related sensor data is growing at 40% per annum and will soon surpass all other data types, including entertainment data.
Our own research tells us that organisations across all industry sectors recognise the need to extract more insight and intelligence from their real-time data. 90% of organisations say that they plan to increase investment in real-time or streaming analytics technologies over the next three years. Add to this a growing demand post-Covid-19 for greater automated decision-making as part of digital transformation initiatives and it’s not hard to see why analyst firm Gartner is forecasting that by 2022 most business systems will feature real-time data capabilities.
From trading floor to trackside
When it comes to understanding the transformative potential of real-time analytics, much can be learnt from the experience of companies in the Financial Services sector. Banks, hedge funds, exchanges and regulators have long relied on technologies that enable sub-second decision making for a range of use cases from optimising trades to identifying and mitigating crime.
While the datasets may be different, any scenario where big data meets fast data presents an opportunity for organisations to benefit from the introduction or enhanced application of real-time analytics. Take automotive; from the processing of edge data from sensors in autonomous vehicles to analysing trackside telemetry and aerodynamics in wind tunnels in motorsports, real-time analytics is a game changer for both enabling new technologies and services and significantly enhancing existing processes.
There are a number of similar use cases in industries as diverse as marketing, manufacturing, utilities and telecommunications. As the volume, variety and velocity of data continues to increase, so too then the need for technologies that can unlock the value of data in real-time, using the context of historical time-series data to capitalize on perishable business moments.
Partners in time
This demand for ultra-high-performance analytics is giving rise to a new ‘fast data’ layer within the wider data management and analytics architecture. In this layer, best in class real-time decisioning engines can augment the storage and batch analytical capabilities of partner technologies, including those offered by the hyperscale cloud vendors and sector-focused platform providers. Whether deployed on-cloud, on prem or in a hybrid model, these fast-data solutions allow companies to add deep and wide historical data as context to inform their real-time decision making. They also tend to be simple to deploy and manage, and are therefore appealing to businesses struggling to make legacy data management and analytics systems fit for purpose when faced with the realities of the ever-growing data landscape.
Continuous actionable intelligence
Ultimately, by combining ultra-high performance analytics with time-series data, companies can build a model of Continuous Actionable Intelligence (CAI), where AI and machine learning models are applied to real-time and historical data to deliver greater automation of business processes and decisions.
Alongside significantly improving operational efficiency, CAI offers the promise of both solving hitherto unseen business problems and unearthing new opportunities providing companies have the right people with the right skills to build and promote the data models.
Again, the findings from our research suggest that across industries, many organisations are struggling with having access to the right people with experience in DataOps and machine learning. Get it right, however, and companies can experience the genuinely transformative results that these technologies offer.