W-JAX Keynote - Big Data and Corporate Evolution

What the Internet of Things should learn from the life sciences. About the utility of open data, ontologies and public repositories as routinely used in the academic life science, but rarely in the IoT.

- A biologist is interested in large, unstructured IoT data to gain insights from connections between different data points, similar to how biologists study connections between biological entities. - Currently the IoT lacks common data formats, ontologies to provide context to things, and data repositories, limiting analytical flexibility and insights that can be gained. - Biologists overcame similar problems by developing standards like gene ontologies, metadata requirements, and public data repositories, enabling knowledge inference from large, diverse datasets. - Key concepts from biology that could help the IoT include developing ontologies to define thing functions, processes, and localizations in order to organize knowledge and enable inferencing across the large, diverse data generated by the Io

This document discusses how to analyze large datasets using Hadoop and BigInsights. It describes how IBM's Watson uses Hadoop to distribute its workload and load information into memory from sources like 200 million pages of text, CRM data, POS data, and social media to provide distilled insights. The document provides two use case examples of how energy companies and global media firms could use big data analytics to analyze weather data and identify unauthorized streaming content.

Big Data 101 provides an overview of big data concepts. It defines big data as data that is too large to fit into a typical database or spreadsheet due to its volume, variety and velocity. It discusses how data is accumulating rapidly from various sources and the challenges of storing and processing all this data. It also introduces common big data techniques like MapReduce and how they can be used to extract insights from large, unstructured data sets.

This document provides an overview of a data science training module that introduces students to key concepts like big data, data science components, types of data scientists, and use cases. The module covers topics such as big data scenarios, challenges, Hadoop, R programming, and machine learning. Students will learn how to analyze real-world datasets and implement machine learning algorithms in both Hadoop and R. The goal is for students to understand how to use tools and techniques to extract insights from large datasets.

In today’s interconnected real world, social and informational entities are interconnected, forming gigantic, interconnected, integrated social and information networks. By structuring these data objects into multiple types, such networks become semi-structured heterogeneous social and information networks. Most real world applications that handle big data, including interconnected social media and social networks, medical information systems, online e-commerce systems, or database systems, can be structured into typed, heterogeneous social and information networks. For example, in a medical care network, objects of multiple types, such as patients, doctors, diseases, medication, and links such as visits, diagnosis, and treatments are intertwined together, providing rich information and forming heterogeneous information networks. Effective analysis of large-scale heterogeneous social and information networks poses an interesting but critical challenge. In this talk, we present a set of data mining scenarios in heterogeneous social and information networks and show that mining typed, heterogeneous networks is a new and promising research frontier in data mining research. However, such mining may raise some serious challenging problems on scalability computation. We identify a set of problems on scalable computation and calls for serious studies on such problems. This includes how to efficiently computation for (1) meta path-based similarity search, (2) rank-based clustering, (3) rank-based classification, (4) meta path-based link/relationship prediction, and (5) topical hierarchies from heterogeneous information networks. We introduce some recent efforts, discuss the trade-offs between query-independent pre-computation vs. query-dependent online computation, and point out some promising research directions.

In this presentation, Wes Eldridge will provide a general overview on data science. The talk will cover a variety of topics, Wes will start with the dirty history of the field which will help add context. After learning about the history of data and data science Wes will discuss the common roles a data scientist holds in business and organizations. Next, he will talk about how to use data in your organization and products. Finally, he'll cover some tools to help you get started in data science. After the presentation, Wes will stick around for Q/A and data discussion.

This document discusses big data and defines it using the four Vs: volume, velocity, variety, and veracity. It states that big data is characterized by extremely large data sets that are difficult to process using traditional data processing applications. Specifically, it provides examples showing that big data is generated in huge volumes (petabytes or exabytes) at very fast rates, comes in many different forms (structured, unstructured, sensor data), and can be unreliable. The document also notes that while big data problems challenge existing technologies and algorithms, many analytics projects currently labeled as "big data" may not truly qualify. It concludes by mentioning some big data technologies like Hadoop that provide improved computing capabilities for processing large and diverse datasets.

This document provides an overview of big data including: - Defining big data as large and complex data sets that cannot be managed with traditional database tools. - Describing the four V's of big data: volume, velocity, variety and veracity. - Providing examples of how big data is being used including by companies to manage legal spend, for litigation strategy and jury selection, and potential uses for law firms like predictive analysis and improving business development.

Data-driven enterprise requires intelligent, sentient, and resilient software systems to address information processing structures to deal with rapid  fluctuations in resource demand and availability.   Communication, Collaboration and Commerce workflows at the speed of light demand always-on anti-fragile systems   Both autonomic computing and neural networks provide a next generation set of technologies to meet the needs of the data-driven enterprise at the speed of light Crypto-Security and New Digital Asset Life-cycle Managent assures the Asset’s Confidentiality, Integrity, Availability providing Privacy & Protection of Individual Rights

Cognitive Computing: How Artificial Intelligence and Machine Learning is transforming Data into Wisdom

The document discusses emerging technologies including quantum computing, artificial intelligence, machine learning, blockchains, the internet of things, cloud computing, edge computing, and data analytics. While these technologies show promise, the document cautions against hype and notes that many technologies do not yet work as described or have practical applications. The author offers experience supporting genomics and building IT infrastructure and is available for consulting work.

Rao Mikkilineni discusses the emergence of cognitive computing models and a new cognitive infrastructure. He argues that increasing data volumes and the need for real-time insights are driving the need for intelligent, sentient, and resilient systems. The new cognitive infrastructure will include a cognitive and infrastructure agnostic control overlay, composable services, and cognitive deep learning integration. It will enable a post-hypervisor cognitive computing era with intelligent, distributed systems.

This document provides a charter and roadmap for a Computing, Data, and Informatics Working Group. It discusses their vision of enabling data, computing, and identity services at unlimited scale. It highlights how information technology has been critical but also a source of tension in large projects like the Human Genome Project. The document outlines current enabling technologies like machine learning, blockchain, and DevOps practices. It identifies key challenge areas the working group will focus on, including identity and authorization, information security and privacy, and issues around data storage in multi-cloud environments. The working group members are then listed.

Knowing what’s happening in your enterprise right now can mark the difference between success and failure. The key is to have a rich view of activity, such that analysts and others can explore in a fully multidimensional fashion. Benefiting from such a detailed perspective can help professionals identify the exact nature of problems or opportunities, thus enabling precise actions that make a difference quickly. Register for this episode of The Briefing Room to hear veteran Analyst Mark Madsen of Third Nature explain how a nexus of innovations for analyzing network traffic can help companies stay on top of their game. He’ll be briefed by Erik Giesa of ExtraHop, who will showcase his company’s stream analytics technology for wire data, which provides real-time, multidimensional views of network traffic. He’ll share success stories of how ExtraHop has solved otherwise intractable problems and enabled a new level of root-cause analysis.

Big data offers significant opportunities for businesses but few have effectively exploited it due to challenges in dealing with the technical and management aspects of big data. An integrated enterprise platform is needed to deploy and use big data effectively alongside existing business processes and information tools. Big data should be introduced incrementally for maximum benefit.

Presentation that I delivered at "Accelerate AI, Europe 2018" in London on Sept 19, 2018. My focus is on socio-cultural perspective as well as proving information about various tools, vendors and partners available to help companies get started using AI.

The document discusses how corporations can become more innovative by adopting post-bureaucratic models that allow for emergent and networked behaviors. It argues that intentionally enabling emergence through open source practices, low barriers to participation, and generative systems can produce unanticipated innovation from broad audiences. The key is impedance matching organizations to the decentralized world through policies, architectures, and cultures that enhance long-tail contribution from self-organizing groups.

Presented at Velocity Conference in September in New York City. The information age is replacing the industrial age and corporations are going through changes as big as their original adoption of bureaucracy as an organizing principle. This talk describes the notion of "Intentional Emergence" in a corporate setting. The idea that managers and technologists should create conditions for emergent outcomes rather than always focusing on the outcome itself. DevOps and other mechanisms for a more dynamic IT culture are important to this end.

This document discusses how organizations can transition from traditional bureaucratic models to more emergent, networked models that are better suited for the information age. It notes that fully centralized planning is not feasible for complex systems, and that allowing for more decentralized, emergent processes can enhance flexibility and growth. The key aspects discussed include: - Adopting more permeable boundaries and "post-bureaucratic" internal characteristics at the organizational edge - Distinguishing between the planned core and more emergent, networked surroundings/edge - Making choices that increase a system's "generativity" through openness, low barriers to participation, and community involvement - Impedance matching the organization with the decentralized,

This document discusses how corporations can leverage digital technologies to enhance their intelligence. It suggests that a corporation's intelligence (IQcorp) depends on factors like fluid intelligence (Gf), crystallized intelligence (Gc), quantitative reasoning (Gq), memory (Gsm/Glr), processing speed (Gs), and decision speed (Gt). While humans currently excel in areas like fluid intelligence, digital technologies can augment corporations in other factors like crystallized intelligence, memory, and processing speed through techniques such as analytics, machine learning, and collaborative workflows. The document argues that through a symbiotic human-machine relationship, corporations can develop super-human levels of intelligence.

The document discusses refactoring, emergent design, and evolutionary architecture. It begins with an overview of refactoring, defining it as restructuring software by applying behavior-preserving transformations to improve code qualities like simplicity and modifiability without changing external behavior. The goal of refactoring is to minimize complexity and duplication through iterative application of refactorings. Scaling involves techniques like emergent design, managing technical debt, and evolutionary architecture. The document is divided into three parts that cover refactoring fundamentals, refactoring at scale, and special refactoring topics.

The document discusses several key points about cyber warfare and coding: 1. Coding is a form of maneuver in cyber warfare, as code can be changed and adapted quickly to respond to new threats or vulnerabilities. Technological agility will be important in cyber warfare where there are no compensating physical assets. 2. Cyber situational awareness is different than traditional battlespace awareness, but they should still be related. Integrating cyber and physical target data will be important for coordination. 3. Culture is important in cyber operations and cannot be done in isolation. Participating in communities of practice and using open source tools and techniques will be necessary to learn from others and adapt quickly to changes.

A peek into the next level of big data analytics. Artificial Intelligence and machine learning combined with big-data.

Big data is large and complex data that exceeds the processing capacity of conventional database systems. It is characterized by high volume, velocity, and variety of data. An enterprise can leverage big data through an analytical use to gain new insights, or through enabling new data-driven products and services. An analogy compares an enterprise's big data architecture to a sugar cane factory that acquires, organizes, analyzes, and generates business intelligence from big data sources to create value for the organization. NoSQL databases are complementary to rather than replacements for relational databases in big data solutions.

Keynote at Web Intelligence 2017: http://webintelligence2017.com/program/keynotes/ Video: https://youtu.be/EIbhcqakgvA Paper: http://knoesis.org/node/2698 Abstract: While Bill Gates, Stephen Hawking, Elon Musk, Peter Thiel, and others engage in OpenAI discussions of whether or not AI, robots, and machines will replace humans, proponents of human-centric computing continue to extend work in which humans and machine partner in contextualized and personalized processing of multimodal data to derive actionable information. In this talk, we discuss how maturing towards the emerging paradigms of semantic computing (SC), cognitive computing (CC), and perceptual computing (PC) provides a continuum through which to exploit the ever-increasing and growing diversity of data that could enhance people’s daily lives. SC and CC sift through raw data to personalize it according to context and individual users, creating abstractions that move the data closer to what humans can readily understand and apply in decision-making. PC, which interacts with the surrounding environment to collect data that is relevant and useful in understanding the outside world, is characterized by interpretative and exploratory activities that are supported by the use of prior/background knowledge. Using the examples of personalized digital health and a smart city, we will demonstrate how the trio of these computing paradigms form complementary capabilities that will enable the development of the next generation of intelligent systems. For background: http://bit.ly/PCSComputing

The document discusses the role and responsibilities of a data scientist. It describes how data scientists take large amounts of messy data and use skills in math, statistics, and programming to organize and analyze the data to uncover solutions to business problems. An effective data scientist has strong skills in both statistics and software engineering. The document also outlines the scientific process that data scientists follow, including developing algorithms and models, testing hypotheses on data, deploying solutions, and continuously monitoring and improving based on results.

This document summarizes a presentation on data science and artificial intelligence. It discusses how AI is transforming businesses in many ways, including automating repetitive tasks, improving customer experiences, and driving revenue growth. It also mentions that while data is important, AI is needed to transform organizations through intelligent process optimization and innovation. The document provides examples of how various companies are applying AI in sales, customer service, and other areas. It emphasizes that AI strategies should focus on innovation, identifying high-impact use cases, and developing people's data science skills.

The document provides an overview of data mining and web mining techniques. It discusses how data mining uses statistical analysis, machine learning, and other techniques to extract patterns and correlations from large datasets. The document also presents results from a case study that analyzed web traffic statistics and visitor behavior on a website to gain insights on how to improve the user experience. Clustering algorithms were used to classify users and generate a web mining model. The case study demonstrated that data mining can efficiently analyze large amounts of web data and provide useful information for website optimization.

Slides for an introductory course on Big Data Tools for Artificial Intelligence. This first set of slides introduces the concept of big data and the current context.

This document discusses enabling analytics as a service (AaaS) on IBM SoftLayer Cloud. It describes how various analytical platforms and workloads have been modernized, migrated, and deployed on the SoftLayer Cloud to provide analytics capabilities as a service. Specifically, it outlines big data analytics platforms like Cloudera, Hortonworks, MapR, and IBM BigInsights that have been implemented on the cloud. It also discusses real-time analytics platforms like VoltDB and Apache Storm that have been deployed on SoftLayer Cloud to enable real-time analytics and processing of fast data streams.

The document discusses big data testing using the Hadoop platform. It describes how Hadoop, along with technologies like HDFS, MapReduce, YARN, Pig, and Spark, provides tools for efficiently storing, processing, and analyzing large volumes of structured and unstructured data distributed across clusters of machines. These technologies allow organizations to leverage big data to gain valuable insights by enabling parallel computation of massive datasets.

Over the past weeks we have been examining the inference process, big data (which feeds this process) and now the inter-dependent nature of digital devices in a digital age. I invite you to follow the links below and explore the constructs of Information Overload and Machine Data. Find others of your own, as well. Do either of these constructs resonate with you? How do they interact with the erosion of boundaries between work and home life that many of you have referenced? How does our dependence on digital devices feed machine data and big data? How does it compromise our personal security? Try weaving one of these ideas into your thinking about ICT. The links are: Big Data Means Information Overload https://www.youtube.com/watch?v=6MpfVD-c-QI http://www.forbes.com/sites/laurashin/2014/11/14/10-steps-to-conquering-information-overload/#7bd535d424fe Machine Data and Operational Intelligence http://www.splunk.com/en_us/resources/machine-data.html http://internetofthingsagenda.techtarget.com/definition/machine-data Solution hadoop .