Graph Databases in Python (PyCon Canada 2012)

These are the slides from my ICWE 2012 Tutorial "An Introduction to SPARQL and Queries over Linked Data"

At Stitch Fix we have a lot of Data Scientists. Around eighty at last count. One reason why I think we have so many, is that we do things differently. To get their work done, Data Scientists have access to whatever resources they need (within reason), because they’re end to end responsible for their work; they collaborate with their business partners on objectives and then prototype, iterate, productionize, monitor and debug everything and anything required to get the output desired. They’re full data-stack data scientists! The teams in the organization do a variety of different tasks: - Clothing recommendations for clients. - Clothes reordering recommendations. - Time series analysis & forecasting of inventory, client segments, etc. - Warehouse worker path routing. - NLP. … and more! They’re also quite prolific at what they do -- we are approaching 4500 job definitions at last count. So one might be wondering now, how have we enabled them to get their jobs done without getting in the way of each other? This is where the Data Platform teams comes into play. With the goal of lowering the cognitive overhead and engineering effort required on part of the Data Scientist, the Data Platform team tries to provide abstractions and infrastructure to help the Data Scientists. The relationship is a collaborative partnership, where the Data Scientist is free to make their own decisions and thus choose they way they do their work, and the onus then falls on the Data Platform team to convince Data Scientists to use their tools; the easiest way to do that is by designing the tools well. In regard to scaling Data Science, the Data Platform team has helped establish some patterns and infrastructure that help alleviate contention. Contention on: Access to Data Access to Compute Resources: Ad-hoc compute (think prototype, iterate, workspace) Production compute (think where things are executed once they’re needed regularly) For the talk (and this post) I only focused on how we reduced contention on Access to Data, & Access to Ad-hoc Compute to enable Data Science to scale at Stitch Fix. With that I invite you to take a look through the slides.

Deep Water is H2O's integration with multiple open source deep learning libraries such as TensorFlow, MXNet and Caffe. On top of the performance gains from GPU backends, Deep Water naturally inherits all H2O properties in scalability. ease of use and deployment. In this talk, I will go through the motivation and benefits of Deep Water. After that, I will demonstrate how to build and deploy deep learning models with or without programming experience using H2O's R/Python/Flow (Web) interfaces. Jo-fai (or Joe) is a data scientist at H2O.ai. Before joining H2O, he was in the business intelligence team at Virgin Media in UK where he developed data products to enable quick and smart business decisions. He also worked remotely for Domino Data Lab in the US as a data science evangelist promoting products via blogging and giving talks at meetups. Joe has a background in water engineering. Before his data science journey, he was an EngD research engineer at STREAM Industrial Doctorate Centre working on machine learning techniques for drainage design optimization. Prior to that, he was an asset management consultant specialized in data mining and constrained optimization for the utilities sector in the UK and abroad. He also holds an MSc in Environmental Management and a BEng in Civil Engineering.

The document compares Neo4j, Titan, and Cassandra graph databases. It provides details on each database such as Neo4j using the Cypher query language, Cassandra being highly distributed and able to scale linearly, and Titan running on Cassandra or HBase but not supporting Cypher queries. It also gives a 15 point comparison of Cassandra vs Neo4j and examples of querying the same data in Gremlin, Cypher, and SQL. The conclusion recommends a graph database like Neo4j for recommendation queries and only using Titan for very large graphs or high loads.

I used this slideset to present our research paper at the 14th Int. Semantic Web Conference (ISWC 2015). Find a preprint of the paper here: http://olafhartig.de/files/HartigPerez_ISWC2015_Preprint.pdf

Introductionary talk and tutorial on Groovy, discussion of its success and popularity, issues with the future of Java with examples

Presented at JAX London In this session we'll look at some of the design and implementation strategies you can employ when building a Neo4j-based graph database solution, including architectural choices, data modelling, and testing.

Graph Databases try to make it easy for developers to leverage huge amounts of connected information for everything from routing to recommendations. Doing that poses a number of challenges on the implementation side. In this talk we want to look at the different storage, query and consistency approaches that are used behind the scenes. We’ll check out current and future solutions used in Neo4j and other graph databases for addressing global consistency, query and storage optimization, indexing and more and see which papers and research database developers take inspirations from.

Erin LeDell's presentation on scalable machine learning in R with H2O from the Portland R User Group Meetup in Portland, 08.17.15

Sparkler is a new open source web crawler that scales horizonatally on Apache Spark. Sparkler was presented at Apache Big Data EU 2016, Seville, Spain

http://www.oscon.com/open-source-2015/public/schedule/detail/41579 In this presentation, an open source developer community considers itself algorithmically. This shows how to surface data insights from the developer email forums for just about any Apache open source project. It leverages advanced techniques for natural language processing, machine learning, graph algorithms, time series analysis, etc. As an example, we use data from the Apache Spark email list archives to help understand its community better; however, the code can be applied to many other communities. Exsto is an open source project that demonstrates Apache Spark workflow examples for SQL-based ETL (Spark SQL), machine learning (MLlib), and graph algorithms (GraphX). It surfaces insights about developer communities from their email forums. Natural language processing services in Python (based on NLTK, TextBlob, WordNet, etc.), gets containerized and used to crawl and parse email archives. These produce JSON data sets, then we run machine learning on a Spark cluster to find out insights such as: * What are the trending topic summaries? * Who are the leaders in the community for various topics? * Who discusses most frequently with whom? This talk shows how to use cloud-based notebooks for organizing and running the analytics and visualizations. It reviews the background for how and why the graph analytics and machine learning algorithms generalize patterns within the data — based on open source implementations for two advanced approaches, Word2Vec and TextRank The talk also illustrates best practices for leveraging functional programming for big data.

Apache Spark is one of the most popular general purpose distributed systems, with built in libraries to support everything from ML to SQL. Spark has APIs across languages including Scala, Java, Python, and R -- with more 3rd party language support (like Julia & C#). Apache BEAM is a cross-platform tool for building on top of different distributed systems, but its in it’s early stages. This talk will introduce the core concepts of Apache Spark, and look to the potential future of Apache BEAM. Apache Spark has two core abstractions for representing distributed data and computations. This talk will introduce the basics of RDDs and Spark DataFrames & Datasets, and Spark’s method for achieving resiliency. Since it’s a big data talk, we will include the almost required wordcount example, and end the Spark part with follow up pointers on Spark’s new ML APIs. For folks who are interested we’ll then talk a bit about portability, and how Apache BEAM aims to improve portability (as well it’s unique approach to cross-language support). Slides from Holden's talk at https://www.meetup.com/Wellington-Data-Scaling-Chats/events/mdcsdpyxcbxb/

PDF and Keynote version of the presentation available here: https://github.com/h2oai/h2o-meetups/tree/master/2017_04_04_HarvardMed_Scalable_Ensembles

This document provides an overview of GraphDB and Neo4j. It discusses why graphs are useful for modeling connected data and common use cases. It also summarizes Neo4j's transactional graph database capabilities, performance advantages, and deployment options. Key topics covered include causal clustering, query planning, and driver and tooling support for developers.

Treasure Data is a cloud-based big data analytics company based in Silicon Valley with about 20 employees. The document discusses Treasure Data's services and architecture, which includes collecting data from various sources using Fluentd, storing the data in a columnar format on AWS S3, and performing analytics using Hadoop and SQL queries. Treasure Data aims to simplify big data adoption through its fully-managed platform and quick setup process. Example customers discussed were able to see results within 2 weeks of signing up.

This document summarizes an introduction to data analysis in Python using Wakari. It discusses why Python is a good language for data analysis, highlighting key Python packages like NumPy, Pandas, Matplotlib and IPython. It also introduces Wakari, a browser-based Python environment for collaborative data analysis and reproducible research. Wakari allows sharing of code, notebooks and data through a web link. The document recommends several talks at the PyData conference on efficient computing, machine learning and interactive plotting.

This document discusses persistent graphs in Python with Neo4j. It begins by explaining the limitations of relational databases and how graph databases like Neo4j focus on modeling complex relationships through nodes and edges. It then provides an overview of Neo4j, describing it as an open source graph database that is stable, actively developed, and can handle billions of nodes and relationships to model complex data.

The document discusses developing website search capabilities in Python. It provides an overview of typical search engine components like indexing, analyzing, and searching. It then compares two Python search libraries - Pylucene and Whoosh. Benchmark tests on indexing, committing, and searching a 1GB dataset showed Whoosh to outperform Pylucene in speed. The document recommends designing search as an independent, pluggable component and considers Whoosh and Pylucene as good options for rapid development and integration into Python web projects.

This document summarizes a talk about making Django and NoSQL databases like MongoDB play nicely together. Currently, Django's ORM is optimized for SQL databases and makes assumptions that don't always apply to NoSQL databases. The talk proposes some changes to address this, including having the Query object do less database-specific work and pushing more of that down to the individual database compilers. This would make the Query more agnostic and allow the compilers to generate queries optimized for their specific databases. An example backend for MongoDB would be built to demonstrate this approach.

This document discusses building knowledge graphs using DIG (Distributed Information Graphs) to integrate heterogeneous data sources. It describes the steps involved, including data acquisition, feature extraction, mapping to an ontology, entity resolution, graph construction, and deployment. As a use case, DIG has been used to build a knowledge graph from over 100 million web pages related to human trafficking to help law enforcement identify victims and prosecute traffickers.

This document compares relational and non-relational databases. It discusses how in 2003 the main databases were relational, but by 2010 non-relational databases grew popular in the "NoSQL movement". However, the document argues that there are no truly new database designs and that relational and non-relational databases can be combined. It advises to choose a database based on the specific problem and features needed rather than general classifications. The document provides examples of which types of databases fit certain data and access needs.

This document describes Bubbles, a Python framework for data processing and quality probing. Bubbles focuses on representing data objects and defining operations that can be performed on those objects. Key aspects include: - Data objects define the structure and representations of data without enforcing a specific storage format. - Operations can be performed on data objects and are dispatched dynamically based on the objects' representations. - A context stores available operations and handles dispatching. - Stores provide interfaces to load and save objects from formats like SQL, CSV, etc. - Pipelines allow sequencing operations to transform and process objects from source to target stores. - The framework includes common operations for filtering, joining, aggreg

Search engines frequently miss the mark when it comes to understanding user intent. This talk will walk through some of the key building blocks necessary to turn a search engine into a dynamically-learning "intent engine", able to interpret and search on meaning, not just keywords. We will walk through CareerBuilder's semantic search architecture, including semantic autocomplete, query and document interpretation, probabilistic query parsing, automatic taxonomy discovery, keyword disambiguation, and personalization based upon user context/behavior. We will also see how to leverage an inverted index (Lucene/Solr) as a knowledge graph that can be used as a dynamic ontology to extract phrases, understand and weight the semantic relationships between those phrases and known entities, and expand the query to include those additional conceptual relationships. As an example, most search engines completely miss the mark at parsing a query like (Senior Java Developer Portland, OR Hadoop). We will show how to dynamically understand that "senior" designates an experience level, that "java developer" is a job title related to "software engineering", that "portland, or" is a city with a specific geographical boundary (as opposed to a keyword followed by a boolean operator), and that "hadoop" is the skill "Apache Hadoop", which is also related to other terms like "hbase", "hive", and "map/reduce". We will discuss how to train the search engine to parse the query into this intended understanding and how to reflect this understanding to the end user to provide an insightful, augmented search experience. Topics: Semantic Search, Apache Solr, Finite State Transducers, Probabilistic Query Parsing, Bayes Theorem, Augmented Search, Recommendations, Query Disambiguation, NLP, Knowledge Graphs

P. ter Horst / 2010

Kovelinas is a painter born in 1978 in Lithuania. He is known for his Divas Powerpoint series of paintings. The document provides brief biographical information about the Lithuanian painter Kovelinas and references one of his art series but does not provide any additional context or details.

This document summarizes different market structures: pure competition, pure monopoly, monopolistic competition, oligopoly, and collusive oligopoly. It describes key characteristics of each like perfect competition having many firms and monopoly having a single firm. It also discusses profit maximization under these structures and provides examples.

Mobisfera és una Agència de Màrqueting Mòbil amb seu a Barcelona. Imaginem, dissenyem i desenvolupem solucions per a dispositius mòbils. A més, apostem per l’Internet of Thinks (IOT) i la Wearable Technology.

LINE Developer Meetup in Fukuoka #9 Talk. Actual slide is http://youhei.github.io/why-and-how-of-java8-at-line-fukuoka/

The document outlines the stages and funding of the Dementias Platform UK project. It establishes cohort(s) in stage 1 with £6M funding and establishes an imaging platform in stage 2 with another £6M. An additional £36M in capital funding will go toward imaging, stem cells, and informatics. It lists the director as John Gallacher from Oxford and describes the 14 work packages and informatics network leads. The informatics network lead is Simon Lovestone and lists various informatics sub-networks and their leads. It provides a conceptual model for the imaging informatics component with a central XNAT hub and nodes at various research centers.

Fastrack is a sub-brand of Titan that was established in 1998 and focuses on watches and accessories. The document discusses Fastrack's digital marketing campaign objectives of engaging 1000 students at top private universities in Bangladesh by 2016 through social media campaigns on platforms like Facebook and YouTube. The campaigns aim to raise brand awareness and engage customers among the target 20-25 year old male and female demographic interested in style, fashion and experiencing new things.

The document discusses three paths to designing digital experiences: structural, community, and customer. It advocates writing an experience brief to define goals and mapping the customer journey. The presentation provides recommendations for libraries to focus on the customer experience by asking questions, emphasizing conversation, and staging experiences on their website. The overall message is that experience design improves the ordinary interactions people have with an organization online.

El documento presenta una lista de autores españoles e italianos, incluyendo breves biografías y extractos cortos de sus obras más famosas. Entre los autores se encuentran Juan Ramón Jiménez, Rafael Alberti, Miguel de Cervantes, Gustavo Adolfo Bécquer, William Shakespeare. El documento proporciona información básica sobre la vida y obras de estos importantes escritores.

The document provides an introduction to hidden Markov models (HMM) and their applications. It begins with an overview of HMM and its advantages for modeling sequential data. It then describes the basic concepts of Markov models, including their graphical representation, definitions, and algorithms for calculating sequence and state probabilities. The document introduces HMM and the hidden aspect, which is the state transition information that cannot be directly observed. It provides the formal definition of HMM and describes the three main problems in HMM: model evaluation, decoding, and training. It focuses on explaining the forward algorithm for efficient model evaluation in linear time complexity. The document uses examples throughout to illustrate key concepts such as Markov models, HMM, and the forward algorithm.

This Nielsen report summarizes the results of a global survey of over 28,000 online consumers in 56 countries regarding their multi-screen media usage. The survey found that watching video on computers has become as popular as watching TV among online users. Reported online and mobile video viewing is rising, with over half of global online consumers watching videos on mobile phones monthly. Smartphone ownership is up significantly since 2010 and tablets are also gaining popularity globally. The report concludes that portable devices will continue affecting media consumption as their adoption increases.

The document summarizes a presentation on using R and Hadoop together. It includes: 1) An outline of topics to be covered including why use MapReduce and R, options for combining R and Hadoop, an overview of RHadoop, a step-by-step example, and advanced RHadoop features. 2) Code examples from Jonathan Seidman showing how to analyze airline on-time data using different R and Hadoop options - naked streaming, Hive, RHIPE, and RHadoop. 3) The analysis calculates average departure delays by year, month and airline using each method.

My talk at August's joint meeting of Chicago's R and Hadoop user groups providing an introduction to using R with Hadoop. It starts with a quick introduction to and overview of available options, then focuses on using RHadoop's rmr library to perform an analysis on the publicly-available 'airline' data set.

This document discusses scalability issues with publishing, exchanging, and consuming large RDF datasets on the semantic web. It proposes an integrated solution called Binary RDF that includes (1) a binary serialization format for efficient publication and exchange of RDF data, and (2) basic data structures for direct consumption without decompression. Preliminary results show Binary RDF in the form of HDT can provide a compact representation of RDF and support direct pattern matching queries during consumption. Further work is needed to fully understand RDF structure and apply it to innovative dictionary and triple indexes.

Presentation given at Drupal Summit Latino 2013 in Loja, Ecuador, covering the topic of the Semantic Web and Drupal 7.

Roberto García presented on exploring linked data. He discussed how semantic data is fine for computers but difficult for people to interact with. He proposed automatically generating user interfaces from ontologies and datasets, including overview menus, faceted browsing, and interaction patterns to allow users to build queries without knowledge of SPARQL or dataset structure. He demonstrated examples of his approach applied to DBPedia and LinkedMDB data.

This document summarizes Rodrigo Dias Arruda Senra's 2012 doctoral thesis defense at the University of Campinas. The thesis studied how to organize digital information for sharing across heterogeneous systems and proposed three main contributions: 1) SciFrame, a conceptual framework for scientific digital data processing; 2) database descriptors to enable loose coupling between applications and database management systems; and 3) organographs, a method for explicitly organizing information based on tasks.

This document discusses using R Shiny and related tools to create cloud-based spatial data analytics applications. It describes a case study of an app called VectorPoint created to analyze spatial disease data from Peru. The app allows users to collect field data via smartphones, calculate disease probabilities on a map, and track inspections. R Shiny allows rapid prototyping by combining R code and interactive web interfaces. While powerful for prototyping, R Shiny has limitations like requiring an online connection and not being optimized for speed.

This is part of an introductory course to Big Data Tools for Artificial Intelligence. These slides introduce students to the use of Apache Pig as an ETL tool over Hadoop.

The document discusses using Python with Hadoop frameworks. It outlines some of the benefits of Hadoop like scalability and schema flexibility, and benefits of Python like succinct code and many data science libraries. It then reviews several projects that aim to bridge Python and Hadoop, including mrjob for MapReduce jobs, Pydoop for faster MapReduce, Pig for higher-level data flows, Snakebite for a Python HDFS client, and PySpark for working with Spark. However, it notes that Python support is often an afterthought or fringe project compared to the native Java support, and lacks commercial backing or cohesive APIs.

This document summarizes an introductory webinar on building an enterprise knowledge graph from RDF data using TigerGraph. It introduces RDF and knowledge graphs, demonstrates loading DBpedia data into a TigerGraph graph database using a universal schema, and provides examples of queries to extract information from the graph such as related people, publishers by location, and related topics for a given predicate. The webinar encourages attendees to learn more about graph databases and TigerGraph through additional resources and future webinar episodes.

Programmers love Python because of how fast and easy it is to use. Python cuts development time in half with its simple to read syntax and easy compilation feature. Debugging your programs is a breeze in Python with its built in debugger. Python is continued to be a favourite option for data scientists who use it for building and using Machine learning applications and other scientific computations. Python has evolved as the most preferred Language for Data Analytics and the increasing search trends on python also indicates that Python is the next "Big Thing" and a must for Professionals in the Data Analytics domain.

This document discusses Grails integration with Neo4j graph databases. It begins with an introduction to graph databases and Neo4j. It then covers the Grails Neo4j plugin which allows using Neo4j as the persistence layer for Grails domain classes. Finally, it addresses some challenges in mapping the Grails domain model to the Neo4j nodespace and potential solutions.

This document proposes a mapping between the Web Ontology Language (OWL) and the OpenAPI Specification (OAS) to generate REST APIs from OWL ontologies. It describes a mapping method, discusses related work, and details the mapping's coverage of OWL constructs. While some constructs like complex boolean restrictions are not supported, the mapping specification and implementation aim to make ontology knowledge graphs accessible via RESTful APIs in accordance with FAIR principles. Future work includes enhancing path/schema naming and adding metadata annotations.