You know Java, but what is AppEngine? In this session Max will walk you from a brief introduction of Servlets to an overview of the Google AppEngine. We learn about the basics such as the data store – which is quite different from SQL -, application versions, back-ends, scheduler, instances and logging. Services such as mail, url fetch, task queue are also explained. Since we use GAE/J since 2010 in Calpano we also talk about costs and experience in practice.
JavaOne 2016
It is time to move your Java EE builds over to Gradle! Gradle continues to gain momentum across the industry. In fact, Google is now pushing Gradle for Android development. Gradle draws on lessons learned from both Ant and Maven and is the next evolutionary step in Java build tools. This session covers the basics of switching existing Java EE projects (that use Maven) over to Gradle and the benefits you will reap, such as incremental compiling, custom distributions, and task parallelization. You’ll see demos of all the goodies you’ve come to expect, such as integration testing and leveraging of Docker. Switching is easier than you think, and no refactoring is required.
www.caucho.com
Using industry standard tool and methodology, Resin Pro web server was put to the test versus Nginx, a popular web server with a reputation for efficiency and performance. Nginx is known to be faster and more reliable under load than the popular Apache HTTPD. Benchmark tests between Resin and Nginx yielded competitive figures, with Resin leading with fewer errors and faster response times. In numerous and varying tests, Resin handled 20% to 25% more load while still outperforming Nginx. In particular, Resin was able to sustain fast response times under extremely heavy load while Nginx performance degraded.
The document discusses the importance of resilience and failure preparation in cloud systems. It introduces the concept of "GameDay", which are large-scale exercises that inject faults across critical systems to increase an organization's resilience. These exercises help identify risks, reduce failure frequency and recovery time, and build confidence and competence in responding to failures. The document advocates designing infrastructure to be rebuildable from source code and data backups alone. It also stresses the importance of automation, emergencies tools testing, and culture of collaboration between dev and ops.
Full stack development with node and NoSQL - All Things Open - October 2017
What is different about this generation of web applications? A solid development approach must consider latency, throughput, and interactivity demanded by users users across mobile devices, web browsers, and IoT. These applications often use NoSQL to support a flexible data model and easy scalability required for modern development.
A full stack application (composed of Couchbase, WebAPI, Angular2, and ASP.NET/ASP.NET Core) will be demonstrated in this session. The individual parts of a stack may vary, but the overall design is the focus.
Integrating multiple CDN providers at Etsy - Velocity Europe (London) 2013
The document discusses Etsy's experience integrating multiple content delivery network (CDN) providers. Etsy began using a single CDN in 2008 but then investigated using multiple CDNs in 2012 to improve resilience, flexibility, and costs. They developed an evaluation criteria and testing process to initially configure and test the CDNs with non-critical traffic before routing production traffic. Etsy then implemented methods for balancing traffic across CDNs using DNS and monitoring the performance of the CDNs and origin infrastructure.
This document discusses using virtualization and containers to improve database deployments in development environments. It notes that traditional database deployments are slow, taking 85% of project time for creation and refreshes. Virtualization allows for more frequent releases by speeding up refresh times. The document discusses how virtualization engines can track database changes and provision new virtual databases in seconds from a source database. This allows developers and testers to self-service provision databases without involving DBAs. It also discusses how virtualization and containers can optimize database deployments in cloud environments by reducing storage usage and data transfers.
http://wiki4.caucho.com/Java_EE_Tutorial_covering_JSP_2.2,_and_Servlets_3.0
This tutorial focuses on using Servlet's and JSP the right way. Servlet and JSP have evolved over the years, and now there is often more than one way to do things. For example, this tutorial uses EL and JSTL not JSP scriptlets, it uses JSPs in a Model 2/MVC style not in a model 1 style, etc. Consider it a tutorial that focuses only on the best practices and not the legacy ways to do things.
There are other tutorials on this subject, but this tutorial is going to be different in that it is going to put all code into github, and you can follow along with Eclipse. Also instead of focusing on JSF, we are going to focus on JSP and Servlets as the main view technology.
Java EE, JSP and Servlets have added a lot of features that are in other frameworks, making those other frameworks less relevant then they were before Java EE garnered these extra abilities. Even is you decide to use JSF, Struts, Stripes, Spring MVC, JSF, etc., this tutorial should help you have a better understanding of the JSP/Servlets core that they build on.
We are going to start by building a simple bookstore. We will progressively add more features to the bookstore and as we do we will use more of Java EE/CDI, JSP and Servlets.
For this tutorial, I am going to use Resin 4.0.x, but you could use any Java EE 6 Web Profile compliant application server.
This document provides information about an upcoming Box Guy event, including sponsors, speakers and sessions. On the first day, there will be a presentation on FusionReactor at 12:30pm and a podcast at 6:30pm. The second day will feature a lunch with Pete Freitag at 12:30pm. Attendees can enter raffles by tweeting with #intothebox. The event also promotes Ortus' consulting services and modernization approach for CFML applications.
This document provides an overview of the remaining tasks to complete the release of Drupal 8. It discusses the 26 remaining critical issues blocking the beta to RC phase. It outlines goals around performance, security, and the upgrade path. Key tasks include finalizing caching, migrating from Drupal 7, and ensuring alternative databases like SQLite and PostgreSQL are fully supported. The document encourages community involvement in testing, profiling, and helping to resolve critical issues.
ContainerDays NYC 2016: "OpenWhisk: A Serverless Computing Platform" (Rodric ...
Slides from Rodric Rabbah & Philippe Suter's talk "OpenWhisk: A Serverless Computing Platform" at ContainerDays NYC 2016: dynamicinfradays.org/events/2016-nyc/programme.html#openwhisk
JavaOne 2016
This session provides 50 best practices for Java EE 7, with examples. The best practices covered focus primarily on JPA, CDI, JAX-WS, and JAX-RS. In addition, topics involving testing and deployment are covered. This presentation points out where best practices have changed, common misconceptions, and antipatterns that should be avoided. This is a fast-paced presentation with many code samples.
http://wiki4.caucho.com/Java_EE_Tutorial_covering_JSP_2.2,_and_Servlets_3.0
This tutorial focuses on using Servlet's and JSP the right way. Servlet and JSP have evolved over the years, and now there is often more than one way to do things. For example, this tutorial uses EL and JSTL not JSP scriptlets, it uses JSPs in a Model 2/MVC style not in a model 1 style, etc. Consider it a tutorial that focuses only on the best practices and not the legacy ways to do things.
There are other tutorials on this subject, but this tutorial is going to be different in that it is going to put all code into github, and you can follow along with Eclipse. Also instead of focusing on JSF, we are going to focus on JSP and Servlets as the main view technology.
Java EE, JSP and Servlets have added a lot of features that are in other frameworks, making those other frameworks less relevant then they were before Java EE garnered these extra abilities. Even is you decide to use JSF, Struts, Stripes, Spring MVC, JSF, etc., this tutorial should help you have a better understanding of the JSP/Servlets core that they build on.
We are going to start by building a simple bookstore. We will progressively add more features to the bookstore and as we do we will use more of Java EE/CDI, JSP and Servlets.
For this tutorial, I am going to use Resin 4.0.x, but you could use any Java EE 6 Web Profile compliant application server.
Developing Java EE Applications on IntelliJ IDEA with Oracle WebLogic 12c
This document provides an overview and agenda for a presentation on developing Java EE applications for WebLogic 12c with IntelliJ IDEA. The presentation agenda includes an overview of Oracle WebLogic 12c, installing and configuring WebLogic 12c and IntelliJ IDEA, and developing Java EE applications. It also discusses features of WebLogic 12c such as dynamic clusters, high availability, database integration, developer tools and technologies.
This document discusses setting up a web application cluster for scaling and load balancing purposes. It introduces vertical and horizontal clustering approaches. Vertical clustering involves using multiple application instances on the same machine, while horizontal clustering uses multiple machines. The document then provides steps to create a private vertical cluster on a local machine using two Tomcat instances fronted by an Apache HTTP server with a weighted round robin load balancing policy.
OSGi ecosystems compared on Apache Karaf - Christian Schneider
OSGi Community Event 2015
A look at three competing OSGi ecosystems (Declarative Services, Blueprint, CDI). Capabilities of each DI framework. Comparison of support for EE technologies like JPA, Security, SOAP and REST services, UIs. Looking into some of the recent advancements like Aries JPA 2 featuring closure based transactions, JAAS Security, JSP and JSF on OSGi. Attendees will get a good overview about the stacks as well as recommendations where each is most applicable.
Google App Engine (GAE) is a popular PaaS offering. Where its scalable and reliable environment is hidden behind a custom API. This makes GAE apps hard to port over to other non-GAE environments.
But what if one could implement such similar environment? And you could simply move your GAE application’s .war file to this new environment and it would just work?
After all, at the end it’s all about the API, plus scalable and reliable services.
JBoss CapeDwarf project aims at making this a reality. This presentation will provide a glimpse into what it takes to implement something as GAE, ranging from runtime integration with JBoss Application Server, actual services implementation to last but not least, automated heavy testing.
Make Your Build Great Again (DroidConSF 2017)Jared Burrows
Slow builds have been plaguing Android development since the very beginning, especially for large multi-dex projects. As libraries tend to grow in size and the more libraries an application consumes it will slow down the build, especially when an application goes over the mutli-dex limit. Libraries aren't the only thing that can slow down the build, adding many Gradle plugins and repositories can increase the time it takes to configure the Gradle build. This talk will be centered around how I was able to decrease Yammer for Android's Gradle build times by optimizing our use of the Android Gradle plugin and the Gradle setup of our multi-project build and will give several tools and tips on how to help you profile and decrease your build times as well.
Apache Lucene is the de-facto standard open source library for Java developers to implement full-text-search capabilities.
While it’s thriving in its field, it is rarely mentioned in the scope of Java EE development.
In this talk we will see for which features many developers love Lucene, make some concrete examples of common problems it elegantly solves, and see some best practices about using it in a Java EE stack.
Finally we'll see how some popular OSS projects such as Hibernate ORM (JPA provider), WildFly (Java EE runtime) and Infinispan (in-memory datagrid, JCache implementor) actually provide great Lucene integration capabilities.
Building Grails Plugins - Tips And TricksMike Hugo
This document provides an overview of building Grails plugins, including tips and tricks. It discusses creating a plugin project structure, testing plugins, adding configuration, events, and internationalization. It also covers integrating plugins into applications, reloading changes during development, and publishing plugins for others to use.
Faster java ee builds with gradle [con4921]Ryan Cuprak
JavaOne 2016
It is time to move your Java EE builds over to Gradle! Gradle continues to gain momentum across the industry. In fact, Google is now pushing Gradle for Android development. Gradle draws on lessons learned from both Ant and Maven and is the next evolutionary step in Java build tools. This session covers the basics of switching existing Java EE projects (that use Maven) over to Gradle and the benefits you will reap, such as incremental compiling, custom distributions, and task parallelization. You’ll see demos of all the goodies you’ve come to expect, such as integration testing and leveraging of Docker. Switching is easier than you think, and no refactoring is required.
www.caucho.com
Using industry standard tool and methodology, Resin Pro web server was put to the test versus Nginx, a popular web server with a reputation for efficiency and performance. Nginx is known to be faster and more reliable under load than the popular Apache HTTPD. Benchmark tests between Resin and Nginx yielded competitive figures, with Resin leading with fewer errors and faster response times. In numerous and varying tests, Resin handled 20% to 25% more load while still outperforming Nginx. In particular, Resin was able to sustain fast response times under extremely heavy load while Nginx performance degraded.
The document discusses the importance of resilience and failure preparation in cloud systems. It introduces the concept of "GameDay", which are large-scale exercises that inject faults across critical systems to increase an organization's resilience. These exercises help identify risks, reduce failure frequency and recovery time, and build confidence and competence in responding to failures. The document advocates designing infrastructure to be rebuildable from source code and data backups alone. It also stresses the importance of automation, emergencies tools testing, and culture of collaboration between dev and ops.
Full stack development with node and NoSQL - All Things Open - October 2017Matthew Groves
What is different about this generation of web applications? A solid development approach must consider latency, throughput, and interactivity demanded by users users across mobile devices, web browsers, and IoT. These applications often use NoSQL to support a flexible data model and easy scalability required for modern development.
A full stack application (composed of Couchbase, WebAPI, Angular2, and ASP.NET/ASP.NET Core) will be demonstrated in this session. The individual parts of a stack may vary, but the overall design is the focus.
Integrating multiple CDN providers at Etsy - Velocity Europe (London) 2013Marcus Barczak
The document discusses Etsy's experience integrating multiple content delivery network (CDN) providers. Etsy began using a single CDN in 2008 but then investigated using multiple CDNs in 2012 to improve resilience, flexibility, and costs. They developed an evaluation criteria and testing process to initially configure and test the CDNs with non-critical traffic before routing production traffic. Etsy then implemented methods for balancing traffic across CDNs using DNS and monitoring the performance of the CDNs and origin infrastructure.
This document discusses using virtualization and containers to improve database deployments in development environments. It notes that traditional database deployments are slow, taking 85% of project time for creation and refreshes. Virtualization allows for more frequent releases by speeding up refresh times. The document discusses how virtualization engines can track database changes and provision new virtual databases in seconds from a source database. This allows developers and testers to self-service provision databases without involving DBAs. It also discusses how virtualization and containers can optimize database deployments in cloud environments by reducing storage usage and data transfers.
Java EE Servlet JSP Tutorial- Cookbook 1billdigman
http://wiki4.caucho.com/Java_EE_Tutorial_covering_JSP_2.2,_and_Servlets_3.0
This tutorial focuses on using Servlet's and JSP the right way. Servlet and JSP have evolved over the years, and now there is often more than one way to do things. For example, this tutorial uses EL and JSTL not JSP scriptlets, it uses JSPs in a Model 2/MVC style not in a model 1 style, etc. Consider it a tutorial that focuses only on the best practices and not the legacy ways to do things.
There are other tutorials on this subject, but this tutorial is going to be different in that it is going to put all code into github, and you can follow along with Eclipse. Also instead of focusing on JSF, we are going to focus on JSP and Servlets as the main view technology.
Java EE, JSP and Servlets have added a lot of features that are in other frameworks, making those other frameworks less relevant then they were before Java EE garnered these extra abilities. Even is you decide to use JSF, Struts, Stripes, Spring MVC, JSF, etc., this tutorial should help you have a better understanding of the JSP/Servlets core that they build on.
We are going to start by building a simple bookstore. We will progressively add more features to the bookstore and as we do we will use more of Java EE/CDI, JSP and Servlets.
For this tutorial, I am going to use Resin 4.0.x, but you could use any Java EE 6 Web Profile compliant application server.
This document provides information about an upcoming Box Guy event, including sponsors, speakers and sessions. On the first day, there will be a presentation on FusionReactor at 12:30pm and a podcast at 6:30pm. The second day will feature a lunch with Pete Freitag at 12:30pm. Attendees can enter raffles by tweeting with #intothebox. The event also promotes Ortus' consulting services and modernization approach for CFML applications.
Plain english guide to drupal 8 criticalsAngela Byron
This document provides an overview of the remaining tasks to complete the release of Drupal 8. It discusses the 26 remaining critical issues blocking the beta to RC phase. It outlines goals around performance, security, and the upgrade path. Key tasks include finalizing caching, migrating from Drupal 7, and ensuring alternative databases like SQLite and PostgreSQL are fully supported. The document encourages community involvement in testing, profiling, and helping to resolve critical issues.
ContainerDays NYC 2016: "OpenWhisk: A Serverless Computing Platform" (Rodric ...DynamicInfraDays
Slides from Rodric Rabbah & Philippe Suter's talk "OpenWhisk: A Serverless Computing Platform" at ContainerDays NYC 2016: dynamicinfradays.org/events/2016-nyc/programme.html#openwhisk
Top 50 java ee 7 best practices [con5669]Ryan Cuprak
JavaOne 2016
This session provides 50 best practices for Java EE 7, with examples. The best practices covered focus primarily on JPA, CDI, JAX-WS, and JAX-RS. In addition, topics involving testing and deployment are covered. This presentation points out where best practices have changed, common misconceptions, and antipatterns that should be avoided. This is a fast-paced presentation with many code samples.
Java EE Servlet/JSP Tutorial- Cookbook 2billdigman
http://wiki4.caucho.com/Java_EE_Tutorial_covering_JSP_2.2,_and_Servlets_3.0
This tutorial focuses on using Servlet's and JSP the right way. Servlet and JSP have evolved over the years, and now there is often more than one way to do things. For example, this tutorial uses EL and JSTL not JSP scriptlets, it uses JSPs in a Model 2/MVC style not in a model 1 style, etc. Consider it a tutorial that focuses only on the best practices and not the legacy ways to do things.
There are other tutorials on this subject, but this tutorial is going to be different in that it is going to put all code into github, and you can follow along with Eclipse. Also instead of focusing on JSF, we are going to focus on JSP and Servlets as the main view technology.
Java EE, JSP and Servlets have added a lot of features that are in other frameworks, making those other frameworks less relevant then they were before Java EE garnered these extra abilities. Even is you decide to use JSF, Struts, Stripes, Spring MVC, JSF, etc., this tutorial should help you have a better understanding of the JSP/Servlets core that they build on.
We are going to start by building a simple bookstore. We will progressively add more features to the bookstore and as we do we will use more of Java EE/CDI, JSP and Servlets.
For this tutorial, I am going to use Resin 4.0.x, but you could use any Java EE 6 Web Profile compliant application server.
Developing Java EE Applications on IntelliJ IDEA with Oracle WebLogic 12cBruno Borges
This document provides an overview and agenda for a presentation on developing Java EE applications for WebLogic 12c with IntelliJ IDEA. The presentation agenda includes an overview of Oracle WebLogic 12c, installing and configuring WebLogic 12c and IntelliJ IDEA, and developing Java EE applications. It also discusses features of WebLogic 12c such as dynamic clusters, high availability, database integration, developer tools and technologies.
This document discusses setting up a web application cluster for scaling and load balancing purposes. It introduces vertical and horizontal clustering approaches. Vertical clustering involves using multiple application instances on the same machine, while horizontal clustering uses multiple machines. The document then provides steps to create a private vertical cluster on a local machine using two Tomcat instances fronted by an Apache HTTP server with a weighted round robin load balancing policy.
OSGi ecosystems compared on Apache Karaf - Christian Schneidermfrancis
OSGi Community Event 2015
A look at three competing OSGi ecosystems (Declarative Services, Blueprint, CDI). Capabilities of each DI framework. Comparison of support for EE technologies like JPA, Security, SOAP and REST services, UIs. Looking into some of the recent advancements like Aries JPA 2 featuring closure based transactions, JAAS Security, JSP and JSF on OSGi. Attendees will get a good overview about the stacks as well as recommendations where each is most applicable.
Google App Engine (GAE) is a popular PaaS offering. Where its scalable and reliable environment is hidden behind a custom API. This makes GAE apps hard to port over to other non-GAE environments.
But what if one could implement such similar environment? And you could simply move your GAE application’s .war file to this new environment and it would just work?
After all, at the end it’s all about the API, plus scalable and reliable services.
JBoss CapeDwarf project aims at making this a reality. This presentation will provide a glimpse into what it takes to implement something as GAE, ranging from runtime integration with JBoss Application Server, actual services implementation to last but not least, automated heavy testing.
Tokopedia uses Go for several services including search, image uploads, and analytics. Previously they used Node.js but faced challenges with unbounded concurrency, lack of type safety, and performance issues with HTTP clients. Go provides an easy way to write unit tests and benchmarks. Documentation is generated from code comments. Go also includes libraries for object caching, metrics collection, and cancelling long-running requests. Migrating from a scripting language to a compiled language like Go changes the deployment process and requires packaging binaries instead of interpreting code.
This document discusses building a social analytics tool using MongoDB from a developer's perspective. It covers using MongoDB for its schema-less data and ability to handle fast read-write operations. Key topics include using aggregation queries to gain insights from data by chaining queries together and filtering/manipulating results at each stage. JavaScript capabilities in MongoDB allow applying business logic directly to data. Examples demonstrate removing garbage data and stopwords. Indexes, current progress, and tips/tricks learned around cloning collections and removing vs dropping are also covered, with a demo planned.
This document discusses using various technologies on Google App Engine including JIQL, GaeVFS, RESTlets, scheduled tasks, JRuby on Rails, task queues, XMPP, and Clojure. JIQL emulates a relational database on App Engine's Bigtable datastore. GaeVFS provides a virtual filesystem on Bigtable. RESTlets make RESTful web services easy to implement in Java on App Engine. Scheduled tasks allow for background processing via cron jobs. JRuby on Rails provides a way to run Ruby on Rails applications on App Engine. Task queues allow for asynchronous background processing. XMPP enables instant messaging and peer-to-peer applications. Clojure can also be used
This document provides an overview of Gaelyk, a lightweight Groovy toolkit for developing applications on Google App Engine. Gaelyk builds on Groovy's servlet support and provides enhancements to the Google App Engine Java SDK to simplify development. It allows using Groovy scripts called Groovlets instead of raw servlets and Groovy templates instead of JSPs. This provides a clean separation of views and logic for developing web applications on Google App Engine using the Groovy programming language.
The State of Java under Oracle at JCertif 2011Arun Gupta
The document discusses Oracle's strategy regarding Java, open source, and middleware. Key points include:
- Delivering a complete, open, integrated stack of hardware, infrastructure, database, middleware, and applications.
- Integrating Oracle software components to provide unique value to customers.
- Priorities for Java platforms include growing the developer base, adoption, competitiveness, and adapting to change.
- Details of new features in Java SE 7 like string switching and automatic resource management.
- Plans for Java SE 8 including language enhancements from Project Lambda and modularity.
Getting Started on Google Cloud PlatformAaron Taylor
This slide deck accompanied a talk I gave at Boston's Google Cloud Meetup group in June of 2016. It chronicles our story of building out the Meta Search product using Google Cloud Platform, particularly App Engine, and finishes with a short walkthrough of a demo application.
The document provides an overview of Google App Engine (GAE) for running Java applications on cloud platforms. It discusses that in GAE, developers do not manage machines directly and instead upload binaries for GAE to run. It describes various services available in GAE like data storage, processing images, and cron jobs. The document also summarizes tools for local development and deployment, limitations of GAE around filesystem and socket access, and advantages like built-in logging and routing by domain headers.
Java 9 is expected to include several new features and changes, including:
- New collection factory methods like Set.of() and Map.of() that provide immutable collections.
- Enhancements to the Stream API such as takeWhile() and dropWhile().
- Syntax changes like allowing effectively final variables in try-with-resources and @SafeVarargs for private methods.
- The addition of JShell to provide a Java REPL.
- Garbage First (G1) garbage collector becoming the default collector.
- Various performance and logging improvements.
Java script nirvana in netbeans [con5679]Ryan Cuprak
This document discusses using NetBeans as an IDE for JavaScript development. It provides an overview of NetBeans' features for JavaScript including syntax highlighting, code completion, debugging, support for frameworks like Angular and Node.js, and mobile development with Apache Cordova. It also demonstrates how to set up and configure NetBeans for common JavaScript tasks like adding libraries, using build tools like Grunt and Gulp, and setting up unit testing with Karma and Jasmine.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1stYuc2.
Brennan Saeta covers aspects of Coursera’s architecture that enables them to rapidly build sophisticated features for their learning platform. Saeta discusses also their experience running containers in production, what works, what doesn’t, and why. He briefly touches upon container threat models, and how to architect a defense-in-depth strategy to mitigate both known and unknown vulnerabilities. Filmed at qconlondon.com.
Brennan Saeta is a Lead Infrastructure Engineer, leading the ‘Cour’ (core) group responsible for the development environment, core libraries, and the common infrastructure powering Coursera.
Secrets of Performance Tuning Java on KubernetesBruno Borges
Java on Kubernetes may seem complicated, but after a bit of YAML and Dockerfiles, you will wonder what all that fuss was. But then the performance of your app in 1 CPU/1 GB of RAM makes you wonder. Learn how JVM ergonomics, CPU throttling, and GCs can help increase performance while reducing costs.
DrupalSouth 2015 - Performance: Not an AfterthoughtNick Santamaria
Nick Santamaria's performance and scalability presentation from DrupalSouth 2015.
https://melbourne2015.drupal.org.au/session/performance-not-afterthought
Patrick Chanezon and Guillaume Laforge are presenting Google App Engine Java and Gaelyk, the lightweight groovy toolkit on top of the GAE SDK, at the Devoxx conference
This document discusses how web design firms can compete with internal GIS teams by providing web-based GIS (WebGIS) applications. It notes that WebGIS requires learning new tools like JavaScript, AJAX, and RESTful services. To protect their work, internal GIS teams need to learn these new web technologies and prioritize usability over features to create responsive applications. The document advocates for an iterative development process with a focus on performance and usability testing.
Realtime Analytics with MongoDB Counters (mongonyc 2012)Scott Hernandez
This document discusses pre-aggregating data with counters in MongoDB to enable dashboard-style reports. It involves defining metrics and aggregations upfront, and using the $inc operator to increment values in aggregation collections during write operations. This allows generating real-time aggregated data without requiring map-reduce or the aggregation framework, providing better performance for dashboard queries.
This document discusses why Node.js is a good choice for building scalable web applications. Node.js uses an event-driven, non-blocking I/O model that allows for highly scalable web servers. It also has a simple API based around callbacks and handles HTTP requests directly. Additionally, Node.js benefits from a large ecosystem of open source libraries and the npm package manager. The document provides examples of how Node.js can be used to build stateless, rapidly deployable servers and packages applications in a way that supports continuous deployment.
Apache Airflow (incubating) NL HUG Meetup 2016-07-19Bolke de Bruin
Introduction to Apache Airflow (Incubating), best practices and roadmap. Airflow is a platform to programmatically author, schedule and monitor workflows.
This document outlines the history and design proposals of Apache Drill from the OpenDremel team. It describes OpenDremel starting in 2010 with an initial implementation based on the Dremel paper. Over time, the design was found to be naive and was restarted with a new architecture called Dazo inspired by BigQuery. The document proposes several design tenets for Apache Drill including supporting multi-tenancy, being flexible and customizable, being efficient through the use of ZeroVM for sandboxing, and having a suggested architecture with a browser frontend and multi-tenant backend.
Similar to Google AppEngine (GAE/J) - Introduction and Overview from a Java Guy (20)
The document discusses how lean startup methods can save time and money compared to traditional approaches. It explains that lean startup involves making assumptions about a business plan and then running cheap experiments to validate the assumptions. This allows companies to learn quickly which assumptions are incorrect and adapt the plan accordingly. The document provides an example comparing the expected costs of a project with and without using lean startup experiments to validate assumptions. It finds that running only the most useful experiments can significantly reduce the total expected costs from wasted effort on invalidated assumptions.
REQUIREMENTS FOR PERSONAL KNOWLEDGE MANAGEMENT TOOLSMax Völkel
This document outlines requirements for personal knowledge management tools based on an analysis of how knowledge workers take notes and manage information. The three main requirements are:
1) Personal knowledge management tools should allow step-wise formalization that balances the costs of authoring content with the benefits of easy retrieval.
2) Tools require a unified conceptual model that can accommodate different levels of formality and supports the conversion of content between representations.
3) Core functions include fast note entry, versioning, linking and grouping of notes, querying, and active assistance with maintenance tasks.
TagFS — Tag Semantics for Hierarchical File SystemsMax Völkel
Finally a file can be on more than one folder! End of the tyranny of the hierarchy.
The talk describes the design and implementation of a tag-based file system. Presented at IKNOW 2006.
The document discusses artificial memory, including its tasks, structure, and performance indicators. The tasks of artificial memory are to improve knowledge and prevent not knowing. It discusses different types of knowledge and not knowing. The structure includes artificial extending memory (remembering), artificial expanding memory (learning), and artificial reflective memory (reflecting). The performance indicators measure remembering recall and precision, learning recall and precision, and reflective recall and precision based on numbers of information chunks.
Personal Knowledge Models for More Productive Knowledge WorkersMax Völkel
Paper presented at PKM2009
First International Workshop on
Personal Knowledge Management (PKM)
at the 5th Conference on Professional Knowledge Management,
25-27 March 2009, Solothurn
A Survey of the Landscape and State-of-Art in Semantic WikiMax Völkel
Semantic Wikis: The Wiki Way to the Semantic Web?
Semantic Wiki Mini-Series1st session: A Survey of the Landscape and State-of-Art in Semantic Wiki
Co-chairs:
Sebastian Schaffert (Salzburg Research, Austria),Max Völkel (FZI-Karlsruhe)
Persönliches Wissensmanagement mit Semantischen TechnologienMax Völkel
Vortrag von Max Völkel am 09.10.2008 beim Symposium des Arbeitskreises Wissensmanagement (AKWM) in Karlsruhe. http://www.arbeitskreis-wissensmanagement.org/
Cost-Benefit Analysis for the Design of Personal Knowledge Management SystemsMax Völkel
The document discusses the costs and benefits of personal knowledge management systems. It argues that quantifying these costs and benefits is important for improving knowledge reuse and productivity. Key factors that influence the cost-benefit ratio include the granularity and formality of knowledge items, considering the full knowledge process, and decreasing costs over time through improved semantics and tools. The goal is to help knowledge workers better manage what they know for increased productivity.
Talk of Max Völkel at SemWiki2008 workshopMax Völkel
The document discusses the motivation, data model, and user interface for a personal knowledge management application called SemWiki. It describes how the application allows users to create and link various types of information snippets called items. The data model uses RDF triples to represent items, their properties and relationships. The user interface is designed to facilitate easy navigation and authoring of items through association and semantic linking. Future work involves integrating the application with the NEPOMUK semantic desktop to link desktop resources and starting a private beta test.
Google AppEngine (GAE/J) - Introduction and Overview from a Java Guy
1. You
know
Java,
but
what
is
AppEngine?
In
this
session
Max
will
walk
you
from
a
brief
introducAon
of
Servlets
to
an
overview
of
the
Google
AppEngine.
We
learn
about
the
basics
such
as
the
data
store
–
which
is
quite
different
from
SQL
-‐,
applicaAon
versions,
back-‐ends,
scheduler,
instances
and
logging.
Services
such
as
mail,
url
fetch,
task
queue
are
also
explained.
Since
we
use
GAE/J
since
2010
in
Calpano
we
also
talk
about
costs
and
experience
in
pracAce.
Dr. Max Völkel, Calpano
1
3. Do you know Servlets?
§ HttpServlet
• void doGet( HttpServletRequest req,
"HttpServletResponse res);"
• void doPost(…)"
§ HttpServletRequest
• get Header(…), get InputStream(...)"
§ HttpServletResponse
• set Header(…), getOutputStream(...)"
See: http://java.sun.com/products/servlet/, AppEngine supports version 2.5 "
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
3
4. AppEngine is a Servlet Container
§ Servlet Container runs a Web Application
Archive (WAR)
• Class files
• Static resources
• web.xml"
§ In AppEngine also:
• appengine-web.xml, queue.xml, cron.xml,
backends.xml, indexes.xml, …
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
4
5. AppEngine Architecture
Java/Python/Go
§ Running Code
HTTP,
Incoming
Mail,
§ Storing Data
Cron,
Taskqueue
• JVM
• Memcache
• Data store (Old vs. High
Replication)
Servlet
Container
§ Connect to the outside world
• Mail (Incoming, Outgoing)
JVM
• Urlfetch
§ Delay or schedule work
• Taskqueue
Memcache
• Cron
Data
Store
Images
§ More helper services
• Images
Urlfetch,
Outgoing
Mail,
• Users
Taskqueue
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
5
6. Hello AppEngine
§ Typical first test:
• Register, Verify mobile number via SMS
• Create new app “helloworld”
• http://helloworld.appspot.com
§ Next step:
• Buy domain name
• Configure …
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
6
7. 1. Appspot.com > Create App
2. Go to “Apps for your Domain”
https://www.google.com/a/calpano.com
3. Organization & Users > Create user
! Only those can send emails
4. Domain Settings >
Domain names >
Add domain name or alias >
5. Go to appspot.com and “Add Domain”
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
7
8. Application Versions
§ http://{version}.{appid}.appspot.com"
§ http://{appid].appspot.com
à main version (Selected in Dashboard)
§ All versions
• Share the same datastore, memcache (!)
• Different logs, stats, instances
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
8
9. Running Code
§ Java 6 VM (experimental Java 7)
§ No Files (only read the ones you put in your .war)
§ Explicit whitelist which Java classes are allowed (many!)
• https://developers.google.com/appengine/docs/java/jrewhitelist
§ F1: 8 cent/hour
• 128MB memory (!),
• 600MHz CPU
§ F2 = F1 * 2; F4 = F1 * 4
§ ! Servlet response max. time is 60 seconds
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
9
10. Storing Data
§ JVM
• Great when “always on”
§ Memcache
• Really just a cache
§ Data store
• Old: Master-Slave (2 data centers)
• High Replication (3 data centers)
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
10
11. Connect to the outside world
§ Mail
• Incoming
• Send as a servlet request
• Outgoing
• Via Java Mail API, only some headers used
• ! Bounce processing
• http://code.google.com/p/googleappengine/issues/
detail?id=1800
§ Urlfetch
• ! Calling Twitter API etc.
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
11
12. Delay or schedule work
§ Taskqueue
Queue queue = QueueFactory.getDefaultQueue();!
queue.add(TaskOptions.Builder.withDefaults()!
.payload(task));!
• Task is a DeferredTask (has a run() method)
• Very quick to enqueue task
• Task queue service fires requests as
HttpRequests at defined rates, unwraps task,
runs it
• Max runtime: 10 minutes
§ Cron
• Scheduled GET-requests at defined urls
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
12
13. More helper services
§ Images
• Looks like a subset of ImageMagick
§ Users
• Log in via Google Account
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
13
14. Where is a request going?
Typical setup
§ DNS lookup: CNAME or A-record to Google
§ GAE scheduler:
• Request queue for your app
• Boot new instance? Takes ca. 10-30 seconds.
§ On instance:
• Run WAR,
• Run servlet
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
14
16. Time is Money. What time?
§ Free vs. Payment enabled
§ Quotas
• Usage – Free = Billable à Pay
• Resource quota à Fair use limit à Error page
§ Data store
• See also
http://blog.xam.de/2011/11/analysis-of-datastore-operation-
cost.html
§ Instance hours
• Usage (code time * complexity + 15 min)
• What???
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
16
17. Real time vs. API time
§ CPU-intensive code is more expensive
• Is magically speed-up by AppEngine
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
17
18. The Scheduler
Appspot > App Settings > Performance
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
18
20. More stuff
§ Backends
• Requests run for hours (or even days?)
§ Logging
• Java.util.logging runs fine
§ Blobstore
§ Channel ~ like websockets
§ XMPP
§ Remote API:
• Call services remotely
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
20
21. Practical Tips on Datastore
§ Don’t use JDO abstraction
• Leaky abstraction anyway
§ Low-level API quite easy
• Objectify (open source) makes using it nice
• Xydra.org adds versioning/synchronisation
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
21
22. Real-World Experiences
§ Xydra
HTTP,
Incoming
Mail,
Cron,
Taskqueue
§ Calpano.com
§ SonicMetrics.com
Servlet
Container
JVM
Memcache
Data
Store
Images
Urlfetch,
Outgoing
Mail,
Taskqueue
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
22
23. When to use AppEngine?
§ Best: Spikes in demand
§ OK: Outsource server management
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
23
24. Where to learn more?
§ Next talk from Florian Krupicka on
Continuous Integration with AppEngine and
Maven
§ Google webseits – Docu, JavaDocs, Articles,
§ Stackoverflow – Googles official developer
support moved there
§ Books
• Programming Google AppEngine – was OK when
it came out, but now too outdated
§ Book a workshop from Calpano Consulting
J
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
24
25. HTTP,
Incoming
Mail,
Cron,
Taskqueue
Servlet
Container
JVM
Memcache
Data
Store
Images
Urlfetch,
Outgoing
Mail,
Taskqueue
Dr. Max Völkel
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
25
26. Contact Information
§ Dr. Max Völkel
§ Max.voelkel@calpano.com
§ +49 171 8359678
§ www.Calpano.com
GDG Karlsruhe DevFest 2012 | Dr. Max Völkel | Calpano.com
26