The Java Microservice Library
- 2. Java microservice development for high-speed services
What is QBit?
❖ Fast queue based service engine!
❖ Services sit behind high-speed queues!
❖ Uses micro-batching and mechanical sympathy to create high speed REST
and WebSocket microservice!
❖ Programming model looks like Spring MVC, but enables high-speed async
model like Akka, Vertx, Reactor, but easier to program!
❖ Uses Consul or Etcd for clustering, health, cloud config, elasticity !
❖ Has event bus that can use Kafka, 0Q, etc.!
❖ Uses JSON for wire protocol, works easily with any language over HTTP or
WebSocket
- 3. Java microservice development for high-speed services
Why QBit?
❖ Just a lib, does not take over the world!
❖ Works with Guice or Spring!
❖ Just POJOs written like Java developers write POJOs no steep
learning curve!
❖ FAST!!
❖ Easier than reactive, worker pools, sharded services and queued
services!
❖ Uses Consul for service discovery, WebSocket for high-speed
messaging and REST
- 4. Java microservice development for high-speed services
Architects / Developers of QBit
❖ Wrote high throughput global services for large high-traffic clients!
❖ F5 crushing load!!
❖ Implemented systems that took other systems 10x to 100x as many servers and 20x
#of developers that could handle only 1/10th of the traffic !
❖ Wrote JSON parser that is 4x faster than mainstream JSON parsers!
❖ Been there, done that for 20+ years. 100+ years building high-speed, reliable systems
combined!
❖ Consulting with airlines, plane manufacturers, banks, commodity trading, electronic
manufacturing, and other trading systems!
❖ Wrote services that powers some of the busiest sites on the Internet!
❖ Wrote fastest disk batcher for JVM, high-speed, distributed k/v store, in-memory db
- 5. Java microservice development for high-speed services
QBit Speed - FAST!
❖ 200 M in-proc messages a second!
❖ 10M to 100M in-proc events per second!
❖ 700K to 1,000,000 RPC calls!
❖ Equates to 1.4M to 2M messages TPS!
❖ Using JSON! One Service Thread! One WebSocket!!
❖ Not uncommon to use a fraction of CPU for 2x to 10x to
100x the same performance as a traditional microservice
- 8. Java microservice development for high-speed services
Service Discovery / Health
❖ Integration with Consul for
health and event bus wiring
of peers!
❖ Cloud config!
❖ Peer discovery!
❖ Health Checks!
❖ Serf/Dead man’s switch,
gossip, custom
- 10. Java microservice development for high-speed services
What is QBit again?
❖ Java microservice development for high-speed services!
❖ JSON!
❖ REST!
❖ WebSocket!
❖ Services look like Spring MVC REST / JAX-RS !
❖ look like Spring Boot, Drop-wizard but work like Spring reactor, Akka, Go,
ErLang, etc. !
❖ Embraces microservices!
❖ Clients can be Web, Mobile, any language that has JSON lib, HTTP lib can use
QBit services
- 11. Java microservice development for high-speed services
QBit Services
❖ WebSocket RPC, HTTP REST, and in-proc messaging!
❖ In-proc, sits behind an actor style queue, gets notified of
queue empty, queue limit, queue init, queue start batch,
etc.!
❖ Allows reduction of sending IO or calls to CPU
intensive services in batches based on queue events!
❖ Batching calls increases system throughput through
the system 10x, 100x, 1000x.
- 12. Java microservice development for high-speed services
Key Concepts
❖ `!
❖ Queue !
❖ A thread managing a queue. !
❖ supports batching. !
❖ queue callbacks for empty, reachedLimit,
startedBatch, idle, etc.!
❖ ServiceQueue!
❖ POJO (plain old Java object) behind a
queue!
❖ receive method calls and events !
❖ thread safe!
❖ queue callback methods!
❖ Java idioms looks like Spring MVC REST,
JAX-RS!
❖ ServiceBundle !
❖ Many POJOs behind one response queue
and many receive queues.
❖ ServiceServer !
❖ Exposes services to REST/JSON and
WebSocket/JSON communication!
❖ EventBus !
❖ send a lot of messages to services !
❖ may be loosely coupled,!
❖ integration with others event buses
(Kafka, STOMP, etc.)!
❖ Metrics/Stats: track in-flow, and outflow
of messages, other metrics for
performance, high-speed time-series
database with REST interface!
❖ Meta-Data-Service: Publish track all
REST JSON schemas for easy integration
and use by third parties
- 13. Java microservice development for high-speed services
Principles of QBit
❖ Web first (JSON, HTTP, WebSocket)!
❖ Integration via high-speed event bus (Kafka, STOMP,
0MQ, ActiveMQ, etc.)!
❖ Embrace Mechanical Sympathy and Queue Theory to
optimize thread hand-off, IO, and core cross talk!
❖ Less is more (be the 95% solution leave the long tail
alone)!
❖ Fun and easy to use!
- 16. Java microservice development for high-speed services
Service REST support
❖ Supports Request params, URI params, JSON POST,
HTTP GET, etc.!
❖ Customize URIs using annotation!
❖ Reasonable Defaults (annotations not needed)!
❖ Methods available via annotations
- 18. Java microservice development for high-speed services
QBit versus Spring XXX
❖ Spring what? DI? MVC? Boot?!
❖ QBit Works easily with DI and Spring Boot!
❖ Competes against Spring MVC but has better thread
model for high-speed services, but not all the “features”.!
❖ Spring MVC does a lot. QBit only does Microservices
only! So only JSON, REST, and WebSocket!
❖ QBit Enable writing in-memory, high-speed services that
can handle 200K TPS to 2M TPS per second single node
- 19. Java microservice development for high-speed services
Akka vs. QBit
❖ Akka uses actor model!
❖ QBit uses queuing actor-like model + active objects model!
❖ Akka focuses on Queue handlers!
❖ QBit focuses on Services that look / act like Java services in
Spring REST, JAX-RS, Java EE, and other frameworks!
❖ QBit is a Java centric model, Akka programming model is similar
but foreign to most Java developers!
❖ QBit center of gravity is Microservices = REST, JSON, WebSocket
- 20. Java microservice development for high-speed services
Akka and QBit similarities/differences
❖ Akka Typed Actor a lot like QBit Service!
❖ QBit Services can be exposed via REST/JSON or
WebSocket/JSON!
❖ Both use queue events (empty, init, limit, etc.) to optimize
output to IO!
❖ Both are fast!
❖ QBit is library and you can use / compose queues,
services as you see fit (you could use QBit in a Spring app)
- 21. Java microservice development for high-speed services
Node.js, Vertx
❖ Node.js and Vertx both use a Reactor style design pattern based on event bus.
(QBit is reactive but using active objects not reactor.)!
❖ At first glance looks a lot like Akka or QBit!
❖ Less granularity on bus, QBit bus speed is a lot faster!
❖ Less control of when queue is idle, empty, etc. QBit has Akka like tuning which is
critical to optimize around back pressure.!
❖ Vertx/Node.js are harder to optimize. Harder to compose other handlers. !
❖ Great for IO and Vertx can be combined with QBit to provide missing pieces for
Speed!
❖ Not bad. Not even close to what QBit can do!
❖ QBit works well with Vertx
- 22. Java microservice development for high-speed services
How does it do?
❖ Faster!
❖ Easier to use!
❖ Focuses just on Micorservices!
❖ Tolerant reader, promiscuous writers!
❖ Uses RAFT and Consul for service discovery
- 24. Java microservice development for high-speed services
Event Bus
❖ QBit has a high-speed event bus modeled loosely on Vertx style
event bus!
❖ Very fast. !
❖ In-proc but can be integrated with others!
❖ Service methods can be invoked by calls (remote and local behind a
queue) !
❖ Service methods can be invoked by listening to a event bus
channels!
❖ You can subscribe to channels or you can consume channels or both
- 25. Java microservice development for high-speed services
Composable Event Bus
❖ EventBus can be composed!
❖ You can use the system event bus for in-proc service to service
messaging!
❖ You can remote replicate events to other servers!
❖ You can tap into an event bus, and/or post events to a service
directly for integration with 3rd party systems like Kafka, 0MQ,
etc.!
❖ You can use Consul integration to find service peers and
compose event bus that allows peers to communicate
- 26. Java microservice development for high-speed services
Event Bus is an Integration point
❖ Event bus is an integration point!
❖ Can be easily integrated with Camel, ActiveMQ, JMS,
Kafka, etc.!
❖ Events come in on same queue as method call!
❖ Event Bus can do remote broadcast and can discover
peers via Consul or Etcd
- 28. Java microservice development for high-speed services
Private event bus versus System event bus
❖ Private event bus!
❖ https://github.com/advantageous/qbit/wiki/%5BRough-Cut%5D-Working-
with-private-event-bus-for-inproc-microservices!
❖ Only your module uses it!
❖ System event bus!
❖ https://github.com/advantageous/qbit/wiki/%5BRough-Cut%5D-Working-
with-event-bus-for-QBit-the-microservice-engine!
❖ Used by default if you don’t define your own!
❖ Event buses are composable!
❖ events are intrinsic to Services and event buses can be wired to services and
other event buses
- 30. Java microservice development for high-speed services
More involved Example
❖ Advantage of using the event bus with QBit services is the events come into
same queue that handles method calls so events method calls are thread
safe!
❖ Event Bus is very fast. Expect speeds up to 10M to 100M messages a second.!
❖ New example: set of services that handles when a new employees is hired!
❖ Add the employee to payroll system, !
❖ Enroll employee into benefits system!
❖ Invite employee to our community outreach program!
❖ Four services but the first service will not know about other services
- 41. Java microservice development for high-speed services
Event Manager
❖ You can use a private event bus instead of a system
event bus!
❖ You can use strongly typed proxy to the event bus to
abstract channel names and provide a strongly typed
bus
- 46. Java microservice development for high-speed services
Working with callbacks
https://github.com/advantageous/qbit/wiki/%5BRough-Cut%5D-QBit-Microservice-Lib-
Working-With-CallBacks
- 54. Java microservice development for high-speed services
Lambdas are great until you can’t read your code anymore
Lambda calling Lambda, calling
Lambda calling Lambda.!
!
Stop using Lambda if your code
starts to look like Perl threw up on
Lisp.!
!
One, to three levels ok.!
!
Five or six, can you break this out
into a method call or two.
- 56. Java microservice development for high-speed services
But gee what if we have to load the user data?
❖ The first rule is don’t block!
❖ Services can call other services!
❖ When a service might do IO then it should take a
callback instead of have a return (generally speaking)
- 63. Java microservice development for high-speed services
When to handle callbacks?
When your queue is
empty, !
when you hit your
queue limit,!
and !
perhaps at the start !
of a new message
batch?!
!
Handle callbacks
before handling new
method calls so
clients don’t wait
around with open
connections
- 64. Java microservice development for high-speed services
Workers for IO, Sharded Services for CPU
❖ QBit allows worker pools for IO bound workers!
❖ QBit allows sharded for CPU bound workers that work
on in-memory data!
❖ Or just regular workers!
❖ Queue system can detect when CPU bound services are
busy and auto-increase queue batch size