Clustering and Shared Data in Vert.x

Question

I'm developing in Vert.x (based on Netty and Hazelcast), and I'm trying to share data between two server instances (eache of those instances in different machines, on the same lan).

My problem is that I don't know how to configure the vert.x servers to allow them to share their concurrent memory maps (the theory says that's possible).

I've read many documents off Vert.x and Hazelcast but I haven't had results yet. (I don't know how to force vert.x to load hazelcast xml configuration files).

Thanks in advance!

Answer 1

There are options for sharing data among vertx instances on different machines

Option 1.

You could use the Vert.x ClusterManager and it's maps:

ClusterManager clusterManager = ((VertxInternal)vertx).clusterManager();
Map map = clusterManager.getSyncMap("mapName"); // shared distributed map

That map is backed by a Hazelcast IMap and is distributed. This assumes you're running vertx with the -cluster parameter and have configured clustering.

However note that this is internal API and is not generally recommended for production. If you are doing a one time experiment then it could be useful.

Option 2.

You can get access to Hazelcast once vertx is started in clustered mode:

Set<HazelcastInstance> instances = Hazelcast.getAllHazelcastInstances();
HazelcastInstance hz = instances.stream().findFirst().get();
Map map = hz.getMap("mapName"); // shared distributed map

Answer 2

With Vert.x 3 - if you configure your Vert.x instances into "clustered mode" (which can be as simple as adding -cluster to the command line of the Vert.x launcher, see here for details), then you can use the SharedData interface to get access to "distributed maps" which allows cluster members to read and write data across the cluster transparently.

Example:

if (vertx.isClustered()) {
    log.info("Using clustered data store");
    vertx.sharedData().<String, MyEntity>getClusterWideMap("entities", 
            res -> {
                AsyncMap<String, MyEntity> dataMap = res.result();
                setDataStore(dataMap);
            });
}

Answer 3

Afaik you can't share data between different instances of vert.x -- from the documentation

"[...] Such a use case is better solved by providing a shared map structure that can be accessed directly by different verticle instances in the same vert.x instance."

Since "vert.x instance" means "jvm instance" you can't use sharedmap/set between different jvm. You can use the event bus for this.

Edit before others people downvotes: my answer is from 2012, 6 years ago, when this was not possible. Now it is possible as others people already said

Answer 4

Vert.x 2 does not support cluster-wide shared data. However, Vert.x 3 does expose an asynchronous API that wraps the underlying Hazelcast cluster manager.

For Vert.x 2, though, you can use the Hazelcast instance directly in your worker verticles. Just use Hazelcast's static methods to get the Vert.x Hazelcast instance:

HazelcastInstance hazelcast = Hazelcast.getAllHazelcastInstances().iterator().next();

Note that you should only access the Hazelcast API directly in this way from within a worker verticle. The Hazelcast API is blocking, so it will block the event loop if used in a normal verticle.

Answer 5

As has been pointed out, the data sharing objects bundled in Vert.x don't support sharing data across multiple Vert.x instances. In order to do that you would have to do either:

1. Use a "normal" shared database.
2. Set up a Verticle which manages the SharedMap instances and publishes and listens for updates to and from the rest of the cluster.
3. Design your application so you don't need to share data in this way.