SlideShare a Scribd company logo

Infinit: Modern Storage Platform for Container Environments

•
•
Docker, Inc.
Docker, Inc.

Providing state to applications in Docker requires a backend storage component that is both scalable and resilient in order to cope with a variety of use cases and failure scenarios. The Infinit Storage Platform has been designed to provide Docker applications with a set of interfaces (block, file and object) allowing for different tradeoffs. This talk will go through the design principles behind Infinit and demonstrate how the platform can be used to deploy a storage infrastructure through Docker containers in a few command lines.

Related slideshows

Docker & kubernetes
Docker & kubernetesDocker & kubernetes
Docker & kubernetes
 
Abc of docker
Abc of dockerAbc of docker
Abc of docker
 
Introduction to Docker - 2017
Introduction to Docker - 2017Introduction to Docker - 2017
Introduction to Docker - 2017
 
1 of 27
Download to read offline
Modern Storage Platform for Container Environments Julien Quintard Member of Technical Staff Docker Inc. (http://infinit.sh)
1. Platform 2. Internals 3. Demo
the CONTEXT The uniqueness of containers compared to traditional virtual machines comes mainly from the fact that they are lightweight, fast and scalable, at least when it comes to stateless applications. Containers also bring new concepts such as portability while opening ways to new techniques such as micro-services in which an application is split into several containers.
the PERSISTENCE Docker has gained in popularity mainly through the use of stateless applications. Developers and enterprises now require statefulness. However, given their uniqueness, containers need more than just traditional storage but a platform allowing stateful applications to be deployed and scaled as easily in spite of failures, be it the host or container itself. More precisely, a modern storage platform should be both elastic (scalable and fault tolerant) and customizable.
1. Platform
the OVERVIEW
the KEY-VALUE STORE Infinit’s key layer is its key-value store whose role is to aggregate storage capacity from a set of nodes. Unlike most key-value store available on the market, Infinit’s benefits from the following: Consensus Infinit’s consensus algorithm is block- based and allows for a strong consistency model on top of which others (eventual) can be emulated. Policies The key-value store can be configured through a set of policies from encryption (e.g key size), block size, compression, deduplication, data placement, redundancy, versioning and more. On top of the key-value store, a number of interfaces is provided, each with its tradeoffs from usage, performance etc. Decentralization While most distributed systems rely on a leader/follower model, Infinit is completely decentralized allowing for better scalability and fault tolerance.
the INTERFACES On top of the key-value store, a number of interfaces is provided, each with its tradeoffs from usage, performance etc.
File System The file system interface was the first one introduced in the Infinit storage platform. It allows for the creation of hierarchical volumes of files and folders. For now only FUSE (Linux/macOS) and Dokan (Windows) are publicly available. POSIX-compliant file system interfaces allows for concurrent reader/writer accesses from multiple client nodes/containers, the drawback being that such interfaces are often quite complex, hence impacting performance. Examples range from storing logs, configuration files, credentials or just running a legacy application.
Object Storage An object storage HTTP-based API allows for storing unstructured data through binary large objects (blob). The most well known of such APIs is Amazon S3 which has become the de-factor standard. Such an interface is more adapted than file system when one is looking for a simple way to store objects which are often independent from one another. Examples range from videos, images etc. in the context of a website for instance.
Block Device The most common network-based block storage interfaces are iSCSI and FCoE. Those provide raw access to a networked partition which can be formatted in the file system of your choice: ext4, NTFS etc. Block devices are interesting because of their inherent limitation. Such devices can only be attached to a single node. As such, concurrent accesses are limited, only possible on the same node, going through the same kernel handling the block storage. This limitation implies that the underlying logic can be optimized to remove the need for concurrent access mechanisms, to store a database for instance.
the EXAMPLE database block (iSCSI) 512-byte block size replication file (NFS) replication versioning deduplication object (AWS S3) 1-megabyte block size datacenter aware erasure coding compression ext4 im ages/videos configurationfiles content Below is an example of a micro-service application relying on a policy- based storage platform providing multiple interfaces: Wordpress.
the DEPLOYMENTS Hyperscale Deploy your storage infrastructure for it to evolve and scale independently of your cluster of applications (Swarm, Kubernetes etc.). #powerful #operators Hyperconverge Your storage infrastructure scales along with your cluster of applications. #simple #developers
2. Internals
the DISTRIBUTION Infinit is completely decentralized (peer-to-peer) meaning that any operation could be carried out by any other node. In other words, every node plays part of the role of the metadata server. This symmetry property is critical to ensure the validation of operations a posteriori. Infinit’s key-value store relies on two basic constructs to benefit from high scalability and fault tolerance. Overlay Network (not related to Docker’s overlay) The overlay network provides a routing mechanism that allows a client to locate the nodes responsible for a piece of data given an identifier. Noteworthy is that there exist different types of overlay network, each with its own advantages. Distributed Hash Table The distributed hash table relies on the overlay network and ensures the redundancy (replication/ erasure coding), consistency (Paxos), self-healing and mutability of the stored items in order to guarantee their availability and durability.
Example: The Chord overlay network can scale to millions of nodes.
Example: In the Kouncil overlay network, every node has complete knowledge of the nodes and blocks.
the FLOW We’ve introduced the key-value store which is implemented through an overlay network and distributed hash table on top of which several interfaces are provided. Let’s take an example to understand how all those layers interconnect through a system call, create(“/foo”), in this case.
Infinit: Modern Storage Platform for Container Environments
the CONSENSUS Most distributed system rely on a set of managers. Those special nodes organize the network and authorize some critical operations. When such an operation must be performed, the quorum of manager nodes is contacted to reach an agreement and maintain consistency. The problem with this design is that the manager nodes can become bottlenecks, potentially leading to a cascading effect, not to mention an ideal target for privilege escalation. Infinit on the contrary, operates in a decentralized manner, meaning that no central authority is involved.
Unlike traditional distributed systems that rely on a global lock, Infinit relies on per-block quorums. As such, the complexity of the consensus algorithm is related to the redundancy factor only, allowing for better scalability, security and fault tolerance. Distributed System Decentralized System
the BLOCKS The key-value store allows for storing blocks of variable size. There are many types of blocks the construct allows one to store, from immutable and mutable blocks to more specific ones such as named blocks that link a string to an address. Mutable Block Such blocks evolve over time based on a version number. As such they are subject to conflicts. Also, clients need to invalidate such blocks in their cache to make sure the latest version is refetched. Finally, mutable blocks rely on heavier cryptographic mechanisms to ensure integrity and security. Immutable Block Immutable blocks, on the other hand, can only exist in one version. The address of such a block is computed by applying a one-way function on the block’s content (content addressed). Should the content be modified, the address would change as well, leading to a new block. Immutable blocks therefore cannot conflict, can be cached forever and are easy to validate by reapplying the one-way function on the retrieved content.
For instance, in the context of a file system, metadata a.k.a inode are stored in a mutable block while actual data blocks are immutable, all stored through the key-value store. As such, a vast majority of the blocks are immutable, leading to better performance because those are cheap to write, fetchable from any source and cachable ad vitam æternam.
3. Demo
Infinit: Modern Storage Platform for Container Environments
the CONCLUSION This demo is really all about the key basics. From that point you can generate passports for other users to connect to your storage infrastructure, you can control accesses on files and folder, scale out with more storage capacity from the cloud and also, obviously, access an Infinit volume in your Docker containers through Infinit’s Docker Volume Plugin. Infinit is still at a very early stage of its development. In the future, we will introduce “infinit cluster” to make it easier to deploy a storage infrastructure (like Docker Swarm through —init/—join), provide more interfaces (block/object) and open source the several layers I talked about today.
Website infinit.sh Twitter @infinit Github github.com/infinit Reddit /r/infinit

More Related Content

Infinit: Modern Storage Platform for Container Environments

  • 1. Modern Storage Platform for Container Environments Julien Quintard Member of Technical Staff Docker Inc. (http://infinit.sh)
  • 3. the CONTEXT The uniqueness of containers compared to traditional virtual machines comes mainly from the fact that they are lightweight, fast and scalable, at least when it comes to stateless applications. Containers also bring new concepts such as portability while opening ways to new techniques such as micro-services in which an application is split into several containers.
  • 4. the PERSISTENCE Docker has gained in popularity mainly through the use of stateless applications. Developers and enterprises now require statefulness. However, given their uniqueness, containers need more than just traditional storage but a platform allowing stateful applications to be deployed and scaled as easily in spite of failures, be it the host or container itself. More precisely, a modern storage platform should be both elastic (scalable and fault tolerant) and customizable.
  • 7. the KEY-VALUE STORE Infinit’s key layer is its key-value store whose role is to aggregate storage capacity from a set of nodes. Unlike most key-value store available on the market, Infinit’s benefits from the following: Consensus Infinit’s consensus algorithm is block- based and allows for a strong consistency model on top of which others (eventual) can be emulated. Policies The key-value store can be configured through a set of policies from encryption (e.g key size), block size, compression, deduplication, data placement, redundancy, versioning and more. On top of the key-value store, a number of interfaces is provided, each with its tradeoffs from usage, performance etc. Decentralization While most distributed systems rely on a leader/follower model, Infinit is completely decentralized allowing for better scalability and fault tolerance.
  • 8. the INTERFACES On top of the key-value store, a number of interfaces is provided, each with its tradeoffs from usage, performance etc.
  • 9. File System The file system interface was the first one introduced in the Infinit storage platform. It allows for the creation of hierarchical volumes of files and folders. For now only FUSE (Linux/macOS) and Dokan (Windows) are publicly available. POSIX-compliant file system interfaces allows for concurrent reader/writer accesses from multiple client nodes/containers, the drawback being that such interfaces are often quite complex, hence impacting performance. Examples range from storing logs, configuration files, credentials or just running a legacy application.
  • 10. Object Storage An object storage HTTP-based API allows for storing unstructured data through binary large objects (blob). The most well known of such APIs is Amazon S3 which has become the de-factor standard. Such an interface is more adapted than file system when one is looking for a simple way to store objects which are often independent from one another. Examples range from videos, images etc. in the context of a website for instance.
  • 11. Block Device The most common network-based block storage interfaces are iSCSI and FCoE. Those provide raw access to a networked partition which can be formatted in the file system of your choice: ext4, NTFS etc. Block devices are interesting because of their inherent limitation. Such devices can only be attached to a single node. As such, concurrent accesses are limited, only possible on the same node, going through the same kernel handling the block storage. This limitation implies that the underlying logic can be optimized to remove the need for concurrent access mechanisms, to store a database for instance.
  • 12. the EXAMPLE database block (iSCSI) 512-byte block size replication file (NFS) replication versioning deduplication object (AWS S3) 1-megabyte block size datacenter aware erasure coding compression ext4 im ages/videos configurationfiles content Below is an example of a micro-service application relying on a policy- based storage platform providing multiple interfaces: Wordpress.
  • 13. the DEPLOYMENTS Hyperscale Deploy your storage infrastructure for it to evolve and scale independently of your cluster of applications (Swarm, Kubernetes etc.). #powerful #operators Hyperconverge Your storage infrastructure scales along with your cluster of applications. #simple #developers
  • 15. the DISTRIBUTION Infinit is completely decentralized (peer-to-peer) meaning that any operation could be carried out by any other node. In other words, every node plays part of the role of the metadata server. This symmetry property is critical to ensure the validation of operations a posteriori. Infinit’s key-value store relies on two basic constructs to benefit from high scalability and fault tolerance. Overlay Network (not related to Docker’s overlay) The overlay network provides a routing mechanism that allows a client to locate the nodes responsible for a piece of data given an identifier. Noteworthy is that there exist different types of overlay network, each with its own advantages. Distributed Hash Table The distributed hash table relies on the overlay network and ensures the redundancy (replication/ erasure coding), consistency (Paxos), self-healing and mutability of the stored items in order to guarantee their availability and durability.
  • 16. Example: The Chord overlay network can scale to millions of nodes.
  • 17. Example: In the Kouncil overlay network, every node has complete knowledge of the nodes and blocks.
  • 18. the FLOW We’ve introduced the key-value store which is implemented through an overlay network and distributed hash table on top of which several interfaces are provided. Let’s take an example to understand how all those layers interconnect through a system call, create(“/foo”), in this case.
  • 20. the CONSENSUS Most distributed system rely on a set of managers. Those special nodes organize the network and authorize some critical operations. When such an operation must be performed, the quorum of manager nodes is contacted to reach an agreement and maintain consistency. The problem with this design is that the manager nodes can become bottlenecks, potentially leading to a cascading effect, not to mention an ideal target for privilege escalation. Infinit on the contrary, operates in a decentralized manner, meaning that no central authority is involved.
  • 21. Unlike traditional distributed systems that rely on a global lock, Infinit relies on per-block quorums. As such, the complexity of the consensus algorithm is related to the redundancy factor only, allowing for better scalability, security and fault tolerance. Distributed System Decentralized System
  • 22. the BLOCKS The key-value store allows for storing blocks of variable size. There are many types of blocks the construct allows one to store, from immutable and mutable blocks to more specific ones such as named blocks that link a string to an address. Mutable Block Such blocks evolve over time based on a version number. As such they are subject to conflicts. Also, clients need to invalidate such blocks in their cache to make sure the latest version is refetched. Finally, mutable blocks rely on heavier cryptographic mechanisms to ensure integrity and security. Immutable Block Immutable blocks, on the other hand, can only exist in one version. The address of such a block is computed by applying a one-way function on the block’s content (content addressed). Should the content be modified, the address would change as well, leading to a new block. Immutable blocks therefore cannot conflict, can be cached forever and are easy to validate by reapplying the one-way function on the retrieved content.
  • 23. For instance, in the context of a file system, metadata a.k.a inode are stored in a mutable block while actual data blocks are immutable, all stored through the key-value store. As such, a vast majority of the blocks are immutable, leading to better performance because those are cheap to write, fetchable from any source and cachable ad vitam æternam.
  • 26. the CONCLUSION This demo is really all about the key basics. From that point you can generate passports for other users to connect to your storage infrastructure, you can control accesses on files and folder, scale out with more storage capacity from the cloud and also, obviously, access an Infinit volume in your Docker containers through Infinit’s Docker Volume Plugin. Infinit is still at a very early stage of its development. In the future, we will introduce “infinit cluster” to make it easier to deploy a storage infrastructure (like Docker Swarm through —init/—join), provide more interfaces (block/object) and open source the several layers I talked about today.