SlideShare a Scribd company logo

gRPC Design and Implementation

•
•
Varun Talwar
Varun Talwar

The document discusses gRPC, an open-source framework for building distributed applications and services. It was motivated by Google's experience with internal RPC systems handling over 1010 RPCs per second. gRPC uses HTTP/2 for transport, allowing it to take advantage of existing network infrastructure while enabling features like bidirectional streaming. It generates client and server code from IDL definitions, supporting multiple languages.

Related slideshows

GRPC.pptx
GRPC.pptxGRPC.pptx
GRPC.pptx
 
Building microservices with grpc
Building microservices with grpcBuilding microservices with grpc
Building microservices with grpc
 
REST vs gRPC: Battle of API's
REST vs gRPC: Battle of API'sREST vs gRPC: Battle of API's
REST vs gRPC: Battle of API's
 
1 of 20
Download to read offline
gRPC Design and Implementation gRPC Team
@grpcio Motivation for RPC systems ● Large-scale distributed systems actually composed of microservices ○ Allows loosely-coupled and even multilingual development ○ Scalability: things, cores, devices, nodes, clusters, and data centers (DCs) ● Communication predominantly structured as RPCs ○ Many models of RPC communication ○ Terminology: Client uses a stub to call a method running on a service/server ○ Easiest interfaces (synchronous, unary) resemble local procedure calls translated to network activity by code generator and RPC library ○ High-performance interfaces (async, streaming) look like Active Messaging ● Long way from textbook description of RPCs!
@grpcio Application composed of microservices Data Store Task Server Computation Task Server Stub UI Task Stub Monitoring Task Server Stub
@grpcio gRPC: Motivation ● Google has had 4 generations of internal RPC systems, called Stubby ○ All production applications and systems built using RPCs ○ Over 1010 RPCs per second, fleetwide ○ APIs for C++, Java, Python, Go ○ Not suitable for open-source community! (Tight coupling with internal tools) ● Apply scalability, performance, and API lessons to external open-source
@grpcio gRPC: Summary ● Multi-language, multi-platform framework ○ Native implementations in C, Java, and Go ○ C stack wrapped by C++, C#, Node, ObjC, Python, Ruby, PHP ○ Platforms supported: Linux, Android, iOS, MacOS, Windows ○ This talk: focus on C++ API and implementation (designed for performance) ● Transport over HTTP/2 + TLS ○ Leverage existing network protocols and infrastructure ○ Efficient use of TCP - 1 connection shared across concurrent framed streams ○ Native support for secure bidirectional streaming ● C/C++ implementation goals ○ High throughput and scalability, low latency ○ Minimal external dependencies
Using HTTP/2 as a transport
@grpcio Using an HTTP transport: Why and How ● Network infrastructure well-designed to support HTTP ○ Firewalls, load balancers, encryption, authentication, compression, ... ● Basic idea: treat RPCs as references to HTTP objects ○ Encode request method name as URI ○ Encode request parameters as content ○ Encode return value in HTTP response
@grpcio ● Request-Response protocol ○ Each connection supports pipelining ○ … but not parallelism (in-order only) ○ Need multiple connections per client-server pair to avoid in-order stalls across multiple requests → multiple CPU-intensive TLS handshakes, higher memory footprint ● Content may be compressed ○ … but headers are text format ● Naturally supports single-direction streaming ○ … but not bidirectional Using an HTTP/1.1 transport and its limitations
@grpcio ● One TCP connection for each client-server pair ● Request → Stream ○ Streams are multiplexed using framing ● Compact binary framing layer ○ Prioritization ○ Flow control ○ Server push ● Header compression ● Directly supports bidirectional streaming HTTP/2 in a Nutshell
HTTP/2HTTP/1.1 http://www.http2demo.io/
gRPC
@grpcio ● IDL to describe service API ● Automatically generates client stubs and abstract server classes in 10+ languages ● Takes advantage of feature set of HTTP/2 gRPC in a nutshell
@grpcio ● Google’s Lingua Franca for serializing data: RPCs and storage ● Binary data representation ● Structures can be extended and maintain backward compatibility ● Code generators for many languages ● Strongly typed ● Not required for gRPC, but very handy An Aside: Protocol Buffers syntax = “proto3”; message Person { string name = 1; int32 id = 2; string email = 3; enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; } message PhoneNumber { string number = 1; PhoneType type = 2; } repeated PhoneNumber phone = 4; }
@grpcio Example gRPC client/server architecture
@grpcio Getting Started Define a service in a .proto file using Protocol Buffers IDL Generate server and client stub code using the protocol buffer compiler Extend the generated server class in your language to fill in the logic of your service Invoke it using the generated client stubs
@grpcio service RouteGuide { rpc GetFeature(Point) returns (Feature); rpc RouteChat(stream RouteNote) returns (stream RouteNote); } Example Service Definition message Point { int32 Latitude = 1; int32 Longitude = 2; } message RouteNote { Point location = 1; string message = 2; } message Feature { string name = 1; Point location = 2; }
@grpcio An (anonymized) case study ● Service needs to support bidirectional streaming with clients ● Attempt 1: Directly use TCP sockets ○ Functional in production data center, but not on Internet (firewalls, etc) ○ Programmer responsible for all network management and data transfer ● Attempt 2: JSON-based RPC over two HTTP/1.1 connections ○ Start two: one for request streaming and one for response streaming ○ But they might end up load-balanced to different back-end servers, so the backing servers require shared state ○ Can only support 1 streaming RPC at a time for a client-server pair ● Attempt 3: gRPC ○ Natural fit
@grpcio Authentication SSL/TLS gRPC has SSL/TLS integration and promotes the use of SSL/TLS to authenticate the server, and encrypt all the data exchanged between the client and the server. Optional mechanisms are available for clients to provide certificates to accomplish mutual authentication. OAuth 2.0 gRPC provides a generic mechanism to attach metadata to requests and responses. Can be used to attach OAuth 2.0 Access Tokens to RPCs being made at a client.
Wrap-up
@grpcio grpc is Open Source We welcome your help! http://grpc.io/contribute https://github.com/grpc irc.freenode.net #grpc @grpcio grpc-io@googlegroups.com Occasional public meetups with free pizza

More Related Content

gRPC Design and Implementation

  • 1. gRPC Design and Implementation gRPC Team
  • 2. @grpcio Motivation for RPC systems ● Large-scale distributed systems actually composed of microservices ○ Allows loosely-coupled and even multilingual development ○ Scalability: things, cores, devices, nodes, clusters, and data centers (DCs) ● Communication predominantly structured as RPCs ○ Many models of RPC communication ○ Terminology: Client uses a stub to call a method running on a service/server ○ Easiest interfaces (synchronous, unary) resemble local procedure calls translated to network activity by code generator and RPC library ○ High-performance interfaces (async, streaming) look like Active Messaging ● Long way from textbook description of RPCs!
  • 3. @grpcio Application composed of microservices Data Store Task Server Computation Task Server Stub UI Task Stub Monitoring Task Server Stub
  • 4. @grpcio gRPC: Motivation ● Google has had 4 generations of internal RPC systems, called Stubby ○ All production applications and systems built using RPCs ○ Over 1010 RPCs per second, fleetwide ○ APIs for C++, Java, Python, Go ○ Not suitable for open-source community! (Tight coupling with internal tools) ● Apply scalability, performance, and API lessons to external open-source
  • 5. @grpcio gRPC: Summary ● Multi-language, multi-platform framework ○ Native implementations in C, Java, and Go ○ C stack wrapped by C++, C#, Node, ObjC, Python, Ruby, PHP ○ Platforms supported: Linux, Android, iOS, MacOS, Windows ○ This talk: focus on C++ API and implementation (designed for performance) ● Transport over HTTP/2 + TLS ○ Leverage existing network protocols and infrastructure ○ Efficient use of TCP - 1 connection shared across concurrent framed streams ○ Native support for secure bidirectional streaming ● C/C++ implementation goals ○ High throughput and scalability, low latency ○ Minimal external dependencies
  • 6. Using HTTP/2 as a transport
  • 7. @grpcio Using an HTTP transport: Why and How ● Network infrastructure well-designed to support HTTP ○ Firewalls, load balancers, encryption, authentication, compression, ... ● Basic idea: treat RPCs as references to HTTP objects ○ Encode request method name as URI ○ Encode request parameters as content ○ Encode return value in HTTP response
  • 8. @grpcio ● Request-Response protocol ○ Each connection supports pipelining ○ … but not parallelism (in-order only) ○ Need multiple connections per client-server pair to avoid in-order stalls across multiple requests → multiple CPU-intensive TLS handshakes, higher memory footprint ● Content may be compressed ○ … but headers are text format ● Naturally supports single-direction streaming ○ … but not bidirectional Using an HTTP/1.1 transport and its limitations
  • 9. @grpcio ● One TCP connection for each client-server pair ● Request → Stream ○ Streams are multiplexed using framing ● Compact binary framing layer ○ Prioritization ○ Flow control ○ Server push ● Header compression ● Directly supports bidirectional streaming HTTP/2 in a Nutshell
  • 11. gRPC
  • 12. @grpcio ● IDL to describe service API ● Automatically generates client stubs and abstract server classes in 10+ languages ● Takes advantage of feature set of HTTP/2 gRPC in a nutshell
  • 13. @grpcio ● Google’s Lingua Franca for serializing data: RPCs and storage ● Binary data representation ● Structures can be extended and maintain backward compatibility ● Code generators for many languages ● Strongly typed ● Not required for gRPC, but very handy An Aside: Protocol Buffers syntax = “proto3”; message Person { string name = 1; int32 id = 2; string email = 3; enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; } message PhoneNumber { string number = 1; PhoneType type = 2; } repeated PhoneNumber phone = 4; }
  • 15. @grpcio Getting Started Define a service in a .proto file using Protocol Buffers IDL Generate server and client stub code using the protocol buffer compiler Extend the generated server class in your language to fill in the logic of your service Invoke it using the generated client stubs
  • 16. @grpcio service RouteGuide { rpc GetFeature(Point) returns (Feature); rpc RouteChat(stream RouteNote) returns (stream RouteNote); } Example Service Definition message Point { int32 Latitude = 1; int32 Longitude = 2; } message RouteNote { Point location = 1; string message = 2; } message Feature { string name = 1; Point location = 2; }
  • 17. @grpcio An (anonymized) case study ● Service needs to support bidirectional streaming with clients ● Attempt 1: Directly use TCP sockets ○ Functional in production data center, but not on Internet (firewalls, etc) ○ Programmer responsible for all network management and data transfer ● Attempt 2: JSON-based RPC over two HTTP/1.1 connections ○ Start two: one for request streaming and one for response streaming ○ But they might end up load-balanced to different back-end servers, so the backing servers require shared state ○ Can only support 1 streaming RPC at a time for a client-server pair ● Attempt 3: gRPC ○ Natural fit
  • 18. @grpcio Authentication SSL/TLS gRPC has SSL/TLS integration and promotes the use of SSL/TLS to authenticate the server, and encrypt all the data exchanged between the client and the server. Optional mechanisms are available for clients to provide certificates to accomplish mutual authentication. OAuth 2.0 gRPC provides a generic mechanism to attach metadata to requests and responses. Can be used to attach OAuth 2.0 Access Tokens to RPCs being made at a client.
  • 20. @grpcio grpc is Open Source We welcome your help! http://grpc.io/contribute https://github.com/grpc irc.freenode.net #grpc @grpcio grpc-io@googlegroups.com Occasional public meetups with free pizza