SlideShare a Scribd company logo
Ing. Eduardo Castro, PhD
ecastro@grupoasesor.net

http://ecastrom.blogspot.com
http://comunidadwindows.org
http://tiny.cc/comwindows
SQL Azure Federation and Scalability
Microsoft SQL Azure
Information Platform as a Service


        Managed Service                                 Scale on Demand                          Innovate Faster


 Platform capabilities delivered as a service   Database as a utility with pay as you   Innovate with new data application
 (Database, Data Sync, Reporting…)              grow scaling                            patterns
 Reduced complexity, greater flexibility        Rely on Business-ready SLAs             Build cloud-based data solutions on a
                                                                                        familiar relational model
 Easy provisioning and deployment               Enable multi-tenant solutions
                                                                                        Build on existing developer skills and
 Auto High availability and fault tolerant      Manage multiple servers                 familiar Transact-SQL syntax
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
A relational DB in the cloud

                                                                                 SQL Azure Database

                                                                                      Data Hub

                                                                                   Others (Future)



                                             .NET Services


      Applications                                     SQL Services
                                                                                    Relational database as a service
                                                                                         Highly available, automatically maintained
                                                                 Live Services
      Windows Azure



                                                                                    Extension of the SQL Server Data Platform
                            Applications


 Windows             Windows               Windows            Others
  Server             Vista/XP               Mobile
Single Logical   Multiple Physical
  Database           Replicas


                      Replica 1




                      Replica 2
        DB


                      Replica 3
Reference       Data Sync
                             Data




  Symmetric Programming Model                                Data Hub Aggregation



• Initial services – core RDBMS capabilities with SQL Azure Database, Data Sync
• Future Offerings
    • Additional data platform capabilities: Reporting, BI
    • New services: Reference Data
Clear Feedback: “I want a database in the Cloud”




Focus on combining the best features of SQL Server running at scale
                        with low friction
Application                                                        Application                                    Application
                                         Browser                                                            Browser
                                                                                                                                               ODBC, OLEDB, ADO.N
                                        Application                                                        Application
               REST Client                                                         REST Client                                   SQL Client*   et PHP, Ruby, …




                                Cloud                                                                     Cloud




                                                       Evolves


                                                                                    HTTP+REST
               HTTP+REST




                                                                                                                 HTTP
                                          HTTP




                                                                                                                                   TDS
                                       Windows Azure                                             Windows Azure

                                         Web App                                     REST (Astoria)                Web App
Data Center




                                                                 Data Center
                                        REST Client                                  ADO.Net + EF                 SQL Client*




                           REST/SOAP + ACE Model                                                      TDS + TSQL Model



                            SDS Current                                                                SDS Next

                                                                               * Client access enabled using TDS for ODBC,
                                                                                 ADO.Net, OLEDB, PHP-SQL, Ruby, …
SQL Azure Database

                                                                        Others (Future)




                                       SQL Azure

Applications
                                             Windows Azure AppFabric

Windows Azure




                     Applications



               Windows              Others
SQL Azure Database


                     TDS      Database


Application
                                            Database

                            Database




SQL Azure Database

  Others (Future)
SQL Azure Database


                         Database
Application


                                           Max database size
              Database                         is 50 GB


                                Database
Application
              Database
Applications use standard SQL client libraries:
                                                                                                                         ODBC, ADO.Net, PHP, …
                                                                                               Application




                                                                                             Internet
                                                                                           Azure Cloud


                                                                         TDS (tcp)
          Security Boundary
                                                                                                                       Load balancer forwards „sticky‟ sessions to TDS protocol tier
                                                                                                   LB




    TDS (tcp)

                       Gateway                     Gateway                       Gateway                     Gateway                      Gateway                          Gateway


                Gateway: TDS protocol gateway, enforces AUTHN/AUTHZ policy; proxy to CloudDB
    TDS (tcp)


L                                 SQL                                       SQL                                    SQL                                           SQL


                                       Scalability and Availability: Fabric, Failover, Replication, and Load balancing
SQL Azure Federation and Scalability
Gateway Process


           TDS Endpoint         AdminSvc Endpoint            Provisioning Endpoint


          Protocol Parser


                                 Business Logic Services


                                   Connection Mgmt




L   SQL                      SQL                           SQL                       SQL


     Scalability and Availability: Fabric, Failover, Replication, and Load balancing
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
Live DNS Cluster
             Customer Browser                                                              Live DNS Svc


                1

                                           Datacenter (Sub-Region)          5
           Portal LB                                                                    Gateway LB
                                                   2
                                                                   4
                                                                                3                      6

Front-end Node                    Front-end Node              Front-end Node                     Front-end Node

Admin Portal                      Admin Portal                    Gateway                           Gateway

                                                                       7

     Backend Node                                  Backend Node                          Backend Node

SQL Server                                 SQL Server                               SQL Server
                        Mgmt.                                 Mgmt.                                         Mgmt.
                       Services              Fabric          Services                Fabric                Services
  Fabric
SQL Azure Federation and Scalability
1
                                                TDS Gateway
                                                Front-end Node
                               TDS Session                       Protocol Parser
                                                      2
                                                                  3
                                         Gateway Logic


                                                                                                          Master Cluster

                                                                                                            Master Node
                                                                                     4
                                                                                                     Master Node Components



    7
                                                  5       8                                          6

Backend Node 1                               Backend Node 2                                          Backend Node 3
  SQL Instance                                 SQL Instance                                              SQL Instance
    SQL DB                                       SQL DB                                                    SQL DB




                 Scalability and and Availability: Fabric,Failover,Replication, and Load balancing
                      Scalability Availability: Fabric, Failover, Replication, and Load balancing
Account




Server




Database
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
Challenges Today
                “Capacity Bursting“
                                                 Elastic scale – database as a
                                                 service
   Capacity




                             Average
                              Usage
                                          Time
                                                 Pay as you grow and shrink
              Compute
              # of Hr‟s




                          Average Usage          Easy to provision and manage
                                                 database
                                                 No hardware, no manual
 Promotions, events, ticket selling
 businesses are “bursts – bound” by              database administration required
 nature
 Capacity constraints limit business agility
 High costs of entry into new business
 Difficult to roll out extra capacity quickly
 Idle capacity “off-bursts” is cost
 prohibitive
T1    T2    T3    T4    T5


T6    T7    T8    T9    T10         All my data
                                  is handled by
T11
                              one DB on one server
      T12   T13   T14   T15


T16   T17   T18   T19   T20
App
− Provisioning                                              Covered by
   − Growing and shrinking capacity                         SQL Azure
                                                              today
− Managing
   − Upgrading, patching, HA for lots of databases

− Routing
   − Where is the directory?
   − How to scale it and use it?                           Coming up in
                                                            SQL Azure:
− Partition Management
                                                            Federations
   − Splitting and Merging, without loss of availability
   − Fan-out
Con-
 fig


Data1   Data2   Data3   Data4   Data5
Root

   Federation “CustData”
   (Federation Key: CustID)

    Member: [min, 100)
 AU           AU           AU
PK=5         PK=25        PK=35


    Member: [100, 488)
  AU          AU            AU
PK=105      PK=235        PK=365


    Member: [488, max)
  AU         AU            AU
PK=555     PK=2545       PK=3565
SalesDB


Federation “Orders_Fed”
 (Federation Key: CustID)


   Member: [min, max)
Create a
                                            Federation use a
                                              BIGINT as the
                                             Federation Key




COS222 What‟s New in Microsoft SQL Azure,
Establish the customerid
 as the Federation Key




     Data with the same
   customerid in these two
   tables should be treated
      as an Atomic Unit
COS222 What‟s New in Microsoft SQL Azure,
COS222 What‟s New in Microsoft SQL Azure,
SalesDB

        Products



Federation “Orders_Fed”
 (Federation Key: CustID)


   Member: [min, max)

  orders             zipcode
SalesDB

        Products


Federation “Orders_Fed”
(Federation Key: CustID)


      Member: [min, max)


 orders             zipcode


      Member: [min, 100)

ord
                    zipcode
ers

      Member: [100, max)

        ord
                    zipcode
        ers
Federation Root
                                              houses the
                                               Directory




                                             Split the Federation
                                             Member into two at
                                             the customerid 100
                                                   boundary

COS222 What‟s New in Microsoft SQL Azure,
Use Federation
     Member
   that contains
customerids < 100




   Use Federation
      Member
    that contains
 customerids > 100
App




       Member: [min, 100)

 AU           AU         AU
PK=5         PK=25      PK=56


             zipcode
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
SQL Azure Federation and Scalability
xyz.database.windows.net                                 efg.database.windows.net

   Master         prod1        Prod2                      prod2
                                                                         DR1         Master
                                                          clone
                                                          copy


                               prod2
                               clone
                               copy



     abc.database.windows.net

                               prod2
   Master         Dev1
                               clone
                               copy




            South Central US                                      North Central US

CREATE DATABASE efg.prod2clone
                abc.prod2clone
CREATE DATABASE xyz.prod2clone         AS COPY OF xyz.prod2
                                       AS COPY OF xyz.prod2
CTP1
                        SQL Azure
                        Database


                          Data Sync Service
Retail Stores             For SQL Azure       Remote Offices




                            Sync




                                                               CTP2
                On-Premises (Headquarters)

More Related Content

SQL Azure Federation and Scalability

  • 1. Ing. Eduardo Castro, PhD ecastro@grupoasesor.net http://ecastrom.blogspot.com http://comunidadwindows.org http://tiny.cc/comwindows
  • 3. Microsoft SQL Azure Information Platform as a Service Managed Service Scale on Demand Innovate Faster Platform capabilities delivered as a service Database as a utility with pay as you Innovate with new data application (Database, Data Sync, Reporting…) grow scaling patterns Reduced complexity, greater flexibility Rely on Business-ready SLAs Build cloud-based data solutions on a familiar relational model Easy provisioning and deployment Enable multi-tenant solutions Build on existing developer skills and Auto High availability and fault tolerant Manage multiple servers familiar Transact-SQL syntax
  • 6. A relational DB in the cloud SQL Azure Database Data Hub Others (Future) .NET Services Applications SQL Services Relational database as a service Highly available, automatically maintained Live Services Windows Azure Extension of the SQL Server Data Platform Applications Windows Windows Windows Others Server Vista/XP Mobile
  • 7. Single Logical Multiple Physical Database Replicas Replica 1 Replica 2 DB Replica 3
  • 8. Reference Data Sync Data Symmetric Programming Model Data Hub Aggregation • Initial services – core RDBMS capabilities with SQL Azure Database, Data Sync • Future Offerings • Additional data platform capabilities: Reporting, BI • New services: Reference Data
  • 9. Clear Feedback: “I want a database in the Cloud” Focus on combining the best features of SQL Server running at scale with low friction
  • 10. Application Application Application Browser Browser ODBC, OLEDB, ADO.N Application Application REST Client REST Client SQL Client* et PHP, Ruby, … Cloud Cloud Evolves HTTP+REST HTTP+REST HTTP HTTP TDS Windows Azure Windows Azure Web App REST (Astoria) Web App Data Center Data Center REST Client ADO.Net + EF SQL Client* REST/SOAP + ACE Model TDS + TSQL Model SDS Current SDS Next * Client access enabled using TDS for ODBC, ADO.Net, OLEDB, PHP-SQL, Ruby, …
  • 11. SQL Azure Database Others (Future) SQL Azure Applications Windows Azure AppFabric Windows Azure Applications Windows Others
  • 12. SQL Azure Database TDS Database Application Database Database SQL Azure Database Others (Future)
  • 13. SQL Azure Database Database Application Max database size Database is 50 GB Database Application Database
  • 14. Applications use standard SQL client libraries: ODBC, ADO.Net, PHP, … Application Internet Azure Cloud TDS (tcp) Security Boundary Load balancer forwards „sticky‟ sessions to TDS protocol tier LB TDS (tcp) Gateway Gateway Gateway Gateway Gateway Gateway Gateway: TDS protocol gateway, enforces AUTHN/AUTHZ policy; proxy to CloudDB TDS (tcp) L SQL SQL SQL SQL Scalability and Availability: Fabric, Failover, Replication, and Load balancing
  • 16. Gateway Process TDS Endpoint AdminSvc Endpoint Provisioning Endpoint Protocol Parser Business Logic Services Connection Mgmt L SQL SQL SQL SQL Scalability and Availability: Fabric, Failover, Replication, and Load balancing
  • 19. Live DNS Cluster Customer Browser Live DNS Svc 1 Datacenter (Sub-Region) 5 Portal LB Gateway LB 2 4 3 6 Front-end Node Front-end Node Front-end Node Front-end Node Admin Portal Admin Portal Gateway Gateway 7 Backend Node Backend Node Backend Node SQL Server SQL Server SQL Server Mgmt. Mgmt. Mgmt. Services Fabric Services Fabric Services Fabric
  • 21. 1 TDS Gateway Front-end Node TDS Session Protocol Parser 2 3 Gateway Logic Master Cluster Master Node 4 Master Node Components 7 5 8 6 Backend Node 1 Backend Node 2 Backend Node 3 SQL Instance SQL Instance SQL Instance SQL DB SQL DB SQL DB Scalability and and Availability: Fabric,Failover,Replication, and Load balancing Scalability Availability: Fabric, Failover, Replication, and Load balancing
  • 25. Challenges Today “Capacity Bursting“ Elastic scale – database as a service Capacity Average Usage Time Pay as you grow and shrink Compute # of Hr‟s Average Usage Easy to provision and manage database No hardware, no manual Promotions, events, ticket selling businesses are “bursts – bound” by database administration required nature Capacity constraints limit business agility High costs of entry into new business Difficult to roll out extra capacity quickly Idle capacity “off-bursts” is cost prohibitive
  • 26. T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 All my data is handled by T11 one DB on one server T12 T13 T14 T15 T16 T17 T18 T19 T20
  • 27. App
  • 28. − Provisioning Covered by − Growing and shrinking capacity SQL Azure today − Managing − Upgrading, patching, HA for lots of databases − Routing − Where is the directory? − How to scale it and use it? Coming up in SQL Azure: − Partition Management Federations − Splitting and Merging, without loss of availability − Fan-out
  • 29. Con- fig Data1 Data2 Data3 Data4 Data5
  • 30. Root Federation “CustData” (Federation Key: CustID) Member: [min, 100) AU AU AU PK=5 PK=25 PK=35 Member: [100, 488) AU AU AU PK=105 PK=235 PK=365 Member: [488, max) AU AU AU PK=555 PK=2545 PK=3565
  • 31. SalesDB Federation “Orders_Fed” (Federation Key: CustID) Member: [min, max)
  • 32. Create a Federation use a BIGINT as the Federation Key COS222 What‟s New in Microsoft SQL Azure,
  • 33. Establish the customerid as the Federation Key Data with the same customerid in these two tables should be treated as an Atomic Unit
  • 34. COS222 What‟s New in Microsoft SQL Azure,
  • 35. COS222 What‟s New in Microsoft SQL Azure,
  • 36. SalesDB Products Federation “Orders_Fed” (Federation Key: CustID) Member: [min, max) orders zipcode
  • 37. SalesDB Products Federation “Orders_Fed” (Federation Key: CustID) Member: [min, max) orders zipcode Member: [min, 100) ord zipcode ers Member: [100, max) ord zipcode ers
  • 38. Federation Root houses the Directory Split the Federation Member into two at the customerid 100 boundary COS222 What‟s New in Microsoft SQL Azure,
  • 39. Use Federation Member that contains customerids < 100 Use Federation Member that contains customerids > 100
  • 40. App Member: [min, 100) AU AU AU PK=5 PK=25 PK=56 zipcode
  • 50. xyz.database.windows.net efg.database.windows.net Master prod1 Prod2 prod2 DR1 Master clone copy prod2 clone copy abc.database.windows.net prod2 Master Dev1 clone copy South Central US North Central US CREATE DATABASE efg.prod2clone abc.prod2clone CREATE DATABASE xyz.prod2clone AS COPY OF xyz.prod2 AS COPY OF xyz.prod2
  • 51. CTP1 SQL Azure Database Data Sync Service Retail Stores For SQL Azure Remote Offices Sync CTP2 On-Premises (Headquarters)