Azure Storage Performance
•Download as PPTX, PDF•
5 likes•1,976 views
This document provides an overview of Azure Storage services including Blob Storage, CDN, Drive Storage, Queue Storage, Table Storage, and File Storage. It discusses key concepts such as containers, accounts, and storage artifacts. Specific services covered in more detail include Blob Storage (block blobs, page blobs, operations), Tables (entities, partitions, queries), and Drives (durability, mounting page blobs). Performance tests demonstrate benefits of multi-threaded uploads, copy operations, and batch operations versus single operations for Blobs and Tables.
Report
Share
Azure Storage Performance
- 1. Azure Storage Anton Boyko Microsoft Azure MVP boyko.ant@live.com
- 2. Agenda Azure Storage • Overview • Blob Storage • CDN • Drive Storage • Queue Storage • Table Storage • File Storage Azure Storage Performance • Blob Storage • Drive Storage • Table Storage
- 3. OVERVIEW
- 4. Storage in the cloud • Exposed via RESTful web services • Available globally
- 5. Storage SDK in many languages C#/.NET Python Ruby Perl Node.JS Java PHP Erlang Common LISP Objective- C
- 6. Storage security • HTTP or HTTPS • Two independent 512bit symmetric master keys • Shared access signatures for more granular access
- 7. Storage artifacts Blobs Drives Queues Tables Files
- 8. BLOBS
- 9. Blob storage concept Block/Page Blob Container Account contoso image pic01.jpg block01 block02 pic02.jpg video mov01.avi page01 page02 https://contoso.blob.core.windows.net/image/pic01.jpg
- 10. Blobs main operations • Get • Put • Delete • Copy
- 11. Quiz time Question Why do I need a “copy” command if it will do pretty much the same as “get” and “put” do together? Answer
- 12. Quiz time Question Why do I need a “copy” command if it will do pretty much the same as “get” and “put” do together? Answer “Copy” command is server- side copy. I do not need to download and upload my blob. Also it’s possible to use different storage accounts for source and destination (even in different locations).
- 13. Blobs main operations • Get • Put • Delete • Copy • Snapshot • Lease
- 15. Block blobs Benefit Efficient continuation and retry Parallel and out of order upload of blocks
- 17. Quiz time Question How can we increase upload speed for page blobs? What is the preferred way to upload 1TB VHD to Azure? Answer
- 18. Quiz time Question How can we increase upload speed for page blobs? What is the preferred way to upload 1TB VHD to Azure? Answer Upload in many threads. Upload only non-zero pages.
- 20. Content Delivery Network • High-bandwidth global content delivery • Many origins – Web App – Cloud Service – Storage Account
- 21. Quiz time Question Can I use CDN for my VM’s in Azure? Answer
- 22. Quiz time Question Can I use CDN for my VM’s in Azure? Answer • Yes. All VMs sit behind Cloud Services.
- 23. Content Delivery Network • High-bandwidth global content delivery • Many origins – Web App – Cloud Service – Storage Account • New origin – Custom URL
- 26. DRIVES
- 27. Azure Drives • Durable NTFS volume for Azure Instances – Use existing NTFS APIs to access a network-attached durable drive – Use System.IO from .NET • Benefits – Move existing apps that are using NTFS more easily to the cloud – Durability and survival of data on instance recycle – Drives can be up to 1TB • Azure Drive is NTFS VHD Page Blob – Mounts Page Blob over the network as NTFS drive – Local cache on instance for read operations – All flushed and unbuffered writes to drive are made durable to the Page Blob
- 28. TABLES
- 29. Table storage concept Entity Table Account contoso users FName=Bob LName=Smith FName=Inga Phone=123 groups Name=Admin
- 30. Table details Table • Create • Query • Delete Entity • Insert • Update – Merge – Replace • Upsert • Query • Delete
- 31. Entity properties • Entity can have up to 255 properties – Up to 1MB per entity • Mandatory Properties for every entity – PartitionKey & RowKey (only indexed properties) • Uniquely identifies an entity • Defines the sort order – Timestamp • Optimistic Concurrency • Exposed as an HTTP Etag • No fixed schema for other properties – Each property is stored as a <name, typed value> pair – No schema stored for a table – Properties can be the standard .NET types String, binary, bool, DateTime, GUID, int and double
- 32. Purpose of Partition Key • Entity Locality – Entities in the same partition will be stored together – Efficient querying and cache locality – Endeavour to include partition key in all queries • Entity Group Transactions – Atomic multiple Insert/Update/Delete in the same partition in a single transaction • Table Scalability – Target throughput – 2 000 tps/partition, 20 000 tps/account – Azure monitors usage patterns of partitions – Automatically load balance partitions – Each partition can be served by a different storage node – Scale to meet traffic needs of your table
- 33. Poker scoring table By players • PK = PlayerID • RK = GameID • Score = 42 By games • PK = GameID • RK = PlayerID • Score = 42 there is no silver bullet
- 35. Blog tables Posts • PK = Date • RK = Time • Authors • Tags • Text PostsByAuthors • PK = UserID • RK = Post PK + RK PostsByTags • PK = Tag • RK = Post PK + RK
- 36. IT’S SHOW TIME
- 37. DRIVES
- 38. Single drive
- 39. 4 * 1/4 drives
- 40. BLOBS
- 41. Upload single-thread Mbytes uploaded: 142.49238300323486 total time in ms: 7377 avg ms per Mbyte: 51.771188357714024 Mbytes uploaded: 1424.9273290634155 total time in ms: 70467 avg ms per Mbyte: 49.45304828023543
- 42. Upload multi-thread total files uploaded: 365 total Mbytes uploaded: 142.4734115600586 total time in ms: 4576 avg ms per Mbyte: 32.11827350727137 total files uploaded: 365 total Mbytes uploaded: 1424.9266424179077 total time in ms: 36323 avg ms per Mbyte: 25.491136819762723
- 43. Copy blob Mbytes uploaded: 127000 total time in ms: 177 avg ms per Mbyte: 0.0013937007874015748 Mbytes uploaded: 127000 total time in ms: 1836 avg ms per Mbyte: 0.014456692913385827
- 44. TABLES
- 45. Insert single entity records inserted: 500 total time in ms: 2920 avg ms per record: 5.84
- 46. Insert batch of 50 records inserted: 500 total time in ms: 470 avg ms per record: 0.94
- 47. Insert batch of 100 (9100 times) records inserted: 910000 total time in ms: 1072417 (~18 min) avg ms per record: 1.1784802197802198
- 48. Select all single-thread records selected: 1500 queries count: 2 total time in ms: 274 avg ms per record: 0.18266666666666667
- 49. Select single partition records selected: 5000 queries count: 5 total time in ms: 815 avg ms per record: 0.163
- 50. Select all single-thread records selected: 1142100 queries count: 1143 total time in ms: 335824 (~6 min) avg ms per record: 0.29404080203134575
- 51. Select all multi-thread records selected: 1142100 queries count partition: 306 queries count data: 1259 total time in ms: 48134 (~48 sec) avg ms per record: 0.04214517117590403 total http queries: +37% total time in ms: -76% avg ms per record: -76%
- 52. Select by custom property records selected: 284927 queries count: 285 total time in ms: 74065 avg ms per record: 0.2599437750722115
- 53. Questions Anton Boyko Microsoft Azure MVP boyko.ant@live.com facebook.com/boyko.ant youtube.com/user/boykoant Google: boykoant
Editor's Notes
- Q: What is MVP?
- Q: Why do we need copy as a separate command?
- Q: Why do we need copy as a separate command?
- Q: Can I use CDN for VM?
- Q: Can I use CDN for VM?