SlideShare a Scribd company logo

AWS S3 and GLACIER

•
•
Mahesh Raj
Mahesh Raj

S3 provides scalable object storage. Objects are stored in buckets and consist of data and metadata. S3 offers 99.99999999% durability and 99.99% availability. Objects can range from 0 bytes to 5TB in size. Buckets organize objects and have restrictions like a 100 bucket limit per account. Objects have keys, versions, metadata and access controls. S3 offers various storage classes and encryption options. Other features include versioning, lifecycle policies, transfer acceleration, cross-region replication and multipart uploads.

Related slideshows

Encryption and Key Management in AWS
Encryption and Key Management in AWSEncryption and Key Management in AWS
Encryption and Key Management in AWS
 
AWS Connectivity, VPC Design and Security Pro Tips
AWS Connectivity, VPC Design and Security Pro TipsAWS Connectivity, VPC Design and Security Pro Tips
AWS Connectivity, VPC Design and Security Pro Tips
 
Object Storage: Amazon S3 and Amazon Glacier
Object Storage: Amazon S3 and Amazon GlacierObject Storage: Amazon S3 and Amazon Glacier
Object Storage: Amazon S3 and Amazon Glacier
 
1 of 16
Download to read offline
Simple Service Storage (S3)
Mahesh TR What is S3 Buckets are used to store objects, which consist of data and metadata that describes the data. Object storage Unlimited storage High availability Durability 99.99999999% Availability 99.99% Object size 0 byte to 5TB
Mahesh TR Buckets ✓Buckets are main storage containers of S3 ✓Tags can be used to organize buckets ✓Each Buckets must have unique name across ALL of AWS. ✓S3 can be used for hosting static websites and used with route53 ✓S3 can act as origin to the CloudFront. ✓Bucket Limitations Only 100 Buckets can be created at time. Bucket Ownership can not be transferred once it is created.
Mahesh TR S3 Objects • S3 is object based, objects consist of the following ✓Key (This is simply the name of the object) ✓Value (This is simply the data and made up of sequence of bytes) ✓Version ID (Important for versioning) ✓Metadata (Data about the data you are storing) ✓Subresources ➢ACL
Mahesh TR Objects ✓Objects are static files which contains meta data ✓Objects stay with in AWS region and synced across all AZ’s (depends on the storage types) ✓Read after write consistency for PUTS of new object ✓Eventually consistency for overwrite PUTS and DELETE (can take some time to propagate) ✓Each object must be assigned a storage type, which defines the object availability, durability and cost. ✓By Default all objects are Private ✓Objects as small as 0 bytes to as large as 5TB ✓Can have multiple versions. “If enabled” ✓Can Encrypt – AES 256 and AWS KMS ✓Automatically switch to different versions
Mahesh TR Object key Amazon S3 data model is a flat structure: you create a bucket, and the bucket stores objects. Amazon S3 supports buckets and objects, there is no hierarchy in Amazon S3. However, the prefixes and delimiters in an object key name, enables the Amazon S3 console and the AWS SDKs to infer hierarchy and introduce concept of folders. ✓ Development/Projects1.xls ✓ Finance/statement1.pdf ✓ Private/taxdocument.pdf ✓ s3-dg.pdf The console uses the key name prefixes (Development/, Finance/, and Private/) and delimiter ('/') to present a folder structure as shown: The s3-dg.pdf key does not have a prefix, so its object appears directly at the root level of the bucket. If you open the Development/ folder, you will see the Projects.xls object in it.
Mahesh TR Permissions ✓All objects and buckets are private by default ✓Bucket policies (which are attached only to Buckets, not IAM users) ✓Both Buckets and Objects has ACLs. ✓Public content (downloadable by URL) ✓Grant permissions to other AWS account
Mahesh TR Storage class Object availability Object durability Frequency of access ✓ Standard (OA = 99.99% and OD= 99.999999999% ) Durable, immediately available & frequently accessed. ✓ Reduced Redundancy Storage (OA= 99.99% and 0D= 99.99%) Data that is easily reproducible, such as thumb nails etc.. ✓ Infrequent access S3-IA (OA = 99.90% and OD= 99.999999999% ) Durable, immediately available & infrequently accessed. ✓ Glacier (OD= 99.999999999% ) Archived data, where you can wait 3 to 5 hrs. before accessing.
Mahesh TR Versioning ✓Store all versions of an object including deleted ✓Once versioning is enabled it cannot be disabled, only suspended. ✓Integrate with lifecycle rules. ✓MFA delete compatibility, which uses multi-factor authentication, can be used to provide an additional layer ✓Cross zone replication, requires versioning enabled on the source bucket.
Mahesh TR Life cycle policies ✓Can be used to conjunction with versioning. ✓Can be applied to current version or previous version. ✓Following actions can be done. ➢ Transition to the standard – Infrequent access storage class (30 days after the creation date) ➢ Archive to Glacier storage (30 days after IA, if relevant) ➢ Permanently Delete
Mahesh TR Encryption ✓ In Trasit ➢ SSL/TLS ✓ At Rest ➢ S3 Managed keys – SSE-S3 ➢ AWS Key Management Service, Managed Keys – SSE – KMS ➢ Server Side Encryption with customer Provided keys- SSE-C
Mahesh TR S3 Transfer acceleration Amazon Simple Storage Service (Amazon S3) transfer acceleration enables fast, easy, and secure transfers of files between your client and an S3 bucket over long distances. This topic describes how to enable Amazon S3 transfer acceleration for a bucket. https://s3-accelerate-speedtest.s3-accelerate.amazonaws.com/en/accelerate-speed-comparsion.html
Mahesh TR Cross-Region Replication (CRR) Cross-region replication is a bucket-level configuration that enables automatic, asynchronous copying of objects across buckets in different AWS Regions. We refer to these buckets as sourcebucket and destination bucket. These buckets can be owned by different AWS accounts. ✓ Replicas have the same key names and the same metadata—for example, creation time, user-defined metadata, and version ID. ✓ Amazon S3 stores object replicas using the same storage class as the source object, unless you explicitly specify a different storage class in the replication configuration. ✓ Assuming that the object replica continues to be owned by the source object owner, when Amazon S3 initially replicates objects, it also replicates the corresponding object access control list (ACL).
Mahesh TR Multipart Upload ✓Amazon S3 Multipart Upload which allows faster, more flexible uploads into Amazon S3. Multipart Upload allows you to upload a single object as a set of parts. After all parts of your object are uploaded, Amazon S3 then presents the data as a single object. ✓You must be allowed to perform the s3:PutObject action on an object to initiate multipart upload. ✓The bucket owner can allow other principals to perform the s3:PutObject action
Mahesh TR
THANKS !

More Related Content

AWS S3 and GLACIER

  • 2. Mahesh TR What is S3 Buckets are used to store objects, which consist of data and metadata that describes the data. Object storage Unlimited storage High availability Durability 99.99999999% Availability 99.99% Object size 0 byte to 5TB
  • 3. Mahesh TR Buckets ✓Buckets are main storage containers of S3 ✓Tags can be used to organize buckets ✓Each Buckets must have unique name across ALL of AWS. ✓S3 can be used for hosting static websites and used with route53 ✓S3 can act as origin to the CloudFront. ✓Bucket Limitations Only 100 Buckets can be created at time. Bucket Ownership can not be transferred once it is created.
  • 4. Mahesh TR S3 Objects • S3 is object based, objects consist of the following ✓Key (This is simply the name of the object) ✓Value (This is simply the data and made up of sequence of bytes) ✓Version ID (Important for versioning) ✓Metadata (Data about the data you are storing) ✓Subresources ➢ACL
  • 5. Mahesh TR Objects ✓Objects are static files which contains meta data ✓Objects stay with in AWS region and synced across all AZ’s (depends on the storage types) ✓Read after write consistency for PUTS of new object ✓Eventually consistency for overwrite PUTS and DELETE (can take some time to propagate) ✓Each object must be assigned a storage type, which defines the object availability, durability and cost. ✓By Default all objects are Private ✓Objects as small as 0 bytes to as large as 5TB ✓Can have multiple versions. “If enabled” ✓Can Encrypt – AES 256 and AWS KMS ✓Automatically switch to different versions
  • 6. Mahesh TR Object key Amazon S3 data model is a flat structure: you create a bucket, and the bucket stores objects. Amazon S3 supports buckets and objects, there is no hierarchy in Amazon S3. However, the prefixes and delimiters in an object key name, enables the Amazon S3 console and the AWS SDKs to infer hierarchy and introduce concept of folders. ✓ Development/Projects1.xls ✓ Finance/statement1.pdf ✓ Private/taxdocument.pdf ✓ s3-dg.pdf The console uses the key name prefixes (Development/, Finance/, and Private/) and delimiter ('/') to present a folder structure as shown: The s3-dg.pdf key does not have a prefix, so its object appears directly at the root level of the bucket. If you open the Development/ folder, you will see the Projects.xls object in it.
  • 7. Mahesh TR Permissions ✓All objects and buckets are private by default ✓Bucket policies (which are attached only to Buckets, not IAM users) ✓Both Buckets and Objects has ACLs. ✓Public content (downloadable by URL) ✓Grant permissions to other AWS account
  • 8. Mahesh TR Storage class Object availability Object durability Frequency of access ✓ Standard (OA = 99.99% and OD= 99.999999999% ) Durable, immediately available & frequently accessed. ✓ Reduced Redundancy Storage (OA= 99.99% and 0D= 99.99%) Data that is easily reproducible, such as thumb nails etc.. ✓ Infrequent access S3-IA (OA = 99.90% and OD= 99.999999999% ) Durable, immediately available & infrequently accessed. ✓ Glacier (OD= 99.999999999% ) Archived data, where you can wait 3 to 5 hrs. before accessing.
  • 9. Mahesh TR Versioning ✓Store all versions of an object including deleted ✓Once versioning is enabled it cannot be disabled, only suspended. ✓Integrate with lifecycle rules. ✓MFA delete compatibility, which uses multi-factor authentication, can be used to provide an additional layer ✓Cross zone replication, requires versioning enabled on the source bucket.
  • 10. Mahesh TR Life cycle policies ✓Can be used to conjunction with versioning. ✓Can be applied to current version or previous version. ✓Following actions can be done. ➢ Transition to the standard – Infrequent access storage class (30 days after the creation date) ➢ Archive to Glacier storage (30 days after IA, if relevant) ➢ Permanently Delete
  • 11. Mahesh TR Encryption ✓ In Trasit ➢ SSL/TLS ✓ At Rest ➢ S3 Managed keys – SSE-S3 ➢ AWS Key Management Service, Managed Keys – SSE – KMS ➢ Server Side Encryption with customer Provided keys- SSE-C
  • 12. Mahesh TR S3 Transfer acceleration Amazon Simple Storage Service (Amazon S3) transfer acceleration enables fast, easy, and secure transfers of files between your client and an S3 bucket over long distances. This topic describes how to enable Amazon S3 transfer acceleration for a bucket. https://s3-accelerate-speedtest.s3-accelerate.amazonaws.com/en/accelerate-speed-comparsion.html
  • 13. Mahesh TR Cross-Region Replication (CRR) Cross-region replication is a bucket-level configuration that enables automatic, asynchronous copying of objects across buckets in different AWS Regions. We refer to these buckets as sourcebucket and destination bucket. These buckets can be owned by different AWS accounts. ✓ Replicas have the same key names and the same metadata—for example, creation time, user-defined metadata, and version ID. ✓ Amazon S3 stores object replicas using the same storage class as the source object, unless you explicitly specify a different storage class in the replication configuration. ✓ Assuming that the object replica continues to be owned by the source object owner, when Amazon S3 initially replicates objects, it also replicates the corresponding object access control list (ACL).
  • 14. Mahesh TR Multipart Upload ✓Amazon S3 Multipart Upload which allows faster, more flexible uploads into Amazon S3. Multipart Upload allows you to upload a single object as a set of parts. After all parts of your object are uploaded, Amazon S3 then presents the data as a single object. ✓You must be allowed to perform the s3:PutObject action on an object to initiate multipart upload. ✓The bucket owner can allow other principals to perform the s3:PutObject action