The AWS Handbook: Learn the Ins and Outs of AWS S3 Glacier Deep Archive | Randomskool | AWS Lecture Series

Welcome to today's class

Today's topic: AWS S3 Glacier Deep Archive

Professor:

Hello, students. Today, we will be discussing AWS S3 Glacier Deep Archive.

Student:

What is AWS S3 Glacier Deep Archive, Professor?

Professor:

AWS S3 Glacier Deep Archive is a secure, durable, and extremely low-cost Amazon S3 storage class for data archiving and long-term backup. It is designed to store data that is infrequently accessed and for which retrieval times of several hours are acceptable.

Student:

How does it differ from other storage classes on AWS?

Professor:

S3 Glacier Deep Archive has the lowest storage cost and longest retrieval times of all the S3 storage classes. It is ideal for storing large amounts of data that are not accessed frequently, such as raw data from scientific instruments, audio and video files, and documents that are retained for regulatory compliance.

Student:

Can we retrieve data from S3 Glacier Deep Archive quickly?

Professor:

Data retrieval from S3 Glacier Deep Archive can take several hours, so it is not suitable for data that needs to be accessed quickly. However, you can use S3 Lifecycle policies to transition data to a more highly available storage class if the data needs to be accessed more frequently.

Student:

Is data stored in S3 Glacier Deep Archive secure?

Professor:

Yes, data stored in S3 Glacier Deep Archive is encrypted and stored in multiple availability zones to ensure durability and security. It also offers multiple options for encrypting data at rest and in transit.

Student:

How do we access data stored in S3 Glacier Deep Archive?

Professor:

You can access data stored in S3 Glacier Deep Archive using the AWS Management Console, the AWS SDKs, or the Amazon S3 REST API. You can also use the AWS CLI or AWS CloudFormation templates to automate data access.

Student:

Thank you for the explanation, Professor. Can we use S3 Glacier Deep Archive for all types of data?

Professor:

S3 Glacier Deep Archive is best suited for storing large amounts of data that are not accessed frequently and for which retrieval times of several hours are acceptable. It may not be the best choice for data that needs to be accessed quickly or for data that is updated frequently. However, you can use S3 Lifecycle policies to automatically transition data to a more appropriate storage class based on your access patterns.

Professor:

One important thing to keep in mind is that there are certain charges associated with retrieving data from S3 Glacier Deep Archive. There are three retrieval tiers: Standard, Bulk, and Expedited. Standard retrieval has the lowest cost but the longest retrieval time, while Expedited retrieval has the highest cost but the shortest retrieval time.

Student:

How do we choose the right retrieval tier for our data?

Professor:

It depends on your access needs. If you need to retrieve your data within several hours, you can use the Standard retrieval tier. If you need to retrieve your data within minutes, you can use the Expedited retrieval tier. The Bulk retrieval tier is intended for retrieving large amounts of data, and it has a lower cost than the Standard retrieval tier but a longer retrieval time.

Student:

Can we store any type of data in S3 Glacier Deep Archive?

Professor:

Yes, you can store any type of data in S3 Glacier Deep Archive, including documents, images, audio and video files, and raw data from scientific instruments. S3 Glacier Deep Archive is designed to store data that is infrequently accessed and for which retrieval times of several hours are acceptable.

Student:

Is there a size limit for the data we can store in S3 Glacier Deep Archive?

Professor:

There is no size limit for individual objects stored in S3 Glacier Deep Archive, but there is a maximum storage capacity for each AWS account. You can store an unlimited amount of data in S3 Glacier Deep Archive as long as you have sufficient storage capacity in your AWS account.

Student:

How do we determine the cost of storing data in S3 Glacier Deep Archive?

Professor:

The cost of storing data in S3 Glacier Deep Archive depends on the volume of data you store, the duration for which you store the data, and the retrieval tier you choose. You can use the AWS Simple Monthly Calculator to estimate the cost of storing data in S3 Glacier Deep Archive based on your specific requirements.

Student:

Can we set expiration dates for objects stored in S3 Glacier Deep Archive?

Professor:

Yes, you can use S3 Lifecycle policies to specify an expiration date for objects stored in S3 Glacier Deep Archive. Once the expiration date is reached, the objects will be deleted automatically, freeing up storage space in your AWS account. You can also use S3 Lifecycle policies to transition data to a more highly available storage class if needed.

Professor:

One advanced feature of S3 Glacier Deep Archive is the ability to store data using additional data redundancy. By default, S3 Glacier Deep Archive stores data in multiple availability zones to ensure durability, but you can choose to store your data with even more redundancy by using the S3 Reduced Redundancy Storage (RRS) option.

Student:

How does the S3 Reduced Redundancy Storage option work?

Professor:

The S3 Reduced Redundancy Storage option stores your data with fewer copies than the default storage option, which can lower the cost of storing data in S3 Glacier Deep Archive. However, it also means that your data is less durable and may be at higher risk of data loss. It is recommended to use the S3 Reduced Redundancy Storage option only for data that is easily reproducible or that can be reconstructed from other sources.

Student:

Can we use S3 Glacier Deep Archive to store data from different regions?

Professor:

Yes, you can store data from different regions in S3 Glacier Deep Archive by using the cross-region replication feature. This feature allows you to automatically replicate data from one region to another, so you can store your data in the region that is most cost-effective for you.

Student:

How do we set up cross-region replication for S3 Glacier Deep Archive?

Professor:

To set up cross-region replication for S3 Glacier Deep Archive, you first need to enable versioning for your S3 bucket. Then, you can use the AWS Management Console or the AWS CLI to set up cross-region replication by specifying the source bucket and the destination bucket, as well as the replication rules. You can also use AWS CloudFormation templates to automate the process.

Student:

Can we use S3 Glacier Deep Archive with other AWS services?

Professor:

Yes, you can use S3 Glacier Deep Archive with a variety of other AWS services to build complex data storage and management solutions. For example, you can use S3 Glacier Deep Archive as the storage layer for Amazon Elasticsearch, Amazon Redshift, and Amazon RDS, or you can use it as the storage layer for a data lake built with Amazon EMR and Amazon Athena.

Student:

Is there a way to monitor the access and usage of S3 Glacier Deep Archive?

Professor:

Yes, you can use Amazon CloudWatch to monitor the access and usage of S3 Glacier Deep Archive. CloudWatch provides metrics and logs for various S3 operations, including data retrieval and deletion, as well as data transfer and storage. You can use CloudWatch to set alarms and trigger automated actions based on the metrics and logs from S3 Glacier Deep Archive.

Professor:

Another advanced feature of S3 Glacier Deep Archive is the ability to store data in a vault lock policy. A vault lock policy allows you to specify certain conditions under which data stored in S3 Glacier Deep Archive can be deleted or transitioned to a different storage class.

Student:

How does a vault lock policy work?

Professor:

A vault lock policy consists of a set of rules that define the conditions under which data stored in S3 Glacier Deep Archive can be deleted or transitioned to a different storage class. For example, you can specify that data can only be deleted or transitioned if a certain number of approvals are obtained, or if the data has been stored for a certain minimum period of time. You can also specify that data can only be deleted or transitioned if certain conditions are met, such as the data being in compliance with a regulatory requirement.

Student:

How do we create a vault lock policy for S3 Glacier Deep Archive?

Professor:

You can create a vault lock policy for S3 Glacier Deep Archive using the AWS Management Console or the AWS CLI. To create a vault lock policy, you need to specify the vault lock policy document, which is a JSON file that defines the rules for the policy. You can use the AWS Management Console to create a policy document by using the policy generator, or you can create a policy document manually by using a text editor.

Student:

Can we update or delete a vault lock policy for S3 Glacier Deep Archive?

Professor:

Yes, you can update or delete a vault lock policy for S3 Glacier Deep Archive using the AWS Management Console or the AWS CLI. To update a vault lock policy, you need to specify the new policy document, which will replace the existing policy. To delete a vault lock policy, you can use the "Delete Vault Lock" operation. However, keep in mind that a vault lock policy cannot be deleted if it is currently in effect for any objects stored in S3 Glacier Deep Archive.

Student:

Can we use S3 Glacier Deep Archive with Amazon Macie?

Professor:

Yes, you can use S3 Glacier Deep Archive with Amazon Macie to discover, classify, and protect sensitive data stored in your AWS account. Amazon Macie is a security service that uses machine learning to automatically discover, classify, and protect sensitive data in S3. By integrating S3 Glacier Deep Archive with Amazon Macie, you can ensure that your data is secure and compliant, even if it is infrequently accessed or stored in a low-cost storage class.

Student:

Is there a way to access data stored in S3 Glacier Deep Archive using a file system interface?

Professor:

Yes, you can use Amazon FSx for Lustre to access data stored in S3 Glacier Deep Archive using a file system interface. Amazon FSx for Lustre is a high-performance file system that allows you to access data stored in S3 using a POSIX-compliant file system interface. You can use Amazon FSx for Lustre to create a file system that is backed by S3 Glacier Deep Archive, and then access the data stored in the file system using standard file system tools and APIs.

Professor:

One way to access data stored in S3 Glacier Deep Archive is by using the AWS SDKs. The AWS SDKs are libraries that provide APIs for various programming languages, such as Java, Python, and .NET, that you can use to access data stored in S3 Glacier Deep Archive.

Student:

Can you give us an example of how to access data stored in S3 Glacier Deep Archive using the AWS SDKs?

Professor:

Sure. Here is an example of how to retrieve an object stored in S3 Glacier Deep Archive using the AWS SDK for Java:

 import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.model.GetObjectRequest; import com.amazonaws.services.s3.model.S3Object; AmazonS3 s3Client = AmazonS3ClientBuilder.standard().build(); GetObjectRequest getObjectRequest = new GetObjectRequest("my-bucket", "my-object"); getObjectRequest.setRequesterPays(true); S3Object s3Object = s3Client.getObject(getObjectRequest); 

This code uses the GetObjectRequest class to specify the bucket and object that you want to retrieve, and the getObject method to initiate the retrieval request. The setRequesterPays method is used to indicate that the requester (i.e., the person making the request) will pay the retrieval fee.

Student:

Can we also access data stored in S3 Glacier Deep Archive using the AWS CLI?

Professor:

Yes, you can use the AWS CLI to access data stored in S3 Glacier Deep Archive. Here is an example of how to retrieve an object stored in S3 Glacier Deep Archive using the AWS CLI:

 aws s3 cp s3://my-bucket/my-object . --request-payer requester 

This command uses the cp command to copy the object from the specified bucket to the current directory, and the --request-payer option to indicate that the requester will pay the retrieval fee.

Student:

Is there a way to automate data retrieval from S3 Glacier Deep Archive using the AWS CLI?

Professor:

Yes, you can use the AWS CLI to automate data retrieval from S3 Glacier Deep Archive by using shell scripts or other automation tools. For example, you can use the aws s3 sync command to synchronize a local directory with a bucket stored in S3 Glacier Deep Archive, or you can use the aws s3 cp command in combination with a loop and a list of objects to retrieve multiple objects in batch.

Student:

How do we specify the retrieval tier when using the AWS CLI to retrieve data from S3 Glacier Deep Archive?

Professor:

To specify the retrieval tier when using the AWS CLI to retrieve data from S3 Glacier Deep Archive, you can use the --expected-bulk-request-charge option. This option allows you to specify the expected cost of the retrieval request in dollars, and the AWS CLI will automatically choose the appropriate retrieval tier based on the expected cost. For example, to retrieve an object using the Bulk retrieval tier, you can use the following command:

 aws s3 cp s3://my-bucket/my-object . --request-payer requester --expected-bulk-request-charge 0.01 

This command specifies an expected cost of $0.01 for the retrieval request, which will cause the AWS CLI to use.

Conclusion

Professor:

In summary, we covered the following topics related to AWS S3 Glacier Deep Archive: * S3 Glacier Deep Archive is a secure, durable, and extremely low-cost storage class for data archiving and long-term backup on AWS. * S3 Glacier Deep Archive has the lowest storage cost and longest retrieval times of all the S3 storage classes. * You can use S3 Lifecycle policies to transition data to a more highly available storage class if needed, or to specify an expiration date for objects stored in S3 Glacier Deep Archive. * You can access data stored in S3 Glacier Deep Archive using the AWS Management Console, the AWS SDKs, or the Amazon S3 REST API. * You can use the AWS Simple Monthly Calculator to estimate the cost of storing data in S3 Glacier Deep Archive. * You can use Amazon CloudWatch to monitor the access and usage of S3 Glacier Deep Archive. * You can use a vault lock policy to specify certain conditions under which data stored in S3 Glacier Deep Archive can be deleted or transitioned to a different storage class. * You can use the AWS SDKs or the AWS CLI to access data stored in S3 Glacier Deep Archive. * You can use Amazon FSx for Lustre to access data stored in S3 Glacier Deep Archive using a file system interface. * You can use Amazon Macie to discover, classify, and protect sensitive data stored in S3 Glacier Deep Archive. I hope this class has been helpful in understanding the features and capabilities of S3 Glacier Deep Archive. If you have any questions, don't hesitate to ask. Have a good day!

We welcome your feedback on this lecture series. Please share any thoughts or suggestions you may have.

To view the full lecture series, please visit this link.

Share this post