AWS S3 Lifecycle Management

(This page is a work in progress. The policies defined below may still be adjusted, and are not actively deployed)

AWS S3 Lifecycle Policy Overview

AWS S3 buckets can be configured with lifecycle policies. These policies allow for automatically changing the storage class of objects based on the last time they were accessed. AWS S3 objects are typically stored in the Standard storage class, which provides easy access, but has relatively high GB/month storage costs. Other storage classes, such as Infrequent Access and Glacier are more suitable for objects that are rarely accessed. These storage classes maintain a much lower GB/month cost as compared to the Standard S3 storage class, but also incur charges for access and retrieval.

It is recommended to utilize the appropriate storage classes for your data.

• If you have data that you do not expect to access more than once a month, AWS Infrequent Access is a reasonable storage class to use.
• If you have data that you do not expect to access more than once a year, AWS Glacier is a reasonable storage class to use.

UCSC GI Automated Policy

In order to reduce monthly S3 storage costs, the UCSC GI has implemented global S3 lifecycle policies that transition objects to AWS Intelligent Tiering, which monitors S3 object access patterns and transitions objects accordingly.

• Objects uploaded to S3 will remain in the Standard storage class for 1 day, at which point they will be transitioned to Intelligent Tiering.
• Old and new S3 buckets will have this lifecycle policy automatically attached.

AWS Intelligent Tiering functionality:

• Intelligent Tiering does not change object access patterns. This means you do not need to execute special API commands to access objects.
• Intelligent Tiering does not incur charges for object retrieval from different tiers.

For more details on AWS Intelligent Tiering, see the AWS Docs