AWS S3 Lifecycle Management - Revision history

Anovak: /* Restoring Objects */

2022-03-10T20:59:42Z

Restoring Objects

← Older revision		Revision as of 13:59, 10 March 2022
Line 29:		Line 29:
	To restore an object, you can use the S3 section of the AWS web console.		To restore an object, you can use the S3 section of the AWS web console.

	You can also restore an object from the command line with the AWS CLI tool. To restore the object '''s3://bucket-name/path/to/object.dat''' ~~and make it accessible for the next week~~, you would issue the command:		You can also restore an object from the command line with the AWS CLI tool. To restore the object '''s3://bucket-name/path/to/object.dat''', you would issue the command:

	<code>		<code>
	aws s3api restore-object --restore-request ~~Days=7~~ --bucket "bucket-name" --key "path/to/object.dat"		aws s3api restore-object --restore-request "{}" --bucket "bucket-name" --key "path/to/object.dat"
	</code>		</code>

			If the object was manually put in Glacier, you would instead need <code>--restore-request "Days=7"</code>, or some other number of days.

	Note that you need to specify the bucket name and key within the bucket separately, instead of using an S3 URI.		Note that you need to specify the bucket name and key within the bucket separately, instead of using an S3 URI.

	'''Restores from Glacier are not immediate, or even particularly fast.''' [http://vignette2.wikia.nocookie.net/starwars/images/0/0e/Citadel_data_vault.png/revision/latest?cb=20161220040411 Jyn Erso has to go down to the Scarif data vault and find the right data-tape], and it takes a few hours, even if your file is small.		'''Restores from Glacier are not immediate, or even particularly fast.''' [http://vignette2.wikia.nocookie.net/starwars/images/0/0e/Citadel_data_vault.png/revision/latest?cb=20161220040411 Jyn Erso has to go down to the Scarif data vault and find the right data-tape], and it takes a few hours, even if your file is small.

Anovak: /* Restoring Objects */

2022-03-09T19:58:16Z

Restoring Objects

← Older revision		Revision as of 12:58, 9 March 2022
Line 36:		Line 36:

	Note that you need to specify the bucket name and key within the bucket separately, instead of using an S3 URI.		Note that you need to specify the bucket name and key within the bucket separately, instead of using an S3 URI.

			'''Restores from Glacier are not immediate, or even particularly fast.''' [http://vignette2.wikia.nocookie.net/starwars/images/0/0e/Citadel_data_vault.png/revision/latest?cb=20161220040411 Jyn Erso has to go down to the Scarif data vault and find the right data-tape], and it takes a few hours, even if your file is small.

Anovak: Explain how to restore

2022-03-09T19:48:13Z

Explain how to restore

← Older revision		Revision as of 12:48, 9 March 2022
Line 16:		Line 16:
	* Intelligent-Tiering '''does not''' incur charges for object retrieval from different tiers.		* Intelligent-Tiering '''does not''' incur charges for object retrieval from different tiers.
	For more details on AWS Intelligent Tiering, see the [https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ AWS Docs]		For more details on AWS Intelligent Tiering, see the [https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ AWS Docs]

			==Restoring Objects==

			If an object has not been accessed for a while, you may encounter an error like this when trying to access it:

			<code>
			An error occurred (InvalidObjectState) when calling the GetObject operation: The operation is not valid for the object's access tier
			</code>

			This means that the object is in Glacier, either because somebody put it there, or because Intelligent-Tiering moved it there after it was not accessed for a while. If you want to access it, you will need to restore it (and our AWS account will be billed for doing so).

			To restore an object, you can use the S3 section of the AWS web console.

			You can also restore an object from the command line with the AWS CLI tool. To restore the object '''s3://bucket-name/path/to/object.dat''' and make it accessible for the next week, you would issue the command:

			<code>
			aws s3api restore-object --restore-request Days=7 --bucket "bucket-name" --key "path/to/object.dat"
			</code>

			Note that you need to specify the bucket name and key within the bucket separately, instead of using an S3 URI.

Righanse at 19:49, 9 December 2021

2021-12-09T19:49:37Z

← Older revision		Revision as of 12:49, 9 December 2021
Line 8:		Line 8:

	==UCSC GI Automated Policy==		==UCSC GI Automated Policy==
	In order to reduce monthly S3 storage costs, the UCSC GI has implemented global S3 lifecycle policies that transition objects to AWS Intelligent Tiering, which monitors S3 object access patterns and transitions objects to more efficient storage classes accordingly.		In order to reduce monthly S3 storage costs, the UCSC GI has implemented global S3 lifecycle policies that transition objects to AWS Intelligent-Tiering, which monitors S3 object access patterns and transitions objects to more efficient storage classes accordingly.
	* Objects uploaded to S3 will remain in the Standard storage class for '''1''' day, at which point they will be transitioned to Intelligent Tiering.		* Objects uploaded to S3 will remain in the Standard storage class for '''1''' day, at which point they will be transitioned to Intelligent-Tiering.
	* Old and new S3 buckets will have this lifecycle policy automatically attached.		* Old and new S3 buckets will have this lifecycle policy automatically attached.

	AWS Intelligent Tiering functionality:		AWS Intelligent Tiering functionality:
	* Intelligent Tiering '''does not''' change object access patterns. This means you do not need to execute special API commands to access objects.		* Intelligent-Tiering '''does not''' change object access patterns. This means you do not need to execute special API commands to access objects.
	* Intelligent Tiering '''does not''' incur charges for object retrieval from different tiers.		* Intelligent-Tiering '''does not''' incur charges for object retrieval from different tiers.
	For more details on AWS Intelligent Tiering, see the [https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ AWS Docs]		For more details on AWS Intelligent Tiering, see the [https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ AWS Docs]

Righanse at 17:19, 9 December 2021

2021-12-09T17:19:48Z

← Older revision		Revision as of 10:19, 9 December 2021
Line 1:		Line 1:
	~~(This page is a work in progress. The policies defined below may still be adjusted, and are not actively deployed)~~

	==AWS S3 Lifecycle Policy Overview==		==AWS S3 Lifecycle Policy Overview==

Righanse at 17:18, 9 December 2021

2021-12-09T17:18:28Z

← Older revision		Revision as of 10:18, 9 December 2021
Line 2:		Line 2:

	==AWS S3 Lifecycle Policy Overview==		==AWS S3 Lifecycle Policy Overview==
	AWS S3 buckets can be configured with lifecycle policies. These policies allow for automatically changing the storage class of objects based on the last time they were accessed. AWS S3 objects are ~~typically~~ stored in the Standard storage class, which provides easy access, but has relatively high GB/month storage costs. Other storage classes, such as Infrequent Access and Glacier are more suitable for objects that are rarely accessed. These storage classes maintain a much lower GB/month cost as compared to the Standard S3 storage class, but also incur charges for access and retrieval.		AWS S3 buckets can be configured with lifecycle policies. These policies allow for automatically changing the storage class of objects based on the last time they were modified or accessed. AWS S3 objects are stored in the Standard storage class by default, which provides easy access, but has relatively high GB/month storage costs. Other storage classes, such as Infrequent Access and Glacier are more suitable for objects that are rarely accessed. These storage classes maintain a much lower GB/month cost as compared to the Standard S3 storage class, but also incur charges for access and retrieval.

	It is recommended to utilize the appropriate storage classes for your data.		It is recommended to utilize the appropriate storage classes for your data.
Line 9:		Line 9:

	==UCSC GI Automated Policy==		==UCSC GI Automated Policy==
	In order to reduce monthly S3 storage costs, the UCSC GI has implemented global S3 lifecycle policies that transition objects to AWS Intelligent Tiering, which monitors S3 object access patterns and transitions objects accordingly.		In order to reduce monthly S3 storage costs, the UCSC GI has implemented global S3 lifecycle policies that transition objects to AWS Intelligent Tiering, which monitors S3 object access patterns and transitions objects to more efficient storage classes accordingly.
	* Objects uploaded to S3 will remain in the Standard storage class for '''1''' day, at which point they will be transitioned to Intelligent Tiering.		* Objects uploaded to S3 will remain in the Standard storage class for '''1''' day, at which point they will be transitioned to Intelligent Tiering.
	* Old and new S3 buckets will have this lifecycle policy automatically attached.		* Old and new S3 buckets will have this lifecycle policy automatically attached.

Righanse at 17:03, 9 December 2021

2021-12-09T17:03:07Z

← Older revision		Revision as of 10:03, 9 December 2021
Line 14:		Line 14:

	AWS Intelligent Tiering functionality:		AWS Intelligent Tiering functionality:
	* Intelligent Tiering '''does not''' change object access patterns. This means you ~~can still treat the object as if it were in the Standard storage class~~.		* Intelligent Tiering '''does not''' change object access patterns. This means you do not need to execute special API commands to access objects.
	* Intelligent Tiering '''does not''' incur charges for object retrieval from different tiers.		* Intelligent Tiering '''does not''' incur charges for object retrieval from different tiers.
	For more details on AWS Intelligent Tiering, see the [https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ AWS Docs]		For more details on AWS Intelligent Tiering, see the [https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ AWS Docs]

Righanse at 16:43, 9 December 2021

2021-12-09T16:43:15Z

← Older revision		Revision as of 09:43, 9 December 2021
Line 2:		Line 2:

	==AWS S3 Lifecycle Policy Overview==		==AWS S3 Lifecycle Policy Overview==
	AWS S3 buckets can be configured with lifecycle policies. These policies allow for automatically changing the storage class of objects based on the last time they were accessed. AWS S3 objects are typically stored in the Standard storage class, which provides easy access, but has relatively high GB/month storage costs. Other storage classes, such as Infrequent Access and Glacier are more suitable for objects that are rarely accessed. These storage ~~class~~ maintain a much lower GB/month cost as compared to the Standard S3 storage class, but also incur charges for access and retrieval.		AWS S3 buckets can be configured with lifecycle policies. These policies allow for automatically changing the storage class of objects based on the last time they were accessed. AWS S3 objects are typically stored in the Standard storage class, which provides easy access, but has relatively high GB/month storage costs. Other storage classes, such as Infrequent Access and Glacier are more suitable for objects that are rarely accessed. These storage classes maintain a much lower GB/month cost as compared to the Standard S3 storage class, but also incur charges for access and retrieval.

	~~In order~~ to ~~reduce monthly S3 storage costs,~~ the ~~UCSC GI has implemented global S3 lifecycle policies that transition objects to cheaper~~ storage classes ~~if they have not been accessed recently~~. ~~Specifically, AWS S3 objects are transitioned as follows:~~		It is recommended to utilize the appropriate storage classes for your data.
	* ~~After~~ '''30''' ~~days of inactivity~~, ~~objects are transitioned to the '''~~Infrequent Access~~'''~~ storage class.		* If you have data that you do not expect to access more than once a '''month''', AWS Infrequent Access is a reasonable storage class to use.
	* ~~After~~ '''~~180~~''' ~~days of inactivity~~, ~~objects are transitioned to the '''~~Glacier~~'''~~ storage class.		* If you have data that you do not expect to access more than once a '''year''', AWS Glacier is a reasonable storage class to use.

	==~~Object Recovery~~==		==UCSC GI Automated Policy==
	In ~~case there are objects in~~ S3 ~~that have been transitioned out of the Standard~~ storage ~~class~~, ~~they can be recovered. Due to~~ the ~~increased access charges of~~ objects ~~in Infrequent Access and Glacier~~, ~~if an~~ object ~~is expected to be accessed frequently, it should be returned~~ to the Standard storage class. ~~The object recovery method depends on the storage class the object is in, with Glacier being more time consuming~~ and ~~challenging to restore from than Infrequent Access~~.		In order to reduce monthly S3 storage costs, the UCSC GI has implemented global S3 lifecycle policies that transition objects to AWS Intelligent Tiering, which monitors S3 object access patterns and transitions objects accordingly.
			* Objects uploaded to S3 will remain in the Standard storage class for '''1''' day, at which point they will be transitioned to Intelligent Tiering.
			* Old and new S3 buckets will have this lifecycle policy automatically attached.

	~~It is important to note that, in~~ the ~~event an~~ object ~~is recovered,~~ the ~~timer~~ for ~~transitioning the~~ object ~~back to Infrequent Access and Glacier is still running~~, ~~and~~ the ~~objects will be moved again in the future if they meet the criteria~~.		AWS Intelligent Tiering functionality:
			* Intelligent Tiering '''does not''' change object access patterns. This means you can still treat the object as if it were in the Standard storage class.
	~~===Infrequent Access===~~		* Intelligent Tiering '''does not''' incur charges for object retrieval from different tiers.
	~~===Glacier===~~		For more details on AWS Intelligent Tiering, see the [https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ AWS Docs]

Righanse at 23:43, 2 December 2021

2021-12-02T23:43:42Z

← Older revision		Revision as of 16:43, 2 December 2021
Line 1:		Line 1:
	(This page is a work in progress. The policies defined below ~~are~~ still be adjusted, and are not actively deployed)		(This page is a work in progress. The policies defined below may still be adjusted, and are not actively deployed)

	==AWS S3 Lifecycle Policy Overview==		==AWS S3 Lifecycle Policy Overview==
	AWS S3 buckets can be configured with lifecycle policies. These policies allow for automatically changing the storage class of objects based on the last time they were accessed. AWS S3 objects are typically stored in the Standard storage class, which provides easy access, but has relatively high GB/month storage costs. Other storage classes, such as Infrequent Access and Glacier are more suitable for objects that are rarely accessed. These storage class maintain a much lower GB/month cost as compared to the Standard S3 storage class.		AWS S3 buckets can be configured with lifecycle policies. These policies allow for automatically changing the storage class of objects based on the last time they were accessed. AWS S3 objects are typically stored in the Standard storage class, which provides easy access, but has relatively high GB/month storage costs. Other storage classes, such as Infrequent Access and Glacier are more suitable for objects that are rarely accessed. These storage class maintain a much lower GB/month cost as compared to the Standard S3 storage class, but also incur charges for access and retrieval.

	In order to reduce monthly S3 storage costs, the UCSC GI has implemented global S3 lifecycle policies that transition objects to cheaper storage classes if they have not been accessed recently. Specifically, AWS S3 objects are transitioned as follows:		In order to reduce monthly S3 storage costs, the UCSC GI has implemented global S3 lifecycle policies that transition objects to cheaper storage classes if they have not been accessed recently. Specifically, AWS S3 objects are transitioned as follows:
	* After '''30''' days of inactivity, objects are transitioned to the '''Infrequent Access''' storage class.		* After '''30''' days of inactivity, objects are transitioned to the '''Infrequent Access''' storage class.
	* After '''180''' days of inactivity, objects are transitioned to the '''Glacier''' storage class.		* After '''180''' days of inactivity, objects are transitioned to the '''Glacier''' storage class.

			==Object Recovery==
			In case there are objects in S3 that have been transitioned out of the Standard storage class, they can be recovered. Due to the increased access charges of objects in Infrequent Access and Glacier, if an object is expected to be accessed frequently, it should be returned to the Standard storage class. The object recovery method depends on the storage class the object is in, with Glacier being more time consuming and challenging to restore from than Infrequent Access.

			It is important to note that, in the event an object is recovered, the timer for transitioning the object back to Infrequent Access and Glacier is still running, and the objects will be moved again in the future if they meet the criteria.

			===Infrequent Access===
			===Glacier===

Righanse at 23:28, 2 December 2021

2021-12-02T23:28:14Z

← Older revision		Revision as of 16:28, 2 December 2021
Line 1:		Line 1:
	~~Test~~ page		(This page is a work in progress. The policies defined below are still be adjusted, and are not actively deployed)

			==AWS S3 Lifecycle Policy Overview==
			AWS S3 buckets can be configured with lifecycle policies. These policies allow for automatically changing the storage class of objects based on the last time they were accessed. AWS S3 objects are typically stored in the Standard storage class, which provides easy access, but has relatively high GB/month storage costs. Other storage classes, such as Infrequent Access and Glacier are more suitable for objects that are rarely accessed. These storage class maintain a much lower GB/month cost as compared to the Standard S3 storage class.

			In order to reduce monthly S3 storage costs, the UCSC GI has implemented global S3 lifecycle policies that transition objects to cheaper storage classes if they have not been accessed recently. Specifically, AWS S3 objects are transitioned as follows:
			* After '''30''' days of inactivity, objects are transitioned to the '''Infrequent Access''' storage class.
			* After '''180''' days of inactivity, objects are transitioned to the '''Glacier''' storage class.