Firewalled Environment Storage Overview - Revision history

Weiler: /* Backups */

2026-04-29T15:10:34Z

Backups

← Older revision		Revision as of 08:10, 29 April 2026
Line 76:		Line 76:
	== Backups ==		== Backups ==

	/private/groups is backed up ~~weekly~~ on ~~Friday nights~~ (which usually takes ~~several days~~ to complete). Please note that the following directories in the tree '''WILL NOT''' be backed up:		/private/groups is backed up monthly on the first of the month (which usually takes a week to complete). Please note that the following directories in the tree '''WILL NOT''' be backed up:

	tmp/		tmp/

Weiler at 18:36, 4 February 2026

2026-02-04T18:36:38Z

← Older revision		Revision as of 11:36, 4 February 2026
Line 34:		Line 34:
	'''Home Directories (/private/home/username)'''		'''Home Directories (/private/home/username)'''

	Your home directory will be located as "/private/home/username" and has a ~~30GB~~ soft quota and a ~~31GB~~ hard quota. Your home directory is meant for small scripts and login data, or a git repo. Please do not try to store large data there or computer on large jobs using data in your home directory.		Your home directory will be located as "/private/home/username" and has a 100GB soft quota and a 100GB hard quota. Your home directory is meant for small scripts and login data, or a git repo. Please do not try to store large data there or computer on large jobs using data in your home directory.

	'''Groups Directories (/private/groups/groupname)'''		'''Groups Directories (/private/groups/groupname)'''

Weiler at 18:25, 4 February 2026

2026-02-04T18:25:36Z

← Older revision		Revision as of 11:25, 4 February 2026
Line 12:		Line 12:
	\|-		\|-
	\| style="font-weight:bold; text-align:left;" \| Default Soft Quota		\| style="font-weight:bold; text-align:left;" \| Default Soft Quota
	\| 30 GB		\| 100 GB
	\| 15 TB		\| 15 TB
	\|-		\|-
	\| style="font-weight:bold; text-align:left;" \| Default Hard Quota		\| style="font-weight:bold; text-align:left;" \| Default Hard Quota
	\| 31 GB		\| 100 GB
	\| 15 TB		\| 15 TB
	\|-		\|-

Weiler: /* Storage */

2025-09-29T15:52:09Z

Storage

← Older revision		Revision as of 08:52, 29 September 2025
Line 21:		Line 21:
	\| style="font-weight:bold; text-align:left;" \| Total Capacity		\| style="font-weight:bold; text-align:left;" \| Total Capacity
	\| 19 TB		\| 19 TB
	\| 1.5 PB		\| 1.7 PB
	\|- style="text-align:left;"		\|- style="text-align:left;"
	\| style="font-weight:bold;" \| Access Speed		\| style="font-weight:bold;" \| Access Speed

Weiler at 15:50, 29 September 2025

2025-09-29T15:50:59Z

← Older revision		Revision as of 08:50, 29 September 2025
Line 21:		Line 21:
	\| style="font-weight:bold; text-align:left;" \| Total Capacity		\| style="font-weight:bold; text-align:left;" \| Total Capacity
	\| 19 TB		\| 19 TB
	\| ~~800 TB~~		\| 1.5 PB
	\|- style="text-align:left;"		\|- style="text-align:left;"
	\| style="font-weight:bold;" \| Access Speed		\| style="font-weight:bold;" \| Access Speed

Weiler: /* Backups */

2024-05-14T18:26:05Z

Backups

← Older revision		Revision as of 11:26, 14 May 2024
Line 84:		Line 84:
	cache/		cache/
	.cache/		.cache/
			scratch/
	*.tmp/		*.tmp/

Weiler: /* Backups */

2024-05-14T18:21:55Z

Backups

← Older revision		Revision as of 11:21, 14 May 2024
Line 76:		Line 76:
	== Backups ==		== Backups ==

	/private/groups is backed up weekly on Friday nights (which usually takes several days to complete). Please ~~not~~ that the following directories in the tree '''WILL NOT''' be backed up:		/private/groups is backed up weekly on Friday nights (which usually takes several days to complete). Please note that the following directories in the tree '''WILL NOT''' be backed up:

	tmp/		tmp/

Weiler at 18:21, 14 May 2024

2024-05-14T18:21:34Z

← Older revision		Revision as of 11:21, 14 May 2024
Line 73:		Line 73:

	Each server will generally have a local /data/scratch filesystem that you can use to store temporary files. '''BE ADVISED''' that /data/scratch is not backed up, and the data there could disappear in the event of a disk failure or anything else. Do not store important data there. If it is important, it should be moved somewhere else very soon after creation.		Each server will generally have a local /data/scratch filesystem that you can use to store temporary files. '''BE ADVISED''' that /data/scratch is not backed up, and the data there could disappear in the event of a disk failure or anything else. Do not store important data there. If it is important, it should be moved somewhere else very soon after creation.

			== Backups ==

			/private/groups is backed up weekly on Friday nights (which usually takes several days to complete). Please not that the following directories in the tree '''WILL NOT''' be backed up:

			tmp/
			temp/
			TMP/
			TEMP/
			cache/
			.cache/
			*.tmp/

			So if you have data that you know isn't important and should be excluded from the backups, put them in a directory suffixed with ".tmp". Such as this example:

			/private/groups/clusteradmin/mybams.tmp/

Weiler: /* Storage Quota Alerting */

2024-05-13T16:31:28Z

Storage Quota Alerting

← Older revision		Revision as of 09:31, 13 May 2024
Line 62:		Line 62:
	== Storage Quota Alerting ==		== Storage Quota Alerting ==

	If you and/or folks in your lab would like an automated alert when the /private/groups/labname quota is getting to a certain percentage of fullness, we can set that up for you and others in your lab. Just email cluster-admin@soe.ucsc.edu with the following information:		If you and/or folks in your lab would like an automated alert when the /private/groups/labname quota is getting to a certain percentage of fullness, we can set that up for you and others in your lab. Just email '''cluster-admin@soe.ucsc.edu''' with the following information:

	1: Which directory you would like to watch quotas on (i.e. /private/groups/somelab)		1: Which directory you would like to watch quotas on (i.e. /private/groups/somelab)
Line 68:		Line 68:
	3: What email addresses you want on the alert list		3: What email addresses you want on the alert list

	After setup, our alerting system will alert folks on that email list every 4 hours until the quota in question is reduced to an amount under the alerting % threshold you asked for. So it is a bit noisy, but will force folks to delete data in order to stop the alerts. When the system notices that the quota usage has decreased to under the alert threshold, you will receive one final email with an "OK" notification that things are OK now.		After setup, our alerting system will alert folks on that email list ''every 4 hours'' until the quota in question is reduced to an amount under the alerting % threshold you asked for. So it is a bit noisy, but will force folks to delete data in order to stop the alerts. When the system notices that the quota usage has decreased to under the alert threshold, you will receive one final email with an "OK" notification that things are OK now.

	== /data/scratch Space on the Servers ==		== /data/scratch Space on the Servers ==

	Each server will generally have a local /data/scratch filesystem that you can use to store temporary files. '''BE ADVISED''' that /data/scratch is not backed up, and the data there could disappear in the event of a disk failure or anything else. Do not store important data there. If it is important, it should be moved somewhere else very soon after creation.		Each server will generally have a local /data/scratch filesystem that you can use to store temporary files. '''BE ADVISED''' that /data/scratch is not backed up, and the data there could disappear in the event of a disk failure or anything else. Do not store important data there. If it is important, it should be moved somewhere else very soon after creation.

Weiler at 16:30, 13 May 2024

2024-05-13T16:30:25Z

← Older revision		Revision as of 09:30, 13 May 2024
Line 59:		Line 59:

	And 15000000000000 divided by 1,000,000,000,000 is 15 TB.		And 15000000000000 divided by 1,000,000,000,000 is 15 TB.

			== Storage Quota Alerting ==

			If you and/or folks in your lab would like an automated alert when the /private/groups/labname quota is getting to a certain percentage of fullness, we can set that up for you and others in your lab. Just email cluster-admin@soe.ucsc.edu with the following information:

			1: Which directory you would like to watch quotas on (i.e. /private/groups/somelab)
			2: What % full you would like an email alert at
			3: What email addresses you want on the alert list

			After setup, our alerting system will alert folks on that email list every 4 hours until the quota in question is reduced to an amount under the alerting % threshold you asked for. So it is a bit noisy, but will force folks to delete data in order to stop the alerts. When the system notices that the quota usage has decreased to under the alert threshold, you will receive one final email with an "OK" notification that things are OK now.

	== /data/scratch Space on the Servers ==		== /data/scratch Space on the Servers ==

	Each server will generally have a local /data/scratch filesystem that you can use to store temporary files. '''BE ADVISED''' that /data/scratch is not backed up, and the data there could disappear in the event of a disk failure or anything else. Do not store important data there. If it is important, it should be moved somewhere else very soon after creation.		Each server will generally have a local /data/scratch filesystem that you can use to store temporary files. '''BE ADVISED''' that /data/scratch is not backed up, and the data there could disappear in the event of a disk failure or anything else. Do not store important data there. If it is important, it should be moved somewhere else very soon after creation.