Access to the Firewalled Compute Servers - Revision history

Weiler at 23:27, 14 June 2023

2023-06-14T23:27:30Z

← Older revision		Revision as of 16:27, 14 June 2023
Line 2:		Line 2:

	[[Requirement for users to get GI VPN access]]		[[Requirement for users to get GI VPN access]]

	~~== Account and Storage Cost ==~~

	Costs for having an active UNIX account and for storage (per TB) are listed in this document, under "Genomics Project Support", specifically "Genomics IT Systems User Support per user" and "Genomics Data Storage per TB":

	~~https://planning.ucsc.edu/budget/rates-and-assessments/recharge-rates/docs/2021-22-approved-recharge-rates.pdf~~

	== Account Expiration ==		== Account Expiration ==

Weiler at 22:23, 14 June 2023

2023-06-14T22:23:30Z

← Older revision		Revision as of 15:23, 14 June 2023
Line 16:		Line 16:

	If your account expires, the account will be suspended and you will no longer be able to login or view any data you may have in our systems. Any automated scripts (owned by you) that run via cron or other mechanisms will cease to function.		If your account expires, the account will be suspended and you will no longer be able to login or view any data you may have in our systems. Any automated scripts (owned by you) that run via cron or other mechanisms will cease to function.

	~~== Server Types and Management==~~

	~~After confirming your VPN software is working, you can ssh into one of the compute servers behind the VPN:~~

	~~'''crimson.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space, CentOS 7.9~~
	~~'''razzmatazz.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space, Ubuntu 22.04~~
	~~'''mustard.prism''': 1.5TB RAM, 160 cores, 9TB local scratch space, Ubuntu 22.04~~

	~~These servers are managed by the Genomics Institute Cluster Admin group. If you need software installed on any of these servers, please make your request by emailing cluster-admin@soe.ucsc.edu.~~

	~~== Storage ==~~

	These servers mount two types of storage; home directories and group storage directories. Your home directory will be located as "/private/home/username" and has a 30GB quota. The group storage directories are created per PI, and each group directory has a 15TB quota. For example, if David Haussler is the PI that you report to directly, then the directory would exist as /private/groups/hausslerlab. Request access to that group directory and you will then be able to write to it. Each of those group directories are shared by the lab it belongs to, so you must be wary of everyone's data usage and share the 15TB available per group accordingly.

	On the compute servers you can check your group's current quota usage by using the '/usr/bin/viewquota' command. You can only check the quota of a group you are part of (you would be a member of the UNIX group of the same name). If you wanted to check the quota usage of /private/groups/hausslerlab for example, you would do:

	~~$ viewquota hausslerlab~~

	~~Project quota on /export (/dev/mapper/export)~~
	~~Project ID Used Soft Hard Warn/Grace~~
	~~---------- ---------------------------------~~
	~~hausslerlab 1.8T 15T 16T 00 [------]~~

	~~== Actually Doing Work and Computing ==~~

	When doing research, running jobs and the like, please be careful of your resource consumption on the server you are on. Don't run too many threads or cores at once if such a thing overruns the RAM available or the disk IO available. If you are not sure of your potential RAM, CPU or disk impact, start small with one or two processes and work your way up from there. Also, before running your stuff, check what else is already happening on the server by using the 'top' command to see who else and what else is running and what kind of resources are already being consumed. If, after starting a process, you realize that the server slows down considerably or becomes unusable, kill your processes and re-evaluate what you need to make things work. These servers are shared resources - be a good neighbor!

	~~== The Firewall ==~~

	All servers are behind a firewall in this environment, and as such, you must connect to the VPN in order to access them. They will not be accessible from the greater Internet without VPN. Although you will be able to connect outbound from them to other servers on the internet to copy data in, sync git repos, stuff like that. It is only inbound connections that will be blocked. All machines behind the firewall have the private domain name suffix of "*.prism".

	~~== /scratch Space on the Servers ==~~

	Each server will generally have a local /scratch filesystem that you can use to store temporary files. '''BE ADVISED''' that /scratch is not backed up, and the data there could disappear in the event of a disk failure or anything else. Do not store important data there. If it is important, it should be moved somewhere else very soon after creation.

	~~== The Phoenix Cluster ==~~

	~~This is a cluster of ~20 Ubuntu 22.04 nodes, some of which have GPUs in them. Each node generally has about 2TB RAM and 128 cores, although the cluster is heterogeneous and has multiple node types.~~

	~~The cluster head node, from which all jobs are submitted via the SLURM job scheduling framework, is '''phoenix.prism'''. To learn more about how to use Slurm, refer to:~~

	~~https://giwiki.gi.ucsc.edu/index.php/Genomics_Institute_Computing_Information#Slurm_at_the_Genomics_Institute~~

	~~For scratch on the cluster, TMPDIR will be set to /data/tmp. That area is cleaned often so don't store any data there that isn't being used by your jobs.~~

Weiler: /* The Phoenix Cluster */

2023-06-12T22:03:12Z

The Phoenix Cluster

← Older revision		Revision as of 15:03, 12 June 2023
Line 59:		Line 59:

	https://giwiki.gi.ucsc.edu/index.php/Genomics_Institute_Computing_Information#Slurm_at_the_Genomics_Institute		https://giwiki.gi.ucsc.edu/index.php/Genomics_Institute_Computing_Information#Slurm_at_the_Genomics_Institute

			For scratch on the cluster, TMPDIR will be set to /data/tmp. That area is cleaned often so don't store any data there that isn't being used by your jobs.

Weiler: /* The Phoenix Cluster */

2023-06-12T22:01:15Z

The Phoenix Cluster

← Older revision		Revision as of 15:01, 12 June 2023
Line 56:		Line 56:
	This is a cluster of ~20 Ubuntu 22.04 nodes, some of which have GPUs in them. Each node generally has about 2TB RAM and 128 cores, although the cluster is heterogeneous and has multiple node types.		This is a cluster of ~20 Ubuntu 22.04 nodes, some of which have GPUs in them. Each node generally has about 2TB RAM and 128 cores, although the cluster is heterogeneous and has multiple node types.

	The cluster head node, from which all jobs are submitted via the SLURM job scheduling framework, is ~~"""~~phoenix.prism~~"""~~. To learn more about how to use Slurm, refer to:		The cluster head node, from which all jobs are submitted via the SLURM job scheduling framework, is '''phoenix.prism'''. To learn more about how to use Slurm, refer to:

	https://giwiki.gi.ucsc.edu/index.php/Genomics_Institute_Computing_Information#Slurm_at_the_Genomics_Institute		https://giwiki.gi.ucsc.edu/index.php/Genomics_Institute_Computing_Information#Slurm_at_the_Genomics_Institute

Weiler at 22:00, 12 June 2023

2023-06-12T22:00:29Z

← Older revision		Revision as of 15:00, 12 June 2023
Line 54:		Line 54:
	== The Phoenix Cluster ==		== The Phoenix Cluster ==

	This is a cluster of ~20 nodes, some of which have GPUs in them. Each node generally has about 2TB RAM and 128 cores, although the cluster is heterogeneous and has multiple node types.		This is a cluster of ~20 Ubuntu 22.04 nodes, some of which have GPUs in them. Each node generally has about 2TB RAM and 128 cores, although the cluster is heterogeneous and has multiple node types.

	The cluster head node, from which all jobs are submitted via the SLURM job scheduling framework, is """phoenix.prism""". To learn more about how to use Slurm, refer to:		The cluster head node, from which all jobs are submitted via the SLURM job scheduling framework, is """phoenix.prism""". To learn more about how to use Slurm, refer to:

			https://giwiki.gi.ucsc.edu/index.php/Genomics_Institute_Computing_Information#Slurm_at_the_Genomics_Institute

Weiler at 21:58, 12 June 2023

2023-06-12T21:58:48Z

← Older revision		Revision as of 14:58, 12 June 2023
Line 51:		Line 51:

	Each server will generally have a local /scratch filesystem that you can use to store temporary files. '''BE ADVISED''' that /scratch is not backed up, and the data there could disappear in the event of a disk failure or anything else. Do not store important data there. If it is important, it should be moved somewhere else very soon after creation.		Each server will generally have a local /scratch filesystem that you can use to store temporary files. '''BE ADVISED''' that /scratch is not backed up, and the data there could disappear in the event of a disk failure or anything else. Do not store important data there. If it is important, it should be moved somewhere else very soon after creation.

			== The Phoenix Cluster ==

			This is a cluster of ~20 nodes, some of which have GPUs in them. Each node generally has about 2TB RAM and 128 cores, although the cluster is heterogeneous and has multiple node types.

			The cluster head node, from which all jobs are submitted via the SLURM job scheduling framework, is """phoenix.prism""". To learn more about how to use Slurm, refer to:

Weiler: /* Server Types and Management */

2023-06-12T21:52:43Z

Server Types and Management

← Older revision		Revision as of 14:52, 12 June 2023
Line 25:		Line 25:
	'''mustard.prism''': 1.5TB RAM, 160 cores, 9TB local scratch space, Ubuntu 22.04		'''mustard.prism''': 1.5TB RAM, 160 cores, 9TB local scratch space, Ubuntu 22.04

	These servers are managed by the Genomics Institute Cluster Admin group. If you need software installed on ~~either or both~~ of these servers, please make your request by emailing cluster-admin@soe.ucsc.edu.		These servers are managed by the Genomics Institute Cluster Admin group. If you need software installed on any of these servers, please make your request by emailing cluster-admin@soe.ucsc.edu.

	== Storage ==		== Storage ==

Weiler: /* Server Types and Management */

2023-06-12T21:52:11Z

Server Types and Management

← Older revision		Revision as of 14:52, 12 June 2023
Line 21:		Line 21:
	After confirming your VPN software is working, you can ssh into one of the compute servers behind the VPN:		After confirming your VPN software is working, you can ssh into one of the compute servers behind the VPN:

	'''crimson.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space		'''crimson.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space, CentOS 7.9
	'''razzmatazz.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space		'''razzmatazz.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space, Ubuntu 22.04
	'''mustard.prism''': 1.5TB RAM, 160 cores, 9TB local scratch space		'''mustard.prism''': 1.5TB RAM, 160 cores, 9TB local scratch space, Ubuntu 22.04

	These servers ~~are running CentOS 7.5 Linux. They~~ are managed by the Genomics Institute Cluster Admin group. If you need software installed on either or both of these servers, please make your request by emailing cluster-admin@soe.ucsc.edu.		These servers are managed by the Genomics Institute Cluster Admin group. If you need software installed on either or both of these servers, please make your request by emailing cluster-admin@soe.ucsc.edu.

	== Storage ==		== Storage ==

Weiler at 18:05, 28 October 2021

2021-10-28T18:05:23Z

← Older revision		Revision as of 11:05, 28 October 2021
Line 2:		Line 2:

	[[Requirement for users to get GI VPN access]]		[[Requirement for users to get GI VPN access]]

			== Account and Storage Cost ==

			Costs for having an active UNIX account and for storage (per TB) are listed in this document, under "Genomics Project Support", specifically "Genomics IT Systems User Support per user" and "Genomics Data Storage per TB":

			https://planning.ucsc.edu/budget/rates-and-assessments/recharge-rates/docs/2021-22-approved-recharge-rates.pdf

	== Account Expiration ==		== Account Expiration ==

Weiler at 22:48, 22 February 2019

2019-02-22T22:48:53Z

← Older revision		Revision as of 15:48, 22 February 2019
Line 2:		Line 2:

	[[Requirement for users to get GI VPN access]]		[[Requirement for users to get GI VPN access]]

			== Account Expiration ==

			Your UNIX account will have an expiration date associated with it after creation. If your account was created in January, February or March, then your account expiration date will be July 1st of the '''current year'''. If the account was created after March, then your expiration date will be July 1st of the '''following year'''.

			You will receive notice by email when your account is about to expire. To renew, simply ask the PI that sponsored you (which will be included in the notice) to email '''cluster-admin@soe.ucsc.edu''' requesting that your account to be renewed for another year.

			If your account expires, the account will be suspended and you will no longer be able to login or view any data you may have in our systems. Any automated scripts (owned by you) that run via cron or other mechanisms will cease to function.

	== Server Types and Management==		== Server Types and Management==