Computing Resources Overview - Revision history

Weiler: /* The Phoenix Cluster */

2023-06-14T23:17:24Z

The Phoenix Cluster

← Older revision		Revision as of 16:17, 14 June 2023
Line 48:		Line 48:

	This is a cluster of ~20 Ubuntu 22.04 nodes, some of which have GPUs in them. Each node generally has about 2TB RAM and 128 cores, although the cluster is heterogeneous and has multiple node types.		This is a cluster of ~20 Ubuntu 22.04 nodes, some of which have GPUs in them. Each node generally has about 2TB RAM and 128 cores, although the cluster is heterogeneous and has multiple node types.

			{\| class="wikitable" style="text-align:center;"
			\|- style="font-weight:bold;"
			! Node Name
			! Operating System<br />
			! CPU Cores
			! GPUs/Type
			! Memory
			! Network Bandwidth
			! Scratch Space
			\|-
			\| style="text-align:left;" \| phoenix-00
			\| style="text-align:left;" \| Ubuntu 22.04
			\| 256
			\| 8 / nVidia A100
			\| 1 TB
			\| 10 Gb/s
			\| 16 TB NVMe
			\|-
			\| style="text-align:left;" \| phoenix-[01-05]
			\| style="text-align:left;" \| Ubuntu 22.04
			\| 128
			\| 8 / nVidia A5500
			\| 2 TB
			\| 10 Gb/s
			\| 16 TB NVMe
			\|-
			\| style="text-align:left;" \| phoenix-[06-08]
			\| style="text-align:left;" \| Ubuntu 22.04
			\| 128
			\| N/A
			\| 2 TB
			\| 10 Gb/s
			\| 16 TB NVMe
			\|-
			\| style="text-align:left;" \| phoenix-09
			\| style="text-align:left;" \| Ubuntu 22.04
			\| 40
			\| 8 / nVidia 1080ti
			\| 256 GB
			\| 10 Gb/s
			\| 1.5 TB NVMe
			\|-
			\| style="text-align:left;" \| phoenix-10
			\| style="text-align:left;" \| Ubuntu 22.04
			\| 36
			\| 4 / nVidia 2080ti
			\| 385 GB
			\| 10 Gb/s
			\| 3 TB NVMe
			\|-
			\| style="text-align:left;" \| phoenix-[11-20]
			\| style="text-align:left;" \| Ubuntu 22.04
			\| 128
			\| N/A
			\| 2 TB
			\| 10 Gb/s
			\| 16 TB NVMe
			\|}

	The cluster head node, from which all jobs are submitted via the SLURM job scheduling framework, is '''phoenix.prism'''. To learn more about how to use Slurm, refer to:		The cluster head node, from which all jobs are submitted via the SLURM job scheduling framework, is '''phoenix.prism'''. To learn more about how to use Slurm, refer to:

Weiler: /* Server Types and Management */

2023-06-14T23:07:49Z

Server Types and Management

← Older revision		Revision as of 16:07, 14 June 2023
Line 6:		Line 6:
	== Server Types and Management ==		== Server Types and Management ==

	After confirming your VPN software is working, you can ssh into one of the compute servers behind the VPN:		After confirming your VPN software is working, you can ssh into one of the shared compute servers behind the VPN:

	{\| class="wikitable"		{\| class="wikitable" style="text-align:center;"
	\|- style="font-weight:bold;"		\|- style="font-weight:bold; text-align:left;"
	! ~~Server~~ Name		! Node Name
	! Operating System<br />		! Operating System<br />
	! CPU Cores		! CPU Cores
Line 17:		Line 17:
	! Scratch Space		! Scratch Space
	\|-		\|-
	\| mustard		\| style="text-align:left;" \| mustard
	~~\| Ubuntu 22.04~~		\| style="text-align:left;" \| Ubuntu 22.04
	\| style="text-align:~~center~~;" \| 160		\| 160
	\| 1.5 TB		\| 1.5 TB
	\| 10 Gb/s		\| 10 Gb/s
	\| 9 TB		\| 9 TB
	\|-		\|-
	\| crimson		\| style="text-align:left;" \| crimson
	~~\| CentOS 7.5~~		\| style="text-align:left;" \| CentOS 7.5
	\| style="text-align:~~center~~;" \| 32		\| 32
	\| 256 GB		\| 256 GB
	\| 10 Gb/s		\| 10 Gb/s
	\| 5.5 TB		\| 5.5 TB
	\|-		\|-
	\| razzmatazz		\| style="text-align:left;" \| razzmatazz
	~~\| Ubuntu 22.04~~		\| style="text-align:left;" \| Ubuntu 22.04
	\| style="text-align:~~center~~;" \| 32		\| 32
	\| 256 GB		\| 256 GB
	\| 10 Gb/s		\| 10 Gb/s
Line 39:		Line 39:
	\|}		\|}

	These servers are managed by the Genomics Institute Cluster Admin group. If you need software installed on any of these servers, please make your request by emailing cluster-admin@soe.ucsc.edu.		These ''shared'' servers are managed by the Genomics Institute Cluster Admin group. If you need software installed on any of these servers, please make your request by emailing cluster-admin@soe.ucsc.edu.

	== The Firewall ==		== The Firewall ==

Weiler: /* Server Types and Management */

2023-06-14T23:03:58Z

Server Types and Management

@@ Line 8: / Line 8: @@
 After confirming your VPN software is working, you can ssh into one of the compute servers behind the VPN:
- '''crimson.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space, CentOS 7.9
+{| class="wikitable"
- '''razzmatazz.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space, Ubuntu 22.04
+|- style="font-weight:bold;"
- '''mustard.prism''': 1.5TB RAM, 160 cores, 9TB local scratch space, Ubuntu 22.04
+! Server Name
 These servers are managed by the Genomics Institute Cluster Admin group.  If you need software installed on any of these servers, please make your request by emailing cluster-admin@soe.ucsc.edu.

Weiler at 22:57, 14 June 2023

2023-06-14T22:57:02Z

← Older revision		Revision as of 15:57, 14 June 2023
Line 1:		Line 1:
	== Doing Work and Computing ==		== Doing Work and Computing ==

	When doing research, running jobs and the like, please be careful of your resource consumption on the server you are on. Don't run too many threads or cores at once if such a thing overruns the RAM available or the disk IO available. If you are not sure of your potential RAM, CPU or disk impact, start small with one or two processes and work your way up from there. Also, before running your stuff, check what else is already happening on the server by using the 'top' command to see who else and what else is running and what kind of resources are already being consumed. If, after starting a process, you realize that the server slows down considerably or becomes unusable, kill your processes and re-evaluate what you need to make things work. These servers are shared resources - be a good neighbor!		When doing research, running jobs and the like, please be careful of your resource consumption on the server you are on. Don't run too many threads or cores at once if such a thing overruns the RAM available or the disk IO available. If you are not sure of your potential RAM, CPU or disk impact, start small with one or two processes and work your way up from there. Also, before running your stuff, check what else is already happening on the server by using the 'top' command to see who else and what else is running and what kind of resources are already being consumed. If, after starting a process, you realize that the server slows down considerably or becomes unusable, kill
			your processes and re-evaluate what you need to make things work. These servers are shared resources - be a good neighbor!

			== Server Types and Management ==

			After confirming your VPN software is working, you can ssh into one of the compute servers behind the VPN:

			'''crimson.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space, CentOS 7.9
			'''razzmatazz.prism''': 256GB RAM, 32 cores, 5.5TB local scratch space, Ubuntu 22.04
			'''mustard.prism''': 1.5TB RAM, 160 cores, 9TB local scratch space, Ubuntu 22.04

			These servers are managed by the Genomics Institute Cluster Admin group. If you need software installed on any of these servers, please make your request by emailing cluster-admin@soe.ucsc.edu.

	== The Firewall ==		== The Firewall ==

Weiler: Created page with "== Doing Work and Computing == When doing research, running jobs and the like, please be careful of your resource consumption on the server you are on. Don't run too many th..."

2023-06-14T22:55:43Z

Created page with "== Doing Work and Computing == When doing research, running jobs and the like, please be careful of your resource consumption on the server you are on. Don't run too many th..."

New page

== Doing Work and Computing ==

When doing research, running jobs and the like, please be careful of your resource consumption on the server you are on. Don't run too many threads or cores at once if such a thing overruns the RAM available or the disk IO available. If you are not sure of your potential RAM, CPU or disk impact, start small with one or two processes and work your way up from there. Also, before running your stuff, check what else is already happening on the server by using the 'top' command to see who else and what else is running and what kind of resources are already being consumed. If, after starting a process, you realize that the server slows down considerably or becomes unusable, kill your processes and re-evaluate what you need to make things work. These servers are shared resources - be a good neighbor!

== The Firewall ==

All servers are behind a firewall in this environment, and as such, you must connect to the VPN in order to access them. They will not be accessible from the greater Internet without VPN. Although you will be able to connect outbound from them to other servers on the internet to copy data in, sync git repos, stuff like that. It is only inbound connections that will be blocked. All machines behind the firewall have the private domain name suffix of "*.prism".

== The Phoenix Cluster ==

This is a cluster of ~20 Ubuntu 22.04 nodes, some of which have GPUs in them. Each node generally has about 2TB RAM and 128 cores, although the cluster is heterogeneous and has multiple node types.

The cluster head node, from which all jobs are submitted via the SLURM job scheduling framework, is '''phoenix.prism'''. To learn more about how to use Slurm, refer to:

https://giwiki.gi.ucsc.edu/index.php/Genomics_Institute_Computing_Information#Slurm_at_the_Genomics_Institute

For scratch on the cluster, TMPDIR will be set to /data/tmp. That area is cleaned often so don't store any data there that isn't being used by your jobs.