Computational Genomics Kubernetes Installation - Revision history

Weiler: /* View the Cluster's Current Activity */

2020-09-08T20:59:10Z

View the Cluster's Current Activity

← Older revision		Revision as of 13:59, 8 September 2020
Line 206:		Line 206:
	Select the "token" login method, and paste in this (long) token:		Select the "token" login method, and paste in this (long) token:

	~~eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9~~.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi0ycDY4cCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImE5ZGI2Y2I0LWMyYWYtNDM3My04ZmM2LWE4YWYwYTBmNGRkNCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.~~ZVQjryG_ksfvReIfq4Frb6M4sE6OVDOXnFy9Aii~~-~~h3mrpdHRE6bgjdAvSGZ0jJSIUEz5GgPBQ0lCwhyZocivHHr4zTrNxMkOFZhPDnpvF6RVIDWTkqmH9Dg6qmro0gTJP75oKBpt7dFN2pW4zvqOAzqPmh7qxfoVusN8X6U13YirMFEf65~~-~~aGL~~-~~_FFNBsEzvjkC~~-~~BgXWbtk3YZc8CJL7xtvlKLyE6u6jC9Qx0SWnwzkALlxmzo_yYTDKpIrWiQGEqzLQOxKml-H0kSYLDX-t4sTivXp4vCw_ruoqwIpLnnQAC7q3ZtSTxHIrxbB7n_M8gfhpXtwprbPav-XmBk1xaQ~~		eyJhbGciOiJSUzI1NiIsImtpZCI6InhhcTVJLWdkXzMzZzAxUENCdjNBYUJBbkZfZlBwSG9lVmd4S1dZbWZ6TncifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi1zNGxkbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImYwNWU4NjYyLWUyY2QtNDY3Yy1hYjY3LTNjNDc4ODVjZmM4YSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.pQQ3iaWgWGr4CVSxl0wvI9R0AqtxEMm_ElisnfejJtKb9g5ki7goL4VQ9n4lY1b0hO7ojfYcZzWC466FHLULPac6r_zRvme2YMi9EwyHU4iYfUVOktmcLPGl-NS_D3k-USJF8npqbn1OFSHS25pJ5924LFAC0dCkukanNODyNgbetplgkl8geG1pR_1dgqamJCB2xwDn2FjQBC-QjtUJnarGqeo1gqG3eeeWAImK3lGLnkYGPcsvwowmtOdjj2ScqCfjqlfkxWymMGAOB-iB7hEruYZ6dD4hrpIGuVSGQCHojm4FJo_AiFgRjBmfHZiRi0PV1PNoLQLRplpXMf2jOg

	The dashboard is read-only, so you won't be able to edit anything, it's mostly for seeing what's going on and where.		The dashboard is read-only, so you won't be able to edit anything, it's mostly for seeing what's going on and where.

Anovak: Suggest --no-progress and talk about AWS secrets

2020-06-23T23:23:05Z

Suggest --no-progress and talk about AWS secrets

@@ Line 93: / Line 93: @@
   https://github.com/rcurrie/kubernetes
 ==Inlining Jobs in Shell and Shell in Jobs==
@@ Line 218: / Line 261: @@
             # Use this particular perf binary that matches the hosts' kernels
             # If it is missing or outdated, get a new one from Erich or cluster-admin
-            aws s3 cp s3://vg-k8s/users/adamnovak/projects/test/perf /usr/bin/perf
+            aws s3 cp --no-progress s3://vg-k8s/users/adamnovak/projects/test/perf /usr/bin/perf
             chmod +x /usr/bin/perf
             # Do your work with perf here.

Anovak: Talk about perf and NUMA issues when profiling

2020-03-13T22:49:05Z

Talk about perf and NUMA issues when profiling

@@ Line 94: / Line 94: @@
   https://github.com/rcurrie/kubernetes
-== Inlining Jobs in Shell and Shell in Jobs ==
+==Inlining Jobs in Shell and Shell in Jobs==
 When interactively developing on Kubernetes, it can be useful to be able to have a shell command you can copy and paste to run a Kubernetes job, rather than having to create YAML files on disk. Similarly, it can be useful to have shell scripting inline in your Kubernetes job definitions, rather than having to bake your experimental script into a Docker container. Here's an example that does both, putting the YAML inside a heredoc and putting the script to run in the container inside a multiline YAML string. We precede this with a command to delete the job, so you can modify your script and re-paste it to replace a failed or failing job. We also make sure to mount the AWS credentials in the container, so that the ''aws'' command will be able to access S3 if you install it.
@@ Line 179: / Line 179: @@
 Once you get in, you should see a drop-down menu near the top left of the screen near "Genomics Institute Grid".  From the drop-down menu, select "CG Kubernetes Cluster".  It will take you to a page detailing the current resource usage and activity on the nodes.  This can be useful for see if anyone else is using the whole cluster, or just to get an idea of how many resources are available for your batch of jobs to assign to the cluster.

Anovak at 22:27, 13 March 2020

2020-03-13T22:27:16Z

← Older revision		Revision as of 15:27, 13 March 2020
Line 93:		Line 93:

	https://github.com/rcurrie/kubernetes		https://github.com/rcurrie/kubernetes

			== Inlining Jobs in Shell and Shell in Jobs ==

			When interactively developing on Kubernetes, it can be useful to be able to have a shell command you can copy and paste to run a Kubernetes job, rather than having to create YAML files on disk. Similarly, it can be useful to have shell scripting inline in your Kubernetes job definitions, rather than having to bake your experimental script into a Docker container. Here's an example that does both, putting the YAML inside a heredoc and putting the script to run in the container inside a multiline YAML string. We precede this with a command to delete the job, so you can modify your script and re-paste it to replace a failed or failing job. We also make sure to mount the AWS credentials in the container, so that the ''aws'' command will be able to access S3 if you install it.

			kubectl delete job username-job
			kubectl apply -f - <<'EOF'
			apiVersion: batch/v1
			kind: Job
			metadata:
			name: username-job
			spec:
			ttlSecondsAfterFinished: 1000
			template:
			spec:
			containers:
			- name: main
			imagePullPolicy: Always
			image: ubuntu:18.04
			command:
			- /bin/bash
			- -c
			- \|
			set -e
			DEBIAN_FRONTEND=noninteractive apt-get update
			DEBIAN_FRONTEND=noninteractive apt-get install -y awscli cowsay
			cowsay "Listing files"
			aws s3 ls s3://vg-k8s/
			volumeMounts:
			- mountPath: /tmp
			name: scratch-volume
			- mountPath: /root/.aws
			name: s3-credentials
			resources:
			limits:
			cpu: 1
			memory: "4Gi"
			ephemeral-storage: "10Gi"
			restartPolicy: Never
			volumes:
			- name: scratch-volume
			emptyDir: {}
			- name: s3-credentials
			secret:
			secretName: shared-s3-credentials
			backoffLimit: 0
			EOF

			Make sure to replace "username-job" with a unique job name that includes ''your'' username.

	==View the Cluster's Current Activity==		==View the Cluster's Current Activity==

Anovak: Note that limits can replace requests if unspecified.

2020-03-13T22:15:58Z

Note that limits can replace requests if unspecified.

← Older revision		Revision as of 15:15, 13 March 2020
Line 76:		Line 76:
	priorityClassName: medium-priority		priorityClassName: medium-priority

	Please note that the "request" and "limit" item fields should be the same. You would think that you could set the limit higher than the request, but in reality they need to match in order for the pod to stay within the kubernetes resource limit bubble. If you set the limit higher than the request, then you are risking the pod using more memory than the scheduler expects, and the node can start killing off random other colocated pods by way of OOM, which is very, very bad for the cluster.		Please note that the "request" and "limit" item fields should be the same. You would think that you could set the limit higher than the request, but in reality they need to match in order for the pod to stay within the kubernetes resource limit bubble. If you set the limit higher than the request, then you are risking the pod using more memory than the scheduler expects, and the node can start killing off random other colocated pods by way of OOM, which is very, very bad for the cluster. If you omit the "requests" section altogether, the limit values will be used, so if you use only one, use "limits".

	Also note the "priorityClassName" line. Available values are:		Also note the "priorityClassName" line. Available values are:

Weiler: /* Running Pods and Jobs with Requests and Limits */

2020-02-05T22:34:40Z

Running Pods and Jobs with Requests and Limits

← Older revision		Revision as of 15:34, 5 February 2020
Line 43:		Line 43:
	==Running Pods and Jobs with Requests and Limits==		==Running Pods and Jobs with Requests and Limits==

	When running jobs and pods on kubernetes, you will always want to specify "requests" and "limits" on resources, otherwise your pods will get stuck with the default limits which are tiny (to protect against runaway pods). You should always have an idea of how much resources will be consumed by your jobs, and not use much more than that, in order not to hog all the resources. It also prevents your ~~jib~~ from "running away" unexpectedly and chewing up more resources than expected.		When running jobs and pods on kubernetes, you will always want to specify "requests" and "limits" on resources, otherwise your pods will get stuck with the default limits which are tiny (to protect against runaway pods). You should always have an idea of how much resources will be consumed by your jobs, and not use much more than that, in order not to hog all the resources. It also prevents your job from "running away" unexpectedly and chewing up more resources than expected.

	Here is a good example of a job file that specifies limits:		Here is a good example of a job file that specifies limits:
Line 74:		Line 74:
	args: ['for i in {1..100}; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done']		args: ['for i in {1..100}; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done']
	restartPolicy: Never		restartPolicy: Never
			priorityClassName: medium-priority

	Please note that the "request" and "limit" item fields should be the same. You would think that you could set the limit higher than the request, but in reality they need to match in order for the pod to stay within the kubernetes resource limit bubble. If you set the limit higher than the request, then you are risking the pod using more memory than the scheduler expects, and the node can start killing off random other colocated pods by way of OOM, which is very, very bad for the cluster.		Please note that the "request" and "limit" item fields should be the same. You would think that you could set the limit higher than the request, but in reality they need to match in order for the pod to stay within the kubernetes resource limit bubble. If you set the limit higher than the request, then you are risking the pod using more memory than the scheduler expects, and the node can start killing off random other colocated pods by way of OOM, which is very, very bad for the cluster.

			Also note the "priorityClassName" line. Available values are:

			high-priority
			medium-priority
			low-priority

			That affects how quickly your jobs move up the queue in the event there are a lot of queued jobs. Always use "medium-priority" as the default unless you specifically know you need it higher or lower. Higher priority jobs will always go in front of lower priority jobs.

	'''NOTE:''' Jobs and pods that have '''completed''' over 72 hours ago but have not been cleaned up will be automatically removed by the garbage collector. Most jobs will have the "ttlSecondsAfterFinished" configuration item in them, so they will automatically cleaned up after that time expires, but leaving the old pods and jobs around pins the disk space they were using while they remain, so it's good to get rid of them as soon as they are done unless you are debugging a failure or something like that.		'''NOTE:''' Jobs and pods that have '''completed''' over 72 hours ago but have not been cleaned up will be automatically removed by the garbage collector. Most jobs will have the "ttlSecondsAfterFinished" configuration item in them, so they will automatically cleaned up after that time expires, but leaving the old pods and jobs around pins the disk space they were using while they remain, so it's good to get rid of them as soon as they are done unless you are debugging a failure or something like that.

Weiler: /* View the Cluster's Current Activity */

2019-12-06T21:12:58Z

View the Cluster's Current Activity

← Older revision		Revision as of 14:12, 6 December 2019
Line 103:		Line 103:
	https://cgl-k8s-dashboard.gi.ucsc.edu/		https://cgl-k8s-dashboard.gi.ucsc.edu/

	Select the "token" login method, and paste in this(long) token:		Select the "token" login method, and paste in this (long) token:

	eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi0ycDY4cCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImE5ZGI2Y2I0LWMyYWYtNDM3My04ZmM2LWE4YWYwYTBmNGRkNCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.ZVQjryG_ksfvReIfq4Frb6M4sE6OVDOXnFy9Aii-h3mrpdHRE6bgjdAvSGZ0jJSIUEz5GgPBQ0lCwhyZocivHHr4zTrNxMkOFZhPDnpvF6RVIDWTkqmH9Dg6qmro0gTJP75oKBpt7dFN2pW4zvqOAzqPmh7qxfoVusN8X6U13YirMFEf65-aGL-_FFNBsEzvjkC-BgXWbtk3YZc8CJL7xtvlKLyE6u6jC9Qx0SWnwzkALlxmzo_yYTDKpIrWiQGEqzLQOxKml-H0kSYLDX-t4sTivXp4vCw_ruoqwIpLnnQAC7q3ZtSTxHIrxbB7n_M8gfhpXtwprbPav-XmBk1xaQ		eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi0ycDY4cCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImE5ZGI2Y2I0LWMyYWYtNDM3My04ZmM2LWE4YWYwYTBmNGRkNCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.ZVQjryG_ksfvReIfq4Frb6M4sE6OVDOXnFy9Aii-h3mrpdHRE6bgjdAvSGZ0jJSIUEz5GgPBQ0lCwhyZocivHHr4zTrNxMkOFZhPDnpvF6RVIDWTkqmH9Dg6qmro0gTJP75oKBpt7dFN2pW4zvqOAzqPmh7qxfoVusN8X6U13YirMFEf65-aGL-_FFNBsEzvjkC-BgXWbtk3YZc8CJL7xtvlKLyE6u6jC9Qx0SWnwzkALlxmzo_yYTDKpIrWiQGEqzLQOxKml-H0kSYLDX-t4sTivXp4vCw_ruoqwIpLnnQAC7q3ZtSTxHIrxbB7n_M8gfhpXtwprbPav-XmBk1xaQ

Weiler: /* View the Cluster's Current Activity */

2019-12-06T21:12:33Z

View the Cluster's Current Activity

← Older revision		Revision as of 14:12, 6 December 2019
Line 98:		Line 98:

	That means the worker nodes, k1, k2 and k3, are using minimal memory, k2 is using 52% CPU but lots of room still open for new jobs. Ignore the master node as that one only handles cluster management and doesn't run jobs or pods for users.		That means the worker nodes, k1, k2 and k3, are using minimal memory, k2 is using 52% CPU but lots of room still open for new jobs. Ignore the master node as that one only handles cluster management and doesn't run jobs or pods for users.

			Another good way to get a lot of details about the current state of the cluster is through the Kubernetes Dashboard:

			https://cgl-k8s-dashboard.gi.ucsc.edu/

			Select the "token" login method, and paste in this(long) token:

			eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi0ycDY4cCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImE5ZGI2Y2I0LWMyYWYtNDM3My04ZmM2LWE4YWYwYTBmNGRkNCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.ZVQjryG_ksfvReIfq4Frb6M4sE6OVDOXnFy9Aii-h3mrpdHRE6bgjdAvSGZ0jJSIUEz5GgPBQ0lCwhyZocivHHr4zTrNxMkOFZhPDnpvF6RVIDWTkqmH9Dg6qmro0gTJP75oKBpt7dFN2pW4zvqOAzqPmh7qxfoVusN8X6U13YirMFEf65-aGL-_FFNBsEzvjkC-BgXWbtk3YZc8CJL7xtvlKLyE6u6jC9Qx0SWnwzkALlxmzo_yYTDKpIrWiQGEqzLQOxKml-H0kSYLDX-t4sTivXp4vCw_ruoqwIpLnnQAC7q3ZtSTxHIrxbB7n_M8gfhpXtwprbPav-XmBk1xaQ

			The dashboard is read-only, so you won't be able to edit anything, it's mostly for seeing what's going on and where.

	You can also take a look at current resource consumption by taking a look at our Ganglia Cluster monitor tool:		You can also take a look at current resource consumption by taking a look at our Ganglia Cluster monitor tool:

Weiler: /* Running Pods and Jobs with Requests and Limits */

2019-12-02T23:32:26Z

Running Pods and Jobs with Requests and Limits

← Older revision		Revision as of 16:32, 2 December 2019
Line 68:		Line 68:
	ephemeral-storage: "2G"		ephemeral-storage: "2G"
	limits:		limits:
	cpu: "2"		cpu: "1"
	memory: "3G"		memory: "2G"
	ephemeral-storage: "3G"		ephemeral-storage: "3G"
	command: ["/bin/bash", "-c"]		command: ["/bin/bash", "-c"]
Line 75:		Line 75:
	restartPolicy: Never		restartPolicy: Never

	~~'''NOTE:''' Jobs~~ and ~~pods that have '''completed''' over 48 hours ago but have not been cleaned up will~~ be ~~automatically removed by~~ the ~~garbage collector~~. ~~Most jobs will have~~ the ~~"ttlSecondsAfterFinished" configuration item~~ in ~~them, so~~ they ~~will automatically cleaned up after that time expires~~, ~~but leaving~~ the ~~old pods~~ and ~~jobs around pins~~ the ~~disk space they were using while they remain~~, ~~so it's good to get rid of them as soon as they are done unless you are debugging a failure or something like that~~.		Please note that the "request" and "limit" item fields should be the same. You would think that you could set the limit higher than the request, but in reality they need to match in order for the pod to stay within the kubernetes resource limit bubble. If you set the limit higher than the request, then you are risking the pod using more memory than the scheduler expects, and the node can start killing off random other colocated pods by way of OOM, which is very, very bad for the cluster.

	Jobs that '''run''' over 48 hours will not be deleted, only the ones that have '''exited''' over 48 hours ago.		'''NOTE:''' Jobs and pods that have '''completed''' over 72 hours ago but have not been cleaned up will be automatically removed by the garbage collector. Most jobs will have the "ttlSecondsAfterFinished" configuration item in them, so they will automatically cleaned up after that time expires, but leaving the old pods and jobs around pins the disk space they were using while they remain, so it's good to get rid of them as soon as they are done unless you are debugging a failure or something like that.

			Jobs that '''run''' over 72 hours will not be deleted, only the ones that have '''exited''' over 72 hours ago.

	A lot of other good information can be viewed on Rob Currie's github page, which includes examples and some "How To" documentation:		A lot of other good information can be viewed on Rob Currie's github page, which includes examples and some "How To" documentation:

Weiler: /* Running Pods and Jobs with Requests and Limits */

2019-09-23T19:40:55Z

Running Pods and Jobs with Requests and Limits

← Older revision		Revision as of 12:40, 23 September 2019
Line 43:		Line 43:
	==Running Pods and Jobs with Requests and Limits==		==Running Pods and Jobs with Requests and Limits==

	When running jobs and pods on kubernetes, you will always want to specify "requests" and "limits" on resources, otherwise ~~you~~ pods will get stuck with the default limits which are tiny (to protect against runaway pods). You should always have an idea of how much resources will be consumed by your jobs, and not use much more than that, in order not to hog all the resources. It also prevents your jib from "running away" unexpectedly and chewing up more resources than expected.		When running jobs and pods on kubernetes, you will always want to specify "requests" and "limits" on resources, otherwise your pods will get stuck with the default limits which are tiny (to protect against runaway pods). You should always have an idea of how much resources will be consumed by your jobs, and not use much more than that, in order not to hog all the resources. It also prevents your jib from "running away" unexpectedly and chewing up more resources than expected.

	Here is a good example of a job file that specifies limits:		Here is a good example of a job file that specifies limits: