Slurm Tips for vg

This page explains how to set up a development environment for vg on the Phoenix cluster.

1. Make yourself a user directory under /private/groups, which is where large data must be stored. For example, if you are in the Paten lab:

mkdir /private/groups/patenlab/$USER

2. (Optional) Link it over to your home directory, so it is easy to use storage there to store your repos. The /private/groups storage may be faster than the home directory storage.

mkdir -p /private/groups/patenlab/$USER/workspace
ln -s /private/groups/patenlab/$USER/workspace ~/workspace

3. Make sure you have SSH keys created and add them to Github.

cat ~/.ssh/id_ed25519.pub || (ssh-keygen -t dsa && cat  ~/.ssh/id_ed25519.pub)
# Paste into https://github.com/settings/ssh/new

4. Make a place to put your clone, and clone vg:

mkdir -p ~/workspace
cd ~/workspace
git clone --recursive git@github.com:vgteam/vg.git
cd vg

5. vg's dependencies should already be installed on the cluster nodes. If any of them seem to be missing, tell cluster-admin@soe.ucsc.edu to install them.

6. Build vg as a Slurm job. This will send the build out to the cluster as a 64-core, 80G memory job, and keep the output logs in your terminal.

srun -c 64 --mem=80G make -j64

This will leave your vg binary at ~/workspace/vg/bin/vg.