Duke Computing Resources- An Overview

  • DSS Studio Server: knight.stat.duke.edu:8787/

    • Can switch between knight, rook, algebra2, and others. Someone please tell me the difference between these!
    • You have to be on VPN to use the server.

  • OIT virtual machine: helpful if the dss Studio server is crashing, or for using other languages.

  • Duke Computing Cluster:

    • The OIT DCC website is a super helpful resource.
    • You can check out an Rstudio server on the cluster. Information here. The DCC OnDemand login site is dcc-ondemand-01.oit.duke.edu.
    • If you aren’t able to log on to any of these DCC resources, you might need to request access. Check rtoolkits to see if you have access.

Duke Computing Cluster- The Basics

Simple steps to use the cluster through your terminal and use slurm:

  1. To log-on, execute the following in the terminal,

     ssh NETID@dcc-login.oit.duke.edu

  2. Navigate to your desired working directory. Details here on the differences between the various directories. I tend to use,

     cd /hpc/group/LAB/NETID

  3. Add/remove files with git or by using sftp:

    • Replace ssh with sftp upon login. Note that tab-complete often doesn’t work well when in an sftp environment.
    • To upload files from your computer to the cluster, execute put FILENAME.
    • To download files from the cluster to your computer, execute get FILENAME.
    • Some useful options: -r for working with directories, -C for working with large files

  4. Life is easier if you can edit files with emacs (or similar) in the terminal. To use emacs, open a file by executing emacs FILENAME.

    • If you’re going to be using a text editor in the terminal, I recommend doing a hot key tutorial.
    • Some useful emacs commands: C-x C-s to save changes, C-x C-c to exit emacs

  5. Submit a job to slurm by executing sbatch FILENAME.sh (see DCC website for helpful resources on setting up your .sh file), check on the status of your jobs with squeue -u NETID, and to cancel jobs use scancel JOB##.