A Brief Introduction to Texas A&M High Performance Research Computing (TAMU-HPRC)

3 minute read

Published:

Overview

TAMU provides free HPRC accounts for everyone

  • Students can have free 5000 SUs per year
  • Advisors can apply for almost unlimited SUs then assign to students

Please be aware of other free computing resources

Background

Basics

  • Google is your friend, and learn to use google search operators when searching
  • Everything is in command line on HPRC, so you need to learn basic command operations (a quick tutorial on lifehacker)
    • make sure you are familiar with commands: cd, ls, mkdir, mv, rm -r, tail -f
    • On WindowsOS: always use PowerShell (do not use cmd), always use Putty (instead of ssh in Powershell)
    • On MacOS: use iterm2
    • On Linux: Congrats! You made the right choice of OS!

A little more

Software Recommendations

In general

  • use git to manage your code (NOT data)
  • (optional) use software like Fugu/WinSCP to sync large files (e.g., your data and tensorflow model), command rsync is another option

Windows

  • best ssh tool: Putty
  • best FTP tool for data transfer: WinSCP

MacOS

Linux

Introduction to TAMU-HPRC

You need to replace NETID below with your own netid

Get Started

  • Apply for an account here
  • Check the hardware summary of different clusters, make sure the cluster meets your requirements (e.g., with GPUs): ada, terra, currie
  • TAMU HPRC staff provide detailed and excellent guides to beginners, make sure you carefully read through these pages: Ada Quick Start Guide, Terra Quick Start Guide

Basic Usage

Suggestions:

  • Test your code on small cases locally (your laptop), try avoiding debugging on HPRC since it could be hard if you are used to IDEs
  • Use folder /scratch/user/NETID/ if you need more storage space (up to 1T), the home folder /home/NETID/ is limited (10GB)

Submit jobs:

  • Matlab Users, HPRC provides a good tool matlabsubmit, see usage here
  • Other Users: use qsub with various options. Some other commonly used commands (note: all these commands have detailed explanations in the quick-start-guide of every cluster)
    • qsub: submit a job to HPRC (common options -w)
    • bjobs: check job status
    • bkill XXXX: cancel a job
    • try to type -h after each command and learn more about them.
  • Monitoring jobs:
    • bjobs JOBID will show the progress of a specific job
    • each job will create a log file (perhaps *.out) , command tail will show the last part of the logfile, and tail -f willl keep showing the end of the file (whenever it is updated)

More

  • check available packages using module spider
  • load a module module load MODULENAME