Simple C, C++, Python program

Start with a C++ program, we can start with a simple C++ program, to print out a text to a file, i.e. example2.cpp

#include <iostream>
#include <fstream>
using namespace std;

int main () {
  ofstream myfile;
  myfile.open ("example2.txt");
  myfile << "Writing this to a file.\n";
  myfile.close();
  return 0;
}

You can choose to compile your program first, or you can compile it on a worker node. You may want to compile on the worker node, in case your program is sensitive to the hardware, e.g. can run with CPU-only, CPU+GPU and/or GPU-only depend on the availability of the worker node.

We now compile the example2 first

make example2

Then you prepare the Slurm submission script, e.g.

#!/bin/bash
#
#SBATCH --qos=cu_hpc
#SBATCH --partition=cpu
#SBATCH --job-name=example2
#SBATCH --output=example2_log.txt
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --time=00:10:00
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=1G

module purge

#To handle PATHs
export MYCODEDIR=`pwd`
echo "MYCODEDIR = "$MYCODEDIR
echo "TMPDIR = "$TMPDIR

#To run C++ program
cd $TMPDIR
cp $MYCODEDIR/example2 .
chmod a+x example2
rm -rf example2.txt
./example2
cp -rf example2.txt $MYCODEDIR

You should get example2.txt as an output from your program, and example2_log.txt as the log from the Slurm.

For C and Python program, user can follow the same way:

1. Create and test your program

2. Write the submission script

3. Submit jobs to Slurm clusters, and get the output back to your working directory

However, for Python, a user may need specific packages or specific versions which are not installed centrally. User can also set the virtual environment, install on what are needed. You can see example in Python with VirtualEnv.