Create a script with the following structure:
#!/bin/bash #SBATCH -J my_project_job #SBATCH --mail-type=END #SBATCH --error=/home/$USER/project<number>/"%x.err.%j" #SBATCH --output=/home/$USER/project<number>/"%x.out.%j" #SBATCH -n 1 # Number of cores #SBATCH --mem-per-cpu=1600 # Main memory in MByte per MPI task #SBATCH -t 01:30:00 # Hours, minutes and seconds, or '#SBATCH -t 10' = only minutes # ------------------------------- srun singularity run -o overlay.img lent_latest.sif foamVersion
The folder project should exist in your $HOME folder. Otherwise you get an error e-mail from HPC and the job will fail.
The command following will not be as simple as foamVersion of course. Remember that within the %runscript section you can specify and abstract away commands and also read arguments provided from the run. For example say we have a command named lentFoam which is available only after the source of some bashrc’s and runs depending on the current working directory. Then we can put the following commands in the %runscript section:
%runscript source /opt/OpenFOAM/OpenFOAM-v1806/etc/bashrc source /opt/OpenFOAM/lent/etc/bashrc cd $1 lentFoam
and initiate our run with the following command:
srun singularity run -o overlay.img lent_latest.sif /opt/OpenFOAM/lent/cases/some/random/folder/for/simulation
Here we only provide a path, which is being picked up by the %runscript (as $1 argument) and is used to change to that directory. Then the lentFoam command is called. Careful, this %runscript might be too restrictive depending on the application. It is up to the user to decide the level of automation and abstraction inside the %runscript.
Run the following command to start the script:
Depending on the way the container permission rights were setup and the output folders of the execution commands, the data should always end up inside the overlay image.