Singularity is a container software. It is basically similar to Docker .
For more information please refer to the literature.
Here are the instructions on how to use the Singularity container for the geophase project.
You can use the following commands with the .sif produced by the Definition file (geophase.def) at the end of the article.
clone : To get the project from the container to the current working directory.
./geophase.sif clone
build: By default the project in the /opt/geophase/ path will be built on a folder named “build” on the host current working directory.
./geophase.sif build
To build a modified geophase project which lies on another path (e.g. ~/modified_geophase) provide the path as a second argument. This way the modified project will use the environment of the container to build inside that folder.
./geophase.sif build ~/modified_geophase/
test: By default the project in the /opt/geophase/ path will be tested on a folder named “test” on the host current working directory. To test a modified geophase project which lies on another path (e.g. ~/modified_geophase) provide the path as a second argument. This way the modified project will use the environment and the build of the container to test inside that folder.
./geophase.sif test ~/modified_geophase/
If a project path is provided, that project has to have already been built inside a build/ folder.
getdata: So far getdata just copies all .csv .png and .vtk files from the build folder of the container to the host current working directory.
In order to make use of the Jupyter notebook , it needs to have a writable filesystem available. For that we have to create an overlay images and embed it inside the read-only container.
To create an overlay:
dd if=/dev/zero of=overlay.img bs=1M count=500 && mkfs.ext3 overlay.img
Note: This will create a 500 MB overlay. Adjust by changing the block size (bs) and the count.
To embed the overlay into the container (named geophase.sif in this example):
singularity sif add --datatype 4 --partfs 2 --parttype 4 --partarch 2 --groupid 1 geophase.sif overlay.img
–datatype 4 means the type of data is Partition, –partfs 2 means the filesystem is Ext3, –parttype 4 means its Overlay partition, –partarch 2 means architecture is amd64 (x86-64).
To run the writable container:
singularity shell --writable geophase.sif
Now, you can create/delete/modify files inside the container, provided that you also have the permissions to do so and these changes are transfered along with the .sif.
To inspect the container:
This command will give you information about the filesystems of the container. Example output provided and we can see that at the end there is a filystem of type Overlay Ext3 which is the one we embedded.
singularity sif list geophase.sif
Container id: 275b9e9f-e6cc-4cb4-b737-299f8552da54
Created on: 2020-04-03 11:57:04 +0200 CEST
Modified on: 2020-04-03 15:16:45 +0200 CEST
----------------------------------------------------
Descriptor list:
ID |GROUP |LINK |SIF POSITION (start-end) |TYPE
------------------------------------------------------------------------------
1 |1 |NONE |32768-36250 |Def.FILE
2 |1 |NONE |36864-36984 |JSON.Generic
3 |1 |NONE |40960-1126567936 |FS (Squashfs/*System/amd64)
4 |1 |NONE |1126567936-1650855936 |FS (Ext3/Overlay/amd64)
To remove an embedded overlay:
singularity sif del 4 geophase.sif
The number after the del command indicates the ID of the object descriptor (in this case the Ext3-Overlay). Important: After this, everything that was saved in the overlay will be deleted and will not be retrievable.
To build the container use:
sudo singularity build geophase.sif geophase.def
Bootstrap: docker
From: archlinux:latest
%post
pacman -Sy --noconfirm reflector
reflector --latest 5 --sort rate --save /etc/pacman.d/mirrorlist
pacman -Syu --noconfirm gcc cmake eigen python python-matplotlib texlive-core python-seaborn python-pandas jupyter make git png++ boost texlive-latexextra
cd /opt/
mkdir geophase
git clone --branch feature/geom-vof-init https://git.rwth-aachen.de/leia/geophase.git geophase/
cd geophase/
mkdir build && cd build/
cmake -DCMAKE_BUILD_TYPE=Release ..
make
chmod -R 777 /opt/geophase/
%runscript
# CMD will carry the command the user provided.
CMD=$1
# WORKDIR will be pointed to the location where files will be built and/or tested.
WORKDIR=/opt/geophase/
# PROJDIR will be pointing to the location of the project.
PROJDIR=`cd ${2:-$WORKDIR} && pwd`
# Check whether user uses overlay or not.
OVERLAY=`cat /proc/mounts | grep overlay | cut -d " " -f 4 | echo $(head -c 2)`
if [ "$OVERLAY" = "rw" ]; then
echo "You are using an Overlay. Everything will be saved in $(echo $WORKDIR)"
elif [ "$OVERLAY" = "ro" ]; then
WORKDIR=$(echo $PWD)
echo "You are not using an Overlay. Everything will be saved in $(echo $WORKDIR)"
else
echo "Cannot determine if using an Overlay or not."
fi
# Move to the wokring directory. Either /opt/geophase/ if overlay is used, or the users PWD if not.
cd $WORKDIR
case $CMD in
getdata) # Copy *.csv *.vtk *.png files to the $WORKDIR.
echo "Copying data into folder data/"
mkdir -p data/
cp /opt/geophase/build/*.csv data/
cp /opt/geophase/build/*.vtk data/
cp /opt/geophase/build/*.png data/
;;
build) # Create a new build folder and rebuild geophase project.
echo "Building geophase($PROJDIR) into folder build/"
# Check if a project directory was provided instead of the default one.
if [ $# -eq 2 ]; then
cd $PROJDIR
fi
# Check if a "build" folder already exists in the project directory.
if [ -d "build" ]; then
while true; do
read -p "A folder named "build" already exists in $WORKDIR. Do you want to overwrite it? " yn
case $yn in
[Yy]* ) rm -rf build/*; break;;
[Nn]* ) exit;;
* ) echo "Please answer with a Y/y or N/n.";;
esac
done
fi
mkdir build/ && cd build/
cmake -DCMAKE_BUILD_TYPE=Release $PROJDIR
make
;;
test) # Create a testing folder and retest the geophase binaries.
echo "Testing geophase($PROJDIR) into folder tests/"
# Check if a project directory was provided instead of the default one.
if [ $# -eq 2 ]; then
cd $PROJDIR
fi
mkdir -p tests/ && rm -rf tests/* && cd tests/
cp $PROJDIR/build/CTestTestfile.cmake .
ctest -V
rm CTestTestfile.cmake
;;
clone) # Copy the 'geophase' project to the CWD without the 'build' folder.
echo "Copying the geophase source code to $(echo $WORKDIR)/geophase"
mkdir -p geophase/
cp -r `ls -Ad /opt/geophase/* | grep -v "build"` ./geophase/
;;
jupyter)
# Check if a project directory was provided instead of the default one.
if [ $# -eq 2 ]; then
cd $PROJDIR/
fi
cd jupyter-notebooks/
jupyter notebook --no-browser --port=8889
esac
%test
cd /opt/geophase/build
ctest -V
%help
This container consists of the environment, the code, the build and the tests for the geophase project (https://git.rwth-aachen.de/leia/geophase).
Usage:
./geophase.sif <command>
singularity run geophase.sif <command>
singularity shell [-o overlay.img] geophase.sif (use with overlay)
singularity shell --writable geophase.sif (use with embedded overlay)
The commands are:
clone copies the geophase from /opt/geophase to the current working directory
getdata copes all *.csv *.vtk *.png files from /opt/geophase/build to the current working directory
build [path-to-project] builds the geophase project. By default, the '/opt/geophase'
test [path-to-project] run the tests on the geophase project. By default, the '/opt/geophase/build'
jupyter [path-to-project] runs a Jupyter notebook. By default, the '/opt/geophase/jupyter-notebooks/geophase-interface-positioning.ipynb'