Practical session

This session will provide a practical illustration of the experimental workflow introduced during the presentation. It will cover the steps of the experimental workflow, namely:

Account creation and first steps
Resource allocation
Deployment of an experimental environment
Launching an experiment
Gathering results

During this session, users will be asked to get a resource allocation, and then they will develop a python script that will interact with Grid'5000 API to:

Find information about a resource allocations, whose ID would be given as an argument
List the nodes of the reservation
Deploy an environment on them
Upload a local bash script
Execute the local bash script
Collect its results

I- Account creation and first steps

A- Creating a G5K account

Firstly, create a Grid'5000 account by visiting this link : Get_an_account.

B- Generating a SSH key pair

Run the following code in a terminal to generate a SSH key :

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

display the content of the generated public key :

cat ~/.ssh/id_rsa.pub

Now we will add some configuration to ssh to ease connection to Grid'5000. Add the following lines to the .ssh/config file located in your home folder :

(username=)

Host *.g5k
	User __username__
	ProxyCommand ssh __username__@access-north.grid5000.fr "nc -q 0 `basename %h .g5k` %p"
	StrictHostKeyChecking no

and upload it in the Grid'5000 user management system

II- Resource allocation

First let see what are the resources that are available. It can be done in two ways:

Checking the gantt charts
Using funk

We will now make a reservation of two nodes of the econome cluster, during one hour. To do so, we will use funk as follow :

funk -m free -r econome:2 -w 1:00:00 -o "-t deploy"

Now connect to nantes:

ssh nantes.g5k

III- Deployment of an experimental environment

Wait until your reservation is running. Once it is the case, connect to a frontend :

ssh nantes.g5k

And connect to the OAR job :

(oar_job_id=)

oarsub -C __job_id__

And run the kadeploy command to deploy an operating system (debian) on the nodes you got with the reservation :

kadeploy3 -f $OAR_NODE_FILE -e jessie-x64-min -k

Now, wait until the deployment is finished (it takes approximately 2 minutes).

IV- Launching an experiment

As the nodes of the reservation are deployed, we will now run some software on them.

Create a local bash script launch_experiment.sh that contains the following instructions :

#!/usb/bin/env bash

echo $(hostname) > output.txt

exit 0

and upload it on nodes of the reservation :

uniq $OAR_NODE_FILE > uniq_hosts.txt
while read -r HOST
do
    scp launch_experiment.sh root@$HOST:launch_experiment.sh
done < uniq_hosts.txt

and run the script on nodes of the reservation :

uniq $OAR_NODE_FILE > uniq_hosts.txt
while read -r HOST
do
    ssh root@$HOST "bash launch_experiment.sh" < /dev/null
done < uniq_hosts.txt

V- Gathering results

mkdir results
uniq $OAR_NODE_FILE > uniq_hosts.txt
while read -r HOST
do
    scp root@$HOST:output.txt results/${HOST}_output.txt
done < uniq_hosts.txt

tar -zcvf results.tgz results

VI- Automatization using Execo

It is possible to automate the execution of these steps by python and the Execo library.

First install python and PIP (a dependency manager for Python). Once it is done, install Execo :

Once Execo is installed, create a ~/.execo.conf.py file in your home folder, with the following content :