Introduction

ECC is an instance within the CloudVeneto cloud infrastructure. Access to ECC is granted through a two-stage process:

1. CloudVeneto registration – Obtain a CloudVeneto/OpenStack account.
2. ECC account creation – Submit an ECC access request describing your intended use of the infrastructure.

Only after completing both steps can you connect to the ECC infrastructure.

Access request

CloudVeneto registration

To request access to ECC, you must first register for a CloudVeneto account through the Horizon OpenStack web interface, as described in the CloudVeneto User Guide.

1. Open https://cloudveneto.ict.unipd.it/dashboard in your browser and click the Register.
2. Log in with your Unipd Single Sign-On account (an email address ending in @unipd.it or @studenti.unipd.it) by clicking on the Unipd logo.
3. Complete the User Registration form, filling in all required fields and selecting ELIXIR Compute Cloud as the existing project.
4. Submit the registration form.

Once your CloudVeneto account has been created, you can proceed with the ECC access request.

ECC access request

Complete the Request form using the link below. The form collects information about your project and the intended use of ECC resources, which will be used by the ECC administrators to evaluate your request.

After submission, your request will be reviewed by the ECC administrators. You will receive an email notifying you whether your request has been approved or denied.

If your request is approved, you will receive an email containing the ECC Acceptable Use Policy (AUP). You must accept the AUP terms before access can be granted.

After the AUP has been accepted, the required accounts will be provisioned and credentials will be sent to you by email.

You will receive:

1. A message from CloudVeneto support containing the credentials required to access the CloudVeneto Gate machine (cv_user and cv_pass).
2. A message from the ECC administrators containing the credentials required to access the ECC infrastructure ECC infrastructure (ecc_user, ecc_pass, headnode_ip).

Connect to the ECC

After receiving your credentials, you can access ECC via the CloudVeneto Gate machine and connect to the ECC head node using the Secure Shell Protocol (SSH). In the following examples, replace cv_user, cv_pass, ecc_user, ecc_pass and headnode_ip with your parameters:

# connect to CloudVeneto Gate machine (it will ask for cv_pass) and create a new password
ssh cv_user@gate.cloudveneto.it

# from Gate machine, connect to ECC headnode (it will ask for ecc_pass) and create a new password
ssh ecc_user@headnode_ip

For a smoother connection experience, it is recommended to use SSH key-based authentication instead of password authentication.

To create SSH keys for authentication and use them to connect to the CloudVeneto Gate, run the following command:

# generate a new key pair locally
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_ecc

# copy the public key to CloudVeneto Gate machine (it will ask for cv_pass)
cat ~/.ssh/id_ed25519_ecc.pub | \
  ssh cv_user@gate.cloudveneto.it 'cat >id_ed25519_ecc.pub && \
  mkdir -p .ssh && \
  chmod 700 .ssh && \
  mv id_ed25519_ecc.pub .ssh/id_ed25519_ecc.pub && \
  cat .ssh/id_ed25519_ecc.pub >>.ssh/authorized_keys'

# copy the private key to CloudVeneto Gate machine (it will ask for cv_pass)
cat ~/.ssh/id_ed25519_ecc | \
  ssh cv_user@gate.cloudveneto.it \
  'cat >.ssh/id_ed25519_ecc && chmod 600 .ssh/id_ed25519_ecc'

# connect to CloudVeneto Gate machine (it will ask for SSH key passphrase, if used)
ssh -i ~/.ssh/id_ed25519_ecc cv_user@gate.cloudveneto.it

# copy the public key from the Gate machine to ECC headnode (it will ask for ecc_pass)
cat ~/.ssh/id_ed25519_ecc.pub | \
  ssh ecc_user@headnode_ip 'cat >id_ed25519_ecc.pub && \
  mkdir -p .ssh && \
  chmod 700 .ssh && \
  mv id_ed25519_ecc.pub .ssh/id_ed25519_ecc.pub && \
  cat .ssh/id_ed25519_ecc.pub >>.ssh/authorized_keys'

# test connection to ECC from Gate machine (it will ask for SSH passphrase, if used)
ssh -i ~/.ssh/id_ed25519_ecc ecc_user@headnode_ip
exit

Accessing ECC from your local machine requires proxying the SSH connection through the CloudVeneto Gate. You can achieve this by using the following SSH command:

# (optionally) add key to ssh-agent (it may ask for SSH key passphrase)
ssh-add ~/.ssh/id_ed25519_ecc

# connect to ECC via proxy
ssh -i ~/.ssh/id_ed25519_ecc \
  -o StrictHostKeyChecking=accept-new \
  -o ProxyCommand="ssh -i ~/.ssh/id_ed25519_ecc \
  -W %h:%p cv_user@gate.cloudveneto.it" \
  ecc_user@headnode_ip

You can simplify the SSH connection to ECC by configuring your SSH config file:

# update ssh config with proxy and headnode
cat <<EOF | tee -a ~/.ssh/config

Host cvgate
	HostName gate.cloudveneto.it
	User cv_user
	IdentityFile ~/.ssh/id_ed25519_ecc

Host ecc
	HostName headnode_ip
	User ecc_user
	IdentityFile ~/.ssh/id_ed25519_ecc
	UserKnownHostsFile /dev/null
	StrictHostKeyChecking=accept-new
	ProxyJump cvgate
EOF

# connect to ECC
ssh ecc

# copy files to and from ECC with scp
scp localdir/file ecc:remotedir/
scp ecc:remotedir/file localdir/
# or rsync
rsync -ahv localdir/ ecc:remotedir/
rsync -ahv ecc:remotedir/ localdir/

Job submission example

The following example submits a dummy, CPU-intensive task using two threads in parallel to the Slurm scheduler:

# create work folder
mkdir slurm-test && cd slurm-test

# create simple.sh worker script
cat <<'EOF' | tee simple.sh
#!/bin/bash
#SBATCH -J simplejob
#SBATCH -o "%x"."%A"."%a".out
#SBATCH -e "%x"."%A"."%a".err
#SBATCH --mail-type=ALL
echo -e "$(date)\tStarting job $SLURM_JOB_ID:$SLURM_ARRAY_TASK_ID on $SLURMD_NODENAME ..."
if [ -n "$1" ]; then
    rnd=$1
else
    rnd=$(shuf -i 5-30 -n 1)
fi
echo "working for $rnd s ...";
yes > /dev/null &
ypid=$!
yes > /dev/null &
ypid2=$!
sleep $rnd
echo "killing job $ypid ..."
{ kill $ypid && wait $ypid; } 2>/dev/null
echo "killing job $ypid2 ..."
{ kill $ypid2 && wait $ypid2; } 2>/dev/null
echo “all done, exiting with 0”
ex=$?
echo -e "$(date)\tJob $SLURM_JOB_ID:$SLURM_ARRAY_TASK_ID ended with $ex"
exit $ex
EOF

# submit as an array job allocating 2 CPUs per job (max runtime of 1min; max 1G memory per job)
rm -f *.{err,out}; sbatch -n2 -a 1-5 --time 1 --mem=1G simple.sh
rm -f *.{err,out}; sbatch -n2 -a 1-5 --time 1 --mem=1G simple.sh 120 # these jobs should timeout

# monitor Slurm queues and job status
watch -d "sinfo -S '-P' -o '%8N %9P %.5T %.13C %.8O %.8e %.6m %.8d %.6w %.8f %20E'|cut -c-\$COLUMNS; echo; echo; squeue --format='%12i %10j %6u %8N %4P %4C %7m %8M %10T %16R %o'|cut -c-\$COLUMNS; echo; echo; sacct -X -a --format JobID,User,JobName,Partition,AllocCPUS,State,ExitCode,End,ElapsedRaw|tail|tac|grep -v 'JobID\|^---'|awk 'BEGIN{print \"       JobID      User    JobName  Partition  AllocCPUS      State ExitCode                 End ElapsedRaw\n------------ --------- ---------- ---------- ---------- ---------- -------- ------------------- ----------\"}{print}'|cut -c-\$COLUMNS"

Privacy policy

The complete privacy policy for the ECC is available here.

Get support

For any issues, questions, problems concerning ECC access, please use the support request form or contact ecc@elixir.biomed.unipd.it.