Install

Dependencies

The software requirements for SCOOP are as follows:

Prerequisites

Linux

You must have the Python headers (to compile pyzmq and greenlet) and pip installed. These should be simple to install using the package manager provided with your distribution.

To get the prerequisites on an Ubuntu system, execute the following in a console:

sudo apt-get install python-dev python-pip

Ensure that your compiler is GCC as it is the tested compiler for pyzmq and greenlet.

Mac

The easiest way to get started is by using Homebrew. Once you’ve brewed your Python version and ZeroMQ, you are ready to install SCOOP.

Windows

Please download and install pyzmq before installing SCOOP. This can be done by using the binary installer provided at their download page. These installers will provide libzmq alongside pyzmq.

You can install pip on windows using either Christoph Gohlke windows installers or the get-pip.py script as shown in the pip-installer.org webpage.

Installation

To install SCOOP, use pip as such:

pip install scoop

POSIX Operating systems

Connection to remote hosts is done using SSH. An implementation of SSH must be installed in order to be able to use this feature.

Windows Operating System

On Windows, this will try to compile libzmq. You can skip this compilation by installing pyzmq using the installer available at their download page. This installer installs libzmq alongside pyzmq.

Furthermore, to be able to use the multi-system capabilities of SCOOP, a SSH implementation must be available. This may be done either by using Cygwin or OpenSSH for Windows.

Remote usage

Because remote host connection needs to be done without a prompt, you must use ssh keys to allow passwordless authentication between every computing node. You should make sure that your public ssh key is contained in the ~/.ssh/authorized_keys file on the remote systems (Refer to the ssh manual). If you have a shared /home/ over your systems, you can do as such:

[~]$ mkdir ~/.ssh; cd ~/.ssh
[.ssh]$ ssh-keygen -t dsa
[.ssh]$ cat id_dsa.pub >> authorized_keys
[.ssh]$ chmod 700 ~/.ssh ; chmod 600 ./id_dsa ; chmod 644 ./id_dsa.pub ./authorized_keys

Note

If your remote hosts needs special configuration (non-default port, some specified username, etc.), you should do it in your ssh client configuration file (by default ~/.ssh/config).

Note

The following parameters of ssh are used by SCOOP:

  • -x : Deactivates X forwarding
  • -n : Prevents reading from stdin (batch mode)
  • -oStrictHostKeyChecking=no : Allow the connection to hosts ssh sees for the first time. Without it, ssh interactively asks to accept the identity of the peer.

HPC usage

If you use an Infiniband network, you may want to use an RDMA accelerated socket alternative instead of TCP over IB. In order to do so, you can use libsdp. This can be done by performing the following steps:

$ wget https://www.openfabrics.org/downloads/libsdp/libsdp-1.1.108-0.17.ga6958ef.tar.gz
$ tar xfvz libsdp-1.1.108-0.17.ga6958ef.tar.gz
$ cd libsdp-1.1.108
$ ./configure --prefix=$HOME && make && make install

Once the compilation is done, you can use it by creating a file containing this (for bash):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/lib/
export LD_PRELOAD=libsdp.so

By passing this file to the --prolog parameter of SCOOP, SDP sockets will be used instead of TCP over IB.