Examples

You can find the examples detailed on this page and more in the examples/ directory of SCOOP.

Please check the API Reference for any implentation detail of the proposed functions.

Introduction to the map() function

A core concept of task-based parallelism as presented in SCOOP is the map. An introductory example to map working is presented in examples/map_doc.py.

1
2
3
4
5
6
7
8
9
from __future__ import print_function
from scoop import futures

def helloWorld(value):
    return "Hello World from Future #{0}".format(value)

if __name__ == "__main__":
    returnValues = list(futures.map(helloWorld, range(16)))
    print("\n".join(returnValues))

Line 1 allows Python 2 users to have a print function compatible with Python 3.

On line 2, SCOOP is imported.

On line 4-5, the function that will be mapped is declared.

The condition on line 7 is a safety barrier that prevents the main program to be executed on every workers. It ensures that the map is issued only by one worker, the root.

The map() function is located on line 8. It launches the helloWorld function 16 times, each time with a different argument value selected from the range(16) argument. This method is compatible with the standard Python map() function and thus can be seamlessly interchanged without modifying its arguments.

The example then prints the return values of every calls on line 9.

You can launch this program using python -m scoop. The output should look like this:

~/scoop/examples$ python -m scoop -n 8 map_doc.py
Hello World from Future #0
Hello World from Future #1
Hello World from Future #2
[...]

Note

Results of a map are always ordered even if their computation was made asynchronously on multiple computers.

Note

You can toy around with the previous example by changing the second parameter of the map() function. Is it working with string arrays, pure strings or other variable types?

Computation of \pi

A Monte-Carlo method to calculate \pi using SCOOP to parallelize its computation is found in examples/pi_calc.py. You should familiarize yourself with Monte-Carlo methods before going forth with this example.

Monte Carlo computation of Pi.

Image from Wikipedia made by CaitlinJo that shows the Monte Carlo computation of \pi.

First, we need to import the needed functions as such:

1
2
3
from math import hypot
from random import random
from scoop import futures

The Monte-Carlo method is then defined. It spawns two pseudo-random numbers that are fed to the hypot function which calculates the hypotenuse of its parameters. This step computes the Pythagorean equation (\sqrt{x^2+y^2}) of the given parameters to find the distance from the origin (0,0) to the randomly placed point (which X and Y values were generated from the two pseudo-random values). Then, the result is compared to one to evaluate if this point is inside or outside the unit disk. If it is inside (have a distance from the origin lesser than one), a value of one is produced (red dots in the figure), otherwise the value is zero (blue dots in the figure). The experiment is repeated tries number of times with new random values.

The function returns the number times a pseudo-randomly generated point fell inside the unit disk for a given number of tries.

1
2
def test(tries):
    return sum(hypot(random(), random()) < 1 for _ in range(tries))

One way to obtain a more precise result with a Monte-Carlo method is to perform the method multiple times. The following function executes repeatedly the previous function to gain more precision. These calls are handled by SCOOP using it’s map() function. The results, that is the number of times a random distribution over a 1x1 square hits the unit disk over a given number of tries, are then summed and divided by the total of tries. Since we only covered the upper right quadrant of the unit disk because both parameters are positive in a cartesian map, the result must be multiplied by 4 to get the relation between area and circumference, namely \pi.

1
2
3
def calcPi(nbFutures, tries):
    expr = futures.map(test, [tries] * nbFutures)
    return 4. * sum(expr) / float(nbFutures * tries)

As previously stated, you must wrap your code with a test for the __main__ name.

1
2
if __name__ == "__main__":
    print("pi = {}".format(calcPi(3000, 5000)))

You can now run your code using the command python -m scoop.

Sharing Constant

A typical usage of the shared constants is to broadcast a value or an object that must be created at runtime and read by every worker, as the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

def getValue(words):
    """Computes the sum of the values of the words."""
    value = 0
    for word in words:
        for letter in word:
            # shared.getConst will evaluate to the dictionary broadcasted by
            # the root Future
            value += shared.getConst('lettersValue')[letter]
    return value


if __name__ == "__main__":
    # Set the values of the letters according to the language and broadcast it
    # This list is set at runtime
    import sys
    if len(sys.argv) > 1 and sys.argv[1] == 'francais':
        shared.setConst(lettersValue={'a': 1, 'b': 3, 'c': 3, 'd': 2, 'e': 1,
        'f': 4, 'g': 2, 'h': 4, 'i': 1, 'j': 8, 'k':10, 'l': 1, 'm': 2, 'n': 1,
        'o': 1, 'p': 3, 'q': 8, 'r': 1, 'r': 1, 's': 1, 't': 1, 'u': 1, 'v': 4,
        'w':10, 'x':10, 'y':10, 'z': 10})
        print("French letter values used.")
    else:
        shared.setConst(lettersValue={'a': 1, 'b': 3, 'c': 3, 'd': 2, 'e': 1,
        'f': 4, 'g': 2, 'h': 4, 'i': 1, 'j': 8, 'k': 5, 'l': 1, 'm': 3, 'n': 1,
        'o': 1, 'p': 3, 'q':10, 'r': 1, 'r': 1, 's': 1, 't': 1, 'u': 1, 'v': 4,
        'w': 4, 'x': 8, 'y': 4, 'z': 10})
        print("English letter values used.")

    # Get the player words (generate a list of random letters 
    import random
    import string
    random.seed(3141592653)
    words = []
    player_quantity = 4
    words_per_player = 10
    word_letters = (1, 6)
    for pid in range(player_quantity):
        player = []
        for _ in range(words_per_player):
            word = "".join(random.choice(string.ascii_lowercase) for _ in range(random.randint(*word_letters)))
            player.append(word)
        print("Player {pid} played words: {player}".format(**locals()))
        words.append(player)
    
    # Compute the score of every player and display it
    results = list(futures.map(getValue, words))
    for pid, result in enumerate(results):
        print("Player {pid}: {result}".format(**locals()))

Overall example

The examples/fullTree.py example holds a wrap-up of available SCOOP functionnalities. It notably shows that SCOOP is capable of handling twisted and complex hierarchical requirements.

Getting acquainted with the previous examples is fairly enough to use SCOOP, no need to dive into this complicated example.