Home | Trees | Indices | Help |
|
---|
|
An alternative implementation of multiprocessing.Pool.map that is faster, more robust and which also works for class methods and non-pickleable functions.
Interface is the same as the map function. The first argument is the name of a function that takes only a single argument. The second argument is an iterable sequence for which every item will become an argument to the function given in the first argument:
from wsatools.Parallel import parallel_map results = parallel_map(self.runOnData, [1,2,3])
Author: R.S. Collins
Organization: WFAU, IfA, University of Edinburgh
Note: Gratuitously taken from this site: http://www.astropython.org/snippet/2010/3/Parallel-map-using-multiprocessing
|
|||
|
|||
|
|||
|
|
|||
__package__ =
|
|
A worker function that maps an input function over a slice of the input iterable. :param f : callable function that accepts argument from iterable :param ii : process ID :param chunk: slice of input iterable :param out_q: thread-safe output queue :param err_q: thread-safe queue to populate on exception :param _lock: thread-safe lock to protect a resource ( useful in extending parallel_map() ) |
A function that executes populated processes and processes the resultant array. Checks error queue for any exceptions. :param procs: list of Process objects :param out_q: thread-safe output queue :param err_q: thread-safe queue to populate on exception :param num : length of resultant array |
A parallelised version of the native Python map function that utilises the Python multiprocessing module to divide and conquer sequence. Example: >>> parallel_map(lambda x: x**2, [1,2,3]) [1, 4, 9] parallel_map does not yet support multiple argument sequences. :param function: callable function that accepts argument from iterable :param sequence: iterable sequence :param numcores: number of cores to use |
Home | Trees | Indices | Help |
|
---|
Generated by Epydoc 3.0.1 on Mon Sep 8 15:46:42 2014 | http://epydoc.sourceforge.net |