in' d Z ddlZddlmZ ddlmZ G d d Z G d de Z G d d
e Z G d d e Z dS )
a# Analysis backends --- :mod:`MDAnalysis.analysis.backends`
============================================================
.. versionadded:: 2.8.0
The :mod:`backends` module provides :class:`BackendBase` base class to
implement custom execution backends for
:meth:`MDAnalysis.analysis.base.AnalysisBase.run` and its
subclasses.
.. SeeAlso:: :ref:`parallel-analysis`
.. _backends:
Backends
--------
Three built-in backend classes are provided:
* *serial*: :class:`BackendSerial`, that is equivalent to using no
parallelization and is the default
* *multiprocessing*: :class:`BackendMultiprocessing` that supports
parallelization via standard Python :mod:`multiprocessing` module
and uses default :mod:`pickle` serialization
* *dask*: :class:`BackendDask`, that uses the same process-based
parallelization as :class:`BackendMultiprocessing`, but different
serialization algorithm via `dask `_ (see `dask
serialization algorithms
`_ for details)
Classes
-------
N)Callable) is_installedc D e Zd ZdZdefdZd Zd Zd Zde de
d e
fd
Z d S ) BackendBasea Base class for backend implementation.
Initializes an instance and performs checks for its validity, such as
``n_workers`` and possibly other ones.
Parameters
----------
n_workers : int
number of workers (usually, processes) over which the work is split
Examples
--------
.. code-block:: python
from MDAnalysis.analysis.backends import BackendBase
class ThreadsBackend(BackendBase):
def apply(self, func, computations):
from multiprocessing.dummy import Pool
with Pool(processes=self.n_workers) as pool:
results = pool.map(func, computations)
return results
import MDAnalysis as mda
from MDAnalysis.tests.datafiles import PSF, DCD
from MDAnalysis.analysis.rms import RMSD
u = mda.Universe(PSF, DCD)
ref = mda.Universe(PSF, DCD)
R = RMSD(u, ref)
n_workers = 2
backend = ThreadsBackend(n_workers=n_workers)
R.run(backend=backend, unsupported_backend=True)
.. warning::
Using `ThreadsBackend` above will lead to erroneous results, since it
is an educational example. Do not use it for real analysis.
.. versionadded:: 2.8.0
n_workersc < || _ | d S )N)r _validate)selfr s f/srv/www/vhosts/g4struct/public_html/venv/lib/python3.11/site-packages/MDAnalysis/analysis/backends.py__init__zBackendBase.__init__[ s " c ` t | j t o
| j dk d| j iS )a" Get dictionary with ``condition: error_message`` pairs that ensure the
validity of the backend instance
Returns
-------
dict
dictionary with ``condition: error_message`` pairs that will get
checked during ``_validate()`` run
r z9n_workers should be positive integer, got self.n_workers=)
isinstancer intr
s r _get_checkszBackendBase._get_checks_ s<
t~s + +
OdnOO
r
c t S )a( Get dictionary with ``condition: warning_message`` pairs that ensure
the good usage of the backend instance
Returns
-------
dict
dictionary with ``condition: warning_message`` pairs that will get
checked during ``_validate()`` run
)dictr s r
_get_warningszBackendBase._get_warningso s
vv
r
c | D ]\ }}|st | | D ]\ }}|st j | dS )a Check correctness (e.g. ``dask`` is installed if using ``backend='dask'``)
and good usage (e.g. ``n_workers=1`` if backend is serial) of the backend
Raises
------
ValueError
if one of the conditions in :meth:`_get_checks` is ``True``
N)r items
ValueErrorr warningswarn)r
checkmsgs r r zBackendBase._validate{ s **,,2244 & &JE3
& oo%
& ,,..4466 # #JE3
#
c""" # #r
func computationsreturnc t )aT map function `func` to all tasks in the `computations` list
Main method that will get called when using an instance of
``BackendBase``. It is equivalent to running ``[func(item) for item in
computations]`` while using the parallel backend capabilities.
Parameters
----------
func : Callable
function to be called on each of the tasks in computations list
computations : list
computation tasks to apply function to
Returns
-------
list
list of results of the function
)NotImplementedErrorr
r r s r applyzBackendBase.apply s
( "!r
N) __name__
__module__ __qualname____doc__r r r r r r listr# r
r r r , s , ,\#
# # # "( "$ "4 " " " " " "r
r c , e Zd ZdZd ZdededefdZdS )
BackendSeriala A built-in backend that does serial execution of the function, without any
parallelization.
Parameters
----------
n_workers : int
Is ignored in this class, and if ``n_workers`` > 1, a warning will be
given.
.. versionadded:: 2.8.0
c | j dk diS )a} Get dictionary with ``condition: warning_message`` pairs that ensure
the good usage of the backend instance. Here, it checks if the number
of workers is not 1, otherwise gives warning.
Returns
-------
dict
dictionary with ``condition: warning_message`` pairs that will get
checked during ``_validate()`` run
z9n_workers is ignored when executing with backend='serial')r r s r r zBackendSerial._get_warnings s
N
M
r
r r r c fd|D S )a
Serially applies `func` to each task object in ``computations``.
Parameters
----------
func : Callable
function to be called on each of the tasks in computations list
computations : list
computation tasks to apply function to
Returns
-------
list
list of results of the function
c & g | ]
} | S r) r) ).0taskr s r
z'BackendSerial.apply.. s! 444tT
444r
r) r" s ` r r# zBackendSerial.apply s 5444|4444r
N)r$ r% r& r' r r r( r# r) r
r r+ r+ sV
5( 5$ 54 5 5 5 5 5 5r
r+ c & e Zd ZdZdededefdZdS )BackendMultiprocessingau A built-in backend that executes a given function using the
:meth:`multiprocessing.Pool.map ` method.
Parameters
----------
n_workers : int
number of processes in :class:`multiprocessing.Pool
` to distribute the workload
between. Must be a positive integer.
Examples
--------
.. code-block:: python
from MDAnalysis.analysis.backends import BackendMultiprocessing
import multiprocessing as mp
backend_obj = BackendMultiprocessing(n_workers=mp.cpu_count())
.. versionadded:: 2.8.0
r r r c ddl m} || j 5 }| || }ddd n # 1 swxY w Y |S )a Applies `func` to each object in ``computations`` using `multiprocessing`'s `Pool.map`.
Parameters
----------
func : Callable
function to be called on each of the tasks in computations list
computations : list
computation tasks to apply function to
Returns
-------
list
list of results of the function
r )Pool) processesN)multiprocessingr6 r map)r
r r r6 poolresultss r r# z BackendMultiprocessing.apply s )(((((
TDN
+
+
+ 3thht\22G 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3s
;??N)r$ r% r& r' r r( r# r) r
r r4 r4 sG 2( $ 4 r
r4 c 6 e Zd ZdZdededefdZ fdZ xZS ) BackendDaskad A built-in backend that executes a given function with *dask*.
Execution is performed with the :func:`dask.compute` function of
:class:`dask.delayed.Delayed` object (created with
:func:`dask.delayed.delayed`) with ``scheduler='processes'`` and
``chunksize=1`` (this ensures uniform distribution of tasks among
processes). Requires the `dask package `_
to be `installed `_.
Parameters
----------
n_workers : int
number of processes in to distribute the workload
between. Must be a positive integer. Workers are actually
:class:`multiprocessing.pool.Pool` processes, but they use a different and
more flexible `serialization protocol
`_.
Examples
--------
.. code-block:: python
from MDAnalysis.analysis.backends import BackendDask
import multiprocessing as mp
backend_obj = BackendDask(n_workers=mp.cpu_count())
.. versionadded:: 2.8.0
r r r c ddl m ddl}fd|D }| |dd| j d }|S )ao Applies `func` to each object in ``computations``.
Parameters
----------
func : Callable
function to be called on each of the tasks in computations list
computations : list
computation tasks to apply function to
Returns
-------
list
list of results of the function
r )delayedNc 8 g | ]} | S r) r) )r0 r1 r? r s r r2 z%BackendDask.apply..7 s+ EEE
d++EEEr
r7 r- ) scheduler chunksize num_workers) dask.delayedr? daskcomputer )r
r r rE r; r? s ` @r r# zBackendDask.apply% st )((((( EEEEE EEE ,, !
r
c p t }t d di}||z S )ak Get dictionary with ``condition: error_message`` pairs that ensure the
validity of the backend instance. Here checks if ``dask`` module is
installed in the environment.
Returns
-------
dict
dictionary with ``condition: error_message`` pairs that will get
checked during ``_validate()`` run
rE z]module 'dask' is missing. Please install 'dask': https://docs.dask.org/en/stable/install.html)superr r )r
base_checkschecks __class__s r r zBackendDask._get_checks@ s> gg))++ ?
V##r
) r$ r% r& r' r r( r# r
__classcell__)rK s @r r= r= sl B( $ 4 6$ $ $ $ $ $ $ $ $r
r= )
r' r typingr MDAnalysis.lib.utilr r r+ r4 r= r) r
r rO s $ $L , , , , , ,s" s" s" s" s" s" s" s"l.5 .5 .5 .5 .5K .5 .5 .5b- - - - -[ - - -`O$ O$ O$ O$ O$+ O$ O$ O$ O$ O$r