我有3D数据（x，y，z）的随机样本，其中x和y是空间维度，z是它在平面上的位置的函数：z = f（x，y） . 我想使用样本中的插值来在常规网格上评估此函数 .

每组样本都在不同的文件中，需要单独插入（每个文件是不同的时间点） . 为了加快这些文件的处理速度，我想使用多处理（mp）模块使用mp池并行执行它们 .

但是，当我尝试在mp池调用的函数内使用mp.griddata时，进程在进入griddata函数时会挂起 . 串行执行相同功能时没有问题 . 这也只发生在某些numpy / scipy构建中 . 当我的numpy.show_config（）如下：

blas_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
lapack_info:
    libraries = ['lapack']
    library_dirs = ['/usr/lib']
    language = f77
atlas_threads_info:
  NOT AVAILABLE
blas_opt_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_blas_threads_info:
  NOT AVAILABLE
lapack_opt_info:
    libraries = ['lapack', 'blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE

一切正常 .

但是当配置是：

lapack_info:
    libraries = ['openblas']
    library_dirs = ['/u/ccor/local/lib']
    language = f77
atlas_threads_info:
    libraries = ['openblas']
    library_dirs = ['/u/ccor/local/lib']
    define_macros = [('ATLAS_WITHOUT_LAPACK', None)]
    language = c
    include_dirs = ['/usr/include']
blas_opt_info:
    libraries = ['openblas']
    library_dirs = ['/u/ccor/local/lib']
    define_macros = [('ATLAS_INFO', '"\\"None\\""')]
    language = c
    include_dirs = ['/usr/include']
atlas_blas_threads_info:
    libraries = ['openblas']
    library_dirs = ['/u/ccor/local/lib']
    define_macros = [('ATLAS_INFO', '"\\"None\\""')]
    language = c
    include_dirs = ['/usr/include']
lapack_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/u/ccor/local/lib']
    define_macros = [('ATLAS_WITHOUT_LAPACK', None)]
    language = f77
    include_dirs = ['/usr/include']
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE

子进程挂起 .

第二个配置是通过手动编译LibOpenBLAS和numpy来尝试优化我的BLAS库以获得具有多个核心的机器 . 在这两种情况下，我都在使用python 2.7.3的virtualenv .

此外，悬挂只发生在griddata与'linear'或'cubic'插值一起使用时; '最近'工作得很好 .

以下是一些试图说明和隔离问题的代码 . 数据是随机生成的，而不是从文件中读取，结果只是作为一个例子随意累积 .

import multiprocessing as mp
import numpy
from scipy import interpolate

def interpolate_grid(it, n_points, grid_dim):

    print 'interpolating: ', it

    numpy.random.seed(it)

    points = 2*numpy.random.random((n_points,2))-0.5
    values = numpy.random.random(n_points)
    x,y = numpy.mgrid[0:1:grid_dim*1j,0:1:grid_dim*1j]
    grid_pnts = numpy.vstack([x.flatten(), y.flatten()]).T
    interp_vals = interpolate.griddata(points, values, grid_pnts, 'cubic')
    print 'done interpolating: ', it
    return interp_vals

def interpolation_test(n_runs = 10, n_points = 1000, grid_dim = 10, parallel = True):

    class Sum(object):
        def __init__(self, grid_dim):
            self.acc = numpy.zeros(grid_dim*grid_dim)

        def accumulate(self, val):
            self.acc += val

    acc = Sum(grid_dim)

    if parallel:
        pool = mp.Pool(mp.cpu_count())
        for i in xrange(n_runs):
            pool.apply_async(interpolate_grid, 
                            args = [i, n_points, grid_dim], 
                            callback = acc.accumulate)
        pool.close()
        pool.join()

    else:
        for i in xrange(n_runs):
            acc.acc += interpolate_grid(i, n_points, grid_dim)

    return acc.acc

def main():

    ser_result = interpolation_test(parallel = False)
    par_result = interpolation_test(parallel = True)

    assert (numpy.abs(par_result-ser_result) < 1e-14).all()
    print 'serial and parallel results the same'

if __name__ == '__main__':
    main()

编辑：我在scikit-learn installation page上发现了这个："Using OpenBLAS can give speedups in some scikit-learn modules, but it doesn’t play nicely with joblib/multiprocessing, so using it is not recommended unless you know what you’re doing."

我通过使用ATLAS而不是OpenBLAS找到了一种解决方法：

aptitude install libatlas-base-dev
ATLAS=/usr/lib
pip install numpy

它不如OpenBLAS快，但适用于多处理，这对我的应用程序更重要，并且比基本的numpy安装快得多 . 有谁“知道他们在做什么”关心为什么OpenBLAS和多处理“不能很好地结合在一起”？或者如何让他们玩得更好？ scikit-learn网站上的警告似乎暗示它是可能的 .

python numpy scipy multiprocessing interpolation