多个GPU keras奇怪的加速

我确实实现了类似的代码，如来自keras的多GPU代码（multiGPU tutorial） . 在具有2个GPU的服务器上运行时，我每个时期有以下培训时间：

• 显示Keras只有一个GPU并设置变量 gpus = 1 （仅使用一个GPU），一个纪元= 32s

• 显示Keras两个GPU， gpus = 1 ，一个纪元= 31秒

• 显示Keras两个GPU， gpus = 2 ，一个纪元= 37秒

输出看起来有点奇怪，而初始化代码似乎每GPU创建多个Tensorflow设备，我不确定这是否是正确的行为 . 但是我看到的其他大多数例子中每个GPU只有一条这样的线 .

第一次测试（显示一个GPU，gpus = 1）：

2017-12-04 14:54:04.071549: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties: 
name: Tesla P100-PCIE-16GB
major: 6 minor: 0 memoryClockRate (GHz) 1.3285
pciBusID 0000:82:00.0
Total memory: 15.93GiB
Free memory: 15.64GiB
2017-12-04 14:54:04.071597: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 
2017-12-04 14:54:04.071605: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y 
2017-12-04 14:54:04.071619: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:82:00.0)
2017-12-04 14:54:21.531654: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:82:00.0)

第二次测试（显示2个GPU，gpus = 1）：

2017-12-04 14:48:24.881733: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 1 with properties: 
...(same as earlier)
2017-12-04 14:48:24.882924: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:82:00.0)
2017-12-04 14:48:24.882931: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla P100-PCIE-16GB, pci bus id: 0000:83:00.0)
2017-12-04 14:48:42.353807: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:82:00.0)
2017-12-04 14:48:42.353851: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla P100-PCIE-16GB, pci bus id: 0000:83:00.0)

奇怪的是例如3（gpus = 2）：

2017-12-04 14:41:35.906828: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 1 with properties:
...(same as earlier)
2017-12-04 14:41:35.907996: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:82:00.0)
2017-12-04 14:41:35.908002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla P100-PCIE-16GB, pci bus id: 0000:83:00.0)
2017-12-04 14:41:52.944335: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:82:00.0)
2017-12-04 14:41:52.944377: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla P100-PCIE-16GB, pci bus id: 0000:83:00.0)
2017-12-04 14:41:53.709812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:82:00.0)
2017-12-04 14:41:53.709838: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla P100-PCIE-16GB, pci bus id: 0000:83:00.0)

代码：

LSTM = keras.layers.CuDNNLSTM
model.add(LSTM(knots, input_shape=(timesteps, X_train.shape[-1]), return_sequences=True))
model.add(LSTM(knots))
model.add(Dense(3, activation='softmax'))

if gpus>=2:
    model_basic = model
    with tf.device("/cpu:0"):
        model = model_basic
    parallel_model = multi_gpu_model(model, gpus=gpus)
    model = parallel_model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])
hist = model.fit(myParameter)

这是典型的行为吗？我的代码有什么问题，每个GPU创建多个设备 . 提前致谢 .