我搜索了几个类似的主题,涵盖类似的问题 . 例如this,this和this等 . 尽管如此,我仍然没有设法解决我的问题,为什么我现在试图问社区 .
我最终要做的是使用CNN和回归预测三个参数 . 输入是矩阵(现在可以在我用几个步骤预处理后绘制为RGB图像),初始大小为(3724,4073,3) . 由于数据集的大小,我使用以下生成器分批(现在为16)提供CNN:
class My_Generator(Sequence):
""" Generates batches of training data and ground truth. Inputs are the image paths and batch size. """
def __init__(self, image_paths, batch_size, normalise=True):
self.image_paths, self.batch_size = image_paths, batch_size
self.normalise = normalise
def __len__(self):
return int(np.ceil(len(self.image_paths) / float(self.batch_size)))
def __getitem__(self, idx):
batch = self.image_paths[idx * self.batch_size:(idx + 1) * self.batch_size]
matrices, parameters = [], []
for file_path in batch:
mat, param, name = get_Matrix_and_Parameters(file_path)
#Transform the matrix from 2D to 3D as a (mat.shape[0], mat.shape[1]) RBG image. Rescale its values to [0,1]
mat = skimage.transform.resize(mat, (mat.shape[0]//8, mat.shape[1]//8, 3),
mode='constant', preserve_range=self.normalise)
param = MMscale_param(param, name) # Rescale the parameters
matrices.append(mat)
parameters.append(param)
MAT, PAM = np.array(matrices), np.array(parameters)
PAM = np.reshape(PAM, (PAM.shape[0], PAM.shape[1]))
print("Shape Matrices: {0}, Shape Parameters: {1}".format(MAT.shape, PAM.shape))
print("Individual PAM shape: {0}".format(PAM[0,:].shape))
return MAT, PAM
生成器也将矩阵的大小调整了8倍,因为当我没有时,我会遇到内存错误 . 函数MMscale_param只是将参数重新调整为[0,1] .
生成的批次现在具有关于要关注的矩阵的形状(16,465,509,3)和关于参数的(16,3) . 这些现在被纳入以下CNN架构:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_1 (Conv2D) (None, 463, 507, 16) 448
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 231, 253, 16) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 229, 251, 32) 4640
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 114, 125, 32) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 112, 123, 64) 18496
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 56, 61, 64) 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 54, 59, 128) 73856
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 27, 29, 128) 0
_________________________________________________________________
conv2d_5 (Conv2D) (None, 25, 27, 256) 295168
_________________________________________________________________
max_pooling2d_5 (MaxPooling2 (None, 12, 13, 256) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 39936) 0
_________________________________________________________________
dense_1 (Dense) (None, 1000) 39937000
_________________________________________________________________
dense_2 (Dense) (None, 100) 100100
_________________________________________________________________
dense_3 (Dense) (None, 20) 2020
_________________________________________________________________
dense_4 (Dense) (None, 3) 63
=================================================================
Total params: 40,431,791
Trainable params: 40,431,791
Non-trainable params: 0
_________________________________________________________________
如上所示,模型中的最后一层要求输入为(None,3) . 如果我理解这是正确的,那么“任何”批量大小值可以用“无”替换,所以我的输入(16,3)或(batch_size,number_of_parameters_to_predict)应该是有效的 . 但是,我仍然收到以下错误消息:
ValueError: Error when checking target: expected dense_4 to have shape (1,) but got array with shape (3,)
我发现非常奇怪的是密集层dense_4具有形状(1,)的说法 . 但不是_314869_一个(3,)形状?这应该适合输入数组的形状(3,) .
我试图以几种方式重塑和/或转置阵列,但没有成功 . 我甚至卸载并重新安装了TensorFlow和Keras,因为他们认为那里出了问题,但仍然没有 .
然而,有效的是尝试仅预测三个参数中的一个,给出输入形状(1,0) . (稍后会产生其他与记忆相关的错误 . )这实际上与我如何塑造dense_4层无关,这意味着(None,1)和(None,3)都有效,根据我的有限知识,它不会有道理 .
添加编译;
batch_size = 16
my_training_batch_generator_NIR = My_Generator(training_paths_NIR, batch_size)
my_validation_batch_generator_NIR = My_Generator(validation_paths_NIR, batch_size)
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')
和培训代码:
model_path = "/Models/weights.best.hdf5"
num_epochs = 10
checkpointer = ModelCheckpoint(filepath=model_path,
verbose=1,
save_best_only=True)
model.fit_generator(generator=my_training_batch_generator_NIR,
steps_per_epoch=(len(validation_paths_NIR) // batch_size),
epochs=num_epochs,
verbose=1,
callbacks=[checkpointer],
validation_data=my_validation_batch_generator_NIR,
validation_steps=(len(validation_paths_NIR) // batch_size),
use_multiprocessing=True,
max_queue_size=1,
workers=1)
总而言之:我遇到了将(3,)数组拟合到我认为是(3,)层的问题 . 然而,据称后者具有形状(1,) . 我必须在这里遗漏一些东西,因为它不是一个错误,可以吗?
任何帮助将不胜感激 .
我在Ubuntu上使用带有TensorFlow 1.9.0后端的Keras 2.2.2版 .
1 回答
这是因为您正在使用的丢失功能 . 替换为
代码应该工作 .