import h5py, os
import caffe
import numpy as np
SIZE = 227 # fixed size to all images
with open( 'train.txt', 'r' ) as T :
lines = T.readlines()
# If you do not have enough memory split data into
# multiple batches and generate multiple separate h5 files
X = np.zeros( (len(lines), 3, SIZE, SIZE), dtype='f4' )
y = np.zeros( (len(lines),1), dtype='f4' )
for i,l in enumerate(lines):
sp = l.split(' ')
img = caffe.io.load_image( sp[0] )
img = caffe.io.resize( img, (SIZE, SIZE, 3) ) # resize to fixed size
# you may apply other input transformations here...
# Note that the transformation should take img from size-by-size-by-3 and transpose it to 3-by-size-by-size
# for example
transposed_img = img.transpose((2,0,1))[::-1,:,:] # RGB->BGR
X[i] = transposed_img
y[i] = float(sp[1])
with h5py.File('train.h5','w') as H:
H.create_dataset( 'X', data=X ) # note the name X given to the dataset!
H.create_dataset( 'y', data=y ) # note the name y given to the dataset!
with open('train_h5_list.txt','w') as L:
L.write( 'train.h5' ) # list all h5 files you are going to use
layer {
type: "HDF5Data"
top: "X" # same name as given in create_dataset!
top: "y"
hdf5_data_param {
source: "train_h5_list.txt" # do not give the h5 files directly, but the list.
batch_size: 32
}
include { phase:TRAIN }
}
layer {
type: "HDF5Data"
top: "X" # same name as given in create_dataset!
top: "y"
hdf5_data_param {
source: "val_h5_list.txt" # do not give the h5 files directly, but the list.
batch_size: 32
}
include { phase:TEST }
}
2 回答
Caffe的LMDB接口的问题在于它只允许single int label per image .
如果您想为每个图像添加多个标签,则必须使用不同的输入图层 .
我建议使用"HDF5Data"图层:
这样可以更灵活地设置输入数据,您可以拥有此层所需的
"top"
. 每个输入图像可能有多个标签,并且您的网络需要多次损失才能进行训练 .有关如何为caffe创建hdf5数据,请参阅this post .
谢谢Shai,
只是试图理解实用的方法..在创建包含图像的所有标签的2个.text文件(一个用于训练,一个用于验证)之后,例如:
运行py脚本:
并创建train.h5和val.h5(是包含图像的X数据集,Y是否包含标签?) .
替换我的网络输入图层:
至
我猜HDF5不需要mean.binaryproto?
接下来,输出层应如何更改以输出多个标签概率?我想我需要交叉熵层而不是softmax?这是当前的输出图层: