似乎我使用 coremltool
和经过训练的.caffemodel有一些转换问题 . 我能够训练和测试 caffe
狗模型(120个类别,20k图像),它通过我的测试直接 caffe
分类 . 不幸的是,在转换为 mlmodel
之后,它没有给出我对同一输入的有效预测 .
Traning model
该模型已经使用Caffe,GoogleNet进行了培训,包含120个类别的20k图像集合到lmdb和大约500k迭代中 . 我已经准备好了图像数据库,所有其余的都放了all files together here
Classification with caffe
classification example由 caffe
. 当我试图对受过训练的 caffemodel
运行分类请求时 - 它的效果很好,很有可能(80-99%),正确的结果:
Classification with Apple iOS 11 CoreML
不幸的是,当我试图将这个 DTDogs.caffemodel
& deploy.txt
打包成Apple iOS 11 CoreML
的.mlmodel耗材时,我有不同的预测结果 . 实际上,没有错误加载和使用模型,但我'm unable to get valid classifications, all the predictions are 0-15% confidence and have wrong labels. In order to test it properly I'm使用与 caffe
直接分类完全相同的图像:
我也从这里尝试了我的iOS应用程序the pre-trained and pre-packed models - 它们工作得很好所以它似乎是包装程序的问题 .
What did I miss?
以下是 caffe
的分类示例:没有问题,正确答案( python
):
import numpy as np
import sys
import caffe
import os
import urllib2
import matplotlib.pyplot as plt
%matplotlib inline
test_folder = '/home/<username>/Desktop/CaffeTest/'
test_image_path = "http://cdn.akc.org/content/hero/irish-terrier-Hero.jpg"
# init caffe net
model_def = test_folder + 'deploy.prototxt'
model_weights = test_folder + 'DTDogs.caffemodel'
# caffe.set_mode_gpu()
net = caffe.Net(model_def, model_weights, caffe.TEST)
# prepare transformer
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_transpose('data', (2,0,1))
transformer.set_raw_scale('data', 255)
transformer.set_channel_swap('data', (2,1,0))
net.blobs['data'].reshape(1, 3, 256, 256)
test_image = urllib2.urlopen(test_image_path)
with open(test_folder + 'testImage.jpg','wb') as output:
output.write(test_image.read())
image = caffe.io.load_image(test_folder + 'testImage.jpg')
transformed_image = transformer.preprocess('data', image)
net.blobs['data'].data[...] = transformed_image
# classify
output = net.forward()
output_prob = output['prob'][0]
output_prob_val = output_prob.max() * 100
output_prob_ind = output_prob.argmax()
labels_file = test_folder + 'labels.txt'
labels = np.loadtxt(labels_file, str, delimiter='\t')
plt.imshow(image)
print 'predicted class is:', output_prob_ind
print 'predicted probabily is:', output_prob_val
print 'output label:', labels[output_prob_ind]
以下是使用 coremltools
打包 DTDogs.mlmodel
模型的示例 . 我看到结果 .mlmodel
文件比原始 .caffemodel
小两倍,但它可能是 coremltools
( python
)的某种归档或压缩优化:
import coremltools;
caffe_model = ('DTDogs.caffemodel', 'deploy.prototxt')
labels = 'labels.txt'
coreml_model = coremltools.converters.caffe.convert(caffe_model, class_labels = labels, image_input_names= "data")
coreml_model.short_description = "Dogs Model v1.14"
coreml_model.save('DTDogs.mlmodel')
以下是在应用程序中使用 DTDogs.mlmodel
的示例 . 我正在使用常规图像选择器来选择我用于 .caffe
分类测试( swift
)的相同图像:
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {
picker.dismiss(animated: true)
print("Analyzing Image…")
guard let uiImage = info[UIImagePickerControllerOriginalImage] as? UIImage
else { print("no image from image picker"); return }
guard let ciImage = CIImage(image: uiImage)
else { print("can't create CIImage from UIImage"); return }
imageView.image = uiImage
do {
let model = try VNCoreMLModel(for: DTDogs().model)
let classificationRequest = VNCoreMLRequest(model: model, completionHandler: self.handleClassification)
let orientation = CGImagePropertyOrientation(uiImage.imageOrientation)
let handler = VNImageRequestHandler(ciImage: ciImage, orientation: Int32(orientation.rawValue))
try handler.perform([classificationRequest])
} catch {
print(error)
}
}
1 回答
通常在这些情况下发生的情况是Core ML传入模型的图像格式不正确 .
对于Caffe模型,通常需要在调用
caffe.convert()
时设置is_bgr=True
,并且通常必须传入将从输入图像中减去的RGB平均值,并且可能还需要缩放值 .换句话说,Core ML需要做与
transformer
在Python脚本中所做的相同的事情 .像这样的东西:
我不确定是否需要
image_scale=255.
但是值得一试 . :-)