首页 文章

如何配置Tensorflow服务以从HDFS服务模型？

提问于
浏览 次
11

我正在尝试使用Tensorflow服务项目从HDFS中提供Tensorflow模型 .

我正在运行tensorflow服务docker容器标签1.10.1 https://hub.docker.com/r/tensorflow/serving

我可以在https://github.com/tensorflow/serving/blob/628702e1de1fa3d679369e9546e7d74fa91154d3/tensorflow_serving/model_servers/BUILD#L341看到tensorflow / serve repo引用Hadoop

"@org_tensorflow//tensorflow/core/platform/hadoop:hadoop_file_system"

这是一个参考

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/hadoop/hadoop_file_system.cc

我设置了以下环境变量：

  • HADOOP_HDFS_HOME指向我的HDFS主页（在我的情况下为/ etc / hadoop） .

  • MODEL_BASE_PATH设为"hdfs://tensorflow/models"

  • MODEL_NAME设置为我要加载的模型的名称

我将Hadoop挂载到docker容器中，可以使用docker exec验证它 .

当我运行docker容器时，我在日志中得到以下内容：

tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:369] FileSystemStoragePathSource encountered a file-system access error: Could not find base path hdfs://tensorflow/models/my_model for servable my_model

我找到了Tensorflow使用HDFS进行培训的示例，但没有使用Tensorflow服务从HDFS服务模型 .

Tensorflow服务能否为HDFS服务？如果是这样，你怎么做？

hadoop tensorflow hdfs tensorflow-serving

1 回答

  • 1

    在 BUILD 的 BUILD 中，在 cc_test 下为 get_model_status_impl_test ，添加此行 @org_tensorflow//tensorflow/core/platform/hadoop:hadoop_file_system ，如下所示：

    cc_test(
    name = "get_model_status_impl_test",
    size = "medium",
    srcs = ["get_model_status_impl_test.cc"],
    data = [
        "//tensorflow_serving/servables/tensorflow/testdata:saved_model_half_plus_two_2_versions",
    ],
    deps = [
        ":get_model_status_impl",
        ":model_platform_types",
        ":platform_config_util",
        ":server_core",
        "//tensorflow_serving/apis:model_proto",
        "//tensorflow_serving/core:availability_preserving_policy",
        "//tensorflow_serving/core/test_util:test_main",
        "//tensorflow_serving/servables/tensorflow:saved_model_bundle_source_adapter_proto",
        "//tensorflow_serving/servables/tensorflow:session_bundle_config_proto",
        "//tensorflow_serving/servables/tensorflow:session_bundle_source_adapter_proto",
        "//tensorflow_serving/test_util",
        "@org_tensorflow//tensorflow/cc/saved_model:loader",
        "@org_tensorflow//tensorflow/cc/saved_model:signature_constants",
        "@org_tensorflow//tensorflow/contrib/session_bundle",
        "@org_tensorflow//tensorflow/core:test",
        "@org_tensorflow//tensorflow/core/platform/hadoop:hadoop_file_system",
    ],
)

    我认为这可以解决你的问题 .

    参考：Fail to load the models from HDFS

    回复于

相关问题