我试图从gethue / hue获得Hue docker图像,但它似乎忽略了我给他的配置,并且总是在localhost上寻找HDFS而不是我要求他寻找的docker容器 .
这是一些背景:
- 我正在使用以下docker compose来启动HDFS集群:
hdfs-namenode:
image: bde2020/hadoop-namenode:1.1.0-hadoop2.7.1-java8
hostname: namenode
environment:
- CLUSTER_NAME=davidov
ports:
- "8020:8020"
- "50070:50070"
volumes:
- ./data/hdfs/namenode:/hadoop/dfs/name
env_file:
- ./hadoop.env
hdfs-datanode1:
image: bde2020/hadoop-datanode:1.1.0-hadoop2.7.1-java8
depends_on:
- hdfs-namenode
links:
- hdfs-namenode:namenode
volumes:
- ./data/hdfs/datanode1:/hadoop/dfs/data
env_file:
- ./hadoop.env
这将从BigDataEurope启动已正确配置的图像,包括:
- the activation of webhdfs (in /etc/hadoop/hdfs-site.xml):
- dfs.webhdfs.enabled set to true
- the hue proxy user (in /etc/hadoop/core-site.xml):
- hadoop.proxyuser.hue.hosts set to *
- hadoop.proxyuser.hue.groups set to *
我按照他们的instructions启动了hue:
首先,我在docker容器中启动一个bash提示符:
docker run -it -p 8888:8888 gethue/hue:latest bash
然后,我修改desktop / conf / pseudo-distributed.ini以指向正确的hadoop“节点”(在我的例子中是一个地址为172.30.0.2的docker容器:
[hadoop]
# Configuration for HDFS NameNode
# ------------------------------------------------------------------------
[[hdfs_clusters]]
# HA support by using HttpFs
[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://172.30.0.2:8020
# NameNode logical name.
## logical_name=
# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
# Default port is 14000 for HttpFs.
## webhdfs_url=http://172.30.0.2:50070/webhdfs/v1
# Change this if your HDFS cluster is Kerberos-secured
## security_enabled=false
# In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
# have to be verified against certificate authority
## ssl_cert_ca_verify=True
然后我使用以下命令启动hue(仍然在hue容器内):
./build/env/bin/hue runserver_plus 0.0.0.0:8888
然后我将浏览器指向localhost:8888,创建一个新用户(在我的情况下为'hdfs'),然后启动HDFS文件浏览器模块 . 然后,我收到以下错误消息:
Cannot access: /user/hdfs/.
HTTPConnectionPool(host='localhost', port=50070): Max retries exceeded with url: /webhdfs/v1/user/hdfs?op=GETFILESTATUS&user.name=hue&doas=hdfs (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 99] Cannot assign requested address',))
有趣的是 it still tries to connect to localhost (当然无法工作),即使我将其配置文件修改为指向172.30.0.2 .
谷歌搜索问题,我找到了另一个配置文件:desktop / conf.dist / hue.ini . 我尝试修改这个并再次启动hue,但结果相同 .
在我的情况下,有没有人知道如何正确配置hue?
在此先感谢您的帮助 .
问候,
洛朗 .
1 回答
您的一次性
docker run
命令与docker-compose容器不在同一网络上 .你需要这样的东西,将
[projectname]
替换为你开始使用的文件夹docker-compose up
我建议将Docker Compose也用于Hue容器和
desktop/conf/
下INI文件的卷装,你可以简单地指定(因为你把
hostname: namenode
放在了撰写文件中)您还需要取消注释WebHDFS行,以使更改生效
所有INI文件都合并到Hue的
conf
文件夹中