首页 文章

Kafka Zookeeper连接问题

提问于
浏览
5

我正在使用Kafka 0.8.2-beta并拥有2个Ubuntu 14虚拟机:

  • 172.30.141.127正在运行Zookeeper

  • 172.30.141.184正在运行Kafka经纪人

我正在启动Zookeeper实例,一切都很好 . 然后我尝试启动代理并将其连接到172.30.141.127:2181 . 它似乎能够在特定端口上连接并 Build 会话,但由于某些异常似乎没有记录,它会丢失连接 .

经纪人输出:

[2015-01-19 11:03:55,029] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,030] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,031] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,031] INFO Client environment:os.arch=i386 (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,032] INFO Client environment:os.version=3.16.0-23-generic (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,033] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,033] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,037] INFO Client environment:user.dir=/home/osboxes/Desktop/kafka_2.11-0.8.2-beta (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,039] INFO Initiating client connection, connectString=172.30.141.127:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@1ecf473 (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,129] INFO Opening socket connection to server 172.30.141.127/172.30.141.127:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:55,186] INFO Socket connection established to 172.30.141.127/172.30.141.127:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:55,203] WARN Session 0x0 for server 172.30.141.127/172.30.141.127:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.io.IOException: Connection reset by peer
    at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
    at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
    at sun.nio.ch.IOUtil.read(IOUtil.java:192)
    at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2015-01-19 11:03:56,552] INFO Opening socket connection to server 172.30.141.127/172.30.141.127:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:56,555] INFO Socket connection established to 172.30.141.127/172.30.141.127:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:56,567] WARN Session 0x0 for server 172.30.141.127/172.30.141.127:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.io.IOException: Connection reset by peer
    at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
    at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
    at sun.nio.ch.IOUtil.read(IOUtil.java:192)
    at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2015-01-19 11:03:57,131] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-01-19 11:03:58,075] INFO Opening socket connection to server 172.30.141.127/172.30.141.127:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:58,077] INFO Socket connection established to 172.30.141.127/172.30.141.127:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:58,195] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:58,196] INFO EventThread shut down (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:58,251] FATAL [Kafka Server 1], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 2000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
    at kafka.server.KafkaServer.initZk(KafkaServer.scala:157)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:83)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:28)
    at kafka.Kafka$.main(Kafka.scala:46)
    at kafka.Kafka.main(Kafka.scala)
[2015-01-19 11:03:58,279] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)
[2015-01-19 11:03:58,295] INFO [Kafka Server 1], shut down completed (kafka.server.KafkaServer)
[2015-01-19 11:03:58,308] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 2000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
    at kafka.server.KafkaServer.initZk(KafkaServer.scala:157)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:83)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:28)
    at kafka.Kafka$.main(Kafka.scala:46)
    at kafka.Kafka.main(Kafka.scala)
[2015-01-19 11:03:58,335] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)

Zookeper输出:

[2015-01-19 11:03:55,245] INFO Accepted socket connection from /172.30.141.184:54089 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-01-19 11:03:55,315] WARN Exception causing close of session 0x0 due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:55,329] INFO Closed socket connection for client /172.30.141.184:54089 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:56,613] INFO Accepted socket connection from /172.30.141.184:54090 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-01-19 11:03:56,615] WARN Exception causing close of session 0x0 due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:56,617] INFO Closed socket connection for client /172.30.141.184:54090 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:58,133] INFO Accepted socket connection from /172.30.141.184:54091 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-01-19 11:03:58,134] WARN Exception causing close of session 0x0 due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:58,135] INFO Closed socket connection for client /172.30.141.184:54091 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)

Ping在两台机器之间工作 . Telnet到2181种工作,因为它连接,但不时断开连接 . 这让我认为问题出在Zookeeper实例上 . 这两个进程都以root身份启动 .

任何想法为什么会这样?谢谢

2 回答

  • 0

    您可能正在达到每个主机的最大连接数 .

    如果您的zookeeper日志中有 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /127.0.0.1 - max is 10 ,则会发生这种情况 .

    通过在 conf/zoo.cfg 中设置 maxClientCnxns=[something more than 10; 0 for unlimited] 来修复它 .

    Docs(搜索maxClientCnxns)

  • 3

    不知道警告,但我有同样的错误问题

    org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 2000
        at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
        ....
    

    在我设置 生产环境 者和消费者的情况下,我提供了错误的IP /端口 . 当我改变它以纠正一个:

    bin/kafka-console-producer.sh --broker-list kafka_ip:kafka_port --topic test
    bin/kafka-console-consumer.sh --zookeeper zookeeper_id:zookeper_port --topic test --from-beginning
    

    我的问题解决了 .

相关问题