首页 文章

使用docker-compose和端口映射进行本地Kafka测试

提问于
浏览
1

我正在尝试使用 docker-compose 获取黑盒服务测试的基础设施并运行 . 我想在Jenkins上为多个服务做到这一点,因此有必要绑定到特定于构建的端口(例如50012而不是9092),因为构建应该能够并行运行 .

问题是,生成消息失败了 .

这是我的 docker-compose.yml

---
version: "3.4"

services:

  zookeeper:
    image: "confluentinc/cp-zookeeper:latest"
    ports:
      - "2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    extra_hosts:
      - "moby:127.0.0.1"

  kafka:
    image: "confluentinc/cp-kafka:latest"
    environment:
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_PROTOCOL_NAME: INSIDE
      KAFKA_ADVERTISED_PROTOCOL_NAME: OUTSIDE
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_LISTENERS: "OUTSIDE://localhost:50012,INSIDE://kafka:9092"
      KAFKA_ADVERTISED_LISTENERS: "OUTSIDE://localhost:50012,INSIDE://kafka:9092"
      KAFKA_BROKER_ID: 1
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
    ports:
      - "50012:9092"
    extra_hosts:
      - "moby:127.0.0.1"
    depends_on:
      - zookeeper

我没有看到任何错误,但这个错误(我认为可以忽略):

ERROR Could not submit metrics to Kafka topic __confluent.support.metrics: Failed to construct kafka producer (io.confluent.support.metrics.BaseMetricsReporter)

当我尝试像这样向Kafka控制台制作人发送消息时

echo 'my-message' | kafka-console-producer.sh --broker-list localhost:50012 --topic test

我收到此异常并且消息未被发送:

[2018-07-25 10:56:31,283] ERROR Error when sending message to topic test with key: null, value: 10 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)

org.apache.kafka.common.errors.TimeoutException:60000毫秒后无法更新元数据 .


更新:感谢Robin的正确答案,我们能够使用以下docker-compose.yml运行它:

---
version: "3.4"

services:

zookeeper:
    image: "confluentinc/cp-zookeeper:latest"
    environment:
    ZOOKEEPER_CLIENT_PORT: 2181
    ZOOKEEPER_TICK_TIME: 2000
    extra_hosts:
      - "moby:127.0.0.1"

kafka:
    image: "confluentinc/cp-kafka:latest"
    environment:
    KAFKA_BROKER_ID: 1
    KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
    KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT"
    KAFKA_INTER_BROKER_LISTENER_NAME: "INSIDE"
    KAFKA_ADVERTISED_LISTENERS: "INSIDE://kafka:29092,OUTSIDE://localhost:50012"
    KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
    ports:
      - "50012:50012"
    extra_hosts:
      - "moby:127.0.0.1"
    depends_on:
      - zookeeper

1 回答

  • 2

    配置问题是

    KAFKA_ADVERTISED_LISTENERS: "OUTSIDE://localhost:50012,INSIDE://kafka:9092"
    […]
        ports:
          - "50012:9092"
    

    KAFKA_ADVERTISED_LISTENERS 就是 - 代理通告的地址 . 所以一方面你把它配置为 localhost:50012 ,但是另一方面你从Docker config 9092 作为外部端口暴露出来 .

    如果你只是使用

    ports:
          - "50012:50012"
    

    那它应该工作正常 .

    查看此docker-compose以获取在外部和内部公开Kafka的工作配置示例 .

相关问题