[docker-compose] zookeeper/kafka 클러스터 구성

docker-compose.yml

version: "3.0"
services:
  zk1:
    image: zookeeper
    restart: always
    hostname: zk1
    ports:
      - 2181:2181
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zk1:2888:3888;2181 server.2=zk2:2888:3888;2181 server.3=zk3:2888:3888;2181
    volumes:
      - "~/zk-cluster/zk1/data:/data"

  zk2:
    image: zookeeper
    restart: always
    hostname: zk2
    ports:
      - "2182:2181"
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zk1:2888:3888;2181 server.2=zk2:2888:3888;2181 server.3=zk3:2888:3888;2181
    volumes:
      - "~/zk-cluster/zk2/data:/data"

  zk3:
    image: zookeeper
    restart: always
    hostname: zk3
    ports:
      - "2183:2181"
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zk1:2888:3888;2181 server.2=zk2:2888:3888;2181 server.3=zk3:2888:3888;2181
    volumes:
      - "~/zk-cluster/zk3/data:/data"

  zoo-navi:
    image: elkozmon/zoonavigator
    ports:
      - 9000:9000
    environment:
      HTTP_PORT: 9000

  kafka1:
    image: confluentinc/cp-kafka
    depends_on:
    - zk1
    - zk2
    - zk3
    restart: on-failure
    ports:
    - 9091:9091
    environment:
    - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://호스트IP:9091
    - KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9091
    - KAFKA_ZOOKEEPER_CONNECT=zk1:2181,zk2:2181,zk3:2181
    - KAFKA_BROKER_ID=1
    - BOOTSTRAP_SERVERS=kafka1:9091,kafka2:9092,kafka3:9093

  kafka2:
    image: confluentinc/cp-kafka
    depends_on:
    - zk1
    - zk2
    - zk3
    restart: on-failure
    ports:
    - 9092:9092
    environment:
    - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://호스트IP:9092
    - KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092
    - KAFKA_ZOOKEEPER_CONNECT=zk1:2181,zk2:2181,zk3:2181
    - KAFKA_BROKER_ID=2
    - BOOTSTRAP_SERVERS=kafka1:9091,kafka2:9092,kafka3:9093

  kafka3:
    image: confluentinc/cp-kafka
    depends_on:
    - zk1
    - zk2
    - zk3
    restart: on-failure
    ports:
    - 9093:9093
    environment:
    - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://호스트IP:9093
    - KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9093
    - KAFKA_ZOOKEEPER_CONNECT=zk1:2181,zk2:2181,zk3:2181
    - KAFKA_BROKER_ID=3
    - BOOTSTRAP_SERVERS=kafka1:9091,kafka2:9092,kafka3:9093

주의할 점은 ADVERTISED_LISTENERS 프로퍼티에 로컬 호스트가 아닌 호스트 IP를 적어야 한다느 점이다.

그 이유는 아래 첫번째 링크 글을 참고하면 된다.

 

 

 

 

ADVERTISED_LISTENERS vs LISTENERS 에 관해 참고할만한 글

 

wurstmeister/kafka-docker

위 글을 읽어보길 강력 추천. 길지 않고 사용된 이미지도 좋아서 이해하기 편하다.

도커의 브릿지 네트워크에서 ADVERTISED_LISTENERS 가 왜 필요한지 쉽게 설명해준다.

---

Kafka Listeners - Explained