Docker - Kafka 설치 (UI 관리자 페이지)

✔준비물 : Linux, Docker, Kafka

Kafka 설치및 설정

docker-compose.yml 작성

version: '2' # docker-compose version
services:
  zookeeper:
    container_name: local-zookeeper
    image: wurstmeister/zookeeper:3.4.6
    ports:
      - "2181:2181"
  kafka:
    container_name: local-kafka
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://외부망IP:9092 # 외부에서 접속하기 위한 리스너
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT # 보안을 위한 프로토콜 매핑    
      KAFKA_CREATE_TOPICS: "stopic:1:1" # Topic명:Partition개수:Replica개수

docker 실행

$sudo docker-compose -f docker-compose.yml up -d
[+] Running 2/2
 ⠿ Container local-zookeeper  Started                                                                                          1.1s
 ⠿ Container local-kafka      Started

컨테이너 ID 확인

$sudo docker ps -a
CONTAINER ID   IMAGE                          COMMAND                   CREATED              STATUS              PORTS                                                                   NAMES
fe0699beebbb   confluentinc/cp-kafka:latest   "/etc/confluent/dock…"   About a minute ago   Up About a minute   0.0.0.0:9092->9092/tcp, :::9092->9092/tcp                               local-kafka
3f490bba9a43   wurstmeister/zookeeper:3.4.6   "/bin/sh -c '/usr/sb…"   About a minute ago   Up About a minute   22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, :::2181->2181/tcp   local-zookeeper

 로그 확인

$sudo docker logs -f 컨테이너ID

Kafka 확인

# 컨테이너 접속
$ sudo docker exec -i -t local-kafka bash

# Proucer Hello 작성
$ kafka-console-producer --bootstrap-server kafkaIP:9092 --topic "stopic"
>Hello

# Consumer Hello 출력 확인
$ kafka-console-consumer --bootstrap-server kafkaIP:9092 --topic "stopic"
Hello

정상 확인 완료

Kafka UI 관리자 페이지

1. docker-compose-kafka-ui.yml 작성

version: '2'
services:
  kafka-ui:
    image: provectuslabs/kafka-ui
    container_name: kafka-ui
    ports:
      - "8989:8080"
    restart: always
    environment:
      - KAFKA_CLUSTERS_0_NAME=local
      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafkaIP:9092 # kafkaIP 수정 필요
      - KAFKA_CLUSTERS_0_ZOOKEEPER=zookeeper-1:2181​

2. 실행

$sudo docker-compose -f docker-compose-kafka-ui.yml up -d​

3. 확인
kafkaIP:8989 접속

Broker, Topics, Consumers를 UI로 확인하면서 관리를 편하게 할 수 있음