zookeeper、kafka搭建部署
4 min read

zookeeper、kafka搭建部署

kafka搭建

单机部署参考文档: https://www.cnblogs.com/wonglu/p/8687488.html

群集部署参考文档:https://www.cnblogs.com/luotianshuai/p/5206662.html

第一步:安装Java环境

①.下载地址:https://download.oracle.com/otn/java/jdk/8u211-b12/478a62b7d4e34b78b671c754eaaf38ab/jdk-8u211-linux-i586.tar.gz

下面安装所使用的安装包为自行下载的离线包。

# tar xf jdk-8u211-linux-x64.tar.gz 
# mv jdk1.8.0_211 /usr/local/jdk 
# chown -R root:root /usr/local/jdk 
# echo "export JAVA_HOME=/usr/local/jdk" >>/etc/profile 
# echo 'export PATH=$JAVA_HOME/bin:$PATH' >>/etc/profile

②.验证Java环境

# java -version 
java version "1.8.0_211" Java(TM) SE Runtime Environment (build 1.8.0_211-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)

第二步:安装zookeeper

①.创建目录

# useradd zookeeper 
# echo zookeeper |passwd --stdin zookeeper
# mkdir -p /data/zookeeper/{zkdata,zkdatalog}
# wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.5.5/apache-zookeeper-3.5.5-bin.tar.gz 
# tar xf apache-zookeeper-3.5.5.tar.gz 
# mv apache-zookeeper-3.5.5 /usr/local/zookeeper 
# chown -R zookeeper:zookeeper /usr/local/zookeeper 
# chown -R zookeeper:zookeeper /data/zookeeper
# cd /usr/local/zookeeper 
# cp zoo_sample.cfg zoo.cfg

②.配置

# cat zoo.cfg 
# The number of milliseconds of each tick 
tickTime=2000 
# The number of ticks that the initial 
# synchronization phase can take 
initLimit=10 
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement 
syncLimit=5 
# the directory where the snapshot is stored. 
# do not use /tmp for storage, /tmp here is just 
# example sakes. 
dataDir=/data/zookeeper/zkdata 
dataLogDir=/data/zookeeper/zkdatalog
# the port at which the clients will connect 
clientPort=12181 
# the maximum number of client connections. 
# increase this if you need to handle more clients 
# maxClientCnxns=60 
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge. 
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html
# sc_maintenance # # The number of snapshots to retain in dataDir 
# autopurge.snapRetainCount=3 
# Purge task interval in hours 
# Set to "0" to disable auto purge feature 
# autopurge.purgeInterval=1 
server.1=192.168.8.40:12888:13888 
server.2=192.168.8.106:12888:13888 
server.3=192.168.8.177:12888:13888

将这个配置文件scp到集群的其他机器上。

③.创建myid文件

在server1机器上

# echo "1" > /data/zookeeper/zkdata/myid

在server2机器上

# echo "2" > /data/zookeeper/zkdata/myid

在server3机器上

# echo "3" > /data/zookeeper/zkdata/myid

④.启动zookeeper

进入到Zookeeper的bin目录下

# cd /usr/local/zookeeper/bin

启动服务(3台都需要操作)

# ./zkServer.sh start

检查服务器状态

# ./zkServer.sh status ./zkServer.sh status JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg

配置文件 Mode: follower #他是否为领导

第三步:安装kafka

①.创建目录

# mkdir -p /data/kafka/kafkalogs

②.下载安装包

# wget https://www.apache.org/dyn/closer.cgi?path=/kafka/2.3.0/kafka_2.11-2.3.0.tgz
# tar xf kafka_2.11-2.3.0.tgz # mv kafka_2.11-2.3.0 /usr/local/kafka

③.配置

# grep -Ev "^#|^$" server.properties 
broker.id=0 
listeners=PLAINTEXT://192.168.8.177:9092 
num.network.threads=3 
num.io.threads=8 
socket.send.buffer.bytes=102400 
socket.receive.buffer.bytes=102400 
socket.request.max.bytes=104857600 
log.dirs=/data/kafka/kafkalogs 
num.partitions=3 
num.recovery.threads.per.data.dir=1 
offsets.topic.replication.factor=1 
transaction.state.log.replication.factor=1 
transaction.state.log.min.isr=1 
log.retention.hours=168 
message.max.byte=5242880 
default.replication.factor=2 
replica.fetch.max.bytes=5242880 
delete.topic.enable=true 
log.cleaner.enable=true 
log.segment.bytes=1073741824 
log.retention.check.interval.ms=300000 
zookeeper.connect=192.168.8.40:12181,192.168.8.106:12181,192.168.8.177:12181 zookeeper.connection.timeout.ms=6000 
group.initial.rebalance.delay.ms=3000

将配置文件scp到另外两台机器,将broker.id和listeners修改一下即可。

④.编写启动文件

# cd /usr/local/kafka/bin 
# vim start.sh 
#!/bin/bash 
/usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties

⑤.启动

# bash /usr/local/kafka/bin/start.sh

⑥.检查

# netstat -lntp 
Active Internet connections (only servers) 
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd 
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 4202/sshd 
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 3138/master 
tcp6 0 0 :::39180 :::* LISTEN 22784/java 
tcp6 0 0 :::111 :::* LISTEN 1/systemd 
tcp6 0 0 :::8080 :::* LISTEN 22270/java 
tcp6 0 0 :::12181 :::* LISTEN 22270/java 
tcp6 0 0 :::22 :::* LISTEN 4202/sshd
tcp6 0 0 192.168.8.177:12888 :::* LISTEN 22270/java 
tcp6 0 0 ::1:25 :::* LISTEN 3138/master 
tcp6 0 0 :::43195 :::* LISTEN 22270/java 
tcp6 0 0 192.168.8.177:13888 :::* LISTEN 22270/java 
tcp6 0 0 192.168.8.177:9092 :::* LISTEN 22784/java
京ICP备19055754号