安装前准备

修改环境变量

vi /etc/profile

export KAFKA_HOME=/opt/kafka

export PATH=$PATH:$KAFKA_HOME/bin

创建用户及用户组

groupadd kafka

useradd -g kafka kafka

格式化磁盘并挂载

1. 格式化磁盘

mkfs.xfs /dev/sd{b,c,d,e,f}

2. 创建挂载目录并挂载

mkdir -p /data{1,2,3,4,5}

mount /dev/sdb /data1

mount /dev/sdc /data2

mount /dev/sdd /data3

mount /dev/sde /data4

mount /dev/sdf /data5

mkdir -p /data{1,2,3,4,5}/kafka

3. 写入fstab文件

vi /etc/fatab

/dev/sdb /data{1,2,3,4,5} xfs defaults 0 0

安装kafka

1. 上传安装包并解压

tar -zxf kafka_2.12-2.8.2.tgz -C /opt/

ln -s kafka_2.12-2.8.2 /opt/kafka

2. 复制目录

scp -rp /opt/kafka_2.12-2.8.2 DPS2-kafka-qas2:/opt

scp -rp /opt/kafka_2.12-2.8.2 DPS2-kafka-qas3:/opt

3. 修改权限

mkdir -p /log/kafka

chown -R kafka:kafka /opt/kafka/

for i in {1..5}; do chown -R kafka:kafka /data$i/kafka/;done

chown -R kafka:kafka /log/kafka

配置kafka

修改配置文件

vi /opt/kafka/config/server.properties

broker.id=0 #每个节点的id不一样

listeners = PLAINTEXT://172.16.xxx.xxx:9092 #IP为每个节点的ip

num.network.threads=3

num.io.threads=8

socket.send.buffer.bytes=102400

socket.receive.buffer.bytes=102400

socket.request.max.bytes=104857600

log.dirs=/data1/kafka,/data2/kafka,/data3/kafka,/data4/kafka,/data5/kafka

num.partitions=1

num.recovery.threads.per.data.dir=1

offsets.topic.replication.factor=1

transaction.state.log.replication.factor=1

transaction.state.log.min.isr=1

log.retention.hours=168

log.segment.bytes=1073741824

log.retention.check.interval.ms=300000

zookeeper.connect=172.16.xxx.xxx:2181,172.16.xxx.xxx:2181,172.16.xxx.xxx:2181/kafka

zookeeper.connection.timeout.ms=18000

group.initial.rebalance.delay.ms=0

修改脚本

vi /opt/kafka/bin/kafka-run-class.sh(208行)

LOG_DIR="$base_dir/logs" 改为LOG_DIR="/log/kafka"

启动kafka

1. 切换用户

苏- kafka

2. 启动服务

cd /opt/kafka

bin/kafka-server-start.sh -daemon config/server.properties