大数据运维（41）kafka 2.3.0集群部署
安装kafka之前先安装zookeeper。
配置DNS解析：
|
1
2
3
4
5
6
7
8
9
10
|
cat > /etc/hosts <<EOF
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4::1 localhost localhost.localdomain localhost6 localhost6.localdomain6192.168.0.51 zookeeper1192.168.0.52 zookeeper2192.168.0.53 zookeeper3192.168.0.57 kafka1192.168.0.58 kafka2192.168.0.59 kafka3EOF |
配置免秘钥：
|
1
2
3
4
|
ssh-keygen
ssh-copy-id -i .ssh/id_rsa.pub root@kafka1
ssh-copy-id -i .ssh/id_rsa.pub root@kafka2
ssh-copy-id -i .ssh/id_rsa.pub root@kafka3
|
安装java：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
tar -xf jdk-8u231-linux-x64.tar.gz -C /usr/local/
cd /usr/local/
ln -sv jdk1.8.0_231/ jdk
cat > /etc/profile.d/java.sh <<EOF
export JAVA_HOME=/usr/local/jdk
export JRE_HOME=\$JAVA_HOME/jre
export CLASSPATH=.:\$JAVA_HOME/lib/dt.jar:\$JAVA_HOME/lib/tools.jar:\$JRE_HOME/lib
export PATH=\$PATH:\$JAVA_HOME/bin:\$JRE_HOME/bin
EOF. /etc/profile.d/java.sh
java -versionjavac -version |
下载安装包：包的前面是对应scala的版本后面的2.3.0才是kafka的版本。
|
1
|
wget http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.3.0/kafka_2.12-2.3.0.tgz
|
复制安装包：
|
1
|
scp kafka_2.12-2.3.0.tgz root@kafka1:/root
|
解压安装：
|
1
2
3
|
tar -xf kafka_2.12-2.3.0.tgz -C /usr/local/
cd /usr/local/
ln -sv kafka_2.12-2.3.0/ kafka
|
设置环境变量：
|
1
2
3
4
5
|
cat > /etc/profile.d/kafka.sh <<EOF
KAFKA_HOME=/usr/local/kafka
PATH=\$PATH:\$KAFKA_HOME/bin
EOF. /etc/profile.d/kafka.sh
|
普通用户环境变量：
|
1
2
3
4
5
|
cat >> ~/.bashrc <<EOF
export KAFKA_HOME=/usr/local/kafka
export PATH=\$PATH:\$KAFKA_HOME/bin
EOF. ~/.bashrc |
创建日志目录：
|
1
|
mkdir /usr/local/kafka/logs
|
修改配置：
|
1
2
|
cd /usr/local/kafka/config
vim server.properties |
kafka1的server.properties：不是追加是修改如下几项
|
1
2
3
4
|
broker.id=0
listeners=PLAINTEXT://kafka1:9092
log.dirs=/usr/local/kafka/logs
zookeeper.connect=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181 |
kafka2的server.properties：
|
1
2
3
4
|
broker.id=1
listeners=PLAINTEXT://kafka2:9092
log.dirs=/usr/local/kafka/logs
zookeeper.connect=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181 |
kafka3的server.properties：
|
1
2
3
4
|
broker.id=2
listeners=PLAINTEXT://kafka3:9092
log.dirs=/usr/local/kafka/logs
zookeeper.connect=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181 |
修改属组属主：
|
1
|
chown -R hadoop.hadoop /usr/local/kafka/ /usr/local/kafka
|
启动kafka：到各节点分别启动kafka。
|
1
|
./kafka-server-start.sh ../config/server.properties &
|
查看运行的进程：
|
1
2
3
|
]$ jps28493 Kafka... |
停止kafka：
|
1
2
|
cd /usr/local/kafka/bin/
./kafka-server-stop.sh
|
版权声明：本文为huanghanyu原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。