一、Hadoop-HA-YARN-HA搭建:YARN-HA配置
### --- YARN-HA工作机制

~~~     官方文档:https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarnsite/ResourceManagerHA.html
二、YARN-HA工作机制,如图
三、配置YARN-HA集群
### --- 环境准备

~~~     修改IP
~~~     修改主机名及主机名和IP地址的映射
~~~     关闭防火墙
~~~     ssh免密登录
~~~     安装JDK,配置环境变量等
~~~     配置Zookeeper集群
四、规划集群
linux121 linux122 linux123
NameNode NameNode  
JournalNode JournalNode JournalNode
DataNode DataNode DataNode
ZK ZK ZK
  ResourceManager ResourceManager
NodeManager NodeManager NodeManager
五、具体配置
### --- yarn-site.xml3

[root@linux121 ~]# vim /opt/yanqi/servers/ha/hadoop-2.9.2/etc/hadoop/yarn-site.xml

<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
</property>

<!--启用resourcemanager ha-->
<property>
        <name>yarn.resourcemanager.ha.enabled</name>
        <value>true</value>
</property>

<!--声明两台resourcemanager的地址-->
<property>
        <name>yarn.resourcemanager.cluster-id</name>
        <value>cluster-yarn</value>
</property>
<property>
        <name>yarn.resourcemanager.ha.rm-ids</name>
        <value>rm1,rm2</value>
</property>
<property>
        <name>yarn.resourcemanager.hostname.rm1</name>
        <value>linux122</value>
</property>
<property>
        <name>yarn.resourcemanager.hostname.rm2</name>
        <value>linux123</value>
</property>

<!--指定zookeeper集群的地址-->
<property>
        <name>yarn.resourcemanager.zk-address</name>
        <value>linux121:2181,linux122:2181,linux123:2181</value>
</property>

<!--启用自动恢复-->
<property>
        <name>yarn.resourcemanager.recovery.enabled</name>
        <value>true</value>
</property>

<!--指定resourcemanager的状态信息存储在zookeeper集群-->
<property>
        <name>yarn.resourcemanager.store.class</name>
        <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
</property>
 
### --- 同步更新其他节点的配置信息

[root@linux121 ~]# rsync-script /opt/yanqi/servers/ha/hadoop-2.9.2/etc/hadoop/yarn-site.xml
六、验证yarn集群
### --- 在linux123上启动yarn的主节点

[root@linux123 ~]# /opt/yanqi/servers/ha/hadoop-2.9.2/sbin/start-yarn.sh 
starting resourcemanager, logging to /opt/yanqi/servers/ha/hadoop-2.9.2/logs/yarn-root-resourcemanager-linux123.out
linux122: starting nodemanager, logging to /opt/yanqi/servers/ha/hadoop-2.9.2/logs/yarn-root-nodemanager-linux122.out
linux121: starting nodemanager, logging to /opt/yanqi/servers/ha/hadoop-2.9.2/logs/yarn-root-nodemanager-linux121.out
linux123: starting nodemanager, logging to /opt/yanqi/servers/ha/hadoop-2.9.2/logs/yarn-root-nodemanager-linux123.out
七、访问ResourceManager服务http://linux123:8088/cluster
### --- 在linux122启动yarn的备用节点
~~~     访问:http://linux122:8088/cluster它会自动跳转到http://linux123:8088/cluster

[root@linux122 ~]# /opt/yanqi/servers/ha/hadoop-2.9.2/sbin/yarn-daemon.sh start resourcemanager
[root@linux122 ~]# jps
10480 ResourceManager
### --- 把linux123上的yarn杀死,验证http://linux122:8088/cluster是否可以访问
~~~     杀死linux123上yarn的resourcemanager进程
~~~     验证http://linux123:8088/cluster访问不通

[root@linux123 ~]# jps
11978 ResourceManager
[root@linux123 ~]# kill -9 11978
八、访问http://linux122:8088/cluster可以正常访问到数据;自动升级为主节点。
九、查看启动的所有服务
### --- linux121

[root@linux121 ~]# jps
11136 QuorumPeerMain
11360 JournalNode
11491 NameNode
12387 NodeManager
11806 DataNode
12095 DFSZKFailoverController
12559 Jps
### --- linux122

[root@linux122 ~]# jps
10480 ResourceManager
9905 DFSZKFailoverController
9747 DataNode
10838 Jps
10087 NameNode
9544 JournalNode
9342 QuorumPeerMain
10318 NodeManager
### --- linux123

[root@linux123 ~]# jps
11522 QuorumPeerMain
12517 Jps
11798 DataNode
12454 ResourceManager
12074 NodeManager
11711 JournalNode

 
 
 
 
 
 
 
 
 

Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm’d both hands before the fire of life.It sinks, and I am ready to depart
                                                                                                                                                   ——W.S.Landor

 
版权声明:本文为yanqi_vip原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://www.cnblogs.com/yanqivip/p/16117302.html