Docker部署hadoop

Docker部署hadoop集群

一、环境准备二、安装并启动Docker三、 配置hadoop四、启动hadoop集群五、关闭hadoop集群六、卸载Docker 一、环境准备

操作系统：Centos7.9 二、安装并启动Docker 安装

方法一：从阿里云镜像获取安装包

sudo yum install -y yum-utilssudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

方法二：从docker官方获取安装包

sudo yum install -y yum-utils sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

建议使用方法一安装过程较快

获取到安装包以后开始安装

yum install -y docker-ce docker-ce-cli containerd.io

安装完成以后可输入docker version验证docker是否安装成功
如果出现
Client: Docker Engine - Community
Version: 20.10.12
API version: 1.41
Go version: go1.16.12
Git commit: e91ed57
Built: Mon Dec 13 11:45:41 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.12
API version: 1.41 (minimum version 1.12)
Go version: go1.16.12
Git commit: 459d0df
Built: Mon Dec 13 11:44:05 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.12
GitCommit: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc:
Version: 1.0.2
GitCommit: v1.0.2-0-g52b36a2
docker-init:
Version: 0.19.0
GitCommit: de40ad0
表示安装成功

启动

sudo systemctl start docker

三、 配置hadoop 下载doker下的hadoop

docker pull kiwenlau/hadoop1.0

在docker中搭建网络

docker network create –d=bridge hadoop

创建hadoop-master容器

docker run -itd --net=hadoop --name hadoop-cluster1 --hostname haddoop-master -p 50070:50070 -p 9000:9000 -p 8088:8088 kiwenlau/hadoop1.0

创建hadoop-slave1容器

docker run -itd --net=hadoop --name hadoop-cluster2 --hostname haddoop-slave1 kiwenlau/hadoop1.0

创建hadoop-slave2容器

docker run -itd --net=hadoop --name hadoop-cluster3 --hostname haddoop-slave2 kiwenlau/hadoop1.0

四、启动hadoop集群 进入hadoop-master容器

docker exec -it hadoop-master bash

启动hadoop集群

./start-hadoop.sh

使用jps命令查看启动的服务

root@hadoop-master:~# jps
1549 SecondaryNameNode
1356 NameNode
8098 Jps
1705 ResourceManager

五、关闭hadoop集群

使用exit可以退出当前容器的命令窗口

exit

此时hadoop容器仍会在后台继续运行
root@hadoop-master:~# exit
exit
[root@qh ~]#

退出容器命令窗口后可使用docker stop hadoop-master命令停止容器，停止后容器不再运行

[root@qh ~]# docker stop hadoop-master

可使用docker ps命令查看所有容器的运行情况

[root@qh ~]# docker ps
ConTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5927d483b848 a59a34125272 “sh -c 'service ssh …” 2 days ago Up 2 days hadoop-slave1
290b68fef58d a59a34125272 “sh -c 'service ssh …” 2 days ago Up 2 days hadoop-slave2
785a90585d6b a59a34125272 “sh -c 'service ssh …” 2 days ago Stop 5 seconds ago 0.0.0.0:8088->8088/tcp, :::8088->8088/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 0.0.0.0:50070->50070/tcp, :::50070->50070/tcp hadoop-master
可见hadoop-master已经停止

六、卸载Docker 卸载 Docker软件包

sudo yum remove docker-ce docker-ce-cli containerd.io

卸载docker后，主机上的容器等一系列文件不会自动删除，删除所有docker有关文件可用：

sudo rm -rf /var/lib/docker sudo rm -rf /var/lib/containerd