部署ceph

时光毁灭记忆、已成空白 提交于 2020-03-19 12:50:33

一、部署准备:

4台虚拟机(linux系统为centos7.6版本)

dlp:192.168.88.50

node1:192.168.88.55

node2:192.168.88.56

node3:192.168.88.57

clip_image001

(1)所有ceph集群节点(包括客户端)设置静态域名解析;

[root@dlp ~]# vim /etc/hosts

clip_image002

(2)所有集群节点(包括客户端)创建cent用户,并设置密码,后执行如下命令:

1. 添加cent用户:

useradd cent && echo "123" | passwd

clip_image003

2.赋予sudo权限:

echo -e 'Defaults:cent !requiretty\ncent ALL = (root) NOPASSWD:ALL' | tee /etc/sudoers.d/ceph

clip_image004

3. 修改权限

chmod 440 /etc/sudoers.d/ceph

(3)在部署节点切换为cent用户,设置无密钥登陆各节点包括客户端节点

[root@dlp ~]# su - cent

[cent@dlp ~]$ ssh-keygen

[cent@dlp ~]$ ssh-copy-id node1

[cent@dlp ~]$ ssh-copy-id node2

[cent@dlp ~]$ ssh-copy-id node3

[cent@dlp ~]$ ssh-copy-id dlp

(4)在部署节点切换为cent用户,在cent用户家目录,设置如下文件:vi~/.ssh/config# create new ( define all nodes and users )

[cent@dlp ~]$ ls -a

clip_image005

[cent@dlp ~]$ cd .ssh/

[cent@dlp .ssh]$ vim config #config是新建的文件,下面是配置内容

Host dlp

Hostname dlp

User cent

Host node1

Hostname node1

User cent

Host node2

Hostname node2

User cent

Host node3

Hostname node3

User cent

[cent@dlp .ssh]$ chmod 600 ~/.ssh/config

二、所有节点配置国内ceph源:

(1)all-node(包括客户端)在/etc/yum.repos.d/创建 ceph-yunwei.repo

[cent@dlp ~]$ cat /etc/redhat-release #查看系统版本

clip_image006

(2)到国内ceph源中https://mirrors.aliyun.com/centos/7.6.1810/storage/x86_64/ceph-jewel/下载如下所需rpm包。注意:红色框中为ceph-deploy的rpm,只需要在部署节点安装,下载需要到https://mirrors.aliyun.com/ceph/rpm-jewel/el7/noarch/中找到最新对应的ceph-deploy-xxxxx.noarch.rpm 下载

[cent@dlp ~]$ ls

clip_image007

[cent@dlp ~]$ cd cephjrpm/

clip_image008

(3)将下载好的rpm拷贝到所有节点,并安装。注意ceph-deploy-xxxxx.noarch.rpm 只有部署节点用到,其他节点不需要,部署节点也需要安装其余的rpm

[root@dlp cephjrpm]# yum localinstall ceph-deploy-1.5.39-0.noarch.rpm -y

[root@dlp cephjrpm]# ceph-deploy --version # 查看 ceph-deploy版本号

clip_image009

[root@dlp cephjrpm]# mv ceph-deploy-1.5.39-0.noarch.rpm /root/

[root@dlp cephjrpm]# yum localinstall -y ./*

在node1,2,3节点上安装除了ceph-deploy-1.5.39-0.noarch.rpm外的所以软件包

(4)在部署节点(cent用户下执行):安装 ceph-deploy,在root用户下,进入下载好的rpm包目录,执行:

yum localinstall -y ./*

创建ceph工作目录

[root@dlp ~]# su - cent

[cent@dlp ~]$ mkdir ceph

[cent@dlp ~]$ cd ceph/

[cent@dlp ceph]$ pwd

/home/cent/ceph

(5)在部署节点(cent用户下执行):配置新集群

[cent@dlp ceph]$ pwd

/home/cent/ceph

[cent@dlp ceph]$ ceph-deploy new node1 node2 node3 #在ceph工作目录下

clip_image010

[cent@dlp ceph]$ vim ceph.conf

[global]

fsid = 062e2b8a-2fb6-4982-aab2-3440d01d2b2c

mon_initial_members = node1, node2, node3

mon_host = 192.168.88.55,192.168.88.56,192.168.88.57

auth_cluster_required = cephx

auth_service_required = cephx

auth_client_required = cephx

osd_pool_default_size = 1 # 保存副本数量

mon_clock_drift_allowed = 2 # 至少由两个节点正常,集群才为正常状态

<br>mon_clock_drift_warn_backoff = 3 # 每3秒做一次ceph集群的健康检查

可选参数如下:

public_network = 192.168.254.0/24

cluster_network = 172.16.254.0/24

osd_pool_default_size = 3

osd_pool_default_min_size = 1

osd_pool_default_pg_num = 8

osd_pool_default_pgp_num = 8

osd_crush_chooseleaf_type = 1

[mon]

mon_clock_drift_allowed = 0.5

[osd]

osd_mkfs_type = xfs

osd_mkfs_options_xfs = -f

filestore_max_sync_interval = 5

filestore_min_sync_interval = 0.1

filestore_fd_cache_size = 655350

filestore_omap_header_cache_size = 655350

filestore_fd_cache_random = true

osd op threads = 8

osd disk threads = 4

filestore op threads = 8

max_open_files = 655350

(6)在部署节点执行,所有节点安装ceph软件

[cent@dlp ceph]$ ceph-deploy install dlp node1 node2 node3 #在工作目录下进行操作

(7)在部署节点初始化集群(cent用户下执行):

[cent@dlp ceph]$ ceph-deploy mon create-initial #工作目录

查看启动状态

[root@node1 cephjrpm]# systemctl status ceph-mon@node1 ceph-mon.target

[root@node2 cephjrpm]# systemctl status ceph-mon@node2 ceph-mon.target

[root@node3 cephjrpm]# systemctl status ceph-mon@node3 ceph-mon.target

(8)每个节点将第二块硬盘做分区,并格式化为xfs文件系统挂载到/data:

[root@node1 ~]# fdisk /dev/sdb

clip_image011

clip_image012

[root@node2 ~]# fdisk /dev/sdb

clip_image013

[root@node3 ~]# fdisk /dev/sdc

clip_image014

(9)准备Object Storage Daemon:

[cent@dlp ceph]$ ceph-deploy osd prepare node1:/dev/sdb1

[cent@dlp ceph]$ ceph-deploy osd prepare node2:/dev/sdb1

[cent@dlp ceph]$ ceph-deploy osd prepare node3:/dev/sdc1

clip_image015

(10)激活Object Storage Daemon:

[cent@dlp ceph]$ ceph-deploy osd activate node1:/dev/sdb1 node2:/dev/sdb1 node3:/dev/sdc1

clip_image016

(11)在部署节点transfer config files

[cent@dlp ceph]$ ceph-deploy admin dlp node1 node2 node3

[cent@dlp ceph]$ sudo chmod 644 /etc/ceph/ceph.client.admin.keyring

[root@node1 ~]# sudo chmod 644 /etc/ceph/ceph.client.admin.keyring

[root@node2 ~]# sudo chmod 644 /etc/ceph/ceph.client.admin.keyring

[root@node3 ~]# sudo chmod 644 /etc/ceph/ceph.client.admin.keyring

(12)在ceph集群中任意节点检测:

[cent@dlp ceph]$ ceph -s

[cent@dlp ceph]$ ceph osd tree

clip_image017

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!