日本老太做爰xx0ld,好姑娘国语免费高清观看电影,日欧一片内射va在线影院

title: ceph：cephadm工具使用

1、環境說明

3臺VMware虛擬機信息，以及準備部署的服務：

ceph05 192.168.10.30 mon、mgr、osd、mds

ceph06 192.168.10.31 mon、mgr、osd

ceph07 192.168.10.32 mon、mgr、osd

節點信息如下：

[root@ceph05 ~]# hostnamectl 
   Static hostname: ceph05
         Icon name: computer-vm
           Chassis: vm
        Machine ID: e8e9768445e2448fb3ff5e5b7138039c
           Boot ID: 47e7dd7c158e4becadb4f6987bfe263b
    Virtualization: vmware
  Operating System: CentOS Linux 7 (Core)
       CPE OS Name: cpe:/o:centos:centos:7
            Kernel: Linux 3.10.0-957.el7.x86_64
      Architecture: x86-64

2、準備工作

3個節點都需要操作：

關閉firewalld、selinux
配置/etc/hosts
安裝docker-ce，并啟動docker和enable docker
安裝python36

配置chrony，30節點為server，其他兩個為client，如果不配置chrony或ntp，添加集群節點的時候會報錯：

 [root@ceph05 ~]# ceph orch host add ceph06
 INFO:cephadm:Inferring fsid 8e5c4a46-bb82-11ea-9b65-000c29fb7811
 INFO:cephadm:Inferring config /var/lib/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/mon.ceph05/config
 INFO:cephadm:Using recent ceph image ceph/ceph:v15
 Error ENOENT: New host ceph06 (ceph06) failed check: ['INFO:cephadm:podman|docker (/bin/docker) is present', 'INFO:cephadm:systemctl is present', 'INFO:cephadm:lvcreate is present', "WARNING:cephadm:No time sync service is running; checked for ['chrony.service', 'chronyd.service', 'systemd-timesyncd.service', 'ntpd.service', 'ntp.service']", 'INFO:cephadm:Hostname "ceph06" matches what is expected.', 'ERROR: No time synchronization is active']

3、開始配置

Cephdm 通過先在單個主機上“引導”一個Ceph 集群，然后再擴展集群以包含任何其他主機，然后部署所需的服務。這里我們選擇30節點為引導節點。

安裝cephadm

1、獲取cephadm腳本，這個腳本是python寫的，下載地址是：https://raw.githubusercontent.com/ceph/ceph/octopus/src/cephadm/cephadm。

2、添加ceph源并安裝cephadm包：

[root@ceph05 ~]# ./cephadm add-repo --release octopus
[root@ceph05 ~]# ./cephadm install
[root@ceph05 ~]# rpm -qa|grep ceph
cephadm-15.2.4-0.el7.x86_64

引導一個新集群

創建相關目錄，并引導一個集群：

[root@ceph05 ~]# mkdir -p /etc/ceph
[root@ceph05 ~]# cephadm bootstrap --mon-ip 192.168.10.30
INFO:cephadm:Verifying podman|docker is present...
···
         URL: https://ceph05:8443/
        User: admin
    Password: ynh08y9mz1
···
INFO:cephadm:Bootstrap complete.

這個命令完成幾個事情:

1、在30節點上創建mon和mgr守護進程。

2、為ceph集群生成一個新的SSH密鑰，并將其添加到/root/.ssh/authorized_keys文件中。

3、生成配置文件/etc/ceph/ceph.conf。

[root@ceph05 ~]# cat /etc/ceph/ceph.conf 
# minimal ceph.conf for ffa6fa3c-bb6b-11ea-9d9a-000c29fb7811
[global]
    fsid = ffa6fa3c-bb6b-11ea-9d9a-000c29fb7811
    mon_host = [v2:192.168.10.30:3300/0,v1:192.168.10.30:6789/0]

4、生成/etc/ceph/ceph.client.admin.keyring。

5、將公鑰的副本（就是步驟2里面生成的ssh公鑰）寫入/etc/ceph/ceph.pub。

訪問集群

因為沒有在節點上安裝任何ceph軟件包，所以不能和之前一樣直接執行ceph命令。

cephdm shell 命令在一個容器中啟動bash shell來連接到剛創建的ceph集群，這個容器中安裝了所有ceph軟件包。默認情況下，使用/etc/ceph/下面的配置和密鑰。為了方便使用，創建一個alias：

[root@ceph05 ~]# alias ceph='cephadm shell -- ceph'
[root@ceph05 ~]# ceph -s
INFO:cephadm:Inferring fsid 8e5c4a46-bb82-11ea-9b65-000c29fb7811
INFO:cephadm:Inferring config /var/lib/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/mon.ceph05/config
INFO:cephadm:Using recent ceph image ceph/ceph:v15
  cluster:
    id:     8e5c4a46-bb82-11ea-9b65-000c29fb7811
    health: HEALTH_WARN
            Reduced data availability: 1 pg inactive
            OSD count 0 < osd_pool_default_size 3
 
  services:
    mon: 1 daemons, quorum ceph05 (age 15h)
    mgr: ceph05.nxiihf(active, since 14h)
    osd: 0 osds: 0 up, 0 in
 
  data:
    pools:   1 pools, 1 pgs
    objects: 0 objects, 0 B
    usage:   0 B used, 0 B / 0 B avail
    pgs:     100.000% pgs unknown
             1 unknown

也可以在節點上安裝ceph-common包，然后直接通過ceph命令來訪問集群：

$ cephadm install ceph-common

查看下啟動的容器：

[root@ceph05 ~]# docker ps
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS               NAMES
6def9562a206        ceph/ceph-grafana:latest    "/bin/sh -c 'grafana…"   31 seconds ago      Up 30 seconds                           ceph-ffa6fa3c-bb6b-11ea-9d9a-000c29fb7811-grafana.ceph05
aa6c2c7360a1        ceph/ceph:v15               "/usr/bin/ceph-crash…"   2 minutes ago       Up 2 minutes                            ceph-ffa6fa3c-bb6b-11ea-9d9a-000c29fb7811-crash.ceph05
f4f65eaf1c2b        prom/alertmanager:v0.20.0   "/bin/alertmanager -…"   2 minutes ago       Up 2 minutes                            ceph-ffa6fa3c-bb6b-11ea-9d9a-000c29fb7811-alertmanager.ceph05
518a1c8d8025        ceph/ceph:v15               "/usr/bin/ceph-mgr -…"   4 minutes ago       Up 4 minutes                            ceph-ffa6fa3c-bb6b-11ea-9d9a-000c29fb7811-mgr.ceph05.opycfi
5b7e67ce5c45        ceph/ceph:v15               "/usr/bin/ceph-mon -…"   4 minutes ago       Up 4 minutes                            ceph-ffa6fa3c-bb6b-11ea-9d9a-000c29fb7811-mon.ceph05

增加集群節點

把集群的SSH公鑰放到ceph06和ceph07節點上，以便cephadm可以ssh免密管理這兩個節點：

[root@ceph05 ~]# ssh-copy-id -f -i /etc/ceph/ceph.pub root@ceph06
[root@ceph05 ~]# ssh-copy-id -f -i /etc/ceph/ceph.pub root@ceph07

然后添加節點：

[root@ceph05 ~]# ceph orch host add ceph06
INFO:cephadm:Inferring fsid 8e5c4a46-bb82-11ea-9b65-000c29fb7811
INFO:cephadm:Inferring config /var/lib/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/mon.ceph05/config
INFO:cephadm:Using recent ceph image ceph/ceph:v15
Added host 'ceph06'

[root@ceph05 ~]# ceph orch host add ceph07
INFO:cephadm:Inferring fsid 8e5c4a46-bb82-11ea-9b65-000c29fb7811
INFO:cephadm:Inferring config /var/lib/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/mon.ceph05/config
INFO:cephadm:Using recent ceph image ceph/ceph:v15
Added host 'ceph07'

一般ceph集群應該部署3-5個mon，當cephadm知道mon應該使用什么ip子網時（默認使用第一個mon的ip子網），如果向集群中添加集群節點時，cephadm會自動在新節點上部署mon，默認最多部署五個mon（可以使用 ceph orch apply mon <NUM>來設置）。

因為cephadm是根據添加的集群節點自動的部署指定個數的mon，如果我們要在一組特定的主機上部署mon:

$ ceph orch apply mon *<host1,host2,host3,...>*    # 注意這里的hosts列表必須包含引導節點，這里就是30節點

如果mon應該使用特定的ip子網，則在增加集群節點之前設置：

$ ceph config set mon public_network xxx.xxx.xxx.xxx/xx

也可以給每個mon都指定自己的ip：

1、首先禁止自動部署mon：

$ ceph orch apply mon --unmanaged

2、添加mon服務：

$ ceph orch daemon add mon hostname:xxx.xxx.xxx.xxx/xx

部署osd

使用orch編排器增加osd：

[root@ceph05 ~]# ceph orch daemon add osd ceph05:/dev/sdb
INFO:cephadm:Inferring fsid 8e5c4a46-bb82-11ea-9b65-000c29fb7811
INFO:cephadm:Inferring config /var/lib/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/mon.ceph05/config
INFO:cephadm:Using recent ceph image ceph/ceph:v15
Created osd(s) 0 on host 'ceph05'

[root@ceph05 ~]# ceph orch apply mds cephfs --placement="1 ceph05"
INFO:cephadm:Inferring fsid 8e5c4a46-bb82-11ea-9b65-000c29fb7811
INFO:cephadm:Inferring config /var/lib/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/mon.ceph05/config
INFO:cephadm:Using recent ceph image ceph/ceph:v15
Scheduled mds.cephfs update..

配置cephfs

這個步驟和之前的差不多，創建pool，然后創建fs：

[root@ceph05 ~]# ceph osd pool create metadata 16 16
INFO:cephadm:Inferring fsid 8e5c4a46-bb82-11ea-9b65-000c29fb7811
INFO:cephadm:Inferring config /var/lib/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/mon.ceph05/config
INFO:cephadm:Using recent ceph image ceph/ceph:v15
pool 'metadata' created

[root@ceph05 ~]# ceph osd pool create data 16 16
INFO:cephadm:Inferring fsid 8e5c4a46-bb82-11ea-9b65-000c29fb7811
INFO:cephadm:Inferring config /var/lib/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/mon.ceph05/config
INFO:cephadm:Using recent ceph image ceph/ceph:v15
pool 'data' created

[root@ceph05 ~]# ceph fs new cephfs metadata data
INFO:cephadm:Inferring fsid 8e5c4a46-bb82-11ea-9b65-000c29fb7811
INFO:cephadm:Inferring config /var/lib/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/mon.ceph05/config
INFO:cephadm:Using recent ceph image ceph/ceph:v15
new fs with metadata pool 2 and data pool 3

掛載kc：

[root@ceph05 ceph]# ceph auth get-or-create client.guest mds 'allow' osd 'allow *' mon 'allow *' > ceph.client.guest.keyring
[root@ceph05 ceph]# mount -t ceph 192.168.10.30:/ /root/mnt/
[root@ceph05 ceph]# 
[root@ceph05 ceph]# df -h
Filesystem               Size  Used Avail Use% Mounted on
···
192.168.10.30:/           18G     0   18G   0% /root/mnt

最后的集群狀態：

[root@ceph05 ~]# ceph -s
  cluster:
    id:     8e5c4a46-bb82-11ea-9b65-000c29fb7811
    health: HEALTH_OK
    
  services:
    mon: 3 daemons, quorum ceph05,ceph06,ceph07 (age 7m)
    mgr: ceph07.czczlh(active, since 65m), standbys: ceph05.nxiihf
    mds: cephfs:1 {0=cephfs.ceph05.wxezvv=up:active}
    osd: 3 osds: 3 up (since 64m), 3 in (since 2h)
 
  task status:
    scrub status:
        mds.cephfs.ceph05.wxezvv: idle
 
  data:
    pools:   3 pools, 33 pgs
    objects: 22 objects, 2.7 KiB
    usage:   3.0 GiB used, 57 GiB / 60 GiB avail
    pgs:     33 active+clean

所有的ceph進程都是以容器方式啟動：

[root@ceph06 ~]# docker ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS               NAMES
0f1133e6b35d        ceph/ceph:v15                "/usr/bin/ceph-osd -…"   2 hours ago         Up 2 hours                              ceph-8e5c4a46-bb82-11ea-9b65-000c29fb7811-osd.1
084d7c5a7720        prom/node-exporter:v0.18.1   "/bin/node_exporter …"   2 hours ago         Up 2 hours                              ceph-8e5c4a46-bb82-11ea-9b65-000c29fb7811-node-exporter.ceph06
f28ec2e9782f        ceph/ceph:v15                "/usr/bin/ceph-mon -…"   2 hours ago         Up 2 hours                              ceph-8e5c4a46-bb82-11ea-9b65-000c29fb7811-mon.ceph06
0b373e81bb54        ceph/ceph:v15                "/usr/bin/ceph-crash…"   2 hours ago         Up 2 hours                              ceph-8e5c4a46-bb82-11ea-9b65-000c29fb7811-crash.ceph06

相關日志：

[root@ceph06 ~]# ll /var/log/ceph/
total 0
drwxrwx--- 2 167 167 51 Jul  2 09:20 8e5c4a46-bb82-11ea-9b65-000c29fb7811
[root@ceph06 ~]# ll /var/log/ceph/8e5c4a46-bb82-11ea-9b65-000c29fb7811/
total 416
-rw-r--r-- 1  167  167 106424 Jul  2 09:20 ceph-osd.1.log
-rw-r--r-- 1 root root 315891 Jul  2 11:33 ceph-volume.log

掛載情況：

[root@ceph06 ~]# df -h
Filesystem               Size  Used Avail Use% Mounted on
···     
overlay                   17G  3.3G   14G  20% /var/lib/docker/overlay2/f09b8b105e4966ca4f4179d72df652625a616f1f7a8287368c8bdb8c5baeca58/merged
overlay                   17G  3.3G   14G  20% /var/lib/docker/overlay2/1165656fb80ee479f897e97424d152422418d889acc31da816d4b5fd02744177/merged
overlay                   17G  3.3G   14G  20% /var/lib/docker/overlay2/5d2500dcc5a784732d6f2671666d19f3513ce1548f7850f39c1a209864e48627/merged
overlay                   17G  3.3G   14G  20% /var/lib/docker/overlay2/143b664b67fbebc92d4522c91233d0905341e791881af1ffff18c8977af21f1e/merged

4、主要區別

管理思想上和之前的有較大區別。
使用容器來運行ceph相關進程。
部署節點不用安裝ceph相關軟件包以及其依賴包，對系統本身干擾較小。
集群整個管理方式有點像k8s。
其他日常運維指令方便也有很多不同，比如重啟osd服務，使用 ceph orch restart ceph-osd@1 等。

5、問題

后面繼續更新。

三个男躁一个女,国精产品一区一手机的秘密,麦子交换系列最经典十句话,欧美 国产 综合 欧美 视频

ceph：cephadm工具使用