CEPH

Aufbau

Storage-Node und Monitor Nodes

• omar
• nelson
• bay

Admin-Node

• kekistan

Vorbreitung

Installation von ssh und ntp

Auf allen Nodes

• sudo apt -y install ntp ssh
• wget -q -O- 'https://download.ceph.com/keys/release.asc' | apt-key add -
• echo deb http://download.ceph.com/debian-luminous/ $(lsb_release -sc) main | tee /etc/apt/sources.list.d/ceph.list
• apt-get update

ssh-key auf allen nodes

ssh-keygen

• root@kekistand:~# ssh-keygen

ssh-copy-id

• root@kekistand:~# ssh-copy-id nelson
• root@kekistand:~# ssh-copy-id bay
• root@kekistand:~# ssh-copy-id omar

Hilfskript

• cat /usr/local/sbin/root-all-nodes

#!/bin/bash
echo localhost
$*
LIST="bay omar nelson"
for k in $LIST
do
echo $k
ssh k $*
done

Anlegen des Users cephadmin und keyhandling

• root-all-nodes useradd -s /bin/bash -m cephadmin
• sudo -u cephadmin ssh-keygen -q
• root-all-nodes mkdir /home/cephadmin/.ssh
• root-all-nodes chown cephadmin.cephadmin /home/cephadmin/.ssh
• root-all-nodes chmod 700 /home/cephadmin/.ssh
• root-all-nodes 'echo "cephadmin ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/ceph'
• scp /home/cephadmin/.ssh/id_rsa.pub /tmp
• scp /home/cephadmin/.ssh/id_rsa.pub bay:/tmp
• scp /home/cephadmin/.ssh/id_rsa.pub nelson:/tmp
• scp /home/cephadmin/.ssh/id_rsa.pub omar:/tmp
• root-all-nodes cat /tmp/id_rsa.pub > /home/cephadmin/.ssh/authorized_keys"

Install

• apt install ceph-deploy
• su - cephadmin
• mkdir my-first-ceph-cluster
• cd my-first-ceph-cluster

Monitor Nodes

• ceph-deploy new bay nelson omar
• echo "public network = 10.0.0.0/8" >> ceph.conf
• echo "osd pool default size = 2" >> ceph.conf

Install Ceph

• ceph-deploy install kekistan bay nelson omar

Initialisierung der Monitore und sammeln der der Keys

• ceph-deploy mon create-initial

Vorbereiten der Storages

• ceph-deploy osd prepare bay:/dev/sdb nelson:/dev/sdb omar:/dev/sdb

Aktivieren der Storages

• ceph-deploy osd activate bay:/dev/sdb1 nelson:/dev/sdb1 omar:/dev/sdb1

Kopieren der Admin-keyrings

• ceph-deploy admin kekistan bay nelson omar
• sudo chmod 644 /etc/ceph/ceph.client.admin.keyring

Status check

• ceph osd tree

ID WEIGHT  TYPE NAME       UP/DOWN REWEIGHT PRIMARY-AFFINITY 
-1 1.44955 root default                                      
-2 0.48318     host bay                                      
 0 0.48318         osd.0        up  1.00000          1.00000 
-3 0.48318     host nelson                                   
 1 0.48318         osd.1        up  1.00000          1.00000 
-4 0.48318     host omar                                     
 2 0.48318         osd.2        up  1.00000          1.00000 

• ceph health

HEALTH_OK

Anelegen eines Metadata Server(Voraussetzung für das CephFS

• ceph-deploy mds create bay nelson omar
• ceph osd pool create cephfs_data 1
• ceph osd pool create cephfs_metadata 1
• ceph fs new cephfs cephfs_metadata cephfs_data
• ceph mds stat

Client

auf dem Client cephadmin einrichten und keys von admin node kopieren ...

• ceph-deploy install bajor
• ceph-deploy admin bajor
• rbd create bar --size 4096 --image-feature layering
• sudo rbd map bar --name client.admin
• sudo mkfs.ext4 /dev/rbd0
• sudo mkdir /mnt/ceph-block-device
• sudo mount /dev/rbd0 /mnt/ceph-block-device

Client File System

auf dem Client das secret speichern

mkdir  /etc/ceph/
vi /etc/ceph/admin.secret

TIPP secret auf dem client soll in der form sein
ÖLUJmb5rfHOH8GigijgGTfu==

• eintrag in der fstab anlegen

vi /etc/fstab
10.1.1.1,10.1.1.2,10.1.1.3:/ /mnt/cephfs ceph name=admin,secretfile=/etc/ceph/admin.secret 0 0

• mountpunkt erstellen & Files System mounten

mkdir /mnt/cephfs
mount /mnt/cephfs

Links

• http://docs.ceph.com/docs/master/start/
• https://linoxide.com/ubuntu-how-to/create-ceph-cluster-ubuntu-16-04/
• https://www.howtoforge.com/tutorial/how-to-install-a-ceph-cluster-on-ubuntu-16-04/
• https://www.howtoforge.com/tutorial/how-to-mount-cephfs-on-centos-7/
• http://www.admin-magazin.de/Das-Heft/2014/04/Die-verteilten-Speichersysteme-GlusterFS-und-Ceph-im-Vergleich/(offset)/12