ceph-salt

KO | EN

gitlite — search

#python #hacktoberfest #react #ai #typescript #go #nodejs #windows #awesome #cli #awesome-list #vue #security #ios

(★ 82)

Ceph cluster deployment with SaltStack

파일 탐색기

최종 버전 다운로드 (.zip)

- ceph-EXAMPLE.sls
- environment-EXAMPLE.sls
- mine_functions.sls
- top.sls
- - ceph.sls
  - client.sls
  - global_vars.jinja
  - mds.sls
  - mon.sls
  - osd.sls
  - repo.sls
  - rest_api.sls
- - ceph.sls
- - grains.sls
  - minion.sls
- top.sls
- - master
  - minion
- - master.pem
  - master.pub
  - node01.pem
  - node01.pub
  - node02.pem
  - node02.pub
- Vagrantfile
.gitignore
LICENSE
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Ceph cluster deployment

Salt states for Ceph cluster deployment.

Support for:

* Ceph MON
 * Ceph OSD
 * Ceph MDS
 * Ceph Clients
 * Ceph REST API

Details:

* Support for Ceph multi-environment deployment from one salt master node.
 * Deploy any number of MONs, OSDs or MDS services. Also, those states could be used to add new nodes after a cluster is created.
 * Support to select which disks are OSDs or Journals.
 * Support for cluster and public network.

Those states are tested on Ubuntu 14.04 with Ceph Hammer release and Salt v2016.3.2.

# Vagrant

If you want to test this deployment on your local machine inside VMs, the easiest way is to use Vagrant with VirtualBox provider. All you need is to go inside vagrant directory and run:

```
cd vagrant && vagrant up
```
This will bring up 3 VMs, one master, and 3 minion nodes. Ceph will be deployed on all three nodes. Also, those VMs will have two additional network interfaces to emulate public and cluster network for Ceph and three additional drives attached to them. Two will be used for OSDs and one for a journal.

Test the connectivity between master and minions:

```
vagrant ssh master
sudo salt -G 'environment:VAGRANT' test.ping
```
If everything is OK you can proceed with the Ceph deployment step: https://github.com/komljen/ceph-salt#deployment

# Local environment

First, you need Salt master and minions installed and running on all nodes and minions keys should be accepted. The easiest way to install SaltStack is using bootstrap script:

Master:

```
curl -L https://bootstrap.saltstack.com | sudo sh -s -- -M -g https://github.com/saltstack/salt.git git v2016.3.2
```
Minions:

```
curl -L https://bootstrap.saltstack.com | sudo sh -s -- -g https://github.com/saltstack/salt.git git v2016.3.2
```

### Master configuration

On the master node you need to include additional options. Edit master config file ```/etc/salt/master```. Replace ```<USER>``` with username where this repository will be cloned:

```
file_recv: True
file_roots:
  base:
    - /home/<USER>/ceph-salt/salt
    - /var/cache/salt/master/minions
pillar_roots:
  base:
    - /home/<USER>/config
    - /home/<USER>/ceph-salt/pillar
worker_threads: 10
hash_type: sha256
jinja_trim_blocks: True
jinja_lstrip_blocks: True
```

New options will make sure that minions can send files to the master and other minions to be able to get those files. Also here you can change where your salt states and config files are located. Salt master restart is required:

```
sudo service salt-master restart
```

### Minions configuration

On all minion nodes, you need to edit the configuration file. Edit minion config file ```/etc/salt/minion```. Replace ```<ENV_NAME>``` and master IP address to match with your environment:

```
master: 192.168.33.10
hash_type: sha256
grains:
  environment: <ENV_NAME>
```

Salt minion restart is required:

```
sudo service salt-minion restart
```

**NOTE:** To add new Ceph environment just install minions and choose new environment name!

### Connection check

On master node accept all minions with:

```
sudo salt-key -A
```
Now all minions are connected and you should be able to send any command to a particular environment. Examples:

```
sudo salt -G 'environment:PROD' test.ping
sudo salt -G 'environment:STAGE' test.ping
```
If everything is fine clone this git repository on the master node. Use the same user you specified in master configuration file:

```
git clone https://github.com/komljen/ceph-salt.git -b master
```
Copy configuration files for each environment except ```top.sls``` file:

```
mkdir -p ~/config
cp ~/ceph-salt/pillar/environment-EXAMPLE.sls ~/config/environment-<ENV_NAME>.sls
cp ~/ceph-salt/pillar/ceph-EXAMPLE.sls ~/config/ceph-<ENV_NAME>.sls
cp ~/ceph-salt/pillar/top.sls ~/config/top.sls
```
Edit ```~/config/top.sls``` file and replace ENV_NAME with your environment:

```
  'environment:<ENV_NAME>':
    - match: grain
    - environment-<ENV_NAME>
    - ceph-<ENV_NAME>
```
If you have more environments add it here. Example:

```
  'environment:PROD':
    - match: grain
    - environment-PROD
    - ceph-PROD

'environment:STAGE':
    - match: grain
    - environment-STAGE
    - ceph-STAGE
```

### Configuration options

Edit ```~/config/environment-<ENV_NAME>.sls``` file to match with your environment. For node names use hostnames (not FQDN):

```
nodes:
  master:
    roles:
      - ceph-osd
      - ceph-mon
      - ceph-mds
      - ceph-client
      - ceph-rest-api
    osds:
      sdc:
        journal: sdb
      sdd:
        journal: sdb
  node01:
    roles:
      - ceph-osd
      - ceph-mon
      - ceph-mds
    osds:
      sdc:
        journal: sdb
      sdd:
        journal: sdb
  node02:
    roles:
      - ceph-osd
      - ceph-mon
      - ceph-mds
    osds:
      sdc:
        journal: sdb
      sdd:
        journal: sdb
```
Now edit ```~/config/ceph-<ENV_NAME>.sls``` if you want to make additional changes to ceph configuration. Take a look at those options to match with your machines:

```
ceph:
  version: hammer
  cluster_name: ceph
  rest_api:
    port: 5000
  global:
    cluster_network: 192.168.36.0/24
    fsid: 294bc494-81ba-4c3c-ac5d-af7b3442a2a5
    public_network: 192.168.33.0/24
  mon:
    interface: eth1 # Should match public_network
```
Proceed with deployment step after all changes are done.

**NOTE:** Generate your FSID with ```uuidgen``` command!

# Deployment

First, you need to run high state to add roles to minions based on ```environment-<ENV_NAME>.sls``` file. All roles for all environments will be applied:

```
sudo salt '*' state.highstate
```
To start Ceph cluster deployment run orchestrate state:

```
sudo salt-run state.orchestrate deploy.ceph pillar='{environment: ENV_NAME}'
```
It will take few minutes to complete. Then you can check ceph cluster status:

```
sudo ceph -s
```

ceph-salt

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

ceph-salt

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README