利用bucket localtion实现rgw集群扩容

扩容思路：ceph集群容量不足的时候，新加的OSD组成新的pool，制定新的rule规则，新建的bucket都存放在这些新加入的pool里面，注意扩容仅限于新加的bucket，已有的bucket扩容不适用。

###1.新建pool

ceph osd pool create .rgw.buckets.custom 4096 4096
ceph osd pool create .rgw.buckets.custom.index 512 512
ceph osd pool create .rgw.buckets.custom.extra 64 64

###2.调整region和zone配置 导出region配置

radosgw-admin region get --name client.radosgw.us-zone1  > region.conf.json

编辑region.conf.json,修改placement_targets部分

	#修改前
    "placement_targets": [
        {
            "name": "default-placement",
            "tags": []
        }
    ],
    "default_placement": "default-placement"

	#修改后
    "placement_targets": [
        {
            "name": "default-placement",
            "tags": []
        },
        {
            "name": "new",
            "tags": []
        }
    ],
    "default_placement": "default-placement"

导出zone配置

radosgw-admin zone get --name client.radosgw.us-zone1 > zone.conf.json

编辑zone.conf.json,修改placement_targets部分

	#修改前
"placement_pools": [
        {
            "key": "default-placement",
            "val": {
                "index_pool": ".us-zone1.rgw.buckets.index",
                "data_pool": ".us-zone1.rgw.buckets",
                "data_extra_pool": ".us-zone1.rgw.buckets.extra"
            }
        }
   ]

	#修改后
"placement_pools": [
        {
            "key": "default-placement",
            "val": {
                "index_pool": ".us-zone1.rgw.buckets.index",
                "data_pool": ".us-zone1.rgw.buckets",
                "data_extra_pool": ".us-zone1.rgw.buckets.extra"
            }
        },
        {
            "key": "new",
            "val": {
                "index_pool": ".rgw.buckets.custom.index",
                "data_pool": ".rgw.buckets.custom",
                "data_extra_pool": ".rgw.buckets.custom.extra"
            }
        }

导入配置

radosgw-admin region set --name client.radosgw.us-zone1 < region.conf.json
radosgw-admin zone set --name client.radosgw.us-zone1 < zone.conf.json

刷新配置信息

radosgw-admin regionmap update --name client.radosgw.us-zone1

3.重启服务

注意：所有radosgw服务节点都要进行这一步

/etc/init.d/radosgw restart

###4. 验证测试

测试方法1： 新建bucket需要指定localtion，boto代码如下

from boto.s3.connection import S3Connection
import boto

access_key = 'xxx'
secret_key = 'xxxx'
host = 's3.ceph.work'
bucket = 'localtest2'

conn = S3Connection(access_key, secret_key, host=host, is_secure=False)

conn = boto.connect_s3(
        aws_access_key_id = access_key,
        aws_secret_access_key = secret_key,
        host = host,
        is_secure=False,
        calling_format = boto.s3.connection.SubdomainCallingFormat(),
        validate_certs = True,
        )
bucket = conn.create_bucket(bucket,location=':new')
bucket = conn.get_bucket(bucket)
print bucket

测试方法2：

s3cmd mb s3://localtest2 --bucket-location=:new

验证

root@ceph.work/# radosgw-admin bucket stats --bucket=localtest2 --name client.radosgw.us-zone1

{
    "bucket": "localtest2",
    "pool": ".rgw.buckets.custom",
    "index_pool": ".rgw.buckets.custom.index",
    "id": "us-zone1.477528.2",
    "marker": "us-zone1.477528.2",
    "owner": "gf",
    "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1",
    "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0",
    "mtime": "2016-07-07 15:50:53.000000",
    "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#",
    "usage": {},
    "bucket_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    }
}

来源：oschina

链接：https://my.oschina.net/u/97044/blog/707834