Greenplum安装

评估存储能力

计算可用的空间

步骤1：初始存储能力=硬盘大小*硬盘数
步骤2：配置RAID10，格式化磁盘空间=(初始存储能力*0.9)/2
步骤3：可用磁盘空间=格式化磁盘空间*0.7
步骤4：用户数据使用空间

使用镜像：(2*用户数据)+用户数据/3=可用磁盘空间

不使用镜像：用户数据+用户数据/3=可用磁盘空间

计算用户数据大小

平均来说，实际占用磁盘空间大小=用户数据*1.4
页面开销：32KB页面需要20 bytes
行开销：每行24 bytes，'append-only'表需要4bytes
索引开销：
B-tree:唯一值*(数据类型大小+24bytes) Bitmap:(唯一值*行数*1bit*压vi缩比率/8)+(唯一值*32)

为元数据和日志计算空间需求

系统元数据：20M

预写日志(WAL)：WAL被拆分成多个64M的文件，WAL文件数最多为

2*checkpoint_segments+1，checkpoint_segments默认值为8。也就意味着每个实例需要1088MB的WAL空间

GP数据库日志文件：日志轮转

性能监控数据

实验环境

硬件环境

VMware虚拟机软件10.0
三台Linux虚拟机：Red Hat Enterprise Linux Serverrelease 5.42

数据库：greenplum-db-4.2.8.0-build-1-RHEL5-x86_64.zip

网卡设置

虚拟机配置

基本信息配置

节点	IP	内存	硬盘
master	192.168.80.200	1024M	20G
slave1	192.168.80.201	1024M	20G
slave2	192.168.80.202	1024M	20G

设置IP地址的命令行方式

vi /etc/sysconfig/network-scripts/ifcfg-eth0

填写ip地址、子网掩码、网关、DNS等

DEVICE=eth0

HWADDR=00:0c:29:b7:7d:69

ONBOOT=yes

BOOTPROTO=static

DNS1=192.168.80.200

IPV6INIT=no

USERCTL=no

IPADDR=192.168.80.200

NETMASK=255.255.255.0

GATEWAY=192.168.80.1

关闭防火墙

[root@localhost ~]# service iptables status

表格：filter

Chain INPUT (policy ACCEPT)

num target prot opt source destination

1 RH-Firewall-1-INPUT all -- 0.0.0.0/0 0.0.0.0/0

Chain FORWARD (policy ACCEPT)

num target prot opt source destination

1 RH-Firewall-1-INPUT all -- 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT)

num target prot opt source destination

Chain RH-Firewall-1-INPUT (2 references)

num target prot opt source destination

1 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0

2 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp type 255

3 ACCEPT esp -- 0.0.0.0/0 0.0.0.0/0

4 ACCEPT ah -- 0.0.0.0/0 0.0.0.0/0

5 ACCEPT udp -- 0.0.0.0/0 224.0.0.251 udp dpt:5353

6 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:631

7 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:631

8 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED

9 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:21

10 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22

11 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:23

12 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:80

13 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:443

14 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited

[root@localhost ~]# service iptables stop

清除防火墙规则： [确定]

把 chains 设置为 ACCEPT 策略：filter [确定]

正在卸载 Iiptables 模块： [确定]

修改主机名和IP映射：

[root@master ~]# vi /etc/sysconfig/network

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME=master

[root@localhost ~]# vi /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomain localhost

::1 localhost6.localdomain6 localhost6

192.168.80.200 master

系统参数配置

包括共享内存、网络、用户限制等，修改或添加/etc/sysctl.conf

# Kernel sysctl configuration file for Red Hat Linux

# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and

# sysctl.conf(5) for more details.

# Controls source route verification

net.ipv4.conf.default.rp_filter = 1

#for greenplum begin

xfs_mount_options = rw,noatime,inode64,allocsize=16m

# Controls the maximum shared segment size, in bytes

kernel.shmmax = 500000000

# Controls the minimum shared segment size, in bytes

kernel.shmmni = 4096

# Controls the maximum number of shared memory segments, in pages

kernel.shmall = 4000000000

kernel.sem = 250 512000 100 2048

# Controls the System Request debugging functionality of the kernel

kernel.sysrq = 1

# Controls whether core dumps will append the PID to the core filename

# Useful for debugging multi-threaded applications

kernel.core_uses_pid = 1

# Controls the maximum size of a message, in bytes

kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue

kernel.msgmax = 65536

# Controls the default minmimum size of a mesage queue

kernel.msgmni = 2048

# Controls the use of TCP syncookies

net.ipv4.tcp_syncookies = 1

# Controls IP packet forwarding

net.ipv4.ip_forward = 0

# Do not accept source routing

net.ipv4.conf.default.accept_source_route = 0

net.ipv4.tcp_tw_recycle = 1

net.ipv4.tcp_max_syn_backlog = 4096

net.ipv4.conf.all.arp_filter = 1

net.ipv4.ip_local_port_range = 1025 65535

net.core.netdev_max_backlog = 10000

vm.overcommit_memory = 2

#for greenplum end

系统设置（所有节点）

配置/etc/security/limits.conf文件

# for greenplum begin

* soft nofile 65536

* hard nofile 65536

* soft nproc 131072

* hard nproc 131072

# for greenplum end

设置磁盘访问I/O调度策略

说明：介绍文章：http://blog.csdn.net/theorytree/article/details/6259104

deadline的调度策略可以平衡IO调度和进程调度，不会造成等待进程频繁的等待

[root@master ~] # cat /sys/block/sda/queue/scheduler

noop anticipatory deadline [cfq]

[root@master ~] # cat /sys/block/fd0/queue/scheduler

noop anticipatory deadline [cfq]

[root@master ~]# dmesg | grep -i scheduler

io scheduler noop registered

io scheduler anticipatory registered

io scheduler deadline registered

io scheduler cfq registered (default)

[root@master ~]# echo deadline > /sys/block/sda/queue/scheduler

[root@master ~]# echo deadline > /sys/block/fd0/queue/scheduler

[root@master ~]# cat /sys/block/sda/queue/scheduler

noop anticipatory [deadline] cfq

[root@master ~]# cat /sys/block/fd0/queue/scheduler

noop anticipatory [deadline] cfq

设置预读块的值为16384;

16384：数据仓库的最大特点是用于保存历史数据，存在大量的数据操作，包括增删改查，当设置的块越大时读取性能越高；16384是greenplum数据库要求的最低要求。

[root@master ~]# /sbin/blockdev --setra 16384 /dev/sda

[root@master ~]# /sbin/blockdev --setra 16384 /dev/sda1

[root@master ~]# /sbin/blockdev --setra 16384 /dev/sda2

[root@master ~]# /sbin/blockdev --setra 16384 /dev/sda3

[root@master ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 653 5245191 83 Linux

/dev/sda2 654 914 2096482+ 82 Linux swap / Solaris

/dev/sda3 915 1045 1052257+ 83 Linux

修改/etc/hosts，添加如下内容

127.0.0.1 localhost.localdomain localhost

::1 localhost6.localdomain6 localhost6

192.168.80.200 master

192.168.80.201 slave1

192.168.80.202 slave2

在Master节点上安装Greenplum软件

准备介质

下载地址：http://gopivotal.com/products/pivotal-greenplum-database

解压

#unzip greenplum-db-4.2.8.0-build-1-RHEL5-x86_64.zip

安装软件

#/bin/bash greenplum-db-4.2.8.0-build-1-RHEL5-x86_64.bin

I HAVE READ AND AGREE TO THE TERMS OF THE ABOVE PIVOTAL SOFTWARE

LICENSE AGREEMENT.

********************************************************************************

Do you accept the Pivotal Database license agreement? [yes|no]

********************************************************************************

yes

********************************************************************************

Provide the installation path for Greenplum Database or press ENTER to

accept the default installation path: /usr/local/greenplum-db-4.2.8.0

********************************************************************************

Install Greenplum Database into </usr/local/greenplum-db-4.2.8.0>? [yes|no]

********************************************************************************

yes

********************************************************************************

/usr/local/greenplum-db-4.2.8.0 does not exist.

Create /usr/local/greenplum-db-4.2.8.0 ? [yes|no]

(Selecting no will exit the installer)

********************************************************************************

yes

********************************************************************************

[Optional] Provide the path to a previous installation of Greenplum Database,

or press ENTER to skip this step. e.g. /usr/local/greenplum-db-4.1.1.3

This installation step will migrate any Greenplum Database extensions from the

provided path to the version currently being installed. This step is optional

and can be run later with:

gppkg --migrate <path_to_old_gphome> /usr/local/greenplum-db-4.2.8.0

********************************************************************************

Extracting product to /usr/local/greenplum-db-4.2.8.0

Skipping migration of Greenplum Database extensions...

********************************************************************************

Installation complete.

Greenplum Database is installed in /usr/local/greenplum-db-4.2.8.0

Pivotal Greenplum documentation is available

for download at http://docs.gopivotal.com/gpdb

********************************************************************************

安装结果

[root@master greenplum-db-4.2.8.0]# pwd

/usr/local/greenplum-db-4.2.8.0

[root@master greenplum-db-4.2.8.0]# ls -ltr

total 328

drwxr-xr-x 3 510 510 4096 Jun 18 2014 share

drwxr-xr-x 2 510 510 4096 Jun 18 2014 demo

drwxr-xr-x 5 510 510 4096 Jun 18 2014 docs

drwxr-xr-x 7 510 510 4096 Jun 18 2014 lib

drwxr-xr-x 6 510 510 4096 Jun 18 2014 include

drwxr-xr-x 3 510 510 4096 Jun 18 2014 ext

drwxr-xr-x 2 510 510 4096 Jun 18 2014 etc

drwxr-xr-x 2 510 510 4096 Jun 18 2014 sbin

-rw-rw-r-- 1 510 510 193083 Jun 18 2014 LICENSE.thirdparty

-rw-rw-r-- 1 510 510 43025 Jun 18 2014 GPDB-LICENSE.txt

drwxr-xr-x 3 510 510 4096 Jun 18 2014 bin

-rw-r--r-- 1 root root 676 Mar 20 19:41 greenplum_path.sh

文件说明：

greenplum_path.sh：Greenplum数据库环境变量文件

GPDB-LICENSE.txt：Greenplum许可协议

bin：管理工具、客户端程序及服务程序

demo：示例程序

docs：帮助文档

etc：OpenSSL的配置示例

ext：一些GP工具使用的捆绑程序

inlcude：C头文件

lib：库文件

sbin：支持的或者内部的脚本和程序

share：共享文件

在所有主机上初始化配置Greenplum

获取环境参数:

# source/usr/local/greenplum-db/greenplum_path.sh

GPHOME=/usr/local/greenplum-db-4.2.8.0

# Replace with symlink path if it is present and correct

if [ -h ${GPHOME}/../greenplum-db ]; then

GPHOME_BY_SYMLINK=`(cd ${GPHOME}/../greenplum-db/ && pwd -P)`

if [ x"${GPHOME_BY_SYMLINK}" = x"${GPHOME}" ]; then

GPHOME=`(cd ${GPHOME}/../greenplum-db/ && pwd -L)`/.

unset GPHOME_BY_SYMLINK

PATH=$GPHOME/bin:$GPHOME/ext/python/bin:$PATH

LD_LIBRARY_PATH=$GPHOME/lib:$GPHOME/ext/python/lib:$LD_LIBRARY_PATH

PYTHONPATH=$GPHOME/lib/python

PYTHONHOME=$GPHOME/ext/python

OPENSSL_CONF=$GPHOME/etc/openssl.cnf

export GPHOME

export PATH

export LD_LIBRARY_PATH

export PYTHONPATH

export PYTHONHOME

export OPENSSL_CONF

创建主机文件all_hosts

文件内容：

master

slave1

slave2

env

[root@master greenplum-db-4.2.8.0]# env

HOSTNAME=master

TERM=vt100

SHELL=/bin/bash

HISTSIZE=1000

SSH_CLIENT=192.168.80.120 49852 22

GPHOME=/usr/local/greenplum-db/.

OLDPWD=/root/installer

SSH_TTY=/dev/pts/2

USER=root

LD_LIBRARY_PATH=/usr/local/greenplum-db/./lib:/usr/local/greenplum-db/./ext/python/lib:

LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:

MAIL=/var/spool/mail/root

PATH=/usr/local/greenplum-db/./bin:/usr/local/greenplum-db/./ext/python/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

INPUTRC=/etc/inputrc

PWD=/usr/local/greenplum-db-4.2.8.0

LANG=en_US.UTF-8

PYTHONHOME=/usr/local/greenplum-db/./ext/python

SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass

SHLVL=1

HOME=/root

OPENSSL_CONF=/usr/local/greenplum-db/./etc/openssl.cnf

PYTHONPATH=/usr/local/greenplum-db/./lib/python

LOGNAME=root

CVS_RSH=ssh

SSH_CONNECTION=192.168.80.120 49852 192.168.80.200 22

LESSOPEN=|/usr/bin/lesspipe.sh %s

G_BROKEN_FILENAMES=1

_=/bin/env

[root@master greenplum-db-4.2.8.0]# ssh slave1

The authenticity of host 'slave1 (192.168.80.201)' can't be established.

RSA key fingerprint is 04:6d:9c:fc:68:e3:9a:24:dc:11:ff:25:14:9e:d1:5b.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'slave1,192.168.80.201' (RSA) to the list of known hosts.

root@slave1's password:

Last login: Fri Mar 20 17:49:51 2015 from 192.168.80.120

运行gpseginstall工具

# gpseginstall -f all_hosts-ugpadmin -p gpadmin

[root@master greenplum-db]# gpseginstall -f all_hosts -u gpadmin -p gpadmin

20150320:19:58:32:003371 gpseginstall:master:root-[INFO]:-Installation Info:

link_name greenplum-db

binary_path /usr/local/greenplum-db-4.2.8.0

binary_dir_location /usr/local

binary_dir_name greenplum-db-4.2.8.0

20150320:19:58:32:003371 gpseginstall:master:root-[INFO]:-check cluster password access

*** Enter password for slave1:

20150320:19:58:38:003371 gpseginstall:master:root-[INFO]:-de-duplicate hostnames

20150320:19:58:38:003371 gpseginstall:master:root-[INFO]:-master hostname: master

20150320:19:58:38:003371 gpseginstall:master:root-[INFO]:-check for user gpadmin on cluster

20150320:19:58:38:003371 gpseginstall:master:root-[INFO]:-add user gpadmin on master

20150320:19:58:41:003371 gpseginstall:master:root-[INFO]:-add user gpadmin on cluster

20150320:19:58:42:003371 gpseginstall:master:root-[INFO]:-chown -R gpadmin:gpadmin /usr/local/greenplum-db

20150320:19:58:43:003371 gpseginstall:master:root-[INFO]:-chown -R gpadmin:gpadmin /usr/local/greenplum-db-4.2.8.0

20150320:19:58:43:003371 gpseginstall:master:root-[INFO]:-rm -f /usr/local/greenplum-db-4.2.8.0.tar; rm -f /usr/local/greenplum-db-4.2.8.0.tar.gz

20150320:19:58:43:003371 gpseginstall:master:root-[INFO]:-cd /usr/local; tar cf greenplum-db-4.2.8.0.tar greenplum-db-4.2.8.0

20150320:19:58:50:003371 gpseginstall:master:root-[INFO]:-gzip /usr/local/greenplum-db-4.2.8.0.tar

20150320:19:59:04:003371 gpseginstall:master:root-[INFO]:-remote command: mkdir -p /usr/local

20150320:19:59:04:003371 gpseginstall:master:root-[INFO]:-remote command: rm -rf /usr/local/greenplum-db-4.2.8.0

20150320:19:59:04:003371 gpseginstall:master:root-[INFO]:-scp software to remote location

20150320:19:59:34:003371 gpseginstall:master:root-[INFO]:-remote command: gzip -f -d /usr/local/greenplum-db-4.2.8.0.tar.gz

20150320:19:59:39:003371 gpseginstall:master:root-[INFO]:-md5 check on remote location

20150320:19:59:41:003371 gpseginstall:master:root-[INFO]:-remote command: cd /usr/local; tar xf greenplum-db-4.2.8.0.tar

20150320:19:59:59:003371 gpseginstall:master:root-[INFO]:-remote command: rm -f /usr/local/greenplum-db-4.2.8.0.tar

20150320:20:00:01:003371 gpseginstall:master:root-[INFO]:-remote command: cd /usr/local; rm -f greenplum-db; ln -fs greenplum-db-4.2.8.0 greenplum-db

20150320:20:00:01:003371 gpseginstall:master:root-[INFO]:-remote command: chown -R gpadmin:gpadmin /usr/local/greenplum-db

20150320:20:00:02:003371 gpseginstall:master:root-[INFO]:-remote command: chown -R gpadmin:gpadmin /usr/local/greenplum-db-4.2.8.0

20150320:20:00:02:003371 gpseginstall:master:root-[INFO]:-rm -f /usr/local/greenplum-db-4.2.8.0.tar.gz

20150320:20:00:02:003371 gpseginstall:master:root-[INFO]:-Changing system passwords ...

20150320:20:00:05:003371 gpseginstall:master:root-[INFO]:-exchange ssh keys for user root

20150320:20:00:16:003371 gpseginstall:master:root-[INFO]:-exchange ssh keys for user gpadmin

20150320:20:00:18:003371 gpseginstall:master:root-[INFO]:-/usr/local/greenplum-db/./sbin/gpfixuserlimts -f /etc/security/limits.conf -u gpadmin

20150320:20:00:20:003371 gpseginstall:master:root-[INFO]:-remote command: . /usr/local/greenplum-db/./greenplum_path.sh; /usr/local/greenplum-db/./sbin/gpfixuserlimts -f /etc/security/limits.conf -u gpadmin

20150320:20:00:21:003371 gpseginstall:master:root-[INFO]:-version string on master: gpssh version 4.2.8.0 build 1

20150320:20:00:21:003371 gpseginstall:master:root-[INFO]:-remote command: . /usr/local/greenplum-db/./greenplum_path.sh; /usr/local/greenplum-db/./bin/gpssh --version

20150320:20:00:21:003371 gpseginstall:master:root-[INFO]:-remote command: . /usr/local/greenplum-db-4.2.8.0/greenplum_path.sh; /usr/local/greenplum-db-4.2.8.0/bin/gpssh --version

20150320:20:00:27:003371 gpseginstall:master:root-[INFO]:-SUCCESS -- Requested commands completed

在所有主机上初始化配置Greenplum

切换到gpamdin用户并获取环境变量

$ su - gpadmin
# source/usr/local/greenplum-db/greenplum_path.sh

使用gpssh工具来测试无密码登录所有主机

$ gpssh -f host_list -e ls -l$GPHOME

[gpadmin@master greenplum-db]$ gpssh -f all_hosts -e ls -l $GPHOME

[master] ls -l /usr/local/greenplum-db/.

[master] total 336

......

[slave1] ls -l /usr/local/greenplum-db/.

[slave1] total 336

......

[slave2] ls -l /usr/local/greenplum-db/.

[slave2] total 336

......

将"./usr/local/greenplum-db-4.2.2.4/greenplum_path.sh"添加到.bashrc文件最后并传送到子服务器

[gpadmin@master ~]$ cat .bashrc

# .bashrc

# Source global definitions

if [ -f /etc/bashrc ]; then

. /etc/bashrc

. /usr/local/greenplum-db/greenplum_path.sh

# User specific aliases and functions

[gpadmin@master ~]$ env

HOSTNAME=master

SHELL=/bin/bash

TERM=vt100

HISTSIZE=1000

GPHOME=/usr/local/greenplum-db/.

USER=gpadmin

LD_LIBRARY_PATH=/usr/local/greenplum-db/./lib:/usr/local/greenplum-db/./ext/python/lib:

MAIL=/var/spool/mail/gpadmin

PATH=/usr/local/greenplum-db/./bin:/usr/local/greenplum-db/./ext/python/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/gpadmin/bin

INPUTRC=/etc/inputrc

PWD=/home/gpadmin

LANG=en_US.UTF-8

PYTHONHOME=/usr/local/greenplum-db/./ext/python

SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass

SHLVL=1

HOME=/home/gpadmin

OPENSSL_CONF=/usr/local/greenplum-db/./etc/openssl.cnf

PYTHONPATH=/usr/local/greenplum-db/./lib/python

LOGNAME=gpadmin

CVS_RSH=ssh

LESSOPEN=|/usr/bin/lesspipe.sh %s

G_BROKEN_FILENAMES=1

_=/bin/env

[gpadmin@master ~]$ scp .bashrc slave1:`pwd`

.bashrc 100% 167 0.2KB/s 00:00

[gpadmin@master ~]$ scp .bashrc slave2:`pwd`

.bashrc 100% 167 0.2KB/s 00:00

验证操作系统设置

创建存储区域

a) 创建Master数据存储区域

# mkdir -p /data/master

b) 改变目录的所有权

# chown gpadmin /data/master

c) 创建一个包含所有segment主机的文件seg_hosts

[root@master ~]# mkdir /tmp/greenplum

[root@master ~]# cd /tmp/greenplum/

[root@master greenplum]# vi seg_hosts

slave1

slave2

"seg_hosts" [New] 2L, 14C written

[root@master greenplum]# cat seg_hosts

slave1

slave2

d) 使用gpssh工具在所有segment主机上创建主数据和镜像数据目录

[root@master greenplum]# . /usr/local/greenplum-db/greenplum_path.sh

[root@master greenplum]# gpssh -f seg_hosts -e 'mkdir -p /data/primary'

[slave1] mkdir -p /data/primary

[slave2] mkdir -p /data/primary

[root@master greenplum]# gpssh -f seg_hosts -e 'mkdir -p /data/mirror'

[slave1] mkdir -p /data/mirror

[slave2] mkdir -p /data/mirror

[root@master greenplum]# gpssh -f seg_hosts -e 'chown gpadmin /data/primary'

[slave1] chown gpadmin /data/primary

[slave2] chown gpadmin /data/primary

[root@master greenplum]# gpssh -f seg_hosts -e 'chown gpadmin /data/mirror'

[slave1] chown gpadmin /data/mirror

[slave2] chown gpadmin /data/mirror

同步系统时间

a) 在Master主机上编辑/etc/ntp.conf来设置如下内容：

[root@master greenplum]# cat /etc/ntp.conf

# Permit time synchronization with our time source, but do not

# permit the source to query or modify the service on this system.

restrict default kod nomodify notrap nopeer noquery

restrict -6 default kod nomodify notrap nopeer noquery

# Permit all access over the loopback interface. This could

# be tightened as well, but to do so would effect some of

# the administrative functions.

restrict 127.0.0.1

restrict -6 ::1

# Hosts on local network are less restricted.

#restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap

# Use public servers from the pool.ntp.org project.

# Please consider joining the pool (http://www.pool.ntp.org/join.html).

server 0.rhel.pool.ntp.org

server 1.rhel.pool.ntp.org

server 2.rhel.pool.ntp.org

#broadcast 192.168.1.255 key 42 # broadcast server

#broadcastclient # broadcast client

#broadcast 224.0.1.1 key 42 # multicast server

#multicastclient 224.0.1.1 # multicast client

#manycastserver 239.255.254.254 # manycast server

#manycastclient 239.255.254.254 key 42 # manycast client

# Undisciplined Local Clock. This is a fake driver intended for backup

# and when no outside source of synchronized time is available.

server 127.127.1.0 # local clock

fudge 127.127.1.0 stratum 10

# Drift file. Put this in a directory which the daemon can write to.

# No symbolic links allowed, either, since the daemon updates the file

# by creating a temporary in the same directory and then rename()'ing

# it to the file.

driftfile /var/lib/ntp/drift

# Key file containing the keys and key identifiers used when operating

# with symmetric key cryptography.

keys /etc/ntp/keys

# Specify the key identifiers which are trusted.

#trustedkey 4 8 42

# Specify the key identifier to use with the ntpdc utility.

#requestkey 8

# Specify the key identifier to use with the ntpq utility.

#controlkey 8

b) 在Segment主机上编辑/etc/ntp.conf

# Undisciplined Local Clock. This is a fake driver intended for backup

# and when no outside source of synchronized time is available.

#server 127.127.1.0 # local clock

server master # local clock

fudge 127.127.1.0 stratum 10

c) 在Master主机上，通过NTP守护进程同步系统时钟

[root@master greenplum]# gpssh -f all_hosts -v -e 'ntpd'

[Reset ...]

[INFO] login master

[INFO] login slave1

[INFO] login slave2

[master] ntpd

[slave1] ntpd

[slave2] ntpd

[INFO] completed successfully

[Cleanup...]

验证操作系统设置

[root@master greenplum]# gpcheck -f all_hosts -m master

.......

20150320:21:20:17:004588 gpcheck:master:root-[INFO]:-gpcheck completing...

[root@master greenplum]# gpcheck -f all_hosts -m master

......

20150320:21:23:52:004727 gpcheck:master:root-[INFO]:-gpcheck completing...

[root@master greenplum]# echo deadline > /sys/block/sr0/queue/scheduler

[root@master greenplum]# gpcheck -f all_hosts -m master

......

20150320:21:27:50:004866 gpcheck:master:root-[INFO]:-gpcheck completing...

创建Greenplum数据库配置文件
初始化Greenplum数据库系统

a) 以gpadmin用户登录
# su - gpadmin
b) 从模板中拷贝一份gpinitsystem_config文件

[gpadmin@master ~]$ cp $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/

[gpadmin@master ~]$ chmod 775 /home/gpadmin/gpinitsystem_config

c) 设置所有必须的参数以及可选参数

[gpadmin@master ~]$ vi gpinitsystem_config

# FILE NAME: gpinitsystem_config

# Configuration file needed by the gpinitsystem

################################################

#### REQUIRED PARAMETERS

################################################

#### Name of this Greenplum system enclosed in quotes.

ARRAY_NAME="EMC Greenplum DW"

#### Naming convention for utility-generated data directories.

SEG_PREFIX=gpseg

#### Base number by which primary segment port numbers

#### are calculated.

PORT_BASE=40000

declare -a DATA_DIRECTORY=(/data/primary )

#### OS-configured hostname or IP address of the master host.

MASTER_HOSTNAME=master

#### File system location where the master data directory

#### will be created.

MASTER_DIRECTORY=/data/master

#### Port number for the master instance.

MASTER_PORT=5432

#### Shell utility used to connect to remote hosts.

TRUSTED_SHELL=ssh

#### Maximum log file segments between automatic WAL checkpoints.

CHECK_POINT_SEGMENTS=8

#### Default server-side character set encoding.

ENCODING=UNICODE

################################################

#### OPTIONAL MIRROR PARAMETERS

################################################

#### Base number by which mirror segment port numbers

#### are calculated.

MIRROR_PORT_BASE=50000

#### Base number by which primary file replication port

#### numbers are calculated.

REPLICATION_PORT_BASE=41000

#### Base number by which mirror file replication port

#### numbers are calculated.

MIRROR_REPLICATION_PORT_BASE=51000

#### File system location(s) where mirror segment data directories

#### will be created. The number of mirror locations must equal the

#### number of primary locations as specified in the

#### DATA_DIRECTORY parameter.

declare -a MIRROR_DATA_DIRECTORY=(/data/mirror)

################################################

#### OTHER OPTIONAL PARAMETERS

################################################

#### Create a database of this name after initialization.

#DATABASE_NAME=name_of_database

#### Specify the location of the host address file here instead of

#### with the the -h option of gpinitsystem.

#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem

初始化数据库

a) 运行初始化工具

[gpadmin@master ~]$ cp /tmp/greenplum/seg_hosts .

[gpadmin@master ~]$ ls

gpinitsystem_config seg_hosts

[gpadmin@master ~]$ gpinitsystem -c gpinitsystem_config -h seg_hosts

20150320:21:53:15:005103 gpinitsystem:master:gpadmin-[FATAL]:-Errors generated from parallel processes

20150320:21:53:15:005103 gpinitsystem:master:gpadmin-[INFO]:-Dumped contents of status file to the log file

20150320:21:53:15:005103 gpinitsystem:master:gpadmin-[INFO]:-Building composite backout file

20150320:21:53:15:gpinitsystem:master:gpadmin-[FATAL]:-Failures detected, see log file /home/gpadmin/gpAdminLogs/gpinitsystem_20150320.log for more detail Script Exiting!

20150320:21:53:15:005103 gpinitsystem:master:gpadmin-[WARN]:-Script has left Greenplum Database in an incomplete state

20150320:21:53:15:005103 gpinitsystem:master:gpadmin-[WARN]:-Run command /bin/bash /home/gpadmin/gpAdminLogs/backout_gpinitsystem_gpadmin_20150320_215158 to remove these changes

20150320:21:53:15:005103 gpinitsystem:master:gpadmin-[INFO]:-Start Function BACKOUT_COMMAND

20150320:21:53:15:005103 gpinitsystem:master:gpadmin-[INFO]:-End Function BACKOUT_COMMAND

经过重新同步各服务器之间的时间后初始化正确：

[gpadmin@master ~]$ gpinitsystem -c gpinitsystem_config -h seg_hosts

20150321:14:23:03:032514 gpstart:master:gpadmin-[INFO]:-----------------------------------------------------

20150321:14:23:03:032514 gpstart:master:gpadmin-[INFO]:-Successfully started 2 of 2 segment instances

20150321:14:23:03:032514 gpstart:master:gpadmin-[INFO]:-----------------------------------------------------

20150321:14:23:03:032514 gpstart:master:gpadmin-[INFO]:-Starting Master instance master directory /data/master/gpseg-1

20150321:14:23:04:032514 gpstart:master:gpadmin-[INFO]:-Command pg_ctl reports Master master instance active

当出现以下信息时初始化成功

[gpadmin@master ~]$ ps -ef|grep post

gpadmin 1444 15784 0 14:27 pts/1 00:00:00 grep post

gpadmin 32570 1 0 14:23 ? 00:00:00 /usr/local/greenplum-db-4.2.8.0/bin/postgres -D /data/master/gpseg-1 -p 5432 -b 1 -z 2 --silent-mode=true -i -M master -C -1 -x 0 -E

gpadmin 32571 32570 0 14:23 ? 00:00:00 postgres: port 5432, master logger process

gpadmin 32574 32570 0 14:23 ? 00:00:00 postgres: port 5432, stats collector process

gpadmin 32575 32570 0 14:23 ? 00:00:00 postgres: port 5432, writer process

gpadmin 32576 32570 0 14:23 ? 00:00:00 postgres: port 5432, checkpoint process

gpadmin 32577 32570 0 14:23 ? 00:00:00 postgres: port 5432, seqserver process

gpadmin 32578 32570 0 14:23 ? 00:00:00 postgres: port 5432, WAL Send Server process

gpadmin 32579 32570 0 14:23 ? 00:00:00 postgres: port 5432, ftsprobe process

gpadmin 32580 32570 0 14:23 ? 00:00:00 postgres: port 5432, sweeper process

b) 设置环境变量
添加“export MASTER_DATA_DIRECTORY=/data/master/gpseg-1”到~/.bashrc文件尾，并同步到其他节点。

[gpadmin@master ~]$ cat .bashrc

# .bashrc

# Source global definitions

if [ -f /etc/bashrc ]; then

. /etc/bashrc

. /usr/local/greenplum-db/greenplum_path.sh

export MASTER_DATA_DIRECTORY=/data/master/gpseg-1

# User specific aliases and functions

[gpadmin@master ~]$ scp .bashrc slave1:`pwd`

.bashrc 100% 217 0.2KB/s 00:00

[gpadmin@master ~]$ scp .bashrc slave2:`pwd`

.bashrc 100% 217 0.2KB/s 00:00

启动和停止数据库

a) 启动数据库 $ gpstart

[gpadmin@master ~]$ gpstart

20150321:14:49:18:007507 gpstart:master:gpadmin-[INFO]:-Starting gpstart with args:

.......

20150321:14:49:26:007507 gpstart:master:gpadmin-[INFO]:-Database successfully started

[gpadmin@master ~]$ psql -d template1

psql (8.2.15)

Type "help" for help.

template1=#

注意：需要检查防火墙是否关闭否则，会有错误。
b) 关闭数据库
$ gpstop

[gpadmin@master ~]$ gpstop

20150321:14:38:08:004265 gpstop:master:gpadmin-[INFO]:-Starting gpstop with args:

Continue with Greenplum instance shutdown Yy|Nn (default=N):

> y

20150321:14:38:14:004265 gpstop:master:gpadmin-[INFO]:-Successfully shutdown 2 of 2 segment instances

20150321:14:38:14:004265 gpstop:master:gpadmin-[INFO]:-Database successfully shutdown with no errors reported

来源：oschina

链接：https://my.oschina.net/u/4402583/blog/4486349

标签

Greenplum DB

Greenplum安装

Greenplum安装

评估存储能力

计算可用的空间

计算用户数据大小

实验环境

硬件环境

网卡设置

虚拟机配置

基本信息配置

设置IP地址的命令行方式

系统参数配置

系统设置（所有节点）

在Master节点上安装Greenplum软件

准备介质

解压

安装软件

安装结果

文件说明：

在所有主机上初始化配置Greenplum

获取环境参数:

创建主机文件all_hosts

运行gpseginstall工具

在所有主机上初始化配置Greenplum

切换到gpamdin用户并获取环境变量

使用gpssh工具来测试无密码登录所有主机

将"./usr/local/greenplum-db-4.2.2.4/greenplum_path.sh"添加到.bashrc文件最后并传送到子服务器

验证操作系统设置

创建存储区域

同步系统时间

验证操作系统设置

创建Greenplum数据库配置文件 初始化Greenplum数据库系统

初始化数据库

启动和停止数据库

创建Greenplum数据库配置文件
初始化Greenplum数据库系统