Greenplum部署-5.增加segment节点

作者: sysit 分类: d 发表于 2018-12-27 210人围观

## 5.1 增加说明
* （1）如果master配置了standby master，添加的segment必须要有mirror segment存在，否则报错。
* （2）按照常规操作，必须增加至少两个主机

## 5.2 segment节点初始配置

### 5.2.1 基础环境配置
* 参见Greenplum基础环境配置部分
* 注意其他主机增加对应的hosts

### 5.2.2 gporca编译

* 我实际测试过程中，发现需要调用gporca编译的库文件，建议一步一步编译gporca
* 编译参照编译安装文档。

## 5.3 新增segment节点环境准备

### 5.3.1 添加主机信任

```
# 创建新节点的host文件
vi /opt/gpinit/new_segment
gpdb5.sysit.cn
gpdb6.sysit.cn

#root操作,加载path
su - 
source /usr/local/gpdb/greenplum_path.sh

# 添加主机信任
#gpssh-exkeys -e hostfile_exkeys -x hostfile_gpexpand
# -e 指定已有主机名文件，包含所有配置的主机名和主机地址（接口名）
# -x 包含所有新增的主机名和主机地址（接口名）对于要添加到Greenplum系统的每个新段主机
# 中间需要输入root的密码

gpssh-exkeys -e /opt/gpinit/all_host -x /opt/gpinit/new_segment

# 创建gpadmin用户
gpssh -f /opt/gpinit/new_segment '/usr/sbin/useradd gpadmin -d /home/gpadmin -s /bin/bash'

# 设置gpadmin密码
gpssh -f /opt/gpinit/new_segment 'echo gpadmin_password | passwd gpadmin --stdin'

# 查看创建的用户目录
gpssh -f /opt/gpinit/new_segment ls -l /home

#创建gpadmin用户的信任（同root）
su - gpadmin
gpssh-exkeys -e /opt/gpinit/all_host -x /opt/gpinit/new_segment
```

### 5.3.2 环境验证

* 检查操作系统
```
su - gpadmin
gpcheck -f /opt/gpinit/new_segment
```

* 检查磁盘IO
```
gpcheckperf -f /opt/gpinit/new_segment -d /data1 -d /data2 -v 
```

* 网络检查
```
gpcheckperf -f /opt/gpinit/new_segment _ic1 -r N -d /tmp > subnet1.out
```

### 5.3.3 初始化segments节点

* 安装gpdb

```
# 将/usr/local/gpdb安装到new_segment节点。
gpseginstall -f /opt/gpinit/new_segment -u gpadmin -p gpadminpassword

# 切换到gpadmin用户
su - gpadmin
#加载path文件
source /usr/local/gpdb/greenplum_path.sh

# 使用gpssh验证是否可以免密登录new_segment，并确认gpdb是否在每一台服务器上安装成功
gpssh -f /opt/gpinit/new_segment -e ls -l $GPHOME

# 如果不能免密登录，执行一下命令。
gpssh-exkeys -f /opt/gpinit/new_segment
```

* 创建目录

> 服务器事先已经挂载好目录则忽略此步骤
> 服务器也可手动创建目录
> gpexpand也会自动创建目录（前提是有权限）
```
# segment primary
gpssh -f /opt/gpinit/new_segment -e 'mkdir -p /gpdisk/primary' 
gpssh -f /opt/gpinit/new_segment -e 'chown gpadmin:gpadmin /gpdisk/primary'

# segment mirror
gpssh -f /opt/gpinit/snew_segment -e 'mkdir -p /gpdisk/mirror' 
gpssh -f /opt/gpinit/new_segment -e 'chown gpadmin:gpadmin /gpdisk/mirror'

```

## 5.4 使用gpexpand增加节点

### 5.4.1 生成扩展文件
```
[gpadmin@gpdb1 ~]$ gpexpand -f /opt/gpinit/new_segment 
20181227:15:23:03:031099 gpexpand:gpdb1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.0.0-alpha.0+dev.13528.g1a9ca69 build dev-oss'
20181227:15:23:04:031099 gpexpand:gpdb1:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4beta1 (Greenplum Database 6.0.0-alpha.0+dev.13528.g1a9ca69 build dev-oss) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.3.1 20170216 (Red Hat 6.3.1-3), 64-bit compiled on Dec 24 2018 15:23:43'
20181227:15:23:05:031099 gpexpand:gpdb1:gpadmin-[INFO]:-Querying gpexpand schema for current expansion state

System Expansion is used to add segments to an existing GPDB array.
gpexpand did not detect a System Expansion that is in progress.

Before initiating a System Expansion, you need to provision and burn-in
the new hardware.  Please be sure to run gpcheckperf to make sure the
new hardware is working properly.

Please refer to the Admin Guide for more information.

Would you like to initiate a new System Expansion Yy|Nn (default=N):
> y

You must now specify a mirroring strategy for the new hosts.  Spread mirroring places
a given hosts mirrored segments each on a separate host.  You must be 
adding more hosts than the number of segments per host to use this. 
Grouped mirroring places all of a given hosts segments on a single 
mirrored host.  You must be adding at least 2 hosts in order to use this.

What type of mirroring strategy would you like?
 spread|grouped (default=grouped):
>

By default, new hosts are configured with the same number of primary
    segments as existing hosts.  Optionally, you can increase the number
    of segments per host.

For example, if existing hosts have two primary segments, entering a value
    of 2 will initialize two additional segments on existing hosts, and four
    segments on new hosts.  In addition, mirror segments will be added for
    these new primary segments if mirroring is enabled.

How many new primary segments per host do you want to add? (default=0):
> 2
Enter new primary data directory 1:
> /gpdisk/primary
Enter new primary data directory 2:
> /gpdisk/primary
Enter new mirror data directory 1:
> /gpdisk/mirror
Enter new mirror data directory 2:
> /gpdisk/mirror

Generating configuration file...

20181227:15:26:04:031099 gpexpand:gpdb1:gpadmin-[INFO]:-Generating input file...

Input configuration file was written to 'gpexpand_inputfile_20181227_152604'.
Please review the file and make sure that it is correct then re-run
with: gpexpand -i gpexpand_inputfile_20181227_152604 -D testgpdb
                
20181227:15:26:04:031099 gpexpand:gpdb1:gpadmin-[INFO]:-Exiting...

# 以上生成了gpexpand文件gpexpand_inputfile_20181227_152604
# testgpdb是我的默认库，在环境变量中自动调用，如果不清楚，见编译安装文档。
```

### 5.4.2 初始化segments节点并扩展schema
```
#gpexpand -i input_file -D database1

gpexpand -i gpexpand_inputfile_20181227_152604 -D testgpdb

```

### 5.4.3 重新分发表
```
#节点添加成功，但数据没有重新分布。
#gpexpand -d duration

gpexpand -d 60:00:00
```

### 5.4.4 清理expansion schema
```
#expansion schema 存储的是系统中每个表的元数据，以便对每个扩展的整个过程的状态进行跟踪，expansion schema由2张表和1个视图组成
#清除重分布产生的schema

gpexpand -c
```

### 5.4.5 初始化失败回滚的方法
```
#gpexpand --rollback -D database_name

gpexpand -r -D testgpdb
```

如果觉得我的文章对您有用，请随意赞赏。您的支持将鼓励我继续创作！