原創文章,轉載須註明出處。訪問我的Github(地址:https://guobo507.github.io)查看最新文章列表。
目錄
環境說明
測試環境,一共4個節點,兩個用作gtm,另外兩個用作gtm_proxy/coordinator和datanode。
具體安裝不說,參考相關文檔即可。
環境規劃如下所示:
目錄規劃:
主機名 | 目錄 |
---|---|
pgxc-lvs | - |
pgxc-gtm1 | /data/gtm |
pgxc-gtm2 | /data/gtm |
pgxc-n1 | /data/coord, /data/data, /data/coord_s, /data/data_s, /data/gtm_proxy |
pgxc-n2 | /data/coord, /data/data, /data/coord_s, /data/data_s, /data/gtm_proxy |
端口規劃:
主機名/ROLE | 端口 |
---|---|
pgxc-lvs | 5432(DB) |
pgxc-gtm1 | 6666 |
pgxc-gtm2 | 6666 |
gtm_proxy1 | 6667 |
gtm_proxy2 | 6667 |
coord1/coord1_pooler Port | 6611/6612 |
coord2/coord2_pooler Port | 6611/6612 |
coord1_slave/coord1_slave_pooler Port | 6621/6622 |
coord2_slave/coord2_slave_pooler Port | 6621/6622 |
data1/data1_pooler Port | 6631/6632 |
data2/data2_pooler Port | 6631/6632 |
data1_slave/data1_slave_pooler Port | 6641/6642 |
data1_slave/data1_slave_pooler Port | 6641/6642 |
軟件版本
我是用的是Postgres-XL提供的版本,所有節點都安裝相同版本的軟件包,如下所示爲其中一個節點:
[pgxc@pgxc-gtm1 ~]$ rpm -qa |grep postgres
postgres-xl95-libs-9.5-1.6.el7.x86_64
postgres-xl95-gtm-9.5-1.6.el7.x86_64
postgres-xl95-contrib-9.5-1.6.el7.x86_64
postgres-xl95-test-9.5-1.6.el7.x86_64
postgres-xl95-9.5-1.6.el7.x86_64
postgres-xl95-server-9.5-1.6.el7.x86_64
postgres-xl95-devel-9.5-1.6.el7.x86_64
說明:官方手冊是使用源碼編譯安裝,懶得用源碼編譯,所以提前用源碼編譯了RPM包,一勞永逸。關於如何創建RPM安裝包,請參考我的Github項目:https://github.com/guobo507/postgres-xl-rpmbuild。同理,如果你想使用postgres-xc版本並自己打包RPM也是可以的,參考https://github.com/guobo507/postgres-xc-rpm。
環境變量
所有節點配置相同的環境變量設置,如下:
[pgxc@pgxc-gtm1 ~]$ tail -n4 ~/.bashrc
export PGHOME=/usr/postgres-xl-9.5
export export PGUSER=pgxc
export LD_LIBRARY_PATH=$PGHOME/lib:$LD_LIBRARY_PATH
export PATH=$PGHOME/bin:$PATH
操作過程
生成模版配置文件
選取任一節點,如pgxc-gtm1節點上操作:
$ ssh [email protected]
$ pgxc_ctl
$ pgxc_ctl
/usr/bin/bash
Installing pgxc_ctl_bash script as /home/pgxc/pgxc_ctl/pgxc_ctl_bash.
ERROR: File "/home/pgxc/pgxc_ctl/pgxc_ctl.conf" not found or not a regular file. No such file or directory
Installing pgxc_ctl_bash script as /home/pgxc/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /home/pgxc/pgxc_ctl/pgxc_ctl_bash --home /home/pgxc/pgxc_ctl --configuration /home/pgxc/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
******** PGXC_CTL START ***************
Current directory: /home/pgxc/pgxc_ctl
PGXC prepare config empty # 生成空的配置文件模板
PGXC exit # 退出,也可以執行`quit`。
往集羣中添加節點
配置gtm節點(master/standby)
pgxc_ctl
add gtm master gtm1 pgxc-gtm1 6666 /data/gtm
add gtm slave gtm2 pgxc-gtm2 6666 /data/gtm
monitor all
quit
設置附加配置文件
生成附加配置文件,這兩個文件的內容分別需要添加到各節點實例的postgresql.conf和pg_hba.conf文件中:
cat > ~/pgxc_ctl/coordExtraConfig.conf <<EOF
#================================================
# Added to all the coordinator postgresql.conf
# Original: \$coordExtraConfig
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
listen_addresses = '*'
max_connections = 256
EOF
cat > ~/pgxc_ctl/coordExtraPgHba.conf <<EOF
#================================================
# Added to all the coordinator master's pg_hba.conf
# Original: \$coordExtraPgHba
host all all 10.128.0.30/32 trust
host all all 10.128.0.31/32 trust
host all all 10.128.0.32/32 trust
host all all 10.128.0.33/32 trust
host all all 10.128.0.34/32 trust
host all all 0.0.0.0/0 md5
EOF
添加協調節點(Master/Slave)
pgxc_ctl
add coordinator master coord1 pgxc-n1 6611 6612 /data/coord coordExtraConfig.conf coordExtraPgHba.conf
add coordinator master coord2 pgxc-n2 6611 6612 /data/coord coordExtraConfig.conf coordExtraPgHba.conf
add coordinator slave coord1 pgxc-n2 6621 6622 /data/coord_s /data/coord_s_arc
add coordinator slave coord2 pgxc-n1 6621 6622 /data/coord_s /data/coord_s_arc
monitor coordinator all
exit
Check backgroud coordinator processes
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i "ls -ld /data/coord*" && echo; done
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i ps -ef |grep -v grep |grep coordinator && echo; done
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i ps -ef |grep -v grep |grep -E '(coordinator|sender|receiver)' && echo; done
添加數據節點(Master/Slave)
pgxc_ctl
add datanode master data1 pgxc-n1 6631 6632 /data/data none coordExtraConfig.conf coordExtraPgHba.conf
add datanode master data2 pgxc-n2 6631 6632 /data/data none coordExtraConfig.conf coordExtraPgHba.conf
add datanode slave data1 pgxc-n2 6641 6642 /data/data_s none /data/data_s_arc
add datanode slave data2 pgxc-n1 6641 6642 /data/data_s none /data/data_s_arc
monitor datanode all
exit
Check backgroud datanode processes
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i "ls -ld /data/data*" && echo; done
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i ps -ef |grep -v grep |grep datanode && echo; done
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i ps -ef |grep -v grep |grep -E '(datanode|sender|receiver)' && echo; done
添加GTM proxy節點
首先將GTM proxy節點添加到集羣中:
pgxc_ctl
add gtm_proxy gtm_proxy1 pgxc-n1 6667 /data/gtm_proxy
add gtm_proxy gtm_proxy2 pgxc-n2 6667 /data/gtm_proxy
quit
默認添加後可能起不來,但集羣會註冊這兩個節點信息。
我們需要分別連接到pgxc-n1和pgxc-n2節點上修改配置文件,然後啓動GTM proxy。
首先,配置pgxc-n1上的GTM proxy,如下所示:
ssh pgxc-n1
vi /data/gtm_proxy/gtm_proxy.conf # 根據需要修改參數配置。
參數配置如下所示:
nodename = 'gtm_proxy1'
listen_addresses = '*'
port = 6667
worker_threads = 2
gtm_host = 'pgxc-gtm1'
gtm_port = 6667
gtm_connect_retry_interval = 1
log_file = 'gtm_proxy.log'
log_min_messages = INFO
啓動GTM proxy:
gtm_proxy -h pgxc-n1 -p 6667 -s pgxc-gtm1 -t 6666 -n 2 -D /data/gtm_proxy/ -l /data/gtm_proxy/gtm_proxy.log &
exit
然後,對pgxc-n2節點執行類似步驟來啓動該節點上的GTM proxy:
ssh pgxc-n2
vi /data/gtm_proxy/gtm_proxy.conf # 根據需要修改參數配置。
gtm_proxy -h pgxc-n2 -p 6667 -s pgxc-gtm1 -t 6666 -n 2 -D /data/gtm_proxy/ -l /data/gtm_proxy/gtm_proxy.log &
exit
然後,回到pgxc_ctl命令行中,觀察所有節點狀態:
[pgxc@pgxc-gtm1 ~]$ pgxc_ctl
/bin/bash
Installing pgxc_ctl_bash script as /home/pgxc/pgxc_ctl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /home/pgxc/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /home/pgxc/pgxc_ctl/pgxc_ctl_bash --home /home/pgxc/pgxc_ctl --configuration /home/pgxc/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
******** PGXC_CTL START ***************
Current directory: /home/pgxc/pgxc_ctl
PGXC monitor all
Running: gtm master
Running: gtm slave
Running: gtm proxy gtm_proxy1
Running: gtm proxy gtm_proxy2
Running: coordinator master coord1
Running: coordinator slave coord1
Running: coordinator master coord2
Running: coordinator slave coord2
Running: datanode master data1
Running: datanode slave data1
Running: datanode master data2
Running: datanode slave data2
PGXC
pgxc_ctl使用幫助
pgxc_ctl
工具也可以執行節點刪除等工作,使用help COMMAND
查看幫助信息,如下所示:
PGXC help
You are using pgxc_ctl, the configuration utility for PGXL
Type:
help <command>
where <command> is either add, Createdb, Createuser, clean,
configure, deploy, failover, init, kill, log, monitor,
prepare, q, reconnect, remove, set, show, start,
stop or unregister
PGXC help remove
remove gtm slave
remove gtm_proxy nodename [ clean ]
remove coordinator [ master| slave ] nodename [ clean ]
remove datanode [ master| slave ] nodename [ clean ]
Removes the specified node from the cluster
For more details, please see the pgxc_ctl documentation
PGXC