GreenPlum 4.3.2.0 installed in CentOS 6.x x64
背景
greenplum从开源的bizgres发展而来, 早些年被EMC收购, 更名为Pivotal Greenplum Database.
有兴趣的朋友可以到我的分享下载bizgres的源码.
http://pan.baidu.com/s/13XU8m
2008年第一次接触GreenPlum的时候, 性能让人眼前一亮, 后来盛大的兄弟来我们单位交流, 我们用6台普通X86机器搭建的GP测试环境, 已经超过数倍盛大用的顶配IBM P570+EMC+ORACLE RAC的性能. 不过gp的底层用的PG版本太老, 还是8.2的版本, 很多PG后期的新特性都没有.
最近又把GP翻出来了, 因为准备做几个测试的DEMO, 见 http://dba.sky-mobi.com/demo/db/
包含 PostgreSQL, Postgres-XL, GreenPlum. 用来对比一些性能指标.
Greenplum for redhat linux目前包含如下组件.
Greenplum Database 4.3.2.0 for Red Hat Enterprise Linux 5 and 6 4.3.2.0
Clients for Red Hat Enterprise Linux x86_64 (RHEL5 and RHEL6) 4.3.2.0
Greenplum Database - Command Center 1.2.2.3 1.2.2.3
Loaders for Red Hat Enterprise Linux x86_64 (RHEL5 and RHEL6) 4.3.2.0
Connectivity for Red Hat Enterprise Linux x86_64 (RHEL5 and RHEL6) 4.3.2.0
Greenplum Database 4.3 - Partner Connector for RHEL 1.2
MADLib 1.6 - GPDB 4.3 MADLib 1.6
Greenplum Database 4.3 - PL/Perl Extension for RHEL6 5.12.4_pv1.2
Greenplum Database 4.3 - PL/Java Extension for RHEL 1.4.0_pv1.1
Greenplum Database 4.3 - PL/R Extension for RHEL 8.3.0.12_pv1.1
Greenplum Database 4.3 - PostGIS Extension for RHEL 2.0.3_pv2.0
Greenplum Database 4.3 - Pgcrypto Extension for RHEL 1.1_pv1.1
要安装GP的话, 只需要安装
Greenplum Database 4.3.2.0 for Red Hat Enterprise Linux 5 and 6
其他的组件, 按需安装即可, 例如需要PL/R的语言扩展, 可以装这个组件.
unzip greenplum-db-4.3.2.0-build-1-RHEL5-x86_64.zip
下载后解压, 有两个文件, 一个README_INSTALL, 还有一个安装文件.
截取README_INSTALL文件的几个内容.
安装说明
***********************************************************
RUNNING THE GREENPLUM DATABASE INSTALLER
***********************************************************
1. Open a command-line terminal.
2. Execute the installer using bash:
/bin/bash greenplum-db-4.3.x.x-<PLATFORM>.bin
3. When prompted, type 'yes' to accept the license agreement.
4. When prompted, provide an installation path. Make sure you have
permissions to write to the installation directory you specify.
For example:
/usr/local/greenplum-db-4.3.x.x
操作系统设置 :
***********************************************************
REQUIRED OS SYSTEM SETTINGS FOR GREENPLUM DATABASE
***********************************************************
These must be set on all Greenplum hosts (master and segments).
Make sure you restart your system after adding or changing
kernel parameters.
----------------
LINUX
----------------
/etc/sysctl.conf:
kernel.shmmax = 500000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 1025 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2
/etc/security/limits.conf:
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072
/etc/security/limits.d/90-nproc.conf
* soft nproc 131072
* hard nproc 131073
/etc/hosts:
Include the host names and all interface address names for every
machine participating in your Greenplum Database system.
XFS文件系统mount推荐参数
File system recommendations:
XFS is the preferred file system on Linux platforms for data storage.
Greenplum recommends the following xfs mount options:
rw,noatime,inode64,allocsize=16m
设置块设备的预读块
Also, each disk device file should have a read-ahead value of 16384.
For example, to verify the read-ahead value of a disk device:
# /sbin/blockdev --getra /dev/sdb
如果有SSD的话, 建议使用flashcache优化IOPS能力.
http://blog.163.com/digoal@126/blog/static/163877040201463101652528/
http://blog.163.com/digoal@126/blog/static/1638770402014528115551323/
安装完后, 目录结构 :
************************************************
ABOUT YOUR INSTALLATION
************************************************
The Greenplum Database installer installs the following files and
directories:
* greenplum_path.sh - Greenplum Database environment variables.
* GPDB-LICENSE.txt - Greenplum license agreement.
* LICENSE.thirdparty - Licenses for third-party tools
* bin - Greenplum Database server programs, client programs, and
management tools.
* demo - Greenplum Database demonstration programs.
* docs - Greenplum Database documentation.
* etc - Sample configuration file for OpenSSL.
* ext - Bundled programs (such as Python) used by some
Greenplum Database utilities.
* include - Greenplum Database and PostgreSQL header files.
* lib - Greenplum Database and PostgreSQL library files.
* sbin - Supporting/Internal scripts and programs.
* share - PostgreSQL sample files and templates.
其他参考文档
************************************************
GREENPLUM DATABASE DOCUMENTATION
************************************************
The following documentation is available for
download from http://support.emc.com
* Greenplum Database <version> Release Notes
Release notes for this version of Greenplum Database.
The release notes contains a list of the current Greenplum
Database documentation.
* Greenplum Database Installation Guide
* Greenplum Database Administrator Guide
* Greenplum Database System Administrator Guide
* Greenplum Database Reference Guide
* Greenplum Database Utility Guide
* Greenplum Database Installation Guide
* Greenplum Database Database Administrator Guide
* Greenplum Database System Administrator Guide
* Greenplum Database Reference Guide
* Greenplum Database Utility Guide
* Greenplum Database Client Tools for UNIX
* Greenplum Database Client Tools for Windows
* Greenplum Database Connectivity Tools for UNIX
* Greenplum Database Connectivity Tools for Windows
* Greenplum Database Load Tools for UNIX
* Greenplum Database Load Tools for Windows
* Greenplum Command Center Administrator Guide
安装步骤:
./greenplum-db-4.3.2.0-build-1-RHEL5-x86_64.bin
********************************************************************************
Do you accept the Pivotal Database license agreement? [yes|no]
********************************************************************************
yes
********************************************************************************
Provide the installation path for Greenplum Database or press ENTER to
accept the default installation path: /usr/local/greenplum-db-4.3.2.0
********************************************************************************
/opt/greenplum-db-4.3.2.0
********************************************************************************
Install Greenplum Database into </opt/greenplum-db-4.3.2.0>? [yes|no]
********************************************************************************
yes
********************************************************************************
/opt/greenplum-db-4.3.2.0 does not exist.
Create /opt/greenplum-db-4.3.2.0 ? [yes|no]
(Selecting no will exit the installer)
********************************************************************************
yes
********************************************************************************
[Optional] Provide the path to a previous installation of Greenplum Database,
or press ENTER to skip this step. e.g. /usr/local/greenplum-db-4.1.1.3
This installation step will migrate any Greenplum Database extensions from the
provided path to the version currently being installed. This step is optional
and can be run later with:
gppkg --migrate <path_to_old_gphome> /opt/greenplum-db-4.3.2.0
********************************************************************************
Enter.
Extracting product to /opt/greenplum-db-4.3.2.0
Skipping migration of Greenplum Database extensions...
********************************************************************************
Installation complete.
Greenplum Database is installed in /opt/greenplum-db-4.3.2.0
Pivotal Greenplum documentation is available
for download at http://docs.gopivotal.com/gpdb
********************************************************************************
目录结构 :
cd /opt/greenplum-db-4.3.2.0
[root@db-172-16-3-150 greenplum-db-4.3.2.0]# ll
total 284
drwxr-xr-x 3 pgdigoal digoal 4096 Jul 13 08:23 bin
drwxr-xr-x 2 pgdigoal digoal 4096 Jul 13 08:07 demo
drwxr-xr-x 5 pgdigoal digoal 4096 Jul 13 08:07 docs
drwxr-xr-x 2 pgdigoal digoal 4096 Jul 13 08:07 etc
drwxr-xr-x 3 pgdigoal digoal 4096 Jul 13 08:07 ext
-rw-rw-r-- 1 pgdigoal digoal 43025 Jul 13 08:23 GPDB-LICENSE.txt
-rw-r--r-- 1 root root 670 Jul 25 16:13 greenplum_path.sh
drwxr-xr-x 6 pgdigoal digoal 4096 Jul 13 08:07 include
drwxr-xr-x 7 pgdigoal digoal 12288 Jul 13 08:07 lib
-rw-rw-r-- 1 pgdigoal digoal 193083 Jul 13 08:23 LICENSE.thirdparty
drwxr-xr-x 2 pgdigoal digoal 4096 Jul 13 08:23 sbin
drwxr-xr-x 3 pgdigoal digoal 4096 Jul 13 07:57 share
参考
1. https://network.gopivotal.com/products/pivotal-gpdb
2. http://blog.163.com/digoal@126/blog/static/163877040201463101652528/
3. http://blog.163.com/digoal@126/blog/static/1638770402014528115551323/