PostgreSQL 如何让 列存(外部列存) 并行起来

11 minute read

背景

PostgreSQL 10已经实现了大部分操作的并行计算,例如(全表扫描,索引扫描,位图扫描,哈希JOIN,哈希聚合,分组聚合,排序,建索引等)。

对于外部表,要实现并行扫描,PostgreSQL有什么方法呢?

PostgreSQL不仅支持单个对象的并行计算,还支持多个对象的并行访问。多个对象的并行访问,包括继承表、分区表、继承外部表、UNION、UNION ALL等语义。

这个patch:

https://commitfest.postgresql.org/15/987/

例如:

某个父表有4个继承表,当查询父表时,可以并行的访问父表以及它的继承表。(继承可以是本地的物化视图、本地表、或者外部表对象)。这就给并行访问外部列存表提供了技术基础。

并行访问外部表带来了一个全新的能力,可以把 PG 10+ 作为一个具备完整SQL语义,强大功能的中间件。

试用这个patch

1、使用这个PATCH

wget https://ftp.postgresql.org/pub/snapshot/dev/postgresql-snapshot.tar.bz2  
  
wget https://www.postgresql.org/message-id/attachment/55680/ParallelAppend_v17.patch  
  
tar -jxvf postgresql-snapshot.tar.bz2  
  
cd postgresql-11devel/  
patch -p 1 < ../ParallelAppend_v17.patch   
  
  
export USE_NAMED_POSIX_SEMAPHORES=1  
LIBS=-lpthread CFLAGS="-O3" ./configure --prefix=/home/dege.zzz/pgsql11  
LIBS=-lpthread CFLAGS="-O3" make world -j 64  
LIBS=-lpthread CFLAGS="-O3" make install-world  

2、初始化数据库

initdb -D $PGDATA -U postgres --locale=C -E SQL_ASCII  

3、配置参数,这个PATCH增加了两个GUC参数

enable_partition_wise_join  
  
enable_parallelappend  

4、配置并行参数,用于并行测试

max_worker_processes = 128              # (change requires restart)  
max_parallel_workers_per_gather = 64    # taken from max_parallel_workers  
max_parallel_workers = 128              # maximum number of max_worker_processes that  
  
enable_parallelappend = on  
  
parallel_tuple_cost = 0         # same scale as above  
parallel_setup_cost = 0 # same scale as above  
min_parallel_table_scan_size = 0  
min_parallel_index_scan_size = 0  

5、创建表和继承关系

create table p(id int, info text);  
  
create table c1(like p) inherits(p);  
create table c2(like p) inherits(p);  
create table c3(like p) inherits(p);  
create table c4(like p) inherits(p);  

6、写入测试数据,每个子表1000万记录。

insert into c1 select generate_series(1,10000000),'test';  
insert into c2 select generate_series(1,10000000),'test';  
insert into c3 select generate_series(1,10000000),'test';  
insert into c4 select generate_series(1,10000000),'test';  

7、设置每个子表的并行度

alter table c1 set (parallel_workers =8);  
alter table c2 set (parallel_workers =8);  
alter table c3 set (parallel_workers =8);  
alter table c4 set (parallel_workers =8);  

8、设置父表的并行度(目前取决于父表的并行度,将来可能会改进)。

alter table p set (parallel_workers = 40);  

9、单表的耗时

explain (analyze,verbose,timing,costs,buffers) select count(*) from c1 where id=1;  
explain (analyze,verbose,timing,costs,buffers) select count(*) from c2 where id=2;  
explain (analyze,verbose,timing,costs,buffers) select count(*) from c3 where id=2;  
explain (analyze,verbose,timing,costs,buffers) select count(*) from c4 where id=2;  
  
每张表114毫秒。  
  
Execution time: 114.160 ms  

10、测试关闭append parallel

postgres=# set enable_parallelappend =off;  
SET  
postgres=# alter table p set (parallel_workers = 8);  
ALTER TABLE  
postgres=# explain (analyze,verbose,timing,costs,buffers) select count(*) from p where id=2;  
                                                             QUERY PLAN                                                                
-------------------------------------------------------------------------------------------------------------------------------------  
 Aggregate  (cost=278720.12..278720.13 rows=1 width=8) (actual time=406.051..406.051 rows=1 loops=1)  
   Output: count(*)  
   Buffers: shared hit=30906  
   ->  Gather  (cost=0.00..278720.11 rows=5 width=0) (actual time=0.376..406.042 rows=4 loops=1)  
         Workers Planned: 8  
         Workers Launched: 8  
         Buffers: shared hit=30906  
         ->  Append  (cost=0.00..278720.11 rows=5 width=0) (actual time=325.088..402.806 rows=0 loops=9)  
               Buffers: shared hit=216220  
               Worker 0: actual time=401.809..401.809 rows=0 loops=1  
                 Buffers: shared hit=23064  
               Worker 1: actual time=402.125..402.125 rows=0 loops=1  
                 Buffers: shared hit=23156  
               Worker 2: actual time=402.255..402.255 rows=0 loops=1  
                 Buffers: shared hit=23494  
               Worker 3: actual time=402.448..402.448 rows=0 loops=1  
                 Buffers: shared hit=23337  
               Worker 4: actual time=305.167..402.598 rows=1 loops=1  
                 Buffers: shared hit=23572  
               Worker 5: actual time=206.445..402.817 rows=1 loops=1  
                 Buffers: shared hit=22803  
               Worker 6: actual time=402.630..402.630 rows=0 loops=1  
                 Buffers: shared hit=22971  
               Worker 7: actual time=402.905..402.905 rows=0 loops=1  
                 Buffers: shared hit=22917  
               ->  Parallel Seq Scan on public.p  (cost=0.00..0.00 rows=1 width=0) (actual time=0.000..0.000 rows=0 loops=9)  
                     Filter: (p.id = 2)  
                     Worker 0: actual time=0.000..0.000 rows=0 loops=1  
                     Worker 1: actual time=0.000..0.000 rows=0 loops=1  
                     Worker 2: actual time=0.000..0.000 rows=0 loops=1  
                     Worker 3: actual time=0.000..0.000 rows=0 loops=1  
                     Worker 4: actual time=0.000..0.000 rows=0 loops=1  
                     Worker 5: actual time=0.000..0.000 rows=0 loops=1  
                     Worker 6: actual time=0.000..0.000 rows=0 loops=1  
                     Worker 7: actual time=0.001..0.001 rows=0 loops=1  
               ->  Parallel Seq Scan on public.c1  (cost=0.00..69680.27 rows=1 width=0) (actual time=93.294..105.312 rows=0 loops=9)  
                     Filter: (c1.id = 2)  
                     Rows Removed by Filter: 1111111  
                     Buffers: shared hit=54055  
                     Worker 0: actual time=104.313..104.313 rows=0 loops=1  
                       Buffers: shared hit=5554  
                     Worker 1: actual time=104.637..104.637 rows=0 loops=1  
                       Buffers: shared hit=5598  
                     Worker 2: actual time=104.762..104.762 rows=0 loops=1  
                       Buffers: shared hit=5857  
                     Worker 3: actual time=104.952..104.952 rows=0 loops=1  
                       Buffers: shared hit=5827  
                     Worker 4: actual time=105.111..105.111 rows=0 loops=1  
                       Buffers: shared hit=5896  
                     Worker 5: actual time=105.328..105.328 rows=0 loops=1  
                       Buffers: shared hit=5664  
                     Worker 6: actual time=105.121..105.121 rows=0 loops=1  
                       Buffers: shared hit=5692  
                     Worker 7: actual time=105.414..105.414 rows=0 loops=1  
                       Buffers: shared hit=5713  
               ->  Parallel Seq Scan on public.c2  (cost=0.00..69679.78 rows=1 width=0) (actual time=89.870..101.103 rows=0 loops=9)  
                     Filter: (c2.id = 2)  
                     Rows Removed by Filter: 1111111  
                     Buffers: shared hit=54055  
                     Worker 0: actual time=101.104..101.104 rows=0 loops=1  
                       Buffers: shared hit=5849  
                     Worker 1: actual time=101.107..101.107 rows=0 loops=1  
                       Buffers: shared hit=5864  
                     Worker 2: actual time=101.102..101.102 rows=0 loops=1  
                       Buffers: shared hit=5900  
                     Worker 3: actual time=101.096..101.096 rows=0 loops=1  
                       Buffers: shared hit=5859  
                     Worker 4: actual time=101.112..101.112 rows=0 loops=1  
                       Buffers: shared hit=5920  
                     Worker 5: actual time=101.095..101.095 rows=0 loops=1  
                       Buffers: shared hit=5798  
                     Worker 6: actual time=101.108..101.108 rows=0 loops=1  
                       Buffers: shared hit=5843  
                     Worker 7: actual time=101.104..101.104 rows=0 loops=1  
                       Buffers: shared hit=5818  
               ->  Parallel Seq Scan on public.c3  (cost=0.00..69679.78 rows=1 width=0) (actual time=87.947..98.939 rows=0 loops=9)  
                     Filter: (c3.id = 2)  
                     Rows Removed by Filter: 1111111  
                     Buffers: shared hit=54055  
                     Worker 0: actual time=98.939..98.939 rows=0 loops=1  
                       Buffers: shared hit=5853  
                     Worker 1: actual time=98.929..98.929 rows=0 loops=1  
                       Buffers: shared hit=5870  
                     Worker 2: actual time=98.952..98.952 rows=0 loops=1  
                       Buffers: shared hit=5918  
                     Worker 3: actual time=98.940..98.940 rows=0 loops=1  
                       Buffers: shared hit=5826  
                     Worker 4: actual time=98.928..98.928 rows=0 loops=1  
                       Buffers: shared hit=5886  
                     Worker 5: actual time=0.017..98.948 rows=1 loops=1  
                       Buffers: shared hit=5608  
                     Worker 6: actual time=98.936..98.936 rows=0 loops=1  
                       Buffers: shared hit=5643  
                     Worker 7: actual time=98.936..98.936 rows=0 loops=1  
                       Buffers: shared hit=5624  
               ->  Parallel Seq Scan on public.c4  (cost=0.00..69680.27 rows=1 width=0) (actual time=86.621..97.446 rows=0 loops=9)  
                     Filter: (c4.id = 2)  
                     Rows Removed by Filter: 1111111  
                     Buffers: shared hit=54055  
                     Worker 0: actual time=97.448..97.448 rows=0 loops=1  
                       Buffers: shared hit=5808  
                     Worker 1: actual time=97.447..97.447 rows=0 loops=1  
                       Buffers: shared hit=5824  
                     Worker 2: actual time=97.435..97.435 rows=0 loops=1  
                       Buffers: shared hit=5819  
                     Worker 3: actual time=97.455..97.455 rows=0 loops=1  
                       Buffers: shared hit=5825  
                     Worker 4: actual time=0.012..97.443 rows=1 loops=1  
                       Buffers: shared hit=5870  
                     Worker 5: actual time=97.439..97.439 rows=0 loops=1  
                       Buffers: shared hit=5733  
                     Worker 6: actual time=97.460..97.460 rows=0 loops=1  
                       Buffers: shared hit=5793  
                     Worker 7: actual time=97.443..97.443 rows=0 loops=1  
                       Buffers: shared hit=5762  
 Planning time: 0.140 ms  
 Execution time: 421.076 ms  
(117 rows)  
  
扫描p, c1, c2, c3, c4是串行的。耗时几乎是累加的。  
  
同时也可以看到,每个表扫描都启动了新的worker。  

11、测试开启append parallel

postgres=# set enable_parallelappend =on;  
SET  
postgres=# alter table p set (parallel_workers = 40);  
ALTER TABLE  
postgres=# explain (analyze,verbose,timing,costs,buffers) select count(*) from p where id=2;  
                                                              QUERY PLAN                                                                
--------------------------------------------------------------------------------------------------------------------------------------  
 Aggregate  (cost=278720.12..278720.13 rows=1 width=8) (actual time=128.447..128.447 rows=1 loops=1)  
   Output: count(*)  
   Buffers: shared hit=9780  
   ->  Gather  (cost=0.00..278720.11 rows=5 width=0) (actual time=0.914..128.438 rows=4 loops=1)  
         Workers Planned: 41  
         Workers Launched: 41  
         Buffers: shared hit=9780  
         ->  Parallel Append  (cost=0.00..278720.11 rows=5 width=0) (actual time=107.692..119.530 rows=0 loops=42)  
               Buffers: shared hit=216220  
               Worker 0: actual time=113.650..113.650 rows=0 loops=1  
                 Buffers: shared hit=3363  
               Worker 1: actual time=114.238..114.238 rows=0 loops=1  
                 Buffers: shared hit=3329  
               Worker 2: actual time=114.214..114.214 rows=0 loops=1  
                 Buffers: shared hit=3333  
               Worker 3: actual time=114.887..114.887 rows=0 loops=1  
                 Buffers: shared hit=3370  
               Worker 4: actual time=115.038..115.038 rows=0 loops=1  
                 Buffers: shared hit=3365  
               Worker 5: actual time=115.084..115.084 rows=0 loops=1  
                 Buffers: shared hit=3389  
               Worker 6: actual time=115.720..115.720 rows=0 loops=1  
                 Buffers: shared hit=3370  
               Worker 7: actual time=115.830..115.830 rows=0 loops=1  
                 Buffers: shared hit=3358  
               Worker 8: actual time=116.457..116.457 rows=0 loops=1  
                 Buffers: shared hit=3430  
               Worker 9: actual time=116.760..116.760 rows=0 loops=1  
                 Buffers: shared hit=3423  
               Worker 10: actual time=116.541..116.541 rows=0 loops=1  
                 Buffers: shared hit=6129  
               Worker 11: actual time=117.355..117.355 rows=0 loops=1  
                 Buffers: shared hit=3447  
               Worker 12: actual time=117.767..117.767 rows=0 loops=1  
                 Buffers: shared hit=6295  
               Worker 13: actual time=118.195..118.195 rows=0 loops=1  
                 Buffers: shared hit=6382  
               Worker 14: actual time=118.150..118.150 rows=0 loops=1  
                 Buffers: shared hit=6194  
               Worker 15: actual time=118.468..118.468 rows=0 loops=1  
                 Buffers: shared hit=6250  
               Worker 16: actual time=119.035..119.035 rows=0 loops=1  
                 Buffers: shared hit=6337  
               Worker 17: actual time=119.092..119.092 rows=0 loops=1  
                 Buffers: shared hit=6472  
               Worker 18: actual time=119.077..119.077 rows=0 loops=1  
                 Buffers: shared hit=6293  
               Worker 19: actual time=119.361..119.361 rows=0 loops=1  
                 Buffers: shared hit=6557  
               Worker 20: actual time=119.519..119.519 rows=0 loops=1  
                 Buffers: shared hit=6446  
               Worker 21: actual time=119.782..119.782 rows=0 loops=1  
                 Buffers: shared hit=6465  
               Worker 22: actual time=120.339..120.339 rows=0 loops=1  
                 Buffers: shared hit=6575  
               Worker 23: actual time=120.350..120.350 rows=0 loops=1  
                 Buffers: shared hit=6596  
               Worker 24: actual time=120.436..120.436 rows=0 loops=1  
                 Buffers: shared hit=6593  
               Worker 25: actual time=121.735..121.735 rows=0 loops=1  
                 Buffers: shared hit=6606  
               Worker 26: actual time=120.821..120.821 rows=0 loops=1  
                 Buffers: shared hit=6607  
               Worker 27: actual time=122.161..122.161 rows=0 loops=1  
                 Buffers: shared hit=6585  
               Worker 28: actual time=121.265..121.265 rows=0 loops=1  
                 Buffers: shared hit=6695  
               Worker 29: actual time=121.616..121.616 rows=0 loops=1  
                 Buffers: shared hit=6703  
               Worker 30: actual time=121.919..121.919 rows=0 loops=1  
                 Buffers: shared hit=3630  
               Worker 31: actual time=121.914..121.914 rows=0 loops=1  
                 Buffers: shared hit=6590  
               Worker 32: actual time=122.310..122.310 rows=0 loops=1  
                 Buffers: shared hit=3714  
               Worker 33: actual time=122.471..122.471 rows=0 loops=1  
                 Buffers: shared hit=6652  
               Worker 34: actual time=122.684..122.684 rows=0 loops=1  
                 Buffers: shared hit=3699  
               Worker 35: actual time=122.737..122.737 rows=0 loops=1  
                 Buffers: shared hit=3744  
               Worker 36: actual time=122.884..122.884 rows=0 loops=1  
                 Buffers: shared hit=3678  
               Worker 37: actual time=123.155..123.155 rows=0 loops=1  
                 Buffers: shared hit=3740  
               Worker 38: actual time=0.014..123.474 rows=1 loops=1  
                 Buffers: shared hit=3647  
               Worker 39: actual time=0.015..123.662 rows=1 loops=1  
                 Buffers: shared hit=3693  
               Worker 40: actual time=0.019..123.742 rows=1 loops=1  
                 Buffers: shared hit=3696  
               ->  Parallel Seq Scan on public.p  (cost=0.00..0.00 rows=1 width=0) (actual time=0.001..0.001 rows=0 loops=1)  
                     Filter: (p.id = 2)  
                     Worker 40: actual time=0.001..0.001 rows=0 loops=1  
               ->  Parallel Seq Scan on public.c1  (cost=0.00..69680.27 rows=1 width=0) (actual time=85.131..93.695 rows=0 loops=14)  
                     Filter: (c1.id = 2)  
                     Rows Removed by Filter: 714286  
                     Buffers: shared hit=54055  
                     Worker 0: actual time=110.100..110.100 rows=0 loops=1  
                       Buffers: shared hit=3260  
                     Worker 4: actual time=111.267..111.267 rows=0 loops=1  
                       Buffers: shared hit=3272  
                     Worker 8: actual time=112.661..112.661 rows=0 loops=1  
                       Buffers: shared hit=3333  
                     Worker 9: actual time=13.433..13.433 rows=0 loops=1  
                       Buffers: shared hit=419  
                     Worker 12: actual time=114.096..114.096 rows=0 loops=1  
                       Buffers: shared hit=6118  
                     Worker 16: actual time=115.113..115.113 rows=0 loops=1  
                       Buffers: shared hit=6166  
                     Worker 17: actual time=13.448..13.448 rows=0 loops=1  
                       Buffers: shared hit=770  
                     Worker 20: actual time=115.961..115.961 rows=0 loops=1  
                       Buffers: shared hit=6281  
                     Worker 24: actual time=116.868..116.868 rows=0 loops=1  
                       Buffers: shared hit=6417  
                     Worker 28: actual time=117.509..117.509 rows=0 loops=1  
                       Buffers: shared hit=6525  
                     Worker 32: actual time=118.585..118.585 rows=0 loops=1  
                       Buffers: shared hit=3617  
                     Worker 33: actual time=13.446..13.446 rows=0 loops=1  
                       Buffers: shared hit=699  
                     Worker 36: actual time=119.327..119.327 rows=0 loops=1  
                       Buffers: shared hit=3579  
                     Worker 40: actual time=0.017..119.913 rows=1 loops=1  
                       Buffers: shared hit=3599  
               ->  Parallel Seq Scan on public.c2  (cost=0.00..69679.78 rows=1 width=0) (actual time=29.164..32.100 rows=0 loops=42)  
                     Filter: (c2.id = 2)  
                     Rows Removed by Filter: 238095  
                     Buffers: shared hit=54055  
                     Worker 0: actual time=2.855..2.855 rows=0 loops=1  
                       Buffers: shared hit=83  
                     Worker 1: actual time=2.909..2.909 rows=0 loops=1  
                       Buffers: shared hit=88  
                     Worker 2: actual time=2.880..2.880 rows=0 loops=1  
                       Buffers: shared hit=91  
                     Worker 3: actual time=114.623..114.623 rows=0 loops=1  
                       Buffers: shared hit=3370  
                     Worker 4: actual time=3.567..3.567 rows=0 loops=1  
                       Buffers: shared hit=93  
                     Worker 5: actual time=2.890..2.890 rows=0 loops=1  
                       Buffers: shared hit=89  
                     Worker 6: actual time=2.893..2.893 rows=0 loops=1  
                       Buffers: shared hit=89  
                     Worker 7: actual time=115.735..115.735 rows=0 loops=1  
                       Buffers: shared hit=3358  
                     Worker 8: actual time=2.921..2.921 rows=0 loops=1  
                       Buffers: shared hit=78  
                     Worker 9: actual time=2.937..2.937 rows=0 loops=1  
                       Buffers: shared hit=80  
                     Worker 10: actual time=2.873..2.873 rows=0 loops=1  
                       Buffers: shared hit=160  
                     Worker 11: actual time=117.272..117.272 rows=0 loops=1  
                       Buffers: shared hit=3447  
                     Worker 12: actual time=2.865..2.865 rows=0 loops=1  
                       Buffers: shared hit=143  
                     Worker 13: actual time=17.020..17.020 rows=0 loops=1  
                       Buffers: shared hit=916  
                     Worker 14: actual time=2.850..2.850 rows=0 loops=1  
                       Buffers: shared hit=160  
                     Worker 15: actual time=118.347..118.347 rows=0 loops=1  
                       Buffers: shared hit=6250  
                     Worker 16: actual time=2.924..2.924 rows=0 loops=1  
                       Buffers: shared hit=137  
                     Worker 17: actual time=3.566..3.566 rows=0 loops=1  
                       Buffers: shared hit=174  
                     Worker 18: actual time=2.888..2.888 rows=0 loops=1  
                       Buffers: shared hit=122  
                     Worker 19: actual time=119.352..119.352 rows=0 loops=1  
                       Buffers: shared hit=6557  
                     Worker 20: actual time=3.553..3.553 rows=0 loops=1  
                       Buffers: shared hit=165  
                     Worker 21: actual time=17.010..17.010 rows=0 loops=1  
                       Buffers: shared hit=927  
                     Worker 22: actual time=2.851..2.851 rows=0 loops=1  
                       Buffers: shared hit=167  
                     Worker 23: actual time=120.346..120.346 rows=0 loops=1  
                       Buffers: shared hit=6596  
                     Worker 24: actual time=3.563..3.563 rows=0 loops=1  
                       Buffers: shared hit=176  
                     Worker 25: actual time=17.017..17.017 rows=0 loops=1  
                       Buffers: shared hit=935  
                     Worker 26: actual time=2.886..2.886 rows=0 loops=1  
                       Buffers: shared hit=163  
                     Worker 27: actual time=121.060..121.060 rows=0 loops=1  
                       Buffers: shared hit=6585  
                     Worker 28: actual time=2.911..2.911 rows=0 loops=1  
                       Buffers: shared hit=135  
                     Worker 29: actual time=2.898..2.898 rows=0 loops=1  
                       Buffers: shared hit=168  
                     Worker 30: actual time=121.689..121.689 rows=0 loops=1  
                       Buffers: shared hit=3630  
                     Worker 31: actual time=2.920..2.920 rows=0 loops=1  
                       Buffers: shared hit=165  
                     Worker 32: actual time=3.572..3.572 rows=0 loops=1  
                       Buffers: shared hit=97  
                     Worker 33: actual time=3.567..3.567 rows=0 loops=1  
                       Buffers: shared hit=171  
                     Worker 34: actual time=2.911..2.911 rows=0 loops=1  
                       Buffers: shared hit=89  
                     Worker 35: actual time=122.732..122.732 rows=0 loops=1  
                       Buffers: shared hit=3744  
                     Worker 36: actual time=2.877..2.877 rows=0 loops=1  
                       Buffers: shared hit=80  
                     Worker 37: actual time=16.989..16.989 rows=0 loops=1  
                       Buffers: shared hit=496  
                     Worker 38: actual time=2.882..2.882 rows=0 loops=1  
                       Buffers: shared hit=86  
                     Worker 39: actual time=0.014..123.337 rows=1 loops=1  
                       Buffers: shared hit=3693  
                     Worker 40: actual time=3.595..3.595 rows=0 loops=1  
                       Buffers: shared hit=97  
               ->  Parallel Seq Scan on public.c3  (cost=0.00..69679.78 rows=1 width=0) (actual time=52.559..58.290 rows=0 loops=21)  
                     Filter: (c3.id = 2)  
                     Rows Removed by Filter: 476190  
                     Buffers: shared hit=54055  
                     Worker 0: actual time=0.689..0.689 rows=0 loops=1  
                       Buffers: shared hit=20  
                     Worker 1: actual time=14.124..14.124 rows=0 loops=1  
                       Buffers: shared hit=419  
                     Worker 2: actual time=111.314..111.314 rows=0 loops=1  
                       Buffers: shared hit=3242  
                     Worker 5: actual time=14.126..14.126 rows=0 loops=1  
                       Buffers: shared hit=421  
                     Worker 6: actual time=112.583..112.583 rows=0 loops=1  
                       Buffers: shared hit=3281  
                     Worker 8: actual time=0.672..0.672 rows=0 loops=1  
                       Buffers: shared hit=19  
                     Worker 9: actual time=0.669..0.669 rows=0 loops=1  
                       Buffers: shared hit=19  
                     Worker 10: actual time=113.632..113.632 rows=0 loops=1  
                       Buffers: shared hit=5969  
                     Worker 12: actual time=0.678..0.678 rows=0 loops=1  
                       Buffers: shared hit=34  
                     Worker 14: actual time=115.273..115.273 rows=0 loops=1  
                       Buffers: shared hit=6034  
                     Worker 16: actual time=0.671..0.671 rows=0 loops=1  
                       Buffers: shared hit=34  
                     Worker 18: actual time=116.184..116.184 rows=0 loops=1  
                       Buffers: shared hit=6171  
                     Worker 22: actual time=117.285..117.285 rows=0 loops=1  
                       Buffers: shared hit=6408  
                     Worker 26: actual time=117.931..117.931 rows=0 loops=1  
                       Buffers: shared hit=6444  
                     Worker 28: actual time=0.679..0.679 rows=0 loops=1  
                       Buffers: shared hit=35  
                     Worker 29: actual time=14.117..14.117 rows=0 loops=1  
                       Buffers: shared hit=794  
                     Worker 31: actual time=118.705..118.705 rows=0 loops=1  
                       Buffers: shared hit=6425  
                     Worker 34: actual time=119.585..119.585 rows=0 loops=1  
                       Buffers: shared hit=3610  
                     Worker 36: actual time=0.675..0.675 rows=0 loops=1  
                       Buffers: shared hit=19  
                     Worker 38: actual time=0.013..120.367 rows=1 loops=1  
                       Buffers: shared hit=3561  
               ->  Parallel Seq Scan on public.c4  (cost=0.00..69680.27 rows=1 width=0) (actual time=92.675..102.616 rows=0 loops=11)  
                     Filter: (c4.id = 2)  
                     Rows Removed by Filter: 909091  
                     Buffers: shared hit=54055  
                     Worker 1: actual time=96.862..96.862 rows=0 loops=1  
                       Buffers: shared hit=2822  
                     Worker 5: actual time=98.057..98.057 rows=0 loops=1  
                       Buffers: shared hit=2879  
                     Worker 9: actual time=99.478..99.478 rows=0 loops=1  
                       Buffers: shared hit=2905  
                     Worker 13: actual time=101.043..101.043 rows=0 loops=1  
                       Buffers: shared hit=5466  
                     Worker 17: actual time=101.879..101.879 rows=0 loops=1  
                       Buffers: shared hit=5528  
                     Worker 21: actual time=102.755..102.755 rows=0 loops=1  
                       Buffers: shared hit=5538  
                     Worker 25: actual time=103.575..103.575 rows=0 loops=1  
                       Buffers: shared hit=5671  
                     Worker 29: actual time=104.456..104.456 rows=0 loops=1  
                       Buffers: shared hit=5741  
                     Worker 33: actual time=105.152..105.152 rows=0 loops=1  
                       Buffers: shared hit=5782  
                     Worker 37: actual time=106.161..106.161 rows=0 loops=1  
                       Buffers: shared hit=3244  
 Planning time: 0.143 ms  
 Execution time: 143.039 ms  
(282 rows)  
  
扫描p, c1, c2, c3, c4是并行的。时间不是累加。  
  
worker 分布不均,如果均匀性能应该是可以更好的。  
  
不过现在还没有正式MERGE这个PATCH,正式提交肯定会改进的。  

12、外部表继承的语法如下:

ALTER FOREIGN TABLE name   
    INHERIT parent_table |   
    NO INHERIT parent_table  

用此法,可以实现任意类型的外部表的并行。当然也包括本文提到的cstore_fdw,列存外部表。

还有mysql_fdw, oracle_fdw, mongo_fdw, file_fdw, oss_fdw, 比较全面的FDW接口,可以参考这个WIKI页面:

https://wiki.postgresql.org/wiki/Fdw

参考

https://commitfest.postgresql.org/15/987/

Flag Counter

digoal’s 大量PostgreSQL文章入口