PostgreSQL & LLVM & Vitesse DB

背景

llvm动态编译，快了不是一点点。当IO不再是瓶颈时，下一个瓶颈CPU。LLVM解决了CPU瓶颈的问题。

http://vitessedata.com/vitesse-x

提供插件形式的测试，闭源。

PostgreSQL performance with gcc, clang and icc

http://blog.pgaddict.com/posts/postgresql-performance-with-gcc-clang-and-icc

Clang build of PostgreSQL

http://lists.cs.uiuc.edu/pipermail/cfe-dev/2011-June/015793.html

Go Faster With Native Compilation

http://www.pgcon.org/2015/schedule/attachments/344_go_faster_with_native_compilation_v3.pdf

http://www.pgcon.org/2015/schedule/track/Performance/776.en.html

The LLVM Compiler Infrastructure

http://llvm.org/

We have been running a similar setup (Postgres -> Foreign Data Wrappers -> LLVM) at AdRoll for over a year. We keep 100TBs+ of raw data in memory, compressed.

We managed to build our solution mostly in Python(!) using Numba for JIT and a number of compression tricks. More about it here:

HOW TO BUILD A SQL-BASED DATA WAREHOUSE

FOR A TRILLION ROWS IN PYTHON

http://tuulos.github.io/pydata-2014

https://www.youtube.com/watch?v=xnfnv6WT1Ng

Vitesse DB is a PostgreSQL database designed for data warehouse, OLAP/BI applications. On top of PostgreSQL, we add dynamic query compilation, parallel query execution, column store, and fast data injection capabilities. The result is astounding: TPCH Q1 runs 180X faster.

http://vitessedata.com/technology

Vitesse DB call for testing

Hi everyone,

Vitesse DB 9.3.5.S is Postgres 9.3.5 with a LLVM-JIT query executor

designed for compute intensive OLAP workload. We have gotten it to a

reasonable state and would like to open it up to the pg hackers

community for testing and suggestions.

Vitesse DB offers

-- LLVM JIT Compilation for compute-intensive queries  
-- CSV parsing with SSE instructions  
-- 100% binary compatibility with PG9.3.5.  

Our results show CSV imports run up to 2X faster, and TPCH Q1 runs 8X faster.

Our TPCH 1GB benchmark results is also available at

http://vitessedata.com/benchmark/ .

Please direct any questions by email to cktan(at)vitessedata(dot)com .

Thank you for your help.

--  
CK Tan  
Vitesse Data, Inc.  

http://www.postgresql.org/message-id/flat/5CFE0CA1-E5CC-4CD1-9D0B-8D72143D81C2@vitessedata.com#5CFE0CA1-E5CC-4CD1-9D0B-8D72143D81C2@vitessedata.com

https://news.ycombinator.com/item?id=8470754

参考

1. http://llvm.org/

digoal’s 大量PostgreSQL文章入口