Systemtap EXP: PostgreSQL IN-BUILD mark Class 3 - checkpoint
背景
本文将讲一下PostgreSQL内建的checkpoint相关探针.
包含:
checkpoint开始, 完成;
checkpoin的clog部分开始, 完成;
checkpoin的subtransaction部分开始, 完成;
checkpoin的multixact部分开始, 完成;
checkpoin的buffer writting部分开始, 写脏页面开始, 单个脏页面写完成, 所有脏页面写完成, checkpoin的buffer write完成;
checkpoin的two phase部分开始, 完成;
从这些探针变量中可以获取到 :
1. checkpoint的类型,
2. checkpoint写了多少buffers, 一共有多少buffers, checkpoint过程中产生了多少xlog文件, 有多少xlog文件可以删除, 多少xlog文件可以重用.
3 .区分是normal checkpoint还是shutdown checkpoint
4. shared buffer中一共有多少buffers(单位page), buffers中有多少脏数据要写
5. buffer ID, 当使用buffer_sync_written时得到, buffer id即当前正在同步shared buffer中的那个page.
6. 真实同步的buffers, 期望同步的buffers, 使用buffer__sync__done探针时得到. 如果两者不相等, 说明其他进程也在flush buffer.
checkpoint探针一般可用于收集checkpoint各个部分的时间开销. checkpoint频度. 各个部分的详细输出等.
结合内核探针, 可以得知checkpoint每个部分带来的io开销.
例子就不举了, 可以参考 :
http://blog.163.com/digoal@126/blog/static/163877040201391684012713/
探针详情 :
| name | parameter | desc |
|---|---|---|
| checkpoint-start | (int) | Probe that fires when a checkpoint is started. arg0 holds the bitwise flags used to distinguish different checkpoint types, such as shutdown, immediate or force. |
| checkpoint-done | (int, int, int, int, int) | Probe that fires when a checkpoint is complete. (The probes listed next fire in sequence during checkpoint processing.) arg0 is the number of buffers written. arg1 is the total number of buffers. arg2, arg3 and arg4 contain the number of xlog file(s) added, removed and recycled respectively. |
| clog-checkpoint-start | (bool) | Probe that fires when the CLOG portion of a checkpoint is started. arg0 is true for normal checkpoint, false for shutdown checkpoint. |
| clog-checkpoint-done | (bool) | Probe that fires when the CLOG portion of a checkpoint is complete. arg0 has the same meaning as for clog-checkpoint-start. |
| subtrans-checkpoint-start | (bool) | Probe that fires when the SUBTRANS portion of a checkpoint is started. arg0 is true for normal checkpoint, false for shutdown checkpoint. |
| subtrans-checkpoint-done | (bool) | Probe that fires when the SUBTRANS portion of a checkpoint is complete. arg0 has the same meaning as for subtrans-checkpoint-start. |
| multixact-checkpoint-start | (bool) | Probe that fires when the MultiXact portion of a checkpoint is started. arg0 is true for normal checkpoint, false for shutdown checkpoint. |
| multixact-checkpoint-done | (bool) | Probe that fires when the MultiXact portion of a checkpoint is complete. arg0 has the same meaning as for multixact-checkpoint-start. |
| buffer-checkpoint-start | (int) | Probe that fires when the buffer-writing portion of a checkpoint is started. arg0 holds the bitwise flags used to distinguish different checkpoint types, such as shutdown, immediate or force. |
| buffer-sync-start | (int, int) | Probe that fires when we begin to write dirty buffers during checkpoint (after identifying which buffers must be written). arg0 is the total number of buffers. arg1 is the number that are currently dirty and need to be written. |
| buffer-sync-written | (int) | Probe that fires after each buffer is written during checkpoint. arg0 is the ID number of the buffer. |
| buffer-sync-done | (int, int, int) | Probe that fires when all dirty buffers have been written. arg0 is the total number of buffers. arg1 is the number of buffers actually written by the checkpoint process. arg2 is the number that were expected to be written (arg1 of buffer-sync-start); any difference reflects other processes flushing buffers during the checkpoint. |
| buffer-checkpoint-sync-start | () | Probe that fires after dirty buffers have been written to the kernel, and before starting to issue fsync requests. |
| buffer-checkpoint-done | () | Probe that fires when syncing of buffers to disk is complete. |
| twophase-checkpoint-start | () | Probe that fires when the two-phase portion of a checkpoint is started. |
| twophase-checkpoint-done | () | Probe that fires when the two-phase portion of a checkpoint is complete. |
探针信息见本文末尾, 探针相关源码文件 :
src/backend/storage/buffer/bufmgr.c
src/backend/access/transam/xlog.c
src/backend/access/transam/clog.c
src/backend/access/transam/subtrans.c
src/backend/access/transam/multixact.c
src/backend/access/transam/twophase.c
针对探针中出现的变量, 从源码中抽取出来简单的解释一下 :
checkpoint的几种类型以bitwise形式定义区分, 源码如下 :
/*
* Perform a checkpoint --- either during shutdown, or on-the-fly
*
* flags is a bitwise OR of the following:
* CHECKPOINT_IS_SHUTDOWN: checkpoint is for database shutdown.
* CHECKPOINT_END_OF_RECOVERY: checkpoint is for end of WAL recovery.
* CHECKPOINT_IMMEDIATE: finish the checkpoint ASAP,
* ignoring checkpoint_completion_target parameter.
* CHECKPOINT_FORCE: force a checkpoint even if no XLOG activity has occurred
* since the last one (implied by CHECKPOINT_IS_SHUTDOWN or
* CHECKPOINT_END_OF_RECOVERY).
*
* Note: flags contains other bits, of interest here only for logging purposes.
* In particular note that this routine is synchronous and does not pay
* attention to CHECKPOINT_WAIT.
*
* If !shutdown then we are writing an online checkpoint. This is a very special
* kind of operation and WAL record because the checkpoint action occurs over
* a period of time yet logically occurs at just a single LSN. The logical
* position of the WAL record (redo ptr) is the same or earlier than the
* physical position. When we replay WAL we locate the checkpoint via its
* physical position then read the redo ptr and actually start replay at the
* earlier logical position. Note that we don't write *anything* to WAL at
* the logical position, so that location could be any other kind of WAL record.
* All of this mechanism allows us to continue working while we checkpoint.
* As a result, timing of actions is critical here and be careful to note that
* this function will likely take minutes to execute on a busy system.
*/
void
CreateCheckPoint(int flags)
{
...
TRACE_POSTGRESQL_CHECKPOINT_START(flags);
checkpoint类型定义 :
src/include/access/xlog.h
/*
* OR-able request flag bits for checkpoints. The "cause" bits are used only
* for logging purposes. Note: the flags must be defined so that it's
* sensible to OR together request flags arising from different requestors.
*/
/* These directly affect the behavior of CreateCheckPoint and subsidiaries */
#define CHECKPOINT_IS_SHUTDOWN 0x0001 /* Checkpoint is for shutdown */
#define CHECKPOINT_END_OF_RECOVERY 0x0002 /* Like shutdown checkpoint,
* but issued at end of WAL
* recovery */
#define CHECKPOINT_IMMEDIATE 0x0004 /* Do it without delays */
#define CHECKPOINT_FORCE 0x0008 /* Force even if no activity */
/* These are important to RequestCheckpoint */
#define CHECKPOINT_WAIT 0x0010 /* Wait for completion */
/* These indicate the cause of a checkpoint request */
#define CHECKPOINT_CAUSE_XLOG 0x0020 /* XLOG consumption */
#define CHECKPOINT_CAUSE_TIME 0x0040 /* Elapsed time */
checkpoint__done探针的变量信息 :
TRACE_POSTGRESQL_CHECKPOINT_DONE(CheckpointStats.ckpt_bufs_written,
NBuffers,
CheckpointStats.ckpt_segs_added,
CheckpointStats.ckpt_segs_removed,
CheckpointStats.ckpt_segs_recycled);
src/include/storage/buf.h
NBuffers指shared buffersz总的pages数. 使用gdb可以输出NBuffers的值, 可以证实这一点.
/*
* Buffer identifiers.
*
* Zero is invalid, positive is the index of a shared buffer (1..NBuffers),
* negative is the index of a local buffer (-1 .. -NLocBuffer).
*/
typedef int Buffer;
在测试过程中遇到1个BUG, 所有和checkpoint__done探针中相关的变量, 只要输出就会报错. 缺乏对应的debuginfo.
已经提交bug报告, 等待回复.
例如 :
[root@db-172-16-3-150 postgresql-9.3.1]# stap --vp 10000 -e 'probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {println($$vars)}'
Pass 1: parsed user script and 96 library script(s) using 152024virt/25212res/2104shr/23932data kb, in 230usr/20sys/252real ms.
WARNING: Can't parse SDT_V3 operand 'CheckpointStats+40(%rip)': identifier '$$vars' at <input>:1:87
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {println($$vars)}
^
WARNING: Can't parse SDT_V3 operand 'NBuffers(%rip)': identifier '$$vars' at :1:87
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {println($$vars)}
^
WARNING: Can't parse SDT_V3 operand 'CheckpointStats+44(%rip)': identifier '$$vars' at :1:87
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {println($$vars)}
^
WARNING: Can't parse SDT_V3 operand 'CheckpointStats+48(%rip)': identifier '$$vars' at :1:87
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {println($$vars)}
^
WARNING: Can't parse SDT_V3 operand 'CheckpointStats+52(%rip)': identifier '$$vars' at :1:87
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {println($$vars)}
^
semantic error: unable to find local 'arg1', [man error::dwarf] dieoffset 0xd00f5 in /home/pg93/pgsql9.3.1/bin/postgres, near pc 0x4b9789 in CreateCheckPoint xlog.c (alternatives: $flags $shutdown $checkPoint $recptr $Insert $rdata $freespace $_logSegNo $vxids $nvxids $__func__): identifier '$$vars' at :1:87
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {println($$vars)}
^
semantic error: SDT asm not understood, requires debuginfo: identifier '$$vars' at :1:87
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {println($$vars)}
^
Pass 2: analysis failed. [man error::pass2]
[root@db-172-16-3-150 postgresql-9.3.1]# stap --vp 10000 -e 'probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("buffer__syn_done") {println($$vars)}'
Pass 1: parsed user script and 96 library script(s) using 152024virt/25212res/2104shr/23932data kb, in 230usr/20sys/252real ms.
WARNING: Can't parse SDT_V3 operand 'NBuffers(%rip)': identifier '$$vars' at <input>:1:89
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("buffer__sync__done") {println($$vars)}
^
semantic error: unable to find local 'arg1', [man error::dwarf] dieoffset 0x43a1d1 in /home/pg93/pgsql9.3.1/bin/postgres, near pc 0x630697 in <unknown> bufmgr.c (alternatives: $buf_id $num_to_scan $num_to_write $num_written $mask): identifier '$$vars' at :1:89
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("buffer__sync__done") {println($$vars)}
^
semantic error: SDT asm not understood, requires debuginfo: identifier '$$vars' at :1:89
source: probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("buffer__sync__done") {println($$vars)}
^
Pass 2: analysis failed. [man error::pass2]
CHECKPOINT_DONE探针中输出的CheckpointStats, 数据类型定义如下 :
src/include/access/xlog.h
/* Checkpoint statistics */
typedef struct CheckpointStatsData
{
TimestampTz ckpt_start_t; /* start of checkpoint */
TimestampTz ckpt_write_t; /* start of flushing buffers */
TimestampTz ckpt_sync_t; /* start of fsyncs */
TimestampTz ckpt_sync_end_t; /* end of fsyncs */
TimestampTz ckpt_end_t; /* end of checkpoint */
int ckpt_bufs_written; /* # of buffers written */
int ckpt_segs_added; /* # of new xlog segments created */
int ckpt_segs_removed; /* # of xlog segments deleted */
int ckpt_segs_recycled; /* # of xlog segments recycled */
int ckpt_sync_rels; /* # of relations synced */
uint64 ckpt_longest_sync; /* Longest sync for one relation */
uint64 ckpt_agg_sync_time; /* The sum of all the individual sync
* times, which is not necessarily the
* same as the total elapsed time for
* the entire sync phase. */
} CheckpointStatsData;
使用$$locals可以输出一些本地变量的值.
[root@db-172-16-3-150 ~]# stap -e '
probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {
println($$locals)
}'
shutdown=? checkPoint={...} recptr=0x603d5cc58 Insert=0x7faaa73e7c00 rdata={...} freespace=? _logSegNo=? vxids=? nvxids=0x0 __func__=[...]
[root@db-172-16-3-150 ~]# stap -e '
probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {
println($recptr$$)
}'
25834147224
[root@db-172-16-3-150 ~]# stap -e '
probe process("/home/pg93/pgsql9.3.1/bin/postgres").mark("checkpoint__done") {
println($Insert$$)
}'
{.PrevRecord=25834147760, .curridx=0, .currpage=0x7faaa73ec000, .currpos="", .RedoRecPtr=25834147704, .forcePageWrites='\000', .fullPageWrites='\001', .exclusiveBackup='\000', .nonExclusiveBackups=0, .lastBackupStart=0}
对于这个BUG的处理请参考另一篇blog.
http://blog.163.com/digoal@126/blog/static/163877040201391883345365/
http://blog.163.com/digoal@126/blog/static/1638770402013918103315822/
参考
1. http://www.postgresql.org/docs/9.3/static/dynamic-trace.html
2.
src/backend/storage/buffer/bufmgr.c
src/backend/access/transam/xlog.c
src/backend/access/transam/clog.c
src/backend/access/transam/subtrans.c
src/backend/access/transam/multixact.c
src/backend/access/transam/twophase.c
src/include/access/xlog.h
2. checkpoint相关探针信息 :
/* TRACE_POSTGRESQL_BUFFER_CHECKPOINT_START ( int) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_BUFFER_CHECKPOINT_START_ENABLED() __builtin_expect (buffer__checkpoint__start_semaphore, 0)
#define postgresql_buffer__checkpoint__start_semaphore buffer__checkpoint__start_semaphore
#else
#define TRACE_POSTGRESQL_BUFFER_CHECKPOINT_START_ENABLED() __builtin_expect (postgresql_buffer__checkpoint__start_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_buffer__checkpoint__start_semaphore __attribute__ ((unused)) __attribute__ ((section
(".probes")));
#define TRACE_POSTGRESQL_BUFFER_CHECKPOINT_START(arg1) \
DTRACE_PROBE1(postgresql,buffer__checkpoint__start,arg1)
/* TRACE_POSTGRESQL_BUFFER_CHECKPOINT_SYNC_START () */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_BUFFER_CHECKPOINT_SYNC_START_ENABLED() __builtin_expect (buffer__checkpoint__sync__start_semaphore, 0)
#define postgresql_buffer__checkpoint__sync__start_semaphore buffer__checkpoint__sync__start_semaphore
#else
#define TRACE_POSTGRESQL_BUFFER_CHECKPOINT_SYNC_START_ENABLED() __builtin_expect (postgresql_buffer__checkpoint__sync__start_semapho
re, 0)
#endif
__extension__ extern unsigned short postgresql_buffer__checkpoint__sync__start_semaphore __attribute__ ((unused)) __attribute__ ((se
ction (".probes")));
#define TRACE_POSTGRESQL_BUFFER_CHECKPOINT_SYNC_START() \
DTRACE_PROBE(postgresql,buffer__checkpoint__sync__start)
/* TRACE_POSTGRESQL_BUFFER_CHECKPOINT_DONE () */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_BUFFER_CHECKPOINT_DONE_ENABLED() __builtin_expect (buffer__checkpoint__done_semaphore, 0)
#define postgresql_buffer__checkpoint__done_semaphore buffer__checkpoint__done_semaphore
#else
#define TRACE_POSTGRESQL_BUFFER_CHECKPOINT_DONE_ENABLED() __builtin_expect (postgresql_buffer__checkpoint__done_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_buffer__checkpoint__done_semaphore __attribute__ ((unused)) __attribute__ ((section (
".probes")));
#define TRACE_POSTGRESQL_BUFFER_CHECKPOINT_DONE() \
DTRACE_PROBE(postgresql,buffer__checkpoint__done)
/* TRACE_POSTGRESQL_CHECKPOINT_START ( int) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_CHECKPOINT_START_ENABLED() __builtin_expect (checkpoint__start_semaphore, 0)
#define postgresql_checkpoint__start_semaphore checkpoint__start_semaphore
#else
#define TRACE_POSTGRESQL_CHECKPOINT_START_ENABLED() __builtin_expect (postgresql_checkpoint__start_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_checkpoint__start_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_CHECKPOINT_START(arg1) \
DTRACE_PROBE1(postgresql,checkpoint__start,arg1)
/* TRACE_POSTGRESQL_CHECKPOINT_DONE ( int, int, int, int, int) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_CHECKPOINT_DONE_ENABLED() __builtin_expect (checkpoint__done_semaphore, 0)
#define postgresql_checkpoint__done_semaphore checkpoint__done_semaphore
#else
#define TRACE_POSTGRESQL_CHECKPOINT_DONE_ENABLED() __builtin_expect (postgresql_checkpoint__done_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_checkpoint__done_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_CHECKPOINT_DONE(arg1,arg2,arg3,arg4,arg5) \
DTRACE_PROBE5(postgresql,checkpoint__done,arg1,arg2,arg3,arg4,arg5)
/* TRACE_POSTGRESQL_CLOG_CHECKPOINT_START ( char) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_CLOG_CHECKPOINT_START_ENABLED() __builtin_expect (clog__checkpoint__start_semaphore, 0)
#define postgresql_clog__checkpoint__start_semaphore clog__checkpoint__start_semaphore
#else
#define TRACE_POSTGRESQL_CLOG_CHECKPOINT_START_ENABLED() __builtin_expect (postgresql_clog__checkpoint__start_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_clog__checkpoint__start_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_CLOG_CHECKPOINT_START(arg1) \
DTRACE_PROBE1(postgresql,clog__checkpoint__start,arg1)
/* TRACE_POSTGRESQL_CLOG_CHECKPOINT_DONE ( char) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_CLOG_CHECKPOINT_DONE_ENABLED() __builtin_expect (clog__checkpoint__done_semaphore, 0)
#define postgresql_clog__checkpoint__done_semaphore clog__checkpoint__done_semaphore
#else
#define TRACE_POSTGRESQL_CLOG_CHECKPOINT_DONE_ENABLED() __builtin_expect (postgresql_clog__checkpoint__done_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_clog__checkpoint__done_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_CLOG_CHECKPOINT_DONE(arg1) \
DTRACE_PROBE1(postgresql,clog__checkpoint__done,arg1)
/* TRACE_POSTGRESQL_SUBTRANS_CHECKPOINT_START ( char) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_SUBTRANS_CHECKPOINT_START_ENABLED() __builtin_expect (subtrans__checkpoint__start_semaphore, 0)
#define postgresql_subtrans__checkpoint__start_semaphore subtrans__checkpoint__start_semaphore
#else
#define TRACE_POSTGRESQL_SUBTRANS_CHECKPOINT_START_ENABLED() __builtin_expect (postgresql_subtrans__checkpoint__start_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_subtrans__checkpoint__start_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_SUBTRANS_CHECKPOINT_START(arg1) \
DTRACE_PROBE1(postgresql,subtrans__checkpoint__start,arg1)
/* TRACE_POSTGRESQL_SUBTRANS_CHECKPOINT_DONE ( char) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_SUBTRANS_CHECKPOINT_DONE_ENABLED() __builtin_expect (subtrans__checkpoint__done_semaphore, 0)
#define postgresql_subtrans__checkpoint__done_semaphore subtrans__checkpoint__done_semaphore
#else
#define TRACE_POSTGRESQL_SUBTRANS_CHECKPOINT_DONE_ENABLED() __builtin_expect (postgresql_subtrans__checkpoint__done_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_subtrans__checkpoint__done_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_SUBTRANS_CHECKPOINT_DONE(arg1) \
DTRACE_PROBE1(postgresql,subtrans__checkpoint__done,arg1)
/* TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_START ( char) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_START_ENABLED() __builtin_expect (multixact__checkpoint__start_semaphore, 0)
#define postgresql_multixact__checkpoint__start_semaphore multixact__checkpoint__start_semaphore
#else
#define TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_START_ENABLED() __builtin_expect (postgresql_multixact__checkpoint__start_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_multixact__checkpoint__start_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_START(arg1) \
DTRACE_PROBE1(postgresql,multixact__checkpoint__start,arg1)
/* TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_DONE ( char) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_DONE_ENABLED() __builtin_expect (multixact__checkpoint__done_semaphore, 0)
#define postgresql_multixact__checkpoint__done_semaphore multixact__checkpoint__done_semaphore
#else
#define TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_DONE_ENABLED() __builtin_expect (postgresql_multixact__checkpoint__done_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_multixact__checkpoint__done_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_MULTIXACT_CHECKPOINT_DONE(arg1) \
DTRACE_PROBE1(postgresql,multixact__checkpoint__done,arg1)
/* TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START () */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START_ENABLED() __builtin_expect (twophase__checkpoint__start_semaphore, 0)
#define postgresql_twophase__checkpoint__start_semaphore twophase__checkpoint__start_semaphore
#else
#define TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START_ENABLED() __builtin_expect (postgresql_twophase__checkpoint__start_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_twophase__checkpoint__start_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START() \
DTRACE_PROBE(postgresql,twophase__checkpoint__start)
/* TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE () */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE_ENABLED() __builtin_expect (twophase__checkpoint__done_semaphore, 0)
#define postgresql_twophase__checkpoint__done_semaphore twophase__checkpoint__done_semaphore
#else
#define TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE_ENABLED() __builtin_expect (postgresql_twophase__checkpoint__done_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_twophase__checkpoint__done_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE() \
DTRACE_PROBE(postgresql,twophase__checkpoint__done)
/* TRACE_POSTGRESQL_BUFFER_SYNC_START ( int, int) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_BUFFER_SYNC_START_ENABLED() __builtin_expect (buffer__sync__start_semaphore, 0)
#define postgresql_buffer__sync__start_semaphore buffer__sync__start_semaphore
#else
#define TRACE_POSTGRESQL_BUFFER_SYNC_START_ENABLED() __builtin_expect (postgresql_buffer__sync__start_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_buffer__sync__start_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_BUFFER_SYNC_START(arg1,arg2) \
DTRACE_PROBE2(postgresql,buffer__sync__start,arg1,arg2)
/* TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN ( int) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN_ENABLED() __builtin_expect (buffer__sync__written_semaphore, 0)
#define postgresql_buffer__sync__written_semaphore buffer__sync__written_semaphore
#else
#define TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN_ENABLED() __builtin_expect (postgresql_buffer__sync__written_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_buffer__sync__written_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(arg1) \
DTRACE_PROBE1(postgresql,buffer__sync__written,arg1)
/* TRACE_POSTGRESQL_BUFFER_SYNC_DONE ( int, int, int) */
#if defined STAP_SDT_V1
#define TRACE_POSTGRESQL_BUFFER_SYNC_DONE_ENABLED() __builtin_expect (buffer__sync__done_semaphore, 0)
#define postgresql_buffer__sync__done_semaphore buffer__sync__done_semaphore
#else
#define TRACE_POSTGRESQL_BUFFER_SYNC_DONE_ENABLED() __builtin_expect (postgresql_buffer__sync__done_semaphore, 0)
#endif
__extension__ extern unsigned short postgresql_buffer__sync__done_semaphore __attribute__ ((unused)) __attribute__ ((section (".probes")));
#define TRACE_POSTGRESQL_BUFFER_SYNC_DONE(arg1,arg2,arg3) \
DTRACE_PROBE3(postgresql,buffer__sync__done,arg1,arg2,arg3)