首页 > 数据库 > 数据库开发技术 > XID耗尽模拟实验,可以更优雅点......

曾在前不久,整理过一篇XID耗尽的模拟实验 (耗尽PostgreSQL中的XID模拟实验),总觉来得不够优雅。为了营造不能VACUUM FREEZE的场景,生生把一个表文件给挪走。有没有不去做什么破坏,只是正常或者接近正常使用的情况下,慢慢把XID耗近的可能呢?
小波同学刚接触PG不久,感觉里边的临时表蛮有意思,反正是做做试验和试用,一些不那么重要的参数,干脆就关了:
archive_mode = off
log_filename = 'postgresql-%Y-%m-%d.log'
logging_collector = on
log_directory = 'log'
他顺便也记录了下机器的基本配置,是一台VM:
RELS7 + PostgreSQL 14.8
/proc/cpuinfo:
Intel(R) Xeon(R) Platinum 8260 CPU @ 2.40GHz 4Core memory: 8G
这次,他偷了个懒,建了一个DB:mydb, 然后新起了一个连接,就建一张普通的临时表,然后把那个连接就挂在那里了。
psql mydb
create temp table t(id int);
ctrl + z
// 进入后台模式。保持那个连接会话还在那里
然后,这边呢,他开始尝试用pgbench疯狂加压,每次操作一个事务,就只是获取当前的事务ID。
script.sql内容如下:
begin;
select pg_current_xact_id();
-- 或者 select txid_current();
end;
接着在命令行窗口,进行漫长的并发加压,目的很简单,就是让XID跑完。
nohup pgbench -c 90 -j 90 -n -f script.sql -T 72000 mydb &
意思比较明了,给俺跑20个小时。
他又用相关的SQL语句以及\watch 60来监控中间的XID值的结果:
select oid,datname,datfrozenxid, age(datfrozenxid) from pg_database where datname='mydb';
oid | datname | datfrozenxid | age
-------+---------+--------------+-----
16384 | mydb | 726 | 15
.................
oid | datname | datfrozenxid | age
-------+---------+--------------+------------
16384 | mydb | 125177483 | 2144483732
(1 row)
Mon 06 Nov 2023 11:53:34 AM UTC (every 60s)
oid | datname | datfrozenxid | age
-------+---------+--------------+------------
16384 | mydb | 125177483 | 2144483732
(1 row)
按照他的监测,发现在那个给定的系统条件下,一分钟,差不多300 - 360万个XID会产生。于是,差不多10-11个小时,就跑到21亿条的临界了。
这个时候,再看看PG的server log:
2023-11-06 11:49:02.011 UTC [13122] WARNING: database "mydb" must be vacuumed within 2999915 transactions
2023-11-06 11:49:02.011 UTC [13122] HINT: To avoid a database shutdown, execute a database-wide VACUUM in that database.
You might also need to commit or roll back old prepared transactions, or drop stale replication slots.
2023-11-06 11:49:17.027 UTC [13135] WARNING: database "mydb" must be vacuumed within 2999915 transactions
2023-11-06 11:49:17.027 UTC [13135] HINT: To avoid a database shutdown, execute a database-wide VACUUM in that database.
You might also need to commit or roll back old prepared transactions, or drop stale replication slots.
2023-11-06 11:49:32.043 UTC [13149] WARNING: database "mydb" must be vacuumed within 2999915 transactions
2023-11-06 11:49:32.043 UTC [13149] HINT: To avoid a database shutdown, execute a database-wide VACUUM in that database.
You might also need to commit or roll back old prepared transactions, or drop stale replication slots.
重启以后,也会发现下边的LOG:
2023-11-06 11:55:26.267 UTC [13489] HINT: To avoid a database shutdown, execute a database-wide VACUUM in that database.
You might also need to commit or roll back old prepared transactions, or drop stale replication slots.
2023-11-06 11:55:26.271 UTC [13487] LOG: database system is ready to accept connections
2023-11-06 11:55:41.312 UTC [13511] ERROR: database is not accepting commands to avoid wraparound data loss in database "mydb"
2023-11-06 11:55:41.312 UTC [13511] HINT: Stop the postmaster and vacuum that database in single-user mode.
You might also need to commit or roll back old prepared transactions, or drop stale replication slots.
2023-11-06 11:55:56.298 UTC [13524] ERROR: database is not accepting commands to avoid wraparound data loss in database "mydb"
2023-11-06 11:55:56.298 UTC [13524] HINT: Stop the postmaster and vacuum that database in single-user mode.
You might also need to commit or roll back old prepared transactions, or drop stale replication slots.
2023-11-06 11:56:11.313 UTC [13537] ERROR: database is not accepting commands to avoid wraparound data loss in database "mydb"
2023-11-06 11:56:11.313 UTC [13537] HINT: Stop the postmaster and vacuum that database in single-user mode.
You might also need to commit or roll back old prepared transactions, or drop stale replication slots.
里边有一句重要的提示:
HINT: Stop the postmaster and vacuum that database in single-user mode.
不过,出现这些LOG的时候,可能SERVER LOG也因为不断的ERROR, HINT,导致累积,可能达到10来个G的大小。
到此,小波有点小兴奋,好像一张普通的临时表,如果得不到及时的释放(释放会话连接,会自动DROP),那么它会导致age越来越大,最后也会出现XID耗光的情况。
后边的过程,就同前一次实验过程中的一样了。使用single-user模式进行vacuum修复。
小波经过一顿折腾,发现临时表的使用,还真的非常小心。比如,在一个会话连接里,建了若干临时表,结果连接忘了关了。也没有什么设置什么SESSION TIMEOUT。最后就把系统给玩坏了。加强必要的监控还是非常重要的。
当然,如果有64位XID,就不会那么快耗尽了。

原文作者:数据库杂记;如有侵权,请联系管理员删除。