在 OceanBase 数据库的日常运维中,内存占用异常是极易引发业务故障的高频问题。近期我们遇到一例典型场景:某业务执行 insert all 语句时频繁报内存超限错误,租户 SqlExecutor 模块内存占用飙升至近 200G,直接触发 OCP 告警。今天就带大家拆解问题根源,给出落地可行的解决方案!
当insert all into tab语句中存在大量的into tab values子句,执行SQL报错ERROR 4013 (HY001): No memory or reach tenant memory limit,ocp告警中出现OBServer内存分配出错的告警信息。
告警详情:[OBServer 内存分配出错] 集群:yx_arm_clu01,主机:10.xx.xx.52,日志类型:observer,日志文件:/home/admin/oceanbase/
log/observer.log,日志级别:WARN,关键字=No memory or reach tenant memory
limit.*\[OOPS\]=,错误码=4013,日志详情=[2026-02-12 13:12:15.495254] WARN common_alloc (ob_tenant_ctx_allocator.cpp:464) [2490563][pnio1][T0][YB420A50A235-00062BA62D7FC190-0-0] [lt=6][errcode=-4013] No memory or reach tenant memory
limit([OOPS]=
"alloc failed reason", msg=tenant memory has reached the upper
limit(tenant_id: 1016, tenant_hold: 247389028352, tenant_limit: 247390116250, alloc_size: 2097152))。
OCP租户 SQL诊断 TOP SQL如下
SQL示例语句如下:
insert all into tab (col1, col2) values (
'a', 1) into tab (col1, col2) values (
'b', 2) into tab (col1, col2) values (
'c', 3) select * from dual;
OBServer日志内存信息如下,DEFAULT_CTX_ID的内存占用高达近200G。
[2026-02-12 11:46:18.507119] INFO [LIB] operator() (ob_malloc_allocator.cpp:567) [2490491][MemDumpTimer][T1016][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant: 1016,
limit: 247,390,116,250 hold: 231,218,413,568 rpc_hold: 0 ca
che_hold: 178,257,920 cache_used: 178,257,920 cache_item_count: 85
[MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes=208,756,539,392
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= MEMSTORE_CTX_ID hold_bytes= 1,679,818,752
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= TRANS_CTX_MGR_ID hold_bytes= 4,194,304
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= PLAN_CACHE_CTX_ID hold_bytes= 18,944,884,736
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= WORK_AREA hold_bytes= 100,663,296
limit= 49,478,023,240
[MEMORY] ctx_id= GLIBC hold_bytes= 153,092,096
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= CO_STACK hold_bytes= 603,979,776
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= META_OBJ_CTX_ID hold_bytes= 161,480,704
limit= 49,478,023,240
[MEMORY] ctx_id= TX_CALLBACK_CTX_ID hold_bytes= 538,968,064
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= LOB_CTX_ID hold_bytes= 2,097,152
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= PS_CACHE_CTX_ID hold_bytes= 12,582,912
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= RPC_CTX_ID hold_bytes= 2,097,152
limit= 247,390,116,200
[MEMORY] ctx_id= TX_DATA_TABLE hold_bytes= 71,368,704
limit= 9,223,372,036,854,775,807
[MEMORY] ctx_id= MDS_DATA_ID hold_bytes= 8,388,608
limit= 9,223,372,036,854,775,807
[MEMORY][PM] tid= 593265 used= 0 hold= 4,194,304 pm=0xffbfff30f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593266 used= 0 hold= 4,194,304 pm=0xffbfff39f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593269 used= 0 hold= 2,097,152 pm=0xffbfff19f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593274 used= 0 hold= 4,194,304 pm=0xffbffed0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593275 used= 0 hold= 4,194,304 pm=0xffbffed9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593281 used= 0 hold= 4,194,304 pm=0xffbffe99f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593280 used= 0 hold= 4,194,304 pm=0xffbffe90f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593283 used= 0 hold= 4,194,304 pm=0xffbffe70f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593285 used= 0 hold= 4,194,304 pm=0xffbffe46f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593286 used= 0 hold= 4,194,304 pm=0xffbffe50f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593288 used= 0 hold= 4,194,304 pm=0xffbffe26f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593289 used= 0 hold= 4,194,304 pm=0xffbffe30f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593290 used= 0 hold= 4,194,304 pm=0xffbffe39f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593291 used= 0 hold= 4,194,304 pm=0xffbffe06f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593292 used= 0 hold= 4,194,304 pm=0xffbffe10f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593294 used= 0 hold= 4,194,304 pm=0xffbffde6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593296 used= 0 hold= 4,194,304 pm=0xffbffdf9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593297 used= 0 hold= 4,194,304 pm=0xffbffdc6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593298 used= 0 hold= 4,194,304 pm=0xffbffdd0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593301 used= 0 hold= 4,194,304 pm=0xffbffdb0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593302 used= 18,780,424,224 hold= 18,935,578,624 pm=0xffbffdb9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593305 used= 0 hold= 4,194,304 pm=0xffbffd99f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593309 used= 0 hold= 4,194,304 pm=0xffbffd46f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593310 used= 0 hold= 4,194,304 pm=0xffbffd50f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593314 used= 0 hold= 4,194,304 pm=0xffbffd39f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593318 used= 0 hold= 4,194,304 pm=0xffbffce6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593319 used= 0 hold= 4,194,304 pm=0xffbffcf0f000 ctx_name=DEFAULT_CTX_ID
:[MEMORY][PM] tid= 593292 used= 0 hold= 4,194,304 pm=0xffbffe10f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593294 used= 0 hold= 4,194,304 pm=0xffbffde6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593296 used= 0 hold= 4,194,304 pm=0xffbffdf9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593297 used= 0 hold= 4,194,304 pm=0xffbffdc6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593298 used= 0 hold= 4,194,304 pm=0xffbffdd0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593301 used= 0 hold= 4,194,304 pm=0xffbffdb0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593302 used= 18,780,424,224 hold= 18,935,578,624 pm=0xffbffdb9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593305 used= 0 hold= 4,194,304 pm=0xffbffd99f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593309 used= 0 hold= 4,194,304 pm=0xffbffd46f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593310 used= 0 hold= 4,194,304 pm=0xffbffd50f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593314 used= 0 hold= 4,194,304 pm=0xffbffd39f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593318 used= 0 hold= 4,194,304 pm=0xffbffce6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593319 used= 0 hold= 4,194,304 pm=0xffbffcf0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593320 used= 0 hold= 4,194,304 pm=0xffbffcf9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593322 used= 0 hold= 4,194,304 pm=0xffbffcd0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593326 used= 0 hold= 4,194,304 pm=0xffbffcb9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593329 used= 0 hold= 4,194,304 pm=0xffbffc99f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593331 used= 0 hold= 4,194,304 pm=0xffbffc70f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593332 used= 0 hold= 4,194,304 pm=0xffbffc79f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593334 used= 0 hold= 4,194,304 pm=0xffbffc50f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593335 used= 0 hold= 4,194,304 pm=0xffbffc59f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593338 used= 0 hold= 4,194,304 pm=0xffbffc39f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593342 used= 0 hold= 4,194,304 pm=0xffbffbe6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593344 used= 0 hold= 4,194,304 pm=0xffbffbf9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593345 used= 0 hold= 4,194,304 pm=0xffbffbc6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593349 used= 0 hold= 4,194,304 pm=0xffbffbb0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593351 used= 0 hold= 4,194,304 pm=0xffbffb86f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593355 used= 95,993,336,496 hold= 96,785,399,808 pm=0xffbffb70f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593357 used= 37,978,587,072 hold= 38,292,488,192 pm=0xffbffb46f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593361 used= 0 hold= 4,194,304 pm=0xffbffb30f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593362 used= 25,274,339,360 hold= 25,483,083,776 pm=0xffbffb39f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593364 used= 0 hold= 4,194,304 pm=0xffbffb10f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593366 used= 0 hold= 4,194,304 pm=0xffbffae6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593367 used= 0 hold= 4,194,304 pm=0xffbffaf0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593368 used= 0 hold= 4,194,304 pm=0xffbffaf9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593369 used= 0 hold= 4,194,304 pm=0xffbffac6f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593371 used= 0 hold= 4,194,304 pm=0xffbffad9f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593373 used= 0 hold= 4,194,304 pm=0xffbffab0f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593375 used= 0 hold= 4,194,304 pm=0xffbffa86f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593380 used= 0 hold= 4,194,304 pm=0xffbffa79f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593383 used= 0 hold= 4,194,304 pm=0xffbffa59f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593384 used= 14,166,628,288 hold= 14,283,374,592 pm=0xffbffa26f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593385 used= 0 hold= 4,194,304 pm=0xffbffa30f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593386 used= 8,319,744 hold= 8,388,608 pm=0xffbffa39f000 ctx_name=DEFAULT_CTX_ID
[MEMORY][PM] tid= 593391 used= 0 hold= 4,194,304 pm=0xffbff9f0f000 ctx_name=DEFAULT_CTX_ID
集群sys租户内存性能视图GV$OB_MEMORY结果如下
MySQL [oceanbase]> select * from GV
$OB_MEMORY
where tenant_id=1016 order by used;
+-----------+--------------+----------+-------------------------------+-----------------+---------+--------------+--------------+
| TENANT_ID | SVR_IP | SVR_PORT | CTX_NAME | MOD_NAME | COUNT | HOLD | USED |
+-----------+--------------+----------+-------------------------------+-----------------+---------+--------------+--------------+
| 1016 | 10.xx.xx.52 | 2882 | UNEXPECTED_IN_500 | KvstorCacheMb | 0 | 0 | 0 |
| 1016 | 10.xx.xx.52 | 2882 | TX_CALLBACK_CTX_ID | KvstorCacheMb | 0 | 0 | 0 |
| 1016 | 10.xx.xx.52 | 2882 | TX_CALLBACK_CTX_ID | KvstorCacheMb | 0 | 0 | 0 |
...
| 1016 | 10.xx.xx.52 | 2882 | MEMSTORE_CTX_ID | Memstore | 516 | 1073676288 | 1073185056 |
| 1016 | 10.xx.xx.52 | 2882 | PLAN_CACHE_CTX_ID | SqlPhyPlan | 305793 | 6853178816 | 6360646353 |
| 1016 | 10.xx.xx.52 | 2882 | MEMSTORE_CTX_ID | Memstore | 4879 | 10152067072 | 10147422264 |
| 1016 | 10.xx.xx.52 | 2882 | DEFAULT_CTX_ID | MysqlRequesReco | 4895 | 10261233664 | 10256201088 |
| 1016 | 10.xx.xx.52 | 2882 | MEMSTORE_CTX_ID | Memstore | 5122 | 10657693696 | 10652817552 |
| 1016 | 10.xx.xx.52 | 2882 | DEFAULT_CTX_ID | MysqlRequesReco | 5226 | 10949967872 | 10944596352 |
| 1016 | 10.xx.xx.52 | 2882 | DEFAULT_CTX_ID | MysqlRequesReco | 5273 | 11047763968 | 11042344320 |
| 1016 | 10.xx.xx.52 | 2882 | PLAN_CACHE_CTX_ID | SqlPhyPlan | 1010678 | 17699686016 | 16976458135 |
| 1016 | 10.xx.xx.52 | 2882 | PLAN_CACHE_CTX_ID | SqlPhyPlObj | 815592 | 19348716992 | 19077125294 |
| 1016 | 10.xx.xx.52 | 2882 | DEFAULT_CTX_ID | SqlExecutor | 94760 | 199055097856 | 198953370500 |
+-----------+--------------+----------+-------------------------------+-----------------+---------+--------------+--------------+
878 rows
in
set (0.010 sec)
SQL执行时发现SqlExecutor模块的内存出现膨胀,随着会话连接的关闭,内存也会进行释放。
gv$ob_sql_audit中SQL执行耗时GET_PLAN_TIME的值较大,接近ELAPSED_TIME的时间,REQUEST_MEMORY_USED的内存值达到了90G。
select * from gv
$ob_sql_audit
where sql_id
in (
'66B25AF9022FD543841E0CF9A1934B53',
'CC8F9BD89C362010C9A1D07E4DB54A82')
limit 10\G;
*************************** 3. row ***************************
SVR_IP: 10.xx.xx.52
SVR_PORT: 2882
REQUEST_ID: 11626260613
SQL_EXEC_ID: 18113079752
TRACE_ID: YB420A50A234-00062AD3381FAF22-0-0
SID: 3221811103
CLIENT_IP: 10.xx.xx.52
CLIENT_PORT: 22156
TENANT_ID: 1016
TENANT_NAME: xxx_t02
EFFECTIVE_TENANT_ID: 1016
USER_ID: 515777
USER_NAME: xxx_user
USER_GROUP: 0
USER_CLIENT_IP: 192.xx.xx.23
DB_ID: 515778
DB_NAME: xxx_DBO
SQL_ID: 66B25AF9022FD543841E0CF9A1934B53
QUERY_SQL: insert all
into TBL_SCM_FLOW_TOUCH_QUOTA (LIST_ID,ONE_ID,NODE_ID,ACTIVITY_CODE) values
...
into TBL_SCM_FLOW_TOUCH_QUOTA (LIST_ID,ONE_ID,NODE_ID,ACTIVITY_CODE) values
( 7592334943,
'cabee4de-3c3b-412b-a73f-26b1486ba805',
'23612587c9ae441895077e67c8d43954',
'52b701b93
PLAN_ID: 0
AFFECTED_ROWS: 0
RETURN_ROWS: 0
PARTITION_CNT: 0
RET_CODE: -5066
QC_ID: 0
DFO_ID: 0
SQC_ID: 0
WORKER_ID: 0
EVENT:
P1TEXT:
P1: 0
P2TEXT:
P2: 0
P3TEXT:
P3: 0
LEVEL: 0
WAIT_CLASS_ID: 100
WAIT_CLASS#: 0
WAIT_CLASS: OTHER
STATE: MAX_WAIT TIME ZERO
WAIT_TIME_MICRO: 0
TOTAL_WAIT_TIME_MICRO: 0
TOTAL_WAITS: 0
RPC_COUNT: 0
PLAN_TYPE: 0
IS_INNER_SQL: 0
IS_EXECUTOR_RPC: 0
IS_HIT_PLAN: 0
REQUEST_TIME: 1770826483379082
ELAPSED_TIME: 48045733798
NET_TIME: 0
NET_WAIT_TIME: 8
QUEUE_TIME: 61
DECODE_TIME: 1
GET_PLAN_TIME: 48045713897
EXECUTE_TIME: 76
APPLICATION_WAIT_TIME: 0
CONCURRENCY_WAIT_TIME: 0
USER_IO_WAIT_TIME: 0
SCHEDULE_TIME: 0
ROW_CACHE_HIT: 0
BLOOM_FILTER_CACHE_HIT: 0
BLOCK_CACHE_HIT: 0
DISK_READS: 0
RETRY_CNT: 0
TABLE_SCAN: 0
CONSISTENCY_LEVEL: -1
MEMSTORE_READ_ROW_COUNT: 0
SSSTORE_READ_ROW_COUNT: 0
DATA_BLOCK_READ_CNT: 0
DATA_BLOCK_CACHE_HIT: 0
INDEX_BLOCK_READ_CNT: 0
INDEX_BLOCK_CACHE_HIT: 0
BLOCKSCAN_BLOCK_CNT: 0
BLOCKSCAN_ROW_CNT: 0
PUSHDOWN_STORAGE_FILTER_ROW_CNT: 0
REQUEST_MEMORY_USED: 98880409680
EXPECTED_WORKER_COUNT: 0
USED_WORKER_COUNT: 0
SCHED_INFO: NULL
FUSE_ROW_CACHE_HIT: 0
PS_CLIENT_STMT_ID: -1
PS_INNER_STMT_ID: -1
TX_ID: 0
SNAPSHOT_VERSION: 0
REQUEST_TYPE: 2
IS_BATCHED_MULTI_STMT: 0
OB_TRACE_INFO: NULL
PLAN_HASH: 0
LOCK_FOR_READ_TIME: 0
PARAMS_VALUE:
RULE_NAME:
PARTITION_HIT: 1
TX_INTERNAL_ROUTING: 0
TX_STATE_VERSION: 0
FLT_TRACE_ID:
NETWORK_WAIT_TIME: 0
3 rows in set (58.455 sec)
ERROR: No query specified
insert all into values 类型 SQL 后跟大量多行数据时,在硬解析时,内存膨胀,另外行数不固定,可能导致不能命中 plan_cache,出现并发硬解析时,同时占用内存就很大,导致租户内存被挤占。
数据库版本:OceanBase 4.2.1.10
租户模式:OB_Oracle
https://www.oceanbase.com/knowledge-base/oceanbase-database-1000000000690996
如果你的业务也遇到 OceanBase 内存异常、SQL 执行效率低等问题,欢迎在评论区留言交流,我们会持续分享更多实战排查案例!
欢迎 关注、 点赞、转发 ,后续我们会持续分享更多实战排查案例,下次见~

来自 “ ITPUB博客 ” ,链接:https://blog.itpub.net/70047467/viewspace-3113894/ ,如需转载,请注明出处,否则将追究法律责任。