ITPub博客

首页 > 数据库 > 国内数据库 > OceanBase 内存报错 - 11049 与‑4013 :单 SQL 超限 vs 租户内存打满

OceanBase 内存报错 - 11049 与‑4013 :单 SQL 超限 vs 租户内存打满

原创 国内数据库 作者:瑞蓝创_RLC_888 时间:2026-08-13 15:28:52 0 删除 编辑

生产环境经常遇到内存类报错,同样是 OOM,-11049 和‑4013 根因、排查路径、处理方案完全不一样。本文结合真实生产案例,帮大家分清两个报错,掌握完整定位与优化手段。

一、报错‑11049

query_memory_limit_percentage用于指定 单条 SQL 可使用的租户内存百分比。当内存使用超过指定的阈值后,就会报错-11049并中断该 SQL 的执行。如果租户规格为8C32G, query_memory_limit_percentage参数值为10%,则单条SQL可使用的内存上限为3.2G。

对于 V4.3.x 版本,该配置项从 V4.3.5 版本开始引入。

对于 V4.2.x 版本,该配置项从 V4.2.5 版本开始引入。

前台应用报错如下


Caused by: java.sql.SQLException: Exceed query memory 
limit (mem_limit=3382286745, mem_hold=3382636544),  please check whether the query_memory_limit_percentage configuration item is reasonable.

at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:130)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:916)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeQuery(ClientPreparedStatement.java:972)
at com.alibaba.druid.pool.DruidPooledPreparedStatement.executeQuery(DruidPooledPreparedStatement.java:213)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:57)

server日志中的表现如下


[errcode=-11049] Exceed query memory 
limit (mem_limit=3382286745, mem_hold=3382636544),  please check whether the query_memory_limit_percentage configuration item is reasonable.

这类问题首先是考虑能否适当调大 query_memory_limit_percentage(业务租户可修改)


ALTER SYSTEM SET query_memory_limit_percentage = xx;

在 query_memory_limit_percentage值已经较大,或遇特殊情况无法进行调整的情况下,定位到问题SQL之后我们可以从audit中的 REQUEST_MEMORY_USED字段,推算该SQL所需的内存大小

如6579103857/1024/1024/1024约是6.1G


ocp_common_ro@[oceanbase]>select * from gv
$ob_sql_audit 
where sql_id=
'63CB94990FCB456DFE3A1FDAB3EAEE28' \G;

*************************** 1. row ***************************
                         SVR_IP: xxx

                       SVR_PORT: 2882

                     REQUEST_ID: 2242026936

                    SQL_EXEC_ID: 152874360

                       TRACE_ID: YB420A6B0B54-00064110B24B6899-0-0

                            SID: 3222942919

                      CLIENT_IP: xxxx

                    CLIENT_PORT: xxx

                      TENANT_ID: 1002

                    TENANT_NAME: 

            EFFECTIVE_TENANT_ID: 1002

                        USER_ID: 500004

                      USER_NAME: 

                     USER_GROUP: 0

                 USER_CLIENT_IP: xxx

                          DB_ID: 500007

                        DB_NAME: 

                         SQL_ID: 5ADE23A64BBAD3EF2837BE40EEE60BD8

                      QUERY_SQL: 

                        PLAN_ID: 17318365

                  AFFECTED_ROWS: 0

                    RETURN_ROWS: 0

                  PARTITION_CNT: 0

                       RET_CODE: 0

                          QC_ID: 0

                         DFO_ID: 0

                         SQC_ID: 0

                      WORKER_ID: 0

                          EVENT: db file data  read

                         P1TEXT: schedule_queue_delay

                             P1: 421902

                         P2TEXT: device_delay

                             P2: 445

                         P3TEXT: callback_delay

                             P3: 123

                          LEVEL: 0

                  WAIT_CLASS_ID: 108

                    WAIT_CLASS #: 8

                     WAIT_CLASS: USER_IO

                          STATE: WAITED KNOWN TIME

                WAIT_TIME_MICRO: 421446

          TOTAL_WAIT_TIME_MICRO: 16073246

                    TOTAL_WAITS: 5879

                      RPC_COUNT: 1

                      PLAN_TYPE: 1

                   IS_INNER_SQL: 0

                IS_EXECUTOR_RPC: 1

                    IS_HIT_PLAN: 1

                   REQUEST_TIME: 1760968374389947

                   ELAPSED_TIME: 40739943

                       NET_TIME: 651

                  NET_WAIT_TIME: -1760968374389947

                     QUEUE_TIME: 1760968266397214

                    DECODE_TIME: 129

                  GET_PLAN_TIME: 0

                   EXECUTE_TIME: 148709879

          APPLICATION_WAIT_TIME: 0

          CONCURRENCY_WAIT_TIME: 0

              USER_IO_WAIT_TIME: 0

                  SCHEDULE_TIME: 0

                  ROW_CACHE_HIT: 0

         BLOOM_FILTER_CACHE_HIT: 0

                BLOCK_CACHE_HIT: 4697

                     DISK_READS: 19808

                      RETRY_CNT: 0

                     TABLE_SCAN: 0

              CONSISTENCY_LEVEL: 3

        MEMSTORE_READ_ROW_COUNT: 52745

         SSSTORE_READ_ROW_COUNT: 39313810

            DATA_BLOCK_READ_CNT: 50976

           DATA_BLOCK_CACHE_HIT: 22713

           INDEX_BLOCK_READ_CNT: 4

          INDEX_BLOCK_CACHE_HIT: 1710

            BLOCKSCAN_BLOCK_CNT: 37510

              BLOCKSCAN_ROW_CNT: 38477181

PUSHDOWN_STORAGE_FILTER_ROW_CNT: 0

            REQUEST_MEMORY_USED: 6579103857

          EXPECTED_WORKER_COUNT: 0

              USED_WORKER_COUNT: 0

                     SCHED_INFO: NULL

             FUSE_ROW_CACHE_HIT: 0

              PS_CLIENT_STMT_ID: -1

               PS_INNER_STMT_ID: -1

                          TX_ID: 0

               SNAPSHOT_VERSION: 1760968266416651000

                   REQUEST_TYPE: 3

          IS_BATCHED_MULTI_STMT: 0

                  OB_TRACE_INFO: NULL

                      PLAN_HASH: 0

             LOCK_FOR_READ_TIME: 0

                   PARAMS_VALUE: 

                      RULE_NAME: 

                  PARTITION_HIT: 1

            TX_INTERNAL_ROUTING: 0

               TX_STATE_VERSION: 0

                   FLT_TRACE_ID: 

                    PL_TRACE_ID: NULL

                PLSQL_EXEC_TIME: 0

                  FORMAT_SQL_ID: 

                      STMT_TYPE: SELECT

  TOTAL_MEMSTORE_READ_ROW_COUNT: 52745

   TOTAL_SSSTORE_READ_ROW_COUNT: 39313810

                     PROXY_USER: 

                        SEQ_NUM: 1760968415128431

              NETWORK_WAIT_TIME: 0

             PLSQL_COMPILE_TIME: 0

     INSERT_DUPLICATE_ROW_COUNT: 0

                    CCL_RULE_ID: 0

                 CCL_MATCH_TIME: 0

2 rows  in  set (3.035 sec)

然后可以使用下面的语句通过trace_id查看算子维度的计划详情,分析瓶颈




select svr_ip,plan_line_id,concat(lpad( ' ', plan_depth,  ' '), plan_operation) op, sum(STARTS) rescan, sum(output_rows) rowss, count(*) threads, min(first_refresh_time) open_time,max(last_refresh_time) close_time,min(first_change_time) first_row_time,max(last_change_time) last_row_eof_time,OTHERSTAT_3_VALUE as ssstore_row,OTHERSTAT_4_VALUE as memstore_row
from oceanbase.gv $sql_plan_monitor
where  trace_id =  'YB420A662F20-0006343D50BD3184-0-0' 
group by plan_line_id, plan_operation,plan_depth order by 1,2;
+--------------+-------------------------------+--------+----------+---------+----------------------------+----------------------------+----------------------------+----------------------------+-------------+--------------+
| plan_line_id | op                            | rescan | rowss    | threads | open_time                  | close_time                 | first_row_time             | last_row_eof_time          | ssstore_row | memstore_row |
+--------------+-------------------------------+--------+----------+---------+----------------------------+----------------------------+----------------------------+----------------------------+-------------+--------------+
|            0 | PHY_DIRECT_RECEIVE            |      0 |    10587 |       1 | 2025-10-20 21:51:06.395578 | 2025-10-20 21:53:35.149535 | 2025-10-20 21:52:54.367164 | 2025-10-20 21:53:35.149535 |           0 |            0 |
|            0 | PHY_VEC_MERGE_GROUP_BY        |      0 |    10587 |       1 | 2025-10-20 21:51:06.419801 | 2025-10-20 21:53:35.129484 | 2025-10-20 21:52:01.897629 | 2025-10-20 21:53:35.126282 |           0 |            0 |
|            1 |  PHY_SORT                     |      0 | 33018160 |       1 | 2025-10-20 21:51:06.419801 | 2025-10-20 21:53:35.127354 | 2025-10-20 21:51:58.621384 | 2025-10-20 21:53:35.124126 |     6425166 |     13762199 |
|            2 |   PHY_VEC_HASH_JOIN           |      0 | 33018160 |       1 | 2025-10-20 21:51:06.419801 | 2025-10-20 21:53:35.127354 | 2025-10-20 21:51:09.068365 | 2025-10-20 21:51:57.419166 |     1440336 |     16777216 |
|            3 |    PHY_VEC_TABLE_SCAN         |      0 |  6347821 |       1 | 2025-10-20 21:51:06.419801 | 2025-10-20 21:53:35.126282 | 2025-10-20 21:51:06.420865 | 2025-10-20 21:51:08.645433 |      260421 |         8073 |
|            4 |    PHY_VEC_HASH_JOIN          |      0 | 33018160 |       1 | 2025-10-20 21:51:06.419801 | 2025-10-20 21:53:35.127354 | 2025-10-20 21:51:09.068365 | 2025-10-20 21:51:57.412807 |          81 |         4096 |
|            5 |     PHY_VEC_SUBPLAN_SCAN      |      0 |      807 |       1 | 2025-10-20 21:51:06.419801 | 2025-10-20 21:53:35.126282 | 2025-10-20 21:51:09.066263 | 2025-10-20 21:51:09.066263 |           0 |            0 |
|            6 |      PHY_VEC_HASH_DISTINCT    |      0 |      807 |       1 | 2025-10-20 21:51:06.419801 | 2025-10-20 21:53:35.126282 | 2025-10-20 21:51:09.066263 | 2025-10-20 21:51:09.066263 |       32768 |            0 |
|            7 |       PHY_VALUES_TABLE_ACCESS |      0 |      807 |       1 | 2025-10-20 21:51:06.419801 | 2025-10-20 21:53:35.126282 | 2025-10-20 21:51:09.066263 | 2025-10-20 21:51:09.066263 |           0 |            0 |
|            8 |     PHY_VEC_TABLE_SCAN        |      0 | 33018160 |       1 | 2025-10-20 21:51:06.419801 | 2025-10-20 21:53:35.127354 | 2025-10-20 21:51:09.068365 | 2025-10-20 21:51:57.412807 |     1177348 |        44670 |
+--------------+-------------------------------+--------+----------+---------+----------------------------+----------------------------+----------------------------+----------------------------+-------------+--------------+

10 rows  in  set (0.207 sec)

这里主要是阻塞性算子merge group by+partition sort+hash join产生临时文件占用内存,存在优化空间,后续使用/*+parallel(2) */ 将计划变为分布式流式计划,内存消耗减少至1-2G

二、报错‑4013

ob_sql_work_area_percentage用于设置 SQL 工作区内存占整个租户内存百分比,默认为 5%,取值范围为 0 ~ 100。如果租户请求并发量较大,且每个请求占用的工作区内存比较多,就可能出现工作区内存不足的错误。


WARN  common_alloc (ob_tenant_ctx_allocator.cpp:465) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=2][errcode=-4013] No memory or reach tenant memory 
limit([OOPS]=
"alloc failed reason", msg=tenant memory has reached the upper 
limit(tenant_id: 1010, tenant_hold: 32845094912, tenant_limit: 32856499815, alloc_size: 16273408))

WDIAG common_alloc (ob_tenant_ctx_allocator.cpp:470) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=6][errcode=-4013] oops, alloc failed, tenant_id=1010, ctx_id=0, ctx_name=DEFAULT_CTX_ID, ctx_hold=31119998976, ctx_limit=9223372036854775807, tenant_hold=32845094912, tenant_limit=32856499815

参考 https://www.oceanbase.com/knowledge-base/oceanbase-database-1000000000704208?back=kb,日志中【tenant memory has reached the upper limit】对应的是 租户内存爆的情况!

使用下面的方法查看那个模块占用高


grep 
'\[MEMORY\] tenant: 1008' observer.log -A 10

如果有hold>limit的模块,则为可能有问题的模块

查看模块的哪个mod占用高:


grep 
'1008 ctx_id=           DEFAULT_CTX_ID' observer.log -A 10

如果有used>hold的mod,则为可能有问题的mod

这个报错比较典型的一个情况是大inlist,这里遇到的一个case是客户在421集群上执行大in查询报错-4013,in值3w+,尝试调整 ob_sql_work_area_percentage没有用,关键日志如下



$grep 
"YB420A663FF0-000646B37999723E-0-0" observer.log |grep 
"1010" |  tail -200

WDIAG [SQL.RESV] get_name_internal (ob_raw_expr.cpp:1085) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=12][errcode=-4019] fail to BUF_PRINTF(ret=-4019)
WDIAG [SQL.RESV] get_name (ob_raw_expr.cpp:344) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=12][errcode=-4019] fail to get_name(buf_len=1048576, pos=1048575, ret=-4019)
WDIAG [SQL.RESV] get_name_internal (ob_raw_expr.cpp:2402) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=5][errcode=-4019] fail to get_name(i=128627, ret=-4019)
WDIAG [SQL.RESV] get_name (ob_raw_expr.cpp:344) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4019] fail to get_name(buf_len=1048576, pos=1048575, ret=-4019)
WDIAG [SQL.RESV] get_name_internal (ob_raw_expr.cpp:2388) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4019] fail to BUF_PRINTF(ret=-4019)
WDIAG [SQL.RESV] get_name (ob_raw_expr.cpp:344) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=8][errcode=-4019] fail to get_name(buf_len=1048576, pos=1048575, ret=-4019)
WDIAG [SQL.OPT] get_plan_item_info (ob_log_table_scan.cpp:1216) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4019] failed to get plan item info(ret=-4019)
WDIAG [SQL] get_plan_tree_infos (ob_sql_plan.cpp:649) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=-4019] failed to get plan item info(ret=-4019)
WDIAG [SQL] get_plan_tree_infos (ob_sql_plan.cpp:660) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4019] failed to get child plan tree infos(ret=-4019)
WDIAG [SQL] get_plan_tree_infos (ob_sql_plan.cpp:660) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=2][errcode=-4019] failed to get child plan tree infos(ret=-4019)
WDIAG [SQL] get_sql_plan_infos (ob_sql_plan.cpp:602) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=2][errcode=-4019] failed to get plan tree infos(ret=-4019)
WDIAG [SQL] store_sql_plan (ob_sql_plan.cpp:92) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=2][errcode=-4019] failed to get sql plan infos(ret=-4019)
WDIAG [SQL] store_sql_plan (ob_sql_plan.cpp:99) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=2][errcode=-4019] failed to store sql plan(ret=-4019)
INFO  [SQL] generate_plan (ob_sql.cpp:3277) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=5] [SQL MEM USAGE] use too much memory(total_mem_used=2584030722, ObString(parse_result.input_sql_len_, parse_result.input_sql_)=select id, dimension_id, sum(use_count) as use_count
INFO  [SQL.PC] inner_add_cache_obj (ob_pcv_set.cpp:247) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=39] has identical pcv(is_same= true, pcv=0x7f66205ba4c0)
INFO  [SQL.PC] add_evolving_plan (ob_spm_evolution_plan.cpp:592) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=10] succ to add evolving plan(plan.get_plan_type()=1, plan_set_=0x7f66205bb240, ctx.sql_ctx_.spm_ctx_.bl_key_={db_id:500172, constructed_sql: "select id, dimension_id, sum(use_count) as use_count
WDIAG [SQL] handle_text_query (ob_sql.cpp:2582) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=50][errcode=-5434] fail to handle physical plan(ret=-5434)
WDIAG [SQL] stmt_query (ob_sql.cpp:222) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=12][errcode=-5434] fail to handle text query(stmt=select id, dimension_id, sum(use_count) as use_count
WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:962) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=32][errcode=-5434] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:0, local_retry_times:0, err_:-5434, err_:"
OB_SQL_RETRY_SPM ", retry_type:1, client_ret:-5434, is_interrupted_by_outer_query:false}, need_retry=true, THIS_WORKER.can_retry()=true, v.ctx_.multi_stmt_item_={is_part_of_multi_stmt:false, seq_num:0, sql:"select id, dimension_id, sum(use_count) as use_count
WDIAG [SERVER] do_process (obmp_query.cpp:759) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=32][errcode=-5434] run stmt_query failed, check  if need retry(ret=-5434, cli_ret=-5434, retry_ctrl_.need_retry()=1, sql=select id, dimension_id, sum(use_count) as use_count
WDIAG [SERVER] do_process (obmp_query.cpp:860) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=23][errcode=-5434] try to execute again(ret=-5434,  type=1, retry_type=1, timeout_remain=7107733790)
WDIAG print_tenant_memblock_info (ob_kvcache_store.cpp:790) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=0] [CACHE] len:     5490 tenant sync wash failed, cache memblock info: 
INFO  [COMMON] try_flush_washable_mb (ob_kvcache_store.cpp:660) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=28] can not find enough memory block to wash(ret=-4273, size_washed=37748736, size_need_washed=42131456)
INFO  [COMMON] try_flush_washable_mb (ob_kvcache_store.cpp:679) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=6] ObKVCache try flush washable memblock details(ret=-4273, force_flush= false, tenant_id=1010, cache_id=-1, size_washed=37748736, size_need_washed=42131456)
WDIAG [COMMON] alloc_chunk (ob_resource_mgr.cpp:86) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=5][errcode=-4273] sync_wash_mbs failed(ret=-4273, tenant_id=1010, wash_size=42131456)
WDIAG [COMMON] alloc_chunk (ob_resource_mgr.cpp:112) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=5][errcode=-4273] after wash from cache, still can 't alloc chunk from chunk_mgr, maybe alloc by other thread(size=37918432, wash_size=42131456, ret=-4273)
WDIAG alloc_block (ob_page_manager.h:197) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=0] oops, alloc failed, tenant_id=1010 ctx_id=0 hold=30852931584 limit=9223372036854775807
INFO  sync_wash (ob_tenant_ctx_allocator.cpp:390) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=5] [MEM][WASH] tenant_id: 1010, ctx_id: 5, washed_size: 204242944
WDIAG print_tenant_memblock_info (ob_kvcache_store.cpp:790) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=10][errcode=0] [CACHE] len:     5494 tenant sync wash failed, cache memblock info: 
INFO  [COMMON] try_flush_washable_mb (ob_kvcache_store.cpp:679) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=22] ObKVCache try flush washable memblock details(ret=0, force_flush=false, tenant_id=1010, cache_id=-1, size_washed=12582912, size_need_washed=12341248)
WDIAG print_tenant_memblock_info (ob_kvcache_store.cpp:790) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=6][errcode=0] [CACHE] len:     5494 tenant sync wash failed, cache memblock info: 
INFO  [COMMON] try_flush_washable_mb (ob_kvcache_store.cpp:660) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=16] can not find enough memory block to wash(ret=-4273, size_washed=0, size_need_washed=20467712)
INFO  [COMMON] try_flush_washable_mb (ob_kvcache_store.cpp:679) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=5] ObKVCache try flush washable memblock details(ret=-4273, force_flush=false, tenant_id=1010, cache_id=-1, size_washed=0, size_need_washed=20467712)
WDIAG [COMMON] alloc_chunk (ob_resource_mgr.cpp:86) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=-4273] sync_wash_mbs failed(ret=-4273, tenant_id=1010, wash_size=20467712)
WDIAG [COMMON] alloc_chunk (ob_resource_mgr.cpp:112) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=6][errcode=-4273] after wash from cache, still can'
t alloc chunk from chunk_mgr, maybe alloc by other thread(size=16253152, wash_size=20467712, ret=-4273)
WDIAG alloc_block (ob_page_manager.h:197) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=5][errcode=0] oops, alloc failed, tenant_id=1010 ctx_id=0 hold=31119998976  limit=9223372036854775807
WDIAG print_tenant_memblock_info (ob_kvcache_store.cpp:790) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=5][errcode=0] [CACHE] len:     5494 tenant sync wash failed, cache memblock info: 
INFO  [COMMON] try_flush_washable_mb (ob_kvcache_store.cpp:660) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=12] can not find enough memory block to wash(ret=-4273, size_washed=0, size_need_washed=20467712)
INFO  [COMMON] try_flush_washable_mb (ob_kvcache_store.cpp:679) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4] ObKVCache try flush washable memblock details(ret=-4273, force_flush= false, tenant_id=1010, cache_id=-1, size_washed=0, size_need_washed=20467712)
1WDIAG [COMMON] alloc_chunk (ob_resource_mgr.cpp:86) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=-4273] sync_wash_mbs failed(ret=-4273, tenant_id=1010, wash_size=20467712)
WDIAG [COMMON] alloc_chunk (ob_resource_mgr.cpp:112) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=-4273] after wash from cache, still can 't alloc chunk from chunk_mgr, maybe alloc by other thread(size=16253152, wash_size=20467712, ret=-4273)
WDIAG alloc_block (ob_page_manager.h:197) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=0] oops, alloc failed, tenant_id=1010 ctx_id=0 hold=31119998976 limit=9223372036854775807
WARN  common_alloc (ob_tenant_ctx_allocator.cpp:465) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=2][errcode=-4013] No memory or reach tenant memory limit([OOPS]="alloc failed reason", msg=tenant memory has reached the upper limit(tenant_id: 1010, tenant_hold: 32845094912, tenant_limit: 32856499815, alloc_size: 16273408))
WDIAG common_alloc (ob_tenant_ctx_allocator.cpp:470) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=6][errcode=-4013] oops, alloc failed, tenant_id=1010, ctx_id=0, ctx_name=DEFAULT_CTX_ID, ctx_hold=31119998976, ctx_limit=9223372036854775807, tenant_hold=32845094912, tenant_limit=32856499815
WDIAG alloc_new_page (page_arena.h:329) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=-4013] cannot allocate memory.sz=16252960, pages_=365,total_=2344598941
ERROR issue_dba_error (ob_log.cpp:1875) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-4013, file="ob_se_array.h", line_no=358, info="no memory")
EDIAG [LIB] reserve (ob_se_array.h:358) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=7][errcode=-4013] no memory(ret=-4013, new_size=16252928, block_size_=7936, count_=53463, capacity_=53463) BACKTRACE:0x13a165e3 0x5903775 0x5887485 0x588703f 0x5886f82 0x5886e0c 0x9f7dc74 0x9f7cec9 0x9f51c04 0xd5d06c3 0x5afe127 0x5afd542 0xd57455d 0xd57497e 0xc35627c 0xdaf7d8c 0xdaf4f28 0xe2b6ed0 0xe2ad43f 0x57b7044 0x57b3087 0x55a3a16 0xad0880b 0x558d830 0x5586341 0x55792ca 0x5563c2f 0xa9b1d74 0x13cd4e0f 0x7f6af23a6dc5 0x7f6af20d3ced
WDIAG [SQL.RESV] get_row_expr_param_type (ob_raw_expr_deduce_type.cpp:2367) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=41][errcode=-4013] push back param type failed(ret=-4013)
WDIAG [SQL.RESV] visit (ob_raw_expr_deduce_type.cpp:674) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=-4013] get row expr param type failed(ret=-4013)
WDIAG [SQL.RESV] postorder_accept (ob_raw_expr.cpp:514) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4013] visit failed(ret=-4013, type=136, name="T_OP_IN")
WDIAG [SQL.RESV] deduce_type (ob_raw_expr.cpp:399) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=12][errcode=-4013] fail to deduce(ret=-4013)
WDIAG [SQL.RESV] formalize (ob_raw_expr.cpp:418) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4013] failed to deduce type(*this={item_type:"T_OP_IN", result_type:{meta:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:1, scale:0}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_IN", "CNT_CONST_EXPR", "CNT_CONST", "CNT_COLUMN", "CNT_STATIC_PARAM", "CNT_IN"], rel_id:[1], children:[{item_type:"T_REF_COLUMN", result_type:{meta:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:11, scale:0}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[1], table_id:18446744073709551614, column_id:18, database_name:"sou_bi", table_name:"user_dimension_count", synonym_name:"", synonym_db_name:"", column_name:"dimension_id", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551584, udt_set_id:0}, {item_type:"T_OP_ROW", result_type:{meta:{type:"NULL", collation:"binary", coercibility:"IGNORABLE"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST_EXPR", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], children:[{item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":2}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":3}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":4}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":5}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":6}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":7}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":8}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":9}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":10}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":11}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":12}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":13}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":14}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":15}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":16}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":17}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":18}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":19}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":20}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":21}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":22}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":23}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":24}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":25}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":26}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":27}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":28}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":29}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":30}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":31}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":32}}, {item_type:"T_QUESTIONMARK", result_type:{meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:6, precision:6, scale:0}, flag:1, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_CONST", "IS_STATIC_PARAM", "CNT_CONST", "CNT_STATIC_PARAM"], rel_id:[], value:{"UNKNOWN":33}}]}]})
WDIAG [SQL.RESV] formalize_relation_exprs (ob_dml_stmt.cpp:1692) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=192][errcode=-4013] failed to formalize expr(ret=-4013)
WDIAG [SQL.RESV] formalize_stmt (ob_dml_stmt.cpp:1638) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=-4013] failed to formalize relation exprs(ret=-4013)
WDIAG [SQL.REWRITE] transform_rule_set (ob_transformer_impl.cpp:310) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4013] failed to formalize stmt(ret=-4013)
WDIAG [SQL.REWRITE] do_transform (ob_transformer_impl.cpp:137) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4013] failed to transform one rule set(ret=-4013)
WDIAG [SQL.REWRITE] transform (ob_transformer_impl.cpp:80) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=-4013] failed to do transform(ret=-4013)
WDIAG [SQL] transform_stmt (ob_sql.cpp:3546) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=-4013] failed to transform statement(ret=-4013)
WDIAG [SQL] generate_plan (ob_sql.cpp:3216) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=12][errcode=-4013] Failed to transform stmt(ret=-4013)
INFO  [SQL] generate_plan (ob_sql.cpp:3277) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=7] [SQL MEM USAGE] use too much memory(total_mem_used=2344598941, ObString(parse_result.input_sql_len_, parse_result.input_sql_)=select id, dimension_id, sum(use_count) as use_count
WDIAG [SQL] generate_physical_plan (ob_sql.cpp:3032) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=37][errcode=-4013] failed to generate plan(ret=-4013)
WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4850) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4013] Failed to generate plan(ret=-4013, result.get_exec_context().need_disconnect()=false)
WDIAG [SQL] handle_text_query (ob_sql.cpp:2582) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4013] fail to handle physical plan(ret=-4013)
WDIAG [SQL] stmt_query (ob_sql.cpp:222) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=3][errcode=-4013] fail to handle text query(stmt=select id, dimension_id, sum(use_count) as use_count
WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:962) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=22][errcode=-4013] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:1, local_retry_times:1, err_:-4013, err_:"OB_ALLOCATE_MEMORY_FAILED", retry_type:0, client_ret:-4013, is_interrupted_by_outer_query:false}, need_retry=false, THIS_WORKER.can_retry()=true, v.ctx_.multi_stmt_item_={is_part_of_multi_stmt:false, seq_num:0, sql:"select id, dimension_id, sum(use_count) as use_count
WDIAG [SERVER] do_process (obmp_query.cpp:759) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=30][errcode=-4013] run stmt_query failed, check if need retry(ret=-4013, cli_ret=-4013, retry_ctrl_.need_retry()=0, sql=select id, dimension_id, sum(use_count) as use_count
WDIAG [SERVER] do_process (obmp_query.cpp:885) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=25][errcode=-4013] query failed(ret=-4013, session={this:0x7f4f31864eb8, id:3221782870, deser:false, tenant:"t7fd012zysg00", tenant_id:1010, effective_tenant:"t7fd012zysg00", effective_tenant_id:1010, database:"sou_bi", user:"sou_bi_rw@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f688cdd18a0}, sql=select id, dimension_id, sum(use_count) as use_count
INFO  [SERVER] send_error_packet (obmp_packet_sender.cpp:319) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=32] sending error packet(ob_error=-4013, client error=-4013, extra_err_info=NULL, lbt()="0x13a165e3 0xacfea90 0xacaaf42 0xad09dc2 0x558d830 0x5586341 0x55792ca 0x5563c2f 0xa9b1d74 0x13cd4e0f 0x7f6af23a6dc5 0x7f6af20d3ced")
INFO  [SERVER] send_error_packet (obmp_packet_sender.cpp:502) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=6] dump txn free route audit_record(value=1048581, session->get_sessid()=3221782870, session->get_proxy_sessid()=13882384978710613955)
WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [23931][T1010_L0_G0][T1010][YB420A663FF0-000646B37999723E-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=6099087651657, cache_obj->added_lc()=false, cache_obj->get_object_id()=96080, cache_obj->get_tenant_id()=1010, lbt()="0x13a165e3 0xb54450e 0x55975af 0xad09368 0x558d830 0x5586341 0x55792ca 0x5563c2f 0xa9b1d74 0x13cd4e0f 0x7f6af23a6dc5 0x7f6af20d3ced")

客户当前版本为 V4.2.1。在该版本中,对于 a IN (1, 2, 3) 这类查询,硬解析阶段优化器会把 IN 列表中的每一个常量都当作一个独立的表达式节点处理(在逻辑上等价于展开为 a = 1 OR a = 2 OR a = 3)。当 IN 列表包含 3 万+ 个常量时,语法树上会产生 3 万+ 个表达式节点,类型推导、参数化、改写、代价估算都要逐一遍历,导致:

  • 硬解析内存消耗巨大(可达 GB 级),最终打爆租户内存上限,报错 -4013;

  • 硬解析耗时超长,占用大量 CPU。

OB 422版本上实现了大inlist SQL语句执行时自动转Values Table的解析优化。当 IN 列表中常量个数超过阈值 _inlist_rewrite_threshold 时,优化器会自动把这堆常量改写成一张内存中的常量表(Values Table),SQL 在逻辑上等价变成:


-- 改写前

select ... from t  where c1  in (1, 2, 3, ..., 500000);

-- 改写后(等价形式)
select ... from t  where c1  in (select v from VALUES_TABLE(1,2,3,...,500000));
  • 数万个常量不再是数万个表达式节点,而是 Values 表中的一批数据行,处理成本大幅降低;
  • 硬解析阶段的类型推导、改写、优化只需处理一个表对象 + 一个连接条件,CPU 与内存消耗从 O(N) 降至接近常数级。

后续推动客户升级至425版本(并推高优化器版本),将in查询改写为values table后,SQL执行不再报错



set global optimizer_features_enable=
'4.2.5.7';


三、‑11049 vs ‑4013 对比总结

这两个报错都与内存相关,但触发层级不一样。-11049 是单条 SQL 查询级别到达内存上限;-4013 是整个租户层面的内存不足。如果 query_memory_limit_percentage设置过大,且查询并发较多的情况下,就可能导致单条SQL打爆租户内存报错-4013。

排查时还是要结合实际情况来看。

如果在操作过程中遇到问题,欢迎在评论区留言交流,后续我们会持续分享更多实战运维干货,记得  关注不迷路  ,下次见~


来自 “ ITPUB博客 ” ,链接:https://blog.itpub.net/70047467/viewspace-3179698/ ,如需转载,请注明出处,否则将追究法律责任。

请登录后发表评论 登录
全部评论
专注于企业信息科技战略咨询、数据中心规划及运维、智能化软件产品研发,致力于为金融、能源、电信、制造等行业提供智能的业务永续及流程自动化解决方案。

注册时间:2025-09-08

  • 博文量
    89
  • 访问量
    86707