首页 > 数据库 > Oracle > dbs目录被core.XXXX 塞满
一、错误描述
RAC 集群节点 2 出现写归档错误,通过查看告警日志和 trace 日志,定位到关键错误信息如下:
1. 告警日志关键内容
<msg time='2025-08-03T18:02:56.692+08:00' org_id='oracle' comp_id='rdbms'
client_id='' type='UNKNOWN' level='16'
host_id='rac2' host_addr='92.178.12.163' module='DBMS_SCHEDULER'
pid='177429'>
<txt>Exception [type: SIGSEGV, Invalid permissions for mapped object] [ADDR:0x7FBC24D8F000] [PC:0x49AE183, __intel_new_memcpy()+5395] [flags: 0x0, count: 1]
</txt>
</msg>
<msg time='2025-08-03T18:02:56.779+08:00' org_id='oracle' comp_id='rdbms'
msg_id='609256791' type='INCIDENT_ERROR' group='Access Violation'
level='1' host_id='rac2' host_addr='92.178.12.163'
prob_key='ORA 7445 [__intel_new_memcpy()+5395]' upstream_comp='' downstream_comp=''
ecid='' errid='426293' detail_path='/u01/app/oracle/diag/rdbms/yajj/yajj2/trace/yajj2_j001_177429.trc'>
<txt>Errors in file /u01/app/oracle/diag/rdbms/yajj/yajj2/trace/yajj2_j001_177429.trc (incident=426293):
ORA-07445: exception encountered: core dump [__intel_new_memcpy()+5395] [SIGSEGV] [ADDR:0x7FBC24D8F000] [PC:0x49AE183] [Invalid permissions for mapped object] []
</txt>
</msg>
核心错误:
多次出现 ORA-07445 异常,伴随 SIGSEGV 信号(无效内存访问)。
涉及函数:__intel_new_memcpy()+5395、pfrtra()+66 等,提示 “Invalid permissions for mapped object”(映射对象权限无效)或 “Address not mapped to object”(地址未映射到对象)。
2. Trace 日志关键内容
截取 trace 日志中与错误直接相关的片段:
Archived Log entry 1845100 added for thread 2 sequence 969570 ID 0xfe827259 dest 1:
Sun Aug 03 18:02:56 2025
Exception [type: SIGSEGV, Invalid permissions for mapped object] [ADDR:0x7FBC24D8F000] [PC:0x49AE183, __intel_new_memcpy()+5395] [flags: 0x0, count: 1]
Errors in file /u01/app/oracle/diag/rdbms/yajj/yajj2/trace/yajj2_j001_177429.trc (incident=426293):
ORA-07445: exception encountered: core dump [__intel_new_memcpy()+5395] [SIGSEGV] [ADDR:0x7FBC24D8F000] [PC:0x49AE183] [Invalid permissions for mapped object] []
...
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x68] [PC:0x8AB20BE, pfrtra()+66] [flags: 0x0, count: 2]
Errors in file /u01/app/oracle/diag/rdbms/yajj/yajj2/trace/yajj2_j001_177429.trc (incident=426294):
ORA-07445: exception encountered: core dump [pfrtra()+66] [SIGSEGV] [ADDR:0x68] [PC:0x8AB20BE] [Address not mapped to object] []
二、分析过程
1. 数据库版本确认
通过 SQL 查询确认数据库版本为 Oracle 11g Enterprise Edition 11.2.0.4.0(64 位),支持 RAC、ASM 等特性:
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
PL/SQL Release 11.2.0.4.0 - Production
CORE 11.2.0.4.0 Production
TNS for Linux: Version 11.2.0.4.0 - Production
NLSRTL Version 11.2.0.4.0 - Production
2. Trace 文件深入分析
查看 yajj2_j001_177429.trc 发现关键线索:
错误发生在 DBMS_SCHEDULER 模块,对应自动作业 ORA$AT_OS_OPT_SY_33509。
日志中提到 内存 Dump 信息,包含表名 EXAMINE,推测自动统计信息收集任务对该表分析时出错。
关键标记:
plaintext
*** MODULE NAME:(DBMS_SCHEDULER) 2025-08-03 18:02:56.691 ---调度异常
*** ACTION NAME:(ORA$AT_OS_OPT_SY_33509) 2025-08-03 18:02:56.691 ---自动作业故障
3. 自动作业状态检查
查询 dba_scheduler_job_run_details 发现,以 ORA$AT_OS_OPT_SY_% 命名的自动作业(属于 auto optimizer stats collection 任务)近期状态均为 STOPPED:
SQL> select a.job_name, a.LOG_ID, a.status, a.actual_start_date
from (select t.job_name, t.LOG_ID, t.status, t.actual_start_date
from dba_scheduler_job_run_details t
where t.job_name like 'ORA$AT_OS_OPT_SY_%'
order by t.actual_start_date desc) a
where rownum <= 100;
JOB_NAME LOG_ID STATUS ACTUAL_START_DATE
----------------------------------- ---------- ------------ ------------------------------
ORA$AT_OS_OPT_SY_33509 286044 STOPPED 03-AUG-25 06.02.21.783210 PM PRC
ORA$AT_OS_OPT_SY_33489 286025 STOPPED 03-AUG-25 02.01.46.994629 PM PRC
...
进一步通过 DBA_AUTOTASK_JOB_HISTORY 确认:
sql
SQL> SELECT CLIENT_NAME, JOB_NAME, JOB_STATUS
FROM DBA_AUTOTASK_JOB_HISTORY
WHERE JOB_NAME LIKE 'ORA$AT_OS_OPT_SY_33509';
CLIENT_NAME JOB_NAME JOB_STATUS
------------------------------- ---------------------- ------------
auto optimizer stats collection ORA$AT_OS_OPT_SY_33509 STOPPED
4. 存储空间占用检查
节点 2 的 dbs 目录下生成大量 core.XXXX 文件(总大小达 770G),耗尽存储空间,导致归档日志无法写入:
[root@rac2 dbs]# ls -lah
total 770G
drwxr-xr-x 2 oracle oinstall 4.0K Aug 6 09:54 .
-rw------- 1 oracle asmadmin 404G Aug 3 06:17 core.127664
-rw------- 1 oracle asmadmin 366G Aug 3 18:25 core.177445
...
三、分析结果
根本原因:
自动统计信息收集任务(auto optimizer stats collection)对表 EXAMINE 进行分析时,因内存访问异常触发 ORA-07445 错误,生成大量 core 文件占用存储空间,最终导致节点 2 归档日志写入失败。
四、解决办法
清理冗余文件:
删除 dbs 目录下的 core.XXXX 文件,释放存储空间:
rm -f /u01/app/oracle/product/11.2.0/db_1/dbs/core.*
重建问题表:
重建 EXAMINE 表,修复可能的表结构或数据异常(建议先备份数据):
-- 备份表数据
CREATE TABLE EXAMINE_BAK AS SELECT * FROM EXAMINE;
-- 重建表
DROP TABLE EXAMINE;
CREATE TABLE EXAMINE (...); -- 按原表结构重建
-- 恢复数据
INSERT INTO EXAMINE SELECT * FROM EXAMINE_BAK;
COMMIT;
监控作业状态:
重建表后,启用自动统计信息收集任务并监控运行状态:
-- 启用任务
BEGIN
dbms_auto_task_admin.enable(
client_name => 'auto optimizer stats collection',
operation => NULL,
window_name => NULL
);
END;
/
-- 查看状态
SELECT client_name, status FROM dba_autotask_client;
参考文档: Bug 21869685 - ORA-7445: exception encountered: core dump [evaudc()+180] (Doc ID 21869685.8)
来自 “ ITPUB博客 ” ,链接:https://blog.itpub.net/30063763/viewspace-3092867/ ,如需转载,请注明出处,否则将追究法律责任。