Applies to:

Oracle Server - Standard Edition - Version: 10.2.0.1 to 11.1.0.7 - Release: 10.2 to 11.1
Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 11.1.0.7 [Release: 10.2 to 11.1]
Information in this document applies to any platform.
Oracle Database

Description

Oracle NUMA (Non Uniform Memory Architecture) support can be used with large SMP multiprocessor environments with NUMA hardware. When enabled Oracle NUMA support facilitates efficient use of underlying NUMA hardware and may improve database performance.

Oracle NUMA support needs the right combination of hardware, operating system and Oracle version.

With 10.2.0.4 and 11.1.0.7 patchsets, Oracle NUMA support can be enabled on common Operating Systems like AIX, HP-UX, Solaris, Linux and Windows if the underlying hardware characteristic is NUMA.

When running an Oracle database with NUMA support in a NUMA capable environment, Oracle will by default detect if the hardware and operating system are NUMA capable and enable Oracle NUMA support.

Some OS upgrades/patches may enable NUMA (For example on Linux NUMA is enabled with kernel release 2.6.9-67). Care should be taken before enabling NUMA support or leaving it on by default. Please see below under the “Recommendation” section. Contact your hardware vendor for recommendation and information on your system and operating system NUMA capabilities

Likelihood of Occurrence

The symptoms described in the following section generally occur when:

• Oracle database NUMA support is enabled
• Both operating system and hardware are NUMA capable.

And :

• Database workload is memory constrained (or apply too much memory pressure on a given NUMA memory pool)
• Dynamic reconfiguration events change the characteristics of the hardware or partition (e.g number of CPUs, memory available).

Some issues are OS/hardware specific. See bugs caused by NUMA section below.

Dynamic reconfiguration events removing resources from NUMA system such as an entire cell and all its processors are not supported (Please review Note:761065.1)

Possible Symptoms

The problems manifest usually with crashes with internal errors including:

• ORA-4031
• ORA-600 with argument KSKRECONFIGNUMA2
• ORA-600 with argument KSBASEND_INTERNAL
• ORA-600 with argument KSMHEAP_ALLOC1
• ORA-27302: FAILURE OCCURRED AT: SSKGXPCRE3

Workaround or Resolution

Recommendation

• Customers who have their SLA’s unaffected with NUMA enabled can continue to run with no changes.
• It is strongly recommended to customers who want to enable NUMA do sufficient testing before going into production.
• Apply all the bug fixes or patchset required for your Oracle database version. Fixes for all known NUMA issues in the Oracle database is available for download. Please review the known bugs section.

To disable NUMA consult the section "Steps to disable NUMA" covering the instructions to disable NUMA at the Oracle database level. To disable NUMA at the operating system or hardware level contact your hardware vendor.

Please review the "Caution" section below when disabling NUMA.

Caution

• Disabling or enabling NUMA can change application performance.
• It is strongly recommended to evaluate the performance after disabling or before enabling NUMA in a test environment.
• Operating system and/or hardware configuration may need to be tuned or reconfigured when disabling Oracle NUMA support. Consult your hardware vendor for more information or recommendation

Steps to disable NUMA

• Customers can download and apply patch for Bug 8199533 to disable NUMA support. This is a database patch and should be applied to the Database home. This patch is available for common platforms on 10204 and 11.1.0.7 releases.
• If you apply patch for Bug 8199533 then Oracle will no longer enabled NUMA support by default even if it detects a NUMA capable environment.
• Oracle support does not recommended using init.ora parameter "_enable_NUMA_optimization" to disable NUMA. Customers should apply fix for Patch 8199533 to disable NUMA . The patch is rolling upgradeable.
• This patch does not need to be applied to the ASM home. However if the same Oracle home is used for both RDBMS and ASM instances then this patch can be applied to the Oracle home.

Patches

Bugs caused due to NUMA being enabled

Below is the list of known bugs caused by Oracle NUMA support being enabled. It is important to note that customers may encounter these bugs **ONLY** if NUMA support is enabled.

If Customers apply the corresponding bug fix or disable NUMA using the patch for Patch 8199533 or at the hardware/OS layer, they would not encounter these issues.

Bug	Bug Description	Fixed release
Bug:5173642	Data not read from cache on second execution with NUMA optimization enabled	10.2.0.4 & 11.1.0.6
Bug 6868080	ORA-4031 with NUMA	10.2.0.5 & 11.1.0.7
Bug 6689903	ORA-27302: FAILURE OCCURRED AT: SSKGXPCRE3	10.2.0.5
Bug 4414666	ORA-600[KSMHEAP_ALLOC1] WHEN STARTING UP WITH NUMA_ON	10.2.0.2 & 11.1.0.6
Bug 4173484	OERI[17131] / instance crash with NUMA using SGA_TARGET	10.1.0.5 & 10.2.0.1
Bug 3802438	OERI[17148] / SGA heap corruption with NUMA pools	10.1.0.4 & 10.2.0.1
Bug 3202031 -P	Tru64 OERI [KSMHEAP_ALLOC1] on STARTUP with NUMA_ON	10.2.0.1
Bug 6730567	ORA-4031 / OERI [17137] with NUMA optimization enabled	11.1.0.7
Bug 6689903	Errors such as ORA-27504 using RAC on NUMA	10.2.0.5
Bug:7232946	ORA-600[KSKRECONFIGNUMA2] CAUSES INSTANCE CRASH
Bug:6086099	ORA-7445[KSBASEND_INTERNAL] OR ORA-7445[KCBZWW] DURING STARTUP	10.2.0.3
Bug:8244734	NUMA POOL UNDERCONFIGURED DUE TO KSS TRUNCATION	10.2.0.5
Bug:8856696	INSTANCES HANG WITH 'FREE BUFFER WAITS' OR HIT ORA-00379	11.1.0.7.1 & 11.2.0.2

• P ==> Port specific bug
• Bugs with empty fixed release means that the bug is either being currently fixed or the bug has been closed due to customer using the workaround (_enable_numa_optimization=FALSE and _db_block_numa=1)

Modification History

Publication date 22-DEC-2008
Updated with some other bugs 17-Feb-2009
Updated the note to change the underscore parameter to regular init.ora parameter 14-Apr-2009
18-Jun-2009, Implemented changes recommended by Ravi

References

BUG:5173642 - DATA NOT READ FROM CACHE ON SECOND EXECUTION ON WIN2003 ON AMD64
BUG:6868080 - NUMA POOLS GENERATED ORA-4031
BUG:7232946 - ORA-600[KSKRECONFIGNUMA2] CAUSES INSTANCE CRASH
BUG:7357301 - DB HANG WHEN START 1 MORE INSTANCE DUE TO NUMA
[@more@]