Introduction
============

        Truss and trace are utilities that exist on SOME Unix boxes.
        These are NOT general purpose tools and should not be used 
        to help progress a problem unless their output has been requested
        by Oracle Support or is understood and is likely to be of help. 

        This short article describes what truss and trace are and how to
        use them.

        Warning: These tools produce a LOT of output.


What do these utilities show?
=============================

        Truss and trace show Unix SYSTEM calls made by an executable
        and the arguments that were passed. When a SYSTEM call occurs processing
        switches from the user code into system code and occurs for actions
        such as:

                opening a file / IO device, 
                reading / writing a file / IO device,
                handling signals, 
                forking (spawning a child process),
                executing a process
                etc...


Where is truss is NOT useful?
=============================

        Specifically tracing is unlikely to help with :

                Memory management / leakage issues

                Getting information on WHICH Oracle function was running

                Getting and stack information

                It is unusual for truss / trace to be of use when trying to 
                locate the source of a core dump or other terminal signal.


Where tracing may be useful
===========================

        There are specific circumstances where truss/trace may be of use:

                Identifying files opened by a process AND any and errors
                on attempted file accesses

                Contents of files opened for reading/writing (truss)

                Whether any sub-processes are being spawned

                The environment when executing a sub-process (truss only)

                Which dynamic linked libraries are being loaded when an
                executable runs. (ldd is better for this if available)

                When a process is spinning - if it is spinning in an I/O
                loop or an internal loop. Eg: repeated reads show an I/O
                loop, no truss output when spinning shows a process is
                doing something other than IO - most likely being stuck in 
                an internal loop.
        
                Obtaining information on SGA segments and attach addresses.

        Reading the output requires an understanding of C and how the 
        system calls operate. You will see a lot of errors in the output
        even when a process is running normally so you MUST NOT EVER assume
        an error reported in truss or trace output pinpoints the problem.

        Note that tracing slows processes down considerably.


Using truss or trace
====================

        Truss/trace can be used in two main ways:

                - You can START a program under truss to see every system 
                  call it makes.

                - You can attach to a running process to see what it is doing.

        In addition you can ask truss to show system calls made by any spawned
        child processes. 'trace' cannot do this.


Running a process from the command line
=======================================

        Find the program you want to trace and issue the relevant command
        below. 'program' below should be the full path name to the executable:

                truss -o /tmp/mytruss  

        or:     

                trace -o /tmp/mytruss  

        To follow child processes:

                truss -o /tmp/mytruss -fae  


Attaching to an existing process
================================

        Use 'ps' to identify the process you wish to trace. 
        Eg:     ps -ef | grep oracleTEST

        Log in as a user who has permission to control the process you want to
        trace. This means logging in as either ROOT or the process owner.
        
        Find a writable directory which has lots of spare space. 
        Eg: We assume here /tmp has lots of free space.

        Enter the relevant command:

                truss -o /tmp/mytruss -p 

        or:     

                trace -o /tmp/mytruss -p 

        To follow child processes:

                truss -o /tmp/mytruss -fae -p 

        This will run until the process (and any traced children) exit.
        You can interrupt the capture of information if only a snapshot
        of the trace is required.


The Output
==========

        Once you have a truss output, check with support on what part of the 
        output is required. Fo not just send the whole output.