****  ORACLE     INTERNAL   ****   ****  ORACLE     INTERNAL   ****
         Copyright (c) 2003-2004 Oracle. All rights reserved.

OCFS v1 installation steps RedHat / SuSE:             Last Update: 30NOV04
-----------------------------------------   Comments: Saar.Maoz@oracle.com
                                                      Lance.Larsh@oracle.com

1. Log in to Linux GUI desktop as the oracle user.

2. Get Latest OCFS RPMS:

   ** FOR REDHAT **
  
   a. Click the "Mozilla" internet browser icon on the task bar.
   b. Type in URL "http://oss.oracle.com/".
   c. Under "Projects", Click "OCFS".
   d. Under "Project: OCFS", click "Files".
   e. Click "Red Hat", Click "RHEL 3", Click "i386".
   f. Click "ocfs-tools-1.0.10-1.i386.rpm".
      Select "Save it to disk"
      Click "OK", Click "Save".
     
   g. Click "ocfs-support-1.0.10-1.i386.rpm".
      Select "Save it to disk"
      Click "OK", Click "Save".
  
   h. Click "1.0.13-1"
   i. Click "ocfs-2.4.21-EL-1.0.13-1.i686.rpm"
  
       Notice we pick nonSMP, otherwise known as UP (Uni-Processor Kernel)
       Please choose smp or hugemem based on your running kernel (uname -a)
       OCFS gets updated every once in a while just pick up the latest
       version.  In our example above the version is "1.0.13-1".
      
       Select "Save it to disk".
       Click "OK"., Click "Save".

      NOTE: tools/support version may not match the ocfs version, it's OK as
            long as you take the latest version see readme on site

   j. Exit Mozilla.

   ** FOR SUSE **
  
   a. Click the "Mozilla" internet browser icon on the task bar.
   b. Type in URL "http://oss.oracle.com/".
   c. Under "Projects", click "OCFS".
   d. Under "Project: OCFS", click "Files".
   e. Click "United Linux", Click "United Linux 1.0", Click "i386".
   f. Navigate to find the latest OCFS rpm for your kernel (uname -a)
      SP3 plain is 2.4.21-138  highest ocfs for it is 1.0.11-1.
      Recommended version is 1.0.13-1 which requires kernel SP3-251.

      For normal hardware navigation would look like:
     
      Click "1.0.13-1", Click "SP3-251",
      Click "ocfs-2.4.21-251-deflt-1.0.13-1.i586.rpm"
     
      Select "Save it to disk"
      Click "OK", Click "Save".
     
      VMWARE users:  VMware only supports plain sp3 kernel (-138) if you
                     upgrade your kernel to -198 or -241, -251 you may see
                     system instabilities inside the VM.  If you choose
                     to use OCFS v1.0.11 along with plain SP3 kernel -138
                     CRS 10.1.0.3 will not run due to bug 3656428.
                     In normal customer case you would simply upgrade the
                     kernel to -251 & OCFS to 1.0.13.  Since we don't want to
                     upgrade the kernel of the VM we provide the needed 1.0.13
                     rpms for the sp3 -138 kernel in the
                     T3_LinuxDemoFiles-10g-.zip under ocfs/suse.
                     In short no need for oss website, but now you know
                     where to get OCFS for real deployment.  
     
   g. For SuSE get v1.1.2 of the OCFS tools (not v1.0)
      Under "Projects", click "OCFS Tools" (on the LEFT sidebar).
   h. Under "Project: OCFS Tools", click "Files".
   i. Click "United Linux", Click "United Linux 1.0", Click "i386".
   j. Click "1.1.2-1"
   k. Click "ocfs-support-1.1.2-1.i386.rpm".
      Select "Save it to disk"
      Click "OK", Click "Save".
     
   l. Click "ocfs-tools-1.1.2-1.i386.rpm".
      Select "Save it to disk"
      Click "OK", Click "Save".

3. Open a terminal window:

   a. REDHAT: Click Main Menu --> System Tools --> Terminal.
      SUSE: Click Main Menu --> System -> Terminals -> Konsole.
   b. Change user to root by typing:

      $ su -
      Password: oracle
      #

   Note: The prompt changed from "$" to "#" to indicate that the shell is
         running with root privileges.  From now on in these scripts, each
         command will begin with either a "$" or "#" character to indicate
         whether the command should be run in the "oracle" window or the
         "root" window.  PAY ATTENTION, it does make a difference.

4. Run the following commands as root:

   ** FOR REDHAT **

   # cd ~oracle
   # rpm -Uvh ocfs-2.4.21-EL-1.0.13-1.i686.rpm
              ocfs-support-1.0.10-1.i386.rpm
              ocfs-tools-1.0.10-1.i386.rpm

   ** FOR SUSE **

   # cd ~oracle
   # rpm -Uvh ocfs-2.4.21-138-deflt-1.0.13-1.i586.rpm
              ocfs-support-1.1.2-1.i386.rpm
              ocfs-tools-1.1.2-1.i386.rpm

   OR Simply the Linux way (using wildcard for rpm selection):
  
   # cd ~oracle
   # rpm -Uvh ocfs-*.rpm
  
5. As root, run "ocfstool &".

   NOTE: Safe to Ignore following warning (see bug 3428900):

   Gtk-WARNING **: libgdk_pixbuf.so.2: cannot open shared object file:
                   No such file or directory
  
   If you insist, the workaround is to comment out this line:
   # include "/usr/share/themes/Bluecurve/gtk/gtkrc"

   in file /root/.gtkrc
  
   a. Select "Tasks" --> "Generate Config...".
      i. Interface: Select "eth1" from the pulldown menu.
     ii. Node name: Enter "raclocal1.us.oracle.com".
    iii. Click "OK".
   b. Select "File" --> "Exit".

6. As root, run "load_ocfs".

   In Red Hat case, ignore message about tainted kernel:

  "Warning: loading /lib/modules/2.4.21-EL-ABI/ocfs/ocfs.o will taint
   the kernel: forced load"

    Refer to bug# 3432568 for details

7. Partitioning ocfs device.
         
   a. Depending on weather you want to place the OCR (Oracle Cluster
      Registry) on raw devices or on OCFS you will follow slightly different
      paths, choose ONLY ONE option.
      There is no difference in performance or features, placing all
      on OCFS is simpler, if you want to practice/learn raw devices
      choose that.  Remember your selection, you will need it later.
     
      As root run:
  
      # fdisk /dev/sdd  
         
      Option 1: OCR+Voting disk ON OCFS Filesystem:

          ** MAKE SURE YOU USE THE SUPPORTTED OCFS, 1.0.11 and above **
             see bug 3467544 for details.

          i. Command: Type "n" and press ENTER.
         ii. Command action: Type "p" and press ENTER.
        iii. Partition number: Type "1" and press ENTER.
         iv. First cylinder: Press ENTER.
          v. Last cylinder: Press ENTER.             (Entire disk)

      Option 2: OCR+Voting disk ON RAW Devices:
   
          Partition 1:  (ocr_file)
          i. Command: Type "n" and press ENTER. 
         ii. Command action: Type "p" and press ENTER.
        iii. Partition number: Type "1" and press ENTER.
         iv. First cylinder: Press ENTER.
          v. Last cylinder: +100M.
   
          Partition 2:  (voting_disk)
          i. Command: Type "n" and press ENTER.
         ii. Command action: Type "p" and press ENTER.
        iii. Partition number: Type "2" and press ENTER.
         iv. First cylinder: Press ENTER.
          v. Last cylinder: +100M.
   
          Partition 3:  (ocfs filesystem)
          i. Command: Type "n" and press ENTER.
         ii. Command action: Type "p" and press ENTER.
        iii. Partition number: Type "3" and press ENTER.
         iv. First cylinder: Press ENTER.
          v. Last cylinder: Press ENTER.            (rest of the drive)
   
   b. View partition table, to make sure it's what you want, Type "p"
      Output will look like (for option 2):
     
   Device Boot    Start       End    Blocks   Id  System
/dev/sdd1             1        13    104391   83  Linux
/dev/sdd2            14        26    104422+  83  Linux
/dev/sdd3            27      1044   8177085   83  Linux
   
   c. Finalize changes: Type "w" and press ENTER.

8. As root, run "ocfstool &".

   a. Select "Tasks" --> "Format...".

      i. Device: Select "/dev/sdd1" (everything on OCFS) or "/dev/sdd3"
                 (if OCR+Voting are on raw devices) from the pulldown menu.
     ii. Volume Label: Enter "ocfs".
    iii. Mountpoint: Enter "/ocfs".
     iv. User: Select "oracle" from the pulldown menu.
      v. Group: Select "dba" from the pulldown menu.
     vi. Click "OK".
    vii. Dialog box: "Are you sure you want to format /dev/sdd1?" or "sdd3?"
         1. Click "Yes".

   b. Select "File" --> "Exit".

9. As root, run "nedit /etc/fstab &" or vi /etc/fstab.
   Avoid using xedit (it forgets to put a newline at end of line)

   a. Add ONE of the following lines (without quotes) to the end of the fstab:

      Everything on OCFS:
      "/dev/sdd1         /ocfs       ocfs       _netdev   0 0"
     
      OCR+Voting on raw devices:
      "/dev/sdd3         /ocfs       ocfs       _netdev   0 0"

   b. Click "File->Save" at the top of the window.
   c. Click "File->Exit" at the top of the window.

10. As root, run "mkdir /ocfs".

11. As root, run "mount /ocfs".

12. (REDHAT Only) Tell updatedb/slocate not to touch ocfs filesystems

    Edit two lines in /etc/updatedb.conf  (Just add the "ocfs" at end)

PRUNEFS="devpts NFS nfs afs sfs proc smbfs autofs auto iso9660 ocfs"
PRUNEPATHS="/tmp /usr/tmp /var/tmp /afs /net /sfs /ocfs"
 
    Because updatedb is called from /etc/cron.daily without the "-c"
    flag the updatedb.conf will not be looked at, lets edit
    /etc/cron.daily/slocate.cron and add ocfs to exclusion lists:

/usr/bin/updatedb -f "nfs,smbfs,ncpfs,proc,devpts,ocfs"
                  -e "/tmp,/var/tmp,/usr/tmp,/afs,/net,/ocfs"

    or use the pre-editted updatedb.conf & slocate.cron files in
    the LinuxDemoFiles zip.

OCFS ROLLING UPGRADE:

Since version 1.0.10 rolling upgrades are OK (check ocfs README for details)
To upgrade ocfs in rolling upgrade fashion:

  1. shut db on node A
  2. unmount ocfs volumes
  3. unload ocfs module
  4. upgrade ocfs rpm
  5. reload ocfs module
  6. mount ocfs volumes
  7. start DB on node A
  8. repeat for node A+1
 
VMWARE NOTES:
   Following is true for OCFS in general but will mostly show up in a VMWare
   environment.  When making copies of a virtual machine or changing
   properties of it, the vmware uuid (unique id, has NOTHING to do with
   OCFS's guid) might change, which in turn will cause the MAC addresses
   of the VM's NIC cards to change.  OCFS will fail to load since the MAC
   address stored in /etc/ocfs.conf does not match the new MAC address
   of the NIC.  To get OCFS up and running:
  
    a. Regenerate the guid (Global unique id) for OCFS:
   
       # ocfs_uid_gen -r
  
       "ocfs_uid_gen: The new GUID is xxxxxx-random-appended-to-MAC-inHEX"
      
       It is harmless to re-run, in case the MAC addresses match
       you will get:
      
       "ocfs_uid_gen: The GUID is already correct.  No action taken."
      
       To auto-fix this at startup just place command in /etc/init.d/ocfs
       or use the pre-editted ocfs file in the LinuxDemoFiles zip.
      
    b. Mount ocfs with "reclaimid" option (needed for first time mount only):
   
       Everything on OCFS:
       # mount -t ocfs -o reclaimid /dev/sdd1  /ocfs
  
       OCR+Voting on raw devices:
       # mount -t ocfs -o reclaimid /dev/sdd3  /ocfs
      
       If you forget the reclaimid option, mount will fail or cause a kernel
       panic (on 1.0.9 or older based OCFS), see bug 3181864 for details.
      
       To auto-fix this at startup just add reclaimid option in /etc/fstab
       or use the pre-editted fstab file in the LinuxDemoFiles zip.
      
        Copyright (c) 2003-2004 Oracle. All rights reserved.
 ****  ORACLE     INTERNAL   ****   ****  ORACLE     INTERNAL   ****