• To Get to Sun Cluster 3.2 and Oracle Real Application Cluster (RAC) Installation / configuration Part 2 Click Here

Hardware Installation and configuration

Before you power up the M5000

You must move one CPU and Memory board

The CPU board should be moved to slot 3
The Memory board should be moved to slot 4
To verify if the move is correct and you see all memory/cpu and IO boards run.

XSCF> showhardconf

First time you plug in the M5000 you will need setup the admin account

login using the login account default

Note: Full xscf configurations details is available here

login: default
Change the panel mode switch to Service and press return...
  (Operation : Locked state -> Service -> Return)
Leave it in that position for at least 5 seconds. Change the panel mode switch to Locked, and press return...
(Operation : Wait more than 5 seconds -> Service state -> Locked -> Return)
XSCF>

xscf login information

login: admin
password: *******

Create an XSCF user account, password and privileges

XSCF> adduser administrator
XSCF> password administrator

XSCF> setprivileges administrator useradm platadm fieldeng auditadm mode

XSCF> setdcl –d 0 –a 1=01-0

Sometimes this will work:
XSCF> addboard –d 0 –c assign 01-0

Sometimes this will work:
XSCF> addboard –d 0 –c assign 01-0

XSCF> showdcl -av

XSCF> showboards -av

Configure network

the network was configured, for a full reference look here
The user is admin
The xscf name is db1-xscf/db2-xscf

San switch Port layout

OS Installation and configuration

To Install the OS using a Jumpstart server

Os version used is Solairs 10 – (05/08)

Add the new servers to the jumpstart hosts file

/etc/hosts
10.10.10.11  db1 db1.domain.com
10.10.10.12  db2 db2.domain.com

Add the servers to the rule file

cat /jumpstart/installs/sol10-0508/conf/rule
hostname db1 - m5000_db post_200805.sh
hostname db2 - m5000_db post_200805.sh

Create the server’s profile m5000_db1 m5000_db2
Note: the /globaldevices md cannot be the same on both nodes

db1:
install_type initial_install
system_type standalone
cluster SUNWCXall
partitioning explicit
filesys mirror:d10 c0t0d0s0 c0t1d0s0 2048 /
filesys mirror:d20 c0t0d0s1 c0t1d0s1 8192 /var
filesys mirror:d30 c0t0d0s3 c0t1d0s3 16384 swap
filesys mirror:d60 c0t0d0s4 c0t1d0s4 512 /globaldevices
filesys mirror:d40 c0t0d0s5 c0t1d0s5 8192 /opt
filesys mirror:d50 c0t0d0s6 c0t1d0s6 free /usr
metadb c0t0d0s7 size 8192 count 3
metadb c0t1d0s7 size 8192 count 3

db2:
install_type initial_install
system_type standalone
cluster SUNWCXall
partitioning explicit
filesys mirror:d10 c0t0d0s0 c0t1d0s0 2048 /
filesys mirror:d20 c0t0d0s1 c0t1d0s1 8192 /var
filesys mirror:d30 c0t0d0s3 c0t1d0s3 16384 swap
filesys mirror:d70 c0t0d0s4 c0t1d0s4 512 /globaldevices
filesys mirror:d40 c0t0d0s5 c0t1d0s5 8192 /opt
filesys mirror:d50 c0t0d0s6 c0t1d0s6 free /usr
metadb c0t0d0s7 size 8192 count 3
metadb c0t1d0s7 size 8192 count 3

Make sure the post install file is correct in our case
/jumpstart/installs/sol10-0508/conf/post_200805.sh

run the check script to make sure all is setup correct

/jumpstart/installs/sol10-0508/conf/check

Get the servers MAC address by typing banner on the console’s ok prompt

banner ......... 0:14:4f:e1:48:69 .......

db1:

/jumpstart/installs/sol10-0805/Solaris_10/Tools/add_install_client -e 0:14:4f:e1:48:69 \
-c jumpstart:/jumpstart/installs/sol10-0805/conf \
-s jumpstart:/jumpstart/installs/sol10-0805 \
-p jumpstart:/jumpstart/installs/sol10-0805/conf db1 sun4u

db2:

/jumpstart/installs/sol10-0805/Solaris_10/Tools/add_install_client -e 0:14:4f:e1:48:69 \
-c jumpstart:/jumpstart/installs/sol10-0805/conf \
-s jumpstart:/jumpstart/installs/sol10-0805 \
-p jumpstart:/jumpstart/installs/sol10-0805/conf db2 sun4u

Finally start the jumpstart install

boot net - install
.............
... wait till done
reboot

Fix OBP to boot from any of the mirror disks

First get the full path of both boot disks and try to of them for example

disk1:
ok boot /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/disk@0,0:a

disk2:
ok boot /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/disk@1,0:a

Then create a temp OBP eeprom file, run

eeprom nvramrc >>  /tmp/obp.out

Then modify / fix and add to the /tmp/obp.out

• set the first disk as rootdisk
• set the second disk as rootmirror

Before change of /tmp/obp.out:
nvramrc=." ChassisSerialNumber BEF08265ED " cr

After change of /tmp/obp.out:
devalias rootdisk /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/disk@0,0:a
devalias rootmirror /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/disk@1,0:a
." ChassisSerialNumber BEF08265ED " cr

Then read in the /tmp/obp.out to the eeprom

eeprom "nvramrc=`cat /tmp/obp.out`"

Make sure to change to use the nvramrc and boot-device

eeprom use-nvramrc?=true
eeprom boot-device="rootdisk rootmirror"

Verify the changes:
eeprom
prtconf -pv |grep boot

Setup Server Patching with the aduva system and patch

• Aduva system steps to be filled in
• Install Orcallator
• Install netbackup
• Install Samhine

To enable MPxIO run

stmsboot -e 
enter y for all

WARNING: stmsboot operates on each supported multipath-capable controller
         detected in a host. In your system, these controllers are

/devices/pci@1,700000/SUNW,qlc@0/fp@0,0
/devices/pci@1,700000/SUNW,qlc@0,1/fp@0,0
/devices/pci@11,700000/SUNW,qlc@0/fp@0,0
/devices/pci@11,700000/SUNW,qlc@0,1/fp@0,0
/devices/pci@0,600000/pci@0/pci@8/pci@0/scsi@1
/devices/pci@10,600000/pci@0/pci@8/pci@0/scsi@1

If you do NOT wish to operate on these controllers, please quit stmsboot
and re-invoke with -D { fp | mpt } to specify which controllers you wish
to modify your multipathing configuration for.

Do you wish to continue? [y/n] (default: y) y
Checking mpxio status for driver fp
Checking mpxio status for driver mpt
WARNING: This operation will require a reboot.
Do you want to continue ? [y/n] (default: y) y
The changes will come into effect after rebooting the system.
Reboot the system now ? [y/n] (default: y) y

Oracle system tuning

add to /etc/system

set semsys:seminfo_semmns=1024
set semsys:seminfo_semvmx=32767
set shmsys:shminfo_shmmax=8589934590
* Sets SVM quick mirroring
set ufs:ldl_minlogsize=0x2000000
set ufs:ldl_maxlogsize=0x12c00000
* Set the Open file limit
set rlim_fd_cur=1024
set rlim_fd_max=65536
* Private Network Interfaces Using the nxge Driver Might Panic Cluster Nodes
set nxge:nxge_rx_threshold_hi=0

Create an Oracle Home Directory

First add user oracle and group dba oinstall

Add to /etc/passwd:
oracle:x:102:101:oracle:/usr/oracle:/bin/bash

Add to /etc/shadow:
oracle:x:12824::::::

Add to /etc/group:
dba::100:oracle,root
oinstall::101:oracle

Create oracle home directory and links

metainit d91
metainit d92
metainit d90
metattach d90 d92
 
mkdir -p /oracle/home
mount /oracle
chown oracle:dba /oracle/home
 
ln -s /oracle/home /usr/oracle

Setup ssh keys for root and oracle

Note: For Oracle, ssh without a password needs to work even going local.

ssh-keygen -t dsa
ssh-keygen -t rsa
enter enter twice

scp .ssh/*.pub authorized_keys
This is needed for oracle install to complete
mv /etc/issue /etc/issue_preoracle

Append path with QFS and Cluster paths

/usr/cluster/bin:/opt/SUNWsamfs/bin:/opt/SUNWsamfs/sbin

Download and Install Sun StorageTek QFS Software

To install Sun StorageTek QFS

mount the QFS iso using lofiadm.

lofiadm -a /var/tmp/StorageTek_QFS_4.6.iso
mount -F hsfs /dev/lofi/1 /mnt
cd /mnt/sparc/2.10

pkgadd -d . SUNWqfsr SUNWqfsu
In order to install SUNWqfsr, you must accept the terms of the Sun License Agreement.
Enter "y" if you do, "n" if you don't, or "v" to view agreement. y

- The administrator commands will be executable by root only (group bin).
If this is the desired value, enter "y".  If you want to change
the specified value enter "c". y

By default, db-remotely is not setup to be remotely managed by
File System Manager. It can only be managed by the File System
Manager if it is installed locally
You can modify the remote management configuration at a later time
using the command fsmadm
If you want to keep the default behaviour, enter "y". Otherwise enter "n". y

umount /mnt
lofiadm -d /dev/lofi/1

Download and Install Sun Cluster 3.2(02/08) Software

Unzip the Sun cluster download and install

unzip suncluster-3_2u1-ga-solaris-sparc.zip
......
cd /var/tmp/cluster/Solaris_sparc
./installer -noconsole
............. 
More coming soon.......

Sun cluster first time configuration

Public and private Network topology

Note: All networks (private/public) are dual homed

To get the network listing in PDF format Click here

Run scinstall and select new cluster

Select nxge0 and nxge4 using switches(nxge0 to nxge0 – nxge4 to nxge4).
Wait till a reboot is complete and check logs for success
For a complete setup example Click here
Note: This error message is related to IPV6 and could be ignored(t_optmgmt: System error: Cannot assign requested address)
More information is available here

Solaris volume manager (SVM) and multi-owner Disk setup and cluster configuration

Before you start you will need to create the resources and resource groups

The setup multi-owner metaset’s you will first need to setup the cluster resources.
Install Oracle UDLM:

zcat ORCLudlm.tar.Z |tar xf -
yes|pkgadd -d . ORCLudlm

Register the resources needed:

clresourcetype register SUNW.rac_udlm
clresourcetype register SUNW.rac_framework
clresourcetype register SUNW.rac_svm

Create the Oracle RAC resource:

clresourcegroup create -S rac-fmwk-rg
clresource create -g rac-fmwk-rg -t SUNW.rac_framework rac-fmwk-rs

Create the Oracle udlm RAC resource:

clresource create -g rac-fmwk-rg \
-t SUNW.rac_udlm \
-p resource_dependencies=rac-fmwk-rs rac-udlm-rs

Create the Sun SVM (aka oban) multi-owner resource:

clresource create -g rac-fmwk-rg \
-t SUNW.rac_svm \
-p resource_dependencies=rac-fmwk-rs rac-svm-rs

Bring the resource online:

clresourcegroup online -emM rac-fmwk-rg

To setup the multi-owner metaset’s

Format all disk-sets on the 9985 and setup lun masking

To get the Disk layout in PDF format Click here

Oracle 4 metaset disk setup

Create metaset’s:
UNIQ3114d3ced42f0b18-toggledisplay-00000020-QINU
Add devicess to metaset:
Note: before runing the metaset … -a run the below
UNIQ3114d3ced42f0b18-toggledisplay-00000021-QINU
This disk luns where add on 08/19/10.
UNIQ3114d3ced42f0b18-toggledisplay-00000022-QINU
Create device’s:
UNIQ3114d3ced42f0b18-toggledisplay-00000023-QINU
Attach the device’s:
UNIQ3114d3ced42f0b18-toggledisplay-00000024-QINU
Click here to get this in a script

Create the Oracle RAC redo logs and data files

Create Data Files – set 1:
UNIQ3114d3ced42f0b18-toggledisplay-00000025-QINU
Create redo Logs – set #1:

metainit -s oradg d400 -p oradg/d30 25m
metainit -s oradg d401 -p oradg/d30 25m
metainit -s oradg d402 -p oradg/d30 25m
metainit -s oradg d403 -p oradg/d30 25m
metainit -s oradg d404 -p oradg/d30 25m
metainit -s oradg d405 -p oradg/d30 25m
metainit -s oradg d406 -p oradg/d30 25m
metainit -s oradg d407 -p oradg/d30 25m
metainit -s oradg d408 -p oradg/d30 25m
metainit -s oradg d409 -p oradg/d30 25m
metainit -s oradg d410 -p oradg/d30 25m
metainit -s oradg d411 -p oradg/d30 25m
metainit -s oradg d412 -p oradg/d30 25m
metainit -s oradg d413 -p oradg/d30 25m
metainit -s oradg d414 -p oradg/d30 25m
metainit -s oradg d415 -p oradg/d30 25m
metainit -s oradg d416 -p oradg/d30 25m

Oracle spfile:

metainit -s oradg d35 -p oradg/d30 25m

Create redo Logs – set #2:

metainit -s oradg d500 -p oradg/d40 25m
metainit -s oradg d501 -p oradg/d40 25m
metainit -s oradg d502 -p oradg/d40 25m
metainit -s oradg d503 -p oradg/d40 25m
metainit -s oradg d504 -p oradg/d40 25m
metainit -s oradg d505 -p oradg/d40 25m
metainit -s oradg d506 -p oradg/d40 25m
metainit -s oradg d507 -p oradg/d40 25m
metainit -s oradg d508 -p oradg/d40 25m
metainit -s oradg d509 -p oradg/d40 25m
metainit -s oradg d510 -p oradg/d40 25m
metainit -s oradg d511 -p oradg/d40 25m
metainit -s oradg d512 -p oradg/d40 25m
metainit -s oradg d513 -p oradg/d40 25m
metainit -s oradg d514 -p oradg/d40 25m
metainit -s oradg d515 -p oradg/d40 25m
metainit -s oradg d516 -p oradg/d40 25m

oraclepw – Oracle Password file:

metainit -s oradg d45 -p oradg/d40 25m

Add on latter to grow the redo logs from 25M to 100M
Clear 1 data space for new redo logs

metaclear -s oradg oradg/d310

Create 33 x Redo logs:

metainit -s oradg d602 -p oradg/d70 100m
metainit -s oradg d603 -p oradg/d70 100m
metainit -s oradg d604 -p oradg/d70 100m
metainit -s oradg d605 -p oradg/d70 100m
metainit -s oradg d606 -p oradg/d70 100m
metainit -s oradg d607 -p oradg/d70 100m
metainit -s oradg d608 -p oradg/d70 100m
metainit -s oradg d609 -p oradg/d70 100m
metainit -s oradg d610 -p oradg/d70 100m
metainit -s oradg d611 -p oradg/d70 100m
metainit -s oradg d612 -p oradg/d70 100m
metainit -s oradg d613 -p oradg/d70 100m
metainit -s oradg d614 -p oradg/d70 100m
metainit -s oradg d615 -p oradg/d70 100m
metainit -s oradg d616 -p oradg/d70 100m
metainit -s oradg d617 -p oradg/d70 100m
metainit -s oradg d618 -p oradg/d70 100m
metainit -s oradg d619 -p oradg/d70 100m
metainit -s oradg d620 -p oradg/d70 100m
metainit -s oradg d621 -p oradg/d70 100m
metainit -s oradg d622 -p oradg/d70 100m
metainit -s oradg d623 -p oradg/d70 100m
metainit -s oradg d624 -p oradg/d70 100m
metainit -s oradg d625 -p oradg/d70 100m
metainit -s oradg d626 -p oradg/d70 100m
metainit -s oradg d627 -p oradg/d70 100m
metainit -s oradg d628 -p oradg/d70 100m
metainit -s oradg d629 -p oradg/d70 100m
metainit -s oradg d630 -p oradg/d70 100m
metainit -s oradg d631 -p oradg/d70 100m
metainit -s oradg d632 -p oradg/d70 100m
metainit -s oradg d633 -p oradg/d70 100m

More slices add latter

/dev/md/oradg/rdsk/d634
/dev/md/oradg/rdsk/d635
/dev/md/oradg/rdsk/d636
/dev/md/oradg/rdsk/d637
/dev/md/oradg/rdsk/d638
/dev/md/oradg/rdsk/d639
/dev/md/oradg/rdsk/d640
/dev/md/oradg/rdsk/d641
/dev/md/oradg/rdsk/d642
/dev/md/oradg/rdsk/d643

Click here to get this in a script

Set the correct permissions on both nodes

chmod 600 /dev/md/oradg/rdsk/d100 ;chown oracle:dba /dev/md/oradg/rdsk/d100
chmod 600 /dev/md/oradg/rdsk/d101 ;chown oracle:dba /dev/md/oradg/rdsk/d101
chmod 600 /dev/md/oradg/rdsk/d102 ;chown oracle:dba /dev/md/oradg/rdsk/d102
chmod 600 /dev/md/oradg/rdsk/d103 ;chown oracle:dba /dev/md/oradg/rdsk/d103
chmod 600 /dev/md/oradg/rdsk/d104 ;chown oracle:dba /dev/md/oradg/rdsk/d104
chmod 600 /dev/md/oradg/rdsk/d105 ;chown oracle:dba /dev/md/oradg/rdsk/d105
chmod 600 /dev/md/oradg/rdsk/d106 ;chown oracle:dba /dev/md/oradg/rdsk/d106
chmod 600 /dev/md/oradg/rdsk/d107 ;chown oracle:dba /dev/md/oradg/rdsk/d107
chmod 600 /dev/md/oradg/rdsk/d108 ;chown oracle:dba /dev/md/oradg/rdsk/d108
chmod 600 /dev/md/oradg/rdsk/d109 ;chown oracle:dba /dev/md/oradg/rdsk/d109
chmod 600 /dev/md/oradg/rdsk/d110 ;chown oracle:dba /dev/md/oradg/rdsk/d110
chmod 600 /dev/md/oradg/rdsk/d200 ;chown oracle:dba /dev/md/oradg/rdsk/d200
chmod 600 /dev/md/oradg/rdsk/d201 ;chown oracle:dba /dev/md/oradg/rdsk/d201
chmod 600 /dev/md/oradg/rdsk/d202 ;chown oracle:dba /dev/md/oradg/rdsk/d202
chmod 600 /dev/md/oradg/rdsk/d203 ;chown oracle:dba /dev/md/oradg/rdsk/d203
chmod 600 /dev/md/oradg/rdsk/d204 ;chown oracle:dba /dev/md/oradg/rdsk/d204
chmod 600 /dev/md/oradg/rdsk/d205 ;chown oracle:dba /dev/md/oradg/rdsk/d205
chmod 600 /dev/md/oradg/rdsk/d206 ;chown oracle:dba /dev/md/oradg/rdsk/d206
chmod 600 /dev/md/oradg/rdsk/d207 ;chown oracle:dba /dev/md/oradg/rdsk/d207
chmod 600 /dev/md/oradg/rdsk/d208 ;chown oracle:dba /dev/md/oradg/rdsk/d208
chmod 600 /dev/md/oradg/rdsk/d209 ;chown oracle:dba /dev/md/oradg/rdsk/d209
chmod 600 /dev/md/oradg/rdsk/d210 ;chown oracle:dba /dev/md/oradg/rdsk/d210
chmod 600 /dev/md/oradg/rdsk/d300 ;chown oracle:dba /dev/md/oradg/rdsk/d300
chmod 600 /dev/md/oradg/rdsk/d301 ;chown oracle:dba /dev/md/oradg/rdsk/d301
chmod 600 /dev/md/oradg/rdsk/d302 ;chown oracle:dba /dev/md/oradg/rdsk/d302
chmod 600 /dev/md/oradg/rdsk/d303 ;chown oracle:dba /dev/md/oradg/rdsk/d303
chmod 600 /dev/md/oradg/rdsk/d304 ;chown oracle:dba /dev/md/oradg/rdsk/d304
chmod 600 /dev/md/oradg/rdsk/d305 ;chown oracle:dba /dev/md/oradg/rdsk/d305
chmod 600 /dev/md/oradg/rdsk/d306 ;chown oracle:dba /dev/md/oradg/rdsk/d306
chmod 600 /dev/md/oradg/rdsk/d307 ;chown oracle:dba /dev/md/oradg/rdsk/d307
chmod 600 /dev/md/oradg/rdsk/d308 ;chown oracle:dba /dev/md/oradg/rdsk/d308
chmod 600 /dev/md/oradg/rdsk/d309 ;chown oracle:dba /dev/md/oradg/rdsk/d309
chmod 600 /dev/md/oradg/rdsk/d310 ;chown oracle:dba /dev/md/oradg/rdsk/d310
chmod 600 /dev/md/oradg/rdsk/d400 ;chown oracle:dba /dev/md/oradg/rdsk/d400
chmod 600 /dev/md/oradg/rdsk/d401 ;chown oracle:dba /dev/md/oradg/rdsk/d401
chmod 600 /dev/md/oradg/rdsk/d402 ;chown oracle:dba /dev/md/oradg/rdsk/d402
chmod 600 /dev/md/oradg/rdsk/d403 ;chown oracle:dba /dev/md/oradg/rdsk/d403
chmod 600 /dev/md/oradg/rdsk/d404 ;chown oracle:dba /dev/md/oradg/rdsk/d404
chmod 600 /dev/md/oradg/rdsk/d405 ;chown oracle:dba /dev/md/oradg/rdsk/d405
chmod 600 /dev/md/oradg/rdsk/d406 ;chown oracle:dba /dev/md/oradg/rdsk/d406
chmod 600 /dev/md/oradg/rdsk/d407 ;chown oracle:dba /dev/md/oradg/rdsk/d407
chmod 600 /dev/md/oradg/rdsk/d408 ;chown oracle:dba /dev/md/oradg/rdsk/d408
chmod 600 /dev/md/oradg/rdsk/d409 ;chown oracle:dba /dev/md/oradg/rdsk/d409
chmod 600 /dev/md/oradg/rdsk/d410 ;chown oracle:dba /dev/md/oradg/rdsk/d410
chmod 600 /dev/md/oradg/rdsk/d411 ;chown oracle:dba /dev/md/oradg/rdsk/d411
chmod 600 /dev/md/oradg/rdsk/d412 ;chown oracle:dba /dev/md/oradg/rdsk/d412
chmod 600 /dev/md/oradg/rdsk/d413 ;chown oracle:dba /dev/md/oradg/rdsk/d413
chmod 600 /dev/md/oradg/rdsk/d414 ;chown oracle:dba /dev/md/oradg/rdsk/d414
chmod 600 /dev/md/oradg/rdsk/d415 ;chown oracle:dba /dev/md/oradg/rdsk/d415
chmod 600 /dev/md/oradg/rdsk/d416 ;chown oracle:dba /dev/md/oradg/rdsk/d416
chmod 600 /dev/md/oradg/rdsk/d500 ;chown oracle:dba /dev/md/oradg/rdsk/d500
chmod 600 /dev/md/oradg/rdsk/d501 ;chown oracle:dba /dev/md/oradg/rdsk/d501
chmod 600 /dev/md/oradg/rdsk/d502 ;chown oracle:dba /dev/md/oradg/rdsk/d502
chmod 600 /dev/md/oradg/rdsk/d503 ;chown oracle:dba /dev/md/oradg/rdsk/d503
chmod 600 /dev/md/oradg/rdsk/d504 ;chown oracle:dba /dev/md/oradg/rdsk/d504
chmod 600 /dev/md/oradg/rdsk/d505 ;chown oracle:dba /dev/md/oradg/rdsk/d505
chmod 600 /dev/md/oradg/rdsk/d506 ;chown oracle:dba /dev/md/oradg/rdsk/d506
chmod 600 /dev/md/oradg/rdsk/d507 ;chown oracle:dba /dev/md/oradg/rdsk/d507
chmod 600 /dev/md/oradg/rdsk/d508 ;chown oracle:dba /dev/md/oradg/rdsk/d508
chmod 600 /dev/md/oradg/rdsk/d509 ;chown oracle:dba /dev/md/oradg/rdsk/d509
chmod 600 /dev/md/oradg/rdsk/d510 ;chown oracle:dba /dev/md/oradg/rdsk/d510
chmod 600 /dev/md/oradg/rdsk/d511 ;chown oracle:dba /dev/md/oradg/rdsk/d511
chmod 600 /dev/md/oradg/rdsk/d512 ;chown oracle:dba /dev/md/oradg/rdsk/d512
chmod 600 /dev/md/oradg/rdsk/d513 ;chown oracle:dba /dev/md/oradg/rdsk/d513
chmod 600 /dev/md/oradg/rdsk/d514 ;chown oracle:dba /dev/md/oradg/rdsk/d514
chmod 600 /dev/md/oradg/rdsk/d515 ;chown oracle:dba /dev/md/oradg/rdsk/d515
chmod 600 /dev/md/oradg/rdsk/d516 ;chown oracle:dba /dev/md/oradg/rdsk/d516
chmod 600 /dev/md/oradg/rdsk/d0 ;chown oracle:dba /dev/md/oradg/rdsk/d0
chmod 600 /dev/md/oradg/rdsk/d10 ;chown oracle:dba /dev/md/oradg/rdsk/d10
chmod 600 /dev/md/oradg/rdsk/d20 ;chown oracle:dba /dev/md/oradg/rdsk/d20
chown oracle:dba  /dev/md/oradg/rdsk/d45 /dev/md/oradg/rdsk/d35
chmod 600 /dev/md/oradg/rdsk/d45 /dev/md/oradg/rdsk/d35

Add on latter

chown oracle:dba /dev/md/oradg/rdsk/d601 ;chmod 600 /dev/md/oradg/rdsk/d601
chown oracle:dba /dev/md/oradg/rdsk/d602 ;chmod 600 /dev/md/oradg/rdsk/d602
chown oracle:dba /dev/md/oradg/rdsk/d603 ;chmod 600 /dev/md/oradg/rdsk/d603
chown oracle:dba /dev/md/oradg/rdsk/d604 ;chmod 600 /dev/md/oradg/rdsk/d604
chown oracle:dba /dev/md/oradg/rdsk/d605 ;chmod 600 /dev/md/oradg/rdsk/d605
chown oracle:dba /dev/md/oradg/rdsk/d606 ;chmod 600 /dev/md/oradg/rdsk/d606
chown oracle:dba /dev/md/oradg/rdsk/d607 ;chmod 600 /dev/md/oradg/rdsk/d607
chown oracle:dba /dev/md/oradg/rdsk/d608 ;chmod 600 /dev/md/oradg/rdsk/d608
chown oracle:dba /dev/md/oradg/rdsk/d609 ;chmod 600 /dev/md/oradg/rdsk/d609
chown oracle:dba /dev/md/oradg/rdsk/d610 ;chmod 600 /dev/md/oradg/rdsk/d610
chown oracle:dba /dev/md/oradg/rdsk/d611 ;chmod 600 /dev/md/oradg/rdsk/d611
chown oracle:dba /dev/md/oradg/rdsk/d612 ;chmod 600 /dev/md/oradg/rdsk/d612
chown oracle:dba /dev/md/oradg/rdsk/d613 ;chmod 600 /dev/md/oradg/rdsk/d613
chown oracle:dba /dev/md/oradg/rdsk/d614 ;chmod 600 /dev/md/oradg/rdsk/d614
chown oracle:dba /dev/md/oradg/rdsk/d615 ;chmod 600 /dev/md/oradg/rdsk/d615
chown oracle:dba /dev/md/oradg/rdsk/d616 ;chmod 600 /dev/md/oradg/rdsk/d616
chown oracle:dba /dev/md/oradg/rdsk/d617 ;chmod 600 /dev/md/oradg/rdsk/d617
chown oracle:dba /dev/md/oradg/rdsk/d618 ;chmod 600 /dev/md/oradg/rdsk/d618
chown oracle:dba /dev/md/oradg/rdsk/d619 ;chmod 600 /dev/md/oradg/rdsk/d619
chown oracle:dba /dev/md/oradg/rdsk/d620 ;chmod 600 /dev/md/oradg/rdsk/d620
chown oracle:dba /dev/md/oradg/rdsk/d621 ;chmod 600 /dev/md/oradg/rdsk/d621
chown oracle:dba /dev/md/oradg/rdsk/d622 ;chmod 600 /dev/md/oradg/rdsk/d622
chown oracle:dba /dev/md/oradg/rdsk/d623 ;chmod 600 /dev/md/oradg/rdsk/d623
chown oracle:dba /dev/md/oradg/rdsk/d624 ;chmod 600 /dev/md/oradg/rdsk/d624
chown oracle:dba /dev/md/oradg/rdsk/d625 ;chmod 600 /dev/md/oradg/rdsk/d625
chown oracle:dba /dev/md/oradg/rdsk/d626 ;chmod 600 /dev/md/oradg/rdsk/d626
chown oracle:dba /dev/md/oradg/rdsk/d627 ;chmod 600 /dev/md/oradg/rdsk/d627
chown oracle:dba /dev/md/oradg/rdsk/d628 ;chmod 600 /dev/md/oradg/rdsk/d628
chown oracle:dba /dev/md/oradg/rdsk/d629 ;chmod 600 /dev/md/oradg/rdsk/d629
chown oracle:dba /dev/md/oradg/rdsk/d630 ;chmod 600 /dev/md/oradg/rdsk/d630
chown oracle:dba /dev/md/oradg/rdsk/d631 ;chmod 600 /dev/md/oradg/rdsk/d631
chown oracle:dba /dev/md/oradg/rdsk/d632 ;chmod 600 /dev/md/oradg/rdsk/d632
chown oracle:dba /dev/md/oradg/rdsk/d633 ;chmod 600 /dev/md/oradg/rdsk/d633

Add disk on latter

Note: Many similar disk configuration’s are snipped

Sun cluster device group configuration

Sun Cluster Device Group metaset Configuration

Setup the 3 needed Device Groups for Oracle RAC to auto failover with Sun cluster the metaset master

This could be done by using clsetup option 3 Data Services
To see all steps needed in this wizard Click Here.
Or you could run the list of commends below

/usr/cluster/bin/clresourcegroup create -p Desired_primaries=2 -p \
nodelist=db1,db2 -p RG_mode=Scalable -p RG_affinities=++rac-fmwk-rg -p Maximum_primaries=2 scaldg-rg
 
/usr/cluster/bin/clresourcetype register SUNW.ScalDeviceGroup:2
 
/usr/cluster/bin/clresource create -t SUNW.ScalDeviceGroup:2 -g scaldg-rg -p \
Resource_dependencies=rac-svm-rs -p DiskGroupName=oradg scaloradg-rs
 
/usr/cluster/bin/clresourcegroup online -emM scaldg-rg
 
/usr/cluster/bin/clresource create -t SUNW.ScalDeviceGroup:2 -g scaldg-rg -p \
Resource_dependencies=rac-svm-rs -p DiskGroupName=ocrdg scalocrdg-rs
 
/usr/cluster/bin/clresourcegroup online -emM scaldg-rg
 
/usr/cluster/bin/clresource create -t SUNW.ScalDeviceGroup:2 -g scaldg-rg -p \
Resource_dependencies=rac-svm-rs -p DiskGroupName=votingdg scalvotingdg-rs
 
/usr/cluster/bin/clresourcegroup online -emM scaldg-rg

To Get to Sun Cluster 3.2 and Oracle Real Application Cluster (RAC) Installation / configuration Part 2 Click Here