Tuesday, February 21, 2017

EMCA Configuration and Reconfiguration && run EMCA in multiple nodes

Use the below command  to run the emca in multiple nodes :

emca- reconfig-dbcontrol-cluster -EM_NODE ocluster02 -EM_NODE_LIST ocluster02

Here is the detailed explanation..

[oracle@ocluster02 ~]$ emca -reconfig dbcontrol -cluster -EM_NODE ocluster02 -EM_NODE_LIST ocluster02

STARTED EMCA at Feb 15, 2017 11:34:05 AM
EM Configuration Assistant, Version 11.2.0.3.0 Production
Copyright (c) 2003, 2011, Oracle.  All rights reserved.

Enter the following information:
Database unique name: ocracdb
Service name: ocracdb
Do you wish to continue? [yes(Y)/no(N)]: y
Feb 15, 2017 11:34:39 AM oracle.sysman.emcp.EMConfig perform
INFO: This operation is being logged at /u01/app/oracle/cfgtoollogs/emca/ocracdb/emca_2017_02_15_11_34_04.log.
Feb 15, 2017 11:34:42 AM oracle.sysman.emcp.util.DBControlUtil stopOMS
INFO: Stopping Database Control (this may take a while) ...
Feb 15, 2017 11:34:45 AM oracle.sysman.emcp.EMAgentConfig performDbcReconfiguration
INFO: Propagating /u01/app/oracle/product/11.2.0/dbhome_1/ocluster02_ocracdb/sysman/config/emd.properties to remote nodes ...
Feb 15, 2017 11:34:46 AM oracle.sysman.emcp.util.DBControlUtil startOMS
INFO: Starting Database Control (this may take a while) ...
Feb 15, 2017 11:35:19 AM oracle.sysman.emcp.EMDBPostConfig performDbcReconfiguration
INFO: Database Control started successfully
Feb 15, 2017 11:35:19 AM oracle.sysman.emcp.EMDBPostConfig showClusterDBCAgentMessage
INFO:
****************  Current Configuration  ****************
 INSTANCE            NODE           DBCONTROL_UPLOAD_HOST
----------        ----------        ---------------------

ocracdb           ocluster01           ocluster01.bli.com
ocracdb           ocluster02           ocluster02.bli.com


Enterprise Manager configuration completed successfully
FINISHED EMCA at Feb 15, 2017 11:35:19 AM

[oracle@ocluster02 ~]$ emctl status dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://ocluster02.bli.com:1158/em/console/aboutApplication
Oracle Enterprise Manager 11g is running.
------------------------------------------------------------------
Logs are generated in directory /u01/app/oracle/product/11.2.0/dbhome_1/ocluster02_ocracdb/sysman/log

Thursday, February 9, 2017

Query to identify columns/fields having special characters in Oracle


To identify the special characters in one column / field in  oracle you can use like operator or REGEXP.

It would be better to use REGEXP instead of like. Here is the sample query to find special characters.

select * from table where regexp_like(, '[0-9\-\@\<\>\_\!\@\#\$\%\^\&\*\(\)\;\"\?\=]');

In the above query you have to specify the special character's which you want to identify.

or you can use


select * from table  where not regexp_like(nama_lgkp, '.*[^a-zA-Z0-9]')

In the above query no need to specify the special characters it will give you all the special characters.

Wednesday, February 1, 2017

Check the status of running Clusterware & DB Status

Check the status of running clusterware & DB Status :::

Once completed the grid installation  need to check and verify the cluster services.

To check the status need to run the below commands. Here is the commands with output.
To run the below commands need to login in as grid user or grid home directory.
Check CRS Status ::

[grid@ocluster01 grid]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

Check Clusterware Resources :::

[grid@ocluster01 grid]$ crs_stat -t -v
Name           Type           R/RA   F/FT   Target    State     Host       
----------------------------------------------------------------------
ora.DATA.dg    ora....up.type 0/5    0/     ONLINE    ONLINE    ocluster01 
ora....ER.lsnr ora....er.type 0/5    0/     ONLINE    ONLINE    ocluster01 
ora....N1.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    ocluster02 
ora....N2.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    ocluster01 
ora....N3.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    ocluster01 
ora.asm        ora.asm.type   0/5    0/     ONLINE    ONLINE    ocluster01 
ora.cvu        ora.cvu.type   0/5    0/0    ONLINE    ONLINE    ocluster01 
ora.gsd        ora.gsd.type   0/5    0/     OFFLINE   OFFLINE              
ora....network ora....rk.type 0/5    0/     ONLINE    ONLINE    ocluster01 
ora.oc4j       ora.oc4j.type  0/1    0/2    ONLINE    ONLINE    ocluster01 
ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    ocluster01 
ora....01.lsnr application    0/5    0/0    ONLINE    ONLINE    ocluster01 
ora....r01.gsd application    0/5    0/0    OFFLINE   OFFLINE              
ora....r01.ons application    0/3    0/0    ONLINE    ONLINE    ocluster01 
ora....r01.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    ocluster01 
ora....SM2.asm application    0/5    0/0    ONLINE    ONLINE    ocluster02 
ora....02.lsnr application    0/5    0/0    ONLINE    ONLINE    ocluster02 
ora....r02.gsd application    0/5    0/0    OFFLINE   OFFLINE              
ora....r02.ons application    0/3    0/0    ONLINE    ONLINE    ocluster02 
ora....r02.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    ocluster02 
ora.ons        ora.ons.type   0/3    0/     ONLINE    ONLINE    ocluster01 
ora.scan1.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    ocluster02 
ora.scan2.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    ocluster01 
ora.scan3.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    ocluster01 
[grid@ocluster01 grid]$

Check Cluster Nodes :::

[grid@ocluster01 grid]$ olsnodes -n
ocluster01    1
ocluster02    2

Check Oracle TNS Listener Process on Both Nodes :::

[grid@ocluster01 grid]$ ps -ef | grep lsnr | grep -v 'grep' | grep -v 'ocfs' | awk '{print $9}'
LISTENER_SCAN2
LISTENER_SCAN3
LISTENER

Confirming Oracle ASM Function for Oracle Clusterware Files :::

[grid@ocluster01 grid]$ srvctl status asm -a
ASM is running on ocluster02,ocluster01
ASM is enabled.


Check Oracle Cluster Registry (OCR) :::

[grid@ocluster01 grid]$
[grid@ocluster01 grid]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
     Version                  :          3
     Total space (kbytes)     :     262120
     Used space (kbytes)      :       2672
     Available space (kbytes) :     259448
     ID                       :   16461733
     Device/File Name         :      +DATA
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

     Cluster registry integrity check succeeded

     Logical corruption check bypassed due to non-privileged user



Check Voting Disk :::

[grid@ocluster01 grid]$ crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5b7bf089cab44f30bf7ba5dd0337f7d9 (ORCL:ASMDISK1) [DATA]
Located 1 voting disk(s).


Step by Step Installation of Oracle RAC 11gR2 on Linux

Oracle Installation Prerequisites ::

Perform either the Automatic setup or the Manual setup to complete the basic prerequisites.

for Automatic setup you need the internet connection with configuration of the yum to http://public-yum.oracle.com , If using oracle linux.

run the blow command from root user.

# yum install oracle-validated

All the necessary prerequisites will be installed automatically.

# yum update

Manual setup ::

check the linux version : uname -r and update if it required.

[root@ocluster01 Ora-Soft]# uname -r
2.6.32-642.4.2.el6.x86_64
[root@ocluster01 Ora-Soft]# uname -a
Linux ocluster01.bli.com 2.6.32-642.4.2.el6.x86_64 #1 SMP Tue Aug 23 10:42:14 PDT 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@ocluster01 Ora-Soft]#

After update the packages :
[root@ocluster02 packages]# uname -r
2.6.32-642.4.2.el6.x86_64
[root@ocluster02 packages]# uname -a
Linux ocluster02.bli.com 2.6.32-642.4.2.el6.x86_64 #1 SMP Tue Aug 23 10:42:14 PDT 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@ocluster02 packages]#

In addition to the basic OS installation, following packages must be installed as the root user.
which includes 64-bit and 32-bit versions of few packages.

rpm -Uvh binutils-2*x86_64*
rpm -Uvh glibc-2*x86_64* nss-softokn-freebl-3*x86_64*
rpm -Uvh glibc-2*i686* nss-softokn-freebl-3*i686*
rpm -Uvh compat-libstdc++-33*x86_64*
rpm -Uvh glibc-common-2*x86_64*
rpm -Uvh glibc-devel-2*x86_64*
rpm -Uvh glibc-devel-2*i686*
rpm -Uvh glibc-headers-2*x86_64*
rpm -Uvh elfutils-libelf-0*x86_64*
rpm -Uvh elfutils-libelf-devel-0*x86_64*
rpm -Uvh gcc-4*x86_64*
rpm -Uvh gcc-c++-4*x86_64*
rpm -Uvh ksh-*x86_64*
rpm -Uvh libaio-0*x86_64*
rpm -Uvh libaio-devel-0*x86_64*
rpm -Uvh libaio-0*i686*
rpm -Uvh libaio-devel-0*i686*
rpm -Uvh libgcc-4*x86_64*
rpm -Uvh libgcc-4*i686*
rpm -Uvh libstdc++-4*x86_64*
rpm -Uvh libstdc++-4*i686*
rpm -Uvh libstdc++-devel-4*x86_64*
rpm -Uvh make-3.81*x86_64*
rpm -Uvh numactl-devel-2*x86_64*
rpm -Uvh sysstat-9*x86_64*
rpm -Uvh compat-libstdc++-33*i686*
rpm -Uvh compat-libcap*
rpm -Uvh  unixODBC*
rpm -Uvh unixODBC-devel*
rpm -Uvh gcc-4.1.2
rpm -Uvh elfutils-libelf-devel-0.125
rpm -Uvh glibc-devel-2.5
rpm -Uvh gcc-c++-4.1.2
rpm -Uvh libaio-devel-0.3.106
rpm -Uvh libstdc++-devel-4.1.2
rpm -Uvh unixODBC-2.2.11
rpm -Uvh unixODBC-devel-2.2.11
rpm -Uvh elfutils-libelf-devel-0.125
rpm -Uvh sysstat-7.0.2


After successfully installed the packages need to restart the server.

to cross check the packages installed or updated need to run the following command .
 [root@ocluster02 ~]# rpm -qa |grep 

Add or append the following lines to "/etc/sysctl.conf" file.

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 1054504960
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=1048586

After successfully add Run the following command to change the current kernel parameters.

/sbin/sysctl -p

Add the following lines to the "/etc/security/limits.conf" file.

grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536



Add the following lines to the "/etc/pam.d/login" file, if it does not already exist.

session    required     pam_limits.so


Creating groups and users :::
Create Groups and User for Grid Infrastructure

Lets start this section by creating the recommended OS groups and user for Grid Infrastructure on both Oracle RAC nodes:

groupadd -g 1000 oinstall
groupadd -g 1200 asmadmin
groupadd -g 1201 asmdba
groupadd -g 1202 asmoper

useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper,dba grid

#or( useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -d /home/grid -s /bin/bash -c "Grid Infrastructure Owner" grid)

Once you have created,you check the id of the particular user by issuing the command "id grid " the following result should come:

"id grid"
uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper)

Set the password for the grid account:
passwd grid



Create Groups and User for Oracle Database Software:::

Next, create the the recommended OS groups and user for the Oracle database software on both Oracle RAC nodes:

groupadd -g 1300 dba
groupadd -g 1301 oper
useradd -m -u 1101 -g oinstall -G dba,oper,asmdba  oracle

# or (useradd -m -u 1101 -g oinstall -G dba,oper,asmdba -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle)

id oracle
uid=1101(oracle) gid=1000(oinstall) groups=1000(oinstall),1201(asmdba),1300(dba),1301(oper)

Set the password for the oracle account:
passwd oracle


Create the directory in which the Oracle software will be installed.

Create directories :::

mkdir -p /u01/app/grid
mkdir -p /u01/app/oraInventory
mkdir -p /u01/app/grid/product/11.2.0/grid    --Grid home
mkdir -p /u01/app/oracle                                --Oracle base
mkdir -p /u01/app/oracle/product/11.2.0/dbhome_1    --Oracle home

Permissions:::

chown -R grid:oinstall /u01/app/grid/product/11.2.0/grid
chown -R grid:oinstall /u01/app/grid
chown -R oracle:oinstall /u01/app/oracle
chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/dbhome_1
chown -R grid:oinstall /u01/app/oraInventory
chmod -R 775 /u01/app/oraInventory
chmod -R 775 /u01

Create bash_profile for Grid user :::

ORACLE_SID=+ASM1; export ORACLE_SID
ORACLE_BASE=/u01/app/grid; export ORACLE_BASE
ORACLE_HOME=/u01/app/11.2.0/grid; export ORACLE_HOME
ORACLE_PATH=/u01/app/oracle/common/oracle/sql; export ORACLE_PATH
NLS_DATE_FORMAT="DD-MON-YYYY HH24:MI:SS"; export NLS_DATE_FORMAT
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH


Create bash_profile for Oracle user:::

ORACLE_SID=OCRACDB1; export ORACLE_SID
ORACLE_UNQNAME=ORCL; export ORACLE_UNQNAME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1; export ORACLE_HOME
ORACLE_PATH=/u01/app/common/oracle/sql; export ORACLE_PATH
JAVA_HOME=/usr/local/java; export JAVA_HOME
NLS_DATE_FORMAT="DD-MON-YYYY HH24:MI:SS"; export NLS_DATE_FORMAT
TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH


Change the setting of SELinux to disabled by editing the "/etc/selinux/config" file, make sure the SELINUX flag is set as follows.

Change Enforcing to disabled

SELINUX=disabled

Checking the raw devices (disks):::

Before format the disk check the available disks
[root@ocluster01 Ora-Soft]# ll /dev/sd*
brw-rw---- 1 root disk 8,  0 Sep 22 18:16 /dev/sda
brw-rw---- 1 root disk 8,  1 Sep 22 18:16 /dev/sda1
brw-rw---- 1 root disk 8,  2 Sep 22 18:16 /dev/sda2
brw-rw---- 1 root disk 8, 16 Sep 22 18:16 /dev/sdb
brw-rw---- 1 root disk 8, 32 Sep 22 18:16 /dev/sdc
brw-rw---- 1 root disk 8, 48 Sep 22 18:16 /dev/sdd
 
You can also check the disks by issuing the command  fdisk -l

Formatting the disks :::

[root@ocluster01 Ora-Soft]# fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x494214d4.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-261, default 1):  >> enter
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-261, default 261):  >> enter
Using default value 261

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@ocluster01 Ora-Soft]#

After successfully created and formatted the disks install the oraclelibs

Installing ORACLEASMLIB :: Do the following process from root user

[root@ocluster01 Ora-soft]# rpm -ivh oracleasm-support-2.1.8-1.el6.x86_64.rpm
warning: oracleasm-support-2.1.8-1.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...                ########################################### [100%]
   1:oracleasm-support      ########################################### [100%]
[root@ocluster01 Ora-soft]# rpm -ivh oracleasmlib-2.0.4-1.el6.x86_64.rpm
warning: oracleasmlib-2.0.4-1.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...                ########################################### [100%]
   1:oracleasmlib           ########################################### [100%]
[root@ocluster01 Ora-soft]# rpm -ivh cvuqdisk-1.0.9-1.rpm
Preparing...                ########################################### [100%]
Using default group oinstall to install package
   1:cvuqdisk               ########################################### [100%]
[root@ocluster01 Ora-soft]#


for configuration of the Oracleasm need to configure it by doing the following process.

[root@ocluster01 Ora-Soft]# /etc/init.d/oracleasm init
Usage: /etc/init.d/oracleasm {start|stop|restart|enable|disable|configure|createdisk|deletedisk|querydisk|listdisks|scandisks|status}

[root@ocluster02 Ora-Soft]# /etc/init.d/oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]




Create Oracle disks :::

[root@ocluster01 Ora-Soft]# /etc/init.d/oracleasm createdisk ASMDISK1 /dev/sdb1
Marking disk "ASMDISK1" as an ASM disk:                    [  OK  ]
[root@ocluster01 Ora-Soft]# /etc/init.d/oracleasm createdisk ASMDISK2 /dev/sdc1
Marking disk "ASMDISK2" as an ASM disk:                    [  OK  ]
[root@ocluster01 Ora-Soft]# /etc/init.d/oracleasm createdisk ASMDISK3 /dev/sdd1
Marking disk "ASMDISK3" as an ASM disk:                    [  OK  ]



[root@ocluster01 ~]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks:               [  OK  ]
[root@ocluster01 ~]# /etc/init.d/oracleasm listdisks
ASMDISK1
ASMDISK2
ASMDISK3
[root@ocluster01 ~]#

Configuration of SSH ::: 
SSH configuration is required for password less authentication to transfer the files between the servers. Configuration of the SSH is must require. To configuration of the SSH need to generate the key in all the nodes.
Below are the steps to generate the key and copy to other nodes for password less authentication.

[oracle@ocluster01 ~]$ ssh-keygen -t rsa

2. loing into oracle user in node1 and node2.
3. ssh-keygen -t rsa
[oracle@ocluster03 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Created directory '/home/oracle/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
d8:5a:2e:7e:25:43:91:f3:12:df:76:fb:86:fe:9c:e5 oracle@ocluster03.bli.com
The key's randomart image is:
+--[ RSA 2048]----+
|         .       |
|        =        |
|         * .     |
|       oo o o .  |
|      ..S. . . . |
|       +o .   .  |
|      o .+     o.|
|     . ..     .o=|
|      ..     ..+E|
+-----------------+
[oracle@ocluster03 ~]$

4. for every question just give enter.
5. in home directory there is hidden folder. in that folder the keys will be generated.
6. now we have to exchange the public keys into the other nodes.
7. to chekc connection was established or not. ssh hostname. (ssh oracle@OCluster02 hostname)
8. copy the public key of both nodes (cat id_rsa) and paste it in notepad as one line. and create new file called "authorized_keys" and past it there.

Checking SSH Connectivity for all nodes :::

[oracle@ocluster01 .ssh]$ ssh ocluster01 date
The authenticity of host '[ocluster01]:77 ([172.22.2.3]:77)' can't be established.
RSA key fingerprint is f5:46:02:89:be:82:ad:76:46:42:82:4a:cd:ee:d1:42.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[ocluster01]:77,[172.22.2.3]:77' (RSA) to the list of known hosts.
Thu Sep 22 18:58:13 WIB 2016
[oracle@ocluster01 .ssh]$ ssh ocluster02 date
The authenticity of host '[ocluster02]:77 ([172.22.2.4]:77)' can't be established.
RSA key fingerprint is f5:46:02:89:be:82:ad:76:46:42:82:4a:cd:ee:d1:42.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[ocluster02]:77,[172.22.2.4]:77' (RSA) to the list of known hosts.
Thu Sep 22 18:58:18 WIB 2016
[oracle@ocluster01 .ssh]$ ssh ocluster02.bli.com date
The authenticity of host '[ocluster02.bli.com]:77 ([172.22.2.4]:77)' can't be established.
RSA key fingerprint is f5:46:02:89:be:82:ad:76:46:42:82:4a:cd:ee:d1:42.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[ocluster02.bli.com]:77' (RSA) to the list of known hosts.
Thu Sep 22 18:58:26 WIB 2016
[oracle@ocluster01 .ssh]$ ssh ocluster01.bli.com date
The authenticity of host '[ocluster01.bli.com]:77 ([172.22.2.3]:77)' can't be established.
RSA key fingerprint is f5:46:02:89:be:82:ad:76:46:42:82:4a:cd:ee:d1:42.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[ocluster01.bli.com]:77' (RSA) to the list of known hosts.
Thu Sep 22 18:58:36 WIB 2016
[oracle@ocluster01 .ssh]$


Check for xhost + which is us-full for graphical interface to continue the installation.
  or if require install xclock and from oracle user you are able to view  xclock :

Follow the above procedure in all other nodes. If your installing in VM then take a sanapshot of the VM and create multiple nodes by changing the IPS and remain basic configuration.

As of now we have successfully completed the prerequisites for installation of Oracle.
better to reboot the all nodes.
========
After successful reboot the nodes run the below command in any one of the node to check pre installation compatibility.

./runcluvfy.sh stage -pre crsinst  -n ocluster01.bli.com,ocluster02.bli.com -r 11gR2 -verbose >/home/grid/crs_1checko.log

Check the log file and all the parameters should pass.If all the parameters are passed then start installation of grid from grid user.

Execute ./runInstaller.sh from grid user.

 Skip the software updates
Based on the requirement you have to choose the option in the above scree,Here we are going to install and configure the grid.
Click on Install and configure grid infrastructure for grid..
Click on Advanced Installation.
Select the language.
 Update the Cluster name,Scan name and port number. Uncheck the GNS

 Check the SSH connectivity
 SSH connectivity was successful.
 Check and update the appropriate public and private interface.
Select ASM.
Give the disk group name and select the disks 
 Password for system accounts
 select don't use IPMI
Update the appropriate groups

 select the Oracle base and software installation location.

 
Try to install the above listed packages using yum or local repository

 if not fixed, or still some packages need to install then use Fix & check again. Here you will get one script from oracle recommended. run the script and check again


 
 Check the above setting and the start installing by clicking finish.


 After successful installation of the grid run the both root scripts from both nodes one by one in given sequence first node1 and then node2
Once successfully run the root scripts then continue the installation.



Below is the sample output when you run the root.sh scripts.

[root@ocluster01 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@ocluster01 ~]# /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding Clusterware entries to upstart
CRS-2672: Attempting to start 'ora.mdnsd' on 'ocluster01'
CRS-2676: Start of 'ora.mdnsd' on 'ocluster01' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ocluster01'
CRS-2676: Start of 'ora.gpnpd' on 'ocluster01' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ocluster01'
CRS-2672: Attempting to start 'ora.gipcd' on 'ocluster01'
CRS-2676: Start of 'ora.cssdmonitor' on 'ocluster01' succeeded
CRS-2676: Start of 'ora.gipcd' on 'ocluster01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ocluster01'
CRS-2672: Attempting to start 'ora.diskmon' on 'ocluster01'
CRS-2676: Start of 'ora.diskmon' on 'ocluster01' succeeded
CRS-2676: Start of 'ora.cssd' on 'ocluster01' succeeded

ASM created and started successfully.

Disk Group DATA created successfully.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 5b7bf089cab44f30bf7ba5dd0337f7d9.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5b7bf089cab44f30bf7ba5dd0337f7d9 (ORCL:ASMDISK1) [DATA]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'ocluster01'
CRS-2676: Start of 'ora.asm' on 'ocluster01' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'ocluster01'
CRS-2676: Start of 'ora.DATA.dg' on 'ocluster01' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@ocluster01 ~]#



After successfully completed the grid installation. need to check the following things.



once completed the grid installation  need to check and verify the cluster services ::