SlideShare a Scribd company logo
1 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
2 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Understanding Oracle RAC Internals – Part 1
for the Oracle RAC SIG
Markus Michalewicz (Markus.Michalewicz@oracle.com)
Senior Principal Product Manager Oracle RAC and Oracle RAC One Node
3 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features
or functionality described for Oracle’s products remains at the sole discretion of Oracle.
4 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Agenda
• The Oracle RAC Architecture
• VIPs and Networks
• Listeners and SCAN
– and Services
5 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The Oracle RAC Architecture
Typical hardware architecture
Interconnect
with switch
Public Lan
SAN switch
Client
Shared Storage
Network
Node 1 Node 2 Node NNode N-1
6 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The Oracle RAC Architecture
Typical software deployment 11g Release 2
Shared Storage
OS OS OS OS
Oracle Grid Infrastructure
Oracle Real Application Clusters (RAC)
7 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The Oracle RAC Architecture
Typical software deployment 11g Release 2
Oracle ASM managed
Shared Storage
OS OS OS OS
Oracle Automatic Storage Management (ASM)
Oracle Clusterware
Oracle Real Application Clusters (RAC)
8 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The Oracle RAC Architecture
Oracle RAC 11g Release 2 per node layer
OS OS
Oracle Grid Infrastructure
Node
Membership
HA Framework
ASM Instance
OS
Oracle RAC
DB Instance
Oracle ASM managed
Shared Storage
• Oracle Clusterware files
• Voting Files (disks)
• Cluster Registry (OCR)
• Database Files
• Redo / Archive Logs
• Control Files
• etc.
• Cloud File System
(aka ASM Cluster File
File System - ACFS)
• For all other files
• 11.2.0.3: For Backup Sets
9 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The Oracle RAC Architecture
Oracle Grid Infrastructure 11g Release 2 process overview
OS OS
Oracle Grid Infrastructure
Node
Membership
HA Framework
ASM Instance
OS• My Oracle Support (MOS)
• Note 1053147.1 - 11gR2 Clusterware
and Grid Home - What You Need to Know
• Note 1050908.1 - How to Troubleshoot
Grid Infrastructure Startup Issues
10 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The Oracle RAC Architecture
Local and Cluster Resources (11.2.0.2 list)
OS OS OS OS
Oracle Grid Infrastructure
ora.asm
ora.dgname.dg
ora.listener.lsnr
ora.ons
ora.gsd
ora.net1.network
ora.registry.acfs
ora.asm
ora.dgname.dg
ora.listener.lsnr
ora.ons
ora.gsd
ora.net1.network
ora.registry.acfs
ora.asm
ora.dgname.dg
ora.listener.lsnr
ora.ons
ora.gsd
ora.net1.network
ora.registry.acfs
ora.asm
ora.dgname.dg
ora.listener.lsnr
ora.ons
ora.gsd
ora.net1.network
ora.registry.acfs
LISTENER_SCAN1
ora.SCAN1.VIP
ora.node1.vip
LISTENER_SCAN2
ora.SCAN2.VIP
ora.node2.vip
LISTENER_SCAN3
ora.SCAN3.VIP
ora.node3.vip
ora.oc4j
ora.cvu
ora.node4.vip
If database is deployed:
• ora.Dbname.db
• ora.Dbname.Srvcname.svc
Command to list:
• crsctl stat res -t
11 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
VIPs and Networks
12 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
VIPs and Networks
Oracle VIPs
• SCAN and node VIPs are cluster resources
• SCAN VIPs can float in the cluster
• Node VIPs per default run on specific nodes
• All VIPs on the public network (SCAN, node,
user or application VIPs) depend on the Oracle
network resource.
• The network resource monitors
the public network interface(s).
• IOCTL based checks vs. default gateway ping
• Default gateway checks as backup
• racgvip for backward compatibility only
OS OS
Oracle Grid Infrastructure
…
ora.listener.lsnr
…
ora.net1.network
LISTENER_SCAN1
ora.SCAN1.VIP
ora.rac1.vip
LISTENER_SCAN2
ora.SCAN2.VIP
ora.rac2.vip
…
ora.listener.lsnr
…
ora.net1.network
rac1 rac2
13 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
VIPs and Networks
Node VIP Details
[GRID]> crsctl status res ora.rac1.vip -p
NAME=ora.rac1.vip
TYPE=ora.cluster_vip_net1.type
...
AGENT_FILENAME=%CRS_HOME%/bin/orarootagent%CRS_EXE_SUFFIX%
...
CHECK_INTERVAL=1
CHECK_TIMEOUT=30
...
START_DEPENDENCIES=hard(ora.net1.network)
pullup(ora.net1.network)
...
STOP_DEPENDENCIES=hard(ora.net1.network)
...
USR_ORA_VIP=rac1-vip
VERSION=11.2.0.2.0
OS OS
Oracle Grid Infrastructure
…
ora.listener.lsnr
…
ora.net1.network
…
…
ora.rac1.vip
…
…
ora.rac2.vip
…
ora.listener.lsnr
…
ora.net1.network
rac1 rac2
1
2
3
14 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
VIPs and Networks
The network resource
OS OS
Oracle Grid Infrastructure
…
ora.listener.lsnr
…
ora.net1.network
…
…
ora.rac1.vip
…
…
ora.rac2.vip
…
ora.listener.lsnr
…
ora.net1.network
rac1 rac2
• The Oracle network resource monitors the
public network interface(s) (e.g. eth0).
• The default is network number 1 (net1)
• There is one network resource per subnet:
[GRID]> srvctl config network
Network exists:
1/192.168.0.0/255.255.255.0/eth0, type static
• New in 11.2.0.2:
• Direct management of the network resource
• srvctl verb network
• Network type support (static, dhcp, mixed)
15 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
VIPs and Networks
How to enable multiple networks in the cluster?
OS OS
Oracle Grid Infrastructure
…
ora.net1.network
ora.net2.network
…
…
ora.net1.network
ora.net2.network
…
[GRID]> srvctl add network –h
Adds a network configuration to the Oracle Clusterware.
Usage: srvctl add network
-k <net_num> network number
(default number is 1)
-S <subnet>/<netmask>/[if1[|if2...]]
NET address spec for network
-w <network_type> The network type
(static, dhcp, mixed)
-h Print usage
-v Verbose output
• For example:
srvctl add network -k 2 -S 10.1.1.0/255.255.255.0
1
16 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
VIPs and Networks
How to enable multiple networks in the cluster?
OS OS
Oracle Grid Infrastructure
…
ora.net1.network
ora.net2.network
…
…
ora.net1.network
ora.net2.network
…
[GRID]> srvctl add vip -h
Adds a VIP to the Oracle Clusterware.
Usage: srvctl add vip
-n <node_name> Node name
-A <vip_name|ip>/<netmask>/[if1[|if2...]]
VIP address spec for node applications
-k <net_num> network number
(default number is 1)
-h Print usage
-v Verbose output
• For example:
srvctl add vip -n rac1 -k 2 -A rac1-vip2/255.255.255.0
srvctl add vip -n rac2 -k 2 -A rac2-vip2/255.255.255.0
2
ora.rac1.vip
ora.rac1-vip2.vip
ora.rac2.vip
ora.rac2-vip2.vip
17 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
VIPs and Networks
Application / user and node VIPs
OS OS
Oracle Grid Infrastructure
…
ora.net1.network
ora.net2.network
…
…
ora.net1.network
ora.net2.network
…
[GRID]> crsctl stat res ora.rac1-vip2.vip -p
NAME=ora.rac1-vip2.vip
TYPE=ora.cluster_vip_net2.type
...
HOSTING_MEMBERS=rac1
...
START_DEPENDENCIES=hard(ora.net2.network)
pullup(ora.net2.network)
...
STOP_DEPENDENCIES=hard(ora.net2.network)
...
USR_ORA_VIP=rac1-vip2
VERSION=11.2.0.2.0
ora.rac1.vip
ora.rac1-vip2.vip
ora.rac2.vip
ora.rac2-vip2.vip
1
This is a node vip!
18 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
VIPs and Networks
Application / user and node VIPs
OS OS
Oracle Grid Infrastructure
…
ora.net1.network
ora.net2.network
…
…
ora.net1.network
ora.net2.network
…
[GRID]> appvipcfg create
-network=1
-ip 192.168.0.66
-vipname=myVIP
-user=root
[GRID]> appvipcfg create
-network=2
-ip 10.1.1.88
-vipname=myNewVIP
-user=root
ora.rac1.vip
ora.rac1-vip2.vip
ora.rac2.vip
ora.rac2-vip2.vip
2
myVIP myNewVIP
This is an application vip!
19 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
VIPs and Networks
Application / user and node VIPs
OS OS
Oracle Grid Infrastructure
…
ora.net1.network
ora.net2.network
…
…
ora.net1.network
ora.net2.network
…
[GRID]> crsctl stat res myVIP -p
NAME=myVIP
TYPE=app.appvip_net1.type
...
HOSTING_MEMBERS=rac1
...
RESTART_ATTEMPTS=0
...
USR_ORA_VIP=192.168.0.66
VERSION=11.2.0.2.0
• You can modify it as needed!
ora.rac1.vip
ora.rac1-vip2.vip
ora.rac2.vip
ora.rac2-vip2.vip
3
myVIP myNewVIP
This is an application vip!
20 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners and SCAN
21 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners
Listeners and dependencies
• In Oracle RAC 11g Release 2 there have been
three major changes regarding listeners:
1. Listeners per default now run from the
Oracle Grid Infrastructure (GRID) home
2. Listeners are managed by an Oracle agent
3. SCAN Listeners were introduced
• A listener depends on a VIP
• A listener depends indirectly on a network OS OS
Oracle Grid Infrastructure
…
ora.listener.lsnr
…
ora.net1.network
LISTENER_SCAN1
ora.SCAN1.VIP
ora.rac1.vip
LISTENER_SCAN2
ora.SCAN2.VIP
ora.rac2.vip
…
ora.listener.lsnr
…
ora.net1.network
rac1 rac2
22 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners
The default LISTENER
[GRID]> srvctl config listener
Name: LISTENER
Network: 1, Owner: oracle  Grid Software Owner
Home: <CRS home>
End points: TCP:1521
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
23 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners
The default LISTENER FAQ
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
• Can you define another port? - YES
• Can you define another owner? – YES
• Can you define another home? – YES, if really needed.
• Can I add another listener on another port? - YES
24 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The default LISTENER
Can you define another port? - YES
[GRID]> srvctl modify listener -l LISTENER -p TCP:1545
[GRID]> srvctl config listener
Name: LISTENER
Network: 1, Owner: oracle
Home: <CRS home>
End points: TCP:1545
• Check: <CRS HOME>/network/admin/endpoints_listener.ora before:
LISTENER_RAC1=(DESCRIPTION=(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=rac1-vip)
(PORT=1521))(ADDRESS=(PROTOCOL=TCP)
(HOST=192.168.0.51)(PORT=1521)(IP=FIRST))))
# line added by Agent
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
1
25 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The default LISTENER
Can you define another port? - YES
[GRID]> srvctl modify listener -l LISTENER -p TCP:1545
[GRID]> srvctl config listener
Name: LISTENER
Network: 1, Owner: oracle
Home: <CRS home>
End points: TCP:1545
• Check: <CRS HOME>/network/admin/endpoints_listener.ora after restart:
LISTENER_RAC1=(DESCRIPTION=(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=rac1-vip)
(PORT=1545))(ADDRESS=(PROTOCOL=TCP)
(HOST=192.168.0.51)(PORT=1545)(IP=FIRST))))
# line added by Agent
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
2
26 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The default LISTENER
Can you define another owner? - YES
[GRID]> srvctl modify listener -l LISTENER -u root
You need to be root…
[GRID]> srvctl config listener
Name: LISTENER
Network: 1, Owner: root  For demonstration
purposes only…
Home: <CRS home>
End points: TCP:1545
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
27 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
The default LISTENER
Can you define another home? - YES
[GRID]> srvctl modify listener -l LISTENER
-o /u01/app/oracle/product/11.2.0/dbhome_1
[GRID]> srvctl config listener
Name: LISTENER
Network: 1, Owner: oracle
Home: /u01/app/oracle/product/11.2.0/dbhome_1
End points: TCP:1545
• This may not be required.
• Instead, create a new listener from a separate home, if needed
and use the SRVCTL command as shown, but try using the GI home. OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
28 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners
Can I add another listener on another port? - YES
[GRID]> srvctl config listener
Name: LISTENER
Network: 1, Owner: oracle
Home: <CRS home>
End points: TCP:1521
Name: LISTENER2011
Network: 1, Owner: oracle
Home: <CRS home>
End points: TCP:2011
• MOS Note 220970.1 - RAC: Frequently Asked Questions
• How to use SCAN and node listeners with different ports?
• Add additional node listeners to the system using different ports
OS OS
Oracle Grid Infrastructure
rac1 rac2
…
ora.LISTENER.lsnr
ora.LISTENERK2.lsnr
ora.net1.network
…
ora.LISTENER.lsnr
ora.LISTENERK2.lsnr
ora.net1.network
29 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners
Can I add another listener on another network? - YES
[GRID]> srvctl add listener -l ListenerK2 -p 1544 -k 2
[GRID]> srvctl config listener
Name: LISTENER
Network: 1, Owner: oracle
Home: <CRS home>
End points: TCP:1545
Name: LISTENERK2
Network: 2, Owner: oracle
Home: <CRS home>
End points: TCP:1544
• MOS Note 1063571.1 - How to Configure A Second
Listener on a Separate Network in 11.2 Grid Infrastructure
OS OS
Oracle Grid Infrastructure
rac1 rac2
…
ora.LISTENER.lsnr
Ora.LISTENERK2.lsnr
ora.net1.network
ora.net2.network
…
…
ora.LISTENER.lsnr
ora.LISTENERK2.lsnr
ora.net1.network
ora.net2.network
…
30 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners
Remember – it’s just another listener …
[GRID]> srvctl config listener
Name: JUSTALISTENER
Network: 1, Owner: root
Home: <CRS home>
End points: TCP:1522
...
[GRID]> vi /u01/app/11.2.0/grid/network/admin/listener.ora
JUSTALISTENER=(DESCRIPTION=(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=IPC)(KEY=JUSTALISTENER))))
# line added by Agent
[GRID]> vi /u01/app/11.2.0/grid/network/admin/endpoints_listener.ora
JUSTALISTENER_RAC1=(DESCRIPTION=(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=rac1-vip)(PORT=1522))
(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.51)(PORT=1522)(IP=FIRST))))
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
LISTENER_SCAN1
ora.SCAN1.VIP
ora.rac1.vip
LISTENER_SCAN2
ora.SCAN2.VIP
ora.rac2.vip
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
1
31 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners
Remember – it’s just another listener …
[GRID]> vi /u01/app/11.2.0/grid/network/admin/listener.ora
JUSTALISTENER=(DESCRIPTION=(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=IPC)(RATE_LIMIT=10)  For demonstration purposes only…
(KEY=JUSTALISTENER)))) # line added by Agent
...
[GRID]> srvctl stop listener -l JustAListener
[GRID]> srvctl start listener -l JustAListener
[GRID]> vi /u01/app/11.2.0/grid/network/admin/listener.ora
...
[GRID]> vi /u01/app/11.2.0/grid/network/admin/listener.ora
JUSTALISTENER=(DESCRIPTION=(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=IPC)(RATE_LIMIT=10)
(KEY=JUSTALISTENER)))) # line added by Agent
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
LISTENER_SCAN1
ora.SCAN1.VIP
ora.rac1.vip
LISTENER_SCAN2
ora.SCAN2.VIP
ora.rac2.vip
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
2
32 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
SCAN
The basics and overview
OS OS
rac1 rac3
LISTENER_SCAN1
ora.SCAN1.VIP
…
LISTENER_SCAN3
ora.SCAN3.VIP
…
OS
rac2
LISTENER_SCAN2
ora.SCAN2.VIP
…
Oracle Grid Infrastructure
• In Oracle RAC 11g Release 2
SCAN Listeners were introduced
• There is one SCAN setup per cluster
• Read about SCAN:
• http://www.oracle.com/technetwork/database/
clustering/overview/scan-129069.pdf
• MOS Note 887522.1 –
11gR2 Grid Infrastructure Single Client
Access Name (SCAN) Explained
• SCAN consists of a bundle containing:
1. SCAN IP
2. Associated (dependency) SCAN Listener
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
SCAN
33 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
SCAN
The SCAN bundle
OS OS
rac1 rac3
LISTENER_SCAN1
ora.SCAN1.VIP
…
LISTENER_SCAN3
ora.SCAN3.VIP
…
OS
rac2
LISTENER_SCAN2
ora.SCAN2.VIP
…
Oracle Grid Infrastructure
• 3 SCANs are the default for HA and LB
• Regardless of number of nodes
• You can define less or more, if really needed.
• SCAN listener resources run with
an active dispersion dependency
• If you use more nodes in the cluster than
SCAN listeners are defined, no node should
run more than one SCAN bundle at a time
• If you use less nodes in the cluster than
SCAN listeners are defined, there will be
nodes running more than one SCAN bundle
at a time.
• SCAN VIP moves with the listener, if possible.
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
…
ora.LISTENER.lsnr
…
ora.net1.network
SCAN
1
34 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
SCAN
The SCAN bundle
SCAN
2
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
LISTENER_SCAN1
ora.SCAN1.VIP
…
…
…
…
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
[GRID]> srvctl config scan
SCAN name: cluster1, Network: 1/192.168.0.0/255.255.255.0/eth0
SCAN VIP name: scan1, IP: /cluster1.us.oracle.com/192.168.0.41
[GRID]> srvctl modify scan
Modifies the SCAN name.
Usage: srvctl modify scan
-n <scan_name> Domain name qualified SCAN name
-h Print usage
• http://www.oracle.com/technetwork/database/
clustering/overview/scan-129069.pdf page 2
35 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
SCAN
The SCAN bundle
3
SCAN
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
LISTENER_SCAN1
ora.SCAN1.VIP
…
…
…
…
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
[GRID]> srvctl add scan -h
Adds a SCAN VIP to the Oracle Clusterware.
Usage: srvctl add scan -n <scan_name>
-n <scan_name> Domain name qualified SCAN name
-k <net_num> network number (default number is 1)
-S <subnet>/<netmask>/[if1[|if2...]]
NET address spec for network
-h Print usage
• Note: SCAN can only operate on one network / in one subnet.
36 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
SCAN
The SCAN FAQ
• From MOS Note 220970.1 –
RAC: Frequently Asked Questions
• How to use SCAN and node listeners with different ports?
• With Oracle RAC 11g Release 2 using SCAN is the default.
• As with other listeners, there is no direct communication
between the node (listeners) and the SCAN listeners.
• Listeners are only aware of the instances and services served,
since the instances (PMON) register themselves and the services
they host with the listeners.
• The instances use the LOCAL and REMOTE Listener
parameters to know with which listeners to register.
• Listeners used for a client connection to Oracle RAC should be
managed by Oracle Clusterware and should be listening on an
Oracle managed VIP.
1
SCAN
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
LISTENER_SCAN1
ora.SCAN1.VIP
…
…
…
…
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
37 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
SCAN
The SCAN FAQ
• Can you define another port? - YES
• See MOS Note 220970.1 –
RAC: Frequently Asked Questions
• How to use SCAN and node listeners with different ports?
• Use srvctl modify scan_listener –p <newPort>
• Can you define another owner? – NO
• Can you define another home? – NO
• Can you have more than one node listener with SCAN? – YES
• Can the SCAN and the node listener ports differ? - YES
• See also MOS Note 972500.1 – How to
Modify SCAN Setting or SCAN Listener Port after Installation
2
SCAN
OS OS
Oracle Grid Infrastructure
…
ora.LISTENER.lsnr
…
ora.net1.network
LISTENER_SCAN1
ora.SCAN1.VIP
…
…
…
…
…
ora.LISTENER.lsnr
…
ora.net1.network
rac1 rac2
38 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners and Instance Registration
Listener changes may require instance changes
• See MOS Note 220970.1 –
RAC: Frequently Asked Questions
• How to use SCAN and node listeners with different ports?
• Whenever you change an instance parameter configuration
a change to the LOCAL_LISTENER or REMOTE_LISTENER
per database (instance) may be required.
• Note that using server-side TNSANMES aliases to
resolve the listener configuration is not recommended.
• The use of the TNSNAMES connector string is the default:
local_listener = ‘(DESCRIPTION= (ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.61)(PORT=2011))
))'
SCAN
OS OS
Oracle Grid Infrastructure
ora.LISTENER.lsnr
…
LISTENER_SCAN1
ora.SCAN1.VIP
…
…
…
…
ora.LISTENER.lsnr
…
rac1 rac2
DB Instance DB Instance
39 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners and Instance Registration
EXAMPLE
• Taken from MOS Note 220970.1 – RAC: FAQ
• How to use SCAN and node listeners with different ports?
• Add additional node listeners to the system using different ports
• srvctl add listener -l LISTENER2011 -p 2011
• srvctl start listener -l LISTENER2011
• srvctl status listener -l LISTENER2011
• lsnrctl status LISTENER2011
[GRID]> srvctl config listener
Name: LISTENER2011
Network: 1, Owner: oracle
Home: <CRS home>
End points: TCP:2011
SCAN
OS OS
Oracle Grid Infrastructure
ora.LISTENER.lsnr
…
LISTENER_SCAN1
ora.SCAN1.VIP
…
…
…
…
ora.LISTENER.lsnr
…
rac1 rac2
DB Instance DB Instance
1
40 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners and Instance Registration
EXAMPLE
• Taken from MOS Note 220970.1 – RAC: FAQ
• How to use SCAN and node listeners with different ports?
• Add additional node listeners to the system using different ports
• New node listener does not serve any database instance
• You must alter the LOCAL_LISTENER parameters
for each instance accordingly:
alter system set local_listener =‘
(DESCRIPTION= (ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.61)(PORT=2011))))
' scope=BOTH SID='OCRL1';
alter system register;
SCAN
OS OS
Oracle Grid Infrastructure
ora.LISTENER.lsnr
…
LISTENER_SCAN1
ora.SCAN1.VIP
…
…
…
…
ora.LISTENER.lsnr
…
rac1 rac2
DB Instance DB Instance
2
41 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners and Instance Registration
EXAMPLE
• Taken from MOS Note 220970.1 – RAC: FAQ
• How to use SCAN and node listeners with different ports?
• Add additional node listeners to the system using different ports
• Note: Use spaces between the various segments of the command
• Reason: the database agent in Oracle Clusterware currently
determines whether the local_listener or remote_listener have
been manually set by a string comparison operation.
• If the string looks unaltered it will reset these parameters
with the default values that it determines on instance startup.
• Slight modifications are therefore required. ER 11772838
has been filed to allow for a more convenient mechanism.
SCAN
OS OS
Oracle Grid Infrastructure
ora.LISTENER.lsnr
…
LISTENER_SCAN1
ora.SCAN1.VIP
…
…
…
…
ora.LISTENER.lsnr
…
rac1 rac2
DB Instance DB Instance
3
42 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners and Instance Registration
Using listeners in different subnets / networks
• SCAN can only operate in one network / subnet
• If listeners in different subnets are used,
• use LISTENER_NETWORKS:
http://download.oracle.com/docs/cd/E11882_01
/server.112/e17110/initparams115.htm#REFRN10313
• Note: Listeners specified by the LISTENER_NETWORKS
parameter should not be used in the LOCAL_LISTENER and
REMOTE_LISTENER parameters. Otherwise, cross registration
will happen and connections will be redirected cross networks.
• For more information, see also
MOS Note 1063571.1 – How to Configure A Second
Listener on a Separate Network in 11.2 Grid Infrastructure
OS OS
Oracle Grid Infrastructure
LISTENER_SCAN1
ora.SCAN1.VIP
…
…
…
…
rac1 rac2
DB Instance DB Instance
…
ora.LISTENER.lsnr
Ora.LISTENERK2.lsnr
ora.net1.network
ora.net2.network
…
…
ora.LISTENER.lsnr
ora.LISTENERK2.lsnr
ora.net1.network
ora.net2.network
…
43 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners, SCAN and Services
44 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Listeners and Service Registration
A summary
• Services depend on a database
• Per default, the database instance will register
all services it hosts with all listeners it registers with.
• Services operate within the following boundaries:
• Network (indirectly via dependency)
• Server Pools (in policy-managed database environments)
• Currently there is no support for service failover
• Between Server Pools
• Between networks
SCAN
OS OS
Oracle Grid Infrastructure
ora.LISTENER.lsnr
…
ora.Dbname.db
ora.Dbname.Srvcname.svc
…
ora.Dbname.db
ora.Dbname.Srvcname.svc
…
ora.LISTENER.lsnr
…
rac1 rac2
DB Instance DB Instance
45 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Services and Dependencies
Dependency on DB and Server Pool
[GRID]> srvctl config service -d ORCL -s MyService
Service name: MyService
Service is enabled
Server pool: ORCL
Cardinality: 1
Disconnect: false
Service role: PRIMARY
Management policy: AUTOMATIC
...
Preferred instances: ORCL_1
Available instances:
SCAN
OS OS
Oracle Grid Infrastructure
ora.LISTENER.lsnr
…
ora.ORCL.db
ora.ORCL.MyService.svc
…
ora.ORCL.db
ora.Dbname.Srvcname.svc
…
ora.LISTENER.lsnr
…
rac1 rac2
DB Instance DB Instance
1
46 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Services and Dependencies
Start / Stop dependencies
[GRID]> crsctl stat res ora.orcl.myservice.svc -p
...
DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=service)
PROPERTY(SERVICE_NAME=%GEN_SERVICE_NAME%)
PROPERTY(DB_UNIQUE_NAME=CONCAT(PARSE(%NAME%, .,
2), STAT(ora.orcl.db, USR_ORA_DOMAIN), .))
ELEMENT(INSTANCE_NAME=STAT(ora.orcl.db,
GEN_USR_ORA_INST_NAME))
...
START_DEPENDENCIES=hard(ora.orcl.db,type:ora.cluster
_vip_net1.type) weak(type:ora.listener.type)
pullup(type:ora.cluster_vip_net1.type)
pullup:always(ora.orcl.db)
dispersion(type:ora.service.type)
STOP_DEPENDENCIES=hard(intermediate:ora.orcl.db,inte
rmediate:type:ora.cluster_vip_net1.type)
...
SCAN
OS OS
Oracle Grid Infrastructure
ora.LISTENER.lsnr
…
ora.Dbname.db
ora.Dbname.Srvcname.svc
…
ora.LISTENER.lsnr
…
rac1 rac2
DB Instance DB Instance
2
ora.ORCL.db
ora.ORCL.MyService.svc
…
47 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
48 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

More Related Content

Understanding oracle rac internals part 1 - slides

  • 1. 1 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  • 2. 2 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Understanding Oracle RAC Internals – Part 1 for the Oracle RAC SIG Markus Michalewicz (Markus.Michalewicz@oracle.com) Senior Principal Product Manager Oracle RAC and Oracle RAC One Node
  • 3. 3 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  • 4. 4 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Agenda • The Oracle RAC Architecture • VIPs and Networks • Listeners and SCAN – and Services
  • 5. 5 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The Oracle RAC Architecture Typical hardware architecture Interconnect with switch Public Lan SAN switch Client Shared Storage Network Node 1 Node 2 Node NNode N-1
  • 6. 6 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The Oracle RAC Architecture Typical software deployment 11g Release 2 Shared Storage OS OS OS OS Oracle Grid Infrastructure Oracle Real Application Clusters (RAC)
  • 7. 7 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The Oracle RAC Architecture Typical software deployment 11g Release 2 Oracle ASM managed Shared Storage OS OS OS OS Oracle Automatic Storage Management (ASM) Oracle Clusterware Oracle Real Application Clusters (RAC)
  • 8. 8 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The Oracle RAC Architecture Oracle RAC 11g Release 2 per node layer OS OS Oracle Grid Infrastructure Node Membership HA Framework ASM Instance OS Oracle RAC DB Instance Oracle ASM managed Shared Storage • Oracle Clusterware files • Voting Files (disks) • Cluster Registry (OCR) • Database Files • Redo / Archive Logs • Control Files • etc. • Cloud File System (aka ASM Cluster File File System - ACFS) • For all other files • 11.2.0.3: For Backup Sets
  • 9. 9 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The Oracle RAC Architecture Oracle Grid Infrastructure 11g Release 2 process overview OS OS Oracle Grid Infrastructure Node Membership HA Framework ASM Instance OS• My Oracle Support (MOS) • Note 1053147.1 - 11gR2 Clusterware and Grid Home - What You Need to Know • Note 1050908.1 - How to Troubleshoot Grid Infrastructure Startup Issues
  • 10. 10 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The Oracle RAC Architecture Local and Cluster Resources (11.2.0.2 list) OS OS OS OS Oracle Grid Infrastructure ora.asm ora.dgname.dg ora.listener.lsnr ora.ons ora.gsd ora.net1.network ora.registry.acfs ora.asm ora.dgname.dg ora.listener.lsnr ora.ons ora.gsd ora.net1.network ora.registry.acfs ora.asm ora.dgname.dg ora.listener.lsnr ora.ons ora.gsd ora.net1.network ora.registry.acfs ora.asm ora.dgname.dg ora.listener.lsnr ora.ons ora.gsd ora.net1.network ora.registry.acfs LISTENER_SCAN1 ora.SCAN1.VIP ora.node1.vip LISTENER_SCAN2 ora.SCAN2.VIP ora.node2.vip LISTENER_SCAN3 ora.SCAN3.VIP ora.node3.vip ora.oc4j ora.cvu ora.node4.vip If database is deployed: • ora.Dbname.db • ora.Dbname.Srvcname.svc Command to list: • crsctl stat res -t
  • 11. 11 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. VIPs and Networks
  • 12. 12 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. VIPs and Networks Oracle VIPs • SCAN and node VIPs are cluster resources • SCAN VIPs can float in the cluster • Node VIPs per default run on specific nodes • All VIPs on the public network (SCAN, node, user or application VIPs) depend on the Oracle network resource. • The network resource monitors the public network interface(s). • IOCTL based checks vs. default gateway ping • Default gateway checks as backup • racgvip for backward compatibility only OS OS Oracle Grid Infrastructure … ora.listener.lsnr … ora.net1.network LISTENER_SCAN1 ora.SCAN1.VIP ora.rac1.vip LISTENER_SCAN2 ora.SCAN2.VIP ora.rac2.vip … ora.listener.lsnr … ora.net1.network rac1 rac2
  • 13. 13 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. VIPs and Networks Node VIP Details [GRID]> crsctl status res ora.rac1.vip -p NAME=ora.rac1.vip TYPE=ora.cluster_vip_net1.type ... AGENT_FILENAME=%CRS_HOME%/bin/orarootagent%CRS_EXE_SUFFIX% ... CHECK_INTERVAL=1 CHECK_TIMEOUT=30 ... START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network) ... STOP_DEPENDENCIES=hard(ora.net1.network) ... USR_ORA_VIP=rac1-vip VERSION=11.2.0.2.0 OS OS Oracle Grid Infrastructure … ora.listener.lsnr … ora.net1.network … … ora.rac1.vip … … ora.rac2.vip … ora.listener.lsnr … ora.net1.network rac1 rac2 1 2 3
  • 14. 14 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. VIPs and Networks The network resource OS OS Oracle Grid Infrastructure … ora.listener.lsnr … ora.net1.network … … ora.rac1.vip … … ora.rac2.vip … ora.listener.lsnr … ora.net1.network rac1 rac2 • The Oracle network resource monitors the public network interface(s) (e.g. eth0). • The default is network number 1 (net1) • There is one network resource per subnet: [GRID]> srvctl config network Network exists: 1/192.168.0.0/255.255.255.0/eth0, type static • New in 11.2.0.2: • Direct management of the network resource • srvctl verb network • Network type support (static, dhcp, mixed)
  • 15. 15 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. VIPs and Networks How to enable multiple networks in the cluster? OS OS Oracle Grid Infrastructure … ora.net1.network ora.net2.network … … ora.net1.network ora.net2.network … [GRID]> srvctl add network –h Adds a network configuration to the Oracle Clusterware. Usage: srvctl add network -k <net_num> network number (default number is 1) -S <subnet>/<netmask>/[if1[|if2...]] NET address spec for network -w <network_type> The network type (static, dhcp, mixed) -h Print usage -v Verbose output • For example: srvctl add network -k 2 -S 10.1.1.0/255.255.255.0 1
  • 16. 16 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. VIPs and Networks How to enable multiple networks in the cluster? OS OS Oracle Grid Infrastructure … ora.net1.network ora.net2.network … … ora.net1.network ora.net2.network … [GRID]> srvctl add vip -h Adds a VIP to the Oracle Clusterware. Usage: srvctl add vip -n <node_name> Node name -A <vip_name|ip>/<netmask>/[if1[|if2...]] VIP address spec for node applications -k <net_num> network number (default number is 1) -h Print usage -v Verbose output • For example: srvctl add vip -n rac1 -k 2 -A rac1-vip2/255.255.255.0 srvctl add vip -n rac2 -k 2 -A rac2-vip2/255.255.255.0 2 ora.rac1.vip ora.rac1-vip2.vip ora.rac2.vip ora.rac2-vip2.vip
  • 17. 17 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. VIPs and Networks Application / user and node VIPs OS OS Oracle Grid Infrastructure … ora.net1.network ora.net2.network … … ora.net1.network ora.net2.network … [GRID]> crsctl stat res ora.rac1-vip2.vip -p NAME=ora.rac1-vip2.vip TYPE=ora.cluster_vip_net2.type ... HOSTING_MEMBERS=rac1 ... START_DEPENDENCIES=hard(ora.net2.network) pullup(ora.net2.network) ... STOP_DEPENDENCIES=hard(ora.net2.network) ... USR_ORA_VIP=rac1-vip2 VERSION=11.2.0.2.0 ora.rac1.vip ora.rac1-vip2.vip ora.rac2.vip ora.rac2-vip2.vip 1 This is a node vip!
  • 18. 18 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. VIPs and Networks Application / user and node VIPs OS OS Oracle Grid Infrastructure … ora.net1.network ora.net2.network … … ora.net1.network ora.net2.network … [GRID]> appvipcfg create -network=1 -ip 192.168.0.66 -vipname=myVIP -user=root [GRID]> appvipcfg create -network=2 -ip 10.1.1.88 -vipname=myNewVIP -user=root ora.rac1.vip ora.rac1-vip2.vip ora.rac2.vip ora.rac2-vip2.vip 2 myVIP myNewVIP This is an application vip!
  • 19. 19 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. VIPs and Networks Application / user and node VIPs OS OS Oracle Grid Infrastructure … ora.net1.network ora.net2.network … … ora.net1.network ora.net2.network … [GRID]> crsctl stat res myVIP -p NAME=myVIP TYPE=app.appvip_net1.type ... HOSTING_MEMBERS=rac1 ... RESTART_ATTEMPTS=0 ... USR_ORA_VIP=192.168.0.66 VERSION=11.2.0.2.0 • You can modify it as needed! ora.rac1.vip ora.rac1-vip2.vip ora.rac2.vip ora.rac2-vip2.vip 3 myVIP myNewVIP This is an application vip!
  • 20. 20 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners and SCAN
  • 21. 21 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners Listeners and dependencies • In Oracle RAC 11g Release 2 there have been three major changes regarding listeners: 1. Listeners per default now run from the Oracle Grid Infrastructure (GRID) home 2. Listeners are managed by an Oracle agent 3. SCAN Listeners were introduced • A listener depends on a VIP • A listener depends indirectly on a network OS OS Oracle Grid Infrastructure … ora.listener.lsnr … ora.net1.network LISTENER_SCAN1 ora.SCAN1.VIP ora.rac1.vip LISTENER_SCAN2 ora.SCAN2.VIP ora.rac2.vip … ora.listener.lsnr … ora.net1.network rac1 rac2
  • 22. 22 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners The default LISTENER [GRID]> srvctl config listener Name: LISTENER Network: 1, Owner: oracle  Grid Software Owner Home: <CRS home> End points: TCP:1521 OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network rac1 rac2
  • 23. 23 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners The default LISTENER FAQ OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network rac1 rac2 • Can you define another port? - YES • Can you define another owner? – YES • Can you define another home? – YES, if really needed. • Can I add another listener on another port? - YES
  • 24. 24 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The default LISTENER Can you define another port? - YES [GRID]> srvctl modify listener -l LISTENER -p TCP:1545 [GRID]> srvctl config listener Name: LISTENER Network: 1, Owner: oracle Home: <CRS home> End points: TCP:1545 • Check: <CRS HOME>/network/admin/endpoints_listener.ora before: LISTENER_RAC1=(DESCRIPTION=(ADDRESS_LIST= (ADDRESS=(PROTOCOL=TCP)(HOST=rac1-vip) (PORT=1521))(ADDRESS=(PROTOCOL=TCP) (HOST=192.168.0.51)(PORT=1521)(IP=FIRST)))) # line added by Agent OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network rac1 rac2 1
  • 25. 25 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The default LISTENER Can you define another port? - YES [GRID]> srvctl modify listener -l LISTENER -p TCP:1545 [GRID]> srvctl config listener Name: LISTENER Network: 1, Owner: oracle Home: <CRS home> End points: TCP:1545 • Check: <CRS HOME>/network/admin/endpoints_listener.ora after restart: LISTENER_RAC1=(DESCRIPTION=(ADDRESS_LIST= (ADDRESS=(PROTOCOL=TCP)(HOST=rac1-vip) (PORT=1545))(ADDRESS=(PROTOCOL=TCP) (HOST=192.168.0.51)(PORT=1545)(IP=FIRST)))) # line added by Agent OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network rac1 rac2 2
  • 26. 26 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The default LISTENER Can you define another owner? - YES [GRID]> srvctl modify listener -l LISTENER -u root You need to be root… [GRID]> srvctl config listener Name: LISTENER Network: 1, Owner: root  For demonstration purposes only… Home: <CRS home> End points: TCP:1545 OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network rac1 rac2
  • 27. 27 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. The default LISTENER Can you define another home? - YES [GRID]> srvctl modify listener -l LISTENER -o /u01/app/oracle/product/11.2.0/dbhome_1 [GRID]> srvctl config listener Name: LISTENER Network: 1, Owner: oracle Home: /u01/app/oracle/product/11.2.0/dbhome_1 End points: TCP:1545 • This may not be required. • Instead, create a new listener from a separate home, if needed and use the SRVCTL command as shown, but try using the GI home. OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network rac1 rac2
  • 28. 28 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners Can I add another listener on another port? - YES [GRID]> srvctl config listener Name: LISTENER Network: 1, Owner: oracle Home: <CRS home> End points: TCP:1521 Name: LISTENER2011 Network: 1, Owner: oracle Home: <CRS home> End points: TCP:2011 • MOS Note 220970.1 - RAC: Frequently Asked Questions • How to use SCAN and node listeners with different ports? • Add additional node listeners to the system using different ports OS OS Oracle Grid Infrastructure rac1 rac2 … ora.LISTENER.lsnr ora.LISTENERK2.lsnr ora.net1.network … ora.LISTENER.lsnr ora.LISTENERK2.lsnr ora.net1.network
  • 29. 29 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners Can I add another listener on another network? - YES [GRID]> srvctl add listener -l ListenerK2 -p 1544 -k 2 [GRID]> srvctl config listener Name: LISTENER Network: 1, Owner: oracle Home: <CRS home> End points: TCP:1545 Name: LISTENERK2 Network: 2, Owner: oracle Home: <CRS home> End points: TCP:1544 • MOS Note 1063571.1 - How to Configure A Second Listener on a Separate Network in 11.2 Grid Infrastructure OS OS Oracle Grid Infrastructure rac1 rac2 … ora.LISTENER.lsnr Ora.LISTENERK2.lsnr ora.net1.network ora.net2.network … … ora.LISTENER.lsnr ora.LISTENERK2.lsnr ora.net1.network ora.net2.network …
  • 30. 30 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners Remember – it’s just another listener … [GRID]> srvctl config listener Name: JUSTALISTENER Network: 1, Owner: root Home: <CRS home> End points: TCP:1522 ... [GRID]> vi /u01/app/11.2.0/grid/network/admin/listener.ora JUSTALISTENER=(DESCRIPTION=(ADDRESS_LIST= (ADDRESS=(PROTOCOL=IPC)(KEY=JUSTALISTENER)))) # line added by Agent [GRID]> vi /u01/app/11.2.0/grid/network/admin/endpoints_listener.ora JUSTALISTENER_RAC1=(DESCRIPTION=(ADDRESS_LIST= (ADDRESS=(PROTOCOL=TCP)(HOST=rac1-vip)(PORT=1522)) (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.51)(PORT=1522)(IP=FIRST)))) OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network LISTENER_SCAN1 ora.SCAN1.VIP ora.rac1.vip LISTENER_SCAN2 ora.SCAN2.VIP ora.rac2.vip … ora.LISTENER.lsnr … ora.net1.network rac1 rac2 1
  • 31. 31 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners Remember – it’s just another listener … [GRID]> vi /u01/app/11.2.0/grid/network/admin/listener.ora JUSTALISTENER=(DESCRIPTION=(ADDRESS_LIST= (ADDRESS=(PROTOCOL=IPC)(RATE_LIMIT=10)  For demonstration purposes only… (KEY=JUSTALISTENER)))) # line added by Agent ... [GRID]> srvctl stop listener -l JustAListener [GRID]> srvctl start listener -l JustAListener [GRID]> vi /u01/app/11.2.0/grid/network/admin/listener.ora ... [GRID]> vi /u01/app/11.2.0/grid/network/admin/listener.ora JUSTALISTENER=(DESCRIPTION=(ADDRESS_LIST= (ADDRESS=(PROTOCOL=IPC)(RATE_LIMIT=10) (KEY=JUSTALISTENER)))) # line added by Agent OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network LISTENER_SCAN1 ora.SCAN1.VIP ora.rac1.vip LISTENER_SCAN2 ora.SCAN2.VIP ora.rac2.vip … ora.LISTENER.lsnr … ora.net1.network rac1 rac2 2
  • 32. 32 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. SCAN The basics and overview OS OS rac1 rac3 LISTENER_SCAN1 ora.SCAN1.VIP … LISTENER_SCAN3 ora.SCAN3.VIP … OS rac2 LISTENER_SCAN2 ora.SCAN2.VIP … Oracle Grid Infrastructure • In Oracle RAC 11g Release 2 SCAN Listeners were introduced • There is one SCAN setup per cluster • Read about SCAN: • http://www.oracle.com/technetwork/database/ clustering/overview/scan-129069.pdf • MOS Note 887522.1 – 11gR2 Grid Infrastructure Single Client Access Name (SCAN) Explained • SCAN consists of a bundle containing: 1. SCAN IP 2. Associated (dependency) SCAN Listener … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network SCAN
  • 33. 33 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. SCAN The SCAN bundle OS OS rac1 rac3 LISTENER_SCAN1 ora.SCAN1.VIP … LISTENER_SCAN3 ora.SCAN3.VIP … OS rac2 LISTENER_SCAN2 ora.SCAN2.VIP … Oracle Grid Infrastructure • 3 SCANs are the default for HA and LB • Regardless of number of nodes • You can define less or more, if really needed. • SCAN listener resources run with an active dispersion dependency • If you use more nodes in the cluster than SCAN listeners are defined, no node should run more than one SCAN bundle at a time • If you use less nodes in the cluster than SCAN listeners are defined, there will be nodes running more than one SCAN bundle at a time. • SCAN VIP moves with the listener, if possible. … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network … ora.LISTENER.lsnr … ora.net1.network SCAN 1
  • 34. 34 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. SCAN The SCAN bundle SCAN 2 OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network LISTENER_SCAN1 ora.SCAN1.VIP … … … … … ora.LISTENER.lsnr … ora.net1.network rac1 rac2 [GRID]> srvctl config scan SCAN name: cluster1, Network: 1/192.168.0.0/255.255.255.0/eth0 SCAN VIP name: scan1, IP: /cluster1.us.oracle.com/192.168.0.41 [GRID]> srvctl modify scan Modifies the SCAN name. Usage: srvctl modify scan -n <scan_name> Domain name qualified SCAN name -h Print usage • http://www.oracle.com/technetwork/database/ clustering/overview/scan-129069.pdf page 2
  • 35. 35 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. SCAN The SCAN bundle 3 SCAN OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network LISTENER_SCAN1 ora.SCAN1.VIP … … … … … ora.LISTENER.lsnr … ora.net1.network rac1 rac2 [GRID]> srvctl add scan -h Adds a SCAN VIP to the Oracle Clusterware. Usage: srvctl add scan -n <scan_name> -n <scan_name> Domain name qualified SCAN name -k <net_num> network number (default number is 1) -S <subnet>/<netmask>/[if1[|if2...]] NET address spec for network -h Print usage • Note: SCAN can only operate on one network / in one subnet.
  • 36. 36 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. SCAN The SCAN FAQ • From MOS Note 220970.1 – RAC: Frequently Asked Questions • How to use SCAN and node listeners with different ports? • With Oracle RAC 11g Release 2 using SCAN is the default. • As with other listeners, there is no direct communication between the node (listeners) and the SCAN listeners. • Listeners are only aware of the instances and services served, since the instances (PMON) register themselves and the services they host with the listeners. • The instances use the LOCAL and REMOTE Listener parameters to know with which listeners to register. • Listeners used for a client connection to Oracle RAC should be managed by Oracle Clusterware and should be listening on an Oracle managed VIP. 1 SCAN OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network LISTENER_SCAN1 ora.SCAN1.VIP … … … … … ora.LISTENER.lsnr … ora.net1.network rac1 rac2
  • 37. 37 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. SCAN The SCAN FAQ • Can you define another port? - YES • See MOS Note 220970.1 – RAC: Frequently Asked Questions • How to use SCAN and node listeners with different ports? • Use srvctl modify scan_listener –p <newPort> • Can you define another owner? – NO • Can you define another home? – NO • Can you have more than one node listener with SCAN? – YES • Can the SCAN and the node listener ports differ? - YES • See also MOS Note 972500.1 – How to Modify SCAN Setting or SCAN Listener Port after Installation 2 SCAN OS OS Oracle Grid Infrastructure … ora.LISTENER.lsnr … ora.net1.network LISTENER_SCAN1 ora.SCAN1.VIP … … … … … ora.LISTENER.lsnr … ora.net1.network rac1 rac2
  • 38. 38 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners and Instance Registration Listener changes may require instance changes • See MOS Note 220970.1 – RAC: Frequently Asked Questions • How to use SCAN and node listeners with different ports? • Whenever you change an instance parameter configuration a change to the LOCAL_LISTENER or REMOTE_LISTENER per database (instance) may be required. • Note that using server-side TNSANMES aliases to resolve the listener configuration is not recommended. • The use of the TNSNAMES connector string is the default: local_listener = ‘(DESCRIPTION= (ADDRESS_LIST= (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.61)(PORT=2011)) ))' SCAN OS OS Oracle Grid Infrastructure ora.LISTENER.lsnr … LISTENER_SCAN1 ora.SCAN1.VIP … … … … ora.LISTENER.lsnr … rac1 rac2 DB Instance DB Instance
  • 39. 39 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners and Instance Registration EXAMPLE • Taken from MOS Note 220970.1 – RAC: FAQ • How to use SCAN and node listeners with different ports? • Add additional node listeners to the system using different ports • srvctl add listener -l LISTENER2011 -p 2011 • srvctl start listener -l LISTENER2011 • srvctl status listener -l LISTENER2011 • lsnrctl status LISTENER2011 [GRID]> srvctl config listener Name: LISTENER2011 Network: 1, Owner: oracle Home: <CRS home> End points: TCP:2011 SCAN OS OS Oracle Grid Infrastructure ora.LISTENER.lsnr … LISTENER_SCAN1 ora.SCAN1.VIP … … … … ora.LISTENER.lsnr … rac1 rac2 DB Instance DB Instance 1
  • 40. 40 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners and Instance Registration EXAMPLE • Taken from MOS Note 220970.1 – RAC: FAQ • How to use SCAN and node listeners with different ports? • Add additional node listeners to the system using different ports • New node listener does not serve any database instance • You must alter the LOCAL_LISTENER parameters for each instance accordingly: alter system set local_listener =‘ (DESCRIPTION= (ADDRESS_LIST= (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.61)(PORT=2011)))) ' scope=BOTH SID='OCRL1'; alter system register; SCAN OS OS Oracle Grid Infrastructure ora.LISTENER.lsnr … LISTENER_SCAN1 ora.SCAN1.VIP … … … … ora.LISTENER.lsnr … rac1 rac2 DB Instance DB Instance 2
  • 41. 41 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners and Instance Registration EXAMPLE • Taken from MOS Note 220970.1 – RAC: FAQ • How to use SCAN and node listeners with different ports? • Add additional node listeners to the system using different ports • Note: Use spaces between the various segments of the command • Reason: the database agent in Oracle Clusterware currently determines whether the local_listener or remote_listener have been manually set by a string comparison operation. • If the string looks unaltered it will reset these parameters with the default values that it determines on instance startup. • Slight modifications are therefore required. ER 11772838 has been filed to allow for a more convenient mechanism. SCAN OS OS Oracle Grid Infrastructure ora.LISTENER.lsnr … LISTENER_SCAN1 ora.SCAN1.VIP … … … … ora.LISTENER.lsnr … rac1 rac2 DB Instance DB Instance 3
  • 42. 42 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners and Instance Registration Using listeners in different subnets / networks • SCAN can only operate in one network / subnet • If listeners in different subnets are used, • use LISTENER_NETWORKS: http://download.oracle.com/docs/cd/E11882_01 /server.112/e17110/initparams115.htm#REFRN10313 • Note: Listeners specified by the LISTENER_NETWORKS parameter should not be used in the LOCAL_LISTENER and REMOTE_LISTENER parameters. Otherwise, cross registration will happen and connections will be redirected cross networks. • For more information, see also MOS Note 1063571.1 – How to Configure A Second Listener on a Separate Network in 11.2 Grid Infrastructure OS OS Oracle Grid Infrastructure LISTENER_SCAN1 ora.SCAN1.VIP … … … … rac1 rac2 DB Instance DB Instance … ora.LISTENER.lsnr Ora.LISTENERK2.lsnr ora.net1.network ora.net2.network … … ora.LISTENER.lsnr ora.LISTENERK2.lsnr ora.net1.network ora.net2.network …
  • 43. 43 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners, SCAN and Services
  • 44. 44 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Listeners and Service Registration A summary • Services depend on a database • Per default, the database instance will register all services it hosts with all listeners it registers with. • Services operate within the following boundaries: • Network (indirectly via dependency) • Server Pools (in policy-managed database environments) • Currently there is no support for service failover • Between Server Pools • Between networks SCAN OS OS Oracle Grid Infrastructure ora.LISTENER.lsnr … ora.Dbname.db ora.Dbname.Srvcname.svc … ora.Dbname.db ora.Dbname.Srvcname.svc … ora.LISTENER.lsnr … rac1 rac2 DB Instance DB Instance
  • 45. 45 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Services and Dependencies Dependency on DB and Server Pool [GRID]> srvctl config service -d ORCL -s MyService Service name: MyService Service is enabled Server pool: ORCL Cardinality: 1 Disconnect: false Service role: PRIMARY Management policy: AUTOMATIC ... Preferred instances: ORCL_1 Available instances: SCAN OS OS Oracle Grid Infrastructure ora.LISTENER.lsnr … ora.ORCL.db ora.ORCL.MyService.svc … ora.ORCL.db ora.Dbname.Srvcname.svc … ora.LISTENER.lsnr … rac1 rac2 DB Instance DB Instance 1
  • 46. 46 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Services and Dependencies Start / Stop dependencies [GRID]> crsctl stat res ora.orcl.myservice.svc -p ... DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=service) PROPERTY(SERVICE_NAME=%GEN_SERVICE_NAME%) PROPERTY(DB_UNIQUE_NAME=CONCAT(PARSE(%NAME%, ., 2), STAT(ora.orcl.db, USR_ORA_DOMAIN), .)) ELEMENT(INSTANCE_NAME=STAT(ora.orcl.db, GEN_USR_ORA_INST_NAME)) ... START_DEPENDENCIES=hard(ora.orcl.db,type:ora.cluster _vip_net1.type) weak(type:ora.listener.type) pullup(type:ora.cluster_vip_net1.type) pullup:always(ora.orcl.db) dispersion(type:ora.service.type) STOP_DEPENDENCIES=hard(intermediate:ora.orcl.db,inte rmediate:type:ora.cluster_vip_net1.type) ... SCAN OS OS Oracle Grid Infrastructure ora.LISTENER.lsnr … ora.Dbname.db ora.Dbname.Srvcname.svc … ora.LISTENER.lsnr … rac1 rac2 DB Instance DB Instance 2 ora.ORCL.db ora.ORCL.MyService.svc …
  • 47. 47 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  • 48. 48 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.