对于Exadata和Exalogic做级联后，Oracle 推荐使用TCP，而不是SDP。TCP协议所能提供的稳定性对系统来说要超过SDP所提供的性能提升。而基于对不同工作负载的一些场景测试, 发现使用TCP协议也可以得到非常好的性能。而对于网络链接的故障切换，由于TCP协议对硬件层面的依赖更少，所以表现也比SDP更好。
对于Exadata和Exadata的级联，配置IB监听可以使用SDP协议。如果要启用和使用SDP协议，请参考Setup Listener on Infiniband Network using both SDP and TCP Protocol (Doc ID 1580584.1)
本文记录了如果在Exadata上添加IB网络为第二个监听网络，也适合于其他RAC环境。
查看IB私有网络IP信息[grid@dm06dbadm01 ~]$ oifcfg iflist -p -n | grep PRIVATE
ib0 192.168.10.0 PRIVATE 255.255.252.0
ib1 192.168.10.0 PRIVATE 255.255.252.0
bondeth0 172.16.10.0 PRIVATE 255.255.255.0
[grid@dm06dbadm01 ~]$
在DNS或/etc/hosts中配置新的IB VIP地址
[root@dm06dbadm01 ~]#
192.168.10.33 dm06db01-ibvip dm06db01-ibvip.oracle.com
192.168.10.34 dm06db02-ibvip dm06db02-ibvip.oracle.com
增加新的网络资源
[root@dm06dbadm01 ~]# /u01/app/12.2.0.1/grid/bin/srvctl add network -k 3 -S 192.168.10.0/255.255.252.0/ib0\|ib1
[root@dm06dbadm01 ~]# /u01/app/12.2.0.1/grid/bin/srvctl config network -k 3
Network 3 exists
Subnet IPv4: 192.168.10.0/255.255.252.0/ib0:ib1, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
[root@dm06dbadm01 ~]#
增加IB VIP资源
[root@dm06dbadm01 ~]# /u01/app/12.2.0.1/grid/bin/srvctl add vip -n dm06dbadm01 -A dm06db01-ibvip/255.255.252.0/ib0\|ib1 -k 3
[root@dm06dbadm01 ~]# /u01/app/12.2.0.1/grid/bin/srvctl add vip -n dm06dbadm02 -A dm06db02-ibvip/255.255.252.0/ib0\|ib1 -k 3
启动IB VIP
[root@dm06dbadm01 ~]# /u01/app/12.2.0.1/grid/bin/srvctl start vip -i dm06db01-ibvip
[root@dm06dbadm01 ~]# /u01/app/12.2.0.1/grid/bin/srvctl start vip -i dm06db02-ibvip
测试IB VIP
[root@dm06dbadm01 ~]# ping 192.168.10.33
PING 192.168.10.33 (192.168.10.33) 56(84) bytes of data.
64 bytes from 192.168.10.33: icmp_seq=1 ttl=64 time=0.010 ms
^C
--- 192.168.10.33 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 903ms
rtt min/avg/max/mdev = 0.010/0.010/0.010/0.000 ms
[root@dm06dbadm01 ~]# ping 192.168.10.34
PING 192.168.10.34 (192.168.10.34) 56(84) bytes of data.
64 bytes from 192.168.10.34: icmp_seq=1 ttl=64 time=0.073 ms
64 bytes from 192.168.10.34: icmp_seq=2 ttl=64 time=0.079 ms
^C
--- 192.168.10.34 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1705ms
rtt min/avg/max/mdev = 0.073/0.076/0.079/0.003 ms
[root@dm06dbadm01 ~]#
在grid用户下添加新的IB监听
[root@dm06dbadm01 ~]# su - grid
[grid@dm06dbadm01 ~]$ /u01/app/12.2.0.1/grid/bin/srvctl add listener -l LISTENER_IB -k 3 -p TCP:1522
[grid@dm06dbadm01 ~]$
启用并查看IB监听
[grid@dm06dbadm01 ~]$ /u01/app/12.2.0.1/grid/bin/srvctl status listener -l LISTENER_IB
Listener LISTENER_IB is enabled
Listener LISTENER_IB is running on node(s): dm06dbadm02,dm06dbadm01
[grid@dm06dbadm01 ~]$
修改local_listener，把IB监听加进去
SQL> alter system set LOCAL_LISTENER='(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=172.16.10.37)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.10.33)(PORT=1522)))' sid='pcdb1' scope=both;
2 3
System altered.
SQL>
SQL> alter system set LOCAL_LISTENER='(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=172.16.10.39)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.10.34)(PORT=1522)))' sid='pcdb2' scope=both;
2 3
System altered.
SQL>
重启监听
[grid@dm06dbadm01 ~] srvctl stop listener -l LISTENER_IB
[grid@dm06dbadm01 ~] srvctl start listener -l LISTENER_IB
查看IB监听下的服务
[grid@dm06dbadm01 ~]$ lsnrctl status LISTENER_IB
Reference
Setup Listener on Infiniband Network using both SDP and TCP Protocol (Doc ID 1580584.1)
