Network multicast support on Azure
Today I would like to talk about multicast support on Azure, and how to make it work. While it's not the most required feature on a Virtual environment, nevertheless, some applications require multicast support for networks. The perfect example is Oracle RAC, where multicast is required starting from version 11.2.0.2. In Oracle RAC, multicast is used for highly available IP (HAIP) on interconnect. If you're thinking about building a training environment with Oracle RAC on Azure you will need the multicast support. How can we check if it works, or if it's working now? First, you can check if it's supported by your kernel using the netstat utility.
[root@oradb5 ~]# netstat -g | grep mcast lo 1 all-systems.mcast.net eth0 1 all-systems.mcast.net eth1 1 all-systems.mcast.net You can see that all my interfaces are ready for the multicast support. That's fine, but how can we check if it works on our network? We can use either iperf utility or a perl script created by Oracle. You can download the script from Oracle support if you have account, from the Oracle note "How to Validate Network and Name Resolution Setup for the Clusterware and RAC (Doc ID 1054902.1)". Here's what I got: I have two Azure VM A3 size with Oracle Linux 6, with two network interfaces each. The VM hostnames were oradb5 and oradb6. You can check out my blog on how to make an Azure VM with two network interface
here. The second interface eth1 is one where we are going to enable multicast. I ran the mcasttest.pl script and saw that:
[oracle@oradb5 mcasttest]$ ./mcasttest.pl -n oradb5,oradb6 -i eth1 ########### Setup for node oradb5 ########## Checking node access 'oradb5' Checking node login 'oradb5' Checking/Creating Directory /tmp/mcasttest for binary on node 'oradb5' Distributing mcast2 binary to node 'oradb5' ########### Setup for node oradb6 ########## Checking node access 'oradb6' Checking node login 'oradb6' Checking/Creating Directory /tmp/mcasttest for binary on node 'oradb6' Distributing mcast2 binary to node 'oradb6' ########### testing Multicast on all nodes ########## Test for Multicast address 230.0.1.0 Nov 24 15:05:23 | Multicast Failed for eth1 using address 230.0.1.0:42000 Test for Multicast address 224.0.0.251 Nov 24 15:05:53 | Multicast Failed for eth1 using address 224.0.0.251:42001 [oracle@oradb5 mcasttest]$ The output clearly tells us that we don't have multicast support for either for 230.0.1.0 or 224.0.0.251 multicast addresses. What does the
Virtual Network FAQ for Azure tell us about it? Here is the answer:
Do VNets support multicast or broadcast? No. We do not support multicast or broadcast. What protocols can I use within VNets? You can use standard IP-based protocols within VNets. However, multicast, broadcast, IP-in-IP encapsulated packets and Generic Routing Encapsulation (GRE) packets are blocked within VNets. Standard protocols that work include: * TCP * UDP * ICMPSo, we need a workaround. Luckily we have one. Some time ago, while discussing RAC on Amazon AWS, I was pointed to an article written by my former colleague Jeremiah Wilton, where he had described how he could work around the same problem on Amazon. You can read the article here. I decided to give a try and see if it works for Azure. We are going to use a Peer-to-Peer VPN n2n provided by ntop. They have mentioned that the development for the product has been put on hold, but the tool is still widely used and provides an acceptable solution for our problem. I used a Stuart Buckell's article on how to set it up and it worked for me. We could just use an already precompiled package, but compiling the utility from the sources provides us with an opportunity to disable encryption and compression, or change any other options. Here is what I've done: Installed kernel headers to be able to compile the n2n :
[root@oradb5 n2n_v2]# yum install kernel-headers ol6_UEK_latest | 1.2 kB 00:00 ol6_u4_base | 1.4 kB 00:00 ol6_u4_base/primary | 2.7 MB 00:00 ol6_u4_base 8396/8396 Setting up Install Process Resolving Dependencies .... Installed subversion utility:
[root@oradb5 /]# yum install subversion.x86_64 Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package subversion.x86_64 0:1.6.11-15.el6_7 will be installed ............. Downloaded the sources using svn:
[root@oradb5 /]# svn co https://svn.ntop.org/svn/ntop/trunk/n2n Error validating server certificate for 'https://svn.ntop.org:443': - The certificate hostname does not match. Certificate information: - Hostname: shop.ntop.org - Valid: from Sun, 15 Nov 2015 00:00:00 GMT until Wed, 14 Nov 2018 23:59:59 GMT - Issuer: COMODO CA Limited, Salford, Greater Manchester, GB - Fingerprint: fb:a6:ff:a7:58:f3:9d:54:24:45:e5:a0:c4:04:18:d5:58:91:e0:34 (R)eject, accept (t)emporarily or accept (p)ermanently? p A n2n/n2n_v1 A n2n/n2n_v1/lzodefs.h A n2n/n2n_v1/README ............... Disabled encryption and compression using
this article Changed directory to n2n/n2n_v2 and compiled it.
[root@oradb5 n2n_v2]# make gcc -g3 -Wall -Wshadow -Wpointer-arith -Wmissing-declarations -Wnested-externs -c n2n.c gcc -g3 -Wall -Wshadow -Wpointer-arith -Wmissing-declarations -Wnested-externs -c n2n_keyfile.c gcc -g3 -Wall -Wshadow -Wpointer-arith -Wmissing-declarations -Wnested-externs -c wire.c gcc -g3 -Wall -Wshadow -Wpointer-arith -Wmissing-declarations -Wnested-externs -c minilzo.c gcc -g3 -Wall -Wshadow -Wpointer-arith -Wmissing-declarations -Wnested-externs -c twofish.c .............................. Copied files to the both my servers (oradb5 and oradb6) to /usr/sbin directory:
[root@oradb5 n2n_v2]# cp supernode /usr/sbin/ [root@oradb5 n2n_v2]# cp edge /usr/sbin/ Start a supernode daemon on the 1-st node. We only need it running on one machine, and it can even be a totally different machine. I am using port 1200 for it:
[root@oradb5 ~]# supernode -l 1200 [root@oradb5 ~]# Started the edge on both servers. On oradb5 I am creating a sub-interface with IP 192.168.1.1 and providing some parameters: -E - Accept multicast MAC addresses (default=drop). -r - Enable packet forwarding through n2n community. -c - n2n community name the edge belongs to. -l - our supernode address:port.
[root@oradb5 ~]# edge -l 10.0.2.11:1200 -c RAC -a 192.168.1.1 -E -r [root@oradb6 ~]# edge -l 10.0.2.11:1200 -c RAC -a 192.168.1.2 -E -r So we are getting an interface edge0 on both nodes and can use it for connection required multicast:
[root@oradb5 ~]# ifconfig edge0 edge0 Link encap:Ethernet HWaddr 52:CD:8E:20:3D:E5 inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::50cd:8eff:fe20:3de5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1400 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:0 (0.0 b) TX bytes:592 (592.0 b) [root@oradb5 ~]# On the second box:
[root@oradb6 ~]# ifconfig edge0 edge0 Link encap:Ethernet HWaddr 7E:B1:F1:41:7B:B7 inet addr:192.168.1.2 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::7cb1:f1ff:fe41:7bb7/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1400 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:5 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:0 (0.0 b) TX bytes:362 (362.0 b) [root@oradb6 ~]# Now we can run our multicast test again for edge0 interface and see how it works.
[oracle@oradb5 ~]$ cd mcasttest/ [oracle@oradb5 mcasttest]$ ./mcasttest.pl -n oradb5,oradb6 -i edge0 ########### Setup for node oradb5 ########## Checking node access 'oradb5' Checking node login 'oradb5' Checking/Creating Directory /tmp/mcasttest for binary on node 'oradb5' Distributing mcast2 binary to node 'oradb5' ########### Setup for node oradb6 ########## Checking node access 'oradb6' Checking node login 'oradb6' Checking/Creating Directory /tmp/mcasttest for binary on node 'oradb6' Distributing mcast2 binary to node 'oradb6' ########### testing Multicast on all nodes ########## Test for Multicast address 230.0.1.0 Nov 24 16:22:12 | Multicast Succeeded for edge0 using address 230.0.1.0:42000 Test for Multicast address 224.0.0.251 Nov 24 16:22:13 | Multicast Succeeded for edge0 using address 224.0.0.251:42001 [oracle@oradb5 mcasttest]$ As you can see, the test has completed successfully. So, the edge0 interface can be used now for any connections requiring multicast support. In my next article I will show you how to create an Oracle RAC on Azure using the created multicast interface and a shared storage.
