Use SD-WAN rules to steer multicast traffic

SD-WAN rules can steer IPv4 and IPv6 multicast traffic. When an SD-WAN member is out of SLA, multicast traffic can fail over to another SD-WAN member, and switch back when SLA recovers.

The pim-use-sdwan option enables or disables the use of SD-WAN for PIM (Protocol Independent Multicast) when checking RP (Rendezvous Point) neighbors and sending packets.

config router multicast
    config pim-sm-global
        set pim-use-sdwan {enable | disable}
    end
end

or

config router multicast6
    config pim-sm-global
        set pim-use-sdwan {enable | disable}
    end
end

When SD-WAN steers multicast traffic, ADVPN is not supported. Use the set shortcut option to disable shortcuts for the service: config system sdwan config service edit <id> set shortcut {enable | disable} next end end

Example 1

In this hub and spoke example for IPv4 multicast traffic, the PIM source is behind the hub FortiGate, and the RP is set to internal port (port2) of the hub firewall. Each spoke connects to the two WAN interfaces on the hub by using an overlay tunnel. The overlay tunnels are members of SD-WAN.

Receivers behind the spoke FortiGates request a stream from the source to receive traffic on tunnel1 by default. When the overlay tunnel goes out of SLA, the multicast traffic fails over to tunnel2 and continues to flow.

Following is an overview of how to configure the topology:

1. Configure the hub FortiGate in front of the PIM source. The RP is configured on internal port (port2) of the hub FortiGate.

2. Configure the spoke FortiGates.

3. Verify traffic failover.

To configure the hub:

1. On the hub, enable multicast routing, configure the multicast RP, and enable PIM sparse mode on each interface:

   config router multicast
       set multicast-routing enable
       config pim-sm-global
           config rp-address
               edit 1
                   set ip-address 172.16.205.1
               next
           end
       end
       config interface
           edit "tport1"
               set pim-mode sparse-mode
           next
           edit "tagg1"
               set pim-mode sparse-mode
           next
           edit "port2"
               set pim-mode sparse-mode
           next
       end
   end

To configure each spoke:

1. Enable SD-WAN with the following settings:

   • Configure the overlay tunnels as member of the SD-WAN zone.

   • Configure a performance SLA health-check using ping.

   • Configure a service rule for the PIM protocol with the following settings:

     • Use the lowest cost (SLA) strategy.

     • Monitor with the ping health-check.

   • Disable ADVPN shortcut.

   config system sdwan
       set status enable
       config zone
           edit "virtual-wan-link"
           next
       end
       config members
           edit 1
               set interface "tunnel1"
           next
           edit 2
               set interface "tunnel2"
           next
       end
       config health-check
           edit "ping"
               set server "172.16.205.1"
               set update-static-route disable
               set members 0
               config sla
                   edit 1
                   next
               end
           next
       end
       config service
           edit 1
               set mode sla
               set protocol 103
               set dst "all"
               config sla
                   edit "ping"
                       set id 1
                   next
               end
               set priority-members 1 2
               set use-shortcut-sla disable
               set shortcut disable
           next
           edit 2
               set mode sla
               set dst "all"
               config sla
                   edit "ping"
                       set id 1
                   next
               end
               set priority-members 1 2
           next
       end
   end

2. Enable multicast routing and configure the multicast RP. Enable PIM sparse-mode on each interface:

   config router multicast
       set multicast-routing enable
       config pim-sm-global
           set spt-threshold disable
           set pim-use-sdwan enable
           config rp-address
               edit 1
                   set ip-address 172.16.205.1
               next
           end
       end
       config interface
           edit "tunnel1"
               set pim-mode sparse-mode
           next
           edit "tunnel2"
               set pim-mode sparse-mode
           next
           edit "port4"
               set pim-mode sparse-mode
           next
       end
   end

To verify traffic failover:

With this configuration, multicast traffic starts on tunnel1. When tunnel1 becomes out of SLA, traffic switches to tunnel2. When tunnel1 is in SLA again, the traffic switches back to tunnel1.

The following health-check capture on the spokes shows tunnel1 in SLA with packet-loss (1.000%):

# diagnose sys sdwan health-check
Health Check(ping):
Seq(1 tunnel1): state(alive), packet-loss(0.000%) latency(0.056), jitter(0.002), mos(4.404), bandwidth-up(999999), bandwidth-dw(1000000), bandwidth-bi(1999999) sla_map=0x1
Seq(2 tunnel2): state(alive), packet-loss(0.000%) latency(0.100), jitter(0.002), mos(4.404), bandwidth-up(0), bandwidth-dw(0), bandwidth-bi(0) sla_map=0x1

# diagnose sys sdwan health-check
Health Check(ping):
Seq(1 tunnel1): state(alive), packet-loss(1.000%) latency(0.056), jitter(0.002), mos(4.404), bandwidth-up(999999), bandwidth-dw(1000000), bandwidth-bi(1999999) sla_map=0x1
Seq(2 tunnel2): state(alive), packet-loss(0.000%) latency(0.100), jitter(0.002), mos(4.404), bandwidth-up(0), bandwidth-dw(0), bandwidth-bi(0) sla_map=0x1

The following example shows tunnel1 out of SLA with packet-loss (3.000%):

# diagnose sys sdwan health-check
Health Check(ping):
Seq(1 tunnel1): state(alive), packet-loss(3.000%) latency(0.057), jitter(0.003), mos(4.403), bandwidth-up(999999), bandwidth-dw(1000000), bandwidth-bi(1999999) sla_map=0x0
Seq(2 tunnel2): state(alive), packet-loss(0.000%) latency(0.101), jitter(0.002), mos(4.404), bandwidth-up(0), bandwidth-dw(0), bandwidth-bi(0) sla_map=0x1

The following example shows tunnel1 back in SLA again:

# diagnose sys sdwan health-check
Health Check(ping):
Seq(1 tunnel1): state(alive), packet-loss(1.000%) latency(0.061), jitter(0.004), mos(4.404), bandwidth-up(999999), bandwidth-dw(1000000), bandwidth-bi(1999999) sla_map=0x0
Seq(2 tunnel2): state(alive), packet-loss(0.000%) latency(0.102), jitter(0.002), mos(4.404), bandwidth-up(0), bandwidth-dw(0), bandwidth-bi(0) sla_map=0x1

# diagnose sys sdwan health-check
Health Check(ping):
Seq(1 tunnel1): state(alive), packet-loss(0.000%) latency(0.061), jitter(0.004), mos(4.404), bandwidth-up(999999), bandwidth-dw(1000000), bandwidth-bi(1999999) sla_map=0x0
Seq(2 tunnel2): state(alive), packet-loss(0.000%) latency(0.102), jitter(0.002), mos(4.404), bandwidth-up(0), bandwidth-dw(0), bandwidth-bi(0) sla_map=0x1

The following example how traffic switches to tunnel2 while tunnel1 health-check is out of SLA. Source (172.16.205.11) sends traffic to the multicast group. Later the traffic switches back to tunnel1 once SLA returns to normal:

195.060797 tunnel1 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
195.060805 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
196.060744 tunnel1 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
196.060752 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
197.060728 tunnel1 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
197.060740 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
198.060720 tunnel2 in 172.16.205.11 -> 225.1.1.1: icmp: echo request        
198.060736 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
199.060647 tunnel2 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
199.060655 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
200.060598 tunnel2 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
200.060604 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
... ...
... ...
264.060974 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
265.060950 tunnel2 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
265.060958 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
266.060867 tunnel2 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
266.060877 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
267.060828 tunnel2 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
267.060835 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
268.060836 tunnel1 in 172.16.205.11 -> 225.1.1.1: icmp: echo request          
268.060854 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
269.060757 tunnel1 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
269.060767 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request
270.060645 tunnel1 in 172.16.205.11 -> 225.1.1.1: icmp: echo request
270.060653 port4 out 172.16.205.11 -> 225.1.1.1: icmp: echo request

Example 2

In this hub and spoke example for IPv4 multicast traffic, the PIM source is behind spoke 1, and the RP is configured on the hub FortiGate. BGP is used for routing. The hub uses embedded SLA in ICMP probes to determine the health of each tunnel, allowing it to prioritize healthy IKE routes.

The receiver is on another spoke. Upon requesting a stream, source passes the traffic to the RP on the hub FortiGate, and routes the traffic to the receiver over tunnel1. If a tunnel falls out of SLA, the multicast traffic fails over to the other tunnel.

In this configuration, SD-WAN steers multicast traffic by using embedded SLA information in ICMP probes. See also Embedded SD-WAN SLA information in ICMP probes. With this feature, the hub FortiGate can use the SLA information of the spoke's health-check to control BGP and IKE routes over tunnels.

Following is an overview of how to configure the topology:

1. Configure the hub FortiGate. The RP is configured on the hub FortiGate.

2. Configure the spoke FortiGate in front of the traffic receiver.

3. Configure the spoke FortiGate in front of the PIM source.

To configure the hub:

1. Configure loopbacks hub-lo1 172.31.0.1 for BGP and hub-lo100 172.31.100.100 for health-check:

   config system interface
       edit "hub-lo1"
           set vdom "hub"
           set ip 172.31.0.1 255.255.255.255
           set allowaccess ping
           set type loopback
           set snmp-index 82
       next
       edit "hub-lo100"
           set vdom "hub"
           set ip 172.31.100.100 255.255.255.255
           set allowaccess ping
           set type loopback
           set snmp-index 81
       next
   end

2. Enable multicast routing with the following settings:

   • Configure internal interface p25-v90 as RP.

   • Enable interfaces for PIM sparse-mode.

   config router multicast
       set multicast-routing enable
       config pim-sm-global
           config rp-address
               edit 1
                   set ip-address 192.90.1.11
               next
           end
       end
       config interface
           edit "p11"
               set pim-mode sparse-mode
           next
           edit "p101"
               set pim-mode sparse-mode
           next
           edit "p25-v90"
               set pim-mode sparse-mode
           next
       end
   end

3. Enable SD-WAN with the following settings:

   • Add interfaces p11 and p101 as members.
   • Configure embedded SLA health-checks to detect ICMP probes from each overlay tunnel. Prioritize based on the health of each tunnel.

   config system sdwan
       set status enable
       config zone
           edit "virtual-wan-link"
           next
       end
       config members
           edit 1
               set interface "p11"
           next
           edit 2
               set interface "p101"
           next
       end
       config health-check
           edit "1"
               set detect-mode remote
               set probe-timeout 60000
               set recoverytime 1
               set sla-id-redistribute 1
               set members 1
               config sla
                   edit 1
                       set link-cost-factor latency
                       set latency-threshold 100
                       set priority-in-sla 10
                       set priority-out-sla 20
                   next
               end
           next
           edit "2"
               set detect-mode remote
               set probe-timeout 60000
               set recoverytime 1
               set sla-id-redistribute 1
               set members 2
               config sla
                   edit 1
                       set link-cost-factor latency
                       set latency-threshold 100
                       set priority-in-sla 15
                       set priority-out-sla 25
                   next
               end
           next
       end
   end

4. Configure BGP to peer with neighbors. Neighbor group is configured for tunnel interface IP addresses:

   config router bgp
       set as 65505
       set router-id 172.31.0.1
       set ibgp-multipath enable
       set additional-path enable
       set recursive-inherit-priority enable
       config neighbor-group
           edit "gr1"
               set remote-as 65505
               set update-source "hub-lo1"
               set additional-path both
               set route-reflector-client enable
           next
       end
       config neighbor-range
           edit 1
               set prefix 10.10.0.0 255.255.0.0
               set neighbor-group "gr1"
           next
           edit 66
               set prefix 172.31.0.66 255.255.255.255
               set neighbor-group "gr1"
           next
       end
       config network
           ....
           edit 90
               set prefix 192.90.0.0 255.255.0.0
           next
       end
   end

To configure the spoke (in front of the receiver):

1. Enable multicast routing to use SD-WAN. Configure the RP address. Enable interfaces for PIM sparse-mode.

   config router multicast
       set multicast-routing enable
       config pim-sm-global
           set spt-threshold disable
           set pim-use-sdwan enable
           config rp-address
               edit 1
                   set ip-address 192.90.1.11
               next
           end
       end
       config interface
           edit "p195"
               set pim-mode sparse-mode
           next
           edit "p196"
               set pim-mode sparse-mode
           next
           edit "internal4"
               set pim-mode sparse-mode
               set static-group "225-1-1-122"
           next
       end
   end

2. Configure SD-WAN with the following settings:

   • Add overlay tunnel interfaces as members.

   • Configure a performance SLA health-check to send ping probes to the hub.

   • Configure a service rule for the PIM protocol. Use the lowest cost (SLA) strategy, and monitor with the ping health-check.

   • Disable ADVPN shortcuts.

   config system sdwan
       set status enable
       config zone
           edit "virtual-wan-link"
           next
       end
       config members
           edit 6
               set interface "p196"
           next
           edit 5
               set interface "p195"
           next
       end
       config health-check
           edit "ping"
               set server "172.31.100.100"
               set update-static-route disable
               set members 0
               config sla
                   edit 1
                       set link-cost-factor latency
                       set latency-threshold 100
                   next
               end
           next
       end
       config service
           edit 1
               set mode sla
               set protocol 103
               set dst "all"
               config sla
                   edit "ping"
                       set id 1
                   next
               end
               set priority-members 5 6
               set use-shortcut-sla disable
               set shortcut disable
           next
           edit 2
               set mode sla
               set dst "all"
               config sla
                   edit "ping"
                       set id 1
                   next
               end
               set priority-members 5 6
           next
       end
   end

3. Configure BGP and set neighbors to the overlay gateway IP address on the hub:

   config router bgp
       set as 65505
       set router-id 122.1.1.122
       set ibgp-multipath enable
       set additional-path enable
       config neighbor
           edit "10.10.100.254"
               set soft-reconfiguration enable
               set remote-as 65505
               set connect-timer 10
               set additional-path both
           next
           edit "10.10.101.254"
               set soft-reconfiguration enable
               set remote-as 65505
               set connect-timer 10
               set additional-path both
           next
       end
       config network
           edit 3
               set prefix 192.84.0.0 255.255.0.0
           next
       end
   end

4. Configure the default gateway to use the SD-WAN zone. Other routes are for the underlay to route traffic to the hub's WAN interfaces:

   config router static
       edit 10
           set distance 1
           set sdwan-zone "virtual-wan-link"
       next
       ....
       next
   end

To configure the spoke (in front of the source):

1. Enable multicast routing to use SD-WAN. Configure the RP address. Enable interfaces for PIM sparse-mode:

   config router multicast
       set multicast-routing enable
       config pim-sm-global
           set pim-use-sdwan enable
           config rp-address
               edit 1
                   set ip-address 192.90.1.11
               next
           end
       end
       config interface
           edit "p198"
               set pim-mode sparse-mode
           next
           edit "p200"
               set pim-mode sparse-mode
           next
           edit "npu0_vlink0"
               set pim-mode sparse-mode
           next
       end
   end

2. Configure loopback interface lo66 for BGP and sourcing SD-WAN traffic:

   config system interface
       edit "lo66"
           set vdom "root"
           set ip 172.31.0.66 255.255.255.255
           set allowaccess ping
           set type loopback
           set snmp-index 21
       next
   end

3. Configure SD-WAN:

   • Add overlay tunnel interfaces as members.

   • Configure a performance SLA health-check to send ping probes to the hub.

   • Configure a service rule for the PIM protocol. Use the lowest cost (SLA) strategy, and monitor with the ping health-check.

   • Disable the use of an ADVPN shortcut.

   In the following example, 11.11.11.11 is the underlay address for one of the WAN links on the hub, and 172.31.100.100 is the loopback address on the server.

   config system sdwan
       set status enable
       config zone
           edit "virtual-wan-link"
           next
           edit "overlay"
           next
       end
       config members
           edit 1
               set interface "p198"
               set zone "overlay"
               set source 172.31.0.66
           next
           edit 2
               set interface "p200"
               set zone "overlay"
               set source 172.31.0.66
           next
       end
       config health-check
           edit "ping"
               set server "11.11.11.11"            
               set members 0
               config sla
                   edit 1
                       set link-cost-factor latency
                       set latency-threshold 100
                   next
               end
           next
           edit "HUB"
               set server "172.31.100.100"        
               set embed-measured-health enable
               set members 0
               config sla
                   edit 1
                       set link-cost-factor latency
                       set latency-threshold 100
                   next
               end
           next
       end
        config service
           edit 1
               set mode sla
               set protocol 103
               set dst "all"
               config sla
                   edit "ping"
                       set id 1
                   next
               end
               set priority-members 1 2
               set use-shortcut-sla disable
               set shortcut disable
           next
           edit 2
               set mode sla
               set dst "all"
               config sla
                   edit "ping"
                       set id 1
                   next
               end
               set priority-members 1 2
           next
       end
   end

4. Configure BGP:

   config router bgp
       set as 65505
       set router-id 123.1.1.123
       set ibgp-multipath enable
       set additional-path enable
       config neighbor
           edit "172.31.0.1"
               set next-hop-self enable
               set soft-reconfiguration enable
               set remote-as 65505
               set update-source "lo66"
           next
       end
       config network
           edit 3
               set prefix 192.87.0.0 255.255.0.0
           next
       end
   end

5. Configure the default gateway to use the SD-WAN zone. Other routes are for the underlay to route to the hub's WAN interfaces:

   config router static
       edit 10
           set distance 1
           set sdwan-zone "virtual-wan-link" "overlay"
       next
       ...
       next
   end

Example 3

In the following example for using SD-WAN rules to steer IPv6 multicast traffic, three PIM-SM enabled tunnels are configured between Spoke-1 and the Hub. The multicast source is located at Hub, and the multicast receiver is attached to Spoke-1.

This example focuses on configuration related allowing SD-WAN rules to steer IPv6 multicast traffic. Following is an overview of the configuration steps:

1. On the hub FortiGate, configure multicast routing for the source and the multicast RP.

2. On the spoke FortiGate, configuring multicast routing and enable SD-WAN for steering.

3. Verify traffic failover for the following scenarios:

   • When the cost of an SD-WAN member changes

   • When a link is in SLA

   • When a link is out of SLA

To configure the Hub:

1. On Hub, configure multicast routing for the source and the multicast RP:

   In this example, port5 is used for the multicast source, and 20000:172:16:205::1 is the IPv6 address for the RP.

   config router multicast6
       set multicast-routing enable
       config interface
           edit "hub-phase1"           
           next
           edit "hub2-phase1"
           next
           edit "port5"
           next
           edit "hub3-phase1"
           next
       end
       config pim-sm-global
           config rp-address
               edit 1
                   set ip6-address 2000:172:16:205::1
               next
           end
       end
   end

2. Configure the firewall policy:

   config firewall multicast-policy6
       edit 1
           set srcintf "port5"
           set dstintf "any"
           set srcaddr "all"
           set dstaddr "all"
       next
   end

3. Verify that all PIM-SM neighbors are established:

   # get router info6 multicast pim sparse-mode neighbor
   Neighbor                    Interface          Uptime/Expires    Ver   DR
   Address                                                                Prio/Mode
   fe80::1                     hub-phase1         06:49:35/00:01:39 v2    1 /
   fe80::2                     hub2-phase1        06:49:34/00:01:42 v2    1 /
   fe80::1                     hub3-phase1        02:41:17/00:01:31 v2    1 /

To configure Spoke-1:

1. On Spoke-1, configure multicast routing and enable SD-WAN for steering:

   In this example, port5 is used for the multicast receiver, the use of SD-WAN for steering is enabled, and 20000:172:16:205::1 is the IPv6 address for the RP.

   config router multicast6
       set multicast-routing enable
       config interface
           edit "spoke11-p1"
           next
           edit "spoke12-p1"
           next
           edit "port2"
           next
           edit "spoke13-p1"
           next
       end
       config pim-sm-global
           set pim-use-sdwan enable
           config rp-address
               edit 1
                   set ip6-address 2000:172:16:205::1
               next
           end
       end
   end

2. Configure the firewall policy:

   config firewall multicast-policy6
       edit 1
           set srcintf "any"
           set dstintf "any"
           set srcaddr "all"
           set dstaddr "all"
       next
   end

3. Configure SD-WAN:

   In this example, the protocol is set to 103 to match PIM-SM join/register messages.

   config system sdwan
       set status enable
       config zone
           edit "virtual-wan-link"
           next
       end
       config members
           edit 1
               set interface "spoke11-p1"
           next
           edit 2
               set interface "spoke12-p1"
           next
           edit 3
               set interface "spoke13-p1"
           next
       end
       config health-check
           edit "1"
               set addr-mode ipv6
               set server "2000::9:0:0:1"
               set update-static-route disable
               set members 1
               config sla
                   edit 1
                   next
               end
           next
           edit "2"
               set addr-mode ipv6
               set server "2000::9:0:0:2"
               set update-static-route disable
               set members 2
               config sla
                   edit 1
                   next
               end
           next
           edit "3"
               set addr-mode ipv6
               set server "2000::9:0:0:3"
               set update-static-route disable
               set members 3
               config sla
                   edit 1
                   next
               end
           next
       end
       config service
           edit 1
               set name "1"
               set addr-mode ipv6
               set mode sla
               set protocol 103
               config sla
                   edit "1"
                       set id 1
                   next
                   edit "2"
                       set id 1
                   next
                   edit "3"
                       set id 1
                   next
               end
               set priority-members 1 2 3
               set sla-compare-method number
               set dst6 "all"
           next
       end
   end

4. Verify that all PIM-SM neighbors are established:

   # get router info6 multicast pim sparse-mode neighbor
   Neighbor                    Interface          Uptime/Expires    Ver   DR
   Address                                                                Prio/Mode
   fe80:10:10:15::253          spoke11-p1         06:49:50/00:01:16 v2    1 / DR
   fe80:10:10:16::253          spoke12-p1         06:49:50/00:01:26 v2    1 / DR
   fe80:10:10:17::253          spoke13-p1         02:41:32/00:01:43 v2    1 / DR

To verify traffic failover:

1. On Spoke-1, diagnose the SD-WAN service. The preferred route is spoke11-p1 to hub-phase1:

   # diagnose sys sdwan service6
   
   Service(1): Address Mode(IPV6) flags=0x4200 use-shortcut-sla use-shortcut
    Tie break: cfg
    Shortcut priority: 2
     Gen(1), TOS(0x0/0x0), Protocol(103): src(1->65535):dst(1->65535), Mode(sla), sla-compare-number
     Members(3):
       1: Seq_num(1 spoke11-p1 virtual-wan-link), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected           >>>>>>> spoke11-p1 which is connected to hub-phase1 is preferred
       2: Seq_num(2 spoke12-p1 virtual-wan-link), alive, sla(0x2), gid(0), cfg_order(1), local cost(0), selected
       3: Seq_num(3 spoke13-p1 virtual-wan-link), alive, sla(0x4), gid(0), cfg_order(2), local cost(0), selected
     Dst6 address(1): ::/0

2. When the receiver initiates IGMP to join group ff15::10, view mroute on Spoke-1 and Hub:

   • On Spoke-1:

     The RPF idx is connected to hub-phase1, indicating that PIM-SM join message follows SD-WAN service and is sent to spoke11-p1, and port2 is connected to the receiver.

     FGT_B (root)# get router info6 multicast pim sparse-mode mroute ff15::10
     IP Multicast Routing Table
     
     ......
     
     (*, ff15::10)
     RP: 2000:172:16:205::1
     RPF nbr: fe80:10:10:15::253
     RPF idx: spoke11-p1 
     Upstream State: JOINED
      Local:
          port2 
      Joined:
      Asserted:
     FCR:
     Source: 2000:172:16:205::100
      Outgoing:
          port2
      KAT timer running, 196 seconds remaining
      Packet count 168
     ...

   • On the Hub:

     We see that hub-phase1 is connected to spoke11-p1 on Spoke-1.

     FGT_A (root) (Interim)# get router info6 multicast pim sparse-mode mroute ff15::10
     IP Multicast Routing Table
     
     ......
     
     (*, ff15::10)
     RP: 2000:172:16:205::1
     RPF nbr: ::
     RPF idx: None
     Upstream State: JOINED
      Local:
      Joined:
          hub-phase1
      Asserted:
     FCR:
     
     ...

3. The server starts to send multicast traffic to group ff15::10, and Hub forwards the traffic to Spoke-1 through hub-phase1.

   FGT_A (root) (Interim)# diagnose sniffer packet any 'host  ff15::10' 4
   interfaces=[any]
   filters=[host  ff15::10]
   0.637174 port5 in 2000:172:16:205::100.38823 -> ff15::10.12345: udp 46 [flowlabel 0x8ea58]
   0.637228 hub-phase1 out 2000:172:16:205::100.38823 -> ff15::10.12345: udp 46 [flowlabel 0x8ea58]

4. When the cost of member spoke11-p1 and spoke12-p1 is increased, SD-WAN prefers spoke13-p1.

   The PIM-SM join message from Spoke-1 to RP is sent to member spoke13-p1, and multicast traffic fails over to hub3-phase1 on the Hub accordingly.

   • On Spoke-1:

     In this example, spoke13-p1, which is connected to hub-phase3, is preferred.

     FGT_B (root) (Interim)# diagnose sys sdwan service6
     
     Service(1): Address Mode(IPV6) flags=0x4200 use-shortcut-sla use-shortcut
      Tie break: cfg
      Shortcut priority: 2
       Gen(1), TOS(0x0/0x0), Protocol(103): src(1->65535):dst(1->65535), Mode(sla), sla-compare-number
       Members(3):
         1: Seq_num(3 spoke13-p1 virtual-wan-link), alive, sla(0x4), gid(0), cfg_order(2), local cost(0), selected
         2: Seq_num(1 spoke11-p1 virtual-wan-link), alive, sla(0x1), gid(0), cfg_order(0), local cost(20), selected
         3: Seq_num(2 spoke12-p1 virtual-wan-link), alive, sla(0x2), gid(0), cfg_order(1), local cost(20), selected
       Dst6 address(1): ::/0

   • On the Hub:

     Once the cost of spoke11-p1 is increased, multicast traffic fails over to hub2-phase1. Once the cost of spoke12-p1 is increased, multicast traffic fails over to hub3-phase1.

     FGT_A (root) (Interim)# diagnose sniffer packet any 'host  ff15::10' 4
     interfaces=[any]
     filters=[host  ff15::10]
     
     ....
     385.497887 port5 in 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     385.497927 hub-phase1 out 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     386.497967 port5 in 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     386.498258 hub2-phase1 out 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     387.498044 port5 in 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     ...
     400.499075 port5 in 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     400.499120 hub2-phase1 out 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d] 
     401.499180 port5 in 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     401.499515 hub3-phase1 out 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     402.499254 port5 in 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     402.499319 hub3-phase1 out 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     403.499330 port5 in 2000:172:16:205::100.41944 -> ff15::10.12345: udp 46 [flowlabel 0xa5e3d]
     ...

5. When spoke13-p1 becomes out of SLA, SD-WAN selects spoke11-p1 as the preferred member.

   This change redirects the PIM-SM join message from Spoke-1 to RP towards spoke11-p1, causing the multicast traffic to failover to hub-phase1 on the Hub.

6. Conversely, when spoke13-p1 is in SLA again, it is prioritized by SD-WAN.

   This adjustment redirects the PIM-SM join message from Spoke-1 to RP towards spoke13-p1, triggering a failover of the multicast traffic to hub3-phase1 on the Hub.