JUNIPER: Why is OSPF adjacency breaking when I enable FBF on an OSPF interface?

Question

I've built a test lab where I am testing Filter Based Forwarding (FBF), aka Policy Based Routing. Question will follow below, but first, the details:

Below is the topology diagram:

OBJECTIVE: Any traffic destined to Staging from Site 1 should route via Link 2 into the WAN and NOT via Link 1. Since Link 1 will be saturated with replication traffic between the two data centers.

• SW-1 & SW-2 are Juniper EX4200 switches
• RTR-1 & RTR-2 are Juniper J4350's
• PE-1 & PE-2 are Cisco 1841 routers running ISIS and MPLS VPN to simulate provider WAN backbone

SW-1, SW-2, RTR-1 and RTR-2 are all OSPF neighbors in Area 0. Both RTR-1 and RTR-2 are ASBR's and injecting BGP learned routes into OSPF. Each router is advertising routes into the WAN for its respective site (as well as pre-pended routes for the other site for redundancy).

Routing traffic from Site 1 to Staging at Site 2 is easily accomplished by simply redistributing the static route to Staging on SW-2 into OSPF with a higher metric. Since that route gets advertised by RTR-2 into the WAN, RTR-1 will learn that route and redistribute it into OSPF with a metric of 0. The OSPF route learned on SW-1 from SW-2 would have a higher metric, thus routing would be preferred over the WAN.

Return traffic from site 2 also needs to flow this way so that we avoid asymmetric routing. FBF is applied on the inbound interface (Link 4) entering SW-2. This filter will take all traffic sourced from Staging (10.100.190 /24) and make the next-hop RTR-2. This portion of the FBF is working, as I have tested in the lab.

Since RTR-2's preferred route back to Site 1 is via Link 1, we need to apply FBF once again at the inbound LAN interface of RTR-2 (facing SW-2).

Here's the problem... When FBF is applied on that router, OSPF adjacency with SW-2 breaks.

QUESTION: Why is OSPF adjacency breaking between RTR-2 and SW-2??

Configuration for RTR-2 and SW-2 are attached:

RTR-2 Configs

root@RTR-2> show configuration interfaces | display set    
set interfaces ge-0/0/0 unit 0 family inet filter input FBF-TEST
deactivate interfaces ge-0/0/0 unit 0 family inet filter
set interfaces ge-0/0/0 unit 0 family inet address 10.100.254.2/24
set interfaces ge-0/0/3 description "Uplink to WAN"
set interfaces ge-0/0/3 unit 0 family inet address 200.200.200.2/30
set interfaces lo0 unit 0 family inet address 10.100.199.4/32

root@RTR-2> show configuration routing-options | display set 
set routing-options interface-routes rib-group inet STAGING-RIB
set routing-options rib-groups STAGING-RIB import-rib inet.0
set routing-options rib-groups STAGING-RIB import-rib PATH-1.inet.0
set routing-options rib-groups STAGING-RIB import-rib PATH-2.inet.0
set routing-options router-id 200.200.200.2
set routing-options autonomous-system 1

root@RTR-2> show configuration routing-instances | display set  
set routing-instances PATH-1 instance-type forwarding
set routing-instances PATH-1 routing-options static route 10.100.30.0/24 next-hop 200.200.200.1
set routing-instances PATH-1 routing-options static route 10.100.30.0/24 qualified-next-hop 10.100.254.1 preference 100
set routing-instances PATH-2 instance-type forwarding
set routing-instances PATH-2 routing-options static route 10.100.30.0/24 next-hop 10.100.254.1
set routing-instances PATH-2 routing-options static route 10.100.30.0/24 qualified-next-hop 200.200.200.1 preference 100

root@RTR-2> show configuration firewall | display set             
set firewall family inet filter FBF-TEST term TERM-1 from source-address 10.100.190.0/24
set firewall family inet filter FBF-TEST term TERM-1 from source-address 10.100.191.0/24
set firewall family inet filter FBF-TEST term TERM-1 then routing-instance PATH-1
set firewall family inet filter FBF-TEST term DEFAULT then routing-instance PATH-2

root@RTR-2> show configuration protocols | display set 
set protocols bgp path-selection cisco-non-deterministic
set protocols bgp log-updown
set protocols bgp group TEST type external
set protocols bgp group TEST local-address 200.200.200.2
set protocols bgp group TEST import REJECT
set protocols bgp group TEST export ADVERTISED
set protocols bgp group TEST peer-as 65000
set protocols bgp group TEST neighbor 200.200.200.1 preference 20
set protocols ospf rib-group STAGING-RIB
set protocols ospf export BGP-to-OSPF
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 priority 150
set protocols ospf area 0.0.0.0 interface lo0.0 passive

SW-2 Configs

root@SW-2> show configuration interfaces | display set 
set interfaces ge-0/0/0 unit 0 family inet address 10.10.10.2/30
set interfaces ge-0/0/1 unit 0 family ethernet-switching port-mode access
set interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members VLAN-254
set interfaces ge-0/0/11 description "Uplink to STAGING"
set interfaces ge-0/0/11 unit 0 family inet filter input FBF-TEST
set interfaces ge-0/0/11 unit 0 family inet address 10.100.100.1/30
set interfaces lo0 unit 0 family inet address 10.100.199.2/32
set interfaces vlan unit 2 family inet address 10.100.2.1/24
set interfaces vlan unit 251 family inet address 10.100.251.1/24
set interfaces vlan unit 254 family inet address 10.100.254.1/24

root@SW-2> show configuration routing-options | display set 
set routing-options nonstop-routing
set routing-options interface-routes rib-group inet STAGING-RIB
set routing-options static route 172.22.128.0/21 next-hop 10.22.76.1
set routing-options static route 10.22.20.0/24 next-hop 10.22.76.1
set routing-options static route 10.100.190.0/24 next-hop 10.100.100.2
set routing-options static route 10.100.191.0/24 next-hop 10.100.100.2
set routing-options rib-groups STAGING-RIB import-rib inet.0
set routing-options rib-groups STAGING-RIB import-rib PATH-1.inet.0
set routing-options rib-groups STAGING-RIB import-rib PATH-2.inet.0
set routing-options router-id 10.100.254.1

root@SW-2> show configuration routing-instances | display set  
set routing-instances PATH-1 instance-type forwarding
set routing-instances PATH-1 routing-options static route 10.100.30.0/24 next-hop 10.100.254.2
set routing-instances PATH-1 routing-options static route 10.100.30.0/24 qualified-next-hop 10.10.10.1 preference 100
set routing-instances PATH-2 instance-type forwarding
set routing-instances PATH-2 routing-options static route 10.100.30.0/24 next-hop 10.10.10.1
set routing-instances PATH-2 routing-options static route 10.100.30.0/24 qualified-next-hop 10.100.254.2 preference 100

root@SW-2> show configuration firewall | display set             
set firewall family inet filter FBF-TEST term TERM-1 from source-address 10.100.190.0/24
set firewall family inet filter FBF-TEST term TERM-1 from source-address 10.100.191.0/24
set firewall family inet filter FBF-TEST term TERM-1 then routing-instance PATH-1
set firewall family inet filter FBF-TEST term DEFAULT then routing-instance PATH-2

root@SW-2> show configuration protocols | display set   
set protocols ospf export ADVERTISED
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface vlan.2 passive
set protocols ospf area 0.0.0.0 interface vlan.251 passive
set protocols ospf area 0.0.0.0 interface vlan.254 priority 250

Answer 1

So, after working with JTAC yesterday, "I", as in I really didn't need JTAC since I figured out the issue on my own.. realized that my firewall filter was a bit redundant and was lacking a "permit any" statement.

OSPF adjacency was breaking because the firewall filter was taking the "else" traffic (term DEFAULT) and sending it to routing-instance PATH-2, which didn't help either way since it was sending traffic right back to SW-2, something a "then accept" statement would have done easily

So, to repair the issue..

New SW-2 & RTR-2 corrected configlets:

delete routing-instances PATH-2
delete firewall family inet filter FBF-TEST term DEFAULT
set firewall family inet filter FBF-TEST term PERMIT-ANY then accept

New config snips for SW-2:

routing-options {
    nonstop-routing;
    interface-routes {
        rib-group inet STAGING-RIB;
    }
    static {
        route 10.100.190.0/24 next-hop 10.100.100.2;
        route 10.100.191.0/24 next-hop 10.100.100.2;
    }
    rib-groups {
        STAGING-RIB {
            import-rib [ inet.0 PATH-1.inet.0 ];
        }
    }
    router-id 10.100.254.1;
}
firewall {
    family inet {
        filter FBF-TEST {
            term TERM-1 {
                from {
                    source-address {
                        10.100.190.0/24;
                        10.100.191.0/24;
                    }
                }
                then {
                    routing-instance PATH-1;
                }
            }
            term PERMIT-ANY {
                then accept;
            }
        }
    }
}
routing-instances {
    PATH-1 {
        instance-type forwarding;
        routing-options {
            static {
                route 10.100.30.0/24 {
                    next-hop 10.100.254.2;
                    qualified-next-hop 10.10.10.1 {
                        preference 100;
                    }
                }
            }
        }
    }
}

New config snips for RTR-2:

routing-options {
    interface-routes {
        rib-group inet STAGING-RIB;
    }
    rib-groups {
        STAGING-RIB {
            import-rib [ inet.0 PATH-1.inet.0 ];
        }
    }
    router-id 200.200.200.2;
    autonomous-system 1;
}
firewall {
    family inet {
        filter FBF-TEST {
            term TERM-1 {
                from {
                    source-address {
                        10.100.190.0/24;
                        10.100.191.0/24;
                    }
                }
                then {
                    routing-instance PATH-1;
                }
            }
            term PERMIT-ANY {
                then accept;
            }
        }
    }
}
routing-instances {
    PATH-1 {
        instance-type forwarding;
        routing-options {
            static {
                route 10.100.30.0/24 {
                    next-hop 200.200.200.1;
                    qualified-next-hop 10.100.254.1 {
                        preference 100;
                    }
                }
            }
        }
    }
}