BGP selective path selection on 2 ISP networks

In this post I will show a solution made for international customer, this is the network topololgy:

The setup is as follow:

The customer main site connected through two different ISP’s: AS65000 and AS65001. AS65000 belongs to international ISP which connected worldwide while the ISP on AS65001 is local while connected also through AS65000.

The customer has two (or more) branch sites which some of them are located on US (R6 AS65006) and some of them located on Europe (R7 AS65007).

Both ISP’s are running MPLS with dedicated VRF for the customer (ISP1 – RED, ISP2 – BLUE) and between the ISP’s there is an NNI connection (R3-R12).

The customer requirements:

All traffic from/to main site to US sites will go through ISP1

All traffic from/to main site to European sites will go through ISP2 

On the customer internal network there is FHRP (HSRP in this case) where R9 is the active router. This can also be changed to some IGP along with redistribution (from BGP to IGP) for optimal routing and traffic flow but that’s not the issue here.

Our first problem is to make R4 prefer the path, to main site networks, through the NNI (R3-R12), while in normal condition R4 prefer the path through R2 due to shorter AS-path:

R4#show ip bgp vpnv4 vrf RED 192.168.11.0/24 BGP routing table entry for 4:100:192.168.11.0/24, version 30 Paths: (2 available, best #2, table RED) Multipath: eiBGP   Advertised to update-groups:      10   Refresh Epoch 2   65001 65089, imported path from 3:100:192.168.11.0/24 (global)     3.3.3.3 (metric 3) from 5.5.5.5 (5.5.5.5)       Origin incomplete, metric 0, localpref 100, valid, internal       Extended Community: RT:100:100 RT:888:888       Originator: 3.3.3.3, Cluster list: 5.5.5.5       mpls labels in/out nolabel/24       rx pathid: 0, tx pathid: 0   Refresh Epoch 2   65089, imported path from 2:100:192.168.11.0/24 (global)     2.2.2.2 (metric 3) from 5.5.5.5 (5.5.5.5)       Origin incomplete, metric 0, localpref 100, valid, internal, best       Extended Community: RT:100:100 RT:999:999       Originator: 2.2.2.2, Cluster list: 5.5.5.5       mpls labels in/out nolabel/30       rx pathid: 0, tx pathid: 0x0

So first we will have to make R3 prefer the path through R12 (because he face the same problem as R4 – shorter path through R2), this has made using route-map with weight attribute:

ip prefix-list PL_R10_NET seq 5 permit 192.168.11.0/24 ip prefix-list PL_R10_NET seq 10 permit 192.168.12.0/24 ip prefix-list PL_R10_NET seq 15 permit 192.168.13.0/24 no cdp log mismatch duplex ! route-map RM_BGP_R12_IN permit 10  match ip address prefix-list PL_R10_NET  set weight 1000 ! router bgp 65000  address-family ipv4 vrf RED   redistribute connected   neighbor 10.1.123.12 remote-as 65001   neighbor 10.1.123.12 activate   neighbor 10.1.123.12 route-map RM_BGP_R12_IN in  exit-address-family

Then I have configured route-map, on R3, which mark all main site networks with extended community 888:888 and export it on the customer VRF RED:

route-map RM_RED_EXPORT permit 10  match ip address prefix-list PL_R10_NET  set extcommunity rt 888:888 additive ! ip vrf RED  rd 3:100  export map RM_RED_EXPORT  route-target export 100:100  route-target import 100:100

Now on R4 I have configured a route-map with weight attribute which prefers all routes marked with 888:888 extended community and import it into the customer VRF RED:

ip extcommunity-list 100 permit 888:888 ! route-map RM_RED_IMPORT permit 10  match extcommunity 100  set weight 1500 ! route-map RM_RED_IMPORT permit 999 ! ip vrf RED  rd 4:100  import map RM_RED_IMPORT  route-target export 100:100  route-target import 100:100

The result:

R4#show ip bgp vpnv4 vrf RED 192.168.11.0/24 BGP routing table entry for 4:100:192.168.11.0/24, version 35 Paths: (2 available, best #1, table RED) Multipath: eiBGP   Advertised to update-groups:      11   Refresh Epoch 2   65001 65089, imported path from 3:100:192.168.11.0/24 (global)     3.3.3.3 (metric 3) from 5.5.5.5 (5.5.5.5)       Origin incomplete, metric 0, localpref 100, weight 1500, valid, internal, best       Extended Community: RT:100:100 RT:888:888       Originator: 3.3.3.3, Cluster list: 5.5.5.5       mpls labels in/out nolabel/24       rx pathid: 0, tx pathid: 0x0   Refresh Epoch 2   65089, imported path from 2:100:192.168.11.0/24 (global)     2.2.2.2 (metric 3) from 5.5.5.5 (5.5.5.5)       Origin incomplete, metric 0, localpref 100, valid, internal       Extended Community: RT:100:100 RT:999:999       Originator: 2.2.2.2, Cluster list: 5.5.5.5       mpls labels in/out nolabel/30       rx pathid: 0, tx pathid: 0

R4 now prefers the path through R3-R12 instead of R2.

Next problem is between R9 and R8, R9 which is the active router, will prefer R2 as the exit point to R7 networks due to eBGP vs. iBGP. In order to solve this problem I have configured route-map with weight attribute on R9:

ip prefix-list PL_R7_NET seq 5 permit 192.168.71.0/24 ip prefix-list PL_R7_NET seq 10 permit 192.168.72.0/24 ip prefix-list PL_R7_NET seq 15 permit 192.168.73.0/24 ! route-map RM_BGP_R8_IN permit 10  match ip address prefix-list PL_R7_NET  set weight 1500 ! route-map RM_BGP_R8_IN permit 999 ! router bgp 65089  bgp log-neighbor-changes  redistribute static route-map RM_BGP_STATIC  neighbor 10.1.10.8 remote-as 65089  neighbor 10.1.10.8 route-map RM_BGP_R8_IN in  neighbor 10.1.29.2 remote-as 65000

Now R9 will prefers R8 as the exit point to R7 networks.

The final result – trace from R10 Lo1 to R7 Lo1:

R10#traceroute 192.168.71.1 source lo1 Type escape sequence to abort. Tracing the route to 192.168.71.1 VRF info: (vrf in name/id, vrf out name/id)   1 10.1.10.9 140 msec 116 msec 92 msec   2 10.1.10.8 172 msec 140 msec 216 msec   3 10.1.118.11 368 msec 272 msec 200 msec   4 10.1.123.12 [MPLS: Label 41 Exp 0] 260 msec 224 msec 268 msec   5 10.1.123.3 316 msec 308 msec 332 msec   6 10.1.35.5 [MPLS: Labels 16/20 Exp 0] 596 msec 480 msec 668 msec   7 10.1.47.4 [MPLS: Label 20 Exp 0] 504 msec 356 msec 400 msec   8 10.1.47.7 544 msec 444 msec 532 msec

From R10 Lo1 to R6 Lo1:

R10#traceroute 192.168.61.1 source lo1 Type escape sequence to abort. Tracing the route to 192.168.61.1 VRF info: (vrf in name/id, vrf out name/id)   1 10.1.10.9 140 msec 140 msec 100 msec   2 10.1.29.2 204 msec 248 msec 144 msec   3 10.1.25.5 [MPLS: Labels 19/30 Exp 0] 316 msec 228 msec 364 msec   4 10.1.16.1 [MPLS: Label 30 Exp 0] 308 msec 264 msec 228 msec   5 10.1.16.6 356 msec 248 msec 396 msec

From R7 Lo1 to R10 Lo1:

R7#traceroute 192.168.11.1 source lo1 Type escape sequence to abort. Tracing the route to 192.168.11.1 VRF info: (vrf in name/id, vrf out name/id)   1 10.1.47.4 [AS 65000] 148 msec 140 msec 180 msec   2 10.1.45.5 [AS 65000] [MPLS: Labels 18/24 Exp 0] 316 msec 364 msec 264 msec   3 10.1.123.3 [AS 65000] [MPLS: Label 24 Exp 0] 252 msec 244 msec 196 msec   4 10.1.123.12 [AS 65000] 272 msec 208 msec 244 msec   5 10.1.118.11 [AS 65001] [MPLS: Label 18 Exp 0] 312 msec 196 msec 284 msec   6 10.1.118.8 [AS 65001] 448 msec 484 msec 356 msec   7 10.1.10.10 [AS 65000] 600 msec 532 msec 460 msec

And from R6 Lo1 to R10 Lo1:

R6#traceroute 192.168.11.1 source lo1 Type escape sequence to abort. Tracing the route to 192.168.11.1 VRF info: (vrf in name/id, vrf out name/id)   1 10.1.16.1 [AS 65000] 112 msec 88 msec 100 msec   2 10.1.15.5 [MPLS: Labels 17/30 Exp 0] 308 msec 184 msec 232 msec   3 10.1.29.2 [AS 65000] [MPLS: Label 30 Exp 0] 240 msec 228 msec 168 msec   4 10.1.29.9 [AS 65000] 288 msec 140 msec 360 msec   5 10.1.10.10 284 msec 304 msec 316 msec

Using this method there can be much more manipulations for specific routes and networks, outbound or inbound traffic flow or even load-sharing.