Here is a scenario:
A Customer, named ABC, has data center and few hundreds of
branches. In the DC there are two routers which connected with 4 links to the
SP network while in each branch there is one router with one or two links.
The gateway for the DC is the firewall which works with
static router toward the HSRP VIP between the two routers.
The customer requirements are as follow:
Secure connectivity between all the sites
Branch routers can communicate only with the DC but not
between each other
All links should be active for redundancy and link efficiency
The ability to control the network without the need of the
SP intervention
Following these demands the most appropriate technology to
use is DM-VPN.
In this post I will show how to configure dual DM-VPN with
single hub which allows traffic load-sharing, both upstream and downstream,
between the hubs and spokes while maintain redundancy in case of link/device
failure.
This is the GNS3 topology that I used:
Let’s start with the SP network configuration which is quite
simple -
All 3 routers use OSPF, MPLS and MP-BGP as PE routers, where
R6 is the route-reflector for AS65000, all three routers also advertise
default-information to their CE respectively.
R1, R2, R3, R4 and R5 are all CE routers connected to VRF
ABC at the SP network.
For building up the DM-VPN network we will need only the
loopback IP’s so the first task is to make sure that all customer routers are
advertising only their loopback IP’s, and receive only each other loopback IP,
through the SP network.
R3 BGP configuration (this configuration is similar for all
customer routers with the relevant IP’s):
#the hubs loopback IP address
ip prefix-list
PL_HUB_LOOPBACK seq 5 permit 1.1.1.1/32
ip prefix-list
PL_HUB_LOOPBACK seq 10 permit 2.2.2.2/32
!
#R3 loopback IP
ip prefix-list PL_LO0 seq 5
permit 3.3.3.3/32
!
route-map RM_BGP_INBOUND_R6
permit 10
match ip address prefix-list PL_HUB_LOOPBACK
!
route-map RM_BGP_INBOUND_R6
deny 999
!
route-map RM_BGP_INBOUND_R8
permit 10
match ip address prefix-list PL_HUB_LOOPBACK
!
route-map RM_BGP_INBOUND_R8
deny 999
!
route-map RM_BGP_OUTBOUND_R8
permit 10
match ip address prefix-list PL_LO0
!
route-map RM_BGP_OUTBOUND_R8
deny 999
!
route-map RM_BGP_OUTBOUND_R6
permit 10
match ip address prefix-list PL_LO0
!
route-map RM_BGP_OUTBOUND_R6
deny 999
!
router bgp 65003
redistribute connected
neighbor 10.1.36.6 remote-as 65000
neighbor 10.1.36.6 soft-reconfiguration
inbound
neighbor 10.1.36.6 route-map
RM_BGP_INBOUND_R6 in
neighbor 10.1.36.6 route-map
RM_BGP_OUTBOUND_R6 out
neighbor 10.1.38.8 remote-as 65000
neighbor 10.1.38.8 route-map
RM_BGP_INBOUND_R8 in
neighbor 10.1.38.8 route-map
RM_BGP_OUTBOUND_R8 out
|
Following this configuration R3 will advertise 3.3.3.3/32
and will receive only 1.1.1.1/32 and 2.2.2.2/32, from both PE routers.
Now that the underlay network is ready and all customer
routers learns each other loopback, we can start configure the overlay network
- DM-VPN.
Each hub router will have single tunnel interface while
spoke routers will have two tunnels, on for each hub.
First the encryption configuration (similar to all routers):
crypto isakmp policy 10
encr aes
hash sha
authentication pre-share
group 14
!
crypto isakmp key cisco
address 0.0.0.0
!
crypto ipsec transform-set
MYSET esp-aes esp-sha-hmac
mode transport
!
crypto ipsec profile DMVPN
set transform-set MYSET
|
As I mentioned before this topology will be constructed from
two DM-VPN networks with single hub.
Tunnel configuration on R1 – Hub router for DM-VPN network
1:
interface Tunnel1
ip address 172.1.0.1 255.255.255.0
no ip redirects
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp network-id 1
tunnel source Loopback1
tunnel mode gre multipoint
tunnel key 1
tunnel protection ipsec profile DMVPN
end
|
Tunnel configuration on R2 – Hub router for DM-VPN network
2:
interface Tunnel2
ip address 172.2.0.2 255.255.255.0
no ip redirects
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp network-id 2
tunnel source Loopback2
tunnel mode gre multipoint
tunnel key 2
tunnel protection ipsec profile DMVPN
end
|
Tunnel configuration for each spoke router (R4 output):
interface Tunnel1
ip address 172.1.0.4 255.255.255.0
no ip redirects
ip nhrp authentication cisco
ip nhrp network-id 1
ip nhrp nhs 172.1.0.1 nbma 1.1.1.1 multicast
if-state nhrp
tunnel source Loopback0
tunnel mode gre multipoint
tunnel key 1
tunnel protection ipsec profile DMVPN shared
!
interface Tunnel2
ip address 172.2.0.4 255.255.255.0
no ip redirects
ip nhrp authentication cisco
ip nhrp network-id 2
ip nhrp nhs 172.2.0.2 nbma 2.2.2.2 multicast
if-state nhrp
tunnel source Loopback0
tunnel mode gre multipoint
tunnel key 2
tunnel protection ipsec profile DMVPN shared
|
Note that this is a DM-VPN phase 1 configuration which set
the hub as the next-hop for the spokes without the ability for the spokes to
communicate with each other directly.
Also note that under the tunnel interface protection profile
for the spokes I used ‘shared’ option, because both tunnels use the same IPSec
profile.
Now configure the dynamic routing protocol between the hubs
and spokes, R1 and R2 will advertise summary-address for default route:
Hubs configuration (R1 output):
ip route 192.168.11.0
255.255.255.0 10.1.12.3
ip route 192.168.12.0
255.255.255.0 10.1.12.3
!
router eigrp 1
network 10.1.12.1 0.0.0.0
network 172.1.0.1 0.0.0.0
redistribute static
!
interface Tunnel1
ip address 172.1.0.1 255.255.255.0
no ip redirects
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp network-id 1
ip summary-address eigrp 1 0.0.0.0 0.0.0.0
tunnel source Loopback1
tunnel mode gre multipoint
tunnel key 1
end
|
In the EIGRP process we configure the internal networks, the
one that behind the router, and the tunnel interface to create an adjacency
between the neighbors. On the hub routers I also configured redistribute static
because the DC networks are not directly connected.
Spoke configuration (R4 output):
router eigrp 1
network 172.1.0.4 0.0.0.0
network 172.2.0.4 0.0.0.0
network 192.168.41.1 0.0.0.0
|
Now the DM-VPN is up and running, both all spokes connects
to both hubs:
R1#show dmvpn
Legend: Attrb --> S -
Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries
with same NBMA peer
NHS Status: E --> Expecting
Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for
a Tunnel
==========================================================================
Interface: Tunnel1, IPv4 NHRP
Details
Type:Hub, NHRP Peers:3,
# Ent
Peer NBMA Addr Peer Tunnel Add State
UpDn Tm Attrb
----- --------------- --------------- -----
-------- -----
1 3.3.3.3 172.1.0.3 UP
2d09h D
1 4.4.4.4 172.1.0.4 UP
2d09h D
1 5.5.5.5 172.1.0.5 UP
2d09h D
R2#show dmvpn
Legend: Attrb --> S -
Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries
with same NBMA peer
NHS Status: E --> Expecting
Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for
a Tunnel
==========================================================================
Interface: Tunnel2, IPv4 NHRP
Details
Type:Hub, NHRP Peers:3,
# Ent
Peer NBMA Addr Peer Tunnel Add State
UpDn Tm Attrb
----- --------------- --------------- -----
-------- -----
1 3.3.3.3 172.2.0.3 UP
2d09h D
1 4.4.4.4 172.2.0.4 UP
2d09h D
1 5.5.5.5 172.2.0.5 UP
2d09h D
|
Each spoke can reach the DC from both tunnels in equal way
(ECMP):
R4#show ip route 0.0.0.0
0.0.0.0
Routing entry for 0.0.0.0/0,
supernet
Known via "eigrp 1", distance 90,
metric 26882560, candidate default path, type internal
Redistributing via eigrp 1
Last update from 172.2.0.2 on Tunnel2,
1d05h ago
Routing Descriptor Blocks:
172.2.0.2, from 172.2.0.2, 1d05h ago, via
Tunnel2
Route metric is 26882560, traffic share
count is 1
Total delay is 50100 microseconds,
minimum bandwidth is 100 Kbit
Reliability 255/255, minimum MTU 1472
bytes
Loading 1/255, Hops 1
* 172.1.0.1, from 172.1.0.1, 1d05h ago, via
Tunnel1
Route metric is 26882560, traffic share
count is 1
Total delay is 50100 microseconds,
minimum bandwidth is 100 Kbit
Reliability 255/255, minimum MTU 1472
bytes
Loading 1/255, Hops 1
|
This will ensure that traffic from the spoke to the hub will
share between the two tunnels on a per-destination method (CEF default
load-share algorithm):
R4#show ip cef 0.0.0.0
0.0.0.0
0.0.0.0/0
nexthop 172.1.0.1 Tunnel1
nexthop 172.2.0.2 Tunnel2
|
This can be changed to per-packet but it’s less recommended
especially for voice traffic.
Now we need to make sure that the hub will see the spoke
networks from two paths…
In the DC the FW is using static route which points to R1
and R2 HSRP VIP (10.1.12.254) where R1 is the active router for the HSRP group.
Due to the fact that I cannot change the FW to work with
dynamic routing protocol, upstream traffic from the DC to spoke will have to go
to hub 1 (which is the active HSRP) and from there to either tunnel 1 or to hub
2-tunnel 2, and from there to the spoke.
Now let’s see how R1 sees network 192.168.41.0/24:
R1#show ip
route 192.168.41.0
Routing
entry for 192.168.41.0/24
Known via "eigrp 1", distance 90,
metric 27008000, type internal
Redistributing via eigrp 1
Last update from 172.1.0.4 on Tunnel1,
19:02:50 ago
Routing Descriptor Blocks:
* 172.1.0.4, from 172.1.0.4, 19:02:50 ago,
via Tunnel1
Route metric is 27008000, traffic share
count is 1
Total delay is 55000 microseconds,
minimum bandwidth is 100 Kbit
Reliability 255/255, minimum MTU 1472
bytes
Loading 1/255, Hops 1
R1#show ip
eigrp topology 192.168.41.0/24
EIGRP-IPv4
Topology Entry for AS(1)/ID(2.2.2.1) for 192.168.41.0/24
State is Passive, Query origin flag is 1, 1
Successor(s), FD is 27008000
Descriptor Blocks:
172.1.0.4 (Tunnel1), from 172.1.0.4, Send
flag is 0x0
Composite metric is (27008000/128256), route
is Internal
Vector metric:
Minimum bandwidth is 100 Kbit
Total delay is 55000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1472
Hop count is 1
Originating router is 192.168.41.1
10.1.12.2 (FastEthernet0/0), from
10.1.12.2, Send flag is 0x0
Composite metric is (27010560/27008000), route is
Internal
Vector metric:
Minimum bandwidth is 100 Kbit
Total delay is 55100 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1472
Hop count is 2
Originating router is 192.168.41.1
|
We can see from this table that the feasibility condition didn’t
met, R1 computed distance is 27008000 where R2 (10.1.12.2) reported distance is
27008000 and that’s the reason why R1 doesn’t have ECMP.
From the metric calculation we can see that we need to lower
R2 metric by 2560 (or more) to make this path valid.
We can also notice that R2 total delay is 55100 where R1
total delay is 55000, so we can subtract 100 microseconds from R2 path to
balance the metric, because we can’t change R1 Fa0/0 delay to 0 (default delay
is 100 microseconds) we need to change R2, tunnel 2 interface, delay and
subtract 100 microseconds*. The default delay on interface tunnel 2 is 50,000
microseconds:
R2#show interfaces tunnel 2
Tunnel2 is up, line protocol
is up
Hardware is Tunnel
Internet address is 172.2.0.2/24
MTU 17912 bytes, BW 100 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 1/255,
rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel source 2.2.2.2 (Loopback2)
<OUTPUT_OMMITED>
|
*We can lower the delay on R2 more than 100 but then we will
have to use variance on R1 to use UCMP.
Remember that the setting is configured in 10th
of microseconds, hence 4990 means 49900 microseconds:
R2(config)#interface tunnel 2
R2(config-if)#delay 4990
|
Now check R1 routing table and EIGRP topology table:
R1#show ip
route 192.168.41.0
Routing
entry for 192.168.41.0/24
Known via "eigrp 1", distance 90,
metric 27008000, type internal
Redistributing via eigrp 1
Last update from 10.1.12.2 on
FastEthernet0/0, 00:10:32 ago
Routing Descriptor Blocks:
* 172.1.0.4, from 172.1.0.4, 00:10:32 ago,
via Tunnel1
Route metric is 27008000, traffic share
count is 1
Total delay is 55000 microseconds,
minimum bandwidth is 100 Kbit
Reliability 255/255, minimum MTU 1472
bytes
Loading 1/255, Hops 1
10.1.12.2, from 10.1.12.2, 00:10:32 ago,
via FastEthernet0/0
Route metric is 27008000, traffic share
count is 1
Total delay is 55000 microseconds,
minimum bandwidth is 100 Kbit
Reliability 255/255, minimum MTU 1472
bytes
Loading 1/255, Hops 2
R1#show ip
eigrp topology 192.168.41.0/24
EIGRP-IPv4
Topology Entry for AS(1)/ID(2.2.2.1) for 192.168.41.0/24
State is Passive, Query origin flag is 1, 2
Successor(s), FD is 27008000
Descriptor Blocks:
172.1.0.4 (Tunnel1), from 172.1.0.4, Send
flag is 0x0
Composite metric is (27008000/128256), route
is Internal
Vector metric:
Minimum bandwidth is 100 Kbit
Total delay is 55000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1472
Hop count is 1
Originating router is 192.168.41.1
10.1.12.2 (FastEthernet0/0), from
10.1.12.2, Send flag is 0x0
Composite metric is (27008000/27005440),
route is Internal
Vector metric:
Minimum bandwidth is 100 Kbit
Total delay is 55000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1472
Hop count is 2
Originating router is 192.168.41.1
|
R1 sees network 192.168.41.0/24 from both paths equally!
R1#sh ip
route 192.168.41.0
Routing
entry for 192.168.41.0/24
Known via "eigrp 1", distance 90,
metric 27008000, type internal
Redistributing via eigrp 1
Last update from 172.1.0.4 on Tunnel1,
00:07:40 ago
Routing Descriptor Blocks:
172.1.0.4, from 172.1.0.4, 00:07:40 ago,
via Tunnel1
Route metric is 27008000, traffic share
count is 1
Total delay is 55000 microseconds,
minimum bandwidth is 100 Kbit
Reliability 255/255, minimum MTU 1472
bytes
Loading 1/255, Hops 1
* 10.1.12.2, from 10.1.12.2, 00:07:40 ago,
via FastEthernet0/0
Route metric is 27008000, traffic share
count is 1
Total delay is 55000 microseconds, minimum
bandwidth is 100 Kbit
Reliability 255/255, minimum MTU 1472
bytes
Loading 1/255, Hops 2
R1#show ip
cef 192.168.41.0
192.168.41.0/24
nexthop 10.1.12.2 FastEthernet0/0
nexthop 172.1.0.4 Tunnel1
|
The final configuration files can be found here:
