GRE Backup

Using the following topology:

 

R1:

interface Tunnel12  backup delay 3 3  backup interface Tunnel13  ip address 10.1.12.1 255.255.255.0  keepalive 5 3  tunnel source FastEthernet1/0  tunnel destination 10.1.123.2 ! interface Tunnel13  ip address 10.1.13.1 255.255.255.0  tunnel source FastEthernet1/0  tunnel destination 10.1.123.3 ! interface FastEthernet0/0  no ip address  shutdown  duplex full ! interface FastEthernet1/0  ip address 10.1.123.1 255.255.255.0  speed auto  duplex auto ! ip route 192.168.41.0 255.255.255.0 Tunnel12 ip route 192.168.41.0 255.255.255.0 Tunnel13 50

R2:

interface Tunnel12  ip address 10.1.12.2 255.255.255.0  keepalive 5 3  tunnel source FastEthernet1/0  tunnel destination 10.1.123.1 ! interface FastEthernet1/0  ip address 10.1.123.2 255.255.255.0  speed auto  duplex auto ! interface FastEthernet1/1  ip address 10.1.234.2 255.255.255.0  standby 1 ip 10.1.234.254  standby 1 priority 150  standby 1 preempt  standby 1 track 1 decrement 100  speed auto  duplex auto ! ip route 192.168.41.0 255.255.255.0 10.1.234.4

R3:

interface Tunnel13  ip address 10.1.13.3 255.255.255.0  tunnel source FastEthernet1/0  tunnel destination 10.1.123.1 ! interface FastEthernet1/0  ip address 10.1.123.3 255.255.255.0  speed auto  duplex auto ! interface FastEthernet1/1  ip address 10.1.234.3 255.255.255.0  standby 1 ip 10.1.234.254  standby 1 priority 110  standby 1 preempt  speed auto  duplex auto ! ip route 192.168.41.0 255.255.255.0 10.1.234.4

R4:

interface Loopback1  ip address 192.168.41.1 255.255.255.0 ! interface FastEthernet1/0  ip address 10.1.234.4 255.255.255.0  speed auto  duplex auto ip route 0.0.0.0 0.0.0.0 10.1.234.254

R1 is configured with backup interface - tunnel 13 backup tunnel 12:

R1# show backup Primary Interface          Secondary Interface        Status -------------------------  -------------------------  ------ Tunnel12                   Tunnel13                   normal operation

When tunnel 12 goes down (I used keepalive but we can use other methods like IP-SLA) tunnel 13 become the backup and the floating route starts working:

R1# *Sep 27 11:32:57.495: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel12, changed state to down R1# *Sep 27 11:33:02.503: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel13, changed state to up R1# *Sep 27 11:33:02.523: %LINK-3-UPDOWN: Interface Tunnel13, changed state to up R1#show back R1#show backup Primary Interface          Secondary Interface        Status -------------------------  -------------------------  ------ Tunnel12                   Tunnel13                   backup mode

Note that tunnel 13 is down as long tunnel 12 is up.

Cisco Nexus 5K with 2248PQ connectivity problems

The Cisco Nexus 2248PQ is a 48 1/10 Gigabit Ethernet and FCoE host interfaces (SFP+) and 4 QSFP+ Gigabit Ethernet and FCoE fabric interfaces (QSFP+).

Data sheet: http://www.cisco.com/c/en/us/products/collateral/switches/nexus-2000-series-fabric-extenders/data_sheet_c78-507093.html

Like all Nexus 2200 switches he uses one or more of his uplinks ports to connect to the parent switch, for the 2248PQ there are 4x 40G QSFP+.

I had 3 options at my hand:

QSFP-H40G-CU3M – which is twinax passive cable

QSFP-4SFP10G-CU3M – which is copper breakout cable 40G to 4x 10G

And FET-40G transceiver with straight MPO-12 cable with MTP on both sides

 

MPO-12 straight

 

 

MTP to MTP connectors

 

Cisco FET-40G brown label

First I used the QSFP-4SFP10G-CU3M to connect 2248PQ port 1 to ETH 1/1-4 ports at the parent switch, this was simply straight forward, and the switch discovers the 2248PQ very quickly.

This is the ports configuration:

interface port-channel103   description FEX-103   switchport mode fex-fabric   fex associate 103 ! interface Ethernet1/1-4   switchport mode fex-fabric   fex associate 103   channel-group 103   no shutdown

Then I tried to use the QSFP-H40G-CU3M and the FET-40G, along with the MPO-12 cable, but the link didn’t come up:

N5K-1(config-if)# show interface eth 3/25 brief -------------------------------------------------------------------------------- Ethernet      VLAN    Type Mode   Status  Reason                   Speed     Por t Interface                                                                    Ch # -------------------------------------------------------------------------------- Eth3/25       1       eth  fabric down    Link not connected          40G(D) --

After a little research I found the following document:

http://www.cisco.com/c/en/us/support/docs/switches/nexus-2000-series-fabric-extenders/116348-technote-fex-00.html

Although the 2248PQ has 4x 40G QSFP+ physical ports we need to treat those ports as 10G when connecting to the parent home, hence the 2248PQ have 16x 10G uplink ports which represented by 4x 40G ports!

So this is the reason why the QSFP-4SFP10G-CU3M works and the other 2 doesn’t. 

But how do we connect 40G interface to the parent switch and make it believe its 4 ports 10G?

According to the module, where the 40G port reside on your parent switch, in my case it was on expansion module 3:

N5K-1(config)#interface breakout slot 3 port 25-26 map 10g-4x ! N5K-1(config)#poweroff module 3 N5K-1(config)#no poweroff module 3

Few notes:

      - We must configure breakout interface in groups – in my case I had to configure both 40G ports which reside on module 3

      - We need to reboot the module after changing the settings to take effect and this may take some time (about the time it’s take to the parent switch to reboot)

      - This cause interruption to the network activity (or more specifically to the hosts connected to the module which we reboot)

After that we get 4 more virtual ports under the single physical port - eth3/25/1-4, which we need to configure the same way as the QSFP-4SFP10G-CU3M.

B.T.W - The FET which stands for Fabric Extender Transceiver is a normal transceiver which Cisco marketing has decided to sell cheap but you won't be able to use them for any other purpose beside connecting FEX's.

B.T.W 2 - i had to use the command 'service unsupported-transceiver' in order for the FET-40G to work although it's genuine Cisco QSFP!!! I'm still waiting for some answers from Cisco TAC.