Practical OTV

Practical OTV
————-

This post is all about OTV (Overlay Transport Virtualization) on the CSR1000v.
I wanted to create the post because there are alot of acronyms and terminology involved.

A secondary objective was to have a “real” multicast network in the middle, as the examples I have seen around the web, have used a direct P2P network for the DCI.
Instead, I wanted to have full multicast running in the SP core in order to gain a full understanding of the packet forwarding and encapsulation.

First off, lets talk about the topology I will be using:

Datacenters:
————
We have 2 Datacenters, one represented by Site 1 and the other by Site 2.
In the middle, we have what is in all respects a SP provider network. In your environment, this may or may not be your own transport network.

In site 1, CSR-1 is our “server”, basically all thats configured on it is an IP address (192.168.100.1/24) on its G1 interface.
SW-9 is our L2 switch, which is configured with 2 VLAN’s (Vlan 100 (SERVER-VLAN) and Vlan 900 (SITE-VLAN)). The port (e0/0) going to CSR-1 is configured as an access-port in Vlan 100.

The ports going to CSR-2 and CSR-3 (e0/1 and e0/2) are trunk ports.

In site 1, the CSR-2 and CSR-3 routers are our OTV Edge devices, which is basically just a naming convention to designate your OTV encapsulation/decapsulation devices. In site 1 we are running two of these in order to show how the redundancy is performed.

Site 2 is very similar, although here I have selected to only have 1 OTV edge device (CSR-7).

In all sites, our OTV edge devices use their G2 interface as whats called their “Join Interface”. All that really means is that this is the L3 interface going towards the DCI “cloud”.

Also on all OTV edge devices, the G1 port is a L2 only interface, connecting to the internals of the respective site.

Transport Network:
——————
We have OSPF running as our IGP between all devices, providing full unicast reachability between our routers.
Inside the transport network, we are running Any Source Multicast (ASM), with the Rendevouz Point (RP) being the loopback0 interface of CSR-5 (5.5.5.5/32). All other routers (CSR-4 and CSR-6) having statically this RP configured.

Its important to note that no PIM adjacency exists between CSR-2 and CSR-4, and the same between CSR-3 and CSR-4. And the other way around no PIM adjacency exists between CSR-6 and CSR-7 either.

The only thing thats required is that we enable PIM on the link on CSR-4 and CSR-6. The reason behind that is that in IOS configuration, this is what also enables IGMP which is what we are really after in this solution. So you can think of CSR-2, CSR-3 and CSR-7 as “clients” of the multicast network sending IGMP joins (actually these are reports) to the transport network, which then handles the real multicast forwarding.

Terminology:
————
We have already covered quite a bit of terminology in the previous introduction, but let me iterate a few here:

– Join interface = Simply the L3 interface on the edge device, which face the transport network.
– Edge Device = Just an OTV router. Sits at the boundary between the L2 network you want transported and the L3 transport network.
– AED Device = Authoritive Edge Device. This is the “active” router, doing the transport of a certain VLAN. Only 1 AED for each VLAN on the site.
– Site VLAN = The Edge devices need to elect an AED for each VLAN that needs transporting. This is the function of the Site VLAN.
– Internal Interface = A L2 interface going towards the internal datacenter site. This is where we receive the frames we need to extend across OTV.
– Overlay Interface = The logical representation within Cisco IOS that ties all the pieces together.

Verification:
————-
Enough theory, lets see this beast in action on the command-line.
First off is our “servers”, which is CSR-1 and CSR-8:

CSR-1#sh run int g1
Building configuration...

Current configuration : 122 bytes
!
interface GigabitEthernet1
 ip address 192.168.100.1 255.255.255.0
 negotiation auto
 no mop enabled
 no mop sysid
end

and CSR-8:

CSR-8#sh run int g1
Building configuration...

Current configuration : 122 bytes
!
interface GigabitEthernet1
 ip address 192.168.100.8 255.255.255.0
 negotiation auto
 no mop enabled
 no mop sysid
end

Very simple. Nothing else of interest is configured on these devices, they simply serve as our validation platform.

What about the configuration on SW-9?

SW-9#sh vlan b      

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Et0/3
100  SERVER-VLAN                      active    Et0/0
900  SITE-VLAN                        active    
1002 fddi-default                     act/unsup 
1003 token-ring-default               act/unsup 
1004 fddinet-default                  act/unsup 
1005 trnet-default                    act/unsup 
SW-9#sh run int e0/0
Building configuration...

Current configuration : 81 bytes
!
interface Ethernet0/0
 switchport access vlan 100
 switchport mode access
end

SW-9#sh run int e0/1
Building configuration...

Current configuration : 90 bytes
!
interface Ethernet0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
end

SW-9#sh run int e0/2
Building configuration...

Current configuration : 90 bytes
!
interface Ethernet0/2
 switchport trunk encapsulation dot1q
 switchport mode trunk
end

And we can see the spanning-tree result for this configuration:

SW-9#sh span vl 900

VLAN0900
  Spanning tree enabled protocol rstp
  Root ID    Priority    33668
             Address     aabb.cc00.0900
             This bridge is the root
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    33668  (priority 32768 sys-id-ext 900)
             Address     aabb.cc00.0900
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Et0/1               Desg FWD 100       128.2    Shr 
Et0/2               Desg FWD 100       128.3    Shr 


SW-9#sh span vl 100

VLAN0100
  Spanning tree enabled protocol rstp
  Root ID    Priority    32868
             Address     aabb.cc00.0900
             This bridge is the root
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32868  (priority 32768 sys-id-ext 100)
             Address     aabb.cc00.0900
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Et0/0               Desg FWD 100       128.1    Shr 
Et0/1               Desg FWD 100       128.2    Shr 
Et0/2               Desg FWD 100       128.3    Shr 

Everything looks good so far!

Now for the interesting part, which is CSR-2, CSR-3 and CSR-7:

CSR-2:

otv site bridge-domain 900
!         
otv site-identifier 0001.0001.0001
!
interface Overlay1
 no ip address
 otv control-group 239.1.1.1
 otv data-group 232.1.1.0/24
 otv join-interface GigabitEthernet2
 no mop enabled
 no mop sysid
 service instance 100 ethernet
  encapsulation dot1q 100
  bridge-domain 100
 !
!
interface GigabitEthernet1
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 100 ethernet
  encapsulation dot1q 100
  bridge-domain 100
 !
 service instance 900 ethernet
  encapsulation dot1q 900
  bridge-domain 900
 !
!         
interface GigabitEthernet2
 ip address 172.2.4.2 255.255.255.0
 ip pim passive
 ip igmp version 3
 negotiation auto
 no mop enabled
 no mop sysid

First off, we set our Site Vlan to be 900, which is again what is used for AED election, locally to the site. This Vlan should never be extended over the OTV tunnel.
Then i set the identifier for our site. This is what is used in the loop prevention, so its very important that this is unique per site!

In the overlay interface configuration, I define a few things. The first of which is the multicast configuration, where we use group address 239.1.1.1 for our control traffic and 232.1.1.0/24 for data traffic. Next I specify that our Join interface is GigabitEthernet2. Finally I configure that we want to extend vlan 100 through the use of a service instance configuration snippet.

Toward the L2 site, we have GigabitEthernet1, where I have configured a L2 configuration using two Vlan’s. We want to have the router “listening” to both our Site Vlan (900) and our Data or Server Vlan (100) which is the one we want to extend across our DCI.

Last, but not least, we have GigabitEthernet2, which is our Join interface. This is a standard L3 interface configuration, with two important statements. First is “ip pim passive” which makes the interface run multicast, but not establish any pim adjacency and the other is “ip igmp version 3” which in effect makes the interface able to utilize SSM.

On CSR-3, the exact same configuration is present, with the exception of a different Join interface IP address:

otv site bridge-domain 900
!         
otv site-identifier 0001.0001.0001
!
interface Overlay1
 no ip address
 otv control-group 239.1.1.1
 otv data-group 232.1.1.0/24
 otv join-interface GigabitEthernet2
 no mop enabled
 no mop sysid
 service instance 100 ethernet
  encapsulation dot1q 100
  bridge-domain 100
 !
!
interface GigabitEthernet1
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 100 ethernet
  encapsulation dot1q 100
  bridge-domain 100
 !
 service instance 900 ethernet
  encapsulation dot1q 900
  bridge-domain 900
 !
!         
interface GigabitEthernet2
 ip address 172.3.4.3 255.255.255.0
 ip pim passive
 ip igmp version 3
 negotiation auto
 no mop enabled
 no mop sysid

Lets use some verification commands to see if everything is as we expect:

The first of which is “show otv adjacency”:

CSR-2#show otv adja
Overlay Adjacency Database for overlay 1
Hostname                       System-ID      Dest Addr       Site-ID        Up Time   State
CSR-7                          001e.e6de.0500 172.6.7.7       0002.0002.0002 00:28:01  UP   
CSR-3                          001e.7aa9.0100 172.3.4.3       0001.0001.0001 00:28:01  UP   

This command verifies that we have ISIS adjacencies up and running toward both CSR-3 (in site: 0001.0001.0001) and CSR-7 in site 0002.0002.0002, which is a very good start.

Next command i want, is to check if our Server vlan (100) is active on CSR-2 and CSR-3 (remember that only one of the routers should be active for this purpose):

CSR-2#show otv vlan
Key:  SI - Service Instance, NA - Non AED, NFC - Not Forward Capable. 

Overlay 1 VLAN Configuration Information
 Inst VLAN BD   Auth ED              State                Site If(s)          
 0    100  100   CSR-3               inactive(NA)        Gi1:SI100
 Total VLAN(s): 1

This command states that for Vlan 100, the AED is CSR-3 and that CSR-2 (this router) is inactive for this Vlan.

The reverse should be true when looking at CSR-3:

CSR-3#show otv vlan
Key:  SI - Service Instance, NA - Non AED, NFC - Not Forward Capable. 

Overlay 1 VLAN Configuration Information
 Inst VLAN BD   Auth ED              State                Site If(s)          
 0    100  100  *CSR-3               active              Gi1:SI100
 Total VLAN(s): 1

Thankfully this output verifies this behavior!

For site 2, we only have one edge device, so this router (CSR-7) should be the AED for site 2:

CSR-7#show otv vlan
Key:  SI - Service Instance, NA - Non AED, NFC - Not Forward Capable. 

Overlay 1 VLAN Configuration Information
 Inst VLAN BD   Auth ED              State                Site If(s)          
 0    100  100  *CSR-7               active              Gi1:SI100
 Total VLAN(s): 1

Which it indeed is.

For the next verification, I need to figure out the Mac addresses of our “servers”, namely CSR-1 and CSR-8:

CSR-1#sh int g1 | incl address
  Hardware is CSR vNIC, address is 5000.0001.0000 (bia 5000.0001.0000)

CSR-8#sh int g1 | incl address
  Hardware is CSR vNIC, address is 5000.0008.0000 (bia 5000.0008.0000)

So lets check out whether or not we have those Mac address announced through the OTV control plane:

On CSR-3 in Site-1:

CSR-3#show otv route

Codes: BD - Bridge-Domain, AD - Admin-Distance,
       SI - Service Instance, * - Backup Route

OTV Unicast MAC Routing Table for Overlay1

 Inst VLAN BD     MAC Address    AD    Owner  Next Hops(s)
----------------------------------------------------------
 0    100  100    001e.bdf9.e2bc 40    BD Eng Gi1:SI100
 0    100  100    5000.0001.0000 40    BD Eng Gi1:SI100
 0    100  100    5000.0008.0000 50    ISIS   CSR-7

3 unicast routes displayed in Overlay1

----------------------------------------------------------
3 Total Unicast Routes Displayed

Great! – We see both addresses, one on our G1 interface on the service 100 instance, which is the L2 interface going to our local server and another address known from CSR-7 from Site-2!

The opposite should be true from CSR-7’s point of view:

CSR-7#show otv route

Codes: BD - Bridge-Domain, AD - Admin-Distance,
       SI - Service Instance, * - Backup Route

OTV Unicast MAC Routing Table for Overlay1

 Inst VLAN BD     MAC Address    AD    Owner  Next Hops(s)
----------------------------------------------------------
 0    100  100    001e.bdf9.e2bc 50    ISIS   CSR-3
 0    100  100    5000.0001.0000 50    ISIS   CSR-3
 0    100  100    5000.0008.0000 40    BD Eng Gi1:SI100

3 unicast routes displayed in Overlay1

----------------------------------------------------------
3 Total Unicast Routes Displayed

To complete the picture of the solution, I would like to show the state of the transport network from a multicast perspective.

CSR-2 and CSR-3 should send an IGMP join/report to select traffic from 239.1.1.1 group which we previously configured, so lets check out if CSR-4 receives this:

CSR-4#sh ip igmp groups 
IGMP Connected Group Membership
Group Address    Interface                Uptime    Expires   Last Reporter   Group Accounted
239.1.1.1        GigabitEthernet4         01:38:12  00:02:53  172.3.4.3       
239.1.1.1        GigabitEthernet3         01:38:17  00:02:42  172.2.4.2       
224.0.1.40       Loopback0                01:38:31  00:02:29  4.4.4.4         

Excellent, CSR-4 has received information on both the G3 and G4 interface, going to CSR-2 and CSR-3 respectively.

When we in turn uses this IGMP, our multicast network, built by PIM should have some state information created:

CSR-4#sh ip mroute 239.1.1.1 |  beg Outgoing
Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.1.1.1), 01:40:59/stopped, RP 5.5.5.5, flags: SJCF
  Incoming interface: GigabitEthernet1, RPF nbr 10.4.5.5
  Outgoing interface list:
    GigabitEthernet4, Forward/Sparse, 01:40:55/00:02:11
    GigabitEthernet3, Forward/Sparse, 01:40:59/00:02:56

(172.6.7.7, 239.1.1.1), 01:40:16/00:01:29, flags: JT
  Incoming interface: GigabitEthernet2, RPF nbr 10.4.6.6
  Outgoing interface list:
    GigabitEthernet3, Forward/Sparse, 01:40:16/00:02:56
    GigabitEthernet4, Forward/Sparse, 01:40:16/00:02:11

(172.3.4.3, 239.1.1.1), 01:40:54/00:03:16, flags: FT
  Incoming interface: GigabitEthernet4, RPF nbr 0.0.0.0
  Outgoing interface list:
    GigabitEthernet2, Forward/Sparse, 01:40:14/00:03:27
    GigabitEthernet3, Forward/Sparse, 01:40:54/00:02:56

(172.2.4.2, 239.1.1.1), 01:40:57/00:03:15, flags: FT
  Incoming interface: GigabitEthernet3, RPF nbr 0.0.0.0
  Outgoing interface list:
    GigabitEthernet2, Forward/Sparse, 01:40:14/00:02:36
    GigabitEthernet4, Forward/Sparse, 01:40:55/00:02:11

This rather large output confirms that we have a (*,G) entry for 239.1.1.1 going to our RP which is CSR-5 and we have individual (S,G) from each of our OTV edge devices!

Somewhat the same information should be present on CSR-6:

CSR-6#sh ip mroute 239.1.1.1 |  beg Outgoing
Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.1.1.1), 01:43:05/stopped, RP 5.5.5.5, flags: SJCF
  Incoming interface: GigabitEthernet1, RPF nbr 10.5.6.5
  Outgoing interface list:
    GigabitEthernet3, Forward/Sparse, 01:43:05/00:02:52

(172.2.4.2, 239.1.1.1), 01:42:26/00:00:55, flags: JT
  Incoming interface: GigabitEthernet2, RPF nbr 10.4.6.4
  Outgoing interface list:
    GigabitEthernet3, Forward/Sparse, 01:42:26/00:02:52

(172.3.4.3, 239.1.1.1), 01:42:26/00:01:00, flags: JT
  Incoming interface: GigabitEthernet2, RPF nbr 10.4.6.4
  Outgoing interface list:
    GigabitEthernet3, Forward/Sparse, 01:42:26/00:02:52

(172.6.7.7, 239.1.1.1), 01:43:03/00:03:14, flags: FT
  Incoming interface: GigabitEthernet3, RPF nbr 0.0.0.0
  Outgoing interface list:
    GigabitEthernet2, Forward/Sparse, 01:42:19/00:03:29

Which indeed is the case.

So now that we have established that all the control-plane technology is working, lets try out the data-plane from CSR-1 to CSR-8:

CSR-1#ping 192.168.100.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.100.8, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/10/24 ms
CSR-1#sh arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.100.1           -   5000.0001.0000  ARPA   GigabitEthernet1
Internet  192.168.100.8           9   5000.0008.0000  ARPA   GigabitEthernet1

And thats it!

To summarize, what I have gone through in this post, is how to use the CSR1K platform to provide for DCI (Data Center Interconnect) using OTV. We have gone through the configuration of the individual devices, as well as having provided a “real” multicast transport network. We then verified the control-plane information and lastly we tested our dataplane connectivity.

I hope you have enjoyed this post!

Useful links for Observium + Rancid

I recently decided that i would like to utilize Observium as well as rancid for configuration backups on my home network. To that effect, the following links really helped me out getting it all setup correctly:

https://docs.observium.org/rancid/

http://packetsandpings.blogspot.com/2013/05/installing-and-configuring-rancid.html

https://layer77.net/2016/08/10/upgrading-from-rancid-2-3-8-to-3-4-1/

Let me know if you run into anything i might help out with.

/Kim

A quote from an Ex-Googler

I really like this paragraph, because almost everyone wants to imitate google. Why? well, the answer to that questions seems to be what everyone is missing!

Google’s solutions were built for scale that basically doesn’t exist outside of a maybe a handful of companies with a trillion dollar valuation. It’s foolish to assume that their solutions are better. They’re just more scalable. But they are actually very feature-poor. There’s a tradeoff there. We should not be imitating what Google did without thinking about why they did it. Sometimes the “whys” will apply to us, sometimes they won’t

The quote comes from Cloud Field Day 4, from Ben Sigelman of LightStep.

Thanks to Tom over at networkingnerd.net for the entire post!

/Kim

Complicated Vs. Complexity

I am currently reading Team of Teams, an excellent book!

In it, it highlights an interesting fact that I think is very relevant for the networking world and that is the difference between something that is complicated versus something that is complex.

There is a distinct difference in that something complicated can be broken down into its building blocks and analysed with a high degree of certainty. Think of a car engine for example. It is a very complicated piece of machinery for sure, but it is not complex, since you can divide its functionality down into components. On the other hand think of something like a virus and how it evolves. This is a complex organism that you you can’t be certain that will evolve in a predetermined fashion.

So im thinking, the way we build networks today, are we building them to be “just” complicated or are they really complex in nature instead? – The answer to this question determines how we need to manage our infrastructure!

Just some food for thought!

/Kim

JNCIA-Junos

I recently completed the entry level Juniper certification. I thought it would be a good idea to study for something other than the mighty Cisco, so Juniper’s JNCIA-Junos seemed like a good choice.

It was a very fair exam I can highly recommend.

Next up is AWS Solution Architect Associate. Need to get some cloud skills as thats where everything is going right? 🙂

VxLAN on the CSR1Kv

By now, VxLAN is becoming the standard way of tunneling in the Datacenter.
Using VxLAN, i will show how to use the CSR1Kv to extend your Datacenter L2 reach between sites as well.

First off, what is VxLAN?
It stands for Virtual Extensible LAN. Basically you have a way of decoupling your vlan’s into a new scheme.

You basically map your VLAN into a VNI (Virtual Network Identifier), which in essence makes your VLAN numbering scheme locally significant.

Also, since the numbering for VNI’s is a 24 bit identifier, you have alot more flexibility than just the regular 4096 definable VLAN’s. (12 Bits .1q tags)

Each endpoint that does the encapsulation/decapsulation is called a VTEP (VxLAN Tunnel EndPoint). In our example this would be CSR3 and CSR5.

After the VxLAN header, the packet is further encapsulated into a UDP packet and forwarded across the network. This is a great solution as it doesnt impose any technical restrictions on the core of the network. Only the VTEPs needs to understand VxLAN (and probably have hardware acceleration for it as well).

Since we wont be using BGP EVPN, we will rely solely on multicasting in the network to establish who is the VTEP’s for the traffic in question. The only supported mode is BiDir mode, which is an optimization of the control plane (not the data plane), since it only has (*,G) in its multicast-routing tables.

Lets take a look at the topology i will be using for the example:

 

I have used a regular IOS based device in Site 1 and Site 2, to represent our L2 devices. These could be servers or end-clients for that matter. What i want to accomplish is to run EIGRP between R1 and R2 over the “fabric” using VxLAN as the tunneling mechanism.

CSR3 is the VTEP for Site 1 and CSR5 is the VTEP for Site 2.

In the “fabric” we have CSR4, along with its loopback0 (4.4.4.4/32), which is the BiDir RP and its announcing this using BSR so that CSR3 and CSR4 knows this RP information (along with the BiDir functionality). We are using OSPF as the IGP in the “fabric” to establish routing between the loopback interfaces, which will be the VTEP’s respectively for CSR3 and CSR5.

Lets verify that routing between the loopbacks are working and our RIB is correct:

on CSR3:

CSR3#sh ip route | beg Gate
Gateway of last resort is not set

      3.0.0.0/32 is subnetted, 1 subnets
C        3.3.3.3 is directly connected, Loopback0
      4.0.0.0/32 is subnetted, 1 subnets
O        4.4.4.4 [110/2] via 10.3.4.4, 00:38:27, GigabitEthernet2
      5.0.0.0/32 is subnetted, 1 subnets
O        5.5.5.5 [110/3] via 10.3.4.4, 00:38:27, GigabitEthernet2
      10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C        10.3.4.0/24 is directly connected, GigabitEthernet2
L        10.3.4.3/32 is directly connected, GigabitEthernet2
O        10.4.5.0/24 [110/2] via 10.3.4.4, 00:38:27, GigabitEthernet2

CSR3#ping 5.5.5.5 so loo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 5.5.5.5, timeout is 2 seconds:
Packet sent with a source address of 3.3.3.3 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/7/22 ms

This means we have full reachability through the “fabric” from VTEP to VTEP.

Lets make sure our multicast routing is working properly and lets take a look at CSR4 first, since its the RP for the network:

CSR4#sh run | incl ip pim|interface
interface Loopback0
 ip pim sparse-mode
interface GigabitEthernet1
 ip pim sparse-mode
interface GigabitEthernet2
 ip pim sparse-mode
interface GigabitEthernet3
interface GigabitEthernet4
ip pim bidir-enable
ip pim bsr-candidate Loopback0 0
ip pim rp-candidate Loopback0 bidir

We can see from this output that we are running PIM on all the relevant interfaces as well as making sure that bidir is enabled. We have also verified that we are indeed running BSR to announce Loopback0 as the RP.

Lets verify the multicast routing table:

CSR4#sh ip mroute | beg Outgoing   
Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*,224.0.0.0/4), 00:45:05/-, RP 4.4.4.4, flags: B
  Bidir-Upstream: Loopback0, RPF nbr: 4.4.4.4
  Incoming interface list:
    GigabitEthernet2, Accepting/Sparse
    GigabitEthernet1, Accepting/Sparse
    Loopback0, Accepting/Sparse

(*, 239.1.1.1), 00:44:03/00:02:46, RP 4.4.4.4, flags: B
  Bidir-Upstream: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    GigabitEthernet1, Forward/Sparse, 00:44:03/00:02:38
    GigabitEthernet2, Forward/Sparse, 00:44:03/00:02:46

(*, 224.0.1.40), 00:45:05/00:01:56, RP 0.0.0.0, flags: DCL
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:45:04/00:01:56

We can see that we do have some (*,G) entries installed (more on the (*, 239.1.1.1) later).
Excellent.

Now lets take a sample from CSR3’s multicast configuration:

CSR3#sh ip pim rp mapping 
PIM Group-to-RP Mappings

Group(s) 224.0.0.0/4
  RP 4.4.4.4 (?), v2, bidir
    Info source: 4.4.4.4 (?), via bootstrap, priority 0, holdtime 150
         Uptime: 00:45:39, expires: 00:02:23

We see that we have learned the RP, its functionality as BiDir and its learned through BSR.

So far so good. Now lets turn our attention to the VxLAN part of the configuration.

The VTEP functionality is implemented by a new interface, called an NVE. This is where the configuration of which source address to use along with the multicast group to use for flooding is defined.

This is the configuration for CSR3:

CSR3#sh run int nve1
Building configuration...

Current configuration : 137 bytes
!
interface nve1
 no ip address
 source-interface Loopback0
 member vni 1000100 mcast-group 239.1.1.1
 no mop enabled
 no mop sysid
end

Whats important here is that we will source our VTEP from loopback0 (3.3.3.3/32) and use multicast group 239.1.1.1 for the VNI 1000100. This number can be whatever you choose, i have just chosen to use a very large number and encode which VLAN this VNI is used for (Vlan 100).

On the opposite side, we have a similar configuration for the NVE:

CSR5#sh run int nve1
Building configuration...

Current configuration : 137 bytes
!
interface nve1
 no ip address
 source-interface Loopback0
 member vni 1000100 mcast-group 239.1.1.1
 no mop enabled
 no mop sysid
end

Its very important that the multicast group matches on both sides as this is the group they will use to flood BUM (Broadcasts, Unknowns and Multicast) traffic. For example ARP.

The next configuration piece is that we need to create an EFP (Ethernet Flow Point) on the interface towards the site routers (R1 and R2) where we accept traffic tagged with vlan 100:

CSR3#sh run int g1
Building configuration...

Current configuration : 195 bytes
!
interface GigabitEthernet1
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 100 ethernet
  encapsulation dot1q 100
  rewrite ingress tag pop 1 symmetric
 !
end

This configuration piece states that the encap is dot1q vlan 100 and to strip the tag inbound before further processing and add it again on egress.

Now for the piece that ties it all together, namely the bridge-domain:

bridge-domain 100 
 member vni 1000100
 member GigabitEthernet1 service-instance 100

Here we have a bridge domain configuration where we have 2 members. The local interface G1 on its service instance 100 and our VNI / VTEP. This is basically the glue to tie the bridge domain together end to end.

The same configuration is present on CSR5 as well.

Let verify the control plane on CSR3:

CSR3#sh bridge-domain 100
Bridge-domain 100 (2 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
    GigabitEthernet1 service instance 100
    vni 1000100
   AED MAC address    Policy  Tag       Age  Pseudoport
   0   AABB.CC00.1000 forward dynamic   298  GigabitEthernet1.EFP100
   0   AABB.CC00.2000 forward dynamic   300  nve1.VNI1000100, VxLAN 
                                             src: 3.3.3.3 dst: 5.5.5.5

This command will show the MAC addresses learned in this particular bridge domain. On our EFP on G1 we have dynamically learned the MAC address of R1’s interface and through the NVE1 interface using VNI 1000100 we have learned the MAC address of R2. Pay attention to the fact that we know which VTEP endpoints to send the traffic to now. This means that further communication between these two end-hosts (R1 and R2) is done solely using unicast between 3.3.3.3 and 5.5.5.5 using VxLAN as the tunneling mechanism.

CSR3#show nve interface nve 1 detail 
Interface: nve1, State: Admin Up, Oper Up, Encapsulation: Vxlan,
BGP host reachability: Disable, VxLAN dport: 4789
VNI number: L3CP 0 L2DP 1
source-interface: Loopback0 (primary:3.3.3.3 vrf:0)
   Pkts In   Bytes In   Pkts Out  Bytes Out
      3273     268627       3278     269026

This command shows the status of our NVE interface. From this we can see that its in an Up/Up state. The VxLAN port is the standard destination port (4789) and we have some packets going back and forth.

Now that we have everything checked out okay in the control plane, lets see if the data plane is working by issuing an ICMP ping on R1 to R2 (they are obviously on the same subnet (192.168.100.0/24)):

R1#ping 192.168.100.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.100.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/11/26 ms
R1#sh arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.100.1           -   aabb.cc00.1000  ARPA   Ethernet0/0.100
Internet  192.168.100.2           8   aabb.cc00.2000  ARPA   Ethernet0/0.100

This looks excellent! and in fact the EIGRP peering i had setup between them works as well:

R1#sh ip eigrp neighbors 
EIGRP-IPv4 Neighbors for AS(100)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
0   192.168.100.2           Et0/0.100                12 04:14:30    4   100  0  3

R1#sh ip route eigrp | beg Gateway
Gateway of last resort is not set

      100.0.0.0/32 is subnetted, 2 subnets
D        100.100.100.2 
           [90/409600] via 192.168.100.2, 04:14:46, Ethernet0/0.100

This address is the loopback of R2.

Finally i want to show how the ICMP ping works in the dataplane by doing a capture on CSR4’s G2 interface:

Here we can see a ping i issued on R1’s loopback interface towards R2’s loopback interface.
I have extended the view, so you can see the encapsulation with the VxLAN header running atop the UDP packet.
The UDP packet has the VTEP endpoints (3.3.3.3/32 and 5.5.5.5/32) as the source and destination.

The VNI is what we selected to use and is used for differentiation on the VTEP.
Finally we have our L2 packet in its entirety.

Thats all I wanted to show for now. Next time I will extend this a bit and involve BGP as the control plane.
Thanks for reading!

ISIS Authentication types (packet captures)

In this post i would like to highlight a couple of “features” of ISIS.
More specifically the authentication mechanism used and how it looks in the data plane.

I will do this by configuring a couple of routers and configure the 2 authentication types available. I will then look at packet captures taken from the link between them and illustrate how its used by the ISIS process.

The 2 types of Authentication are link-level authentication of the Hello messages used to establish an adjacency and the second type is the authentication used to authenticate the LSP’s (Link State Packet) themselves.

First off, here is the extremely simple topology, but its all thats required for this purpose:

Simple, right? 2 routers with 1 link between them on Gig1. They are both running ISIS level-2-only mode, which means they will only try and establish a L2 adjacency with their neighbors. Each router has a loopback interface, which is also advertised into ISIS.

First off, lets look at the relevant configuration of CSR-02 for the Link-level authentication:

key chain MY-CHAIN
 key 1
  key-string WIPPIE
!
interface GigabitEthernet1
 ip address 10.1.2.2 255.255.255.0
 ip router isis 1
 negotiation auto
 no mop enabled
 no mop sysid
 isis authentication mode md5
 isis authentication key-chain MY-CHAIN

Without the same configuration on CSR-01, this is what we see in the data path (captured on CSR-02’s G1 interface):

And we also see that we dont have a full adjacency on CSR-01:

CSR-01#sh isis nei

Tag 1:
System Id       Type Interface     IP Address      State Holdtime Circuit Id
CSR-02          L2   Gi1           10.1.2.2        INIT  26       CSR-02.01

Lets apply the same authentication configuration on CSR-01 and see the result:

key chain MY-CHAIN
 key 1
  key-string WIPPIE
!
interface GigabitEthernet1
 ip address 10.1.2.1 255.255.255.0
 ip router isis 1
 negotiation auto
 no mop enabled
 no mop sysid
 isis authentication mode md5
 isis authentication key-chain MY-CHAIN

We now have a full adjacency:

CSR-01#sh isis neighbors 

Tag 1:
System Id       Type Interface     IP Address      State Holdtime Circuit Id
CSR-02          L2   Gi1           10.1.2.2        UP    8        CSR-02.01     

And we have routes from CSR-02:

CSR-01#sh ip route isis | beg Gate
Gateway of last resort is not set

      2.0.0.0/32 is subnetted, 1 subnets
i L2     2.2.2.2 [115/20] via 10.1.2.2, 00:01:07, GigabitEthernet1

Now, this is what we now see from CSR-02’s perspective again:

The Link-level authentication is fairly easy to spot in no time, because you simply wont have a stable adjacency formed.

The second type is LSP authentication. Lets look at the configuration of CSR-02 for this type of authentication:

CSR-02#sh run | sec router isis
 ip router isis 1
 ip router isis 1
router isis 1
 net 49.0000.0000.0002.00
 is-type level-2-only
 authentication mode text
 authentication key-chain MY-CHAIN

In this example, i have selected plain-text authentication, which i certainly dont recommend in production, but its great for our example.

Again, this is what it looks like in the data packet (from CSR-01 to CSR-02) without authentication enabled on CSR-01:

As you can see, we have the LSP that contains CSR-01’s prefixes, but nowhere is authentication present in the packet.

Lets enable it on CSR-01 and see the result:

CSR-01#sh run | sec router isis
 ip router isis 1
 ip router isis 1
router isis 1
 net 49.0000.0000.0001.00
 is-type level-2-only
 authentication mode text
 authentication key-chain MY-CHAIN

The result in the data packet:

Here we clearly have the authentication (with type = 10 (cleartext)) and we can see the password (WIPPIE) because we have selected cleartext.

The result is we a validated ISIS database on both routers.

Thats all folks, hope it helps to understand the difference between the 2 types of authentication in ISIS.

Take care!

 

 

Progress update – 10/07-2017

Hello folks,

Im currently going through the INE DC videos and learning a lot about fabrics and how they work along with a fair bit of UCS information on top of that!

Im spending an average of 2.5 hours on weekdays for study and a bit more in the weekends when time permits.

I still have no firm commitment to the CCIE DC track, but at some point I need to commit to it and really get behind it. One of these days 😉

I mentioned it to the wife-to-be a couple of days ago and while she didn’t applaud the idea, at least she wasn’t firmly against it, which is always something I guess! Its very important for me to have my family behind me in these endeavours!

Im still a bit concerned about the lack of rack rentals for DCv2 from INE, which is something I need to have in place before I order a bootcamp or more training materials from them. As people know by now, I really do my best learning in front of the “system”, trying out what works and what doesn’t.

Now to spin up a few N9K’s in the lab and play around with NX-OS unicast and multicast routing!

Take care.

New Lab Server & random updates

New Server:

So I just completed a purchase off eBay for a new server for my lab purposes.

For a while now I’ve been limited to 32Gb of memory on my old ESXi server, which is really more like 20Gb when my regular servers have had their share. Running a combination of different types of devices, each taking at least 4Gb of memory, doesn’t leave much room for larger labs.

I decided to go with a “real” server this time around. So I got an older Cisco UCS C200 M2 server with 2 x Xeon 5570 processors and an additional 96 Gb ram (on top of the 24 it came with). That stil leaves room for a bit of memory upgrades in the future (it supports a total of 192Gb) (had a budget on this one, so couldn’t go crazy).

Work:

Work has been crazy lately. 2 of my Team members just resigned so a lot of workload has to be shifted until we find suitable replacements. That means I’ve been working 65+ hour work weeks for a while now. Something that I dont find even remotely amusing to be honest. But I’ve been reassured that everything is being done to interview candidates, so im hopeful it will work out after the summer holidays.

We have a lot of interesting projects coming up, fx. our first production environment running Cisco ACI. This also included some very good training. Its really a different ball-game compared to the old way of doing Datacenters.

Also on my plate is some iWan solutions. Pretty interesting all in all.

Study:

Im still reading my way through the Cisco Intelligent WAN (IWAN) book. Its still on my list of things to do to take the exam I mentioned earlier, but keeping the work network running takes priority. Also I can’t help but feeling the pull of another CCIE when time permits, but its still just a thought (we all know how that usually goes right? 🙂 )

Personal:

September 16, my long-time girlfriend and I are getting married! Yes.. Married. Scary, but still something I look forward to. We’ve been together for an amazing 11 years on that date so its about time (she keeps telling me). As you may know, I proposed when I went to Las Vegas for Cisco Live last year, so its very memorable 🙂

Thats about it for now!

Take care!

Onto the next one…

Yesterday I passed the CCNA-W exam. Now onto the next partner certification I need to do before summer.

Its called 500-452 ENCWE – Enterprise Networks Core and WAN Essentials and a large part of it involves iWAN, which im not too familiar with.

To that effect I have ordered the official Cisco Press iWAN book and downloaded all the presentations I could find on iWAN from CiscoLive365. That should keep me busy for the foreseeable future 🙂

I will hopefully be doing some labs on iWAN and will post any findings I have here. It should be fun!

Im still debating whether or not I will goto CLUS this year. Whats really pulling me over there is the people I rarely get to meet. I need to make up my mind soon though.

Take care!

/Kim