Category Archives: CCDE

CCDE – A different Journey

Wednesday the 22nd of February, in a testing center in the middle of London, my journey towards achieving the CCDE certification, finally ended in me passing this beast of an exam.

This learning journey was a very different one than either of my CCIE’s. Whereas going for the CCIE meant spending countless hours at the command-line, the CCDE meant spending all of those hours reading and discussing use cases for technologies. It also meant stepping my toes into the business side, picking up the “Why?” behind selecting a specific technology.

It all started a few years ago when my friend Daniel (lostintransit.se) and I started going back and forth on how to approach this thing. We decided we should team up and share notes, discuss technologies and generally use each other as a sparring partner.

At that point I had already decided, that this was going to be a marathon for me, because I could at the time, not allocate as much time each day for study, as I had been for the CCIE’s. Fast forward a good amount of time and I had finally passed the written part of the exam and was ready to really focus on the practical aspect.

Anyone reading the CCDE Practical blueprint basically come to the same conclussion in that it involves every technology under the sun and then some. This in turn made us start a small study group using Slack, with only 4 members to begin with. This was the turning point for me as we had discussions around scenarios and use cases. It helped tremendously that we all come from different backgrounds and industries as well as diverse geographies.

I attended Jeremy Filiben’s CCDE bootcamp in Orlando in April 2016 (http://www.jeremyfilliben.com), which was a great experience. Jeremy is a very good teacher and his scenarios are top notch. Going over them really makes you think in a completely different way than fx. the CCIE track(s). As an implementation engineer, you tend to think in a certain way which is not doing you any good in a design context.

Early on, I also purchased access to an All Access Pass from INE in order to gain access to their CCDE training material. Unfortunally it has not been updated since and they have no plan to 🙁

I also used the Self-Paced material from Orhan Ergun, especially the Quizzes was of help to me. (orhanergun.net).

I had my first attempt at the practical in late summer 2016 and I did fairly well, but it certainly opened my eyes to what i had gotten myself into. At the same time I had the fortune of meeting up with some of the Slack study group members which was an added bonus! (Thanks Andre for introducing me to a proper beer 🙂 )…

At this point I had some real life stuff to attend to (we purchased a house with all that entails). So I was unable to commit to the November testing date. However a good number of people from our study group passed which really motivated me to get on with my studies. So ever since late November I have been doing 3-4 hours of study each week day and 5-6 hours each day during the weekend.

The last month and a half before the exam i focused on doing practice scenarios and watching recordings made from our study group discussing designs. I can also highly recommend watching the new Safari Livelessons on QoS and Large Scale Network Designs. If time is of the essence to you, do some speed-viewing which is what i did as well.

I left for London on the day before the exam and checked into my hotel, which is only a 10 minute walk from the testing center. I had already mentally prepared on what to do on the day itself, so basically the day before was the last day to recharge my “batteries” and try and find some focus. I spent an hour or so going over my notes, especially the QoS and IPv6 parts, then went for dinner. I went to bed early as planned.

On the day I woke up very early, which is not unusual for me in any way, so I basically followed what I had mentally prepared for myself which was to take a long and hot shower, get a decent breakfast (hard part for me as i dont normally have breakfast) and then back to the room to collect my things (Exam registration, wallet and passport (You need two forms of ID)). I left the hotel at 7:20’ish and took the short walk to the testing center. Since I was early i waited outside for a bit waiting for the other guy from our study group to show up to say Hi before the exam, but he was running a bit late, so I finally decided that I had to get started and went inside.

Since the exam is under heavy NDA like any other Cisco exam, I wont get into any details regarding the experience, except to say that at no point in the exam did i feel really comfortable about passing it, quite the contrary in fact. However I knew this is a feeling most candidates experience during the CCDE, so I just decided to press on and do my very best.

At lunch I took the advice of several people and had a bite to eat (even though i still didnt feel like eating) and some water and used close to the full hour of lunch available. It being London and all, there are several good places for lunch very close by to the testing center, so you dont need to go very far at all.
The afternoon went by and I finally clicked the End Exam button and instantly my mood picked up. I had passed!! I honestly didnt know what to do with myself at this point 🙂

I had a brisk walk back to the hotel where i could finally put my “guard” down and smile a bit!
I called my better half and told her the good news and she was even more estatic than I was.

So what sort of advice can I give to others?

Read, Watch Sessions (Cisco Live, Safari, Nanog etc.), constantly asking “Why?” to everything. Learn to read through documents and pick up on Business requirements and details that will pertain to certain design choices. Read some more.

If you are like me and you are easily distracted during your studies, I can highly recommend using the “Pomodoro”-method (Look it up), which is essentially 25 minute slots of focus on your studies and then a short break. It gives you a scope of focus and helps keep you on track. On top of that I mark down everything i do, study wise, into my calendar so I can look back at it and get a good “feel” for how much time i spend on it. It helps to give you a boost when you feel that you havent put enough effort into it.

If you want some recommendation on which book(s) to read, here’s a subset of the books I’ve read:

1) Definitive MPLS Network Designs.
2) CCDE Study Guide.
3) Optimal Routing Design.
4) End-To-End QoS.

These are by far the most important ones, but by no means the only ones you want to read through. You have to assess which technologies you need to learn (more) about and then pick the right material for those cases. The books above are very good for general theory but especially Definitive MPLS Network Designs is good for putting all the relevant pieces into 4 distinct use cases.

Some of the Cisco Live sessions i went through includes:

– Scaling BGP (BRKRST-3321)
– Wan and Remote-Site Deployment using CVDs (BRKRST-2040)
– Highly available Wide Area Network Design (BRKRST-2042)
– WAN Architectures and Design Principles (BRKRST-2041)
– Layer 3 Network Virtualization Design Concepts over the WAN (BRKRST-2045)
– Deploying a virtualized campus network infrastructure (BRKCRS-2033)
– Best practices to deploy HA in SP edge and aggregation architectures (BRKSPG-2402)
– Advanced enterprise campus design: routed access (BRKCRS-3036)
– Deploying BGP Fast Convergence / BGP PIC (BRKIPM-2265)
– The QoS Paradigm Shift (BRKRST-2056)
– Deploying OSPF in modern networks (BRKRST-2337)
– ISIS Deployment in modern networks (BRKRST-2338)
– IPv6 Transition Technologies (BRKSPG-2067)
– Choosing the right VPN technology for your network (BRKSEC-1050)
– Firewall architectures in the Data Centre (BRKSEC-2021)
– MAP-E/MAP-T IPv6 transitioning (BRKSPG-2606)

There are a bunch of others and I recommend you search for “Design” and “Use Case” on CiscoLive365.com (It really is an awesome resource for learning).

The last piece of advice I can give, is to join a study group, or if you are more inclined, create one yourself along with some of your friends who are serious about the CCDE as well.
Have discussions revolving around technologies and have even more discussion on scenarios and use cases for those technologies. It really is quite important to expand your horizont in order to be successful in this exam.

If you would like, at certain times we have openings in our study group (which counts 60+ people at the time of writing, including several well known others and vendors), so get in touch if thats your preference.

With that said, I would like to thank you for reading through this post. Its been a fun learning experience all the way through my CCDE – Journey!

/Kim (CCDE #20170021)

Snippet: The story of the EFP

For a while now, the concept of EVC’s (Ethernet Virtual Circuits) and EFP’s (Ethernet Flow Points), has eluded me.

In this short post, i will provide you with a simple example of a couple of EFP’s. In a later post i will discuss the MEF concept of EVC’s.

As always, here is the topology i will be using:

topology

Its a very simple setup. R1 connects to R2 through its G1 interface and connects to R3 through its G2 interface.

On R2 and R3, we have the very common configuration of using subinterfaces for the individual Vlan’s in question. Namely Vlan 10 for the connection between R1 and R2 and Vlan 20 between R1 and R3.

Here is the configuration of R2 and R3:

R2#sh run int g1.10
Building configuration...
Current configuration : 98 bytes
!
interface GigabitEthernet1.10
encapsulation dot1Q 10
ip address 10.10.10.2 255.255.255.0
end

R3#sh run int g1.20
Building configuration...
Current configuration : 98 bytes
!
interface GigabitEthernet1.20
encapsulation dot1Q 20
ip address 10.10.10.3 255.255.255.0
end

Now on R1 is where the “different” configuration takes place:

R1#sh run int g1
Building configuration...
Current configuration : 182 bytes
!
interface GigabitEthernet1
no ip address
negotiation auto
service instance 10 ethernet
encapsulation dot1q 10
rewrite ingress tag pop 1 symmetric
bridge-domain 10
!
end

R1#sh run int g2
Building configuration...
Current configuration : 182 bytes
!
interface GigabitEthernet2
no ip address
negotiation auto
service instance 20 ethernet
encapsulation dot1q 20
rewrite ingress tag pop 1 symmetric
bridge-domain 10
!
end

R1#sh run int bdi10
Building configuration...
Current configuration : 96 bytes
!
interface BDI10
description -= Our L3 interface =-
ip address 10.10.10.1 255.255.255.0
end

So what does this all mean!? – Well, basically what you are looking at is the very nature of an EFP. One on each physical interface in this case. It is defined under the “service instance” command structure.

An Ethernet Flow Point (EFP) is a way to match a certain ethernet frame, do an action on it ingress (and also in our case egress). On top of that you can attach it to a bridge-domain.

The result of the above configuration is that on G1, we match on the dot1q tag when its tagged with vlan 10. On ingress we then pop 1 tag before performing any other “upstream” action. With the symmetric keyword, we attach the vlan 10 tag when egressing.

On G2, we are doing the same, but with vlan 20 instead.

With both EFP’s we attach a bridge-domain (ID 10), which can be verified like this:

R1#show bridge-domain
Bridge-domain 10 (3 ports in all)
State: UP Mac learning: Enabled
Aging-Timer: 300 second(s)
BDI10 (up)
GigabitEthernet1 service instance 10
GigabitEthernet2 service instance 20
AED MAC address Policy Tag Age Pseudoport
- 001E.7AE0.11BF to_bdi static 0 BDI10

Right now we only have one mac address learned, namely of our L3 BDI interface. But we can see that both G1 and G2 has a service instance in this bridge-domain.

Lets try and do some ICMP tests from R2:

R2#ping 10.10.10.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/120/598 ms

Lets again verify our bridge-domain on R1:

R1#show bridge-domain
Bridge-domain 10 (3 ports in all)
State: UP Mac learning: Enabled
Aging-Timer: 300 second(s)
BDI10 (up)
GigabitEthernet1 service instance 10
GigabitEthernet2 service instance 20
AED MAC address Policy Tag Age Pseudoport
- 001E.7AE0.11BF to_bdi static 0 BDI10
0 0050.56BE.18D8 forward dynamic 276 GigabitEthernet1.EFP10

What we see now, is that a Mac address has been dynamically learned through the G1.EFP10 EFP.

Since we are technically “bridging” these two distinct vlans, we should be able to ping R3 from R2 as well:

R2#ping 10.10.10.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/24/104 ms

And again on R1:

R1#show bridge-domain
Bridge-domain 10 (3 ports in all)
State: UP Mac learning: Enabled
Aging-Timer: 300 second(s)
BDI10 (up)
GigabitEthernet1 service instance 10
GigabitEthernet2 service instance 20
AED MAC address Policy Tag Age Pseudoport
- 001E.7AE0.11BF to_bdi static 0 BDI10
0 0050.56BE.320A forward dynamic 287 GigabitEthernet2.EFP20
0 0050.56BE.18D8 forward dynamic 287 GigabitEthernet1.EFP10

We have now learned all the Mac addresses in our small test environment.

So thats basically all there is to an EFP. A simple way of providing a flexible way of matching frames.

Until next time! – Take care.

 

New practice lab(s) available…

In case you are serious about going for the CCDE certification, I highly recommend you check out my friend Martin Duggan’s new lab(s) on Leanpub. His writing style is very good and its easy to follow along and i look forward to hitting this lab myself.

So go ahead and pay the man and get an additional CCDE lab for your studies. Take care!

https://leanpub.com/ccdepracticalstudies-practicelab1

 

Practical DMVPN Example

In this post, I will put together a variety of different technologies involved in a real-life DMVPN deployment.

This includes things such as the correct tunnel configuration, routing-configuration using BGP as the protocol of choice, as well as NAT toward an upstream provider and front-door VRF’s in order to implement a default-route on both the Hub and the Spokes and last, but not least a newer feature, namely Per-Tunnel QoS using NHRP.

So I hope you will find the information relevant to your DMVPN deployments.

First off, lets take a look at the topology I will be using for this example:
Topology

As can be seen, we have a hub router which is connected to two different ISP’s. One to a general purpose internet provider (the internet cloud in this topology) which is being used as transport for our DMVPN setup, as well as a router in the TeleCom network (AS 59701), providing a single route for demonstration purposes (8.8.8.8/32). We have been assigned the 70.0.0.0/24 network from TeleCom to use for internet access as well.

Then we have to Spoke sites, with a single router in each site (Spoke-01 and Spoke-02 respectively).
Each one has a loopback interface which is being announced.

The first “trick” here, is to use the so-called Front Door VRF feature. What this basically allows is to have your transport interface located in a separate VRF. This allows us to have 2 default (0.0.0.0) routes. One used for the transport network and one for the “global” VRF, which is being used by the clients behind each router.

I have created a VRF on the 3 routers in our network (the Hub, Spoke-01 and Spoke-02) called “Inet_VRF”. Lets take a look at the configuration for this network along with its routing information (RIB):


HUB#sh run | beg vrf defi
vrf definition Inet_VRF
!
address-family ipv4
exit-address-family

HUB#sh ip route vrf Inet_VRF | beg Ga
Gateway of last resort is 130.0.0.1 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 130.0.0.1
130.0.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 130.0.0.0/30 is directly connected, GigabitEthernet1
L 130.0.0.2/32 is directly connected, GigabitEthernet1

Very simple indeed. We are just using the IPv4 address-family for this VRF and we have a static default route pointing toward the Internet Cloud.

The spokes are very similar:

Spoke-01:





Spoke-01#sh ip route vrf Inet_VRF | beg Gat
Gateway of last resort is 140.0.0.1 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 140.0.0.1
140.0.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 140.0.0.0/30 is directly connected, GigabitEthernet1
L 140.0.0.2/32 is directly connected, GigabitEthernet1



Spoke-02:




Spoke-02#sh ip route vrf Inet_VRF | beg Gat
Gateway of last resort is 150.0.0.1 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 150.0.0.1
150.0.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 150.0.0.0/30 is directly connected, GigabitEthernet1
L 150.0.0.2/32 is directly connected, GigabitEthernet1



With this in place, we should have full reachability to the internet interface address of each router in the Inet_VRF:


HUB#ping vrf Inet_VRF 140.0.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 140.0.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/22/90 ms

HUB#ping vrf Inet_VRF 150.0.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 150.0.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/16/65 ms

With this crucial piece of configuration for the transport network, we can now start building our DMVPN network, starting with the Hub configuration:

HUB#sh run int tun100
Building configuration...



Current configuration : 452 bytes
!
interface Tunnel100
ip address 172.16.0.100 255.255.255.0
no ip redirects
ip mtu 1400
ip nat inside
ip nhrp network-id 100
ip nhrp redirect
ip tcp adjust-mss 1360
load-interval 30
nhrp map group 10MB-Group service-policy output 10MB-Parent
nhrp map group 30MB-Group service-policy output 30MB-Parent
tunnel source GigabitEthernet1
tunnel mode gre multipoint
tunnel vrf Inet_VRF
tunnel protection ipsec profile DMVPN-PROFILE shared
end



There are a fair bit of configuration here. However, pay attention to the “tunnel vrf Inet_VRF” command, as this is the piece that ties into your transport address for the tunnel. So basically we use G1 for the interface, and this is located in the Inet_VRF.

Also notice that we are running crypto on top of our tunnel to protect it from prying eyes. The relevant configuration is here:

crypto keyring MY-KEYRING vrf Inet_VRF
pre-shared-key address 0.0.0.0 0.0.0.0 key SUPER-SECRET
!
!
!
!
!
crypto isakmp policy 1
encr aes 256
hash sha256
authentication pre-share
group 2
!
!
crypto ipsec transform-set TRANSFORM-SET esp-aes 256 esp-sha256-hmac
mode transport
!
crypto ipsec profile DMVPN-PROFILE
set transform-set TRANSFORM-SET



Pretty straightforward with a static pre shared key in place for all nodes.

With the crypto in place, you should have a SA for it installed:


HUB#sh crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
130.0.0.2 150.0.0.2 QM_IDLE 1002 ACTIVE
130.0.0.2 140.0.0.2 QM_IDLE 1001 ACTIVE



One for each spoke is in place an in the correct state (QM_IDLE = Good).

So now, lets verify the entire DMVPN solution with a few “show” commands:

HUB#sh dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
T1 - Route Installed, T2 - Nexthop-override
C - CTS Capable
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel100, IPv4 NHRP Details
Type:Hub, NHRP Peers:2,

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 140.0.0.2 172.16.0.1 UP 05:24:09 D
1 150.0.0.2 172.16.0.2 UP 05:24:03 D


We have 2 spokes associated with our Tunnel100. One where the public address (NBMA) is 140.0.02 and the other 150.0.0.2. Inside the tunnel, the spokes have an IPv4 address of 172.16.0.1 and .2 respectively. Also, we can see that these are being learned Dynamically (The D in the 5 column).

All is well and good so far. But we still need to run a routing protocol across the tunnel interface in order to exchange routes. BGP and EIGRP are good candidates for this and in this example I have used BGP. And since we are running Phase 3 DMVPN, we can actually get away with just receiving a default route on the spokes!. At this point remember that our previous default route pointing toward the internet was in the Inet_VRF table, so these two wont collide.

Take a look at the BGP configuration on the hub router:


HUB#sh run | beg router bgp
router bgp 100
bgp log-neighbor-changes
bgp listen range 172.16.0.0/24 peer-group MYPG
network 70.0.0.0 mask 255.255.255.0
neighbor MYPG peer-group
neighbor MYPG remote-as 100
neighbor MYPG next-hop-self
neighbor MYPG default-originate
neighbor MYPG route-map ONLY-DEFAULT-MAP out
neighbor 133.1.2.2 remote-as 59701
neighbor 133.1.2.2 route-map TO-UPSTREAM-SP out



And the referenced route-maps:


ip prefix-list ONLY-DEFAULT-PFX seq 5 permit 0.0.0.0/0
!
ip prefix-list OUR-SCOPE-PFX seq 5 permit 70.0.0.0/24
!
route-map TO-UPSTREAM-SP permit 5
match ip address prefix-list OUR-SCOPE-PFX
!
route-map TO-UPSTREAM-SP deny 10
!
route-map ONLY-DEFAULT-MAP permit 10
match ip address prefix-list ONLY-DEFAULT-PFX



We are using the BGP listen feature, which makes it dynamic to setup BGP peers. We allow everything in the 172.16.0.0/24 network to setup a BGP session and we are using the Peer-Group MYPG for controlling the settings. Notice that we are sending out only a default route to the spokes.

Also pay attention to the fact that we are sending the 70.0.0.0/24 upstream to the TeleCom ISP. Since we are going to use this network for NAT’ing purposes only, we have a static route to Null0 installed as well:


HUB#sh run | incl ip route
ip route 70.0.0.0 255.255.255.0 Null0



For the last part of our BGP configuration, lets take a look at the Spoke configuration, which is very simple and straightforward:


Spoke-01#sh run | beg router bgp
router bgp 100
bgp log-neighbor-changes
redistribute connected route-map ONLY-LOOPBACK0
neighbor 172.16.0.100 remote-as 100



And the associated route-map:


route-map ONLY-LOOPBACK0 permit 10
match interface Loopback0

So basically thats a cookie-cutter configuration thats being reused on Spoke-02 as well.

So what does the routing end up with on the Hub side of things:


HUB# sh ip bgp | beg Networ
Network Next Hop Metric LocPrf Weight Path
0.0.0.0 0.0.0.0 0 i
*>i 1.1.1.1/32 172.16.0.1 0 100 0 ?
*>i 2.2.2.2/32 172.16.0.2 0 100 0 ?
*> 8.8.8.8/32 133.1.2.2 0 0 59701 i
*> 70.0.0.0/24 0.0.0.0 0 32768 i

We have a default route, which we injected using the default-information originate command. Then we receive the loopback addresses from each of the two spokes. Finally we have the network statement, the hub inserted, sending it to the upstream TeleCom. Finally we have 8.8.8.8/32 from TeleCom 🙂

Lets look at the spokes:


Spoke-01#sh ip bgp | beg Network
Network Next Hop Metric LocPrf Weight Path
*>i 0.0.0.0 172.16.0.100 0 100 0 i
*> 1.1.1.1/32 0.0.0.0 0 32768 ?

Very straightforward and simple. A default from the hub and the loopback we ourself injected.

Right now, we are in a situation where we have the correct routing information and we are ready to let NHRP do its magic. Lets take a look what happens when Spoke-01 sends a ping to Spoke-02’s loopback address:


Spoke-01#ping 2.2.2.2 so loo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/17/32 ms

Everything works, and we the math is right, we should see an NHRP shortcut being created for the Spoke to Spoke tunnel:


Spoke-01#sh ip nhrp shortcut
2.2.2.2/32 via 172.16.0.2
Tunnel100 created 00:00:06, expire 01:59:53
Type: dynamic, Flags: router rib
NBMA address: 150.0.0.2
Group: 30MB-Group
172.16.0.2/32 via 172.16.0.2
Tunnel100 created 00:00:06, expire 01:59:53
Type: dynamic, Flags: router nhop rib
NBMA address: 150.0.0.2
Group: 30MB-Group

and on Spoke-02:


Spoke-02#sh ip nhrp shortcut
1.1.1.1/32 via 172.16.0.1
Tunnel100 created 00:01:29, expire 01:58:31
Type: dynamic, Flags: router rib
NBMA address: 140.0.0.2
Group: 10MB-Group
172.16.0.1/32 via 172.16.0.1
Tunnel100 created 00:01:29, expire 01:58:31
Type: dynamic, Flags: router nhop rib
NBMA address: 140.0.0.2
Group: 10MB-Group

And the RIB on both routers should reflect this as well:


Gateway of last resort is 172.16.0.100 to network 0.0.0.0



B* 0.0.0.0/0 [200/0] via 172.16.0.100, 06:09:37
1.0.0.0/32 is subnetted, 1 subnets
C 1.1.1.1 is directly connected, Loopback0
2.0.0.0/32 is subnetted, 1 subnets
H 2.2.2.2 [250/255] via 172.16.0.2, 00:02:08, Tunnel100
172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks
C 172.16.0.0/24 is directly connected, Tunnel100
L 172.16.0.1/32 is directly connected, Tunnel100
H 172.16.0.2/32 is directly connected, 00:02:08, Tunnel100

The “H” routes are from NHRP

And just to verify that our crypto is working, heres a capture from wireshark on the internet “Cloud” when pinging from Spoke-01 to Spoke-02:

wireshark

Now lets turn our attention to the Quality of Service aspect of our solution.

We have 3 facts to deal with.

1) The Hub router has a line-rate Gigabit Ethernet circuit to the Internet.
2) The Spoke-01 site has a Gigabit Ethernet circuit, but its a subrate to 10Mbit access-rate.
3) The Spoke-02 site has a Gigabit Ethernet circuit, but its a subrate to 30Mbit access-rate.

We somehow want to signal to the Hub site to “respect” these access-rates. This is where the “Per-Tunnel QoS” feature comes into play.

If you remember the Hub tunnel100 configuration, which looks like this:


interface Tunnel100
ip address 172.16.0.100 255.255.255.0
no ip redirects
ip mtu 1400
ip nat inside
ip nhrp network-id 100
ip nhrp redirect
ip tcp adjust-mss 1360
load-interval 30
nhrp map group 10MB-Group service-policy output 10MB-Parent
nhrp map group 30MB-Group service-policy output 30MB-Parent
tunnel source GigabitEthernet1
tunnel mode gre multipoint
tunnel vrf Inet_VRF
tunnel protection ipsec profile DMVPN-PROFILE shared



We have these 2 “nhrp map” statements. What these in effect does is to provide a framework for the Spokes to register using one of these two maps for the Hub to use to that individual spoke.

So these are the policy-maps we reference:


HUB#sh policy-map
Policy Map 30MB-Child
Class ICMP
priority 5 (kbps)
Class TCP
bandwidth 50 (%)

Policy Map 10MB-Parent
Class class-default
Average Rate Traffic Shaping
cir 10000000 (bps)
service-policy 10MB-Child

Policy Map 10MB-Child
Class ICMP
priority 10 (%)
Class TCP
bandwidth 80 (%)


Policy Map 30MB-Parent
Class class-default
Average Rate Traffic Shaping
cir 30000000 (bps)
service-policy 30MB-Child



We have a hiearchical policy for both the 10Mbit and 30Mbit groups. each with their own child policies.

On the Spoke side of things, all we have to do is to tell the Hub which group to use:


interface Tunnel100
bandwidth 10000
ip address 172.16.0.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp map 172.16.0.100 130.0.0.2
ip nhrp map multicast 130.0.0.2
ip nhrp network-id 100
ip nhrp nhs 172.16.0.100
ip nhrp shortcut
ip tcp adjust-mss 1360
load-interval 30
nhrp group 10MB-Group
qos pre-classify
tunnel source GigabitEthernet1
tunnel mode gre multipoint
tunnel vrf Inet_VRF
tunnel protection ipsec profile DMVPN-PROFILE



Here on Spoke-01, we request that the QoS group to be used is 10MB-Group.

And on Spoke-02:


interface Tunnel100
bandwidth 30000
ip address 172.16.0.2 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp map 172.16.0.100 130.0.0.2
ip nhrp map multicast 130.0.0.2
ip nhrp network-id 100
ip nhrp nhs 172.16.0.100
ip nhrp shortcut
ip tcp adjust-mss 1360
load-interval 30
nhrp group 30MB-Group
qos pre-classify
tunnel source GigabitEthernet1
tunnel mode gre multipoint
tunnel vrf Inet_VRF
tunnel protection ipsec profile DMVPN-PROFILE



We request the 30MB-Group.

So lets verify that the Hub understands this and applies it accordingly:


HUB#sh nhrp group-map
Interface: Tunnel100
NHRP group: 10MB-Group
QoS policy: 10MB-Parent
Transport endpoints using the qos policy:
140.0.0.2



NHRP group: 30MB-Group
QoS policy: 30MB-Parent
Transport endpoints using the qos policy:
150.0.0.2



Excellent. and to see that its actually applied correctly:


HUB#sh policy-map multipoint tunnel 100

 

Interface Tunnel100 <--> 140.0.0.2

Service-policy output: 10MB-Parent

Class-map: class-default (match-any)
903 packets, 66746 bytes
30 second offered rate 0000 bps, drop rate 0000 bps
Match: any
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 900/124744
shape (average) cir 10000000, bc 40000, be 40000
target shape rate 10000000

Service-policy : 10MB-Child

queue stats for all priority classes:
Queueing
queue limit 512 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 10/1860

Class-map: ICMP (match-all)
10 packets, 1240 bytes
30 second offered rate 0000 bps, drop rate 0000 bps
Match: protocol icmp
Priority: 10% (1000 kbps), burst bytes 25000, b/w exceed drops: 0
Class-map: TCP (match-all)
890 packets, 65494 bytes
30 second offered rate 0000 bps, drop rate 0000 bps
Match: access-group 110
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 890/122884
bandwidth 80% (8000 kbps)

Class-map: class-default (match-any)
3 packets, 12 bytes
30 second offered rate 0000 bps, drop rate 0000 bps
Match: any

queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 0/0

Interface Tunnel100 <--> 150.0.0.2

Service-policy output: 30MB-Parent

Class-map: class-default (match-any)
901 packets, 66817 bytes
30 second offered rate 0000 bps, drop rate 0000 bps
Match: any
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 901/124898
shape (average) cir 30000000, bc 120000, be 120000
target shape rate 30000000

Service-policy : 30MB-Child

queue stats for all priority classes:
Queueing
queue limit 512 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 10/1860

Class-map: ICMP (match-all)
10 packets, 1240 bytes
30 second offered rate 0000 bps, drop rate 0000 bps
Match: protocol icmp
Priority: 5 kbps, burst bytes 1500, b/w exceed drops: 0
Class-map: TCP (match-all)
891 packets, 65577 bytes
30 second offered rate 0000 bps, drop rate 0000 bps
Match: access-group 110
Queueing
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 891/123038
bandwidth 50% (15000 kbps)

Class-map: class-default (match-any)
0 packets, 0 bytes
30 second offered rate 0000 bps, drop rate 0000 bps
Match: any

queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 0/0


The last piece of the QoS puzzle is to make sure you have a service-policy applied on the transport interfaces on the spokes as well:


Spoke-01#sh run int g1
Building configuration...

&nbsp;


Current configuration : 210 bytes
!
interface GigabitEthernet1
description -= Towards Internet Router =-
bandwidth 30000
vrf forwarding Inet_VRF
ip address 140.0.0.2 255.255.255.252
negotiation auto
service-policy output 10MB-Parent
end



and on Spoke-02:


poke-02#sh run int g1
Building configuration...

&nbsp;


Current configuration : 193 bytes
!
interface GigabitEthernet1
description -= Towards Internet Router =-
vrf forwarding Inet_VRF
ip address 150.0.0.2 255.255.255.252
negotiation auto
service-policy output 30MB-Parent
end



The last thing i want to mention is the NAT on the hub to use the 70.0.0.0/24 network for the outside world. Pretty straight forward NAT (inside on the tunnel interface 100 and outside on the egress interface toward Telecom, G2):


HUB#sh run int g2
Building configuration...

&nbsp;


Current configuration : 106 bytes
!
interface GigabitEthernet2
ip address 133.1.2.1 255.255.255.252
ip nat outside
negotiation auto
end



Also the NAT configuration itself:


ip nat pool NAT-POOL 70.0.0.1 70.0.0.253 netmask 255.255.255.0
ip nat inside source list 10 pool NAT-POOL overload
!
HUB#sh access-list 10
Standard IP access list 10
10 permit 1.1.1.1
20 permit 2.2.2.2



We are only NAT’ing the two loopbacks from the spokes on our network.

Lets do a final verification on the spokes to 8.8.8.8/32:




Spoke-01#ping 8.8.8.8 so loo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/23/56 ms

and Spoke-02:


Spoke-02#ping 8.8.8.8 so loo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of 2.2.2.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 10/14/27 ms



Lets verify the NAT state table on the hub:


HUB#sh ip nat trans
Pro Inside global Inside local Outside local Outside global
icmp 70.0.0.1:1 1.1.1.1:5 8.8.8.8:5 8.8.8.8:1
icmp 70.0.0.1:2 2.2.2.2:0 8.8.8.8:0 8.8.8.8:2
Total number of translations: 2



All good!.

I hope you have had a chance to look at some of the fairly simple configuration snippets thats involved in these techniques and how they fit together in the overall scheme of things.

If you have any questions, please let me know!

Have fun with the lab!

(Configurations will be added shortly!)

GETVPN Example

A couple of weeks ago I had the good fortune of attending Jeremy Filliben’s CCDE Bootcamp.
It was a great experience, which I will elaborate on in another post. But one of the technology areas I had a bit of difficult with, was GETVPN.

So in this post a I am going to setup a scenario in which a customer has 3 sites, 2 “normal” sites and a Datacenter site. The customer wants to encrypt traffic from Site 1 to Site 2.

Currently the customer has a regular L3VPN service from a provider (which is beyond the scope of this post). There is full connectivity between the 3 sites through this service.

The topology is as follows:

Topology

GETVPN consists of a few components, namely the Key Server (KS) and Group Members (GM’s), which is where it derives its name: Group Encrypted Transport. A single SA (Security Association) is used for the encryption. The Key Server distributes the information to the Group Members through a secure transport, where the Group Members then use this information (basically an ACL) to encrypt/decrypt the data packets.

The routing for the topology is fairly simple. (See Routing Diagram) Each client as well as the KeyServer uses a default route to reach the rest of the topology. Each CE router runs eBGP with the provider, where it redistributes the conntected interfaces into BGP for full reachability between the sites.

Routing-Topology

At this point, lets verify that we have full connectivty through the L3VPN SP.

On CE-1:

CE1#sh ip bgp
BGP table version is 7, local router ID is 192.168.12.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>  10.10.1.0/24     0.0.0.0                  0         32768 ?
 *>  10.10.2.0/24     10.10.1.2                              0 100 100 ?
 *>  10.10.3.0/24     10.10.1.2                              0 100 100 ?
 *>  192.168.12.0     0.0.0.0                  0         32768 ?
 *>  192.168.23.0     10.10.1.2                              0 100 100 ?
 *>  192.168.34.0     10.10.1.2                              0 100 100 ?

We are learning the routes to the other sites.

And connectivity from Client-1:

Client-1#ping 192.168.34.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.34.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/10/25 ms

The interesting part takes place on the KeyServer along wi th CE1 and CE3.

If we take a look at the configuration on the KeyServer.

First off, we have a regular extended ACL that defines what traffic we want to encrypt. This ACL is the one that gets “downloaded” to CE1 and CE3:

ip access-list extended CRYPTO_ACL
 permit ip 192.168.12.0 0.0.0.255 192.168.34.0 0.0.0.255
 permit ip 192.168.34.0 0.0.0.255 192.168.12.0 0.0.0.255
 

 

Register-ACL-Download

Next up we have an ISAKMP policy which is used during the information communication with the KeyServer. This policy is present on all the Group Members (GM’s) and the KeyServer:

crypto isakmp policy 10
 encr aes 256
 hash sha256
 authentication pre-share
 group 2
crypto isakmp key SUPERSECRET address 0.0.0.0        

In this example we use a simple Pre Shared Key with the Any address form. This can (and probably should) be either certificate based. However, this complicates matters, so i skipped that.

Next is the transform set for IPsec which will be used. Notice that we use tunnel mode.

crypto ipsec transform-set GET-VPN-TRANSFORM-SET esp-aes esp-sha256-hmac 
 mode tunnel

This transform set is being referenced in a IPsec profile configuration:

crypto ipsec profile GETVPN-PROFILE
 set transform-set GET-VPN-TRANSFORM-SET 

This is nesecary in order for the next configuration, which is the entire GDOI aspect:

crypto gdoi group GDOI-GROUP
 identity number 100
 server local
  rekey authentication mypubkey rsa GETVPN-KEY
  rekey transport unicast
  sa ipsec 1
   profile GETVPN-PROFILE
   match address ipv4 CRYPTO_ACL
   replay counter window-size 64
   no tag
  address ipv4 192.168.23.1

Here we are creating a GDOI configuration, where we have a unique identifier for this group configuration (100). We are telling the router that its the server. Next is the public key we have created with an name this time (“crypto key generate rsa label “). This is used for rekeying purposes. And notice that we are using unicasting for the key material. This could just as well have been multicast, but again, that requires you have your infrastructure multicast capable and ready.

We then reference our previous IPsec profile and specify our crypt “ACL”. Lastly we specify which “update source” should be used for this server (which the other GM’s will use to communicate to/from).

If we then match this to what is configured on CE1 and CE3:

crypto isakmp policy 10
 encr aes 256
 hash sha256
 authentication pre-share
 group 2
crypto isakmp key SUPERSECRET address 0.0.0.0        
crypto gdoi group GDOI-GROUP
 identity number 100
 server address ipv4 192.168.23.1
crypto map MYMAP 10 gdoi 
 set group GDOI-GROUP
 crypto map MYMAP

And on the interface towards the SP we apply the crypto map:

CE1#sh run int g1.10
Building configuration...

Current configuration : 115 bytes
!
interface GigabitEthernet1.10
 encapsulation dot1Q 10
 ip address 10.10.1.1 255.255.255.0
 crypto map MYMAP
end

 

Crypto Map Topology

We can see that we have the ISAKMP configuration which I mentioned thats being used for a secure communication channel. Next we simply have the location of our KeyServer and the Identifier and thats pretty much all. Everything else is being learned from the Key Server.

After everything has been configured, you can see the log showing the registration process:

*May 15 10:37:53.245: %CRYPTO-5-GM_REGSTER: Start registration to KS 192.168.23.1 for group GDOI-GROUP using address 10.10.3.1 fvrf default ivrf default
*May 15 10:38:23.356: %GDOI-5-SA_TEK_UPDATED: SA TEK was updated
*May 15 10:38:23.395: %GDOI-5-SA_KEK_UPDATED: SA KEK was updated 0x5DB57E80F97A9A1DC16B9DBBCF7CB169
*May 15 10:38:23.395: %GDOI-5-GM_REGS_COMPL: Registration to KS 192.168.23.1 complete for group GDOI-GROUP using address 10.10.3.1 fvrf default ivrf default
*May 15 10:38:23.668: %GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 192.168.23.1 for group GDOI-GROUP & gm identity 10.10.3.1 fvrf default ivrf default

Another form of verification is the “show crypto gdoi” command structure, which gives you alot of information on the process:

CE1#sh crypto gdoi 
GROUP INFORMATION

    Group Name               : GDOI-GROUP
    Group Identity           : 100
    Group Type               : GDOI (ISAKMP)
    Crypto Path              : ipv4
    Key Management Path      : ipv4
    Rekeys received          : 0
    IPSec SA Direction       : Both

     Group Server list       : 192.168.23.1
                               
Group Member Information For Group GDOI-GROUP:
    IPSec SA Direction       : Both
    ACL Received From KS     : gdoi_group_GDOI-GROUP_temp_acl

    Group member             : 10.10.1.1       vrf: None
       Local addr/port       : 10.10.1.1/848
       Remote addr/port      : 192.168.23.1/848
       fvrf/ivrf             : None/None
       Version               : 1.0.16
       Registration status   : Registered
       Registered with       : 192.168.23.1
       Re-registers in       : 1580 sec
       Succeeded registration: 1
       Attempted registration: 3
       Last rekey from       : 0.0.0.0
       Last rekey seq num    : 0
       Unicast rekey received: 0
       Rekey ACKs sent       : 0
       Rekey Received        : never
       DP Error Monitoring   : OFF
       IPSEC init reg executed    : 0
       IPSEC init reg postponed   : 0
       Active TEK Number     : 1
       SA Track (OID/status) : disabled

       allowable rekey cipher: any
       allowable rekey hash  : any
       allowable transformtag: any ESP

    Rekeys cumulative
       Total received        : 0
       After latest register : 0
       Rekey Acks sents      : 0

 ACL Downloaded From KS 192.168.23.1:
   access-list   permit ip 192.168.12.0 0.0.0.255 192.168.34.0 0.0.0.255
   access-list   permit ip 192.168.34.0 0.0.0.255 192.168.12.0 0.0.0.255

KEK POLICY:
    Rekey Transport Type     : Unicast
    Lifetime (secs)          : 84613
    Encrypt Algorithm        : 3DES
    Key Size                 : 192     
    Sig Hash Algorithm       : HMAC_AUTH_SHA
    Sig Key Length (bits)    : 2352    

TEK POLICY for the current KS-Policy ACEs Downloaded:
  GigabitEthernet1.10:
    IPsec SA:
        spi: 0xA3D6592E(2748733742)
        KGS: Disabled
        transform: esp-aes esp-sha256-hmac 
        sa timing:remaining key lifetime (sec): (1815)
        Anti-Replay(Counter Based) : 64
        tag method : disabled
        alg key size: 16 (bytes)
        sig key size: 32 (bytes)
        encaps: ENCAPS_TUNNEL

Among the most interesting is the KEK policy and the ACL thats in place.

If we then verify from Client-1, we can see that we have a couple of seconds timeout while the encryption is being setup, and from there we have connectivity:

Client-1#ping 192.168.34.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.34.1, timeout is 2 seconds:
..!!!
Success rate is 60 percent (3/5), round-trip min/avg/max = 2/2/2 ms

So from something thats in theory very complex, this is very efficient from a both a configuration as well as a control-plane point of view. I know it certainly helped me understand the steps involved in setting GETVPN to create this lab, so I hope its been relevant for you as well!

Passed the CCDE written. Now what?

I was fortunate enough to finally pass the CCDE written exam yesterday morning.

That begs the question of “Now What?”

Well, I will spend a couple of days putting together a study strategy, based on where I am now compared to where I need to be in order to pass the exam. As it looks now, I am probably going for a fall 2016 exam date. That gives me enough time to settle into a new job with everything that entails.

It also means that I will need to spend 2-3 hours of study per day (some weekends more than that), with a combination of watching Cisco Live 365 videos and reading CVD’s/Books.

On top of that, my good friend Daniel Dib and I, along with hopefully a few others will have some design discussions using Webex. We have been told its really important to iron out different design ideas with other people. Especially if we can get a group together with people from different areas of expertise (Datacenter, Service Provider, Enterprise etc.).

Alas, an update to this story will come shortly! 🙂

Take care!

MPLS VPN’s over mGRE

This blog post outlines what “MPLS VPNs over mGRE” is all about as well as provide an example of such a configuration.

So what is “MPLS VPNs over mGRE”? – Well, basically its taking regular MPLS VPN’s and using it over an IP only core network. Since VPN’s over MPLS is one of the primary drivers for implementing an MPLS network in the first place, using the same functionality over an IP-only core might be very compelling for some not willing/able to run MPLS label switching in the core.

Instead of using labels to switch the traffic from one PE to another, mGRE (Multipoint GRE) is used as the encapsulation technology instead.

Be advised that 1 label is still being used however. This is the VPN label that’s used to identify which VRF interface to switch the traffic to when its received by a PE. This label is, just as in regular MPLS VPN’s, assigned by the PE through MP-BGP.

So how is this actually performed? – Well, lets take a look at an example.

The topology I will be using is as follows:

Topology for MPLS VPN's over mGRE

Topology for MPLS VPN’s over mGRE

** Note: I ran into an issue with VIRL, causing my CSR-3 to R3 to fail when establishing EIGRP adjacency. So i will not be using this in the examples to come. I noted this behavior on the VIRL community forums in case you are interested.

In this topology we have a core network, consisting of CSR-1 to CSR-5. They are all running OSPF in area 0. No MPLS is configured, so its pure IP routing end-to-end.

Lets take a look at CSR-5’s RIB:

CSR-5#sh ip route | beg Gateway
Gateway of last resort is not set

      1.0.0.0/32 is subnetted, 1 subnets
O        1.1.1.1 [110/2] via 192.168.15.1, 00:39:00, GigabitEthernet2
      2.0.0.0/32 is subnetted, 1 subnets
O        2.2.2.2 [110/2] via 192.168.25.2, 00:38:50, GigabitEthernet3
      3.0.0.0/32 is subnetted, 1 subnets
O        3.3.3.3 [110/2] via 192.168.35.3, 00:38:50, GigabitEthernet4
      4.0.0.0/32 is subnetted, 1 subnets
O        4.4.4.4 [110/2] via 192.168.45.4, 00:39:10, GigabitEthernet5
      5.0.0.0/32 is subnetted, 1 subnets
C        5.5.5.5 is directly connected, Loopback0
      192.168.15.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.15.0/24 is directly connected, GigabitEthernet2
L        192.168.15.5/32 is directly connected, GigabitEthernet2
      192.168.25.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.25.0/24 is directly connected, GigabitEthernet3
L        192.168.25.5/32 is directly connected, GigabitEthernet3
      192.168.35.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.35.0/24 is directly connected, GigabitEthernet4
L        192.168.35.5/32 is directly connected, GigabitEthernet4
      192.168.45.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.45.0/24 is directly connected, GigabitEthernet5
L        192.168.45.5/32 is directly connected, GigabitEthernet5

And to verify that we are not running any MPLS switching:

CSR-5#sh mpls for
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
Label      Label      or Tunnel Id     Switched      interface

So we have our connected interfaces along with the loopbacks of all the routers in the core network.

Lets take a look at CSR-1’s configuration, with regards to its VRF configuration in particular:

vrf definition CUST-A
 rd 100:100
 !
 address-family ipv4
  route-target export 100:100
  route-target import 100:100
 exit-address-family
!
!
interface GigabitEthernet2
 vrf forwarding CUST-A
 ip address 10.0.1.1 255.255.255.0
 negotiation auto
!
router eigrp 1
!
address-family ipv4 vrf CUST-A autonomous-system 100
  redistribute bgp 1 metric 1 1 1 1 1
  network 0.0.0.0
 exit-address-family

We have our VRF CUST-A configured, with a RD of 100:100 along with 100:100 as both import and export Route-Targets. Just as we would configure for a regular MPLS L3 VPN.

We use our GigabithEthernet2 interface as our attachment circuit to our CUST-A. In addition we have EIGRP 100 running as the VRF aware IGP towards R1. And finally we are redistributing BGP into the VRF.

Lets make sure we are receiving routes from R1 into the VRF RIB:

CSR-1#sh ip route vrf CUST-A eigrp | beg Gateway
Gateway of last resort is not set

      100.0.0.0/32 is subnetted, 3 subnets
D        100.100.100.1 [90/130816] via 10.0.1.100, 00:45:37, GigabitEthernet2

Looks good, we are receiving the loopback prefix from R1. This is as we would expect.

A similar configuration exists on CSR-2, CSR-3 and CSR-4. Nothing different from a regular MPLS L3 VPN service.

Now for the core configuration utilizing MP-BGP.
We are using CSR-5 as a VPN-v4 route-reflector in order to avoid having a full mesh of iBGP sessions.

So the configuration on R5 looks like this:

CSR-5#sh run | sec router bgp
router bgp 1
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 1.1.1.1 remote-as 1
 neighbor 1.1.1.1 update-source Loopback0
 neighbor 2.2.2.2 remote-as 1
 neighbor 2.2.2.2 update-source Loopback0
 neighbor 3.3.3.3 remote-as 1
 neighbor 3.3.3.3 update-source Loopback0
 neighbor 4.4.4.4 remote-as 1
 neighbor 4.4.4.4 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 1.1.1.1 activate
  neighbor 1.1.1.1 send-community extended
  neighbor 1.1.1.1 route-reflector-client
  neighbor 2.2.2.2 activate
  neighbor 2.2.2.2 send-community extended
  neighbor 2.2.2.2 route-reflector-client
  neighbor 3.3.3.3 activate
  neighbor 3.3.3.3 send-community extended
  neighbor 3.3.3.3 route-reflector-client
  neighbor 4.4.4.4 activate
  neighbor 4.4.4.4 send-community extended
  neighbor 4.4.4.4 route-reflector-client
 exit-address-family

Pretty straightforward really.

Then on CSR-1:

 CSR-1#sh run | sec router bgp
 router bgp 1
  bgp log-neighbor-changes
  no bgp default ipv4-unicast
  neighbor 5.5.5.5 remote-as 1
  neighbor 5.5.5.5 update-source Loopback0
  !
  address-family ipv4
  exit-address-family
  !
  address-family vpnv4
   neighbor 5.5.5.5 activate
   neighbor 5.5.5.5 send-community extended
   neighbor 5.5.5.5 route-map MODIFY-INBOUND in
  exit-address-family
  !
  address-family ipv4 vrf CUST-A
   redistribute eigrp 100
  exit-address-family

Here we have a single neighbor configured (R5 being the RR) using our loopback address. We are also redistributing routes from the VRF into BGP for VPNv4 announcements to the other PE’s. Whats really important (and differs from regular MPLS L3 VPN’s) is the route-map we apply inbound (MODIFY-INBOUND). Lets take a closer look at that:

CSR-1#sh route-map
route-map MODIFY-INBOUND, permit, sequence 10
  Match clauses:
  Set clauses:
    ip next-hop encapsulate l3vpn L3VPN-PROFILE
  Policy routing matches: 0 packets, 0 bytes

So all this does is set the next-hop according to a l3vpn profile called L3VPN-PROFILE. Now this is really the heart of the technology. Lets look at the profile in more detail:

CSR-1#sh run | beg L3VPN
l3vpn encapsulation ip L3VPN-PROFILE
 !

Well, that wasnt very informative. It simply defines a standard profile (which means mGRE) with our desired name.
You can get more detail by using the show commands:

CSR-1#sh l3vpn encapsulation ip 

 Profile: L3VPN-PROFILE
  transport ipv4 source Auto: Loopback0
  protocol gre
  payload mpls
   mtu default
  Tunnel Tunnel0 Created [OK]
  Tunnel Linestate [OK]
  Tunnel Transport Source (Auto) Loopback0 [OK]

So this tells us, that by default Loopback0 was chosen as the source of the tunnel and that Tunnel0 was created automatically. So lets take a look at the Tunnel0 in more detail:

 CSR-1#sh interface Tunnel0
 Tunnel0 is up, line protocol is up
   Hardware is Tunnel
   Interface is unnumbered. Using address of Loopback0 (1.1.1.1)
   MTU 9976 bytes, BW 10000 Kbit/sec, DLY 50000 usec,
      reliability 255/255, txload 1/255, rxload 1/255
   Encapsulation TUNNEL, loopback not set
   Keepalive not set
   Tunnel linestate evaluation up
   Tunnel source 1.1.1.1 (Loopback0)
    Tunnel Subblocks:
       src-track:
          Tunnel0 source tracking subblock associated with Loopback0
           Set of tunnels with source Loopback0, 1 member (includes iterators), on interface <OK>
   Tunnel protocol/transport multi-GRE/IP
     Key disabled, sequencing disabled
     Checksumming of packets disabled
   Tunnel TTL 255, Fast tunneling enabled
   Tunnel transport MTU 1476 bytes
   Tunnel transmit bandwidth 8000 (kbps)
   Tunnel receive bandwidth 8000 (kbps)
   Last input never, output never, output hang never
   Last clearing of "show interface" counters 00:54:16
   Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 3
   Queueing strategy: fifo
   Output queue: 0/0 (size/max)
   5 minute input rate 0 bits/sec, 0 packets/sec
   5 minute output rate 0 bits/sec, 0 packets/sec
      0 packets input, 0 bytes, 0 no buffer
      Received 0 broadcasts (0 IP multicasts)
      0 runts, 0 giants, 0 throttles
      0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
      0 packets output, 0 bytes, 0 underruns
      0 output errors, 0 collisions, 0 interface resets
      0 unknown protocol drops
      0 output buffer failures, 0 output buffers swapped out

Whats important here is that the Tunnel protocol/transport is multi-GRE/IP, which is the whole point of it all.

So to recap, when we receive prefixes reflected by our RR (this is besides the point, it could just as well be a full mesh), we set our IP Next-Hop to the other PE’s loopback address and tell the router to do the mGRE encapsulation when traffic is to be routed to these prefixes.

Lets take a look at our BGP table on CSR-1:

CSR-1#sh bgp vpnv4 uni vrf CUST-A | beg Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 100:100 (default for vrf CUST-A)
 *>  10.0.1.0/24      0.0.0.0                  0         32768 ?
 *>i 10.0.2.0/24      2.2.2.2                  0    100      0 ?
 *>i 10.0.3.0/24      3.3.3.3                  0    100      0 ?
 *>i 10.0.4.0/24      4.4.4.4                  0    100      0 ?
 *>  100.100.100.1/32 10.0.1.100          130816         32768 ?
 *>i 100.100.100.2/32 2.2.2.2             130816    100      0 ?
 *>i 100.100.100.4/32 4.4.4.4             130816    100      0 ?

(**Note: Remember CSR-3 is broken because of VIRL)

Lets take a look at what information is present for 100.100.100.2/32:

 CSR-1#sh bgp vpnv4 uni vrf CUST-A 100.100.100.2/32
 BGP routing table entry for 100:100:100.100.100.2/32, version 19
 Paths: (1 available, best #1, table CUST-A)
   Not advertised to any peer
   Refresh Epoch 1
   Local
     2.2.2.2 (metric 3) (via default) (via Tunnel0) from 5.5.5.5 (5.5.5.5)
       Origin incomplete, metric 130816, localpref 100, valid, internal, best
       Extended Community: RT:100:100 Cost:pre-bestpath:128:130816
         0x8800:32768:0 0x8801:100:128256 0x8802:65281:2560 0x8803:65281:1500
         0x8806:0:1684300802
       Originator: 2.2.2.2, Cluster list: 5.5.5.5
       mpls labels in/out nolabel/17
       rx pathid: 0, tx pathid: 0x0

Important to note here is that we are being told to use label nr. 17 as the VPN label for this prefix when sending it to 2.2.2.2 (CSR-2).

And finally lets take a look at what CEF thinks about it all:

CSR-1#sh ip cef vrf CUST-A 100.100.100.2 detail
100.100.100.2/32, epoch 0, flags [rib defined all labels]
  nexthop 2.2.2.2 Tunnel0 label 17

So CEF will assign label 17 to the packet and then use Tunnel0 to reach CSR-2. Just as we would expect.

As a final verification ive done an Embedded Packet Capture on CSR-5 while doing a ping from R1’s loopback to R2’s loopback and this is what you can see here:

 6  142   29.028990   1.1.1.1          ->  2.2.2.2          GRE
  0000:  FA163E39 EC39FA16 3ECEC705 08004500   ..>9.9..>.....E.
  0010:  00800000 0000FF2F B5490101 01010202   ......./.I......
  0020:  02020000 88470001 11FE4500 00640000   .....G....E..d..
  0030:  0000FE01 2BCD6464 64016464 64020800   ....+.ddd.ddd...

   7  142   29.106989   1.1.1.1          ->  2.2.2.2          GRE
  0000:  FA163E39 EC39FA16 3ECEC705 08004500   ..>9.9..>.....E.
  0010:  00800001 0000FF2F B5480101 01010202   ......./.H......
  0020:  02020000 88470001 11FE4500 00640001   .....G....E..d..
  0030:  0000FE01 2BCC6464 64016464 64020800   ....+.ddd.ddd...

   8  142   29.184988   1.1.1.1          ->  2.2.2.2          GRE
  0000:  FA163E39 EC39FA16 3ECEC705 08004500   ..>9.9..>.....E.
  0010:  00800002 0000FF2F B5470101 01010202   ......./.G......
  0020:  02020000 88470001 11FE4500 00640002   .....G....E..d..
  0030:  0000FE01 2BCB6464 64016464 64020800   ....+.ddd.ddd...

   9  142   29.241037   1.1.1.1          ->  2.2.2.2          GRE
  0000:  FA163E39 EC39FA16 3ECEC705 08004500   ..>9.9..>.....E.
  0010:  00800003 0000FF2F B5460101 01010202   ......./.F......
  0020:  02020000 88470001 11FE4500 00640003   .....G....E..d..
  0030:  0000FE01 2BCA6464 64016464 64020800   ....+.ddd.ddd...

  10  142   29.287024   1.1.1.1          ->  2.2.2.2          GRE
  0000:  FA163E39 EC39FA16 3ECEC705 08004500   ..>9.9..>.....E.
  0010:  00800004 0000FF2F B5450101 01010202   ......./.E......
  0020:  02020000 88470001 11FE4500 00640004   .....G....E..d..
  0030:  0000FE01 2BC96464 64016464 64020800   ....+.ddd.ddd...

As you can see, the encapsulation is GRE, just as expected.

So thats all there is to this technology. Very useful if you have an IP-only core network.

I hope its been useful and i will soon attach all the configurations for the routers in case you want to take a closer look.

Thanks for reading!

Update: Link to configs here

Overwhelmed – CCDE

I am preparing for the CCDE written exam, which I have coming up at CLUS.

There are so much material to go through, both to read as well as watch CL presentations, it can be a bit overwhelming at times.

So if you are thinking about going down this path, learn how to speed-read 🙂

Just under a month until we leave for the US. We are looking very much forward to the break!

Just a quick thought for the day!

Cisco Live US 2015 – Session Schedule

This is my current schedule for the Cisco Live US 2015 event.

Most are related to my CCDE studies and a few are with technologies and products that im interested in in general.

Monday:
08:00 – 09:30 BRKSAN-2101 FCoE for small and mid size enterprises.
10:00 – 12:00 BRKCRS-2031 Enterprise Campus Design: Multilayer Architectures and Design Principles
13:00 – 15:00 BRKARC-2001 Cisco ASR1000 Series routers: System and Solution Architectures
15:30 – 17:00 GENKEY-1001 Cisco Vision Keynote

Tuesday:
08:00 – 09:30 BRKRST-2124 Introduction to Segment Routing
13:00 – 15:00 BRKSPG-2210 Designing Service Provider Access Networks
15:30 – 17:00 BRKDCT-2049 Overlay Transport Virtualization

Wednesday:
09:00 – 12:00 CCDE Written Exam
13:00 – 15:00 BRKRST-3363 Routed Fast Convergence
15:30 – 17:00 BRKRST-2338 ISIS Deployment in Modern Networks

Thursday:
08:00 – 09:30 BRKMPL-2333 E-VPN & PBB-EVPN: the Next Generation of MPLS-based L2VPN
10:00 – 12:00 BRKRST-2311 IPv6 Planning, Deployment and Troubleshooting
13:00 – 14:30 BRKRST-2044 Enterprise Multihomed Internet Edge Architectures
15:00 – 16:00 GENKEY-1004 Guest Closing Keynote: Mike Rowe

Cant wait 🙂

Upcoming webinar with Daniel Dib

Just wanted to let you know that Daniel from lostintransit.se is doing a webinar on network design. I will be attending and helping out any way i can.

Go here to learn more: https://learningnetwork.cisco.com/blogs/community_cafe/2015/01/21/network-design-fundamentals-webinar-with-ciscovip-daniel-dib