Showing posts with label VPN. Show all posts
Showing posts with label VPN. Show all posts

Thursday, February 6, 2014

NTS: L3VPN Redistribution

L3VPN Redistribution




Configuration Steps

  • Configure the VRFs
  • Configure the RDs
  • Configure the import/export RTs
  • Assign the PE=>CE interfaces to VRFs
  • Configure IGP/BGP between PE-CE
  • Configure MP-BGP between PEs
  • Mutually redistribute between MP-BGP and the PE-CE IGP



BGP<=>RIP

RIP=>BGP
RIP metric => BGP MED (auto)

RIP=>BGP=>RIP
RIP metric => BGP MED => RIP metric (auto)

OTHER=>BGP=>RIP
BGP X => RIP metric (manual)

If "auto" doesn't work (for whatever reason), you can trying clearing the vrf routing table on the PE or you can use the following to set manually the RIP metric:
  • redistribute bgp 100 metric transparent
  • redistribute bgp 100 metric X
  • redistribute bgp 100 route-map X
Clearing of vrf routing table might be needed every time a new prefix is redistributed.

If version 2 is to be used, then it must be defined under the ipv4 vrf address-family on the PE.

RIP metric = hops (0-16)

Configuration

IOS
router rip
 address-family ipv4 vrf VPN
  redistribute bgp 200
!
router bgp 100
 address-family ipv4 vrf VPN

  redistribute rip


IOS-XR
router rip
 vrf VPN
  redistribute bgp 200
!

router bgp 200
 vrf VPN
  address-family ipv4 unicast
   redistribute rip





BGP<=>EIGRP

EIGRP=>BGP
EIGRP composite metric => BGP MED (auto)
EIGRP vector metrics => BGP Extended Cost Community (auto)

EIGRP=>BGP=>EIGRP
EIGRP composite metric => BGP MED => EIGRP composite metric (auto)
EIGRP vector metrics => BGP Extended Cost Community => EIGRP vector metrics (auto)
original internal EIGRP routes appear as internal EIGRP routes when redistributed
original external EIGRP routes appear as external EIGRP routes when redistributed

OTHER=>BGP=>EIGRP
BGP X => EIGRP metrics (manual)
original routes appear as external EIGRP routes when redistributed

If "auto" doesn't work (for whatever reason), you can trying clearing the vrf routing table on the PE or you can use the following to set manually the EIGRP metrics:
  • redistribute bgp 100 metric K1 K2 K3 K4 K5
  • redistribute bgp 100 route-map X
  • redistribute bgp 100 route-policy X
  • redistribute bgp 100 & default-metric K1 K2 K3 K4 K5
Clearing of vrf routing table might be needed every time a new prefix is redistributed.

EIGRP vector metrics = K1 K2 K3 K4 K5 (i.e. 1000 10 255 1 1500)

Configuration 

IOS
router eigrp 100
 address-family ipv4 vrf VPN autonomous-system 1
  redistribute bgp 200
 exit-address-family

!
router bgp 200
 address-family ipv4 vrf VPN

  redistribute eigrp 1


IOS-XR
router eigrp 100
 vrf VPN
  address-family ipv4
   autonomous-system 1
   redistribute bgp 200
!

router bgp 200
 vrf VPN
  address-family ipv4 unicast
   redistribute eigrp 1



Redistribution of EIGRP into the BGP vrf requires the EIGRP autonomous-system number to be redistributed. Some software releases may accept the global EIGRP process too.

You can use the SoO extended community to prevent any possible loops.



BGP<=>ISIS

ISIS=>BGP
ISIS metric => BGP MED (auto)

ISIS=>BGP=>ISIS
ISIS metric => BGP MED => ISIS metric (auto)

OTHER=>BGP=>ISIS
BGP X => ISIS metric (manual)

You can use the following to set manually the ISIS metric:
  • redistribute bgp 100 metric X
  • redistribute bgp 100 route-map X
Clearing of vrf routing table might be needed every time a new prefix is redistributed.

ISIS metric = hops (10)

Configuration

IOS
router isis 100
 address-family ipv4 vrf VPN
  redistribute bgp 200
!
router bgp 200
 address-family ipv4 vrf VPN

  redistribute isis 100


IOS-XR
router isis 100
 vrf VPN
  redistribute bgp 200
!

router bgp 200
 vrf VPN
  address-family ipv4 unicast
   redistribute isis 100


Redistribution doesn't take into account the IS-IS connected routes. You have to explicitly define them.

In order to void a possible loop while doing redistribution (when L1 is involved), you can change the distance of the ISIS advertised routes (excluding connected) on the PE to be higher than BGP's.

IOS
router isis 100
 vrf VPN
 distance 201 0.0.0.0 255.255.255.255 ISIS-NOT-CONNECTED-ACL




BGP<=>OSPF

OSPF=>BGP
OSPF metric => BGP MED + 1 (auto)
OSPF Area/LSA => BGP extended community "OSPF RT" (auto)

OSPF=>BGP=>OSPF
OSPF metric => BGP MED + 1 => OSPF metric (auto)
  • original intra-area routes appear as inter-area routes when redistributed (if same OSPF Domain-ID)
  • original intra-area routes appear as external-2 routes when redistributed (if different OSPF Domain-ID)
  • type-4 LSAs are not redistributed into BGP
  • original external routes appear as external-2 routes when redistributed (requires "match external" in redistribution from OSPF to BGP)

OTHER=>BGP=>OSPF
BGP X => OSPF metric (manual)

You can always use the following to manually set the OSPF metric:
  • redistribute bgp 200 metric X
  • redistribute bgp 200 route-map X
Clearing of vrf routing table might be needed every time a new prefix is redistributed.

OSPF metric = interface cost (0-65535)


"OSPF RT" Extended Community

"OSPF RT" format is "Area:LSA-Type:External-Type"

LSA Type to OSPF RT conversion
  • Type-1/2 => RT 2
  • Type-3 => RT 3
  • Type-5 => RT 5
  • Type-7 => RT 7
  • Sham-links => RT 129
Examples
  • OSPF RT:0.0.0.0:2:0
    • area 0.0.0.0
    • LSA-Type 1/2
  • OSPF RT:0.0.0.0:5:0
    • LSA-Type 5
    • External 1
  • OSPF RT:0.0.0.0:5:1
    • LSA-Type 5
    • External 2


Configuration

IOS
router ospf 100 vrf VPN
 redistribute bgp 200 subnets
!
router bgp 200
 address-family ipv4 vrf VPN

  redistribute ospf 100 vrf VPN

IOS-XR
router ospf 100
 vrf VPN

  redistribute bgp 200
!
router bgp 200
 vrf VPN
  address-family ipv4 unicast
   redistribute ospf 100



In IOS, if you don't include the vrf name in the redistribution of OSPF into BGP, it gets automatically added to the configuration.



The DN Bit and the VPN Route Tag

For a PE it is necessary to know if a particular prefix has been learned from another PE router, in order to avoid re-advertisement of it into BGP and cause a loop.

Two mechanisms are mainly used for loop prevention when OSPF is used as PE-CE protocol.
  • the DN bit
  • the VPN Route (or OSPF Domain) tag
By default, when a type 3, 5, 7 LSA is sent from a PE to a CE, the DN bit is set by the PE.

When another PE receives from a CE router, a type 3, 5, 7 LSA with the DN bit set, the prefix information from that LSA is not used during the OSPF route calculation, which means that the prefix doesn't get installed into the PE's BGP table.

Almost all Cisco software releases support the setting of DN bit only for Type-3 LSAs and they use a 32-bit VPN Route tag for Type-5/7 LSAs. The configuration and inclusion of the VPN Route Tag is required by all implementations for backward compatibility with older implementations that do not set the DN bit in type 5/7 LSAs.

If a PE router receives an LSA that contains the same VPN Route Tag as the locally configured tag, then the local PE router knows that another PE router (from the same domain)  generated this route and the LSA is ignored.

  • 16bit ASNs
    • VPN Route tag Format: 1101 000000000000 ASN_of_VPN_Backbone
  • 32bit ASNs
    • VPN Route tag must be defined manually

You can change this default value by using the "domain-tag" command within the OSPF VRF process configuration.

IOS
router ospf 100 vrf VPN
 domain-tag 12345
 

IOS-XR
router ospf 100
 vrf TEST
  domain-tag 12345



In case of Multi-VRF (VRF-Lite), the router that is accepting the LSA with the DN bit is actually a CE router with no BGP VPNv4 functionality, so there is no danger of redistributing this prefix into BGP. In order to bypass this DN bit check, the following configuration can be enabled.

IOS
router ospf 100 vrf VPN
 capability vrf-lite

IOS-XR
router ospf 100
 vrf VPN
  disable-dn-bit-check



Verification

IOS
R1#sh ip ospf 100 database summary 10.7.7.7

            OSPF Router with ID (10.1.3.1) (Process ID 100)

                Summary Net Link States (Area 0)

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 1196
  Options: (No TOS-capability, DC, Downward)
  LS Type: Summary Links(Network)
  Link State ID: 10.7.7.7 (summary Network Number)
  Advertising Router: 10.1.2.2
  LS Seq Number: 80000005
  Checksum: 0x2761
  Length: 28
  Network Mask: /32
        MTID: 0         Metric: 2



R1#sh ip ospf 100 database external 7.7.7.7

            OSPF Router with ID (10.1.3.1) (Process ID 100)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 1302
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 10.1.2.2
  LS Seq Number: 80000004
  Checksum: 0x6DCF
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 3489661028



Links



OSPF Domain-ID

OSPF Domain-ID is an attribute that defines how (internal, external) the OSPF routes will be transferred from one CE to another CE over their PEs BGP VPNv4 session.

On a PE, if the OSPF Domain-ID of the received BGP prefixes (encoded as extended community) is the same as the OSPF Domain-ID of the local OSPF process, then:
  • the MPLS core is treated like a SuperBackbone area (which is considered higher than area 0)
  • the PE is treated like an ABR (instead of an ASBR)
  • internal routes are being redistributed as Type-3 LSAs (instead of Type-5)

IOS-XR uses a null Domain-ID by default, so this needs to be changed if the other PE is running IOS (which is encoding the OSPF process-id as domain-id). OSPF Domain-ID needs to be changed on the PEs (where redistribution between BGP and OSPF takes place), not on the CEs.

The "type" value can be different is some cases for backwards compatibility (like in 0005 vs 8005).

Detailed Steps
OSPF=>BGP redistribution on PE1
  • if the OSPF Domain tag of the local OSPF process is the same as the VPN Route tag of the prefix, then that route isn't installed into BGP
  • if the OSPF DN bit check is enabled in the local OSPF process and the OSPF route has this bit set, then that route isn't installed into BGP
  • if the route is installed into BGP
    • the Domain-ID of the local OSPF process is encoded into OSPF DOMAIN ID community on the prefix
    • the area and the LSA type of the OSPF prefix is encoded into OSPF RT community on the prefix
    • the Router-ID of the local OSPF process is encoded into OSPF ROUTER ID community on the prefix
BGP=>OSPF redistribution on PE2
  • if the Domain-ID of the local OSPF process is the same as the OSPF DOMAIN ID community of the prefix, then that route is passed to the CE as internal else as external

Configuration

IOS
router ospf 100 vrf VPN
 domain-id type 0005 value 000000440101

IOS-XR
router ospf 100
 vrf VPN
  domain-id type 0005 value
000000440101


Verification

You can use "sh ip ospf" to see the Domain-ID of the local OSPF process.

You can use "sh bgp vpn4 unicast" to see the Domain-ID encoded as extended community in the BGP prefixes (OSPF RT is included too).

R2#sh ip ospf 100
 Routing Process "ospf 100" with ID 10.1.2.2
   Domain ID type 0x0005, value 0x
000000440101
 Start time: 00:13:37.092, Time elapsed: 00:36:17.144
 Supports only single TOS(TOS0) routes
 Supports opaque LSA
 Supports Link-local Signaling (LLS)
 Supports area transit capability
 Connected to MPLS VPN Superbackbone, VRF VPN
 Event-log disabled
 It is an area border and autonomous system boundary router
 Redistributing External Routes from,
    bgp 100, includes subnets in redistribution




R2#sh bgp vpnv4 unicast vrf VPN 1.1.1.1/32
BGP routing table entry for 100:1:1.1.1.1/32, version 2
Paths: (1 available, best #1, table VPN)
  Advertised to update-groups:
     1
  Local
    10.1.2.1 from 0.0.0.0 (2.2.2.2)
      Origin incomplete, metric 2, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:100:1 OSPF DOMAIN ID:0x0005:0x
000000440101
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.2.2:0
      mpls labels in/out 28/nolabel



R2#sh ip ospf 100 database

            OSPF Router with ID (10.1.2.2) (Process ID 100)
...
                Summary Net Link States (Area 0)

Link ID         ADV Router      Age         Seq#       Checksum
1.1.1.1         10.1.1.1        980         0x80000002 0x00F336
...



LSA Type-3 (Summary) in local OSPF table is encoded as "OSPF RT:0.0.0.0:3:0" in local BGP table.


Propagation of OSPF routes between CE1 and CE2
  • same domain-id
    • CE1 O => CE2 IA
  • different domain-id
    • CE1 O => CE2 E2
  • sham-link (regardless of domain-id)
    • CE1 O => CE2 O

Extra care needs to be taken if route tags are changed manually on OSPF=>BGP redistribution, because external OSPF routes are tagged by the BGP ASN when BGP=>OSPF redistribution takes place, which means that the original tag is lost (which could lead to a loop)

IOS
R6#sh ip route 1.1.1.1
Routing entry for 1.1.1.1/32
  Known via "ospf 100", distance 110, metric 2
  Tag Complete, Path Length == 1, AS 100, , type extern 2, forward metric 1
  Last update from 10.10.10.5 on POS4/0, 00:00:03 ago
  Routing Descriptor Blocks:
  * 10.10.10.5, from 10.10.10.5, 00:00:03 ago, via POS4/0
      Route metric is 2, traffic share count is 1
      Route tag 3489661028





If a BGP VPNv4 route is redistributed into OSPF, then redistributed into another IGP like RIP (where all the information (DN bit, VPN Route-Tag) needed to prevent looping is lost), and then redistributed back into OSPF, then it is possible that it could be redistributed back into BGP as a VPNv4 route, thereby causing a loop.

You can use route tags at every step of redistribution in order to avoid possible routing loops, either caused by the above scenario or by mutual redistribution in two places.



NTS: CsC

CsC




CsC (Carrier supporting Carrier) is defined in RFC 4364.



Control-Plane
  • The Customer Carrier PEs run BGP VPNv4 in order to exchange VPN labels
  • The Customer Carrier routers run IGP+LDP (or iBGP+Label) in order to exchange all their internal BGP next-hops and their labels
  • The CsC-PEs and CsC-CEs run eBGP (or IGP) in order to exchange BGP next-hop prefixes
  • The CsC-PEs and CsC-CEs run eBGP+Label (or IGP+LDP) in order to exchange labels for the BGP next-hop prefixes
  • The Backbone Carrier routers run IGP+LDP in order to exchange all their internal BGP next-hops and their labels

The Backbone Carrier offers a MPLS VPN service to the Customer Carrier which in turn offers a MPLS VPN or Internet service to its customers.

The Backbone Carrier doesn't need to know the final customer prefixes.

Using IGP+LDP in CsC is not as risky as with Inter-AS MPLS VPN Option 3 because:
  • Customer Carrier internal routes are put into a specific VRF in the Backbone Carrier
  • No Backbone Carrier internal routes are distributed into the Customer Carrier network

You can have multiple Backbone Carriers, using Inter-AS MPLS L3VPN for interconnection.

By default a CsC-PE runs PHP towards the CsC-CE. If using an ipv4-labeled PE-CE session, you can change this behavior (in order to keep the QoS consistent across providers) by using the "neighbor x.x.x.x send-label explicit-null" on the CsC-CE.

IOS-XR supports only the use of Labeled BGP as a PE-CE protocol in CsC topologies. LDP (+IGP) is not supported.



CsC Load Balancing

Load balancing between CsC-PE and CsC-CE can be achieved with:
  • directly connected loopback peering for one pair of PE/CE
    • one eBGP session between neighbors
    • multiple static routes for each other's loopback
    • mpls forwarding on all directly connected physical interfaces
  • eBGP multipath for multiple pairs of PEs/CEs
    • maximum-paths under bgp & vrf address family on PE
    • maximum-paths under bgp on CE

When using static routes, you also need to define the outgoing interface and the next-hop.

"mpls bgp forwarding" is not automatically added, because the BGP session is not between directly connected neighbors. You have to add it yourself.



Configuration


BC = Backbone Carrier (AS10)
CC = Customer Carrier (AS100)
C = Customer (AS200)

Backbone Carrier runs IS-IS or OSPF with MPLS/LDP in its core

Backbone Carrier (CsC-PE1) runs OSPF+LDP with Customer Carrier (CsC-CE1)
Backbone Carrier (CsC-PE2) runs eBGP+Label with Customer Carrier (CsC-CE2)

Customer Site 1 (C-CE1) runs OSPF with Customer Carrier (CC-PE1)
Customer Site 2 (C-CE2) runs ISIS with Customer Carrier (CC-PE2)

CC-PE (Customer Carrier PE serving the final customer site) and CsC-CE (Carrier supporting Carrier CE) functionalities can be collapsed into a single router.

CsC-PE1 and CsC-PE2 run iBGP VPNv4 in order to exchange Customer Carrier prefixes/labels
CsC-CE1 and CsC-CE2 run iBGP VPNv4 in order to exchange Customer prefixes/labels



IGP+LDP between CsC-PE1 and CsC-CE1


CsC-PE1 (IOS)
! for connectivity to BC core (IGP+LDP)
mpls ldp router-id Loopback0
mpls label protocol ldp
!
interface Ethernet0/2
 description ** Link to BC core **
 ip address x.x.x.x

 mpls ip
!
router isis/ospf x
!
! for connectivity to CsC-CE1 (OSPF+LDP)
vrf definition CC-VPN
 rd 10:X

 route-target 10:X
!
 address-family ipv4

 exit-address-family
!
interface Ethernet1/0

 description ** Link to CsC-CE1 **
 vrf forwarding CC-VPN
 ip address x.x.x.x
 mpls ip
!
router ospf 10 vrf CC-VPN
 redistribute bgp 10 subnets
 network x.x.x.x area 0
!

! for connectivity to BC-PE2 (iBGP VPNv4)
router bgp 10

 no bgp default ipv4-unicast
 neighbor BC-PE2 remote-as 10
 neighbor BC-PE2 update-source Loopback0
 !
 address-family vpnv4
  neighbor BC-PE2 activate
  neighbor BC-PE2 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf CC-VPN
  redistribute ospf 10 vrf CC-VPN
 exit-address-family




CsC-CE1 (IOS)
! for connectivity to CsC-PE1 (OSPF+LDP)
mpls ldp router-id Loopback0
mpls label protocol ldp
!
interface Ethernet1/0
 description ** Link to CsC-PE1 **
 ip address x.x.x.x
 mpls ip
!

router ospf 10
 network x.x.x.x area 0
!


CC-PE1 (IOS)
! for connectivity to C-CE1 (OSPF+VRF)  
vrf definition C-VPN
 rd 100:Y
 route-target 100:Y
 !
 address-family ipv4
 exit-address-family
!

interface Ethernet1/3
 description ** Link to C-CE1 **
 vrf forwarding C-VPN
 ip address y.y.y.y
!

router ospf 200 vrf C-VPN
 redistribute bgp 100 subnets
 network y.y.y.y area 0
!

! for connectivity to CC-PE2 (iBGP VPNv4)  
router bgp 100
 no bgp default ipv4-unicast
 neighbor CC-PE2 remote-as 100
 neighbor
CC-PE2 update-source Loopback0
 !
 address-family vpnv4
  neighbor
CC-PE2 activate
  neighbor
CC-PE2 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf C-VPN
  redistribute ospf 200 vrf C-VPN
 exit-address-family
!



BGP+Label between CsC-PE2 and CsC-CE2


CsC-PE2 (IOS-XR)
! for connectivity to BC core (IGP+LDP)
mpls ldp router-id Loopback0
mpls label protocol ldp
!
router isis/ospf x
!

mpls ldp
 router-id x.x.x.x
 interface x
!

! for connectivity to CsC-CE2  (eBGP+Label)
vrf CC-VPN
 address-family ipv4 unicast
  import route-target
   10:X
  export route-target
   10:X
!
interface GigabitEthernet0/2/1/1
 description ** Link to CsC-CE2 **
 vrf CC-VPN
 ipv4 address x.x.x.x
!

router static
 vrf CC-VPN
  address-family ipv4 unicast
   CsC-CE2/32 GigabitEthernet0/2/1/1

!
router bgp 10
 address-family ipv4 unicast
 !

 vrf CC-VPN
  rd 10:X
  address-family ipv4 unicast
   network x.x.x.x
   allocate-label all
  !
  neighbor CsC-CE2
   remote-as 100
   address-family ipv4 unicast
    route-policy PASS-RPL in
    route-policy PASS-RPL out
    as-override
    send-extended-community-ebgp
   !
   address-family ipv4 labeled-unicast
    route-policy PASS-RPL in
    route-policy PASS-RPL out
    as-override
    send-extended-community-ebgp
!

route-policy PASS-RPL 
  pass
end-policy

!
! for connectivity to BC-PE1 (iBGP VPNv4)  
router bgp 10
 address-family vpnv4 unicast
 !
 neighbor BC-PE1
  remote-as 10
  update-source Loopback0
  address-family vpnv4 unicast



CsC-CE2 (IOS)
! for connectivity to CsC-PE2  (eBGP+Label)
interface Ethernet1/0
 description ** Link to CsC-PE2 **
 ip address x.x.x.x
 mpls bgp forwarding
!

router bgp 100
 no bgp default ipv4-unicast
 neighbor CsC-PE2 remote-as 10
 !
 address-family ipv4
  neighbor CsC-PE2 activate
  neighbor CsC-PE2 send-label
 exit-address-family

!
! for connectivity to C-CE2 (ISIS+VRF)
vrf definition C-VPN
 rd 100:Y

 route-target 100:Y
 !
 address-family ipv4
 exit-address-family
!

interface Ethernet1/3
 description ** Link to C-CE2 **
 vrf forwarding C-VPN
 ip address y.y.y.y
 ip router isis 200
!
router isis 200
 vrf C-VPN
  redistribute bgp 100
!

! for connectivity to CC-PE1 (iBGP VPNv4)  
router bgp 100
 neighbor CC-PE1 remote-as 100
 neighbor CC-PE1 update-source Loopback0
 !
 address-family vpnv4
  neighbor CC-PE1 activate
  neighbor CC-PE1 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf C-VPN
  redistribute isis 200

 exit-address-family


IOS-XR configuration is similar to IOS, with the major difference of using the labeled unicast address-family instead of the send-label keyword.

Don't forget to create a /32 static route for the CsC-PE/CE next-hop in IOS-XR when using eBGP+Label. Always verify the installation of labels for /32 next-hops.



Verification
  • Customer Carrier PEs must have a BGP VPNv4 route and a label for the VPN prefix
  • Customer Carrier routers must have a label for the VPN prefix's next-hop
  • CsC-PEs must have a BGP VPNv4 route and a label for the VPN prefix's next-hop
  • Backbone Carrier routers must have a label for the next-hop of VPN prefix's next-hop



    Example

    Assume the following network:

    R1-R2-R3-R4-R5-R6-R7-R8-R9-R10

    where

    Customer Carrier Network
    Backbone Carrier Network


    Then the following would happen for a VPN packet originating at R1 and terminating at R10.
    • R1 (1.1.1.1) (Customer Carrier PE router) - vrf VPN
      • Transport label is 18, VPN label is 20
      • next-hop is R10 (10.10.10.10)
    • R2 (2.2.2.2) (Customer Carrier P router)
      • Transport label is 20, VPN label is 20
      • next-hop is R10 (10.10.10.10)
    • R3 (3.3.3.3) (CsC-CE)
      • Transport label is 26, VPN label is 20
      • next-hop is R10 (10.10.10.10)
    • R4 (4.4.4.4) (CsC-PE) - vrf CSC
      • Transport label is 16/21, VPN label is 20
      • next-hop is R7 (7.7.7.7)
    • R5 (5.5.5.5) Backbone Carrier P router)
      • Transport label is 16/21, VPN label is 20
      • next-hop is R7 (7.7.7.7)
    • R6 (6.6.6.6) (Backbone Carrier P router)
      • Transport label is 21, VPN label is 20
      • next-hop is R7 (7.7.7.7)
    • R7 (7.7.7.7) (CsC-PE) - vrf CSC
      • Transport label is 18, VPN label is 20
      • next-hop is R10 (10.10.10.10)
    • R8 (8.8.8.8) (CsC-CE)
      • Transport label is 17, VPN label is 20
      • next-hop is R10 (10.10.10.10)
    • R9 (9.9.9.9) (Customer Carrier P router)
      • Transport label is removed, VPN label is 20
      • next-hop is R10 (10.10.10.10)
    • R10 (10.10.10.10) (Customer Carrier PE router) - vrf VPN
      • VPN label is removed, destination reached in next-hop


    R1#trace vrf VPN 99.99.99.99

    Type escape sequence to abort.
    Tracing the route to 99.99.99.99

      1 20.1.2.2 [MPLS: Labels 18/20 Exp 0] 10 msec 10 msec 8 msec
      2 20.2.3.3 [MPLS: Labels 20/20 Exp 0] 7 msec 7 msec 7 msec

      3 20.3.4.4 [MPLS: Labels 26/20 Exp 0] 8 msec 8 msec 7 msec
      4 20.4.5.5 [MPLS: Labels 16/21/20 Exp 0] 7 msec 7 msec 8 msec
      5 20.5.6.6 [MPLS: Labels 16/21/20 Exp 0] 6 msec 5 msec 5 msec
      6 20.6.7.7 [MPLS: Labels 21/20 Exp 0] 3 msec 3 msec 3 msec

      7 20.7.8.8 [MPLS: Labels 18/20 Exp 0] 1 msec 1 msec 1 msec
      8 20.8.9.9 [MPLS: Labels 17/20 Exp 0] 1 msec 1 msec 1 msec
      9 20.9.10.10 [MPLS: Label 20 Exp 0] 1 msec 1 msec 1 msec

     10 30.10.10.99 1 msec 2 msec 2 msec



    Verification in every hop


    Customer Carrier PE router
    R1#sh ip route vrf VPN 99.99.99.99

    Routing Table: VPN
    Routing entry for
    99.99.99.99/32
      Known via "bgp 100", distance 200, metric 1, type internal
      Redistributing via rip
      Advertised by rip metric transparent
      Last update from 10.10.10.10 00:56:37 ago
      Routing Descriptor Blocks:
      *
    10.10.10.10 (default), from 10.10.10.10, 00:56:37 ago
          Route metric is 1, traffic share count is 1
          AS Hops 0
          MPLS label: 20
          MPLS Flags: MPLS Required


    R1#sh bgp vpnv4 unicast vrf VPN 99.99.99.99/32
    BGP routing table entry for 100:1:
    99.99.99.99/32, version 11
    Paths: (1 available, best #1, table VPN)
      Not advertised to any peer
      Local
       
    10.10.10.10 (metric 20) from 10.10.10.10 (10.10.10.10)
          Origin incomplete, metric 1, localpref 100, valid, internal, best
          Extended Community: RT:100:1
          mpls labels in/out nolabel/20


    VPN label is 20

    R1#sh mpls forwarding-table 10.10.10.10 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    23         18        
    10.10.10.10/32   0             Fa0/0.12   20.1.2.2
            MAC/Encaps=18/22, MRU=1500, Label Stack{18}
            CA0113DC0000CA03079400088100026C8847 00012000
            No output feature configured


    R1#sh ip cef vrf VPN 99.99.99.99 det99.99.99.99/32, epoch 0, flags rib defined all labels
      recursive via
    10.10.10.10 label 20
        nexthop 20.1.2.2 FastEthernet0/0.12 label 18


    Transport label is 18, VPN label is 20


    From here you have 2 options:
    • follow the next-hops
    • follow the labels (i prefer this one)



    Following the next-hops



    Customer Carrier P router
    R2#sh mpls forwarding-table 10.10.10.10 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    18         20        
    10.10.10.10/32   13707         Fa0/0.23   20.2.3.3
            MAC/Encaps=18/22, MRU=1500, Label Stack{20}
            CA0610240000CA0113DC00008100002E8847 00014000
            No output feature configured


    Transport label is 20, VPN label is 20

    CsC-CE
    R3#sh mpls forwarding-table 10.10.10.10 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    20         26        
    10.10.10.10/32   15048         Fa0/0.34   20.3.4.4
            MAC/Encaps=18/22, MRU=1500, Label Stack{26}
            CA0207940008CA0610240000810001A38847 0001A000
            No output feature configured


    Transport label is 26, VPN label is 20

    CsC-PE
    R4#sh mpls forwarding-table vrf CSC 10.10.10.10 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    26         21        
    10.10.10.10/32[V]16033         Fa0/0.45   20.4.5.5
            MAC/Encaps=18/26, MRU=1496, Label Stack{16 21}
            C20911080000CA0207940008810003338847 0001000000015000
            VPN route: CSC
            No output feature configured


    2 Transport labels are used (use "detail" to see them)

    R4#sh bgp vpnv4 unicast vrf CSC 10.10.10.10
    BGP routing table entry for 200:1:
    10.10.10.10/32, version 12
    Paths: (1 available, best #1, table CSC)
      Advertised to update-groups:
         3
      100
        7.7.7.7 (metric 4) from 7.7.7.7 (7.7.7.7)
          Origin incomplete, metric 20, localpref 100, valid, internal, best
          Extended Community: RT:200:1
          mpls labels in/out 26/21


    VPN label (21) for Backbone Carrier is actually Transport label (21) for Customer Carrier

    R4#sh mpls forwarding-table 7.7.7.7 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    16         16         7.7.7.7/32       0             Fa0/0.45   20.4.5.5
            MAC/Encaps=18/22, MRU=1500, Label Stack{16}
            C20911080000CA0207940008810003338847 00010000
            No output feature configured


    Transport label is 16/21, VPN label is 20

    Backbone Carrier P router
    R5#sh mpls forwarding-table 7.7.7.7 detail
    Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
    tag    tag or VC   or Tunnel Id      switched   interface
    16     16          7.7.7.7/32        44218      Fa0/0.56   20.5.6.6
            MAC/Encaps=18/22, MRU=1500, Tag Stack{16}
            C20811080000C209110800008100004E8847 00010000
            No output feature configured
        Per-packet load-sharing


    Transport label is 16/21, VPN label is 20

    Backbone Carrier P router
    R6#sh mpls forwarding-table 7.7.7.7 detail
    Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
    tag    tag or VC   or Tunnel Id      switched   interface
    16     Pop tag     7.7.7.7/32        42398      Fa0/0.67   20.6.7.7
            MAC/Encaps=18/18, MRU=1504, Tag Stack{}
            CA0415180000C20811080000810000118847
            No output feature configured
        Per-packet load-sharing


    Transport label is 21, VPN label is 20

    CsC-PE
    R7#sh mpls forwarding-table vrf CSC 10.10.10.10 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    21         18        
    10.10.10.10/32[V]24156         Fa0/0.78   20.7.8.8
            MAC/Encaps=18/22, MRU=1500, Label Stack{18}
            CA0710240000CA04151800008100000D8847 00012000
            VPN route: CSC
            No output feature configured


    Transport label is 18, VPN label is 20

    CsC-CE
    R8#sh mpls forwarding-table 10.10.10.10 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    18         17        
    10.10.10.10/32   24703         Fa0/0.89   20.8.9.9
            MAC/Encaps=18/22, MRU=1500, Label Stack{17}
            CA0013DC0000CA0710240000810000238847 00011000
            No output feature configured


    Transport label is 17, VPN label is 20

    Customer Carrier P router
    R9#sh mpls forwarding-table 10.10.10.10 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    17         Pop Label 
    10.10.10.10/32   23892         Fa0/0.910  20.9.10.10
            MAC/Encaps=18/18, MRU=1504, Label Stack{}
            CA0515180000CA0013DC0000810000198847
            No output feature configured


    Transport label is removed, VPN label is 20

    Customer Carrier PE router
    R10#sh mpls forwarding-table vrf VPN 99.99.99.99 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    20         No Label  
    99.99.99.99/32[V]1770          Fa0/0.1010 30.10.10.99
            MAC/Encaps=18/18, MRU=1504, Label Stack{}
            C20A0F840000CA05151800008100001D0800
            VPN route: VPN
            No output feature configured


    VPN label is removed, destination reached



    Following the labels


    Customer Carrier P router
    R2#sh mpls forwarding-table labels 18 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    18         20        
    10.10.10.10/32   13852         Fa0/0.23   20.4.6.4
            MAC/Encaps=18/22, MRU=1500, Label Stack{20}
            CA0610240000CA0113DC00008100002E8847 00014000
            No output feature configured



    Transport label is 20, VPN label is 20

    CsC-CE
    R3#sh mpls forwarding-table labels 20 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    20         26        
    10.10.10.10/32   15338         Fa0/0.34   20.4.19.19
            MAC/Encaps=18/22, MRU=1500, Label Stack{26}
            CA0207940008CA0610240000810001A38847 0001A000
            No output feature configured


    Transport label is 26, VPN label is 20

    CsC-PE
    R4#sh mpls forwarding-table labels 26 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    26         21        
    10.10.10.10/32[V]16645         Fa0/0.45   20.4.5.5
            MAC/Encaps=18/26, MRU=1496, Label Stack{16 21}
            C20911080000CA0207940008810003338847 0001000000015000
            VPN route: CSC
            No output feature configured


    2 Transport labels are used (use "detail" to see them)

    Transport label is 16/21, VPN label is 20

    Backbone Carrier P router
    R5#sh mpls forwarding-table labels 16 detail
    Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
    tag    tag or VC   or Tunnel Id      switched   interface
    16     16          7.7.7.7/32        30421      Fa0/0.56   20.5.6.6
            MAC/Encaps=18/22, MRU=1500, Tag Stack{16}
            C20811080000C209110800008100004E8847 00010000
            No output feature configured
        Per-packet load-sharing


    Transport label is 16/21, VPN label is 20

    Backbone Carrier P router
    R6#sh mpls forwarding-table labels 16 detail
    Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
    tag    tag or VC   or Tunnel Id      switched   interface
    16     Pop tag     7.7.7.7/32        29337      Fa0/0.67   20.6.7.7
            MAC/Encaps=18/18, MRU=1504, Tag Stack{}
            CA0415180000C20811080000810000118847
            No output feature configured
        Per-packet load-sharing


    Transport label is 21, VPN label is 20

    CsC-PE
    R7#sh mpls forwarding-table labels 21 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    21         18        
    10.10.10.10/32[V]17260         Fa0/0.78   20.7.8.8
            MAC/Encaps=18/22, MRU=1500, Label Stack{18}
            CA0710240000CA04151800008100000D8847 00012000
            VPN route: CSC
            No output feature configured


    Transport label is 18, VPN label is 20

    CsC-CE
    R8#sh mpls forwarding-table labels 18 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    18         17        
    10.10.10.10/32   17726         Fa0/0.89   20.8.9.9
            MAC/Encaps=18/22, MRU=1500, Label Stack{17}
            CA0013DC0000CA0710240000810000238847 00011000
            No output feature configured


    Transport label is 17, VPN label is 20

    Customer Carrier P router
    R9#sh mpls forwarding-table labels 17 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    17         Pop Label 
    10.10.10.10/32   17200         Fa0/0.910  20.9.10.10
            MAC/Encaps=18/18, MRU=1504, Label Stack{}
            CA0515180000CA0013DC0000810000198847
            No output feature configured


    Transport label is removed, VPN label is 20

    Customer Carrier PE router
    R10#sh mpls forwarding-table labels 20 detail
    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
    Label      Label      or Tunnel Id     Switched      interface
    20         No Label  
    99.99.99.99/32[V]1770          Fa0/0.1010 30.10.10.99
            MAC/Encaps=18/18, MRU=1504, Label Stack{}
            C20A0F840000CA05151800008100001D0800
            VPN route: VPN
            No output feature configured


    VPN label is removed, destination reached