Encapsulated Remote SPAN (ERSPAN)

There are three types of Switch Port Analyser (SPAN) supported on Cisco routers and switches:

Local SPAN: Mirrors traffic from one or more interface on the switch to one or more interfaces on the same switch.

Remote SPAN (RSPAN): An extension of SPAN called remote SPAN or RSPAN which allows to capture traffic and send it to a remote switch via a Layer 2 network.

Encapsulated Remote SPAN (ERSPAN): as the name indicates, ERSPAN encapsulates capture traffic in GRE and allows it to be transported to a remote port across a Layer 3 network.

ERSPAN is a Cisco proprietary feature and is available only to Catalyst 6500, 7600, Nexus, and ASR 1000 platforms to date. The ASR 1000 supports ERSPAN source (monitoring) only on Fast Ethernet, Gigabit Ethernet, and port-channel interfaces.

Firstly we will go through a typical scenario, in which traffic on port Gi1.23 of Router R2 is captured, and sent to interface Gi2 of R1 traffic using ERSPAN.

Topology

Configuration

ERSPAN Source Router

CSR2# 

monitor session 10 type erspan-source
 source interface GigabitEthernet1
 filter vlan 23    ! Specify Sub-interface
 destination
 erspan-id 100
 ip address 1.1.1.1
 origin ip address 2.2.2.2
 no shutdown   !   Default is shutdown

ERSPAN Destination Router

CSR1#
 
monitor session 10 type erspan-destination
 destination interface GigabitEthernet2
 source
  erspan-id 100
  ip address 1.1.1.1
 no shutdown   !   Default is shutdown

Verification

CSR2#show monitor session all
Session 10
----------
Type                   : ERSPAN Source Session
Status                 : Admin Enabled
Source Ports           : 
    Both               : Gi1
Filter VLANs           : 23
Destination IP Address : 1.1.1.1
MTU                    : 1464
Destination ERSPAN ID  : 100
Origin IP Address      : 2.2.2.2


CSR1#show monitor session all
Session 10
----------
Type                   : ERSPAN Destination Session
Status                 : Admin Enabled
Destination Ports      : Gi2
Source IP Address      : 1.1.1.1
Source ERSPAN ID       : 100

CSR1#show interface gi2 stats 
GigabitEthernet2
          Switching path    Pkts In   Chars In   Pkts Out  Chars Out
               Processor          1         77          4        308
             Route cache          0          0          0          0
       Distributed cache         47       7745        112      31780
                   Total         48       7822        116      32088

Capture files:

ERSPAN transport traffic, encapsulated within a GRE tunnel, as seen on R1 interface Gi1.12
https://www.cloudshark.org/captures/19b7c6b1c70b

ERSPAN capture result – on R1 Gi2
https://www.cloudshark.org/captures/00c5e863ecfe

ERSPAN without the destination router

Since ERSPAN is a Cisco proprietary protocol supported on a limited number of platforms, one may ask if we can capture traffic and send it via GRE to a remote laptop without having a destination ERSPAN router. We might run into this situation if the destination router does not support ERSPAN, or the PC is not connected directly to a physical port of the ERSPAN destination router (e.g. via a LAN switch, or via an Wireless AP).

The answer is, yes we can do this without the ERSPAN destination router!

The ERSPAN destination router is not a critical requirement. GRE/ERSPAN protocol in this case is not equipped with a reliable transmission mechanism. Nor it has a mechanism to verify the status of the ERSPAN session destination, before sending the capture traffic. Unlike ERSPAN, in the general GRE tunnel interface configuration on IOS (similarly in Juniper JUNOS) we can enable the “keep-alive” function under the GRE tunnel interface, and routers at both ends of the tunnel can check the status of the other end, and can bring down the interface if the keep-alive is not received within a configurable period, and can bring the interface up once the keep-alive messages are received again. The ERSPAN implementation of GRE does not have this feature.

ERSPAN destination router is needed if we want the traffic send to the destination interface appear in the same format as the one captured from the source, i.e. without it being encapsulated in GRE/ERSPAN headers.

If the remote PC is reachable via an IP address (connected to the network via a LAN switch or Wireless AP) the ERSPAN source router can send encapsulated traffic directly to this IP address. The PC does not need to have an GRE tunnel nor ERSPAN session configured . PC can not run ERSPAN anyway, because it is a Cisco proprietary protocol, and there is no “ERSPAN client” software released for PC.

If the remote PC does not have a reachable IP address (quite typical usage case) but connect physically to a router interface, we can still force the ERSPAN traffic to the interface assigned to this PC, by implementing a static route and static ARP. In this example below, 1.1.1.1 is the destination ERSPAN address (configured on R2), and R1 does not support ERSPAN.

Configuration

ERSPAN Source router (2.2.2.2)
CRS2#
! Configuration is unchanged from the previous case
monitor session 10 type erspan-source
 source interface GigabitEthernet1
 filter vlan 23    ! Specify Sub-interface
 destination
 erspan-id 100
 ip address 1.1.1.1
 origin ip address 2.2.2.2
 no shutdown   !   Default is shutdown



CSR1#
! Destination router does not support ERSPAN. 
! The destination IP address 1.1.1.1 is now NOT a real IP "behind" the monitoring PC.

! Remove ERSPAN session
no monitor session 10

! Remove 1.1.1.1 from Loopback interface.
no interface Loopback0

! Configure a dummy "transit" IP address on Gi2 
! And add a static route for destination 1.1.1.1 to force ERSPAN traffic out this way.

interface GigabitEthernet2
 ip address 10.1.1.1 255.255.255.0
!
ip route 1.1.1.1 255.255.255.255 10.1.1.2 name Force_Traffic_Out_Gi2

! Note that we need static ARP for the dummy next hop IP. 
! Otherwise, router will keep ARPing, without sending the actual ERSPAN traffic out Gi2
 
arp 10.1.1.2 6400.f1e2.0112 ARPA

Capture file:

ERSPAN capture result – as seen on on R1 Gi2. Note that the captured traffic is now encapsulated within GRE/ERSPAN header, similar to the transit traffic captured on R1 Gi1.12 in the previous example.

https://www.cloudshark.org/captures/76ce4261df29

Local ERSPAN

In this example, we’d like to mirror traffic from interface Gi1 to Gi2 on a local router R1. As traffic is copied from one interface to other on the same router, we wont be able to capture the actual transport traffic encapsulated within GRE/ERSPAN.

Below is the configuration & verification steps.

CSR1#

monitor session 10 type erspan-source
 source interface GigabitEthernet1
 destination
 erspan-id 100
 ip address 1.1.1.1
 origin ip address 1.1.1.1
 shutdown   !   Default
 
monitor session 20 type erspan-destination
 destination interface GigabitEthernet2
 source
  erspan-id 100
  ip address 1.1.1.1
 shutdown   !   Default
 

CSR1#! Before turning montor session ON
CSR1#
CSR1#
CSR1#show interfaces gigabitEthernet 1 stats 
GigabitEthernet1
          Switching path    Pkts In   Chars In   Pkts Out  Chars Out
               Processor          0          0          0          0
             Route cache          0          0          0          0
       Distributed cache         17       1540         15       1268
                   Total         17       1540         15       1268
CSR1#show interfaces gigabitEthernet 2 stats 
GigabitEthernet2
          Switching path    Pkts In   Chars In   Pkts Out  Chars Out
               Processor          0          0          0          0
             Route cache          0          0          0          0
       Distributed cache          0          0          0          0
                   Total          0          0          0          0
                   
                   
Note that there is no traffic sent to Gi2  


CSR1#
config t
monitor session 10
 no shutdown
monitor session 20
 no shutdown


CSR1#! After turning montor session ON


CSR1#show interfaces gigabitEthernet 1 stats 
GigabitEthernet1
          Switching path    Pkts In   Chars In   Pkts Out  Chars Out
               Processor          0          0          0          0
             Route cache          0          0          0          0
       Distributed cache         48       4264         45       3782
                   Total         48       4264         45       3782
CSR1#show interfaces gigabitEthernet 2 stats 
GigabitEthernet2
          Switching path    Pkts In   Chars In   Pkts Out  Chars Out
               Processor          0          0          0          0
             Route cache          0          0          0          0
       Distributed cache          0          0         14       1192
                   Total          0          0         14       1192

References

Configuring ERSPAN

http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/lanswitch/configuration/xe-3s/lanswitch-xe-3s-book/lnsw-conf-erspan.html#GUID-152D9875-169B-461F-A34B-ABAABD0C1FF8

Understanding SPAN, RSPAN, and ERSPAN

https://supportforums.cisco.com/document/139236/understanding-spanrspanand-erspan

Inter-AS L3VPN Option B

In this post, we are going to go through the implementation and verification steps of Inter-AS Option B (defined in RFC4364 http://tools.ietf.org/html/rfc4364#page-32 ). Please refer to previous post Inter-AS Options to for the definitions, and differences of the three.

In option B implementation, ASBRs themselves are PE routers. They learn VPN-IPv4 routes from other PEs from the same AS via MP-iBGP (either directly or via Route Reflectors). The ASBR from one AS then have an eBGP session(s) with other AS ASBR to redistribute the VPN-IPv4 routes to the other AS. The later ASBR then redistributes those VPN-IPv4 labeled routes to the rest of the PEs belonging to that AS.

Topology

In this topology, we use Cisco IOS routers (R1 – R3) in AS100, as well as Juniper routers (R4 – R6) in AS200, just for demonstration of configuration and verification on both platforms. Typically for connectivity beween ASes, one eBGP session between ASBRs is sufficient. In this example, however, we set up two eBGP sessions between ASBR R3 and R4 to demonstrate VPN-based policy routing, by route filtering based on VPN route targets. We would like to route traffic for VPN1 via the first connection, and VPN2 via the other.

Configuration

R3-ASBR#

interface Loopback0
 description Management Loopback
 ip address 3.3.3.3 255.255.255.255
!
interface GigabitEthernet1.23
 description R3 -> R2-P
 encapsulation dot1Q 23
 ip address 20.2.3.3 255.255.255.0
 mpls ip
!
interface GigabitEthernet1.34
 description R3 -> R4 | First eBGP
 encapsulation dot1Q 34
 ip address 20.3.4.3 255.255.255.0
 mpls bgp forwarding
!
interface GigabitEthernet1.342
 description R3 -> R4 | Second eBGP
 encapsulation dot1Q 342
 ip address 20.3.42.3 255.255.255.0
 mpls bgp forwarding


! Enable OSPF
 
router ospf 1
 network 3.3.3.3 0.0.0.0 area 0
 network 20.2.3.3 0.0.0.0 area 0
!

! Enable MP-iBGP to other PE, and MP-eBGP to other ASBR
router bgp 100
 bgp log-neighbor-changes
 ! By default, PE do not accept VPNv4 routes 
 ! that do not belong to one of its VRF
 ! This command force it to learn & propagate all routes to other PEs.
 no bgp default route-target filter
 neighbor 1.1.1.1 remote-as 100
 neighbor 1.1.1.1 update-source Loopback0
 neighbor 20.3.4.4 remote-as 200
 neighbor 20.3.42.4 remote-as 200
 !
 address-family ipv4
  no neighbor 1.1.1.1 activate
  no neighbor 20.3.4.4 activate
  no neighbor 20.3.42.4 activate
 exit-address-family
 !
 address-family vpnv4
  neighbor 1.1.1.1 activate
  neighbor 1.1.1.1 send-community extended
  neighbor 1.1.1.1 next-hop-self
  neighbor 20.3.4.4 activate
  neighbor 20.3.4.4 send-community extended
  neighbor 20.3.42.4 activate
  neighbor 20.3.42.4 send-community extended
 exit-address-family
 

! Policy to only import and export VPN2 routes via the second eBGP connection
! We can apply a similar policy on the first eBGP connection on R3
! But for demonstration purpose, we choose to apply that policy on R4 (Junos)

ip extcommunity-list standard Community_VPN2 permit rt 100:2
ip extcommunity-list standard Community_VPN2 permit rt 200:2
!
route-map VPN2-EXPORT-ONLY permit 10
 match extcommunity Community_VPN2
!
route-map VPN2-EXPORT-ONLY deny 1000
 description Deny all else
!
route-map VPN2-IMPORT-ONLY permit 10
 match extcommunity Community_VPN2
 set local-preference 200
!
route-map VPN2-IMPORT-ONLY deny 1000
 description Deny all else
!
router bgp 100 
 address-family vpnv4
  neighbor 20.3.42.4 route-map VPN2-IMPORT-ONLY in
  neighbor 20.3.42.4 route-map VPN2-EXPORT-ONLY out
 exit-address-family 
 
  


R4-ASBR# 

interfaces {
    ge-0/0/1 {
        vlan-tagging;
        unit 34 {
            description "R4 -> R3 | First eBGP";
            vlan-id 34;
            family inet {
                address 20.3.4.4/24;
            }                           
            family mpls;
        }
        unit 45 {
            description "R4 -> R5";
            vlan-id 45;
            family inet {
                address 20.4.5.4/24;
            }
            family mpls;
        }
        unit 342 {
            description "R4 -> R3 | Second eBGP";
            vlan-id 342;
            family inet {
                address 20.3.42.4/24;
            }
            family mpls;
        }
    }
    lo0 {
        unit 0 {
            description "Management Loopback";
            family inet {
                address 4.4.4.4/32;     
            }
            family mpls;
        }
    }
}


routing-options {
    router-id 4.4.4.4;
    autonomous-system 200;
}
protocols {
    mpls {
        interface ge-0/0/1.45;
        interface ge-0/0/1.34;
        interface ge-0/0/1.342;
    }
    bgp {
        group IBGP {
            type internal;
            local-address 4.4.4.4;
            family inet-vpn {
                unicast;
            }
            export next-hop-self;
            neighbor 6.6.6.6;           
        }
        group EBGP1 {
            type external;
            description "L3VPN Inter-AS";
            import VPN1-IMPORT-ONLY;
            family inet-vpn {
                unicast;
            }
            export VPN1-EXPORT-ONLY;
            neighbor 20.3.4.3 {
                peer-as 100;
            }
        }
        group EBGP2 {
            type external;
            description "2nd L3VPN Inter-AS";
            family inet-vpn {
                unicast;
            }
            neighbor 20.3.42.3 {
                peer-as 100;
            }
        }
    }                                   
    ospf {
        area 0.0.0.0 {
            interface ge-0/0/1.45;
            interface lo0.0;
        }
    }
    ldp {
        interface ge-0/0/1.45;
        interface lo0.0;
    }
}



policy-options {
    policy-statement VPN1-EXPORT-ONLY {
        term VPN1 {
            from community [ Community_100_1 Community_200_1 ];
            then accept;
        }
        term Reject-All {
            then reject;
        }
    }
    policy-statement VPN1-IMPORT-ONLY {
        term VPN1 {
            from community Community_100_1;
            then {                      
                local-preference 200;   
                accept;                 
            }
        }
        term Reject-All {
            then reject;
        }
    }
    policy-statement next-hop-self {
        then {
            next-hop self;
        }
    }
    community Community_100_1 members target:100:1;
    community Community_100_2 members target:100:2;
    community Community_200_1 members target:200:1;
    community Community_200_2 members target:200:2;
}





/// Other router configuration

R1-PE#
!
hostname R1-PE
!
ip vrf VPN1
 rd 1.1.1.1:1
 route-target export 100:1
 route-target import 100:1
 route-target import 200:1
!
ip vrf VPN2
 rd 1.1.1.1:2
 route-target export 100:2
 route-target import 100:2
 route-target import 200:2

interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
interface Loopback1
 ip vrf forwarding VPN1
 ip address 10.1.1.1 255.255.255.0
!
interface Loopback2
 ip vrf forwarding VPN2
 ip address 10.1.2.1 255.255.255.0
!
interface GigabitEthernet1.12
 encapsulation dot1Q 12
 ip address 20.1.2.1 255.255.255.0
 mpls ip
!
router ospf 1
 network 1.1.1.1 0.0.0.0 area 0
 network 20.1.2.1 0.0.0.0 area 0
!
router bgp 100
 bgp log-neighbor-changes
 neighbor 3.3.3.3 remote-as 100
 neighbor 3.3.3.3 update-source Loopback0
 !
 address-family vpnv4
  neighbor 3.3.3.3 activate
  neighbor 3.3.3.3 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf VPN1
  redistribute connected
 exit-address-family
 !
 address-family ipv4 vrf VPN2
  redistribute connected
 exit-address-family
!


R2-P#

interface Loopback0
 ip address 2.2.2.2 255.255.255.255
!
interface GigabitEthernet1.12
 encapsulation dot1Q 12
 ip address 20.1.2.2 255.255.255.0
 mpls ip
!
interface GigabitEthernet1.23
 encapsulation dot1Q 23
 ip address 20.2.3.2 255.255.255.0
 mpls ip
!
router ospf 1
 network 2.2.2.2 0.0.0.0 area 0
 network 20.1.2.2 0.0.0.0 area 0
 network 20.2.3.2 0.0.0.0 area 0
 
 





lab@R5-P> show configuration

interfaces {
    ge-0/0/1 {
        vlan-tagging;
        unit 45 {
            vlan-id 45;
            family inet {
                address 20.4.5.5/24;
            }
            family mpls;                
        }
        unit 56 {
            vlan-id 56;
            family inet {
                address 20.5.6.5/24;
            }
            family mpls;
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 5.5.5.5/32;
            }
            family mpls;
        }
    }
}
routing-options {
    router-id 5.5.5.5;
    autonomous-system 200;
}
protocols {
    mpls {                              
        interface ge-0/0/1.45;
        interface ge-0/0/1.56;
    }
    ospf {
        area 0.0.0.0 {
            interface lo0.0;
            interface ge-0/0/1.45;
            interface ge-0/0/1.56;
        }
    }
    ldp {
        interface ge-0/0/1.45;
        interface ge-0/0/1.56;
        interface lo0.0;
    }
}



lab@R6-PE> show configuration 

interfaces {
    ge-0/0/1 {
        vlan-tagging;
        unit 56 {
            vlan-id 56;
            family inet {
                address 20.5.6.6/24;
            }
            family mpls;                
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 6.6.6.6/32;
            }
            family mpls;
        }
        unit 1 {
            family inet {
                address 10.6.1.1/24;
            }
        }
        unit 2 {
            family inet {
                address 10.6.2.1/24;
            }
        }
    }
}
routing-options {
    router-id 6.6.6.6;
    autonomous-system 200;              
}
protocols {
    mpls {
        interface ge-0/0/1.56;
    }
    bgp {
        group IBGP {
            type internal;
            local-address 6.6.6.6;
            family inet-vpn {
                unicast;
            }
            neighbor 4.4.4.4;
        }
    }
    ospf {
        area 0.0.0.0 {
            interface lo0.0;
            interface ge-0/0/1.56;
        }
    }
    ldp {
        interface ge-0/0/1.56;
        interface lo0.0;                
    }
}
policy-options {
    policy-statement VPN1-EXPORT {
        then {
            community add Community_200_1;
            accept;
        }
    }
    policy-statement VPN1-IMPORT {
        term Import {
            from community [ Community_200_1 Community_100_1 ];
            then accept;
        }
    }
    policy-statement VPN2-EXPORT {
        then {
            community add Community_200_2;
            accept;
        }
    }
    policy-statement VPN2-IMPORT {
        term Import {
            from community [ Community_200_2 Community_100_2 ];
            then accept;
        }
    }
    community Community_100_1 members target:100:1;
    community Community_100_2 members target:100:2;
    community Community_200_1 members target:200:1;
    community Community_200_2 members target:200:2;
}

routing-instances { 
    VPN1 {
        instance-type vrf;
        interface lo0.1;
        route-distinguisher 6.6.6.6:1;
        vrf-import VPN1-IMPORT;
        vrf-export VPN1-EXPORT;
    }
    VPN2 {
        instance-type vrf;
        interface lo0.2;
        route-distinguisher 6.6.6.6:2;
        vrf-import VPN2-IMPORT;
        vrf-export VPN2-EXPORT;
    }
}

Verification

R3-ASBR#show ip bgp vpnv4 all         
BGP table version is 31, local router ID is 3.3.3.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1.1.1.1:1
 *>i 10.1.1.0/24      1.1.1.1                  0    100      0 ?
Route Distinguisher: 1.1.1.1:2
 *>i 10.1.2.0/24      1.1.1.1                  0    100      0 ?
Route Distinguisher: 6.6.6.6:1
 *>  10.6.1.0/24      20.3.4.4                               0 200 i
 *>  10.6.1.1/32      20.3.4.4                               0 200 i
Route Distinguisher: 6.6.6.6:2
 *>  10.6.2.0/24      20.3.42.4                     200      0 200 i
 *>  10.6.2.1/32      20.3.42.4                     200      0 200 i
 
 
R3-ASBR#show mpls forwarding-table 
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         Pop Label  20.3.4.4/32      0             Gi1.34     20.3.4.4    
17         Pop Label  20.3.42.4/32     0             Gi1.342    20.3.42.4   
18         Pop Label  2.2.2.2/32       0             Gi1.23     20.2.3.2    
19         Pop Label  20.1.2.0/24      0             Gi1.23     20.2.3.2    
20         16         1.1.1.1/32       0             Gi1.23     20.2.3.2    
27         300976     6.6.6.6:1:10.6.1.0/24   \
                                       0             Gi1.34     20.3.4.4    
28         300992     6.6.6.6:1:10.6.1.1/32   \
                                       0             Gi1.34     20.3.4.4    
29         301008     6.6.6.6:2:10.6.2.0/24   \
                                       0             Gi1.342    20.3.42.4   
30         301024     6.6.6.6:2:10.6.2.1/32   \
                                       0             Gi1.342    20.3.42.4   
31         18         1.1.1.1:1:10.1.1.0/24   \
                                       0             Gi1.23     20.2.3.2    
32         19         1.1.1.1:2:10.1.2.0/24   \
                                       0             Gi1.23     20.2.3.2  



lab@R4-ASBR> show route                 

inet.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
...

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

5.5.5.5/32         *[LDP/9] 02:27:16, metric 1
                    > to 20.4.5.5 via ge-0/0/1.45
6.6.6.6/32         *[LDP/9] 02:27:16, metric 1
                    > to 20.4.5.5 via ge-0/0/1.45, Push 301040

mpls.0: 13 destinations, 13 routes (13 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 06:09:45, metric 1
                      Receive
1                  *[MPLS/0] 06:09:45, metric 1
                      Receive
2                  *[MPLS/0] 06:09:45, metric 1
                      Receive
13                 *[MPLS/0] 06:09:45, metric 1
                      Receive
300944             *[LDP/9] 02:27:16, metric 1
                    > to 20.4.5.5 via ge-0/0/1.45, Pop      
300944(S=0)        *[LDP/9] 02:27:16, metric 1
                    > to 20.4.5.5 via ge-0/0/1.45, Pop      
300960             *[LDP/9] 02:27:16, metric 1
                    > to 20.4.5.5 via ge-0/0/1.45, Swap 301040
300976             *[VPN/170] 02:27:08, metric2 1, from 6.6.6.6
                    > to 20.4.5.5 via ge-0/0/1.45, Swap 300080, Push 301040(top)
300992             *[VPN/170] 02:27:08, metric2 1, from 6.6.6.6
                    > to 20.4.5.5 via ge-0/0/1.45, Swap 300096, Push 301040(top)
301008             *[VPN/170] 02:27:08, metric2 1, from 6.6.6.6
                    > to 20.4.5.5 via ge-0/0/1.45, Swap 300112, Push 301040(top)
301024             *[VPN/170] 02:27:08, metric2 1, from 6.6.6.6
                    > to 20.4.5.5 via ge-0/0/1.45, Swap 300128, Push 301040(top)
301040             *[VPN/170] 02:26:38
                    > to 20.3.4.3 via ge-0/0/1.34, Swap 31
301056             *[VPN/170] 02:26:38
                    > to 20.3.42.3 via ge-0/0/1.342, Swap 32
                                        
bgp.l3vpn.0: 6 destinations, 7 routes (6 active, 0 holddown, 1 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1:1:10.1.1.0/24                
                   *[BGP/170] 02:26:38, localpref 200
                      AS path: 100 ?
                    > to 20.3.4.3 via ge-0/0/1.34, Push 31
1.1.1.1:2:10.1.2.0/24                
                   *[BGP/170] 02:26:38, localpref 100
                      AS path: 100 ?
                    > to 20.3.42.3 via ge-0/0/1.342, Push 32
6.6.6.6:1:10.6.1.0/24                
                   *[BGP/170] 02:27:08, localpref 100, from 6.6.6.6
                      AS path: I
                    > to 20.4.5.5 via ge-0/0/1.45, Push 300080, Push 301040(top)
6.6.6.6:1:10.6.1.1/32                
                   *[BGP/170] 02:27:08, localpref 100, from 6.6.6.6
                      AS path: I
                    > to 20.4.5.5 via ge-0/0/1.45, Push 300096, Push 301040(top)
6.6.6.6:2:10.6.2.0/24                
                   *[BGP/170] 02:27:08, localpref 100, from 6.6.6.6
                      AS path: I
                    > to 20.4.5.5 via ge-0/0/1.45, Push 300112, Push 301040(top)
6.6.6.6:2:10.6.2.1/32                   
                   *[BGP/170] 02:27:08, localpref 100, from 6.6.6.6
                      AS path: I
                    > to 20.4.5.5 via ge-0/0/1.45, Push 300128, Push 301040(top)
                    
                    




Verification on PE routers


R1-PE#sh ip route vrf VPN1

      10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
C        10.1.1.0/24 is directly connected, Loopback1
L        10.1.1.1/32 is directly connected, Loopback1
B        10.6.1.0/24 [200/0] via 3.3.3.3, 02:30:47
B        10.6.1.1/32 [200/0] via 3.3.3.3, 02:30:47


R1-PE#sh ip bgp vpnv4 vrf VPN1 10.6.1.0/24
BGP routing table entry for 1.1.1.1:1:10.6.1.0/24, version 20
Paths: (1 available, best #1, table VPN1)
  Not advertised to any peer
  Refresh Epoch 1
  200, imported path from 6.6.6.6:1:10.6.1.0/24 (global)
    3.3.3.3 (metric 3) from 3.3.3.3 (3.3.3.3)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:200:1
      mpls labels in/out nolabel/27
      rx pathid: 0, tx pathid: 0x0
      
lab@R6-PE> show route 

inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
...

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

4.4.4.4/32         *[LDP/9] 02:33:44, metric 1
                    > to 20.5.6.5 via ge-0/0/1.56, Push 301056
5.5.5.5/32         *[LDP/9] 02:33:46, metric 1
                    > to 20.5.6.5 via ge-0/0/1.56

VPN1.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.1.1.0/24        *[BGP/170] 02:33:06, localpref 200, from 4.4.4.4
                      AS path: 100 ?
                    > to 20.5.6.5 via ge-0/0/1.56, Push 301040, Push 301056(top)
10.6.1.0/24        *[Direct/0] 03:58:06
                    > via lo0.1
10.6.1.1/32        *[Local/0] 03:58:06
                      Local via lo0.1

VPN2.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
                                        
10.1.2.0/24        *[BGP/170] 02:33:06, localpref 100, from 4.4.4.4
                      AS path: 100 ?
                    > to 20.5.6.5 via ge-0/0/1.56, Push 301056, Push 301056(top)
10.6.2.0/24        *[Direct/0] 03:58:06
                    > via lo0.2
10.6.2.1/32        *[Local/0] 03:58:06
                      Local via lo0.2

mpls.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 04:35:27, metric 1
                      Receive
1                  *[MPLS/0] 04:35:27, metric 1
                      Receive
2                  *[MPLS/0] 04:35:27, metric 1
                      Receive
13                 *[MPLS/0] 04:35:27, metric 1
                      Receive
300048             *[LDP/9] 02:33:46, metric 1
                    > to 20.5.6.5 via ge-0/0/1.56, Pop      
300048(S=0)        *[LDP/9] 02:33:46, metric 1
                    > to 20.5.6.5 via ge-0/0/1.56, Pop      
300064             *[LDP/9] 02:33:44, metric 1
                    > to 20.5.6.5 via ge-0/0/1.56, Swap 301056
300080             *[VPN/170] 02:33:36
                      receive table VPN1.inet.0, Pop      
300096             *[VPN/170] 02:33:36
                      receive table VPN1.inet.0, Pop      
300112             *[VPN/170] 02:33:36
                      receive table VPN2.inet.0, Pop      
300128             *[VPN/170] 02:33:36
                      receive table VPN2.inet.0, Pop      

bgp.l3vpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1:1:10.1.1.0/24                
                   *[BGP/170] 02:33:06, localpref 200, from 4.4.4.4
                      AS path: 100 ?
                    > to 20.5.6.5 via ge-0/0/1.56, Push 301040, Push 301056(top)
1.1.1.1:2:10.1.2.0/24                
                   *[BGP/170] 02:33:06, localpref 100, from 4.4.4.4
                      AS path: 100 ?
                    > to 20.5.6.5 via ge-0/0/1.56, Push 301056, Push 301056(top)

Which BGP peer did initiate the TCP connection?

A quick BGP question: How to check which BGP peer did initiate the connection?

The answer is very simple! It is in the output of the command show bgp neighbor . We sometime overlook the info in the output of this command, because it contains so much information.

Here you go, lets have a close look at that output again. In Junos, this info shows up in the very first line!

Junos

lab@PE1> show bgp neighbor 192.168.1.2 
Peer: 192.168.1.2+62512 AS 65512 Local: 192.168.1.1+179 AS 65512
  Type: Internal    State: Established    Flags: 
  Last State: OpenConfirm   Last Event: RecvKeepAlive
  Last Error: None
  Options: 
  Local Address: 192.168.1.1 Holdtime: 90 Preference: 170
  Number of flaps: 0
  Peer ID: 192.168.1.2     Local ID: 192.168.1.1       Active Holdtime: 90
  Keepalive Interval: 30         Peer index: 0   
  BFD: disabled, down
  NLRI for restart configured on peer: inet-unicast
  NLRI advertised by peer: inet-unicast
  NLRI for this session: inet-unicast
  Peer supports Refresh capability (2)
  Restart time configured on the peer: 120
  Stale routes from peer are kept for: 300
  Restart time requested by this peer: 120
  NLRI that peer supports restart for: inet-unicast
  NLRI that restart is negotiated for: inet-unicast
  NLRI of received end-of-rib markers: inet-unicast
  NLRI of all end-of-rib markers sent: inet-unicast
  Peer supports 4 byte AS extension (peer-as 65512)
  Peer does not support Addpath
  Table inet.0 Bit: 10001
    RIB State: BGP restart is complete
    Send state: in sync
    Active prefixes:              0
    Received prefixes:            0
    Accepted prefixes:            0
    Suppressed due to damping:    0
    Advertised prefixes:          0
  Last traffic (seconds): Received 19   Sent 16   Checked 57  
  Input messages:  Total 11     Updates 1       Refreshes 0     Octets 253
  Output messages: Total 11     Updates 0       Refreshes 0     Octets 272
  Output Queue[0]: 0

The output shows that this router is a server (with the TCP port 179), and the other end is the one that initiated the TCP connection (source port 62512, which happens to be its AS number).

In Cisco IOS, this info shows in the middle:

Cisco IOS

R1#sh ip bgp neighbors 12.0.0.2
BGP neighbor is 12.0.0.2,  remote AS 200, external link
  BGP version 4, remote router ID 12.0.0.2
  BGP state = Established, up for 00:00:11
  Last read 00:00:11, last write 00:00:11, hold time is 180, keepalive interval is 60 seconds
  Neighbor sessions:
    1 active, is multisession capable
  Neighbor capabilities:
    Route refresh: advertised and received(new)
    Address family IPv4 Unicast: advertised and received
    Multisession Capability: advertised and received
  Message statistics, state Established:
    InQ depth is 0
    OutQ depth is 0
    
                         Sent       Rcvd
    Opens:                  1          1
    Notifications:          0          0
    Updates:                0          0
    Keepalives:             1          1
    Route Refresh:          0          0
    Total:                  2          2
  Default minimum time between advertisement runs is 30 seconds
          
 For address family: IPv4 Unicast
  Session: 12.0.0.2 session 1
  BGP table version 1, neighbor version 0/0
  Output queue size : 0
  Index 1, Offset 0, Mask 0x2
  1 update-group member
                                 Sent       Rcvd
  Prefix activity:               ----       ----
    Prefixes Current:               0          0
    Prefixes Total:                 0          0
    Implicit Withdraw:              0          0
    Explicit Withdraw:              0          0
    Used as bestpath:             n/a          0
    Used as multipath:            n/a          0

                                   Outbound    Inbound
  Local Policy Denied Prefixes:    --------    -------
    Total:                                0          0
  Number of NLRIs in the update sent: max 0, min 0

  Address tracking is enabled, the RIB does have a route to 12.0.0.2
  Connections established 1; dropped 0
  Last reset never
  Transport(tcp) path-mtu-discovery is enabled
  Graceful-Restart is disabled
Connection state is ESTAB, I/O status: 1, unread input bytes: 0        
Connection is ECN Disabled
Mininum incoming TTL 0, Outgoing TTL 1
Local host: 12.0.0.1, Local port: 179
Foreign host: 12.0.0.2, Foreign port: 54889
Connection tableid (VRF): 0

Enqueued packets for retransmit: 0, input: 0  mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x63FB4):
Timer          Starts    Wakeups            Next
Retrans             2          0             0x0
TimeWait            0          0             0x0
AckHold             2          1             0x0
SendWnd             0          0             0x0
KeepAlive           0          0             0x0
GiveUp              0          0             0x0
PmtuAger            0          0             0x0
DeadWait            0          0             0x0
Linger              0          0             0x0

iss: 1400246787  snduna: 1400246857  sndnxt: 1400246857     sndwnd:  16315
irs: 2844375924  rcvnxt: 2844375994  rcvwnd:      16315  delrcvwnd:     69
          
SRTT: 70 ms, RTTO: 1683 ms, RTV: 1613 ms, KRTT: 0 ms
minRTT: 24 ms, maxRTT: 300 ms, ACK hold: 200 ms
Status Flags: passive open, gen tcbs
Option Flags: nagle, path mtu capable

Datagrams (max data segment is 1460 bytes):
Rcvd: 4 (out of order: 0), with data: 2, total data bytes: 69
Sent: 4 (retransmit: 0 fastretransmit: 0),with data: 2, total data bytes: 69

We can filter irrelevant info with the following “pipe” option:

R1#sh ip bgp neighbors 12.0.0.2 | in port:
Local host: 12.0.0.1, Local port: 179
Foreign host: 12.0.0.2, Foreign port: 54889 

In this example, the Cisco router works as the server, listening to TCP port 179, while its peer initiated the connection from the source port 54889.

VRRP

Task

Configure basic VRRP functionality.

Topology

Configuration

Configuration on Junos3 router:

lab@Junos3# show                      
unit 0 {
    family inet {
        address 23.0.0.3/24 {
            vrrp-group 23 {
                virtual-address 23.0.0.23;
                priority 150;
                accept-data;
            }
        }
    }
}

Configuration on a Cisco router R2:

R2#
!
interface Vlan23
 ip address 23.0.0.2 255.255.255.0
 vrrp 23 ip 23.0.0.23
 ! vrrp 23 priority 100   ! Default Priority, not shown in the config
end

Verification

Verification on the Juniper router:

lab@Junos3> show vrrp summary 
Interface     State       Group   VR state       VR Mode    Type   Address 
ge-1/0/1.0    up             23   master          Active    lcl    23.0.0.3           
                                                            vip    23.0.0.23          

lab@Junos3> show vrrp detail 
Physical interface: ge-1/0/1, Unit: 0, Address: 23.0.0.3/24
  Index: 70, SNMP ifIndex: 527, VRRP-Traps: disabled
  Interface state: up, Group: 23, State: master, VRRP Mode: Active
  Priority: 150, Advertisement interval: 1, Authentication type: none
  Delay threshold: 100, Computed send rate: 0
  Preempt: yes, Accept-data mode: yes, VIP count: 1, VIP: 23.0.0.23          
  Advertisement Timer: 0.563s, Master router: 23.0.0.3
  Virtual router uptime: 00:15:11, Master router uptime: 00:10:48
  Virtual Mac: 00:00:5e:00:01:17 
  Tracking: disabled 

lab@Junos3> show vrrp extensive 
Interface: ge-1/0/1.0, Interface index :70, Groups: 1, Active :1
  Interface VRRP PDU statistics
    Advertisement sent                       :956       
    Advertisement received                   :82        
    Packets received                         :82         
    No group match received                  :0         
  Interface VRRP PDU error statistics
    Invalid IPAH next type received          :0         
    Invalid VRRP TTL value received          :0         
    Invalid VRRP version received            :0         
    Invalid VRRP PDU type received           :0         
    Invalid VRRP authentication type received:0         
    Invalid VRRP IP count received           :0         
    Invalid VRRP checksum received           :0

Physical interface: ge-1/0/1, Unit: 0, Address: 23.0.0.3/24
  Index: 70, SNMP ifIndex: 527, VRRP-Traps: disabled
  Interface state: up, Group: 23, State: master, VRRP Mode: Active
  Priority: 150, Advertisement interval: 1, Authentication type: none
  Delay threshold: 100, Computed send rate: 0
  Preempt: yes, Accept-data mode: yes, VIP count: 1, VIP: 23.0.0.23          
  Advertisement Timer: 0.404s, Master router: 23.0.0.3
  Virtual router uptime: 00:15:14, Master router uptime: 00:10:51
  Virtual Mac: 00:00:5e:00:01:17 
  Tracking: disabled 
  Group VRRP PDU statistics             
    Advertisement sent                       :956       
    Advertisement received                   :82        
  Group VRRP PDU error statistics
    Bad authentication Type received         :0         
    Bad password received                    :0         
    Bad MD5 digest received                  :0         
    Bad advertisement timer received         :0         
    Bad VIP count received                   :0         
    Bad VIPADDR received                     :0         
  Group state transition statistics
    Idle to master transitions               :0         
    Idle to backup transitions               :2         
    Backup to master transitions             :3         
    Master to backup transitions             :1

Verification on the Cisco router:

R2#show vrrp 
Vlan23 - Group 23 
  State is Backup  
  Virtual IP address is 23.0.0.23
  Virtual MAC address is 0000.5e00.0117
  Advertisement interval is 1.000 sec
  Preemption enabled
  Priority is 100 
  Master Router is 23.0.0.3, priority is 150 
  Master Advertisement interval is 1.000 sec
  Master Down interval is 3.609 sec (expires in 3.477 sec)

R1>ping 23.0.0.23

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 23.0.0.23, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

Using Traceoptions (debugging) for VRRP

Set the tracing options:

lab@Junos3# show protocols vrrp     
traceoptions {
    file vrrp;
    flag state;
    flag general;
}

Alternatively, if we want to Copy & Paste from Notepad:

lab@Junos3# delete protocols vrrp 

[edit]
lab@Junos3# load merge terminal 
[Type ^D at a new line to end input]
protocols vrrp traceoptions {
    file vrrp;
    flag state;
    flag general;
}    
load complete

[edit]


# Other trace options:
[edit protocols vrrp]
lab@Junos3# set traceoptions flag ?
Possible completions:
  all                  Trace all events
  database             Trace database
  general              Trace general events
  interfaces           Trace interface messages
  normal               Trace normal events
  packets              Trace packets
  ppm                  VRRP PPM related messages
  state                Trace state transitions
  timer                Trace timer events

[edit]
lab@Junos3# commit 
commit complete

lab@Junos3> monitor start vrrp 

Trigger VRRP state change by setting a higher priority for R2, making it the new Master:

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#int vlan 23
R2(config-if)#vrrp 23 priority 200
R2(config-if)#
6w4d: %VRRP-6-STATECHANGE: Vl23 Grp 23 state Backup -> Master

Debugging messages as seen on Junos3

lab@Junos3> 
*** vrrp ***
Aug 18 14:50:18 vrrp_fsm_update IFD: ge-1/0/1.000.000.000.0000:0000:0000:0000:0000:0000:1700:0003.023 event: backup
Aug 18 14:50:18 vrrp_fsm_backup: ge-1/0/1.000.000.000.0000:0000:0000:0000:0000:0000:1700:0003.023 state from: master
Aug 18 14:50:18 vrrp_fsm_update_for_inherit IFD: ge-1/0/1.000.000.000.0000:0000:0000:0000:0000:0000:1700:0003.023 event: backup
Aug 18 14:50:18 Signalled dcd (PID 1008) to reconfig
Aug 18 14:50:18 vrrpd_rts_async_ifd_msg, Received Async message for: ge-1/0/1
Aug 18 14:50:18 vrrpd_rts_async_ifa_msg, Received Async message for: (null) index: 70, family 0x2 op: 0x3
Aug 18 14:50:19 vrrpd_rtsock_query_timer_expiry for ge-1/0/1.000.000.000.0000:0000:0000:0000:0000:0000:1700:0003.023
Aug 18 14:50:19 vrrpd_rts_get_ifd_state for ge-1/0/1.000.000.000.0000:0000:0000:0000:0000:0000:1700:0003.023
Aug 18 14:50:19 Interface ge-1/0/1 ifindex: 143 flags 0xc000
Aug 18 14:50:19 Interface ge-1/0/1 ifl ifindex: 70 flags 0x8000
Aug 18 14:50:19 ifd_fsm_update IFD: ge-1/0/1.000.000.000.0000:0000:0000:0000:0000:0000:1700:0003.023 event: up

Note

• VRRP relies on Multicast (address 224.0.0.18). The multicast is not supported on Olive running on VMWare. As the result, this lab need to be tested on a real Juniper router.
• Without “accept-data” option, the Master VRRP Juniper router does not respond to any traffic address to that IP. If we try to ping from R1, to the virtual router 23.0.0.23, it will fail ! We need to have that option enabled.
• Default behaviour for traffic addressed to the VRRP virtual IP is different on Cisco and on Juniper platform. By default Cisco allow the Master router to respond to that traffic.

Reference