MPLS Hashing: Control Word and Entropy Label

A good presentation on this topic: https://ripe66.ripe.net/presentations/141-RIPE66_Understanding_MPLS_Hashing-Snijders.pdf

• MPLS Lable doesn't have encapsulated protocol information. So after the bottom label, 2 possibilities:
• IPv4/v6 packets if L3VPN, the 1st 4-bit is ipVersion, 4=ipv4, 6=ipv6
• Ethernet frames if L2VPN, the 1st 48-bit is dstMAC
• Along the LSP, if P routers like to do ECMP by hashing, it has to guess like above to do hashing the 5-tuple fields (src/dstAddr, src/dstPort, protoNum)
• So the problem happens if the dstMac starts with 4 or 6

The solution for the above issue is MPLS control word (RFC 4385, Feb 2006), a 4-byte all-zero (same length as a label) control word after the bottom label. 

• Arista EOS 4.21.3F: Configurable L2 EVPN MPLS control word
• mpls evpn; label-stack control-word
• Arista EOS 4.22.1F: Control Word support on LDP Pseudowire
• mpls ldp; pseudowire <name>; control-word

So the MPLS control word is to prevent incorrect hashing. How does the LSR do the load-balancing? The answer is Entropy Label (RFC 6790, Nov 2012)

• Increase the label stack by 2: 1 for ELI (entropy label indicator = 7), 1 for EL (entropy label)
• Aristea EOS 4.26.0F: LDP Entropy Label Support
• 4.25.2F: LSR support
• 4.26.0F: LER support
• mpls ldp; entropy-label
• This feature has a limitation: PHP LSR can't pop ELI + EL labels, so has to rely on LER to pop up [ELI, ELI, VPN] labels. 

Another solution is FAT - Flow Aware Transport on LDP pseudowires (4.24.0F) based on RFC 6391

• mpls ldp; pseudowire; pseudowire <name>; label flow

L2 Control Packet Fwding on J-based system

[Update, Nov 1, 2019] Starting from 4.22.0F, EOS can support multiple L2 protocols packets but NEED to create/apply a TCAM profile. 

The L2 control packets like LACP/LLDP/STP are trapped by a switch and sent to CPU for processing. But for some use cases, this behavior requires to be changed. For example, In the below diagram, the CE1 and CE2 want to form a port-channel across the service provider's MPLS cloud network. 



Another use case is the create an L2 harness switch. As shown below, R1 and R2 can establish a port-channel across a harness switch transparently. 




Starting from Arista EOS 4.21.3F, a feature called - L2 protocol forwarding was introduced. With that, above requirement can be achieved. 

The configurations are quite straightforward, 
1) Step 1 - create l2-protocol forwarding profile and enable lacp forward
2) Step 2 - apply this profile under all the client facing interfaces
3) Step 3 - configure LDP type 5 pw - local patch to stitch 2 local eth ports

1. The port-channel is up. 

R1.gts425.15:54:21#sh port-ch 100
Port Channel Port-Channel100:
  Active Ports: Ethernet21 Ethernet22

2. "show lldp neighbor" shows remote and intermediate devices

R1.gts425.16:15:26#sh lldp nei et21, 22
Port       Neighbor Device ID   Neighbor Port ID           TTL
Et21       lp232                Ethernet12/1               120
Et21       R2.hss457            Ethernet21                 120
Et22       lp232                Ethernet12/2               120
Et22       R2.hss457            Ethernet22                 120

Limitation:

• In 4.21.3F, only LACP/LLDP supported. STP and other L2 protocols will be supported in later releases. 
• From 4.22.0F, more protocols are supported, but need to create a tcam profile. 

Arista MPLS VPN Step by Step (2c) L2VPN - Type 4 PW, MTU

In the LDP PW configuration, one thing needs to be careful is the MTU. You have to ensure the MTU along the underlying LSP matched. Here is an example:

So we have OSPF and ISIS enabled on VLAN 301/302 on CEs. But only OSPF up, no ISIS neighbor at all

CE1#sh ip os neighbor vrf ce1-pw4-301
Neighbor ID     VRF      Pri State                  Dead Time   Address         Interface
101.255.2.2     ce1-pw4-301 1   FULL/DR                00:00:32    101.101.101.2   Vlan301
CE1#sh isis neighbors vrf ce1-pw4-302
CE1#

Well the ping works as expected, CE1 can ping CE2's vlan 302 address - 102.102.102.2

CE1#ping vrf ce1-pw4-302 102.102.102.2
PING 102.102.102.2 (102.102.102.2) 72(100) bytes of data.
80 bytes from 102.102.102.2: icmp_seq=1 ttl=64 time=0.276 ms
80 bytes from 102.102.102.2: icmp_seq=2 ttl=64 time=0.119 ms
80 bytes from 102.102.102.2: icmp_seq=3 ttl=64 time=0.112 ms
80 bytes from 102.102.102.2: icmp_seq=4 ttl=64 time=0.111 ms
80 bytes from 102.102.102.2: icmp_seq=5 ttl=64 time=0.110 ms

--- 102.102.102.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.110/0.145/0.276/0.066 ms, ipg/ewma 0.202/0.208 ms

Both sides configured with MTU 9000

CE1#sh run int vlan 302
interface Vlan302
   mtu 9000
   vrf forwarding ce1-pw4-302
   ip address 102.102.102.1/24
   isis enable isis-ce1-pw4-302

CE2#sh run int vlan 302
interface Vlan302
   mtu 9000
   vrf forwarding ce2-pw4-302
   ip address 102.102.102.2/24
   isis enable isis-ce2-pw4-302

Let's try ping the destination with size of data plus DF-bit

CE1#ping vrf ce1-pw4-302 102.102.102.2 size 9000 df-bit
PING 102.102.102.2 (102.102.102.2) 8972(9000) bytes of data.
--- 102.102.102.2 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 40ms

Ok, it fails but where is the culprit?

PE1 and PE2 are good:

PE1(config-if-Et47/2)#sh ip int brief | grep 88
Ethernet3/1            88.1.11.11/24      up         up              9082
Ethernet45/3           88.3.11.11/24      down       down            9082

PE2(config-if-Et51/1)#sh ip int brief | grep 88
Ethernet33/1           88.4.22.22/24      up         up              9082
Ethernet51/1           88.2.22.22/24      up         up              9082

Let's move on to P routers:

P1#sh ip int brief | grep 88
Ethernet35/1           88.1.4.1/24        up         up              8000
Ethernet51/1           88.1.2.1/24        up         up              8000
Ethernet52/1           88.1.3.1/24        up         up              9082
Ethernet54/1           88.1.11.1/24       up         up              9082

Ok, here is the issue. Let's correct them and ISIS neighbor is up!

CE1#sh isis neighbors vrf ce1-pw4-302

Instance  VRF      System Id        Type Interface          SNPA              State Hold time   Circuit Id
isis-ce1- ce1-pw4- CE2              L2   Vlan302            44:4c:a8:11:6b:e1 UP    8           CE2.02

So, the reason is, the ISIS pads hello packet upto MTU and the oversized packets will be dropped siliently on underlying P routers. Hence ISIS fails to form neighbor. 

The OSPF just negotiates the MTU of connecting interfaces, which are 9000 in both sides, so ospf neighbor is up w/o any issues. But if you have a large LSA database, and ospf utilize the full MTU to transmit it, you will hit this MTU mismatch later, but not in the neigobor discoverage stage. 

Arista MPLS VPN Step by Step (2b) L2VPN - Type 5 PW

This blog covers type 5 pw configuration. Here is the topology:

Configuration-wise, type 5 is simpler than type 4. And between 2 CEs, it is just a like direct ethernet link. From above topology, there is 2 vlans between CEs over 1 pw. 

PE1 Configuration:

mpls ldp
   router-id 88.255.11.11
   transport-address interface Loopback88
   fec filter prefix-list fec-pfx-list
   no shutdown
   !
   pseudowires
      pseudowire to-ce2-type5-pw
         neighbor 88.255.22.22
         pseudowire-id 303
         mtu 9082
!
patch panel
   patch ce1-ce2-pw5-1
      connector 1 interface Ethernet47/3
      connector 2 pseudowire ldp to-ce2-type5-pw
!
interface Ethernet47/3
   description mpls.ldp5pw.pe1=>ce1.et51/3
   !! PE to CE interface MUST be routed port !!
   no switchport

L2 Control Packets

One different between type4 and 5 is the forwarding behavior of L2 control packets. From above link, the LLDP is forwarded. So "show lldp neighbor" on CE1 shows CE2, not PE1

CE1(config)#ln
Et51/1     PE1                              Ethernet47/1               120
Et51/2     PE1                              Ethernet47/2               120
Et51/3     PE1                              Ethernet47/3               120
Et51/3     CE2                              Ethernet52/3               120

Arista MPLS VPN Step by Step (2a) L2VPN - Type 4 PW

Based on the MPLS LDP configuration completed in the previous session, I will talk about the MPLS LDP type 4 pseudowire configuration.

Regarding the LDP/BGP/IGP configuration, please refer to this blog.

Here is the topology:



Here we have 2 pseudowires: ce1-ce2-pw4-vlan-301 and ce1-ce2-pw4-vlan-302 to L2 connectivity between CE1 and CE2 across LDP cloud. From CEs' perspective, the pw works like a directly connected L2 link. So the VLANs are in same subnet and running IGPs (OSPF/ISIS) on the top of it. 

PE 1 configuration:

interface Ethernet3/1
   description mpls.pe1=>p1:et54/1
   mtu 9082
   no switchport
   ip address 88.1.11.11/24
!
interface Ethernet45/3
   description mpls.pe1=>p3:et45/3
   mtu 9082
   no switchport
   ip address 88.3.11.11/24
!
interface Ethernet47/2
   description mpl1.pe1=>ce1:et51/1
   mtu 9082
   ! pw intf to CE1 must be routed port, otherwise pw down !
   no switchport
!
interface Loopback88
   ip address 88.255.11.11/32
!
mpls ip
!
mpls ldp
   ! mpls ldp config is same as mpls ldp l3vpn !
   router-id 88.255.11.11
   transport-address interface Loopback88
   fec filter prefix-list fec-pfx-list
   no shutdown
   !
   pseudowires
      pseudowire to-ce2-vlan-301
         ! 88.255.22.22 is PE2's ldp/pw loopback ip addr !
         neighbor 88.255.22.22
         pseudowire-id 301
         mtu 9082
      !
      pseudowire to-ce2-vlan-302
         neighbor 88.255.22.22
         pseudowire-id 302
         mtu 9082
!
patch panel
   patch ce1-ce2-pw4-vlan-301
      ! note, e47/2 is cfg'ed as routed port, NOT sw/trunk port !
      connector 1 interface Ethernet47/2 dot1q vlan 301
      connector 2 pseudowire ldp to-ce2-vlan-301
   !
   patch ce1-ce2-pw4-vlan-302
      connector 1 interface Ethernet47/2 dot1q vlan 302
      connector 2 pseudowire ldp to-ce2-vlan-302

PE 2 configuration:

interface Ethernet33/1

   description pe2-p4=>p4.et33/1

   mtu 9082

   no switchport

   ip address 88.4.22.22/24

!

interface Ethernet37/2

   description pe2-ce2=>ce2.et52/2

   mtu 9082

   no switchport

!

interface Ethernet51/1

   description pe2-p2=>p2.et51/1

   mtu 9082

   no switchport

   ip address 88.2.22.22/24

!

interface Loopback88

   ip address 88.255.22.22/32

!

mpls ip

!

mpls ldp

   router-id 88.255.22.22

   transport-address interface Loopback88

   fec filter prefix-list fec-pfx-list

   no shutdown

   !

   pseudowires

      pseudowire to-ce1-vlan-301

         neighbor 88.255.11.11

         pseudowire-id 301

         mtu 9082

      !

      pseudowire to-ce1-vlan-302

         neighbor 88.255.11.11

         pseudowire-id 302

         mtu 9082

!

patch panel

   patch ce1-ce2-pw4-vlan-301

      connector 1 interface Ethernet37/2 dot1q vlan 301

      connector 2 pseudowire ldp to-ce1-vlan-301

   !

   patch ce1-ce2-pw4-vlan-302

      connector 1 interface Ethernet37/2 dot1q vlan 302

      connector 2 pseudowire ldp to-ce1-vlan-302

For the CE's configuration, please note that I use VRF on CE to simulate multiple hosts. 

CE1 configuration:

vrf definition ce1-pw4-301

   rd 301:1

!

vrf definition ce1-pw4-302

   rd 302:1

!

interface Ethernet51/2

   description mpls.ce1=>pe1:et47/2

   ! compare with PE1's et47/2, here is sw + trunk !

   switchport trunk allowed vlan 301-302

   switchport mode trunk

!

interface Vlan301

   vrf forwarding ce1-pw4-301

   ip address 101.101.101.1/24

!

interface Vlan302

   mtu 9000

   vrf forwarding ce1-pw4-302

   ip address 102.102.102.1/24

   isis enable isis-ce1-pw4-302

!

interface Loopback301

   vrf forwarding ce1-pw4-301

   ip address 101.255.1.1/24

!

interface Loopback302

   vrf forwarding ce1-pw4-302

   ip address 102.255.1.1/24

   isis enable isis-ce1-pw4-302

!

ip routing vrf ce1-pw4-301

ip routing vrf ce1-pw4-302

!

router isis isis-ce1-pw4-302 vrf ce1-pw4-302

   net 47.0002.0001.0001.0001.00

   is-type level-2

   !

   address-family ipv4 unicast

!

router ospf 301 vrf ce1-pw4-301

   network 101.0.0.0/8 area 0.0.0.0

   max-lsa 12000

CE2 configuration:

interface Ethernet52/2

   description mpls.ce2=>pe2:et37/2

   switchport trunk allowed vlan 301-302

   switchport mode trunk

!

vrf definition ce2-pw4-301

   rd 301:2

!

vrf definition ce2-pw4-302

   rd 302:2

!

interface Loopback301

   vrf forwarding ce2-pw4-301

   ip address 101.255.2.2/24

!

interface Loopback302

   vrf forwarding ce2-pw4-302

   ip address 102.255.2.2/24

   isis enable isis-ce2-pw4-302

!

interface Vlan301

   vrf forwarding ce2-pw4-301

   ip address 101.101.101.2/24

!

interface Vlan302

   mtu 9000

   vrf forwarding ce2-pw4-302

   ip address 102.102.102.2/24

   isis enable isis-ce2-pw4-302

!

ip routing vrf ce2-pw4-301

ip routing vrf ce2-pw4-302

!

router isis isis-ce2-pw4-302 vrf ce2-pw4-302

   net 47.0002.0002.0002.0002.00

   is-type level-2

   !

   address-family ipv4 unicast

!

router ospf 301 vrf ce2-pw4-301

   network 101.0.0.0/8 area 0.0.0.0

   max-lsa 12000

Arista MPLS VPN Step by Step (0)

In this step-by-step series, I will cover the Arista MPLS VPN solution. All of the configuration is based on the following topology. For the technical details, please refer to "Arista MPLS IPv4/v6 MPLS VPN Configuration Guide" 

And some other references:

• Arista Software TOI - click TOI
• 4-15-0F:
• MPLS Label Encap
• Static MPLS Label Pop/Swap (T2)
• 4-17F:
• MPLS LDP (4-17-0F)
• ISIS Segment Routing
• MPLS over GRE
• LDP Pw
• 4-18-1F:
• Static MPLS Label Push (T2/TH)
• 4-20F:
• L3 EVPN MPLS
• LDP Pw type 5
• BGP LU/SR
• L2 EVPN MPLS
• 4-21F:
• MPLS Traceroute
• Inter-VRF Route Leaking
• Arista MPLS Manual
• Arista EVPN Manual