L2 Control Packet Fwding on J-based system

[Update, Nov 1, 2019] Starting from 4.22.0F, EOS can support multiple L2 protocols packets but NEED to create/apply a TCAM profile. 

The L2 control packets like LACP/LLDP/STP are trapped by a switch and sent to CPU for processing. But for some use cases, this behavior requires to be changed. For example, In the below diagram, the CE1 and CE2 want to form a port-channel across the service provider's MPLS cloud network. 



Another use case is the create an L2 harness switch. As shown below, R1 and R2 can establish a port-channel across a harness switch transparently. 




Starting from Arista EOS 4.21.3F, a feature called - L2 protocol forwarding was introduced. With that, above requirement can be achieved. 

The configurations are quite straightforward, 
1) Step 1 - create l2-protocol forwarding profile and enable lacp forward
2) Step 2 - apply this profile under all the client facing interfaces
3) Step 3 - configure LDP type 5 pw - local patch to stitch 2 local eth ports

1. The port-channel is up. 

R1.gts425.15:54:21#sh port-ch 100
Port Channel Port-Channel100:
  Active Ports: Ethernet21 Ethernet22

2. "show lldp neighbor" shows remote and intermediate devices

R1.gts425.16:15:26#sh lldp nei et21, 22
Port       Neighbor Device ID   Neighbor Port ID           TTL
Et21       lp232                Ethernet12/1               120
Et21       R2.hss457            Ethernet21                 120
Et22       lp232                Ethernet12/2               120
Et22       R2.hss457            Ethernet22                 120

Limitation:

• In 4.21.3F, only LACP/LLDP supported. STP and other L2 protocols will be supported in later releases. 
• From 4.22.0F, more protocols are supported, but need to create a tcam profile. 

Arista BGP Peering Options (5) - v6 link-local peering/NH

Arista TOI Link

Compared with "v4 over v6 NH", the configuration is quite similar, except the interface address and neighbor use link-local address. 

So the benefits here are obvious:

• Minimizing the configuration efforts, no need ipv4 and global ipv6 address. 
• IPv4 NLRI over IPv6 link-local NH. 

Because the link-local address is a local address and not accessible by remote devices, the bgp configure is considered to be a special one. The following regular bgp knobs don't work as usually:

• iBGP
• eBGP multi-hop
• confed BGP
• local AS if changing to iBGP
• Link-local eBGP learnt routes are not advertised to iBGP peers, because of lacking NH address

Please note that this is ONLY supported in Gated, not in multi agent mode. 

Arista BGP Peering Options (4) - IPv4 NRLI over IPv6 NH

This blog is about the v4 NRLI over v6 NH. Based on Arista 4.17.0F TOI, this feature comprises 2 parts:

• IPv4 forwarding on IPv6 interface
• IPv6 NRLI with IPv6 NH

Compared with previous configurations, 2 configures are needed:

1. IPv4 forwarding on IPv6 interface, which is enabled by "ip routing ipv6 interfaces  vrf v3"
2. neighbor <ipv6Nei> next-hop address-family ipv6 originate

Another way of 2nd part is to enable "bgp next-hop address-family ipv6". But in the above topology, the NRLI is originated directly, so have to put originate. 

From the output of "show ip route", you can see the v4 route's NH is the ipv6 address

R1.lp231.20:13:59(config-router-bgp-vrf-v3)#sh ip route bgp detail

VRF: v3

 B E   103.2.2.0/24 [200/0]

       via 12:12:103::2, Vlan103

R1.lp231.20:14:07(config-router-bgp-vrf-v3)#sh ip bgp 103.2.2.0

BGP routing table information for VRF v3

Router identifier 103.2.2.2, local AS number 1

BGP routing table entry for 103.2.2.0/24

 Paths: 1 available

  2

    12:12:103::2 from 12:12:103::2 (103.2.2.2)

      Origin IGP, metric 0, localpref 100, IGP metric 1, weight 0, received 00:18:46 ago, valid, external, best

      Rx SAFI: Unicast

Arista BGP Peering Options (3) - IPv6 NRLI over IPv4 Peering

This peering configuration is the reverse version of the previous one. So the configuration-wise, it is very similar. 

The benefit is clear, we only need 1 BGP peering to carry both address families. But we still need to keep a dual stack on the interfaces to be used as NH. In today data center network, ECMP is very popular and wide. Evening using /31, the finest network mask, a /24 can only afford 128 links. 

And this youtube explains very well on the requirement behind RFC 5549, ipv4 over v6 NH, which is shown in the next blog

Arista BGP Peering Options (2) - IPv4 NRLI over IPv6 Peering

Here is the TOI link

On the top of regular IPv6 BGP configuration, you need to configure 2 things:

1) Enable IPv4 NLRI over v6 peering and 2 ways to do it:

1. Under ipv4 address-family, activate it on ipv6 peering, which is shown in R1's configuration
2. Global configuration of "bgp default ipv4-unicast transport ipv6" in R2's show run

2) Specify the IPv4 NH of IPv4 prefixes, and we can do it 3 ways:

1. Global automatic NH address - "bgp auto-local-addr" in R'2 configuration. But this is a global knob, so if you have IPv6 prefixes over IPv4 NH, it will break peering silently. 
2. Neighbor-base automatic NH address - "neighbor 12:12:101::1 auto-local-addr"
3. Neighbor-base manual NH address in R1's configuration - "neighbor 12:12:101::2 local-v4-addr 12.12.101.1"

Here is the output of "sh ip bgp 101.2.2.0" (I enable "routing-context vrf v1", so no need vrf)

R1.lp231.16:28:58(config)#sh ip bgp 101.2.2.0

BGP routing table information for VRF v1

Router identifier 101.1.1.1, local AS number 1

BGP routing table entry for 101.2.2.0/24

 Paths: 1 available

  2

    12.12.101.2 from 12:12:101::2 (101.2.2.2)

      Origin IGP, metric 0, localpref 100, IGP metric 1, weight 0, received 00:02:03 ago, valid, external, best

      Rx SAFI: Unicast

And both ipv4/ipv6 ping work as expected. 

R1.lp231.16:29:11(config)#ping 101.2.2.2

PING 101.2.2.2 (101.2.2.2) 72(100) bytes of data.

80 bytes from 101.2.2.2: icmp_seq=1 ttl=64 time=0.242 ms

80 bytes from 101.2.2.2: icmp_seq=2 ttl=64 time=0.100 ms

80 bytes from 101.2.2.2: icmp_seq=3 ttl=64 time=0.093 ms

80 bytes from 101.2.2.2: icmp_seq=4 ttl=64 time=0.091 ms

80 bytes from 101.2.2.2: icmp_seq=5 ttl=64 time=0.093 ms

--- 101.2.2.2 ping statistics ---

5 packets transmitted, 5 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.091/0.123/0.242/0.060 ms, ipg/ewma 0.174/0.180 ms

R1.lp231.16:31:39(config)#ping ipv6 101:2:2::2

PING 101:2:2::2(101:2:2::2) 72 data bytes

80 bytes from 101:2:2::2: icmp_seq=1 ttl=64 time=0.262 ms

80 bytes from 101:2:2::2: icmp_seq=2 ttl=64 time=0.106 ms

80 bytes from 101:2:2::2: icmp_seq=3 ttl=64 time=0.103 ms

80 bytes from 101:2:2::2: icmp_seq=4 ttl=64 time=0.099 ms

80 bytes from 101:2:2::2: icmp_seq=5 ttl=64 time=0.103 ms

--- 101:2:2::2 ping statistics ---

5 packets transmitted, 5 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.099/0.134/0.262/0.064 ms, ipg/ewma 0.184/0.196 ms

Arista BGP Peering Options (1)

In this series of blogs, I will discuss a bit on the Arista EOS BGP peering options. As of March 2019, the EOS has the following peering options:



Here is the Arista official TOI links:

• BGP IPv4 NLRI over IPv6 Transport
• BGP IPv4 NLRI over IPv6 NH
• BGP IPv6 Link-local Peering

So, why do we need them, what's the difference and how to configure them?

1) Why
Today's network needs to support dual address spaces - IPv4 and IPv6, which requires providers to run 2 parallel control/forward planes. So the purpose of all the above features is to decrease the complexity by reducing control plane even forward plane.

For example, a data center with public ipv4/ipv6 address and needs support worldwide ipv4 and ipv6 access. Is that possible to use ipv4/ipv6 address space at the server pool and service routers, and only ipv6 infrastructure? The answer is yes.

2)  What's the difference between the above 3 options?

1. The basic way is to have dual ipv4/ipv6 infrastructures and control planes, which means, 2 BGP sessions and 2 address planes. 
2. IPv4 NRLI over IPv6 Transport is to send v4 prefixes over v6 peering. So it saves 1 just 1 control plane - ipv6 bgp but still needs ipv4 next hop as ipv4 prefixes, which means 2 data planes or dual v4/v6 stack. 
3. RFC 5549 - IPv4 NRLI over IPv6 NH, this feature is one step further, ipv4 prefixes can use ipv6 NH. So 1 BGP session and 1 data planes (v6 infrastructure)
4. BGP IPv6 link-local peering, this utilizes the ipv6 link-local address to establish BGP peering w/o assigning global IPv6 interface addresses. 

3) How to configure them? In the following blogs, I will show the configuration 1 by 1. (Please note that I use vrf under router bgp, in order to configure them together in one place. But it is not required and most of the time, you only need ONE)

Install Arista EOS Swix

Reference:

• Arista's - "Package RPM into an Arista swix"
• Arista Extension ylb.jp

In the Linux world, RPM package is used for software installation. Simply put, the swix (software image extension) is a software package which contains multiple RPM files and a manifest file, which is used to install software or software patch on Arista EOS. 

Generally, a swix is installed in the following steps:

1. copy to local extension directory - "scp, copy"

2. install - "extension"

3. copy them to boot-extension - "copy"

4. if a modular system, install on the standby supervisor - "session"

5. restart agents or reset hardware components if required - "agent terminate"

Additionally, 

1. uninstall - "no extension"

2. remove packet - "delete" 

3. If both sup running mismatched swix, login console and reset boot-extension file

Below is the step by step:

Step 1 - copy

[admin@eosrouter flash]$ scp solomonyang@server:/users/solomonyang/swix/*swix /mnt/flash

patch.swix                                                                            100%   11MB  11.4MB/s   00:00

eosrouter#copy flash:patch.swix extension:

eosrouter#show extensions

Name                          Version/Release      Status      Extension

----------------------------- -------------------- ----------- ---------

patch.swix                    1.0.4 .              A, NI       6

A: available | NA: not available | I: installed | NI: not installed | F: forced

eosrouter#show extensions detail

       Name: patch.swix

    Version: 1.0.4

    Release: blahblah

   Presence: available

     Status: not installed
    Summary: SandFap agent related libs

   Packages:

 Total size: 0 bytes

Description: SandFap agent related libs.

Step 2 - install

eosrouter#extension patch.swix

eosrouter#show extensions

Name                          Version/Release      Status      Extension

----------------------------- -------------------- ----------- ---------

patch.swix                    1.0.4                A, I        6

Step 3 - copy to boot extension

eosrouter#show installed-extensions

patch.swix

eosrouter#show boot-extensions

eosrouter#copy installed-extensions boot-extensions

Copy completed successfully.

eosrouter#show boot-extensions

patch.swix

Step 4 - install them on standby sup

eosrouter#copy flash:patch.swix supervisor-peer:/mnt/flash

Copy completed successfully.

eosrouter#session peer-supervisor

Warning: Permanently added '[127.1.0.2]:3601' (RSA) to the list of known hosts.

WARNING - you are currently logged in to the standby supervisor.

Not all cli commands are available or supported. Configuration mode

is disabled for SSO.

s1:eosrouter#copy flash:patch.swix extension:

Copy completed successfully.

s1:eosrouter#extension patch.swix

s1:eosrouter#copy installed-extensions boot-extensions

Copy completed successfully.

Step 5 - restart agents or reset hardware component

For some software patches, it is required to restart software agents or hardware components to make patch effective. 

eosrouter#show agent names | grep SandFap

SandFap-Linecard3

SandFap-Linecard4

SandFap-Linecard5

eosrouter#agent SandFap-Linecard3 SandFap-Linecard4 SandFap-Linecard5 terminate

SandFap-Linecard3 was terminated

SandFap-Linecard4 was terminated

SandFap-Linecard5 was terminated

Also you need to do the same on the standby sup

How to fix unmatched swix'ed sup

If you run into unmatched swix supervisor due to mis-configuration, the way to fix them is as follows:

1. Login in the console of both supervisor. Sometimes the standby sup is at bad shape, then you have to fix it in Aboot mode

Arista Networks EOS 4.20.5.2F

localhost login: admin

[PyServer ar.Aaa not responding, still trying -- is it running?]

2. Reboot both and press ctrl-C to enter Aboot mode

3. Vi /mnt/flash/boot-extensions to remove swix entries. Then reboot so you will have a clean EOS w/o any extensions. 

Arista EOS - dot1q tunnel

There are 2 common use cases for dot1q tunnel:
1) Service providers tunnel customer's VLAN-tag traffic across different sites;
2) Lab harness switch for flexible connectivity between devices

But with one limitation: L2 control packets can't traverse. For example, "show lldp neighbor" on cd644 indicates yo412 instead of remote router.