Arista EOS/Networking Tips by Solomon Yang

4/30/2019

Arista 7160S-32CQ Error Msg: % Error: Cannot apply ip ACL test2 to Ethernet2/1 (Out of SW resources on Chip-0)

Platform: Arista 7160S-32CQ, XP80, 32QSFP100
2 large ip ACL with 1500 entries
System is kind of strenched, and hitting limit with hash collision with mask-group
IP Access List test2
10 deny pim host 70.0.70.210 any
20 deny ospf host 0.70.210.70 any
30 deny pim host 140.140.70.210 any
40 permit gre host 70.0.0.0 any
50 permit ospf host 70.140.0.210 any
60 deny icmp host 140.70.210.70 any
70 permit igmp host 0.70.0.0 any
80 permit vrrp host 0.0.0.210 any
90 deny tcp host 70.70.210.70 any
100 deny ospf host 0.210.210.140 any
110 deny gre host 210.140.0.210 any
120 deny ip host 0.0.210.210 any
130 deny igmp host 140.70.140.70 any
140 permit ip host 70.0.70.0 any

dut23:44:51(config-if-Et2/1)#ip access-group test2 in

% Error: Cannot apply ip ACL test2 to Ethernet2/1 (Out of SW resources on Chip-0)

fu590.09:21:58#show platform algo chip 0 acl table usage

Table ID Entries used

-------- ------------

0 7968

1 5567

2 6415

3 0

4 8010

5 2520

show platform algo chip 0 acl mask-group

Initial revision

Hw Acl ID 1

Mask group ID Mask group descriptor Entries

------------- ------------------------------------------ -------

2 ipv4PAcl,sip:8,dip:0,sport:0,dport:0 159

1 ipv4PAcl,sip:8,dip:0,sport:0,dport:0,proto 1385

3 ipv4PAcl,sip:9,dip:0,sport:0,dport:0,proto 19

Final revision

Hw Acl ID 1

Mask group ID Mask group descriptor Entries

------------- ------------------------------------------ -------

3 ipv4PAcl,sip:9,dip:0,sport:0,dport:0,proto 30480

show platform algo chip 0 acl table la

Final revision

Hw Acl ID 1

Table ID Mask group ID Mask group descriptor

-------- ------------- ------------------------------------------

0 3 ipv4PAcl,sip:9,dip:0,sport:0,dport:0,proto

1 3 ipv4PAcl,sip:9,dip:0,sport:0,dport:0,proto

2 3 ipv4PAcl,sip:9,dip:0,sport:0,dport:0,proto

4 3 ipv4PAcl,sip:9,dip:0,sport:0,dport:0,proto

5 3 ipv4PAcl,sip:9,dip:0,sport:0,dport:0,proto

ECN: Explicit Congestion Notifcation, PFC

在DC里面，特别是Storage Network/FCOE，对pkt drop很敏感，常用的是2种QOS

PFC - Prority Flow Control.
ECN - Explicit Congestion Notification,

是DSCP后面的2 bit.
00 - Not supported
01/10 - Supported
11 - CE : congestion encountered

如何Work:

Server pkt 通过 Router
Router发现有Egress congestion, 或者定义的min/max threshold, 开始set ECN to CE
Client收到CE pkt后，通过TCP adjust window to slow down traffic.

一般是定义在Egress interface，config在后面

interface et3/1/1

mtu 9000

speed force 40gfull

no switchport

ip address 10.1.1.1/24

qos trust dscp

tx-queue 0

random-detect ecn minimum-threshold xxxx bytes max yyyy bytes

LACP rate fast vs MLAG

Arista does NOT recommend LACP rate fast enabled with MLAG, which is not reliable especially in scale environment.

In 4.20.2.1F Release Note, on page 21

"LACP PDU fast rate ("lacp rate fast" on an Arista switch) should not be configured on any port in an MLAG pair, or any port connected to an MLAG pair. (13950)"

4/29/2019

How to see the open bug in Arista EOS

Generally, you can see the list of the open bugs in the software release note. Another way is to go to arista.com, click Support in the menu bar, then select the "Software Bug Portal".

Or if you know the specific bug id, you can get it like https://www.arista.com/en/support/software-bug-portal/bugdetail?bug_id=<bug_id>

4/16/2019

How to debug ISIS/BFD neighbor issue

Topology:

[R1/DCS-7512N]:et11/36/1--{180.13.1.0/24}--et11/1:[R2/DCS-7050QX]

* both routers run 4.20.11M
* Between R1 and R2, ISIS/BFD is enabled shown as below

Router Configuration:

R1 configuration:

router isis isis.def
net 49.0001.0000.0000.0001.00
is-type level-1
log-adjacency-changes
redistribute connected
spf-interval 1 50 100
timers lsp generation 1 10 1000
!
address-family ipv4 unicast
maximum-paths 32
!
address-family ipv6 unicast
maximum-paths 32
!
interface Ethernet11/36/1
description UpEt_A_3.1
speed forced 10000full
no switchport
ip address 180.13.1.1/24
ipv6 address 180:13:1::1/64
isis enable isis.def
isis bfd
isis circuit-type level-1
isis network point-to-point
isis authentication mode md5 level-1
isis authentication key 7 QI3r+/7nP3Q= level-1

R2 configuration:
router isis isis.def
net 49.0001.0000.0000.0103.00
is-type level-1
log-adjacency-changes
spf-interval 1 50 100
timers lsp generation 1 10 1000
!
address-family ipv4 unicast
maximum-paths 32
!
address-family ipv6 unicast
maximum-paths 32
!
interface Ethernet11/1
description UpEt_A_3.1
speed forced 10000full
no switchport
ip address 180.13.1.100/24
ipv6 address 180:13:1::100/64
isis enable isis.def
isis bfd
isis circuit-type level-1
isis network point-to-point
isis authentication mode md5 level-1
isis authentication key 7 QI3r+/7nP3Q= level-1

Symptom:

R1: No ISIS and BFD neighbor

R1.13:53:14(config)#show isis nei | grep uspine3
R1.13:53:16(config)#show bfd nei | grep 180.13
R1.13:53:18(config)#

R2: ISIS neighbor stuck in INIT, and BFD is down

R2#sh isis nei
Instance VRF System Id Type Interface SNPA State Hold time Circuit Id
isis.def default R1 L1 Ethernet11/1 P2P INIT 26 2D

R2#sh bfd nei
VRF name: default
-----------------
DstAddr MyDisc YourDisc Interface Type LastUp LastDown LastDiag State
-------------------- --------------- --------------- --------------------- ------------ -------------------- -------------------- ----------------------- -----
180.13.1.1 2628313401 0 Ethernet11/1(47) normal 04/15/19 13:25 04/15/19 13:26 Nbr Signaled Down Down

Step1: R1/R2 - "show lldp nei" good and ping works, so no connectivity issue but high latency

-- show lldp nei --
R2#sh lldp nei
Port Neighbor Device ID Neighbor Port ID TTL
Et11/1 R2 Ethernet11/36/1 120

-- ping -- R2#ping 180.13.1.1
PING 180.13.1.1 (180.13.1.1) 72(100) bytes of data.
80 bytes from 180.13.1.1: icmp_seq=1 ttl=64 time=390 ms
80 bytes from 180.13.1.1: icmp_seq=2 ttl=64 time=437 ms
....

-- show interface <intf> | grep error --
R1.13:46:01(config)#sh int et11/36/1 | grep error
0 input errors, 0 CRC, 0 alignment, 0 symbol, 0 input discards

0 output errors, 0 collisions

R2.13:45:34(config)#sh int et11/1 | grep error
0 input errors, 0 CRC, 0 alignment, 0 symbol, 0 input discards

0 output errors, 0 collisions

Step2: R2's Down BFD shows NO incoming bfd pkts

-- show bfd neighbor dest-ip <ip> detail --
R2#sh bfd neighbors dest-ip 180.13.1.1 detail
VRF name: default
-----------------
Peer Addr 180.13.1.1, Intf Ethernet11/1, Type normal, State Down
VRF default, LAddr 180.13.2.100, LD/RD 2628313401/0
Session state is Down and not using echo function
Last Up Apr 15 13:25:27 2019
Last Down Apr 15 13:26:23 2019
Last Diag: Nbr Signaled Down
TxInt: 1000, RxInt: 1000, Multiplier: 3
Received RxInt: 300, Received Multiplier: 3
Rx Count: 0, Rx Interval (ms) min/max/avg: 0/0/0 last: never
Tx Count: 87681, Tx Interval (ms) min/max/avg: 508/996/870 last: 627 ms ago
Detect Time: 3000
Sched Delay: 1*TxInt: 87898, 2*TxInt: 0, 3*TxInt: 0, GT 3*TxInt: 0

R2 sends out BFD but no receiving

-- bash tcpdump -ni <intf> udp --
[admin@R2 ~]$ tcpdump -ni et11_1 udp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on et11_1, link-type EN10MB (Ethernet), capture size 262144 bytes
11:48:22.309536 28:99:3a:3b:15:39 > 44:4c:a8:a5:11:40, ethertype IPv4 (0x0800), length 66: 180.13.1.100.57657 > 180.13.1.1.bfd-control: BFDv1, Control, State Down, Flags: [none], length: 24

11:48:23.301574 28:99:3a:3b:15:39 > 44:4c:a8:a5:11:40, ethertype IPv4 (0x0800), length 66: 180.13.1.100.57657 > 180.13.1.1.bfd-control: BFDv1, Control, State Down, Flags: [none], length: 24

Same observation on R1, only receiving no sending

R1.13:53:18(config)#bash tcpdump -ni et11_36_2 udp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on et11_36_2, link-type EN10MB (Ethernet), capture size 262144 bytes
13:55:10.568533 28:99:3a:3b:15:39 > 44:4c:a8:a5:11:40, ethertype IPv4 (0x0800), length 66: 180.13.2.100.57657 > 180.13.2.1.bfd-control: BFDv1, Control, State Down, Flags: [none], length: 24

13:55:11.445074 28:99:3a:3b:15:39 > 44:4c:a8:a5:11:40, ethertype IPv4 (0x0800), length 66: 180.13.2.100.57657 > 180.13.2.1.bfd-control: BFDv1, Control, State Down, Flags: [none], length: 24

ISIS should notify the BFD to establish session after ISIS neighbor up. So the symptom is narrowed down to ISIS neighbor issue.

Step 3: Tcpdump ISIS IIH (isis hello) on both sides

-- show ver | grep MAC to get system MAC --

R1#show ver | grep MAC
System MAC address: 444c.a8a5.1140

R2#show ver | grep MAC
System MAC address: 2899.3a3b.1539

R2: IIH 2-way and looks good, R1/R2 send IIH every 10 seconds

-- bash tcpdump -ni <intf> iih --
[admin@R2 ~]$ tcpdump -ni et11_1 iih
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on et11_1, link-type EN10MB (Ethernet), capture size 262144 bytes
11:18:16.104275 44:4c:a8:a5:11:40 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0001, length 1497
11:18:18.000062 28:99:3a:3b:15:39 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0103, length 1497
11:18:24.723990 44:4c:a8:a5:11:40 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0001, length 1497
11:18:27.597320 28:99:3a:3b:15:39 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0103, length 1497

R1: IIH, only 1 way, from R1 to R2, no R2 to R1

[admin@bn303 ~]$ tcpdump -ni et11_36_2 iih
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on et11_36_2, link-type EN10MB (Ethernet), capture size 262144 bytes
11:23:53.268387 44:4c:a8:a5:11:40 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0001, length 1497

11:24:02.823894 44:4c:a8:a5:11:40 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0001, length 1497

IIH packets are destinated to well-known LAN broadcast address (0900:2b00:0005). These packets can't reach from R2 to R1.

Step 4: Arping

Arping shows similar, R1 to R2 is good, but seeing broadcast packets (all ff) drop on the direction of R2 to R1 even with arp ping

* Arp ping is to send our brocast L2 arp request to probe hosts

[admin@R1 ~]$ arping -b -I et11_36_1 180.13.1.100
ARPING 180.13.1.100 from 180.13.1.1 et11_36_1
Unicast reply from 180.13.1.100 [28:99:3A:3B:15:39] 419.967ms
Unicast reply from 180.13.1.100 [28:99:3A:3B:15:39] 409.205ms
Unicast reply from 180.13.1.100 [28:99:3A:3B:15:39] 401.616ms
Unicast reply from 180.13.1.100 [28:99:3A:3B:15:39] 410.797ms
^CSent 4 probes (4 broadcast(s))
Received 4 response(s)

[admin@R2 ~]$ arping -b -I et11_1 180.13.1.1
ARPING 180.13.1.1 from 180.13.1.100 et11_1
^CSent 9 probes (9 broadcast(s))

Received 0 response(s)

[admin@R2 ~]$ tcpdump -ni et11_1 arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on et11_1, link-type EN10MB (Ethernet), capture size 262144 bytes
11:46:04.859574 28:99:3a:3b:15:39 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 180.13.1.1 (Broadcast) tell 180.13.1.100, length 28

11:46:05.859645 28:99:3a:3b:15:39 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 180.13.1.1 (Broadcast) tell 180.13.1.100, length 28

At this point, the issue is narrowed down to, broadcast packets from R2 to R1 are dropped unexpectedly, which causes R1 not seeing R2's IIH packets, then no ISIS/BFD neighbor.

Step 5: hardware issue, but who drops the packets?

Ok, now we can say, most likely the problem is hardware, but who? The best way to nail down the culprit is to mirror packets to cpu, to see if the broadcast packets leave the R2.

* The feature of mirroring interface packets to CPU is only supported on Sand (Jericho/Arad) platform.

R1.14:32:14(config)#monitor session 1 source ethernet 11/36/1 rx
R1.14:32:19(config)#monitor session 1 destination cpu

Seeing IIH from R2 to R1 on the mirror cpu. << This is the packets received on the wire of et11/36/1

R1.14:32:20(config)#bash tcpdump -ni mirror0 iih
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on mirror0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:33:04.105249 28:99:3a:3b:15:39 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0103, length 1497
14:33:13.257565 28:99:3a:3b:15:39 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0103, length 1497
14:33:21.554871 28:99:3a:3b:15:39 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0103, length 1497

But not on the physical ethernet port << This is the packets received by the kernel via et11/36/1

R1.14:33:22(config)#bash tcpdump -ni et11_36_1 iih
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on et11_36_1, link-type EN10MB (Ethernet), capture size 262144 bytes
14:33:35.056748 44:4c:a8:a5:11:40 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0001, length 1497

14:33:42.990686 44:4c:a8:a5:11:40 > 09:00:2b:00:00:05, 802.3, length 1500: LLC, dsap OSI (0xfe) Individual, ssap OSI (0xfe) Command, ctrl 0x03: OSI NLPID IS-IS (0x83): p2p IIH, src-id 0000.0000.0001, length 1497

Now we know the broadcast packets are dropped on R1 between eth and kernel.

Step 6: Counters

And this is approved by the counter output on both R1 and R2

R2/Trident - No loss

R2#clear counters
R2#clear platform trident counters
R2#sh cpu counters queue | nz

--------------------------------------------------------------------------------
Linecard0/0
--------------------------------------------------------------------------------
Queue Counter/pkts* Drops/pkts
--------------- ------------------- -------------------
OSPF/ISIS 2 0
BFD 10 0
R2#sh platform trident counters interface e11/1 | egrep -i 'card|drop' | nz
R2#

R1/Jericho - Loss seen

R1#clear hardware counter drop
R1#clear counters

R1#sh cpu counters queue | grep 'Et11/36/1' | nz

CoppSystemLldp Et11/36/1 4 924 0 0

R1#sh hardware counter drop | grep o11/1
A Jericho11/1 DeqDeletePktCnt : 39148 : 2019-04-16 11:59:39 : 2019-04-16 11:59:39
P Jericho11/1 dropVoqInNullRoute : 3800339040 : 2019-04-16 11:59:39 : 2019-04-16 11:59:39

R2#sh hardware counter drop | grep o11/1
A Jericho11/1 DeqDeletePktCnt : 39153 : 2019-04-16 11:59:39 : 2019-04-16 11:59:45
P Jericho11/1 dropVoqInNullRoute : 3800605701 : 2019-04-16 11:59:39 : 2019-04-16 11:59:45

So, should goes to DeqDeletePktCnt or dropVoqInNullRoute. What does they mean?

3/22/2019

L2 Control Packet Fwding on J-based system

[Update, Nov 1, 2019] Starting from 4.22.0F, EOS can support multiple L2 protocols packets but NEED to create/apply a TCAM profile.

The L2 control packets like LACP/LLDP/STP are trapped by a switch and sent to CPU for processing. But for some use cases, this behavior requires to be changed. For example, In the below diagram, the CE1 and CE2 want to form a port-channel across the service provider's MPLS cloud network.

Another use case is the create an L2 harness switch. As shown below, R1 and R2 can establish a port-channel across a harness switch transparently.

Starting from Arista EOS 4.21.3F, a feature called - L2 protocol forwarding was introduced. With that, above requirement can be achieved.

The configurations are quite straightforward,
1) Step 1 - create l2-protocol forwarding profile and enable lacp forward
2) Step 2 - apply this profile under all the client facing interfaces
3) Step 3 - configure LDP type 5 pw - local patch to stitch 2 local eth ports

1. The port-channel is up.

R1.gts425.15:54:21#sh port-ch 100
Port Channel Port-Channel100:
Active Ports: Ethernet21 Ethernet22

2. "show lldp neighbor" shows remote and intermediate devices

R1.gts425.16:15:26#sh lldp nei et21, 22
Port Neighbor Device ID Neighbor Port ID TTL
Et21 lp232 Ethernet12/1 120
Et21 R2.hss457 Ethernet21 120
Et22 lp232 Ethernet12/2 120
Et22 R2.hss457 Ethernet22 120

Limitation:

In 4.21.3F, only LACP/LLDP supported. STP and other L2 protocols will be supported in later releases.
From 4.22.0F, more protocols are supported, but need to create a tcam profile.

3/14/2019

Arista BGP Peering Options (5) - v6 link-local peering/NH

Arista TOI Link

Compared with "v4 over v6 NH", the configuration is quite similar, except the interface address and neighbor use link-local address.

So the benefits here are obvious:

Minimizing the configuration efforts, no need ipv4 and global ipv6 address.
IPv4 NLRI over IPv6 link-local NH.

Because the link-local address is a local address and not accessible by remote devices, the bgp configure is considered to be a special one. The following regular bgp knobs don't work as usually:

iBGP
eBGP multi-hop
confed BGP
local AS if changing to iBGP
Link-local eBGP learnt routes are not advertised to iBGP peers, because of lacking NH address

Please note that this is ONLY supported in Gated, not in multi agent mode.

3/13/2019

Arista BGP Peering Options (4) - IPv4 NRLI over IPv6 NH

This blog is about the v4 NRLI over v6 NH. Based on Arista 4.17.0F TOI, this feature comprises 2 parts:

Compared with previous configurations, 2 configures are needed:

IPv4 forwarding on IPv6 interface, which is enabled by "ip routing ipv6 interfaces vrf v3"
neighbor <ipv6Nei> next-hop address-family ipv6 originate

Another way of 2nd part is to enable "bgp next-hop address-family ipv6". But in the above topology, the NRLI is originated directly, so have to put originate.

From the output of "show ip route", you can see the v4 route's NH is the ipv6 address

R1.lp231.20:13:59(config-router-bgp-vrf-v3)#sh ip route bgp detail

VRF: v3

B E 103.2.2.0/24 [200/0]

via 12:12:103::2, Vlan103

R1.lp231.20:14:07(config-router-bgp-vrf-v3)#sh ip bgp 103.2.2.0

BGP routing table information for VRF v3

Router identifier 103.2.2.2, local AS number 1

BGP routing table entry for 103.2.2.0/24

Paths: 1 available

12:12:103::2 from 12:12:103::2 (103.2.2.2)

Origin IGP, metric 0, localpref 100, IGP metric 1, weight 0, received 00:18:46 ago, valid, external, best

Rx SAFI: Unicast

Arista BGP Peering Options (3) - IPv6 NRLI over IPv4 Peering

This peering configuration is the reverse version of the previous one. So the configuration-wise, it is very similar.

The benefit is clear, we only need 1 BGP peering to carry both address families. But we still need to keep a dual stack on the interfaces to be used as NH. In today data center network, ECMP is very popular and wide. Evening using /31, the finest network mask, a /24 can only afford 128 links.

And this youtube explains very well on the requirement behind RFC 5549, ipv4 over v6 NH, which is shown in the next blog

Arista BGP Peering Options (2) - IPv4 NRLI over IPv6 Peering

Here is the TOI link

On the top of regular IPv6 BGP configuration, you need to configure 2 things:

1) Enable IPv4 NLRI over v6 peering and 2 ways to do it:

Under ipv4 address-family, activate it on ipv6 peering, which is shown in R1's configuration
Global configuration of "bgp default ipv4-unicast transport ipv6" in R2's show run

2) Specify the IPv4 NH of IPv4 prefixes, and we can do it 3 ways:

Global automatic NH address - "bgp auto-local-addr" in R'2 configuration. But this is a global knob, so if you have IPv6 prefixes over IPv4 NH, it will break peering silently.
Neighbor-base automatic NH address - "neighbor 12:12:101::1 auto-local-addr"
Neighbor-base manual NH address in R1's configuration - "neighbor 12:12:101::2 local-v4-addr 12.12.101.1"

Here is the output of "sh ip bgp 101.2.2.0" (I enable "routing-context vrf v1", so no need vrf)

R1.lp231.16:28:58(config)#sh ip bgp 101.2.2.0

BGP routing table information for VRF v1

Router identifier 101.1.1.1, local AS number 1

BGP routing table entry for 101.2.2.0/24

Paths: 1 available

12.12.101.2 from 12:12:101::2 (101.2.2.2)

Origin IGP, metric 0, localpref 100, IGP metric 1, weight 0, received 00:02:03 ago, valid, external, best

Rx SAFI: Unicast

And both ipv4/ipv6 ping work as expected.

R1.lp231.16:29:11(config)#ping 101.2.2.2

PING 101.2.2.2 (101.2.2.2) 72(100) bytes of data.

80 bytes from 101.2.2.2: icmp_seq=1 ttl=64 time=0.242 ms

80 bytes from 101.2.2.2: icmp_seq=2 ttl=64 time=0.100 ms

80 bytes from 101.2.2.2: icmp_seq=3 ttl=64 time=0.093 ms

80 bytes from 101.2.2.2: icmp_seq=4 ttl=64 time=0.091 ms

80 bytes from 101.2.2.2: icmp_seq=5 ttl=64 time=0.093 ms

--- 101.2.2.2 ping statistics ---

5 packets transmitted, 5 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.091/0.123/0.242/0.060 ms, ipg/ewma 0.174/0.180 ms

R1.lp231.16:31:39(config)#ping ipv6 101:2:2::2

PING 101:2:2::2(101:2:2::2) 72 data bytes

80 bytes from 101:2:2::2: icmp_seq=1 ttl=64 time=0.262 ms

80 bytes from 101:2:2::2: icmp_seq=2 ttl=64 time=0.106 ms

80 bytes from 101:2:2::2: icmp_seq=3 ttl=64 time=0.103 ms

80 bytes from 101:2:2::2: icmp_seq=4 ttl=64 time=0.099 ms

80 bytes from 101:2:2::2: icmp_seq=5 ttl=64 time=0.103 ms

--- 101:2:2::2 ping statistics ---

5 packets transmitted, 5 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.099/0.134/0.262/0.064 ms, ipg/ewma 0.184/0.196 ms

Arista BGP Peering Options (1)

In this series of blogs, I will discuss a bit on the Arista EOS BGP peering options. As of March 2019, the EOS has the following peering options:

Arista EOS BGP peering features

Here is the Arista official TOI links:

So, why do we need them, what's the difference and how to configure them?

1) Why
Today's network needs to support dual address spaces - IPv4 and IPv6, which requires providers to run 2 parallel control/forward planes. So the purpose of all the above features is to decrease the complexity by reducing control plane even forward plane.

For example, a data center with public ipv4/ipv6 address and needs support worldwide ipv4 and ipv6 access. Is that possible to use ipv4/ipv6 address space at the server pool and service routers, and only ipv6 infrastructure? The answer is yes.

2) What's the difference between the above 3 options?

The basic way is to have dual ipv4/ipv6 infrastructures and control planes, which means, 2 BGP sessions and 2 address planes.
IPv4 NRLI over IPv6 Transport is to send v4 prefixes over v6 peering. So it saves 1 just 1 control plane - ipv6 bgp but still needs ipv4 next hop as ipv4 prefixes, which means 2 data planes or dual v4/v6 stack.
RFC 5549 - IPv4 NRLI over IPv6 NH, this feature is one step further, ipv4 prefixes can use ipv6 NH. So 1 BGP session and 1 data planes (v6 infrastructure)
BGP IPv6 link-local peering, this utilizes the ipv6 link-local address to establish BGP peering w/o assigning global IPv6 interface addresses.

3) How to configure them? In the following blogs, I will show the configuration 1 by 1. (Please note that I use vrf under router bgp, in order to configure them together in one place. But it is not required and most of the time, you only need ONE)

3/05/2019

Install Arista EOS Swix

Reference:

Arista's - "Package RPM into an Arista swix"
Arista Extension ylb.jp

In the Linux world, RPM package is used for software installation. Simply put, the swix (software image extension) is a software package which contains multiple RPM files and a manifest file, which is used to install software or software patch on Arista EOS.

Generally, a swix is installed in the following steps:

1. copy to local extension directory - "scp, copy"

2. install - "extension"

3. copy them to boot-extension - "copy"

4. if a modular system, install on the standby supervisor - "session"

5. restart agents or reset hardware components if required - "agent terminate"

Additionally,

1. uninstall - "no extension"

2. remove packet - "delete"

3. If both sup running mismatched swix, login console and reset boot-extension file

Below is the step by step:

Step 1 - copy

[admin@eosrouter flash]$ scp solomonyang@server:/users/solomonyang/swix/*swix /mnt/flash

patch.swix 100% 11MB 11.4MB/s 00:00

eosrouter#copy flash:patch.swix extension:

eosrouter#show extensions

Name Version/Release Status Extension

----------------------------- -------------------- ----------- ---------

patch.swix 1.0.4 . A, NI 6

A: available | NA: not available | I: installed | NI: not installed | F: forced

eosrouter#show extensions detail

Name: patch.swix

Version: 1.0.4

Release: blahblah

Presence: available

Status: not installed
Summary: SandFap agent related libs

Packages:

Total size: 0 bytes

Description: SandFap agent related libs.

Step 2 - install

eosrouter#extension patch.swix

eosrouter#show extensions

Name Version/Release Status Extension

----------------------------- -------------------- ----------- ---------

patch.swix 1.0.4 A, I 6

Step 3 - copy to boot extension

eosrouter#show installed-extensions

patch.swix

eosrouter#show boot-extensions

eosrouter#copy installed-extensions boot-extensions

Copy completed successfully.

eosrouter#show boot-extensions

patch.swix

Step 4 - install them on standby sup

eosrouter#copy flash:patch.swix supervisor-peer:/mnt/flash

Copy completed successfully.

eosrouter#session peer-supervisor

Warning: Permanently added '[127.1.0.2]:3601' (RSA) to the list of known hosts.

WARNING - you are currently logged in to the standby supervisor.

Not all cli commands are available or supported. Configuration mode

is disabled for SSO.

s1:eosrouter#copy flash:patch.swix extension:

Copy completed successfully.

s1:eosrouter#extension patch.swix

s1:eosrouter#copy installed-extensions boot-extensions

Copy completed successfully.

Step 5 - restart agents or reset hardware component

For some software patches, it is required to restart software agents or hardware components to make patch effective.

eosrouter#show agent names | grep SandFap

SandFap-Linecard3

SandFap-Linecard4

SandFap-Linecard5

eosrouter#agent SandFap-Linecard3 SandFap-Linecard4 SandFap-Linecard5 terminate

SandFap-Linecard3 was terminated

SandFap-Linecard4 was terminated

SandFap-Linecard5 was terminated

Also you need to do the same on the standby sup

How to fix unmatched swix'ed sup

If you run into unmatched swix supervisor due to mis-configuration, the way to fix them is as follows:

1. Login in the console of both supervisor. Sometimes the standby sup is at bad shape, then you have to fix it in Aboot mode

Arista Networks EOS 4.20.5.2F

localhost login: admin

[PyServer ar.Aaa not responding, still trying -- is it running?]

2. Reboot both and press ctrl-C to enter Aboot mode

3. Vi /mnt/flash/boot-extensions to remove swix entries. Then reboot so you will have a clean EOS w/o any extensions.

Arista EOS - dot1q tunnel

There are 2 common use cases for dot1q tunnel:
1) Service providers tunnel customer's VLAN-tag traffic across different sites;
2) Lab harness switch for flexible connectivity between devices

But with one limitation: L2 control packets can't traverse. For example, "show lldp neighbor" on cd644 indicates yo412 instead of remote router.