Arista EOS/Networking Tips by Solomon Yang: October 2018

10/31/2018

sum dropped pkts in "show platform jericho counter drop"

bn302.12:51:12#show platform jericho counters drops | grep "Counts Match" | awk '{print $9}' | awk '{sum += $1} END {print sum}'
52508425

10/30/2018

EOS: collect data and save to flash

bash dmesg | gzip > /mnt/flash/dmesg-$HOSTNAME-$(date +%Y-%m-%d_%H-%M).gz

show tech-support | gzip > /mnt/flash/shtech-$HOSTNAME-$(date +%Y-%m-%d_%H-%M).gz

show log sys | cat > /mnt/flash/shlogsys-$HOSTNAME-$(date +%Y-%m-%d_%H%M).txt

show agent log | gzip > /mnt/flash/shagentlog-$HOSTNAME-$(date +%Y-%m-%d_%H-%M).gz

show agent qt | gzip > /mnt/flash/shagentqt-$HOSTNAME-$(date +%Y-%m-%d_%H-%M).gz

10/25/2018

Arista EOS: Change the default shape and bandwidth of CoPP class

The purpose of CoPP is to protect the CPU from unnecessary or malicious packets. Arista EOS has a default CoPP set with pre-defined bandwidth and shape.

R1#sh policy-map type control-plane copp-system-policy
Service-policy input: copp-system-policy
Number of units programmed: 8
Hardware programming status: Successful

Class-map: copp-system-arp-inspect (match-any)
shape : 250000 kbps
bandwidth : 25000 kbps

<....>

Most of the class names are intuitive, some need a little explanation.

l3destmiss - unicast routed packets but no ARP info, need ARP resolution.
l3lpmoverflow - ip broadcast destined to router???
linklocal - multicast packets to 224.0.0.*
l3slowpath - header option

How to change:

bn302.15:27:12(config)#policy-map type control-plane copp-system-policy

bn302.15:27:29(config-pmap-control-plane-copp-system-policy)#class copp-system-l3destmiss

bn302.15:27:43(config-pmap-c-control-plane-copp-system-policy-copp-system-l3destmiss)#shape kbps 999999

bn302.15:27:49(config-pmap-c-control-plane-copp-system-policy-copp-system-l3destmiss)#bandwidth kbps 88888

Policy map changes are applied when existing configuration mode. Use 'abort' to ignore any changes.

How to verify:

bn302.16:47:57#show policy-map interface control-plane copp-system-policy

....

Class-map: copp-system-l3destmiss (match-any)

shape : 999999 kbps

bandwidth : 88888 kbps

bn302.15:29:48(config-cp)#sh platform jericho copp mapping | egrep 'Shape|destmiss'

Class name SysPhyPort First Voq# Total Voqs Bandwidth(kbps) Shape(kbps)

copp-system-l3destmiss 36 1680 0-40 250 2500

10/23/2018

Arista EOS Tunneling (3) - IPinIP

IP-in-IP format:

Size = 20B, just an ip header,
Protocol# = 4

Digram and configuration:

Verification:

R4.cd681#sh ip decap-group
NOTE: "D" column indicates dynamic entries
D | Name | Type | Info | Version | Address Type
--|--------------|----------------|--------------------|---------|-------------
| ipinip | IP-in-IP | 100.20.255.4 | IPv4 |

R4.cd681#show platform trident l3 shadow tunnel encap raw
EGR IP Tunnel:
--------------------------------------------------------------------------------
maxEntries: 512, numEntries: 1
EGR_IP_TUNNEL Address: 0xd72817a4, magic:aeaeaeae, maxEntries: 512, numEntries: 1

EGR_IP_TUNNEL[2]: 0x01 | 0x1 0x0 0x0 0x100000 0x9053fc11 0x9053fc0d 0x1 0x0

R4.cd681#show platform trident l3 shadow tunnel encap
EGR IP Tunnel:
--------------------------------------------------------------------------------
Unit Id TnlType EntType TTL DscpSel SrcIp DstIp
0 2 0 IPV4 64 1 100.20.255.4 100.20.255.3

R4.cd681#show nexthop-group
ipip
Id 1
Type ipInIp
Size 1
TTL 64
Source IP 100.20.255.4 (Loopback20)
Entries (left most label is the top of the stack)
0 100.20.255.3
Via 100.20.34.3, Ethernet31 (100.20.255.3/32)
00:1c:73:54:27:cb, Ethernet31

10/22/2018

Arista EOS Tunneling (2) - GRE

Supported and Release:
The GRE Tunneling in EOS is supported from 4.15.0F (Apr 2015)

Not like the interface tunnel configuration in Cisco OS's, Arista uses nexthop-group and decap-group as the encap and decap endpoints. (EOS also has tunnel interfaces, but it is performed by software and ONLY for control plane purpose).

GRE Header Info:

Protocol #: 47 (0x2F)
Between Outer IP Header and Payload packet:

GRE Flags (2B)
Protocol Type (2B) = 0x0800 (ipv4)
Key (4B) if enabled tunnel-key shown as below

Diagram and configuration:

Configuration details:

nexthop-group <tunnel_intf_name> type gre

on Arad platform: there is 4 encap - gre, ip-in-ip, mpls and mpls-over-gre

size 1:

# of ecmp path

tunnel-source intf <intf, loopback most of time>

Source ip of outer GRE/IP header

entry 0 tunnel-destination <ip>

Dest ip of GRE/IP header
match above size #, starting from 0

tunnel-key ingress-interface

if enabled, will add 4B of GRE key value after 4B GRE header(flag + protocol#). This key is calculated from ingress interface + vlan id.

Show output:

R1.yo412#sh ip decap-group

NOTE: "D" column indicates dynamic entries

--|--------------------|------------|-------------------|---------|-------------

| gre-tunnel-1 | GRE | 100.1.255.1 | |

R1.yo412#sh platform arad decap-group

DecapIp | LIF

------------------------+-------

100.1.255.1 | 1

R1.yo412#show nexthop-group

gre-tunnel-1

Id 2

Type gre (key: ingress-interface)

Size 1

TTL 64

Source IP 100.1.255.1 (Loopback1)

Counters shared

Entries (left most label is the top of the stack)

0 100.1.255.2

Via 100.1.12.2, Ethernet7/36/1 (100.1.255.2/32)

00:1c:73:44:58:d0, Ethernet7/36/1

R1.yo412#sh platform arad ip nexthop-group

gre-tunnel-1

Type: GRE (key: ingress-interface)

Ecmp Fec Id: -

---------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------

|0 |100.1.255.2 |100.1.255.1 [0 ]|64 [0 ]|1047 |6144 |GRE |Et7/36/1 |3983|00:1c:73:44:58:d0|

R1.yo412#sh platform arad ip route | egrep '222|VRF|ID'

|0 |222.100.0.0/16 |ROUTE| Et5/1/1 |3984 |4094 | 00:1c:73:3b:ce:c9 | - |1046 | -

|0 |222.200.0.0/16 |ROUTE| Et7/36/1 |3983 | - | 00:1c:73:44:58:d0 | - |1047 |G 100.1.255.2

Arista EOS Tunneling (1)

Arista EOS Tunneling is a little different from other vendors' implmentations

Encap and decap are separated, which are defined in nexthop-group and decap-group.

On 1 router configure nexthop-group and on other router only has decap group.
For example, you can have internet traffic to FW, which tunnels the packets to severs. And servers directly route the returning traffic to users.

nexthop-group does NOT equal to tunnel interface. It is not an interface, actually it is encap endpoint.
Starting from 4.21.1F, EOS support hw GRE tunnel interface. Only on Jericho platforms - 7020R, 7280R, 7500R.

Tunneling vs platforms(chipset) and releases.

Sand platform - Arad/Jericho/J+/Qurman

GRE encap + decap
IPinIP decap only

Strata platform - Trident 2/Thomahawk

IPinIP encap + decap

In this quick lab, we will configure the GRE and IPinIP tunnels in the following topology:

Use ONE router - RtrAB to simulate end hosts by dividing into VRFs.
222.10.10.1 -- 222.10.20.1 over GRE tunnel
222.20.10.1 -- 222.20.20.1 over IPinIP tunnel

10/15/2018

FB's back bone network - EBB

https://code.fb.com/networking-traffic/building-express-backbone-facebook-s-new-long-haul-network/

Why need EBB? Inter-DC traffic growth is much faster than internet egress traffic and WAN technology evolution.

What needed for the new EBB?

Incremental deployment of software features. 不是一步到位，而且可以roll back.
NO RSVP-TE, because of inefficiency and complexity. 嗯，锤！
Use MPLS SR so network state lean. 这倒是真的，SR分离CP和Local segment management.

How

Use DC network ideas

Small routing table - on the order of K routes
4 planes.

3 Steps:

IGP + full-mesh iBGP for packet forwarding
TE + controller
OPEN/R to replace IGP

Components:

3 server-side:

SFlow - read
BGP route injector - write
Controller - think

2 device-side:

Open/R = IGP
SR = forwarding

还有clean fault domain splitting, 这是咋做的？

Software Design:

Hybrid approach: centralized controller + SR, best way

path computation is at central level with global view
network failure is handled at distributed level.

Open/R

how to open/R on arista devices?

Traffic estimator:

SFlow, need src/dst pair
what is the sample rate? 1 of 1000? how accurate

LSP agent:

With thrift-based API (eapi on eos) to program hw
In future? need failure detection and fails over within hundreds of msec.

Next step:

per-service model
scheduler for bulk transfer

10/10/2018

20+ sec traffic seen during Mlag peer reload

https://eos.arista.com/mlag-issu/

The reason of such long period loss is most likely caused by STP flap or the STP is not stable before system reload. Another symptom is the output of "show hardware counter drop". If you see high # of drop due to dropVoqInPortNotVlanMember, it is likely the STP issue.

Use these 2 commands to verify STP status:

bn303.14:47:42(config)#show mlag issu warnings
Stp is not restartable. Topology changes will occur during the upgrade
process.

bn303.14:49:21(config)#show spanning-tree bridge det | more
Stp Detailed Status:
Stp agent restartable : False
MST-PVST interoperation : Disabled

One of the reasons of unstable STP is, this switch receives its own BPDU on one port. That happens if both mlag peers have 1 port connected to same switch then form a loop. Even the STP kicks in and put the port in "discarding" mode, but this bouncing back BPDU still causing the STP unstable.

In this case, use command - "show spanning topology status" to find those looped interfaces.

bn302.14:49:44(config)#show spanning-tree topology status | more
Topology: Cist
Mapped Vlans: 1
Ethernet12/32/1: forwarding
Ethernet14/32/1: forwarding
PeerEthernet12/32/1: discarding <<<<<< check this peer port
Port-Channel2000: forwarding

bn303.15:32:02(config)#tcpdump ver interface et12/32/1
....
15:32:21.036169 28:99:3a:84:8d:68 > 01:80:c2:00:00:00, 802.3, length 105: LLC, dsap STP (0x42) Individual, ssap STP (0x42) Command, ctrl 0x03: STP 802.1s, Rapid STP, CIST Flags [Proposal, Learn, Forward, Agreement], length 102
port-role Designated, CIST root-id 2000.46:4c:a8:a5:11:40, CIST ext-pathcost 0
CIST regional-root-id 2000.46:4c:a8:a5:11:40, CIST port-id 8789,
message-age 0.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
v3len 64, MCID Name , rev 0,
digest 64b4d42c08d6bff5b647fc0be11d4ebb, CIST int-root-pathcost 0,
CIST bridge-id 2000.46:4c:a8:a5:11:40, CIST remaining-hops 20