Arista EOS/Networking Tips by Solomon Yang: December 2018

12/28/2018

EOS - CPU monitor session

monitor session 1 source Ethernet3/1
monitor session 1 destination Cpu

wa463.vtepB2(config)#show monitor sess

Session 1
------------------------
Source Ports:
Both: Et3/1
Destination Ports:
Cpu : active (mirror0) <<< use this interface to tcpdump

wa462.vtepB1#bash tcpdump -nvvi mirror0 udp
tcpdump: listening on mirror0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:56:32.121346 44:4c:a8:a5:11:40 > 44:4c:a8:97:8c:51, ethertype IPv4 (0x0800), length 164: (tos 0x0, ttl 63, id 47642, offset 0, flags [DF], proto UDP (17), length 150)
200.255.200.34.62729 > 200.255.200.56.4789: VXLAN, flags [I] (0x08), vni 2000
00:1c:73:ff:3c:30 > 44:4c:a8:97:72:b7, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 32537, offset 0, flags [none], proto ICMP (1), length 100)

12/27/2018

Arista EOS Tips - alias

1. Multiple line alias
alias ccc
10 clear arp
20 clear mac add dyn
30 bash clear

2. Alias argument

alias da bash tcpdump -nvvvi %1 arp

12/18/2018

Arista EOS: VARP - GARP

If you have the following VARP configuration:

ip virtual-router mac-address 00:00:33:33:44:44
ip virtual-router mac-address advertisement-interval 1
interface Vlan2000
ip address 20.0.134.14/16
ip virtual-router address 20.0.134.1

You will see the incoming GARP every 1 sec (collecting by "bash tcpdump -nvvi vlan2000 -w arp.cap")

Arista EOS

Use opcode=2 (reply) as GARP
SrcMAC = vMAC, DstMAC = all FF
Send/Target MAC/IP = vMAC/vIP
You see 2 GARP in 1 sec because of 2 mlag peers

12/17/2018

FB: A Billion user load balancer

https://www.usenix.org/conference/lisa16/conference-program/presentation/shuff

A 2016 presentation by PE - Patrick Shuff
Tb of egress traffic leaving FB routers, 85% from outside US

FB Req Flow vs Architecture

Client - DNS AAAA req for facebook.com
Client - Router
Router - ECMP to L4LB(ipvs)
L4LB(ipvs) - Http GET to L7LB(proxygen)
L7LB(proxygen) - Http GET to HHVM (websever)
Router + L4LB + L7LB + HHVM = a cluster/DC
Data flow:

router/ECMP --> L4LB (ipvs) --> L7LB(proxygen) --> HHVM
L4LB = 10+
L7LB = 100+
HHVM = 1000+

And L4LB/L7LB/HHVM are NOT dedicted servers or devices. All x86 servers dynamically allocated.

L4/L7 LB:

L4LB (ipvs) + xBGP, a python BGP dameon to talk to TOR to announce

ipv4:/32,
ipv6:/64

Router to L4LB, just ECMP hash
L4LB to L7LB, hash + state table
要考虑两种failures

1) L4LB down, 新的L4LB用同样的hash，到原来的HHVM;
2) L7LB down, TCP breaks, L4LB hash到不同的L7LB, 如果L7LB back up, 因为有state table，不会影响TCP

DSR = direct server access

return traffic bypass the L4LB, L7LB to router.
L4LB ==ipInip== L7LB
然后L7LB直接 original ipv4 packet back to client. 和微软很像

Speed up

用PoP early terminates客户TCP/SSL handshake，这样加快链接建立的速度，而不是到DC
Pop has direct SSL connections to DC.
需要看TCP (3-way), SSL (4-way)

DNS

Real time monitor

Arista DCS-7050QX-32S-F enable eth1-4

On the DCS-7050QX-32S-F, there are 4x10G SFP on the front panel. This 4 ports share the hardware with Eth5/1-4. So to switch the hardware of eth5/1 to eth1-4 of 10G, use this command

ck475.vtepX4(config)#hardware port-group 1 select ?
Et1-4 Activate ports Et1-4
Et5/1-4 Activate ports Et5/1-4

Arista EOS MLAG - domain name must be matched

snp262.vtepA1#sh run | sec mlag config
mlag configuration
domain-id vtepA
local-interface Vlan4094
peer-address 200.255.134.2
peer-link Port-Channel2000

snp263.vtepA2(config-mlag)#sh run sec mlag config
mlag configuration
domain-id vtepA2 <<<<< a typo in mlag domain-name
local-interface Vlan4094
peer-address 200.255.134.1
peer-link Port-Channel2000
reload-delay mode lacp standby

snp263.vtepA2#sh mlag det | grep State
State : inactive
Peer State : unknown
State changes : 3

snp263.vtepA2(config)#mlag config
snp263.vtepA2(config-mlag)#domain-id vtepA
snp263.vtepA2(config-mlag)#end

After correcting it, the mlag is up

snp263.vtepA2#sh mlag det | grep State
State : primary
Peer State : secondary
State changes : 4

12/03/2018

Arista EOS BGP wait-for-install and wait-for-convergence

"update wait-for-install"

Quite straightforward feature, update the prefixes until they are installed in hardware

Obviously, this prevents packet loss. Without this feature, routers start to advertise prefixes when hardware forwarding not ready yet. When its peers start to flood traffic to it, the only thing it can do is to drop them.
Very useful when working with MLAG, which requires the routers to put Mlag downstream interface to hold state during initialization.

"update wait-for-convergence"

https://aristaeos.blogspot.com/2018/03/arista-eos-bgp-update-wait-for.html
This is a feature for spine routers. The key point behind it is, don't advertise prefixes until fully convergence (which means you have full routing info).

Where to use them:

On MLAG or L2/L3 boundary, ONLY use wait-for-install
On spine, use both

FB: Eng Egress with Edge Fabric

https://research.fb.com/wp-content/uploads/2017/08/sigcomm17-final177-2billion.pdf?

PR's BGP connection types:

Transit: private link with dedicated b/w
Peers:

private peer: dedicated PN ?~= transit ?
public: via public fabric
route server: prefix redirected by RS and traffic via public fabric

How prefixes are preferred:

Prefer peer routes over transit (via local_pref), as_path tiebreaker

In/egress traffic over same path

If still tie, private peer > public > route server peer, using MED

to avoid cross-congestion over fabric

BGP multi-path vs. ECMP

Cisco BGP multipath doc
Requirement of path characteristics to be multipath

Weight
local_pref
as_path length
origin
MED
one of these:

neighbor AS or sub-as
as_path

BGP limitation

Not capacity-aware + ECMP

unbalances links get equal load

Static bgp policy likely optimizes traffic, but

as-path != performance

Avoid congestion

Input:

Prefix via BMP, BGP only has 1 best
controller does best selection
sFLOW, IPFIX, traffic info
SNMP, interface info

Output, via BGP update by using higher local_pref

Performance-based routing

servers set DSCP
PBR on PRs, 1 DSCP - 1 route (table?)
PR ISIS-SR/MPLS to ASW
eBPF - extended berkley packet filter

change pkt egress this server

to be continued.....

How to check # of BGP attributes

Check BGP export/shadown table

First enable "sflow extension bgp" then run command - "show bgp export-table path-attribute" or "show bgp shadown-table bgp-attribute" if older releases

bn302.17:01:15#show bgp export-table path-attribute
Bgp Attribute Info Table
key nextHop origin pathFlags originatorId aspType med localPref asPathAttrInfoId commListId extCommListId
64 210.100.22.1 0 16 0 2 0 0 3 4294967295 4294967295
80 2000:210:255:254:: 0 0 3539992321 1 0 100 4294967295 4294967295 4294967295
88 210.100.11.1 0 24 3539992321 1 0 100 2 4294967295 4294967295
56 210.100.22.1 2 16 0 2 0 0 2 4294967295 4294967295
20 2000:210:100:32::1 0 48 0 2 0 0 1 4294967295 4294967295

A new command:
bn302.17:01:08#bash smash -p ar/Smash/routing/bgp/export/pathAttrTableInfo/default/pathAttrEntry | egrep '^bucket' | wc -l
23

MAC 11:11:xx:.... is not valid MAC address

On IXIA if you configured MAC/Eth address to be 11:11:00:02:01:01, the switch doesn't take them.

Thanks Paul Lavelle for the why :-)

The least significant bit of the first byte is known as the IG bit. An IG bit of 0 indicates that this is a unicast MAC address, an IG bit of 1 indicates a multicast or broadcast address. It is illegal to use a multicast source MAC address in ARP replies.

References:

https://en.wikipedia.org/wiki/Organizationally_unique_identifier#Format

https://blog.ipspace.net/2012/05/arp-reply-with-multicast-source-mac-is.html

I use a simple trick: If you are manually setting a MAC address on Ixia, make sure the first byte of the address is an even number :)

12/02/2018

Arista EOS - customized CLI

https://www.arista.com/assets/data/pdf/Whitepapers/Arista_EOS_parser.pdf

Arista EOS CLI is implemented in Python, so you can customize the CLI yourself. Of course it is limited to simple output instead of complicated contents.

Step 1: Modify the CLI script

bn303.17:40:17#bash

Arista Networks EOS shell

[admin@bn303 ~]$ cd /usr/lib/python2.7/site-packages/CliPlugin/
[admin@bn303 CliPlugin]$ vi RoutingBgpShowCli.py
[admin@bn303 CliPlugin]$ sudo vi RoutingBgpCli.py

Step 2: save the new CLI script and load up during boot

copy the modified to /mnt/flash
the /usr/lib is file system in memory, unsustainable after reboot.
vi /mnt/flash/rc.eos, so copy the save script to the location