Arista EOS/Networking Tips by Solomon Yang

10/31/2018

sum dropped pkts in "show platform jericho counter drop"

bn302.12:51:12#show platform jericho counters drops | grep "Counts Match" | awk '{print $9}' | awk '{sum += $1} END {print sum}'
52508425

10/30/2018

EOS: collect data and save to flash

bash dmesg | gzip > /mnt/flash/dmesg-$HOSTNAME-$(date +%Y-%m-%d_%H-%M).gz

show tech-support | gzip > /mnt/flash/shtech-$HOSTNAME-$(date +%Y-%m-%d_%H-%M).gz

show log sys | cat > /mnt/flash/shlogsys-$HOSTNAME-$(date +%Y-%m-%d_%H%M).txt

show agent log | gzip > /mnt/flash/shagentlog-$HOSTNAME-$(date +%Y-%m-%d_%H-%M).gz

show agent qt | gzip > /mnt/flash/shagentqt-$HOSTNAME-$(date +%Y-%m-%d_%H-%M).gz

10/25/2018

Arista EOS: Change the default shape and bandwidth of CoPP class

The purpose of CoPP is to protect the CPU from unnecessary or malicious packets. Arista EOS has a default CoPP set with pre-defined bandwidth and shape.

R1#sh policy-map type control-plane copp-system-policy
Service-policy input: copp-system-policy
Number of units programmed: 8
Hardware programming status: Successful

Class-map: copp-system-arp-inspect (match-any)
shape : 250000 kbps
bandwidth : 25000 kbps

<....>

Most of the class names are intuitive, some need a little explanation.

l3destmiss - unicast routed packets but no ARP info, need ARP resolution.
l3lpmoverflow - ip broadcast destined to router???
linklocal - multicast packets to 224.0.0.*
l3slowpath - header option

How to change:

bn302.15:27:12(config)#policy-map type control-plane copp-system-policy

bn302.15:27:29(config-pmap-control-plane-copp-system-policy)#class copp-system-l3destmiss

bn302.15:27:43(config-pmap-c-control-plane-copp-system-policy-copp-system-l3destmiss)#shape kbps 999999

bn302.15:27:49(config-pmap-c-control-plane-copp-system-policy-copp-system-l3destmiss)#bandwidth kbps 88888

Policy map changes are applied when existing configuration mode. Use 'abort' to ignore any changes.

How to verify:

bn302.16:47:57#show policy-map interface control-plane copp-system-policy

....

Class-map: copp-system-l3destmiss (match-any)

shape : 999999 kbps

bandwidth : 88888 kbps

bn302.15:29:48(config-cp)#sh platform jericho copp mapping | egrep 'Shape|destmiss'

Class name SysPhyPort First Voq# Total Voqs Bandwidth(kbps) Shape(kbps)

copp-system-l3destmiss 36 1680 0-40 250 2500

10/23/2018

Arista EOS Tunneling (3) - IPinIP

IP-in-IP format:

Size = 20B, just an ip header,
Protocol# = 4

Digram and configuration:

Verification:

R4.cd681#sh ip decap-group
NOTE: "D" column indicates dynamic entries
D | Name | Type | Info | Version | Address Type
--|--------------|----------------|--------------------|---------|-------------
| ipinip | IP-in-IP | 100.20.255.4 | IPv4 |

R4.cd681#show platform trident l3 shadow tunnel encap raw
EGR IP Tunnel:
--------------------------------------------------------------------------------
maxEntries: 512, numEntries: 1
EGR_IP_TUNNEL Address: 0xd72817a4, magic:aeaeaeae, maxEntries: 512, numEntries: 1

EGR_IP_TUNNEL[2]: 0x01 | 0x1 0x0 0x0 0x100000 0x9053fc11 0x9053fc0d 0x1 0x0

R4.cd681#show platform trident l3 shadow tunnel encap
EGR IP Tunnel:
--------------------------------------------------------------------------------
Unit Id TnlType EntType TTL DscpSel SrcIp DstIp
0 2 0 IPV4 64 1 100.20.255.4 100.20.255.3

R4.cd681#show nexthop-group
ipip
Id 1
Type ipInIp
Size 1
TTL 64
Source IP 100.20.255.4 (Loopback20)
Entries (left most label is the top of the stack)
0 100.20.255.3
Via 100.20.34.3, Ethernet31 (100.20.255.3/32)
00:1c:73:54:27:cb, Ethernet31

10/22/2018

Arista EOS Tunneling (2) - GRE

Supported and Release:
The GRE Tunneling in EOS is supported from 4.15.0F (Apr 2015)

Not like the interface tunnel configuration in Cisco OS's, Arista uses nexthop-group and decap-group as the encap and decap endpoints. (EOS also has tunnel interfaces, but it is performed by software and ONLY for control plane purpose).

GRE Header Info:

Protocol #: 47 (0x2F)
Between Outer IP Header and Payload packet:

GRE Flags (2B)
Protocol Type (2B) = 0x0800 (ipv4)
Key (4B) if enabled tunnel-key shown as below

Diagram and configuration:

Configuration details:

nexthop-group <tunnel_intf_name> type gre

on Arad platform: there is 4 encap - gre, ip-in-ip, mpls and mpls-over-gre

size 1:

# of ecmp path

tunnel-source intf <intf, loopback most of time>

Source ip of outer GRE/IP header

entry 0 tunnel-destination <ip>

Dest ip of GRE/IP header
match above size #, starting from 0

tunnel-key ingress-interface

if enabled, will add 4B of GRE key value after 4B GRE header(flag + protocol#). This key is calculated from ingress interface + vlan id.

Show output:

R1.yo412#sh ip decap-group

NOTE: "D" column indicates dynamic entries

--|--------------------|------------|-------------------|---------|-------------

| gre-tunnel-1 | GRE | 100.1.255.1 | |

R1.yo412#sh platform arad decap-group

DecapIp | LIF

------------------------+-------

100.1.255.1 | 1

R1.yo412#show nexthop-group

gre-tunnel-1

Id 2

Type gre (key: ingress-interface)

Size 1

TTL 64

Source IP 100.1.255.1 (Loopback1)

Counters shared

Entries (left most label is the top of the stack)

0 100.1.255.2

Via 100.1.12.2, Ethernet7/36/1 (100.1.255.2/32)

00:1c:73:44:58:d0, Ethernet7/36/1

R1.yo412#sh platform arad ip nexthop-group

gre-tunnel-1

Type: GRE (key: ingress-interface)

Ecmp Fec Id: -

---------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------

|0 |100.1.255.2 |100.1.255.1 [0 ]|64 [0 ]|1047 |6144 |GRE |Et7/36/1 |3983|00:1c:73:44:58:d0|

R1.yo412#sh platform arad ip route | egrep '222|VRF|ID'

|0 |222.100.0.0/16 |ROUTE| Et5/1/1 |3984 |4094 | 00:1c:73:3b:ce:c9 | - |1046 | -

|0 |222.200.0.0/16 |ROUTE| Et7/36/1 |3983 | - | 00:1c:73:44:58:d0 | - |1047 |G 100.1.255.2

Arista EOS Tunneling (1)

Arista EOS Tunneling is a little different from other vendors' implmentations

Encap and decap are separated, which are defined in nexthop-group and decap-group.

On 1 router configure nexthop-group and on other router only has decap group.
For example, you can have internet traffic to FW, which tunnels the packets to severs. And servers directly route the returning traffic to users.

nexthop-group does NOT equal to tunnel interface. It is not an interface, actually it is encap endpoint.
Starting from 4.21.1F, EOS support hw GRE tunnel interface. Only on Jericho platforms - 7020R, 7280R, 7500R.

Tunneling vs platforms(chipset) and releases.

Sand platform - Arad/Jericho/J+/Qurman

GRE encap + decap
IPinIP decap only

Strata platform - Trident 2/Thomahawk

IPinIP encap + decap

In this quick lab, we will configure the GRE and IPinIP tunnels in the following topology:

Use ONE router - RtrAB to simulate end hosts by dividing into VRFs.
222.10.10.1 -- 222.10.20.1 over GRE tunnel
222.20.10.1 -- 222.20.20.1 over IPinIP tunnel

10/15/2018

FB's back bone network - EBB

https://code.fb.com/networking-traffic/building-express-backbone-facebook-s-new-long-haul-network/

Why need EBB? Inter-DC traffic growth is much faster than internet egress traffic and WAN technology evolution.

What needed for the new EBB?

Incremental deployment of software features. 不是一步到位，而且可以roll back.
NO RSVP-TE, because of inefficiency and complexity. 嗯，锤！
Use MPLS SR so network state lean. 这倒是真的，SR分离CP和Local segment management.

How

Use DC network ideas

Small routing table - on the order of K routes
4 planes.

3 Steps:

IGP + full-mesh iBGP for packet forwarding
TE + controller
OPEN/R to replace IGP

Components:

3 server-side:

SFlow - read
BGP route injector - write
Controller - think

2 device-side:

Open/R = IGP
SR = forwarding

还有clean fault domain splitting, 这是咋做的？

Software Design:

Hybrid approach: centralized controller + SR, best way

path computation is at central level with global view
network failure is handled at distributed level.

Open/R

how to open/R on arista devices?

Traffic estimator:

SFlow, need src/dst pair
what is the sample rate? 1 of 1000? how accurate

LSP agent:

With thrift-based API (eapi on eos) to program hw
In future? need failure detection and fails over within hundreds of msec.

Next step:

per-service model
scheduler for bulk transfer

10/10/2018

20+ sec traffic seen during Mlag peer reload

https://eos.arista.com/mlag-issu/

The reason of such long period loss is most likely caused by STP flap or the STP is not stable before system reload. Another symptom is the output of "show hardware counter drop". If you see high # of drop due to dropVoqInPortNotVlanMember, it is likely the STP issue.

Use these 2 commands to verify STP status:

bn303.14:47:42(config)#show mlag issu warnings
Stp is not restartable. Topology changes will occur during the upgrade
process.

bn303.14:49:21(config)#show spanning-tree bridge det | more
Stp Detailed Status:
Stp agent restartable : False
MST-PVST interoperation : Disabled

One of the reasons of unstable STP is, this switch receives its own BPDU on one port. That happens if both mlag peers have 1 port connected to same switch then form a loop. Even the STP kicks in and put the port in "discarding" mode, but this bouncing back BPDU still causing the STP unstable.

In this case, use command - "show spanning topology status" to find those looped interfaces.

bn302.14:49:44(config)#show spanning-tree topology status | more
Topology: Cist
Mapped Vlans: 1
Ethernet12/32/1: forwarding
Ethernet14/32/1: forwarding
PeerEthernet12/32/1: discarding <<<<<< check this peer port
Port-Channel2000: forwarding

bn303.15:32:02(config)#tcpdump ver interface et12/32/1
....
15:32:21.036169 28:99:3a:84:8d:68 > 01:80:c2:00:00:00, 802.3, length 105: LLC, dsap STP (0x42) Individual, ssap STP (0x42) Command, ctrl 0x03: STP 802.1s, Rapid STP, CIST Flags [Proposal, Learn, Forward, Agreement], length 102
port-role Designated, CIST root-id 2000.46:4c:a8:a5:11:40, CIST ext-pathcost 0
CIST regional-root-id 2000.46:4c:a8:a5:11:40, CIST port-id 8789,
message-age 0.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
v3len 64, MCID Name , rev 0,
digest 64b4d42c08d6bff5b647fc0be11d4ebb, CIST int-root-pathcost 0,
CIST bridge-id 2000.46:4c:a8:a5:11:40, CIST remaining-hops 20

9/28/2018

Nexthop Group (2)

NHG supported features are different per Arista platforms:

This is DCS-7060CX based on Tomahwak/Strata, only supports IP next hop, ip-in-ip and mpls

DCS-7060CX(config)#nexthop-group test type ?
ip ecmp next hop
ip-in-ip ip-in-ip tunnel type
mpls mpls tunnel type

This is DCS-7512N based on Jericho/Sand, which supports GRE, ip next hop, mpls and mpls-over-gre.

DCS-7512N(s1)(config)#nexthop-group test type ?
gre gre tunnel type
ip ecmp next hop
mpls mpls tunnel type
mpls-over-gre mpls-over-gre tunnel type

Nexthop Group (1)

为什么需要一个这样的功能？比方说，以下一个典型的JY网络

[internet]<---- user: 200.1.1.1
/ \
/ \
[borderLeaf] [borderLeaf]
||| ||||
[...........spine nodes..........]
||| ||||
[Leaf] [Leaf ]
/ \ / \
[LB1] [LB2] [SVR1] [SVR2]
[customer.com] [10.1.1.1]
[100.1.1.1 ] [10.1.1.1]

数据包流是这样的

每个LB advertise 一组LB IP给用户，应该是internet public ip
用户的请求先到LB，LB再通过IP-in-IP，把数据包发给服务器，extDstIP = SvrIP
服务器Decap pkts，处理完之后直接返回给用户
所以数据流是: internet --> LB --> Svr --> internet

这个功能做啥哪？主要是switch offload SLB workload

第一个是 IPinIP encap

在LB上面的Leaf上，create static routes to Svr by NHG-IPinIP-tunnel。LB直接就转发native ip pkts, switch负责encap ip-in-ip tunnel header，因为是hw，比LB高速有效

Server LB - 上图里面的SVR1/2都有DstIP = 10.1.1.1，ECMP hashing会选一个final dest.
可以在NHG-IPinIP-tunnel增加 ECMP，从而达到network LB

需要啥？

T+ platform
LB using ECMP
v4-in-v4, v6-in-v6，以后的releae应该可以是v4-over-v6 or verse vice
Scale: 512 v4/256 v6 tunnels
EAPI: 1) config NHG; 2) config static route pointing to NHG

CLI:

size = # of NH entries in NHG, default = 256
entry <index> tunnel dest <dstIP>, dstIP = svrIP，这里在做LB!

index# = 0..size-1, 如果缺失就是drop, 注意！
所以hashing很少啊，srcIP唯一, proto#=4(IPinIP)也是唯一，全靠dstIP

ip route 10.1.1.1/32 nexthop-group NHG
如果route to dstIP本身就是multipath, 那么就选第一条

注意的几点

这个NHG不是interface，什么意思哪？

他不会像正常的interface去解析Nexthop
正常的情况下，解析是ASIC fwd to software due to missing NH info. 在Tunnel情况，就是software自己去解析
为啥有drop entry，主要就不会generate ICMP unreachable msg

multiple NHG可以share tunnels
如果多个routes to host ip，gated show多个，但是hw里面只有NHG
可以VRF吗？不知道
IP routing on/off，会重写hw，改变interface ip也是
MTU必须注意，增加了20B (v4) or 40B (v6)，需要config correct MTU on ALL interfaces
RPF: srcIP随意配置，所以注意不要被RPF给drop了
Linux Kernel has a drop entry for each IP, how to check?

为啥不叫IPinIP tunneling? 为啥取个新名字

NHG != tunnel interface, 3个区别:

没有路由协议
不会接受数据包
而且NHG可以有不同种类的Encap: LSP/MPLS, GRE, IPinIP

而且这是一个比interface更底层的概念，更像Adj/NHG = fib entry, 或者via = ECMP adj

(Z1446)

9/26/2018

EOS.swi vs /mnt/flash issue

就是 mount /dev/sda1 /mnt/flash, 然后vi boot-config, show boot, 其他没啥

Loading an engineering EOS.swi, the dut stuck at "wait for /mnt/flash timeout". Somehow, this image failed to mount /mnt/flash. The easiest way is to mount manually and change /mnt/flash/boot-config to load up a stable image.

mkdir /mnt/flash
mount /dev/sda1 /mnt/flash
vi /mnt/flash/boot-config

9/25/2018

Mlag reload-delay timers tunning

General rules:

No.1 rule - side-way up first

Peer-link up first. Recommended (if not must) to have an iBGP peering between MLAG peers. So always have a side-way.
Why need a side-way? Because there is for sure a gap between non-mlag (uplinks) and mlag (downlinks) ready to forward, in this case, there is a fire exit to avoid traffic drop.

No.2 rule - non-mlag relay < mlag delay

"In a topology where non-mlag links are only L3 ports that want to reach hosts behind MLAG interface, then it is preferable to keep non-mlag reload-delay smaller compared to mlag reload-delay to have L3 protocols converge before the L2 links ( MLAG interfaces ) are brought up."
Why? don't overwhelm the peerLink. Most of the time, servers's ingress << egress.

No.3 rule - if "lacp standby" enabled, mlag delay < non-mlag

Bring up interfaces to allow hw programing. So non-mlag (uplinks) up first, N-S traffic in but downlinks not ready.

Primary和Secondary最大的区别是STP，Sec上的STP is held.

MLAG reboot/SSO, all ports except peerlink are in err-disabled state。这个期间，系统需要bring up hardware，Mlag peer可以sync states. From 4.15.2F (released Sep 2015), the default-relay is automatically adjusted based on platform, 1800s in Sand(Arad/J/J+) modular, 1200s in Strata(Trident/TH) modular and 300s fixed.

如果non-mlag links都是L3 ports，最好是non-mlag delay < mlag delay，所以uplink先起来，等Mlag converged了，S-N已经有routes ready了。但是这个时候，会Attract N-S traffic，但是我们peerlink已经up了，就走peerlink, no loss.

LACP-standby就是可以更新LAG成员表，MAC地址表，ports appear as linkActive. 如果这个是ON，那么Non-Mlag delay >= Mlag delay. 不如N-S traffic进来，以为Mlag ports up，就drop了

如果使用VRRP，就需要configure VRRP reload daly time higher than MLAG reload delay.

Z1554

9/20/2018

Disable Mlag Peerlink

An EOS engineering command to shutdown the peerLink port-channel:

errordisable test interface po2000

9/12/2018

FHRP, VRRP and VARP

VARP (Z636)

"ip virtual mac <mac>"

= treat <mac> as own mac;
和MLAG合用=poor-man's VRRP，
好处是active-active，
坏处是static ARP on neighbor hosts，为啥坏啊？

如果SVI有"ip virtual addr"

respond ARP req for vIP + vMAC, 但是srcMAC还是phyMAC;
GARP, srcMAC = vMAC, 刷switch MAC table

under bash, 还有command?

varp vlan3 1.2.3.4 00:1c:73:00:00:01

VRRP over MLAG (Z1223)

传统上，MLAG最好的选择的是VARP，用VRRP is kinda dumb(DE's comments)
最大的问题是，VRRP Backup不fwd traffic，而且peerlink上不学MAC，结果哪？

hash到backup的traffic，会被flood，连switch都没有，因为peerlink不学mac
纪录在Y31356
Solution是write vrrp mac address into mlag host

还有mlag reload delay + VRRP

peerlink先起来，vrrp prempt所以newly up peer becomes master
可是the new peer还在reload delay，black hole traffic!!
纪录在Y30494
Workaround: config preempt delay reload #1 > reload-delay #2

这个Z1223最后没有做

FHRP: HSRP, VRRP and VARP

HSRP, VRRP and VARP use vMAC. GLBP uses phyMAC for LB;

HSRP vMAC = 0000:0c07:ac**
VRRP vMAC = 0000:5e00:01xx, xx = VRID (1-256)
VARP vMAC = self-configured
GLBP其实也用vMAC，应该是不同的vMAC = phyMAC

Assigned MAC address (side note)

00-00-5e, IANA (internet assign num association) ucast

00-00-5e-00-01/02-xx, VRRP v4/v6

01-00-5e, IANA mcast

00-00-00 to 7f-ff-ff: v4 mcast
90-00-01: bfd on LAG

最大的区别是，Active-Active vs Active-Standby, 如何做到的？

都用vMAC, hosts send packets with dstMAC = vMAC
VARP是active-active，直接route out.
而VRRP是bridge to peer via peerLink

GARP是刷switch mac table + 通知全部的hosts, ip/MAC mapping of vMAC = vIP

GARP和普通ARP Reply一样，只是dstMAC = FF or hostMAC
只有GARP pkt里面的srcMAC是vMAC. 这是唯一pkt!!. 其他data甚至ARP的srcMAC都是phyMAC.

需要phyIP吗？

'ip virtual address' 无论有没有mask，都需要phyIP. 不过w/ mask可以是dummy ip.
'ip address virtual' 不要phyIP

VARP = ip virtual address - IVA

GARP和ARP Reply一样，srcMAC, arp.sndMAC = vMAC
ARP request里面, 里外Eth/ARP都是Switch System MAC，arp.sndIP = phyIP，因为要确保Arp reply回到Src Mlag Peer!!!

VARP w/mask = ip virtual address w/ mask - IVAM

GARP + ARP Reply = VARP way
关键是ARP Req, 没有phyIP under this subnet, 所以里外都是vMAC/vIP. 好了有问题了，如果Host ARP Reply hashed到里外一个Peer，咋办？
所以这个VxLAN VARP必须有 ARP Sync!

VxLAN Anycast = ip address virtual + vMac

both Mlag peer都是一个Addr. 不需要phyIP, vIP就可以
没有GARP，为啥？因为ARP reply另外都是vMAC，不需要GARP刷switch
host知道GW，必须靠ARP reply by mlag peer. 里外都是vMac + vIP.
ARP Req = VARP w/mask, 因为没有phyIP；
所以也需要ARP Sync.

Python Tips

1. Is vs ==, mutable and immutable

== is to compare value, is is compare address

== is to compare by calling object.__eq__()

list1 = list2, address is assigned, not value. need list1=list2[:]

>>> a = [1,2,3]
>>> id(a)
4463753048 <<<< address is 048
>>> a[0] = 11
>>> id(a)
4463753048
>>> a
[11, 2, 3]
>>> a.append(44)
>>> a
[11, 2, 3, 44]
>>> id(a) <<<< can change item and append, address is same
4463753048
>>> b = a
>>> id(b) <<<< list= is point to same address
4463753048
>>> b[0] = 111
>>> a
[111, 2, 3, 44] <<<< change b = change a
>>> c = a[:]
>>> id(c)
4463919328
>>> c[0] = 1234
>>> a
[111, 2, 3, 44]
>>> b
[111, 2, 3, 44]
>>> c
[1234, 2, 3, 44]
>>>

2. keyword argument

>>> def ff(*args, **kwargs):
... print args
... print kwargs
...

>>> ff(1, '22', k1=333, k2='4444')
(1, '22')
{'k2': '4444', 'k1': 333}

>>> d = {'ka':'aaa', 'kb':123}
>>> ff(**d)
()
{'kb': 123, 'ka': 'aaa'}

3. Regexp \符号

https://docs.python.org/3/howto/regex.html

Remove全部的None-Alphanumeric char from string. 应该是

s = re.sub('\W', '', s)

还有记住以下的

\d, \D = [0-9]
\s, \S = [ \t\n\r\v\f]

\w, \W = alphanumeric,[0-9a-bA-Z_]

4. arstCli Script模版

from arstCliLib import *
import sys

dut = sys.argv[1]
openSshOnDut( dut )
setAccessMethod(dut, 'ssh')
cmd = ['show ip int brief | grep " 10\." | grep Vlan']
output = sendCmd(dut, cmd, prompt='enable', raw=True)

print '\n'.join(output)

[solomonyang@syscon] ~ $ python testCli.py
Traceback (most recent call last):
File "testCli.py", line 4, in <module>
from arstCliLib import *
ImportError: No module named arstCliLib

PATH=$PATH:$HOME/bin:$HOME/py
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/git/systest-infra/lib/
PYTHONPATH=$PYTHONPATH:$HOME/git/systest-infra/lib/

export PATH
export LD_LIBRARY_PATH
export PYTHONPATH

5. List vs Tuple, based on stackoverflow post

1. Literal/语法

>>> t = (1,2); l = [1,2]; t[1]; l[0]
2
1

2. Size/内存量，区别大概list +12%

>>> t = tuple(range(100000)); l = list(range(100000)); t.__sizeof__(); l.__sizeof__()
800024
900088

3. Mutable vs Inmutable/可变 vs 不可变
>>> t = (1,2); l = [1,2]
>>> l[0] = 11; print l
[11, 2]
>>> t[0] = 11; print t
TypeError: 'tuple' object does not support item assignment

4. 都可以增加，但是不同，tuple += (3)是返回一个新的obj，而list.add()是改原来的
>>> t = (1,2); l = [1,2]; id(t); id(l); t+=(3,); l+=[3]; id(t); id(l); l.append(3); id(l)
4306859000 <<<<< t
4307014720 <<<<< 1, 都一个地址
4306913952 <<<<< t +=(3,)，新地址
4307014720

4307014720

5. 因为不可变，所以tuple (1,2)可以是Dict[key]
>>> d = {}; d[t] = '1,2'
>>> d = {}; d[l] = '1,2'
TypeError: unhashable type: 'list'

6. 用途
比方说，(10,11)是个bookmark，第10页的第11行，一般没有必要改；而list of bookmark = [ (1,10), (10,11), (22, 1)]

7. Tuple != constant list
http://news.e-scribe.com/397
但是这么了解哪，好像也撮合:-) 好像更多的是字面的了解，tuple = lightweight record，比方说：DB API's fetchmany() 返回的是List of tuple. 每个tuple是一个record，不能改其中一项，改了就没有意义了。

List vs Set

1. Literal/语法/内存量
>>> s = set(range(100000)); s.__sizeof__(); l = list(range(100000)); l.__sizeof__()
4194504

900088

2. Set没有Index, 没有重复，可以数学操作 &, -, ^
>>> s1=set(range(1,20,2)); s2=set(range(10,30)); print 's1->', s1; print 's2->', s2; print '-:', s1 - s2; print '&:', s1&s2; s1^s2
s1-> set([1, 3, 5, 7, 9, 11, 13, 15, 17, 19])
s2-> set([10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29])
-: set([1, 3, 9, 5, 7])
&: set([19, 17, 11, 13, 15])

set([1, 3, 5, 7, 9, 10, 12, 14, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29])

9/11/2018

ISIS Segment Routing

CCIE SR discussion

说起来不难，就是用IGP(ISIS)来signal labels，而不是LDP或者RSVP，更简单
但是还是用MPLS，所以软件升级就可以了
有Node Segment和Adj Segment，Ingress Router构建Label Stack就可以控制LSP

Arista MPLS SR

网络的难点是：

如何Classify traffic 和 Engineer path.
SDN如何control traffic path/flow，特别是native solution for ipv6
MPLS解决了一些，但是太复杂，例如TE，而且还没有ipv6 native support.
Segment是Arista认为的solution

SR operation:

网络分成Segment, 给一个SID - segment id. 利用BGP/ISIS/OSPF extension来distribute. 而不在需要Label protocols - LDP/RSVP
SID有Global Unique或者Local significant, Base + Index
3种 global SID:

1) prefix;
2) node; loopback of node
3) anycast; loopback shared by a set of routers, ecmp

Prefix SID: 全网都是same value, 非常关键，reducing DP state;

图上的例子是, 所有的router SRGB(SR Global Block)都是900,000-965,535, Rtr5's 5.5.5.5/32, prefix-SID是10，所有的router全部assign label 900,000+10 = 900,010

Adj Sid是locally significant, 只给neighbor，只installed at neighbor
最简单应用，用到ECMP，ingress LER就push一个SR label. 或者是push a set of label

SR vs LDP:

相同的地方：

easy configured, "plug and play"
Both form stateless Mp-to-pt LSP

不同：

LDP全是local signficant label, SR是global unique labels, 减少DP state; scale 好；
SR有TE，LDP TE没有流行，v6也没有

SR vs RSVP-TE

RSVP-TE的特点：

constraints routing like b/w, shared link risk group and explicit paths，可以不按照IGP shortest path;
有b/w
FRR，有pre-computed backup paths

不好的地方：

full-mesh p2p TE tunnels, 没有ECMP
failure后有churn，需要re-signaling.
scalability issue，所以不那么流行

SR利用SDN，在head加入stack of mpls label，而不需要中间router纪录state。还有scale
什么都好，没有Multicast?

Arista SR:

必须是R-series with FlexRoute?

三种SR Solutions:

Static MPLS push + NHG

在ingress LER上，configure a route pointing to a label stack via CLI;
ECMP = multiple tunnels
Class-based service policy
Easy start

Controller based using Eos SDK
BGP-LU (labeled unicast/RFC 3107)

应用：Cloud-WAN, CDN, NFV

Arista ISIS SR (from 4-17-0f)

9/10/2018

Tutorial: Segment Routing

https://www.youtube.com/watch?v=BEo5MdB3o3Y

Take-away points:

SR中最重要的是TE
Segment (1-hop or n-hop) vs SID
SR是hybrid TE approaches: path info in packet and network
CSPF - attribute(b/w, color), requirement and shortest path
CP = controller

What's SR?

A tunneling tech - nothing fancy
A TE! - steer packet instead of routing path. ~= RSVP TE

Terminology:

Domain
SR path
Segment = SID, can be multiple hops

Traditional TE Approaches

path info in packet

like IPv4 strict routing option

path info in the network

RSVP-signaled MPLS

SR is hybrid TE approach

Segment types:

1 router hop
multiple router hops
Types:

Adjacency (IGP adjacency, 1-hop)
Prefix = IGP least cost path to a prefix

SR Encapsulation

MPLS

Label = segment

IPv6:

Segment Routing Extension Header (SRH)
A list of ipv6 addr
each ipv6 addr = a segment

Local Labels

Some SIDs have node-local significance
Adjancency
Why important

Stack can be too big, ASIC cannot handle
MTU

Global Labels

Some have domain-wide significance
Each node reserves a block of labels.
SRGB base

IPv6 forwarding

SRH, segment routing hdr
A list of ipv6 address as SID

Binding SIDs

Pointing to another SR paths or tunnels
Reasons:

Label stack
MTU size too big

Control Plane

Path computation: on SR ingress or central controller.

CSPF, 这个讲的清楚！

Attributes to segment: color, b/w, SRLG
Req to each path
Shortest path meeting req.
LSDB, TED = extension to carry info.
Alternative path to protect
All info in LSDB, no need for RSVP or LDP

SR convergence after failures

Fast recovery by IGP reconvergence.
TI-FLA, speed up convergence if not fast enuf
Use anycast SID (ecmp)

SR benefits:

If using SR to reserve b/w, MUST go for controller
Central controller has global view

Controller Protocol options:

pull LSDB:

controller: a passive mbr in IGP
BGP-LS

push segment list to ingress SR

PCEP
BGP

push policy

binding what traffic to which path
PCEP or BGP

Conclusion:

SR moves state from network to packet - simplified
Some open issues: OAM, Fast Reroute
Need experience

8/31/2018

Arista EOS - hardware counter feature

Since on Sand platform, Ingress/Egress ipv4 ACL, Qos and PBR counters occupy the same counter engineer, so they will not be able to work together. You have to specify which counter could use this shared engineer.

yo411.mlagB.profA.11:49:40(config)#hardware counter feature ?
acl ACL counter feature
mpls MPLS LFIB counter feature
nexthop Nexthop counter feature
pdp PDP counter feature
subinterface Subinterface counter feature
traffic-class Traffic-class counter feature
vlan VLAN counter feature

8/28/2018

Arista/EOS, MLAG ipv6 partial traffic loss

Topology (a typical MLAG network):

[mlagA] ======= [mlagB]
\ /
+---[Leaf]---+

Symptom:
1) about 10% L3 ipv6 traffic, WE and SN
2) No L2 or L3 ipv4 traffic loss

Get one problem destination - 2000:120:4d:d::1

1) show ipv6 route is good,
bn303.mlagA.profA.16:17:04(config)#sh ipv6 route 2000:120:4d:d::1
C 2000:120:4d::/48 [0/1]

via Vlan2077, directly connected

bn302.mlagB.profA.16:17:03(config-if-Vl2199)#sh ipv6 route 2000:120:4d:d::1
C 2000:120:4d::/48 [0/1]
via Vlan2077, directly connected

2) show ipv6 route host is NOT right
bn303.mlagA.profA.16:18:20(config)#sh ipv6 route host | grep 2000:120:4d:d
A 2000:120:4d:d::1 on Vlan2077
A 2000:120:4d:d::3 on Vlan2077 <<<<<< missing ::2
A 2000:120:4d:d::4 on Vlan2077

bn302.mlagB.profA.16:18:20(config-if-Vl2199)#sh ipv6 route host | grep 2000:120:4d:d
A 2000:120:4d:d::2 on Vlan2077 <<<<<< missing ::1
A 2000:120:4d:d::3 on Vlan2077
A 2000:120:4d:d::4 on Vlan2077

Root cause:
missing configuration - "ip virtual-router mac-address mlag-peer"

Arista EOS Debug Tips on Traffic Loss

1. check what/how many pkts punted to CPU?

- "show cpu counter queue", this is the place we see the software drop

- output is by switch ASIC and cpu queues

- CpuQueueL3DstMiss: pkt destinated to unknown address, like unARP'ed host address

- CpuQueueL3LpmOverflow - ?

2. hardward drop

- show hardware counter drop

3. show platform fap interrup

- check hw interrupts

4. Check drops on which interfaces

- show interface counter discard | nz

Arista EOS: %QOS-3-POLICY_HW_RESOURCE_FULL

When applying Qos policy under port-channel, the system doesn't accept it.

wa463.bug228215.16:12:21(config-if-Po20)#service-policy type qos input SPTest
% Error: Cannot apply service-policy to Port-Channel20 ()

And show logg displays an error msg:
Mar 8 16:01:25 wa463 SandAcl: %QOS-3-POLICY_HW_RESOURCE_FULL: Insufficient hardware resources to program the input policy-map SPTest.

It is because the TCAM is running out. One possible reason is PDP (per-port data policy) which uses up quite some TCAM. So try the EOS-Int image.

wa462.bug228215.16:17:04#sh platform jericho acl tcam summary
The total number of TCAM lines per Jericho bank is 2048

========================================================
Jericho0:
========================================================
Bank Used Used % Used By
0 2046 99 IP RACLs
1 1554 75 IP RACLs
2, 3 2048 100 IPv6 RACLs
4, 5 2048 100 IPv6 RACLs
6, 7 2048 100 IPv6 RACLs
8, 9 2046 99 IPv6 RACLs
10,11 210 10 IPv6 RACLs
14 79 61 Pdp IP, Pdp Tunnel, Pdp NonIp, Pdp IPv6, Pdp Mpls

Total Number of TCAM lines used is: 20479

After changing to INT image, this issue is gone.

wa462.bug228215.16:28:09(config)#int po20
wa462.bug228215.16:28:11(config-if-Po20)#service-policy type qos input SPTest
wa462.bug228215.16:28:13(config-if-Po20)#show ver
Arista DCS-7280CR-48-F
Hardware version: 11.01
Serial number: JPE16473148
System MAC address: 444c.a897.8c51

Software image version: 4.20.0F-INT-7767198.bloomingtonrel (engineering build)