Arista EOS Lag/ECMP Hashing Features

References/Manual:

• EOS Manual Section 12.4: Load Balancing Hash Algorithms

References/TOI:

• EOS 4.24.2F TOI - Support for dynamic load balancing on ECMP groups (Trident Only)
• EOS 4.22.1F TOI - ECMP Load Balance Profile Support (Sand/Jericho Only)
• EOS 4.21.3F TOI - Symmetric header selection in LAG load-balancing profiles (Jericho Only)
• EOS 4.21.0F TOI - UDF flex hashing improvements (Trident Only)
• EOS 4.20.6F TOI - LAG hashing enhancement for configuring member-selection method (Jericho Only)
• EOS 4.20.5F TOI - LAG hashing on Ingress Interface
• EOS 4.20.5F TOI - Lag Hashing Key Shift
• EOS 4.20.5F TOI - VxLAN over MPLS LAG hashing optimizations for Bridged ports
• EOS 4.20.1F TOI - LAG and ECMP selectable hashing fields (Jericho Only)
• EOS 4.18.0F TOI - Hardware load-balancing for ingress/egress and fabric/egress replication
• EOS 4.17.2F TOI - Configurable field selection for ECMP hash, (Trident Only)
• EOS 4.17.2F TOI - Disable usage of ingress interface in ECMP hashing
• EOS 4.17.1F TOI - Inner IP hashing for MPLSoGRE
• EOS 4.17.0F TOI - Global LAG Hashing Profiles (Jericho Only)
• EOS 4.17.0F TOI - ECMP Hash Visibility
• EOS 4.15.2F TOI - GRE in LAG hash
• EOS 4.15.2F TOI - LAG/ECMP hashing on TTL/Hop Limit
• EOS 4.15.0F TOI - Dynamic and symmetric LAG hashing
• EOS 4.15.0F TOI - Tap aggregation enhancements

Sample: In this below sample, I like to show how to use the above features to fix an ipv6 multicast unbalanced issue. 

Says, in the above topology, there are 2000 ipv6 multicast streams flowing ixia-harness-[vrrp primary| secondary]-nexthop router. Let's see the output of srnz on VRRP primary. 

1) w/o any hashing config, lowest vs highest = 220 vs 317

ghs259-GW2.21:24:15(config-if-Po2)#srnz | grep HUB

Et3/1     GW2-HUB2     0:05       0.0   0.0%        0     220.1   0.2%       35

Et4/1     GW2-HUB2     0:05       0.0   0.0%        0     316.9   0.3%       50

Et13/1    GW2-HUB2     0:05       0.0   0.0%        0     266.1   0.3%       42

Et14/1    GW2-HUB2     0:05       0.0   0.0%        0     269.3   0.3%       43

Po2       GW2-HUB2     0:05       0.0   0.0%        0    1072.3   0.3%      171

2) hash key shift

Let's try 1 hashing feature - Lag Hashing Key Shift. Create a LB profile - mcast-v6 + key shift, apply it globally. 

ghs259-GW2.21:37:22(config)#load-balance policies

ghs259-GW2.21:37:28(config-load-balance-policies)#   load-balance sand profile mcast-v6

ghs259-GW2.21:37:35(config-sand-load-balance-profile-mcast-v6)#      ecmp hash key shift 3

ghs259-GW2.21:37:57(config)#port-channel load-balance sand profile mcast-v6

ghs259-GW2.21:39:50#srnz | grep HUB

Et3/1     GW2-HUB2     0:05       0.0   0.0%        0     220.1   0.2%       35

Et4/1     GW2-HUB2     0:05       0.0   0.0%        0     316.9   0.3%       50

...

Clearly, not much help. 

3) hash seed

ghs259-GW2.21:42:47(config-load-balance-policies)#load-balance sand profile mcast-v6

! profile mcast-v6 is the current global profile

ghs259-GW2.21:42:52(config-sand-load-balance-profile-mcast-v6)#no ecmp hash key shift

ghs259-GW2.21:43:03(config-sand-load-balance-profile-mcast-v6)#ecmp hash seed 1001

ghs259-GW2.21:43:25(config-sand-load-balance-profile-mcast-v6)#end

ghs259-GW2.21:43:36#srnz | grep HUB

Et3/1     GW2-HUB2     0:05       0.0   0.0%        0     220.2   0.2%       35

Et4/1     GW2-HUB2     0:05       0.0   0.0%        0     316.9   0.3%       50

...

4) hash polynomial + Hardware load-balancing for ingress/egress and fabric/egress replication (4.18.0F)

ghs259-GW2.21:45:26(config)#load-balance policies

ghs259-GW2.21:45:30(config-load-balance-policies)#load-balance sand profile mcast-v6

! profile mcast-v6 is the current global profile

ghs259-GW2.21:45:35(config-sand-load-balance-profile-mcast-v6)#no ecmp hash seed 1001

ghs259-GW2.21:45:42(config-sand-load-balance-profile-mcast-v6)#port-channel hash polynomial 5

ghs259-GW2.21:45:43(config-load-balance-policies)#    port-channel load-balance sand replication egress

ghs259-GW2.21:46:52#srnz | grep HUB

Et3/1     GW2-HUB2     0:05       0.0   0.0%        0     265.3   0.3%       42

Et4/1     GW2-HUB2     0:05       0.0   0.0%        0     272.3   0.3%       43

Nice! The hash polynomial works! Actually EOS 8 polynomial functions, the #5 works. 

Understanding the output of "show qos interface eth#"

In this post, I like to explain the output of the EOS command - "show qos interface eth#" based on my understanding from EOS document. 

wa461.00:58:25#sh qos interfaces e17/1
Ethernet17/1:
   Trust Mode: DSCP
   Default COS: 0
   Default DSCP: 0

   Port shaping rate: disabled
   Burst-size: disabled

  Tx    Bandwidth         Shape Rate               Burst-Size          Priority   ECN/WRED
 Queue  (percent)          (units)                  (units)
 ------------------------------------------------------------------------------------------
   7      - / -       - / -          ( - )           -  /  -           SP / SP       D
   6      - / -       - / -          ( - )           -  /  -           SP / SP       D
   5      - / -       - / -          ( - )           -  /  -           SP / SP       D
   4      - / -       - / -          ( - )           -  /  -           SP / SP       D
   3      - / -       - / -          ( - )           -  /  -           SP / SP       D
   2     20 / 20    1.2 / 1.0        (Gbps)     2048 KB / 2048 KB      RR / RR       D
   1     30 / 30      - / -          ( - )           -  /  -           RR / SP       D
   0     50 / 50      - / -          ( - )           -  /  -           RR / SP       D

Note: Values are displayed as Operational/Configured
Legend:
RR -> Round Robin
SP -> Strict Priority
 - -> Not Applicable / Not Configured
 % -> Percentage of line rate

• Values are displayed as Operational/Configured, like RR/SP which means this Q is configured as strict priority but operational as round-robin. 
• If one queue is configured as no priority (RR), then all the lower queues are changed to RR. 
• In this example, Q 2 is RR, then 0 and 1 are automatically changed to RR. 
• And Q 0 and 1 are RR/SP, which means their configuration are SP by default, but operational mode is RR.
• If both interface and tx-queue are configured with shape, which is effective? 
• From EOS manual chapter 27.5 - Enabling port shaping on an FM6000 interface disables queue shaping internally. Disabling port shaping restores queue shaping as specified in running-config.

interface Ethernet17/1
   speed forced 10000full
   !
   tx-queue 0
      bandwidth percent 50
   !
   tx-queue 1
      bandwidth percent 30
   !
   tx-queue 2
      no priority
      bandwidth percent 20
      shape rate 1000000

• Bandwidth vs shape.  
• Bandwidth% is the b/w percent this RR queue can get. Says the above configuration:
• In the sample below, the interface 17/1 is 10 Gbps interface 
• Q3-7 are the strict priority and, say use total 2 Gbps traffic, which left 8Gbps for Q0-2
• The tx-Q 2 can have 20% of left-over capacity which is 1.6Gbps
• But the shape rate is 1.2Gbps
• So the maximum throughput of tx-Q 2 is 1.2 Gbps, even it is assigned with 1.6Gbps.

EOS: A simple Qos design example

This article - "A Simple Quality of Service Design Example" is a very good starting point for understanding the EOS Qos architecture and starting a Qos design. 

Some points:

• 3 ways in the ingress points to map packets to Tx queues:
• qos cos trust + cos-tc map
• qos dscp trust + dscp-tc map
• service-policy + policy-map
• 3 big categories of traffic:
• network-control = control plane
• latency/jitter sensitive traffic
• best-efforts = scavenger traffic
• qos profile = 
• policy-map for input
• tx-queue set for output
• "no priority" in a tx-queue, all lower queues become RR

Support for CPU traffic policy

The default EOS Copp only provides protocol level traffic policy, for example, the maximum throughput of bgp traffic destined to the cpu. But there is no granularity of source address. So this feature - Support for CPU traffic policy is for this purpose. 

Set DSCP value for CPU outbound packets

In Arista EOS, the following protocol packets are able to set a DSCP value other than the default value 0:

• RADIUS, TACACS, SNMP, SSH, and sFlow (feature supported from 4.18.1F)
• NTP, DNS, Syslog, Traceroute, OSPF, and OSPF3 protocols (from 4.22.1F)

Notes:

• The setting must be done individually and under the protocol section
• hostname is not supported. 

For example, setting all locally originated CP packets to DSCP value nc1/cs6/110000/63:

logging qos dscp 48

!

dns qos dscp 48

!

ntp qos dscp 48

!

traceroute qos dscp 48

!

sflow qos dscp 48

！

snmp-server qos dscp 48

！

radius-server qos dscp 48

！

tacacs-server qos dscp 48

！

management ssh

   qos dscp 48

!

router ospf general

   qos dscp 48

EOS: % Not supported when show bgp summary

If you see the error message with EOS command - show bgp <AF> summary, it is probably caused the routing mode. To be more specific, you are probably running ribd mode and the CLI - "show bgp <AF> summary" is only supported in multi-agent mode

ghs259#show bgp ipv6 unicast summary
% Not supported

ghs259-CIN-DPA2.23:32:18#show ip route summary
Operating routing protocol model: ribd
Configured routing protocol model: multi-agent (will apply after next reboot)