Trouble-shoot BGP peering issue over GRE tunnel

Starting from 4.21.1F, Arista EOS starts to support the hardware GRE tunnel interface and BGP session over the tunnel on Jericho platforms. Before the tunnel is implemented by nexthop-group and decap group. 

Here is a very simple and straightforward setup of eBGP over GRE tunnel. 

But BGP session fails to come up as shown below:

R1.gts425#sh ip bgp sum

BGP summary information for VRF default

Router identifier 1.1.1.1, local AS number 1

Neighbor Status Codes: m - Under maintenance

  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State  PfxRcd PfxAcc

  10.100.100.4     4  4               4766       257    0    0 00:20:40 Connect 0      0

Tunnel interface is up and works fine. 

R1.gts425#sh int tunnel 100

Tunnel100 is up, line protocol is up (connected)

  Hardware is Tunnel, address is 0101.0101.0800

  Description: tunnel-gre-sand-to-sand

  Internet address is 10.100.100.1/24

  Broadcast address is 255.255.255.255

  Tunnel source 1.1.1.1, destination 4.4.4.4

  Tunnel protocol/transport GRE/IP

   Key disabled, sequencing disabled

   Checksumming of packets disabled

  Tunnel TTL 0, Hardware forwarding not supported

  Tunnel TOS 0

  Path MTU Discovery

  Tunnel transport MTU 1476 bytes

  Up 22 minutes, 2 seconds

Ping with MTU size works totally fine

R1.gts425#ping 10.100.100.4 size 1476

...

80 bytes from 10.100.100.4: icmp_seq=5 ttl=64 time=0.118 ms

--- 10.100.100.4 ping statistics ---

5 packets transmitted, 5 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.118/0.155/0.295/0.070 ms, ipg/ewma 0.223/0.222 ms

Now let's run the tcpdump on R1 to see if hello packet out

R1.gts425(config-router-bgp)#bash tcpdump -nvvi et21 host 1.1.1.1

tcpdump: listening on et21, link-type EN10MB (Ethernet), capture size 262144 bytes

11:19:19.774224 28:99:3a:8f:91:bf > 44:4c:a8:c1:78:69, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 1, id 39325, offset 0, flags [DF], proto GRE (47), length 84)

    1.1.1.1 > 4.4.4.4: GREv0, Flags [none], proto IPv4 (0x0800), length 64

(tos 0xc0, ttl 1, id 6538, offset 0, flags [DF], proto TCP (6), length 60)

    10.100.100.1.46931 > 10.100.100.4.bgp: Flags [S], seq 3552659475, win 28720, options [mss 1436,sackOK,TS val 18764185 ecr 0,nop,wscale 7], length 0

11:19:19.774420 44:4c:a8:c1:78:69 > 28:99:3a:8f:91:bf, ethertype IPv4 (0x0800), length 126: (tos 0xc0, ttl 64, id 29177, offset 0, flags [none], proto ICMP (1), length 112)

    10.1.2.2 > 1.1.1.1: ICMP time exceeded in-transit, length 92

(tos 0x0, ttl 1, id 39325, offset 0, flags [DF], proto GRE (47), length 84)

    1.1.1.1 > 4.4.4.4: GREv0, Flags [none], proto IPv4 (0x0800), length 64

(tos 0xc0, ttl 1, id 6538, offset 0, flags [DF], proto TCP (6), length 60)

    10.100.100.1.46931 > 10.100.100.4.bgp: Flags [S], seq 3552659475, win 28720, options [mss 1436,sackOK,TS val 18764185 ecr 0,nop,wscale 7], length 0

Now we can see the reason clearly. The eBGP TCP session is default with ttl 1 and copied to outer GRE packets, so the packets get TTL expired at 10.1.2.2 which is R2. 

To fix this issue, just need to set TTL under tunnel interface

R1.gts425(config-router-bgp)#int tu 100

R1.gts425(config-if-Tu100)#tunnel ttl 10

R1.gts425(config-router-bgp)#sh ip bgp sum

BGP summary information for VRF default

Router identifier 1.1.1.1, local AS number 1

Neighbor Status Codes: m - Under maintenance

  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State  PfxRcd PfxAcc

  10.100.100.4     4  4                  4         4    0    0 00:00:01 Estab  0      0