[FS#1494] Second routing table not working

Sat Apr 14 07:03:34 PDT 2018

A new Flyspray task has been opened.  Details are below. 

User who did this - Jeff (bugger) 

Attached to Project - OpenWrt/LEDE Project
Summary - Second routing table not working
Task Type - Bug Report
Category - Kernel
Status - Unconfirmed
Assigned To - 
Operating System - All
Severity - Low
Priority - Very Low
Reported Version - Trunk
Due in Version - Undecided
Due Date - Undecided
Details - I'm using my router as a VPN client and by default all traffic from my network is sent over the VPN.  There's one device one the network that needs to connect without the VPN, so I've created a second routing table and a corresponding rule to route that device over the new table.  This setup is working great on 17.01.4 with kernel 4.4, however if I upgrade to a new snapshot with kernel 4.14 the device going over the alternate route stops working in a weird way.  It's still sending packets out and it receives a preliminary response from the host, but then the connection stalls.  All connections going over the secondary routewill do the same thing: open the port, transmit the first packet and then stall.  Pinging works just fine, too, it just TCP connections that seem to get stuck.

Here's my routing table with the 0.0.0.0 and 128.0.0.0 entries added by openvpn through he "redirect-gateway def1" option.  This routes all outgoing traffic through the VPN.  Note that my WAN connection is on wlan0 and has ip 192.168.2.18 and gateway 192.168.2.1 and my LAN is br-lan with IP 192.168.1.1 on the local 192.168.1.0/24 network.  The VPN host is 161.22.33.44.

root at LEDE:~# ip route
0.0.0.0/1 via 10.9.8.1 dev tun0
default via 192.168.2.1 dev wlan0  proto static  src 192.168.2.18
10.9.8.1 dev tun0  proto kernel  scope link  src 10.9.8.2
128.0.0.0/1 via 10.9.8.1 dev tun0
161.22.33.44 via 192.168.2.1 dev wlan0
192.168.2.0/24 dev wlan0  proto kernel  scope link  src 192.168.2.18
192.168.2.254 dev wlan0  proto static  scope link  src 192.168.2.18
192.168.1.0/24 dev br-lan  proto kernel  scope link  src 192.168.1.1

This is what the tcpdump on the router shows when the traffic is going over the default route

user at local.client:> curl http://remote.host

tcpdump:
14:36:17.060423 IP local.client.46412 > remote.host.80: Flags [S], seq 3190523033, win 29200, options [mss 1460,sackOK,TS val 27124528 ecr 0,nop,wscale 7], length 0
14:36:17.107303 IP remote.host.80 > local.client.46412: Flags [S.], seq 3924485213, ack 3190523034, win 29200, options [mss 1350,nop,nop,sackOK], length 0
14:36:17.107597 IP local.client.46412 > remote.host.80: Flags [.], ack 1, win 29200, length 0
14:36:17.107818 IP local.client.46412 > remote.host.80: Flags [P.], seq 1:79, ack 1, win 29200, length 78: HTTP: GET / HTTP/1.1
14:36:17.156123 IP remote.host.80 > local.client.46412: Flags [.], ack 79, win 29200, length 0
14:36:17.166236 IP remote.host.80 > local.client.46412: Flags [P.], seq 1:182, ack 79, win 29200, length 181: HTTP: HTTP/1.1 200 OK
14:36:17.166475 IP local.client.46412 > remote.host.80: Flags [.], ack 182, win 30016, length 0
14:36:17.166528 IP remote.host.80 > local.client.46412: Flags [F.], seq 182, ack 79, win 29200, length 0
14:36:17.166728 IP local.client.46412 > remote.host.80: Flags [F.], seq 79, ack 183, win 30016, length 0
14:36:17.213095 IP remote.host.80 > local.client.46412: Flags [.], ack 80, win 29200, length 0

And here's what happens when I build a second table and route it over that (note, all of this works just fine on 17.01.4)

root at LEDE:~# ip route add default via 192.168.2.1 dev wlan0 table 201
root at LEDE:~# ip rule add from 192.168.1.60 lookup 201

user at local.client:> curl http://remote.host

tcpdump:
14:40:18.797240 IP local.client.46620 > remote.host.80: Flags [S], seq 1629062849, win 29200, options [mss 1460,sackOK,TS val 27184962 ecr 0,nop,wscale 7], length 0
14:40:18.845435 IP remote.host.80 > local.client.46620: Flags [S.], seq 3094935245, ack 1629062850, win 29200, options [mss 1460,nop,nop,sackOK], length 0
14:40:18.845696 IP local.client.46620 > remote.host.80: Flags [.], ack 1, win 29200, length 0
14:40:18.846146 IP local.client.46620 > remote.host.80: Flags [P.], seq 1:79, ack 1, win 29200, length 78: HTTP: GET / HTTP/1.1
14:40:19.095838 IP local.client.46620 > remote.host.80: Flags [P.], seq 1:79, ack 1, win 29200, length 78: HTTP: GET / HTTP/1.1
14:40:19.347850 IP local.client.46620 > remote.host.80: Flags [P.], seq 1:79, ack 1, win 29200, length 78: HTTP: GET / HTTP/1.1
14:40:19.875877 IP local.client.46620 > remote.host.80: Flags [P.], seq 1:79, ack 1, win 29200, length 78: HTTP: GET / HTTP/1.1
14:40:20.899852 IP local.client.46620 > remote.host.80: Flags [P.], seq 1:79, ack 1, win 29200, length 78: HTTP: GET / HTTP/1.1
14:40:22.915865 IP local.client.46620 > remote.host.80: Flags [P.], seq 1:79, ack 1, win 29200, length 78: HTTP: GET / HTTP/1.1

I'm not really sure where to go from here. Any pointers to help track this down would be appreciated!

More information can be found at the following URL:
https://bugs.openwrt.org/index.php?do=details&task_id=1494