Looking for pointers debugging missing X-CSTP-Split-Exclude in Fedora 32

Mon Nov 16 20:22:10 EST 2020

You've done a pretty impressive deep-dive here.

The short summary, unfortunately, is that NetworkManager itself simply
does not support split-exclude routes. Any split-exclude routes
provided by the server are ignored by NM-OpenConnect:
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/380

Options:

1. Use the OpenConnect CLI instead. It uses the "real" vpnc-script
(https://gitlab.com/openconnect/vpnc-scripts/blob/master/vpnc-script),
which *does* support split-excludes. You can even replace the standard
vpnc-script with https://github.com/dlenski/vpn-slice, which I wrote
to allow you to completely customize the routes that go through the
VPN.

2. Use a NM post-connection script as a kludge to setup the
split-exclude routes automatically after you connect, as alluded to in
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/380#note_437592

3. Just use a shell script to set up the split-exclude routes by hand.
Here's a lightly touched-up version of what I used to use with a VPN
that blocked a whole bunch of sites… ¯\_(ツ)_/¯

```
#!/bin/bash

real="wlan0"

fix_ip_get_output () {
        sed -e 's/ /\n/g' | \
            sed -ne "1 s|\$|${1}|p;/via/{N;p};/dev/{N;p};/src/{N;p};/mtu/{N;p}"
}

dig +short -t A \
    BUNCH OF HOSTNAMES YOU WANT TO EXCLUDE \
    | sort | uniq | grep -Ev '[a-z]' |
    while read IP
    do
        rte=$(ip route get $IP dev $real | fix_ip_get_output)
        [[ -z "$rte" ]] && echo "Couldn't get route for $IP" >&2 \
           || sudo ip route replace $rte
    done
```

-Dan

On Mon, Nov 16, 2020 at 4:50 PM Rob Sherwood <rob.sherwood at gmail.com> wrote:
>
> Hi,
>
> I'm running an openconnect client (versions below) that connects
> perfectly to a VPN server which advertises a bunch of split exclude
> routes using X-CSTP-Split-Exclude.  While the connection, the auth,
> and the actual VPN bits all work correctly, unfortunately, those extra
> routes are not showing up on my system and I'm trying to understand
> why.
>
> If I understand correctly, what should be happening is the server
> sends X-CSTP-Split-Exclude headers which then get parsed and passed as
> environmental variables
> (https://github.com/openconnect/openconnect/blob/master/script.c#L90)
> to the vpnc-script specified in the --script parameter.  And then from
> there, it's the script's job to correctly process and add those
> routes.  Does that sound correct?
>
> But on my system (default Fedora 32, AFAICT), there is no (!?)
> vpnc-script but it uses
> /usr/libexec/nm-openconnect-service-openconnect-helper instead (source
> code at https://github.com/NetworkManager/NetworkManager-openconnect)
>
> [running instance]
>
> /usr/sbin/openconnect --servercert <cert> --syslog --cookie-on-stdin
> --script /usr/libexec/nm-openconnect-service-openconnect-helper
> --interface vpn0 <ip>
>
> I've wrapped /usr/libexec/nm-openconnect-service-openconnect-helper
> with a quick shell script to capture the env variables on calling it
> and they look right (e.g., I'm seeing lots of CISCO_SPLIT_EXC_*
> variables).  But looking through the
> nm-openconnect-service-openconnect-helper code, it seems like it there
> is code to parse these envs (e.g.,
> https://github.com/NetworkManager/NetworkManager-openconnect/blob/master/src/nm-openconnect-service-openconnect-helper.c#L317
> ), but the result seems to be to send then back to the
> openconnect-service
> (https://github.com/NetworkManager/NetworkManager-openconnect/blob/master/src/nm-openconnect-service-openconnect-helper.c#L715)
> via dbus.  And from there I've not been able to continue following the
> chain or understanding how to dump dbus to see if the right info is
> coming out of the helper.
>
> Long story short : there is a lot of different places that could break
> in the chain of passing the routes around and I'm hoping someone on
> this list can help (1) verify what I've discovered so far, (2)
> hopefully point me at some experiments or things to try to see why
> this might all be breaking.
>
> Relevant logs:
> $ journalctl -S today | grep openconnect
> [snip]
> Nov 16 16:19:06 rsher-fedora-R90RRLFQ openconnect[44831]: Connected to [$IP]:443
> Nov 16 16:19:06 rsher-fedora-R90RRLFQ openconnect[44831]: SSL
> negotiation with [$IP]
> Nov 16 16:19:06 rsher-fedora-R90RRLFQ openconnect[44831]: Server
> certificate verify failed: signer not found
> Nov 16 16:19:06 rsher-fedora-R90RRLFQ openconnect[44831]: Connected to
> HTTPS on [$IP] with ciphersuite
> (TLS1.2)-(DHE-CUSTOM2048)-(RSA-SHA512)-(AES-256-GCM)
> Nov 16 16:19:07 rsher-fedora-R90RRLFQ openconnect[44831]: Got CONNECT
> response: HTTP/1.1 200 OK
> Nov 16 16:19:07 rsher-fedora-R90RRLFQ openconnect[44831]: CSTP
> connected. DPD 30, Keepalive 20
> Nov 16 16:19:07 rsher-fedora-R90RRLFQ openconnect[44831]: Connected as <ip>
>  + <ip>, using SSL, with DTLS in progress
> Nov 16 16:19:07 rsher-fedora-R90RRLFQ openconnect[44831]: Established
> DTLS connection (using GnuTLS). Ciphersuite
> (DTLS1.2)-(ECDHE-RSA)-(AES-256-GCM).
> Nov 16 16:19:09 rsher-fedora-R90RRLFQ openconnect[44831]: SIOCSIFMTU:
> Operation not permitted
>
>
> [rsher at rsher-fedora-R90RRLFQ ~]$ rpm -qa | grep openconnect
> NetworkManager-openconnect-gnome-1.2.6-3.fc32.x86_64
> NetworkManager-openconnect-1.2.6-3.fc32.x86_64
> openconnect-8.10-1.fc32.x86_64
>
> Thank you in advance for any help you can provide!
>
> - Rob
> .
>
>
> On Mon, Nov 16, 2020 at 4:32 PM Rob Sherwood <rob.sherwood at gmail.com> wrote:
> >
> > Hi,
> >
> > I'm running an openconnect client (versions below) that connects perfectly to a VPN server which advertises a bunch of split exclude routes using X-CSTP-Split-Exclude.  While the connection, the auth, and the actual VPN bits all work correctly, unfortunately, those extra routes are not showing up on my system and I'm trying to understand why.
> >
> > If I understand correctly, what should be happening is the server sends X-CSTP-Split-Exclude headers which then get parsed and passed as environmental variables (https://github.com/openconnect/openconnect/blob/master/script.c#L90) to the vpnc-script specified in the --script parameter.  And then from there, it's the script's job to correctly process and add those routes.  Does that sound correct?
> >
> > But on my system (default Fedora 32, AFAICT), there is no (!?) vpnc-script but it uses /usr/libexec/nm-openconnect-service-openconnect-helper instead (source code at https://github.com/NetworkManager/NetworkManager-openconnect)
> >
> > [running instance]
> >
> > /usr/sbin/openconnect --servercert <cert> --syslog --cookie-on-stdin --script /usr/libexec/nm-openconnect-service-openconnect-helper --interface vpn0 <ip>
> >
> > I've wrapped /usr/libexec/nm-openconnect-service-openconnect-helper with a quick shell script to capture the env variables on calling it and they look right (e.g., I'm seeing lots of CISCO_SPLIT_EXC_* variables).  But looking through the nm-openconnect-service-openconnect-helper code, it seems like it there is code to parse these envs (e.g., https://github.com/NetworkManager/NetworkManager-openconnect/blob/master/src/nm-openconnect-service-openconnect-helper.c#L317 ), but the result seems to be to send then back to the openconnect-service (https://github.com/NetworkManager/NetworkManager-openconnect/blob/master/src/nm-openconnect-service-openconnect-helper.c#L715) via dbus.  And from there I've not been able to continue following the chain or understanding how to dump dbus to see if the right info is coming out of the helper.
> >
> > Long story short : there is a lot of different places that could break in the chain of passing the routes around and I'm hoping someone on this list can help (1) verify what I've discovered so far, (2) hopefully point me at some experiments or things to try to see why this might all be breaking.
> >
> > Relevant logs:
> > $ journalctl -S today | grep openconnect
> > [snip]
> > Nov 16 16:19:06 rsher-fedora-R90RRLFQ openconnect[44831]: Connected to [$IP]:443
> > Nov 16 16:19:06 rsher-fedora-R90RRLFQ openconnect[44831]: SSL negotiation with [$IP]
> > Nov 16 16:19:06 rsher-fedora-R90RRLFQ openconnect[44831]: Server certificate verify failed: signer not found
> > Nov 16 16:19:06 rsher-fedora-R90RRLFQ openconnect[44831]: Connected to HTTPS on [$IP] with ciphersuite (TLS1.2)-(DHE-CUSTOM2048)-(RSA-SHA512)-(AES-256-GCM)
> > Nov 16 16:19:07 rsher-fedora-R90RRLFQ openconnect[44831]: Got CONNECT response: HTTP/1.1 200 OK
> > Nov 16 16:19:07 rsher-fedora-R90RRLFQ openconnect[44831]: CSTP connected. DPD 30, Keepalive 20
> > Nov 16 16:19:07 rsher-fedora-R90RRLFQ openconnect[44831]: Connected as <ip>
> >  + <ip>, using SSL, with DTLS in progress
> > Nov 16 16:19:07 rsher-fedora-R90RRLFQ openconnect[44831]: Established DTLS connection (using GnuTLS). Ciphersuite (DTLS1.2)-(ECDHE-RSA)-(AES-256-GCM).
> > Nov 16 16:19:09 rsher-fedora-R90RRLFQ openconnect[44831]: SIOCSIFMTU: Operation not permitted
> >
> >
> > [rsher at rsher-fedora-R90RRLFQ ~]$ rpm -qa | grep openconnect
> > NetworkManager-openconnect-gnome-1.2.6-3.fc32.x86_64
> > NetworkManager-openconnect-1.2.6-3.fc32.x86_64
> > openconnect-8.10-1.fc32.x86_64
> >
> > Thank you in advance for any help you can provide!
> >
> > - Rob
> > .
>
> _______________________________________________
> openconnect-devel mailing list
> openconnect-devel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/openconnect-devel