openconnect+OpenSSL failing DTLS handshake with ocserv+GnuTLS

[Vuille, Martin (Martin)]

Client was built from sources, for ARM, using OpenSSL:

       OpenConnect version v8.05-275-g9d287e4
       Using OpenSSL 1.0.2u-fips  20 Dec 201. Features present: TPM (OpenSSL ENGINE not present), HOTP software token, TOTP software token, DTLS, ESP
       Supported protocols: anyconnect (default), nc, gp, pulse

The server is Fedora-provided RPM, using GnuTLS:

       ocserv 1.1.1
       Compiled with: seccomp, oath, radius, gssapi, PAM, PKCS#11, AnyConnect
       GnuTLS version: 3.6.16 (compiled with 3.6.14)

Non-comment lines in ocserv.conf are:

       auth = "certificate"
       listen-host = 10.215.0.62
       tcp-port = 8443
       udp-port = 8443
       run-as-user = ocserv
       run-as-group = ocserv
       socket-file = ocserv.sock
       chroot-dir = /var/lib/ocserv
       server-cert = /etc/pki/ocserv/public/server-cert.pem
       server-key = /etc/pki/ocserv/private/server-key.pem
       ca-cert = /etc/pki/ocserv/public/ca-cert.pem
       isolate-workers = true
       max-clients = 16
       max-same-clients = 2
       rate-limit-ms = 100
       server-stats-reset-time = 604800
       keepalive = 32400
       dpd = 90
       mobile-dpd = 1800
       switch-to-tcp-timeout = 25
       try-mtu-discovery = false
       cert-user-oid = 2.5.4.3
       tls-priorities = "NORMAL:%SERVER_PRECEDENCE"
       auth-timeout = 240
       min-reauth-time = 300
       max-ban-score = 80
       ban-reset-time = 1200
       cookie-timeout = 300
       deny-roaming = false
       rekey-time = 172800
       rekey-method = ssl
       use-occtl = true
       device = vpns
       predictable-ips = true
       ipv4-network = 192.215.0.0
       ipv4-netmask = 255.255.255.252
       ping-leases = true
       cisco-client-compat = false
       dtls-psk = true
       dtls-legacy = false

Client is invoked with the command:

       openconnect --protocol=anyconnect -c user-cert.pem -k user-key.pem --cafile=ca-cert.pem --dump -vvv 10.215.0.62:8443

DTLS handshake fails, server logs show:

       Jul 06 09:56:00 gna ocserv[3394]: worker[user]: 10.215.0.36 suggesting DPD of 90 secs
       Jul 06 09:56:00 gna ocserv[3394]: worker[user]: 10.215.0.36 configured link MTU is 1500
       Jul 06 09:56:00 gna ocserv[3394]: worker[user]: 10.215.0.36 peer's link MTU is 1500
       Jul 06 09:56:00 gna ocserv[3394]: worker[user]: 10.215.0.36 sending IPv4 192.215.0.2
       Jul 06 09:56:00 gna ocserv[3394]: worker[user]: 10.215.0.36 X-DTLS-App-ID: 14ff6e6e26aca40eea9e3f05df2eb9613da6e6bfe400c83aa7d956b5e318c805
       Jul 06 09:56:00 gna ocserv[3394]: worker[user]: 10.215.0.36 DTLS ciphersuite: PSK-NEGOTIATE
       Jul 06 09:56:00 gna ocserv[3394]: worker[user]: 10.215.0.36 Link MTU is 1500 bytes
       Jul 06 09:56:00 gna ocserv[3394]: worker[user]: 10.215.0.36 setting up DTLS-PSK connection
       Jul 06 09:56:00 gna ocserv[3394]: worker[user]: 10.215.0.36 worker-vpn.c:1440: error in DTLS handshake: A packet with illegal or unsupported version was re>

Wireshark capture shows client sending DTLS ClientHello with DTLS 1.0 version (0xfeff) in both the record layer and in the handshake.

This configuration works fine if openconnect is built with GnuTLS:

       OpenConnect version v8.05-275-g9d287e4
       Using GnuTLS 3.5.3. Features present: HOTP software token, TOTP software token, System keys, DTLS, ESP
       Supported protocols: anyconnect (default), nc, gp, pulse

In that case, Wireshark shows the client sending DTLS ClientHello with DTLS 1.0 version in the record layer but DTLS 1.2 (0xfefd) in the handshake.

I'm about to start digging into the code, but hoping this might be "obvious"
to someone with more experience with openconnect and DTLS.

Regards,
MV