From jonathan_jung at web.de  Thu May 12 16:12:08 2011
From: jonathan_jung at web.de (jonathan_jung at web.de)
Date: Thu, 12 May 2011 22:12:08 +0200 (CEST)
Subject: Dead Peer Detection
Message-ID: <1410133832.1006604.1305231129008.JavaMail.fmail@mwmweb034>

An HTML attachment was scrubbed...
URL: <http://lists.infradead.org/pipermail/openconnect-devel/attachments/20110512/5799d3d1/attachment.html>

From dwmw2 at infradead.org  Thu May 12 19:33:20 2011
From: dwmw2 at infradead.org (David Woodhouse)
Date: Fri, 13 May 2011 00:33:20 +0100
Subject: Dead Peer Detection
In-Reply-To: <1410133832.1006604.1305231129008.JavaMail.fmail@mwmweb034>
References: <1410133832.1006604.1305231129008.JavaMail.fmail@mwmweb034>
Message-ID: <1305243202.30435.311.camel@i7.infradead.org>

On Thu, 2011-05-12 at 22:12 +0200, jonathan_jung at web.de wrote:
> Thanks for the help and sorry for the strange/bad english!

The English is perfectly sufficient, but please could you post plain
text email only, not HTML? I accepted the message after it was trapped
for moderation, but now I'm regretting it as I try to make sense of your
output :)


> Connected tun0 as 141.89.47.28, using SSL
> No work to do; sleeping for 20000 ms...
> DTLS handshake timed out
> DTLS handshake failed: 2
> Send CSTP Keepalive
> No work to do; sleeping for 10000 ms...
> Send CSTP DPD
> No work to do; sleeping for 15000 ms...
> Send CSTP DPD
> No work to do; sleeping for 15000 ms...
> Send CSTP DPD
> No work to do; sleeping for 15000 ms...
> CSTP Dead Peer Detection detected dead peer!

The 'dead peer detection' is just a simple ping/pong. We send a DPD
request to the server, and we expect it to reply so that we know we're
still connected.

Except we're *not* still connected. We never get the reply.

> Failed to reconnect to host wlanvpn.uni-potsdam.de
> sleep 10s, remaining timeout 300s

... and we don't even manage to reconnect.

Your tcpdump seems to start just before the dead peer message, and shows
that we keep sending SYN packets to the server to make a new connection,
but they never get a response.

> Nr. 2 - sbin/route -n before the dead peer detection happens:
> /sbin/route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 172.16.3.251 172.16.3.254 255.255.255.255 UGH 0 0 0 wlan0
> 141.89.46.0 141.89.46.106 255.255.255.0 UG 0 0 0 tun0
> 172.16.0.0 0.0.0.0 255.255.252.0 U 0 0 0 wlan0
> 0.0.0.0 141.89.46.106 0.0.0.0 UG 0 0 0 tun0

Ick that's unreadable. I'll try undoing the HTML-damage, and it looks
like this:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.16.3.251    172.16.3.254    255.255.255.255 UGH   0      0        0 wlan0
141.89.46.0     141.89.46.106   255.255.255.0   UG    0      0        0 tun0
172.16.0.0      0.0.0.0         255.255.252.0   U     0      0        0 wlan0
0.0.0.0         141.89.46.106   0.0.0.0         UG    0      0        0 tun0

This looks suspicious.

Your local wireless network is 172.16.0.0/22, so every host from
172.16.0.0 through to 172.16.3.255 is directly on that network segment?

Your VPN server is 172.16.3.251, so shouldn't it be *directly* on the
same network as you? But you have a route to the VPN server *through*
the gateway at 172.16.3.254?

That might explain why your packets aren't reaching the server.

What does your routing table look like after you join the VPN, *before*
you try to connect?

We have special logic in the vpnc-script to preserve your route to the
VPN server, but route everything *else* through the VPN tunnel. I bet
that isn't very well-tested with a VPN server that is actually *on* your
local subnet, and that's why you end up with this strange routing.

If you just run '/sbin/route del 172.16.3.251' as soon as it's
connected, and before it times out, that should delete the rogue route
to the VPN server via that gateway, and let you route to the VPN server
normally. I suspect that'll fix it?

I'll try connecting to my own VPN server through a *proxy* on the local
network, which should trigger the same kind of routing situation for
vpnc-script to screw up, and see if I can reproduce and fix it properly.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse at intel.com                              Intel Corporation



From dwmw2 at infradead.org  Thu May 12 19:46:01 2011
From: dwmw2 at infradead.org (David Woodhouse)
Date: Fri, 13 May 2011 00:46:01 +0100
Subject: Dead Peer Detection
In-Reply-To: <1305243202.30435.311.camel@i7.infradead.org>
References: <1410133832.1006604.1305231129008.JavaMail.fmail@mwmweb034>
	<1305243202.30435.311.camel@i7.infradead.org>
Message-ID: <1305243962.30435.313.camel@i7.infradead.org>

On Fri, 2011-05-13 at 00:33 +0100, David Woodhouse wrote:
> I'll try connecting to my own VPN server through a *proxy* on the
> local network, which should trigger the same kind of routing situation
> for vpnc-script to screw up, and see if I can reproduce and fix it
> properly.

Failed to reproduce on first attempt, but that's because I was doing it
on a machine with iproute (/sbin/ip) installed. If I trick the
vpnc-script into using the obsoltete /sbin/route tool instead, *then* I
see the problem.

So your quick fix is 'apt-get install iproute'.

Perhaps we should make the openconnect package depend on iproute?

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse at intel.com                              Intel Corporation



From hkawakat at gmail.com  Wed May 25 05:11:13 2011
From: hkawakat at gmail.com (Hiroyuki Kawakatsu)
Date: Wed, 25 May 2011 10:11:13 +0100
Subject: Error: either "to" is duplicate, or "ipid" is a garbage.
Message-ID: <BANLkTinfFYXT72iXGvfabC7rJ7dNM-eaUg@mail.gmail.com>

Hi,

Using

polya# openconnect --version
OpenConnect version v2.25

(the debian package), I get (after the connect banner)

Error: either "to" is duplicate, or "ipid" is a garbage.
Connected tun0 as 136.206.133.50, using SSL
DTLS handshake failed: 2

and trying to access the server results in

CSTP Dead Peer Detection detected dead peer!

Does the error message give anyone a clue as to what might be going wrong?
Thanks for any hints/help,

h.
-- 
+---
| Hiroyuki Kawakatsu
| Business School, Dublin City University
| Dublin 9, Ireland. Tel +353 (0)1 700 7496


From dwmw2 at infradead.org  Wed May 25 07:09:00 2011
From: dwmw2 at infradead.org (David Woodhouse)
Date: Wed, 25 May 2011 12:09:00 +0100 (BST)
Subject: Error: either "to" is duplicate, or "ipid" is a garbage.
In-Reply-To: <BANLkTinfFYXT72iXGvfabC7rJ7dNM-eaUg@mail.gmail.com>
References: <BANLkTinfFYXT72iXGvfabC7rJ7dNM-eaUg@mail.gmail.com>
Message-ID: <alpine.LFD.2.02.1105251202340.3164@localhost6.localdomain6>

On Wed, 25 May 2011, Hiroyuki Kawakatsu wrote:

> polya# openconnect --version
> OpenConnect version v2.25

Very old... :(

> (the debian package), I get (after the connect banner)
> 
> Error: either "to" is duplicate, or "ipid" is a garbage.

Hm, is your vpnc-script just as out of date? I bet Debian is still 
shipping the old one from vpnc itself rather than a separate 
vpnc-script package?

Can you try this with the latest script from my git repository (as linked 
from the opoenconnect web page)?

If that doesn't fix it, please send '/sbin/ip route' output before and 
after connection, along with openconnect output with the '-v' option. 
Since you're using such an old version, be careful to elide the 'webvpn' 
cookie before posting. Current versions don't print that.

-- 
dwmw2



From hkawakat at gmail.com  Fri May 27 01:50:59 2011
From: hkawakat at gmail.com (Hiroyuki Kawakatsu)
Date: Fri, 27 May 2011 06:50:59 +0100
Subject: Error: either "to" is duplicate, or "ipid" is a garbage.
In-Reply-To: <BANLkTiksB14xjeST4opfH8Dm55ajac52Ow@mail.gmail.com>
References: <BANLkTinfFYXT72iXGvfabC7rJ7dNM-eaUg@mail.gmail.com>
	<alpine.LFD.2.02.1105251202340.3164@localhost6.localdomain6>
	<BANLkTingPsOFbKUY3Wtju6k6k9TicPtsPw@mail.gmail.com>
	<1306339113.7218.125.camel@i7.infradead.org>
	<BANLkTiksB14xjeST4opfH8Dm55ajac52Ow@mail.gmail.com>
Message-ID: <BANLkTi=dmrEpHbouASiB2Ehuk1Qi6WUfAA@mail.gmail.com>

Thanks to off-line help from David Woodhouse, I uncommented the `set
-x' line in vpnc-script and tracked the error to a call to `ip route'.
A google search gave me the following clear (even to a complete n00b
like me) diagnosis and fix to the vpnc-script. This fix (the regexp
can be improved?) to the latest vpnc-script also works for me with
openconnect v2.25:

https://lkml.org/lkml/2011/3/24/645

h.
-- 
+---
| Hiroyuki Kawakatsu
| Business School, Dublin City University
| Dublin 9, Ireland. Tel +353 (0)1 700 7496


From mkitchin.public at gmail.com  Fri May 27 16:25:47 2011
From: mkitchin.public at gmail.com (Matthew Kitchin (public/usenet))
Date: Fri, 27 May 2011 15:25:47 -0500
Subject: reconnect script
Message-ID: <4DE008CB.8040305@gmail.com>

Openconnect version: OpenConnect version v2.25-unknown (That is what it 
reports)
Openwrt Backfire

I'm currently running the very low tech reconnect script below.

ping -c 4 10.85.0.1 2>/dev/null 1>/dev/nul
       if [ $? -ne 0 ]; then
          killall openconnect
          echo passwd|openconnect https://myIP --no-cert-check --script 
/etc/vpn-script --user=username --passwd-on-stdin
       fi

It runs every minute. I know it is not the ideal way to do it. I tried 
to get it working in a cleaner way with an init script, but I never 
could get it to reconnect properly.
After about 3 days, I lose all connectivity between the devices on my 
192.168.1.x subnet. The devices can still get to the internet, still get 
to the remote vpn subnets, but cannot see each other. Killing the 
openconnect process fixes this. Rebooting the router fixes everything.
Again, I completely admit this is not the best way to do this. Before I 
deal with trying to change it, I wanted to see if you thought that would 
cause the problem I'm experiencing. Would you suspect this was some sort 
of openwrt issue instead?

Thanks,
Matthew


From dwmw2 at infradead.org  Fri May 27 17:27:42 2011
From: dwmw2 at infradead.org (David Woodhouse)
Date: Fri, 27 May 2011 22:27:42 +0100
Subject: reconnect script
In-Reply-To: <4DE008CB.8040305@gmail.com>
References: <4DE008CB.8040305@gmail.com>
Message-ID: <1306531663.2029.173.camel@i7.infradead.org>

On Fri, 2011-05-27 at 15:25 -0500, Matthew Kitchin (public/usenet)
wrote:
> Openconnect version: OpenConnect version v2.25-unknown (That is what it 
> reports)

That'll be because it's patched so it's not a "clean" v2.25. You ought
to upgrade to v3.02.

> After about 3 days, I lose all connectivity between the devices on my 
> 192.168.1.x subnet. The devices can still get to the internet, still get 
> to the remote vpn subnets, but cannot see each other.

How are the machines connected to one another? And if they can still
happily see everything but each other, wtf is your 'ping 10.85.0.1'
doing?

What does your routing table look like before you kill the offending
openconnect process, what does it look like when openconnect is dead,
and what does it look like when you've run a new openconnect process and
everything is working again?

>  Killing the openconnect process fixes this. Rebooting the router
> fixes everything.

Where does the router fit in the network? Can you describe your network
topology? I'm completely lost.

Do you have 'openconnect -v' output from the time when this goes wrong?
I assume it thinks it's still working correctly, given that traffic
*over* it is still working OK?

-- 
dwmw2



From mkitchin.public at gmail.com  Sat May 28 23:27:58 2011
From: mkitchin.public at gmail.com (Matthew Kitchin (public/usenet))
Date: Sat, 28 May 2011 22:27:58 -0500
Subject: reconnect script
In-Reply-To: <1306531663.2029.173.camel@i7.infradead.org>
References: <4DE008CB.8040305@gmail.com>
	<1306531663.2029.173.camel@i7.infradead.org>
Message-ID: <4DE1BD3E.7020205@gmail.com>

On 5/27/2011 4:27 PM, David Woodhouse wrote:
> On Fri, 2011-05-27 at 15:25 -0500, Matthew Kitchin (public/usenet)
> wrote:
>> Openconnect version: OpenConnect version v2.25-unknown (That is what it
>> reports)
> That'll be because it's patched so it's not a "clean" v2.25. You ought
> to upgrade to v3.02.
> i
I tried to compile a package for Openwrt last time, and it didn't go 
very well. I spent a great deal of time, and made little progress. I 
believe that is why I'm running a patched version.
>> After about 3 days, I lose all connectivity between the devices on my
>> 192.168.1.x subnet. The devices can still get to the internet, still get
>> to the remote vpn subnets, but cannot see each other.
> How are the machines connected to one another? And if they can still
> happily see everything but each other, wtf is your 'ping 10.85.0.1'
> doing?
>
All the machines at my house are connected by wifi to the Openwrt box. 
Openwrt has the vpn connection to remote office. All home machines are 
on the 192.168.1.X subnet. 10.85.0.1 is a router IP at my corporate 
office I ping to check if the VPN connection is up or not.
> What does your routing table look like before you kill the offending
> openconnect process, what does it look like when openconnect is dead,
> and what does it look like when you've run a new openconnect process and
> everything is working again?
>
Will do as soon as it dies again.

>>   Killing the openconnect process fixes this. Rebooting the router
>> fixes everything.
> Where does the router fit in the network? Can you describe your network
> topology? I'm completely lost.
6 devices at home on 192.168.1.x network. Router is Openwrt backfire 
with openconnect. IT connects to corporate office, and routes traffic 
for 10.85.0.x and 10.92.x.x
> Do you have 'openconnect -v' output from the time when this goes wrong?
> I assume it thinks it's still working correctly, given that traffic
> *over* it is still working OK?
>
Will do when it dies again.



From dwmw2 at infradead.org  Sun May 29 04:33:52 2011
From: dwmw2 at infradead.org (David Woodhouse)
Date: Sun, 29 May 2011 09:33:52 +0100
Subject: reconnect script
In-Reply-To: <4DE1BD3E.7020205@gmail.com>
References: <4DE008CB.8040305@gmail.com>
	<1306531663.2029.173.camel@i7.infradead.org>
	<4DE1BD3E.7020205@gmail.com>
Message-ID: <1306658033.2029.349.camel@i7.infradead.org>

On Sat, 2011-05-28 at 22:27 -0500, Matthew Kitchin (public/usenet)
wrote:
> I tried to compile a package for Openwrt last time, and it didn't go 
> very well. I spent a great deal of time, and made little progress. I 
> believe that is why I'm running a patched version.

I don't think your current problem is related to the fact that you're
running v2.25.

> >> After about 3 days, I lose all connectivity between the devices on my
> >> 192.168.1.x subnet. The devices can still get to the internet, still get
> >> to the remote vpn subnets, but cannot see each other.
> > How are the machines connected to one another? And if they can still
> > happily see everything but each other, wtf is your 'ping 10.85.0.1'
> > doing?
> >
> All the machines at my house are connected by wifi to the Openwrt box. 
> Openwrt has the vpn connection to remote office. All home machines are 
> on the 192.168.1.X subnet. 10.85.0.1 is a router IP at my corporate 
> office I ping to check if the VPN connection is up or not.

OK, but you said that when the problem happens, they can still get to
the remote VPN subnets. So this 'ping 10.85.0.1' check would still be
*working*, and thus fail to detect the problem?

Or did I misunderstand, and your 'reconnect script' isn't actually
solving the problem with the 192.168.1.x hosts on the wireless not being
able to talk to each other? Except that when it *happens* to trigger
because the VPN goes down, that somehow solves the (unrelated?) wlan
issue?

It sounds like this is an OpenWRT routing problem. If hosts on your
*wireless* cannot see each other, that really shouldn't be anything to
do with openconnect. If restarting openconnect 'fixes' it then that
really makes me suspect the routing setup, because that's all that
openwer will really change. So I'll need to see those routing tables in
all three cases.

-- 
dwmw2



From dwmw2 at infradead.org  Sun May 29 04:59:08 2011
From: dwmw2 at infradead.org (David Woodhouse)
Date: Sun, 29 May 2011 09:59:08 +0100
Subject: reconnect script
In-Reply-To: <4DE008CB.8040305@gmail.com>
References: <4DE008CB.8040305@gmail.com>
Message-ID: <1306659549.2029.355.camel@i7.infradead.org>

On Fri, 2011-05-27 at 15:25 -0500, Matthew Kitchin (public/usenet) wrote:
> 
> ping -c 4 10.85.0.1 2>/dev/null 1>/dev/nul
>        if [ $? -ne 0 ]; then
>           killall openconnect
>           echo passwd|openconnect https://myIP --no-cert-check --script 
> /etc/vpn-script --user=username --passwd-on-stdin
>        fi 

FWIW this is the script I use to keep openconnect up. It pauses between
reconnect attempts if it's failing to connect, and it aborts if it gets
a login failure...

OpenConnect itself should always exit when it's no longer working, as
long as DPD is running. And if for some reason it isn't requested by
your server, you can use the --force-dpd option to enable it anyway.

#!/bin/bash

VPNSERVER=x.x.x.x
read -p "Enter VPN password: " -s PASS

while true; do 
    route add default gw 90.155.92.193
    THEN=`date +%s`
	echo $PASS | openconnect -c /home/dwmw2/.cert/certificate.p12 --key-password-from-fsid $VPNSERVER -x foo.xml --script /etc/vpnc/vpnc-script --mtu 1266 -u david.woodhouse at intel.com --passwd-on-stdin
    if [ $? -eq 2 ]; then
	echo Exit code 2
	exit 1
    fi
    NOW=`date +%s`
    if [ $NOW -lt $(($THEN + 60)) ]; then
	echo Need to wait $(($THEN + 60 - $NOW)) seconds
	sleep $(($THEN + 60 - $NOW))
    fi
done

-- 
dwmw2



From mkitchin.public at gmail.com  Sun May 29 09:06:11 2011
From: mkitchin.public at gmail.com (Matthew Kitchin (public/usenet))
Date: Sun, 29 May 2011 08:06:11 -0500
Subject: reconnect script
In-Reply-To: <1306658033.2029.349.camel@i7.infradead.org>
References: <4DE008CB.8040305@gmail.com>	
	<1306531663.2029.173.camel@i7.infradead.org>
	<4DE1BD3E.7020205@gmail.com>
	<1306658033.2029.349.camel@i7.infradead.org>
Message-ID: <4DE244C3.2020207@gmail.com>

On 5/29/2011 3:33 AM, David Woodhouse wrote:
> On Sat, 2011-05-28 at 22:27 -0500, Matthew Kitchin (public/usenet)
> wrote:
>> I tried to compile a package for Openwrt last time, and it didn't go
>> very well. I spent a great deal of time, and made little progress. I
>> believe that is why I'm running a patched version.
> I don't think your current problem is related to the fact that you're
> running v2.25.
>
>>>> After about 3 days, I lose all connectivity between the devices on my
>>>> 192.168.1.x subnet. The devices can still get to the internet, still get
>>>> to the remote vpn subnets, but cannot see each other.
>>> How are the machines connected to one another? And if they can still
>>> happily see everything but each other, wtf is your 'ping 10.85.0.1'
>>> doing?
>>>
>> All the machines at my house are connected by wifi to the Openwrt box.
>> Openwrt has the vpn connection to remote office. All home machines are
>> on the 192.168.1.X subnet. 10.85.0.1 is a router IP at my corporate
>> office I ping to check if the VPN connection is up or not.
> OK, but you said that when the problem happens, they can still get to
> the remote VPN subnets. So this 'ping 10.85.0.1' check would still be
> *working*, and thus fail to detect the problem?
Correct. They can still get to 10.85.0.1. That check was only put in 
there to detect when the VPN connection goes down. I believe my VPN 
server is set to disconnect at 12 hours. I have not put in anything 
automated to try and fix this local lan problem
> Or did I misunderstand, and your 'reconnect script' isn't actually
> solving the problem with the 192.168.1.x hosts on the wireless not being
> able to talk to each other? Except that when it *happens* to trigger
> because the VPN goes down, that somehow solves the (unrelated?) wlan
> issue?
Correct. It is not solving this problem. When the problem has occurred, 
if I simply kill the openconnect process, the local connection between 
the devices on the 192.168.1.x subnet is resolved. Perhaps this problem 
is happening more often than I thought, and it is being resolved every 
time the script runs to reconnect the VPN connection.
> It sounds like this is an OpenWRT routing problem. If hosts on your
> *wireless* cannot see each other, that really shouldn't be anything to
> do with openconnect. If restarting openconnect 'fixes' it then that
> really makes me suspect the routing setup, because that's all that
> openwer will really change. So I'll need to see those routing tables in
> all three cases.
>
I don't disagree at all. I thought I would start here because stopping 
openconnect seemed to resolve the problem. Here is routing info right 
now when all is well.
Openwrt:
root at OpenWrt:~# route -e
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt 
Iface
10.85.0.10      10.70.6.134     255.255.255.255 UGH       0 0          0 
tun0
10.85.0.11      10.70.6.134     255.255.255.255 UGH       0 0          0 
tun0
216.248.9.102   c-68-63-248-1.h 255.255.255.255 UGH       0 0          0 
eth1
10.70.6.0       10.70.6.134     255.255.255.0   UG        0 0          0 
tun0
192.168.1.0     *               255.255.255.0   U         0 0          0 
br-lan
68.63.248.0     *               255.255.248.0   U         0 0          0 
eth1
172.27.0.0      10.70.6.134     255.255.0.0     UG        0 0          0 
tun0
10.85.0.0       10.70.6.134     255.255.0.0     UG        0 0          0 
tun0
10.92.0.0       10.70.6.134     255.255.0.0     UG        0 0          0 
tun0
default         c-68-63-248-1.h 0.0.0.0         UG        0 0          0 
eth1


Windows machine on wifi:
C:\Users\Matthew Kitchin>route print
===========================================================================
Interface List
  10...00 1e 58 3f a4 95 ......D-Link DWA-556 Xtreme N PCIe Desktop Adapter
   9...00 1e 0b a1 f9 0b ......Intel(R) 82566DM-2 Gigabit Network Connection
   1...........................Software Loopback Interface 1
  15...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter
  25...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
  13...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
           0.0.0.0          0.0.0.0      192.168.1.1    192.168.1.221     25
         127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
         127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
   127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
       192.168.1.0    255.255.255.0         On-link     192.168.1.221    281
     192.168.1.221  255.255.255.255         On-link     192.168.1.221    281
     192.168.1.255  255.255.255.255         On-link     192.168.1.221    281
         224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
         224.0.0.0        240.0.0.0         On-link     192.168.1.221    281
   255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
   255.255.255.255  255.255.255.255         On-link     192.168.1.221    281
===========================================================================
Persistent Routes:
   None

IPv6 Route Table
===========================================================================
Active Routes:
  If Metric Network Destination      Gateway
  13     58 ::/0                     On-link
   1    306 ::1/128                  On-link
  13     58 2001::/32                On-link
  13    306 2001:0:4137:9e76:30be:3346:bbc0:62a/128
                                     On-link
  10    281 fe80::/64                On-link
  13    306 fe80::/64                On-link
  13    306 fe80::30be:3346:bbc0:62a/128
                                     On-link
  10    281 fe80::7830:bd1c:38ae:22fc/128
                                     On-link
   1    306 ff00::/8                 On-link
  13    306 ff00::/8                 On-link
  10    281 ff00::/8                 On-link
===========================================================================
Persistent Routes:
   None

C:\Users\Matthew Kitchin>






