[Bisected] Connection failures when using wl driver (Broadcom) since rev. f2a6ad01

Dan Williams dcbw at redhat.com
Wed Jan 4 08:25:26 PST 2017 

On Fri, 2016-12-23 at 21:56 +0300, Evgenii Shatokhin wrote:
> On 19.12.2016 18:53, Dan Williams wrote:
> > On Mon, 2016-12-19 at 15:11 +0300, Evgenii Shatokhin wrote:
> > > On 19.12.2016 14:00, Jouni Malinen wrote:
> > > > 
> > > > On Mon, Dec 19, 2016 at 12:28:45PM +0300, Evgenii Shatokhin
> > > > wrote:
> > > > > 
> > > > > After update of wpa_supplicant from v.2.4 to v2.6, the system
> > > > > can
> > > > > no
> > > > > longer connect to WiFi networks. The problem persists from
> > > > > the
> > > > > current version of wpa_supplicant from git (rev. aaa9c60b) as
> > > > > well.
> > > > > 
> > > > > NetworkManager shows "Configuring interface" for the chosen
> > > > > SSID
> > > > > and
> > > > > after some time just tells that the connection failed
> > > > > ("disconnected")
> > > > > 
> > > > > The problem only happens when I use the proprietary driver
> > > > > (wl)
> > > > > from
> > > > > Broadcom for the WiFi adapter. Everything seems to be OK with
> > > > > the
> > > > > opensource driver though.
> > > > > 
> > > > > The log of wpa_supplicant is attached.
> > > > 
> > > > What kind of changes are there in wpa_supplicant on top of the
> > > > actually
> > > > released versions? This logs show following line:
> > > > 
> > > > properties_get_or_set: Set(PreassocMacAddr)
> > > 
> > > No additional patches, I just built the code from the git repo,
> > > master
> > > branch. The configuration is attached, if that matters.
> > 
> > I would take this discussion to the NetworkManager list for now.
> >   Randomized MAC addresses and TLS v1.2 have been problematic for a
> > couple of reasons.
> > 
> > First we've found a number of drivers (like brcmfmac) that don't
> > work
> > with randomized MAC addresses because they don't actually set that
> > address immediately, but defer it to a workqueue for later and
> > return
> > success.  That causes some issues between NM and the supplicant's
> > event
> > processing behavior when the MAC address finally does change.  I
> > wouldn't be surprised if the wl driver code does the same thing.
> > 
> > TLS v1.2 has also been problematic in some cases, but I would
> > suspect
> > randomized MAC addresses first.
> > 
> > Anyway, better to take this to the NM list to root-cause the issue
> > and
> > if it does turn out to be the supplicant, we can come back around.
> > 
> > Dan
> 
> Well, I have updated NM to version 1.4.2 (previously version 1.2.x
> was 
> used) and the problem seems to be gone. It is not quite obvious to
> me 
> from NM's git repo yet, how exactly if was fixed but still.

After the randomization was deployed, the NM team found some issues
where NM didn't wait long enough (or verify well enough) that some
drivers had actually changed the MAC address before continuing with
configuration.  That impacted drivers like wl, brcmfmac, and some
others that report "success!" to the MAC change request, but don't
actually change the MAC address for a short period of time.  NM was not
waiting for the change to actually happen in the device, and was
assuming that the driver had changed the MAC after reporting success.

Dan


> I should have guessed earlier the culprit is in NM, that would save
> some 
> time.
> 
> If the problem shows up again, I will report it to the NM's
> developers 
> like you suggest.
> 
> Anyway, thanks to everyone for explanations and quick replies.
> 
> > 
> > > > 
> > > > 
> > > > while PreassocMacAddr does not seem to exist in hostap.git
> > > > version
> > > > at
> > > > all.. Please also note that this configures random MAC address
> > > > use
> > > > which
> > > > is failing in the debug log apparently due to driver not
> > > > supporting
> > > > it.
> > > > 
> > > > > 
> > > > > Version 2.4 worked OK with both drivers.
> > > > 
> > > > Using random MAC addresses?
> > > 
> > > I made no changes in the connection settings.
> > > 
> > > I just checked out tag hostap_2_4 in git repo, copied the
> > > attached
> > > config file to .config, then - cd wpa_supplicant && make. Then as
> > > follows:
> > > 
> > > systemctl stop wpa_supplicant
> > > cp wpa_supplicant wpa_cli wpa_passphrase /usr/sbin/
> > > systemctl start wpa_supplicant
> > > 
> > > iw dev wlan0 scan # to force the scan for the NM to see the
> > > connections
> > > 
> > > After that, I selected the appropriate SSID in NM and the
> > > connection
> > > was
> > > established.
> > > 
> > > Checkout 'master' again, do the same steps => connection failure.
> > > 
> > > > 
> > > > 
> > > > > 
> > > > > git bisect pointed to the following commit as the first "bad"
> > > > > one:
> > > > > commit f2a6ad01a943103c658de5721c2d7f7e91ee7fa4
> > > > > Author: Jouni Malinen <j at w1.fi>
> > > > > Date:   Sun Nov 29 18:59:27 2015 +0200
> > > > > 
> > > > >       TLS client: Add support for server certificate probing
> > > > 
> > > > I would find it very unlikely for this commit to have anything
> > > > to
> > > > do
> > > > with this issue.. Are you even building in the code this
> > > > touches
> > > > (i.e.,
> > > > CONFIG_TLS=internal)? In any case, that functionality is not
> > > > used
> > > > before
> > > > the association is established.
> > > > 
> > > > > 
> > > > > dbus: org.freedesktop.DBus.Properties.Set
> > > > > (/fi/w1/wpa_supplicant1/Interfaces/1) [ssv]
> > > > > properties_get_or_set: Set(PreassocMacAddr)
> > > > > preassoc_mac_addr=1
> > > > 
> > > > This functionality does not exist in upstream version over D-
> > > > Bus
> > > > interface..
> > > 
> > > Could it be that NetworkManager messes with these things somehow?
> > > Because, I used the original, unpached code for testing.
> > > 
> > > By the way, if that matters, the following service file is used
> > > for
> > > wpa_supplicant in that system:
> > > 
> > > -----------------
> > > [Unit]
> > > Description=WPA Supplicant daemon
> > > Before=network.target
> > > After=syslog.target
> > > 
> > > [Service]
> > > Type=dbus
> > > BusName=fi.w1.wpa_supplicant1
> > > EnvironmentFile=-/etc/sysconfig/wpa_supplicant
> > > ExecStart=/usr/sbin/wpa_supplicant -c /etc/wpa_supplicant.conf
> > > $INTERFACES $DRIVERS $OTHER_ARGS
> > > 
> > > [Install]
> > > WantedBy=multi-user.target
> > > -----------------
> > > 
> > > /etc/sysconfig/wpa_supplicant:
> > > -----------------
> > > INTERFACES=""
> > > DRIVERS=""
> > > OTHER_ARGS="-u -f /var/log/wpa_supplicant.log -P
> > > /var/run/wpa_supplicant.pid"
> > > -----------------
> > > 
> > > /etc/wpa_supplicant.conf - the same as
> > > wpa_supplicant/wpa_supplicant.conf in the git repo but with
> > > network={}
> > > blocks commented out.
> > > 
> > > No mention of PreassocMacAddr or similar stiff here which makes
> > > me
> > > think
> > > it is set by something else.
> > > 
> > > > 
> > > > 
> > > > > 
> > > > > wlan0: Starting radio work 'scan'@0x1fb5440 after 0.000057
> > > > > second
> > > > > wait
> > > > > Could not set interface wlan0 hwaddr: Too many open files in
> > > > > system
> > > > > nl80211: failed to set_mac_addr for wlan0 to
> > > > > 96:a4:b3:79:fd:89
> > > > > wlan0: Failed to set random MAC address
> > > > > wlan0: Failed to assign random MAC address for a scan
> > > > 
> > > > and this is where that operation enabled by preassoc_mac_addr=1
> > > > is
> > > > failing in the driver..
> > > > 
> > > > > 
> > > > > dbus: org.freedesktop.DBus.Properties.Set
> > > > > (/fi/w1/wpa_supplicant1/Interfaces/1) [ssv]
> > > > > properties_get_or_set: Set(MacAddr)
> > > > > mac_addr=0
> > > > 
> > > > and that one does not exists either in upstream..
> > > > 
> > > > 
> > > > In other words, it looks like there are some custom changes in
> > > > wpa_supplicant and those changes are in the area that the used
> > > > driver
> > > > does not support. I'd test what happens with that functionality
> > > > (random
> > > > MAC address) disabled.
> > > > 
> > > 
> > > Hm, it looks like using random MAC addresses is not enabled in
> > > the
> > > NM's
> > > gui for that connection, but it could be a bug in the gui.
> > > 
> > > I will try to hard-code the same MAC address as the device has,
> > > just
> > > in
> > > case. And/or try without NM to take it out of the question.
> > > 
> > > Thanks for a quick reply, by the way.
> 
> Regards,
> Evgenii
> 
> 
> _______________________________________________
> Hostap mailing list
> Hostap at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/hostap

More information about the Hostap mailing list