Broadcast storms - Voyage Linux, Engenius NL2611, hostap
Wed Jun 20 08:32:41 PDT 2007
Please forgive if this is not a hostap issue but I have not been able to
find any other discussion on the problem and it does seem possible that
hostap could be involved. If so, I would appreciate comments or
redirection to a more appropriate forum to help resolve the issue.
I'm running Voyage Linux on WRAP boards, typically with a 5GHz backhaul
in one slot and an Engenius NL2611 in the other set up as an AP.
Engenius F/W is
prism2_hw_init: initialized in 374 ms
wifi0: NIC: id=0x8013 v1.0.0
wifi0: PRI: id=0x15 v1.1.1
wifi0: STA: id=0x1f v1.8.2
dpkg -l shows:
lsmod shows that hostap and hostap_pci are loaded
On occasion, the AP will suffer broadcast storms of ICMP dest
unreachable packets from all the CB3 (also Engenius 2611 radios) clients
as evidenced from the iptraf display of MAC addresses. I also have a
few older Tranzeo units with the same radio that will participate in
these storms. When they occur, I will see over 1000 pkts/sec hitting
the AP, effectively resulting in DOS for the customers. I have 11 of
these units in use with a customer base per AP ranging from 36 down to 6
users per AP. One AP in particular has these storms almost weekly and
two others have had them infrequently while the remainder have *never*
experienced the problem. The storms can occur under the most benign
conditions, ie, light network load, no weather issues, etc. A local
power outage where most/all clients come back on line at once has been
observed to initiate a storm.
The most heavily loaded AP has the same version as the problem child but
itself has never had a broadcast storm.
My primary tool for observation has been iptraf such that I can easily
see the type of packet, the source MAC, the pkts/s and also the normal
traffic interspersed with the storm packets and subsequently recovery
and normal behavior.
I have observed the following characteristics during my ongoing battle
with this problem:
1. The current repair procedure is a script that creates an empty
whitelist, kicks all associated clients off using kickmac and adds them
back one at a time at roughly 10 sec intervals. During this repair
procedure, certain MACs can be more prone than others to re-initiate the
storm. Pausing the repair script can sometimes allow the system to
stabilize on its own. Not infrequently, there may be a very brief storm
of 20 or so packets when adding the next client and then it stabilizes.
2. Reducing the total number of customers has allowed the repair process
to generally work the first time through the 28 customers without the
storm reoccurring. Prior to that, it sometimes took several hours of
flushing and re-adding the users before it would settle down and run
3. Iptables rules to deny packets has no effect.
4. The WRAP board and radio has been swapped out at the radio and board
level with no change in behavior. In all other respects, the system has
been utterly reliable - uptime of 260 days and counting.
5. I have not seen this occur with Atheros based APs but also have some
virtually identical units to the problem one that have never been a problem.
From a high level view, the behavior seems like a classic resonance
problem and could be a phenomenon unique to the particular equipment
installed and even their respective distances from the AP thereby
causing some micro-timing issues in transmission/reception with the AP.
I'm at my wit's end in chasing this ghost. :-( I guess the next step
would be to simply switch to an Atheros-based AP but the Engenius cards
work well in all other respects and I'd really like to use them without
fearing the dreaded storms.
Again, I have no "proof" that the hostap modules are involved in this
but am honestly looking to see if anyone else has even experienced the
problem, and best of all, identified a solution.
My thanks for your time to read this and offer any suggestions. I've
tried to provide sufficient information describing my system without
burying the reader in minutiae. If I've left out a key piece, please
let me know and I'll submit that information.
Edwin Whitelaw, P.E.
New River Valley Unwired, LLC
2200 Lonesome Dove Dr
Christiansburg, VA 24073
More information about the Hostap