<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 11 Sep 2015, at 15:43, N.Leiten <<a href="mailto:nickleiten@gmail.com" class="">nickleiten@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">In email dated Пятница - 11 сентября 2015 13:49:26 user Felix Fietkau wrote:</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">On 2015-09-11 13:33, N.Leiten wrote:<br class=""><blockquote type="cite" class="">Fix instability of Ralink WiFi general queue management on high load.<br class="">rt2x00 driver logs in dmesg "Dropping frame due to full queue ..." several times and at some point get stuck.<br class=""><br class="">Solutions in patch:<br class="">1) Increasing number of frames in each TX queue helps with speed and<br class="">decreases queue overflows. Actually 256 frames can be increased to<br class="">512 (this number of frames used in proprietary drivers for every<br class="">queue).<br class=""></blockquote>512 frames seems to be overly excessive. Ever heard of bufferbloat?<br class="">It seems to me that the driver should simply call ieee80211_stop_queue<br class="">earlier to ensure that mac80211 does not attempt to fill the queues as<br class="">much. The driver queue length should probably be around 64 or less.<br class=""><br class=""></blockquote><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">I agree. The patch was made for internal use on own hardware reference design in our company, so I had to look through proprietary and opensource drivers to see the difference. The point was to use recent versions of OpenWRT with mac80211 stack because proprietary still uses linux-2.6. And it makes porting full of pain in nails and bloody eyes reading that piece of code. For now mac80211 is preferrable.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">As to bufferbloat - yes I'm aware of it. Maybe I'm wrong in observations, but simply increasing queue num to 128 frames make driver more stable but not enough, so I started digging in ralink drivers for maximum value, tried it and it increased stability even more, but still hang at some point. So I decreased it to 256. Well, I'll try to remove this part and test again.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></blockquote><div><br class=""></div>I think I solved the stuck tx-queue issue a few years ago by increasing the size of the txstatus fifo. Now looking at the code, I suggest you to try the following. </div><div><br class=""></div><div><b class="">in rt2800mmio.c:</b></div><div><b class=""><span class="Apple-tab-span" style="white-space:pre"> </span>static void rt2800mmio_txstatus_interrupt(struct rt2x00_dev *rt2x00dev…</b></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>I don’t know to what end was this added here but anyways remove the read limit, since it’s obvious that there might be more than just one tx-queue being used. Exiting the interrupt handler here with half full hw fifo (16 entries) will lead to lost txstatuses for sure:</div><div><div><span class="Apple-tab-span" style="white-space:pre"> <b class=""> </b></span><b class="">if (++i >= rt2x00dev->tx->limit)</b></div><div><b class=""> break;</b></div><div><br class=""></div><div><b class="">in rt2x00dev.c</b></div><div><b class=""><span class="Apple-tab-span" style="white-space:pre"> </span>static int rt2x00lib_probe_hw(struct rt2x00_dev *rt2x00dev)…</b></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>I used a static value here, but it seems that the hw might generate excess txstatuses (who knows why). So bigger is better!</div><div><span class="Apple-tab-span" style="white-space:pre"> <b class=""> </b></span><b class="">-int kfifo_size = roundup_pow_of_two(rt2x00dev->ops->tx_queues * rt2x00dev->tx->limit * sizeof(u32));</b></div><div><b class=""><span class="Apple-tab-span" style="white-space:pre"> </span>+int kfifo_size = roundup_pow_of_two(2048 * sizeof(u32));</b></div><div><br class=""></div><div>With these you should no longer be experiencing stuck tx-queues!</div><div class=""><br class=""></div></div><div><blockquote type="cite" class=""><br class=""></blockquote><blockquote type="cite" class=""><div class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class="">2) Setting number of frames in TX/RX queues to equal values<br class="">resolves async speed behaviour with better RX on AP-side (uplink from<br class="">STAs), where it needs to be at least equal or better on TX queue on<br class="">AP (download to STA).<br class=""></blockquote>That also doesn't make much sense to me as queueing behavior is<br class="">completely different between rx and tx. Rx queue processing speed is<br class="">determined by CPU speed, Tx queue processing speed is determined by<br class="">shared medium speed.<br class=""><br class=""></blockquote><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">I agree, that was another step in digging. As I can understand it's better even use NAPI for RX queue. But I found only one mention of it in mac80211 sources, need to explore this.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></blockquote><div><br class=""></div><div>The driver already uses tasklets and rx-interrupts stay disabled until the driver "catches up” with the rx-queue. So, there’s little to none performance gain here with napi. Although, I did see a noticeable performance increase by enabling delayed rx-interrupts from the hw. As if the hw is sending a BA every time when RX_CRX_IDX register is written? </div><br class=""><blockquote type="cite" class=""><div class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">I'll remove it, test again and make new patch. Can you direct me with this kind of behaviour? On rt3092 (2T2R) we can get 30-40Mbit/s download and 80-120Mbit/s upload speeds in separate tests, on 1T1R and 2T2R stations.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class="">3) In rt2x00mac.c additional check for queue<br class="">full added and reassignment in this case, so interface will not drop<br class="">frame.<br class=""></blockquote>Why? If the queues are full, it's better to just drop packets instead of<br class="">making the problem worse.<br class=""><br class=""></blockquote><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">It was the simpliest solution at this point. I'll try to use ieee80211_stop_queue instead, but don't know how much time it'll consume.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Maybe there's problem in "kick/pause" queue mechanism? I managed to explore behaviour after hang-point. It seems that only QID_AC_* queues stuck, station assoc/auth goes fine but actual data not transmitted on AP-side, but it receives DHCP-requests from station and it looks like no network access on hanged AP.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></blockquote><div><br class=""></div><div>When the queue is stuck, you can see that the queue is filled to it’s threshold and it’s paused. So, the kick&pause works as it should.</div><br class=""><blockquote type="cite" class=""><div class=""><br class=""></div></blockquote><blockquote type="cite" class=""><div class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class="">4) Fixes in queue initialization. Default values for AC_BK,<br class="">AC_BE, AC_VI, AC_VO set from WMM.<br class=""></blockquote>Why do you hardcode that stuff inside the driver? What difference do<br class="">these values make?<br class=""><br class=""><blockquote type="cite" class="">Tested on RT3883, RT5350, MT7620 SoCs and on RT3092 pcie interface for 10 days.<br class=""><br class="">I'm planning to send this patch to mac80211 soon, but need to be sure that it works on other Ralink/Mediatek platforms and it's appropriate to do so.<br class=""><br class=""><br class="">Signed-off-by: Nick Leiten <<a href="mailto:nickleiten@gmail.com" class="">nickleiten@gmail.com</a>><br class="">diff --git a/package/kernel/mac80211/patches/999-rt2x00-queue-update.patch b/package/kernel/mac80211/patches/999-rt2x00-queue-update.patch<br class="">new file mode 100644<br class="">index 0000000..9239bec<br class="">--- /dev/null<br class="">+++ b/package/kernel/mac80211/patches/999-rt2x00-queue-update.patch<br class="">@@ -0,0 +1,142 @@<br class="">+Only in compat-wireless-2015-03-09/drivers/net/wireless/rt2x00: limit<br class="">+diff -c -r compat-wireless-2015-03-09-orig/drivers/net/wireless/rt2x00/rt2800mmio.c compat-wireless-2015-03-09/drivers/net/wireless/rt2x00/rt2800mmio.c<br class="">+*** compat-wireless-2015-03-09-orig/drivers/net/wireless/rt2x00/rt2800mmio.c<span class="Apple-tab-span" style="white-space: pre;"> </span>2015-06-16 13:02:30.000000000 +0300<br class="">+--- compat-wireless-2015-03-09/drivers/net/wireless/rt2x00/rt2800mmio.c<span class="Apple-tab-span" style="white-space: pre;"> </span>2015-09-04 11:50:09.665148666 +0300<br class="">+***************<br class="">+*** 700,706 ****<br class="">+ <br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>switch (queue->qid) {<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>case QID_RX:<br class="">+!<span class="Apple-converted-space"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->limit = 128;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->data_size = AGGREGATION_SIZE;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->desc_size = RXD_DESC_SIZE;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->winfo_size = rxwi_size;<br class="">+--- 700,706 ----<br class="">+ <br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>switch (queue->qid) {<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>case QID_RX:<br class="">+!<span class="Apple-converted-space"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->limit = 256;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->data_size = AGGREGATION_SIZE;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->desc_size = RXD_DESC_SIZE;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->winfo_size = rxwi_size;<br class="">+***************<br class="">+*** 711,717 ****<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>case QID_AC_VI:<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>case QID_AC_BE:<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>case QID_AC_BK:<br class="">+!<span class="Apple-converted-space"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->limit = 64;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->data_size = AGGREGATION_SIZE;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->desc_size = TXD_DESC_SIZE;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->winfo_size = txwi_size;<br class="">+--- 711,717 ----<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>case QID_AC_VI:<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>case QID_AC_BE:<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span>case QID_AC_BK:<br class="">+!<span class="Apple-converted-space"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->limit = 256;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->data_size = AGGREGATION_SIZE;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->desc_size = TXD_DESC_SIZE;<br class="">+ <span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue->winfo_size = txwi_size;<br class=""></blockquote>Wrong patch style, please run make package/mac80211/refresh.<br class=""><br class=""></blockquote><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Ok, will fix it.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">- Felix<br class=""></blockquote><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">openwrt-devel mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:openwrt-devel@lists.openwrt.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">openwrt-devel@lists.openwrt.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel</a></div></blockquote></div><br class=""></body></html>