What to do about hung firmware?

[Ben Greear]

On 11/05/2013 11:07 PM, Michal Kazior wrote:
> On 5 November 2013 19:51, Ben Greear <greearb at candelatech.com> wrote:
>> I'm seeing cases where it appears the firmware just gets
>> stuck and will not answer any WMI requests.
>
> You probably mean FW doesn't replenish HTT TX credits for WMI.

Maybe so..I added some debugging to check on that, but then of course
I could not reproduce the problem.

>> ath10k just patiently keeps timing out WMI commands,
>> (while holding locks, and making the whole system run slow).
>>
>> Should we maybe keep a last-msg-from firmware time stamp
>> and just whack the firmware if we detect it hung?  In addition
>> to this, we could add some 'ping' message that will get sent
>> periodically to the firmware to make sure it is alive.
>> We should be able to do this with existing WMI API, just
>> need to pick a message to send that expects some response.
>
> Probably the easiest/shortest way to do this is to store a timestamp
> in ath10k_wmi_op_ep_tx_credits() and check against it in
> ath10k_wmi_cmd_send(). Once you deem FW stopped responding you could
> queue ar->restat_work.
>
> You probably could try WMI_ECHO_CMDID to implement a keep alive when
> idling (i.e. not sending WMI commands for a few seconds at least).

Sounds good.  I'll work on this if I start seeing the lockups
again...

Thanks,
Ben


-- 
Ben Greear <greearb at candelatech.com>
Candela Technologies Inc  http://www.candelatech.com