[PATCH v4 28/33] dmaengine: sf-pdma: route error callbacks through channel BH

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH work,
but SF-PDMA invokes the client callback directly from its error tasklet
after exhausting its retries. That descriptor is never actually
completed: it stays on the issued list and in chan->desc, the cookie is
never advanced, and the channel does not move on to the next transfer.

Complete the failed descriptor through virt-dma instead, mirroring what
the done tasklet does for a successful transfer: unlink it from the
issued list, mark it DMA_TRANS_ABORTED with the untransferred size as
residue, hand it to vchan_cookie_complete() so the client callback runs
from the channel BH, and start the next issued descriptor if there is
one. Reset the retry budget at the same point, as the done tasklet does
after a successful retry, so the next descriptor does not inherit an
exhausted retry count.

During removal, kill the producer tasklets before the channel BH so they
cannot enqueue callback work after it has been drained.

Signed-off-by: Allen Pais <allen.lkml at gmail.com>
---
 drivers/dma/sf-pdma/sf-pdma.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/dma/sf-pdma/sf-pdma.c b/drivers/dma/sf-pdma/sf-pdma.c
index ca60fb46ade0..992f9f103702 100644
--- a/drivers/dma/sf-pdma/sf-pdma.c
+++ b/drivers/dma/sf-pdma/sf-pdma.c
@@ -326,9 +326,21 @@ static void sf_pdma_errbh_tasklet(struct tasklet_struct *t)
 
 	spin_lock_irqsave(&chan->lock, flags);
 	if (chan->retries <= 0) {
-		/* fail to recover */
+		/* fail to recover: abort this descriptor and move on */
+		chan->retries = MAX_RETRY;
+		chan->xfer_err = false;
 		spin_unlock_irqrestore(&chan->lock, flags);
-		dmaengine_desc_get_callback_invoke(desc->async_tx, NULL);
+
+		spin_lock_irqsave(&chan->vchan.lock, flags);
+		list_del(&desc->vdesc.node);
+		desc->vdesc.tx_result.result = DMA_TRANS_ABORTED;
+		desc->vdesc.tx_result.residue = desc->xfer_size;
+		vchan_cookie_complete(&desc->vdesc);
+
+		chan->desc = sf_pdma_get_first_pending_desc(chan);
+		if (chan->desc)
+			sf_pdma_xfer_desc(chan);
+		spin_unlock_irqrestore(&chan->vchan.lock, flags);
 	} else {
 		/* retry */
 		chan->retries--;
@@ -601,9 +613,9 @@ static void sf_pdma_remove(struct platform_device *pdev)
 		devm_free_irq(&pdev->dev, ch->txirq, ch);
 		devm_free_irq(&pdev->dev, ch->errirq, ch);
 		list_del(&ch->vchan.chan.device_node);
-		dmaengine_kill_bh(&ch->vchan.chan);
 		tasklet_kill(&ch->done_tasklet);
 		tasklet_kill(&ch->err_tasklet);
+		dmaengine_kill_bh(&ch->vchan.chan);
 	}
 
 	if (pdev->dev.of_node)
-- 
2.43.0