[PATCH v3] platform: generic/andes: flush caches before setting PMA regions

[Randolph]

From: Randolph Lin <randolph at andestech.com>

Non-cacheable accesses do not remove the lines a region left in the
caches while it was cacheable. When andes_sbi_set_pma() or
andes_pma_setup_regions() makes such a region non-cacheable, memory
and the caches can disagree: data stored by the CPU can stay in a
cache line that a non-coherent DMA master never sees, and a dirty
line can be written back later over data stored by the DMA master.

Write back and invalidate the L1 data cache of the calling hart and
the last level cache before programming the PMA entries. The PMA CSRs
are per hart, so a caller that sets a region on every hart also
flushes every L1 data cache.

Also execute fence.i on every platform. PMA changes are rare, so the
cost of invalidating the instruction cache does not matter.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Randolph Lin <randolph at andestech.com>
---
Changes in v3:
- Execute fence.i on every platform. PMA changes are rare, so the cost
  of invalidating the instruction cache does not matter.
- Rename andes_pma_flush_dcache() to andes_pma_flush_caches() and change
  the subject from "flush data caches" to "flush caches", because it now
  also invalidates the instruction cache.

v2: <link to v2 on the list>

 platform/generic/andes/andes_pma.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/platform/generic/andes/andes_pma.c b/platform/generic/andes/andes_pma.c
index 28d33a2f..be64e2d3 100644
--- a/platform/generic/andes/andes_pma.c
+++ b/platform/generic/andes/andes_pma.c
@@ -15,6 +15,7 @@
 #include <sbi/riscv_io.h>
 #include <sbi/sbi_console.h>
 #include <sbi/sbi_error.h>
+#include <sbi_utils/cache/fdt_cmo_helper.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 
 static inline bool not_napot(unsigned long addr, unsigned long size)
@@ -111,6 +112,24 @@ static bool has_pma_region_overlap(unsigned long start, unsigned long size)
 	return false;
 }
 
+/*
+ * Non-cacheable accesses do not remove the lines a region left in the
+ * caches while it was cacheable, so flush the caches before a region
+ * changes its PMA.
+ */
+static void andes_pma_flush_caches(void)
+{
+	if ((csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
+	    (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK))
+		csr_write(CSR_MCCTLCOMMAND, MCCTLCOMMAND_L1D_WBINVAL_ALL);
+
+	RISCV_FENCE_I;
+
+#ifdef CONFIG_FDT_CACHE
+	fdt_cmo_llc_flush_all();
+#endif
+}
+
 static unsigned long andes_pma_setup(const struct andes_pma_region *pma_region,
 				     unsigned int entry_id)
 {
@@ -257,6 +276,8 @@ int andes_pma_setup_regions(void *fdt,
 	if (!andes_sbi_probe_pma())
 		return SBI_ENOTSUPP;
 
+	andes_pma_flush_caches();
+
 	/* Configure the PMA regions */
 	dt_populate_cnt = 0;
 	for (i = 0; i < pma_regions_count; i++) {
@@ -326,6 +347,8 @@ int andes_sbi_set_pma(unsigned long pa, unsigned long size, u8 flags)
 		return SBI_ERR_FAILED;
 	}
 
+	andes_pma_flush_caches();
+
 	rc = andes_pma_setup(&region, entry_id);
 	if (rc == SBI_EINVAL) {
 		sbi_printf("ERROR %s(): Failed to set PMAADDR%d\n",

base-commit: 332798211e7232da60158b0545b23c61b5338516
-- 
2.34.1