Is it possible to run a linux on multiple clusters if CCI is missing

[Li Chen]

Hi Arnd,
 ---- On Tue, 12 Jul 2022 23:43:06 +0800  Arnd Bergmann <arnd at arndb.de> wrote --- 
 > On Tue, Jul 12, 2022 at 5:02 PM Li Chen <me at linux.beauty> wrote:
 > >
 > > Hi all,
 > >
 > > Say that if one SoC has 4 clusters, and each cluster has an A78. Every A78 has its own SCU to maintain cache coherence,
 > > but this SoC doesn't have CCI, so there is no hardware to maintain cache coherence between the 4 clusters.
 > >
 > > My question: is it possible to run a single Linux kernel on 4 clusters?
 > > If so, how to do it and how can the cluster cache coherence be maintained?
 > 
 > Hi Li,
 > 
 > Please note that sending an open question like this to the linux-arm-kernel list
 > probably doesn't get you a reply, since almost nobody reads all the emails sent
 > to the list, it's more common to just Cc the list to have an archive of the
 > discussion but have other recipients.  A better place for this might be the
 > #armlinux IRC channel on irc.libera.chat.
 > 
 > I did get your email in my inbox since I recently replied to another thread
 > of yours.

Got it. I will come to #armlinux after fixing the IRC connection issue with my company network.
But I prefer mailing list over IRC in that most IRC channels don't have archives available, so they
are searchable on google.

 
 > Regarding your question, I'm pretty sure that you cannot run Linux across
 > multiple clusters without a CCI, as the kernel among other things on behavior
 > documented in Documentation/memory-barriers.txt that is not guaranteed
 > otherwise.
 
Good point, but I don't know how CCI deals with the memory barrier, can you share more about it?

Apart from memory barriers and coherence, can TLB invalidation not also work properly among four clusters without CCI?

 > The only way I can think of for using that kind of system would be to run
 > a separate kernel on each cluster, and assigning each device to one of the
 > instances, and use explicit cache management for a communication
 > channel between them.

Sorry for my three noob questions:
1. How to assign each devices to one of the instances? Can NIC do it?
2. When you say "explicit cache management", do you mean flush/invalidate cache in kernel manually with flush_cache*
    and fush_tlb*?
3. What kind of communication channel? share a region of memory as communication and monitor it with PMU?

Regards,
Li