HIGHMEM is broken when working in SMP V6 mode

[Russell King - ARM Linux]

On Sun, Jan 23, 2011 at 04:38:01PM +0200, saeed bishara wrote:
> Hi,
> I've port 2.6.35 to SMP system that runs in V6 mode, this system
> doesn't support TLB operations broadcasting by hw, so it uses IPI
> messages for that.  when enabling DEBUG_LOCKDEP, I got the following
> error message while booting the system from NFS:

You've bypassed this check:

		if (is_smp() && tlb_ops_need_broadcast()) {
                        /*
                         * kmap_high needs to occasionally flush TLB entries,
                         * however, if the TLB entries need to be broadcast
                         * we may deadlock:
                         *  kmap_high(irqs off)->flush_all_zero_pkmaps->
                         *  flush_tlb_kernel_range->smp_call_function_many
                         *   (must not be called with irqs off)
                         */
                        reason = "without hardware TLB ops broadcasting";
                }

so you lose.  There's reasons why such checks are put in.  We can not
support SMP and highmem on systems which do not have TLB broadcasting.
That's not because the code doesn't support it, it's because there are
deadlocks which will occur.

The fact is that it is unsafe to send IPIs with IRQs disabled, which
means you can't IPI a TLB operation and wait for it to complete with IRQs
disabled.

The kmap code sets up and tears down mappings with IRQs disabled because
of the need to work with the DMA API.  This means that it when it wants
to do a TLB operation (to flush out old mappings) its calling context is
incompatible with what's required to broadcast that operation in
software.

There is no solution to this.  SMP without hardware TLB broadcast is
incompatible with the requirements of highmem.