[PATCH v3 00/21] KVM: arm64: Rewrite page-table code and fault handling

[Will Deacon]

On Thu, Sep 03, 2020 at 09:48:18PM +1000, Gavin Shan wrote:
> On 9/3/20 9:13 PM, Gavin Shan wrote:
> > On 9/3/20 5:34 PM, Gavin Shan wrote:
> > > On 8/25/20 7:39 PM, Will Deacon wrote:
> > > > Hello folks,
> > > > 
> > > > This is version three of the KVM page-table rework that I previously posted
> > > > here:
> > > > 
> > > >    v1: https://lore.kernel.org/r/20200730153406.25136-1-will@kernel.org
> > > >    v2: https://lore.kernel.org/r/20200818132818.16065-1-will@kernel.org
> > > > 
> > > > Changes since v2 include:
> > > > 
> > > >    * Rebased onto -rc2, which includes the conflicting OOM blocking fixes
> > > >    * Dropped the patch trying to "fix" the memcache in kvm_phys_addr_ioremap()
> > > > 
> > > 
> > > It's really nice work, making the code unified/simplified greatly.
> > > However, it seems it doesn't work well with HugeTLBfs. Please refer
> > > to the following test result and see if you have quick idea, or I
> > > can debug it a bit :)

Nice testing matrix, and thanks for reporting the problem!

> > > Machine         Host                     Guest              Result
> > > ===============================================================
> > > ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Passed
> > >               PAGE_SIZE: 64KB                    64KB     passed
> > >               THP:       disabled
> > >               HugeTLB:   disabled
> > > ---------------------------------------------------------------
> > > ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Passed
> > >               PAGE_SIZE: 64KB                    64KB     passed
> > >               THP:       enabled
> > >               HugeTLB:   disabled
> > > ----------------------------------------------------------------
> > > ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Fail[1]
> > >               PAGE_SIZE: 64KB                    64KB     Fail[1]
> > >               THP:       disabled
> > >               HugeTLB:   enabled
> > > ---------------------------------------------------------------
> > > ThunderX2    VA_BITS:   39           PAGE_SIZE:  4KB     Passed
> > >               PAGE_SIZE: 4KB                     64KB     Passed
> > >               THP:       disabled
> > >               HugeTLB:   disabled
> > > ---------------------------------------------------------------
> > > ThunderX2    VA_BITS:   39           PAGE_SIZE:  4KB     Passed
> > >               PAGE_SIZE: 4KB                     64KB     Passed
> > >               THP:       enabled
> > >               HugeTLB:   disabled
> > > --------------------------------------------------------------
> > > ThunderX2    VA_BITS:   39           PAGE_SIZE: 4KB     Fail[2]
> > >               PAGE_SIZE: 4KB                    64KB     Fail[2]
> > >               THP:       disabled
> > >               HugeTLB:   enabled
> > > 
> > 
> > I debugged the code and found the issue is caused by the following
> > patch.
> > 
> > [PATCH v3 06/21] KVM: arm64: Add support for stage-2 map()/unmap() in generic page-table

(I think this is just a symptom of the page-table being out of whack)

> Sorry that the guest could hang sometimes with above changes. I have no idea what
> has been happening before I'm going to debug for more.. I'm pasting the used command
> and output from guest.

Can you try the diff below, please? I think we can end up sticking down a
huge-page-sized mapping at an unaligned address, which causes us both to
overmap and also to fail to use the huge granule for a block mapping.

Cheers,

Will

--->8

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index f28e03dcb897..3bff942e5f33 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -737,11 +737,11 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
        bool exec_fault;
        bool device = false;
        unsigned long mmu_seq;
-       gfn_t gfn = fault_ipa >> PAGE_SHIFT;
        struct kvm *kvm = vcpu->kvm;
        struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;
        struct vm_area_struct *vma;
        short vma_shift;
+       gfn_t gfn;
        kvm_pfn_t pfn;
        bool logging_active = memslot_is_logging(memslot);
        unsigned long vma_pagesize;
@@ -780,7 +780,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
        }
 
        if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE)
-               gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT;
+               fault_ipa &= huge_page_mask(hstate_vma(vma));
+
+       gfn = fault_ipa >> PAGE_SHIFT;
        mmap_read_unlock(current->mm);
 
        if (fault_status != FSC_PERM) {