[PATCH v3 00/21] KVM: arm64: Rewrite page-table code and fault handling

[Gavin Shan]

Hi Will,

On 9/3/20 10:16 PM, Will Deacon wrote:
> On Thu, Sep 03, 2020 at 09:48:18PM +1000, Gavin Shan wrote:
>> On 9/3/20 9:13 PM, Gavin Shan wrote:
>>> On 9/3/20 5:34 PM, Gavin Shan wrote:
>>>> On 8/25/20 7:39 PM, Will Deacon wrote:
>>>>> Hello folks,
>>>>>
>>>>> This is version three of the KVM page-table rework that I previously posted
>>>>> here:
>>>>>
>>>>>     v1: https://lore.kernel.org/r/20200730153406.25136-1-will@kernel.org
>>>>>     v2: https://lore.kernel.org/r/20200818132818.16065-1-will@kernel.org
>>>>>
>>>>> Changes since v2 include:
>>>>>
>>>>>     * Rebased onto -rc2, which includes the conflicting OOM blocking fixes
>>>>>     * Dropped the patch trying to "fix" the memcache in kvm_phys_addr_ioremap()
>>>>>
>>>>
>>>> It's really nice work, making the code unified/simplified greatly.
>>>> However, it seems it doesn't work well with HugeTLBfs. Please refer
>>>> to the following test result and see if you have quick idea, or I
>>>> can debug it a bit :)
> 
> Nice testing matrix, and thanks for reporting the problem!
> 
>>>> Machine         Host                     Guest              Result
>>>> ===============================================================
>>>> ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Passed
>>>>                PAGE_SIZE: 64KB                    64KB     passed
>>>>                THP:       disabled
>>>>                HugeTLB:   disabled
>>>> ---------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Passed
>>>>                PAGE_SIZE: 64KB                    64KB     passed
>>>>                THP:       enabled
>>>>                HugeTLB:   disabled
>>>> ----------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Fail[1]
>>>>                PAGE_SIZE: 64KB                    64KB     Fail[1]
>>>>                THP:       disabled
>>>>                HugeTLB:   enabled
>>>> ---------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   39           PAGE_SIZE:  4KB     Passed
>>>>                PAGE_SIZE: 4KB                     64KB     Passed
>>>>                THP:       disabled
>>>>                HugeTLB:   disabled
>>>> ---------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   39           PAGE_SIZE:  4KB     Passed
>>>>                PAGE_SIZE: 4KB                     64KB     Passed
>>>>                THP:       enabled
>>>>                HugeTLB:   disabled
>>>> --------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   39           PAGE_SIZE: 4KB     Fail[2]
>>>>                PAGE_SIZE: 4KB                    64KB     Fail[2]
>>>>                THP:       disabled
>>>>                HugeTLB:   enabled
>>>>
>>>
>>> I debugged the code and found the issue is caused by the following
>>> patch.
>>>
>>> [PATCH v3 06/21] KVM: arm64: Add support for stage-2 map()/unmap() in generic page-table
> 
> (I think this is just a symptom of the page-table being out of whack)
> 
>> Sorry that the guest could hang sometimes with above changes. I have no idea what
>> has been happening before I'm going to debug for more.. I'm pasting the used command
>> and output from guest.
> 
> Can you try the diff below, please? I think we can end up sticking down a
> huge-page-sized mapping at an unaligned address, which causes us both to
> overmap and also to fail to use the huge granule for a block mapping.
> 

Since the the following changes have been folded to v4, I reran the test cases
on v4 and everything works fine.

Thanks,
Gavin

> --->8
> 
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index f28e03dcb897..3bff942e5f33 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -737,11 +737,11 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
>          bool exec_fault;
>          bool device = false;
>          unsigned long mmu_seq;
> -       gfn_t gfn = fault_ipa >> PAGE_SHIFT;
>          struct kvm *kvm = vcpu->kvm;
>          struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;
>          struct vm_area_struct *vma;
>          short vma_shift;
> +       gfn_t gfn;
>          kvm_pfn_t pfn;
>          bool logging_active = memslot_is_logging(memslot);
>          unsigned long vma_pagesize;
> @@ -780,7 +780,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
>          }
>   
>          if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE)
> -               gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT;
> +               fault_ipa &= huge_page_mask(hstate_vma(vma));
> +
> +       gfn = fault_ipa >> PAGE_SHIFT;
>          mmap_read_unlock(current->mm);
>   
>          if (fault_status != FSC_PERM) {
>