[PATCH v2 9/9] arm64: KVM: vgic: deal with GIC sub-page alignment

[Peter Maydell]

On 25 June 2014 20:34, Joel Schopp <joel.schopp at amd.com> wrote:
> It doesn't work for me.  Maybe I'm doing something wrong, but I can't see
> what.  I am unique in that I'm running a gic-400 (gicv2m) on aarch64
> hardware with 64k pages.  I'm also unique in that my hardware maps each 4K
> gic entry to a 64K page (aliasing each 4k of gic 16 times in a 64K page, ie
> the gic virtual ic is at 0xe1140000 and 0xe1141000 and 0xe1142000, etc).
>
> This is inline with appendix F of the server base system architecture.  This
> is inconvenient when the size is 0x2000 (8K).  As a result all the offsets
> in the device tree entries are to the last 4K in the page so that an 8K read
> will read the last 4k from one page and the first 4k from the next and
> actually get 8k of the gic.
>
>
>         gic: interrupt-controller at e1101000 {
>                 compatible = "arm,gic-400";
>                 #interrupt-cells = <3>;
>                 #address-cells = <0>;
>                 interrupt-controller;
>                 msi-controller;
>                 reg = <0x0 0xe1110000 0 0x1000>, /* gic dist */
>                       <0x0 0xe112f000 0 0x2000>, /* gic cpu */
>                       <0x0 0xe114f000 0 0x2000>, /* gic virtual ic*/
>                       <0x0 0xe116f000 0 0x2000>, /* gic virtual cpu*/
>                       <0x0 0xe1180000 0 0x1000>; /* gic msi */

Right, this is the oddball case we don't yet support for 64K pages
(though as you say it is a permitted configuration per the SBSA).

>                 interrupts = <1 8 0xf04>;
>         };
>
>
> My concern here is that if userspace is going to look at 8k starting at the
> beginning of the page, guest offset 0 in your terminology, (say 0xe1140000)
> instead of starting at the last 4k of the page, offset 0xf000 (say
> 0xe114f000) it is going to get the second 4k wrong by reading 0xe1141000
> instead of 0xe1150000.

Userspace doesn't actually look at anything in the GICC. It just asks
the kernel to put the guest GICC (ie the mapping of the host GICV)
at a particular base address which happens to be a multiple of 64K.
In this case if the host kernel is using 64K pages then the KVM
kernel code ought to say "sorry, can't do that" when we tell it the
base address. (That is, it's impossible to give the guest a VM
where the GICC it sees is at a 64K boundary on your hardware
and host kernel config, and hopefully we report that in a not totally
opaque fashion.)

If you hack QEMU's memory map for the virt board so instead of
    [VIRT_GIC_CPU] = { 0x8010000, 0x10000 },
we have
    [VIRT_GIC_CPU] = { 0x801f000, 0x2000 },

does it work? If QEMU supported this VGIC_GRP_ADDR_OFFSET
query then all it would do would be to change that offset and size.
It would be good to know if there are other problems beyond that...

(Conveniently, Linux guests won't currently try to look at the second
4K page of their GICC...)

thanks
-- PMM