答复: [PATCH v11 0/6] Add RAS virtualization support for armv8 SEA and SEI
gengdongjiu
gengdongjiu at huawei.com
Fri Aug 18 10:17:38 PDT 2017
Loop Laszlo.
> -----邮件原件-----
> 发件人: gengdongjiu
> 发送时间: 2017年8月18日 22:24
> 收件人: mst at redhat.com; imammedo at redhat.com; Zhaoshenglong <zhaoshenglong at huawei.com>; peter.maydell at linaro.org;
> pbonzini at redhat.com; qemu-devel at nongnu.org; qemu-arm at nongnu.org; kvm at vger.kernel.org; edk2-devel at lists.01.org;
> christoffer.dall at linaro.org; marc.zyngier at arm.com; will.deacon at arm.com; james.morse at arm.com; tbaicar at codeaurora.org;
> ard.biesheuvel at linaro.org; mingo at kernel.org; bp at suse.de; Shiju Jose <shiju.jose at huawei.com>; zjzhang at codeaurora.org;
> linux-arm-kernel at lists.infradead.org; kvmarm at lists.cs.columbia.edu; linux-kernel at vger.kernel.org; linux-acpi at vger.kernel.org;
> devel at acpica.org; John Garry <john.garry at huawei.com>; Jonathan Cameron <jonathan.cameron at huawei.com>; Shameerali Kolothum
> Thodi <shameerali.kolothum.thodi at huawei.com>; huangdaode <huangdaode at hisilicon.com>; Wangzhou (B) <wangzhou1 at hisilicon.com>
> 抄送: Huangshaoyu <huangshaoyu at huawei.com>; Wuquanming <wuquanming at huawei.com>; Linuxarm <linuxarm at huawei.com>;
> gengdongjiu <gengdongjiu at huawei.com>; Zhengqiang (turing) <zhengqiang10 at huawei.com>
> 主题: [PATCH v11 0/6] Add RAS virtualization support for armv8 SEA and SEI
>
> In the armv8 platform, the mainly processor hardware error notification type are synchronous external abort(SEA) and SError Interrupt (SEI),
> For the ARMv8 SEA/SEI, KVM or host kernel will deliver SIGBUS or use other interface to notify user space. After user space gets the
> notification, it will record the CPER to simulate GHES for guest OS and inject the a
> exception(SEA/SEI) to KVM.
>
> This series patch has two parts, one part handles synchronous external
> abort(SEA) exception and SError Interrupt (SEI) exception; another part is generating APEI table when guest OS boot up, and dynamically
> record CPER for the guest OS about the generic hardware errors. Currently the userspace only handles the memory section hardware
> errors. Before Qemu record the CPER, it needs to check the ACK value written by the guest OS to avoid read-write race condition. In the
> simulated APEI/GHESV2/CPER table, the max number of error soure is 11, which is classified by notification type, now only enable the
> SEA/SEI notification type error source to avoid OS boot warning.
>
>
> About the whole solution we ever discuessed it in here before:
> https://patchwork.kernel.org/patch/9633105/
>
> Below is the APEI/GHESV2/CPER table layout, the max number of error soure is 11:
>
> etc/acpi/tables etc/hardware_errors
> ==================== ==========================================
> + +--------------------------+ +------------------+
> | | HEST | | address | +--------------+
> | +--------------------------+ | registers | | Error Status |
> | | GHES0 | | +----------------+ | Data Block 0 |
> | +--------------------------+ +--------->| |status_address0
> | +--------------------------+ +--------->| ||------------->|
> | +--------------------------+ +--------->| |+------------+
> | | ................. | | | +----------------+ | | CPER |
> | | error_status_address-----+-+ +------->| |status_address1 |----------+ | | CPER |
> | | ................. | | | +----------------+ | | | .... |
> | | read_ack_register--------+-+ | | ............. | | | | CPER |
> | | read_ack_preserve | | | +------------------+ | | +------------+
> | | read_ack_write | | | +----->| |status_address10|--------+ | | Error Status |
> + +--------------------------+ | | | | +----------------+ | | | Data Block 1 |
> | | GHES1 | +-+-+----->| | ack_value0 | | +-->| +------------+
> + +--------------------------+ | | | +----------------+ | | | CPER |
> | | ................. | | | +--->| | ack_value1 | | | | CPER |
> | | error_status_address-----+---+ | | | +----------------+ | | | .... |
> | | ................. | | | | | ............. | | | | CPER |
> | | read_ack_register--------+-----+-+ | +----------------+ | +-+------------+
> | | read_ack_preserve | | +->| | ack_value10 | | | |.......... |
> | | read_ack_write | | | | +----------------+ | | +------------+
> + +--------------------------| | | | | Error Status |
> | | ............... | | | | | Data Block 10|
> + +--------------------------+ | | +---->| +------------+
> | | GHES10 | | | | | CPER |
> + +--------------------------+ | | | | CPER |
> | | ................. | | | | | .... |
> | | error_status_address-----+-----+ | | | CPER |
> | | ................. | | +-+------------+
> | | read_ack_register--------+---------+
> | | read_ack_preserve |
> | | read_ack_write |
> + +--------------------------+
>
>
> ----------------------------------------------------------------------------------------------
> How to test guest OS do SEA/SEI recovery:
>
> 1. In the guest OS, trigger a SEA or SEI.
> 2. Then you will see below error log that printed by the memory failure 3. Memory failure will do the recovery for the error.
>
> Such as the below shown kernel log:
> [ 21.101216] Synchronous External Abort: synchronous external abort (0x96000010) at 0xffffff8008064018
> [ 21.104969] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 8
> [ 21.106918] {1}[Hardware Error]: event severity: recoverable
> [ 21.109027] {1}[Hardware Error]: Error 0, type: recoverable
> [ 21.110362] {1}[Hardware Error]: section_type: memory error
> [ 21.111705] {1}[Hardware Error]: physical_address: 0x000000007a200000
> [ 21.113255] {1}[Hardware Error]: error_type: 3, multi-bit ECC
> [ 21.118528] Internal error: : 96000010 [#1] SMP
> [ 21.119587] Modules linked in:
> [ 21.120307] CPU: 0 PID: 509 Comm: devmem Not tainted 4.12.0-rc4ajb-00990-g954379b-dirty #67
> [ 21.122307] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
> [ 21.123915] task: ffffffc03da32900 task.stack: ffffffc03dbbc000
> [ 21.125302] PC is at __do_user_fault+0x58/0x110
> [ 21.126370] LR is at __do_user_fault+0x54/0x110
> [ 21.127433] pc : [<ffffff8008097528>] lr : [<ffffff8008097524>] pstate: 80000145
> [ 21.129164] sp : ffffffc03dbbfd20
> [ 21.129940] x29: ffffffc03dbbfd20 x28: ffffffc03da32900
> [ 21.131204] x27: 0000000000000000 x26: 0000007f7edc5001
> [ 21.132439] x25: ffffff8008648438 x24: ffffffc03dbbfec0
> [ 21.133689] x23: 0000000000030001 x22: 0000007f7edc5001
> [ 21.134934] x21: 0000000000000007 x20: 0000000092000021
> [ 21.136195] x19: ffffffc03da32900 x18: 0000007fdd4c18f0
> [ 21.137439] x17: 0000007f7ecb9ebc x16: 0000000000412058
>
> ------------------------------------------------------------------------------------------------
> how to test guest OS APTI/GHES:
> 1. In the guest OS, use this command to dump the APEI table:
> "iasl -p ./HEST -d /sys/firmware/acpi/tables/HEST"
> 2. And find the address for the generic error status block
> according to the notification type
> 3. then find the CPER record through the generic error status block.
>
> For example(notification type is SEA):
>
> (1) root at genericarmv8:~# iasl -p ./HEST -d /sys/firmware/acpi/tables/HEST
> (2) root at genericarmv8:~# cat HEST.dsl
> /*
> * Intel ACPI Component Architecture
> * AML/ASL+ Disassembler version 20170728 (64-bit version)
> * Copyright (c) 2000 - 2017 Intel Corporation
> *
> * Disassembly of /sys/firmware/acpi/tables/HEST, Mon Sep 5 07:59:17 2016
> *
> * ACPI Data Table [HEST]
> *
> * Format: [HexOffset DecimalOffset ByteLength] FieldName : FieldValue
> */
>
> ..................................................................................
> [308h 0776 2] Subtable Type : 000A [Generic Hardware Error Source V2]
> [30Ah 0778 2] Source Id : 0008
> [30Ch 0780 2] Related Source Id : FFFF
> [30Eh 0782 1] Reserved : 00
> [30Fh 0783 1] Enabled : 01
> [310h 0784 4] Records To Preallocate : 00000001
> [314h 0788 4] Max Sections Per Record : 00000001
> [318h 0792 4] Max Raw Data Length : 00001000
>
> [31Ch 0796 12] Error Status Address : [Generic Address Structure]
> [31Ch 0796 1] Space ID : 00 [SystemMemory]
> [31Dh 0797 1] Bit Width : 40
> [31Eh 0798 1] Bit Offset : 00
> [31Fh 0799 1] Encoded Access Width : 04 [QWord Access:64]
> [320h 0800 8] Address : 00000000785D0040
>
> [328h 0808 28] Notify : [Hardware Error Notification Structure]
> [328h 0808 1] Notify Type : 08 [SEA]
> [329h 0809 1] Notify Length : 1C
> [32Ah 0810 2] Configuration Write Enable : 0000
> [32Ch 0812 4] PollInterval : 00000000
> [330h 0816 4] Vector : 00000000
> [334h 0820 4] Polling Threshold Value : 00000000
> [338h 0824 4] Polling Threshold Window : 00000000
> [33Ch 0828 4] Error Threshold Value : 00000000
> [340h 0832 4] Error Threshold Window : 00000000
>
> [344h 0836 4] Error Status Block Length : 00001000
> [348h 0840 12] Read Ack Register : [Generic Address Structure]
> [348h 0840 1] Space ID : 00 [SystemMemory]
> [349h 0841 1] Bit Width : 40
> [34Ah 0842 1] Bit Offset : 00
> [34Bh 0843 1] Encoded Access Width : 04 [QWord Access:64]
> [34Ch 0844 8] Address : 00000000785D0098
>
> [354h 0852 8] Read Ack Preserve : 00000000FFFFFFFE
> [35Ch 0860 8] Read Ack Write : 0000000000000001
>
> [364h 0868 2] Subtable Type : 000A [Generic Hardware Error Source V2]
> [366h 0870 2] Source Id : 0009
> [368h 0872 2] Related Source Id : FFFF
> [36Ah 0874 1] Reserved : 00
> [36Bh 0875 1] Enabled : 01
> [36Ch 0876 4] Records To Preallocate : 00000001
> [370h 0880 4] Max Sections Per Record : 00000001
> [374h 0884 4] Max Raw Data Length : 00001000
>
> [378h 0888 12] Error Status Address : [Generic Address Structure]
> [378h 0888 1] Space ID : 00 [SystemMemory]
> [379h 0889 1] Bit Width : 40
> [37Ah 0890 1] Bit Offset : 00
> [37Bh 0891 1] Encoded Access Width : 04 [QWord Access:64]
> [37Ch 0892 8] Address : 00000000785D0048
>
> [384h 0900 28] Notify : [Hardware Error Notification Structure]
> [384h 0900 1] Notify Type : 09 [SEI]
> [385h 0901 1] Notify Length : 1C
> [386h 0902 2] Configuration Write Enable : 0000
> [388h 0904 4] PollInterval : 00000000
> [38Ch 0908 4] Vector : 00000000
> [390h 0912 4] Polling Threshold Value : 00000000
> [394h 0916 4] Polling Threshold Window : 00000000
> [398h 0920 4] Error Threshold Value : 00000000
> [39Ch 0924 4] Error Threshold Window : 00000000
>
> [3A0h 0928 4] Error Status Block Length : 00001000
> [3A4h 0932 12] Read Ack Register : [Generic Address Structure]
> [3A4h 0932 1] Space ID : 00 [SystemMemory]
> [3A5h 0933 1] Bit Width : 40
> [3A6h 0934 1] Bit Offset : 00
> [3A7h 0935 1] Encoded Access Width : 04 [QWord Access:64]
> [3A8h 0936 8] Address : 00000000785D00A0
>
> [3B0h 0944 8] Read Ack Preserve : 00000000FFFFFFFE
> [3B8h 0952 8] Read Ack Write : 0000000000000001
> .....................................................................................
> (3) according to above table, the address that contains the physical address of a block
> of memory that holds the error status data for SEA notification error source is 0x00000000785D0040
> (4) the address for SEA notification error source is 0x785d8108
> (qemu) xp /1 0x00000000785D0040
> 00000000785d0040: 0x785d80b0
>
> (5) check the content of generic error status block and generic error data entry
> (qemu) xp /100x 0x785d80b0
> 00000000785d80b0: 0x00000001 0x00000000 0x00000000 0x00000098
> 00000000785d80c0: 0x00000000 0xa5bc1114 0x4ede6f64 0x833e63b8
> 00000000785d80d0: 0xb1837ced 0x00000000 0x00000300 0x00000050
> 00000000785d80e0: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d80f0: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8100: 0x00000000 0x00000000 0x00000000 0x00004002
> 00000000785d8110: 0x00000000 0x00000000 0x00000000 0x00001111
> 00000000785d8120: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8130: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8140: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8150: 0x00000000 0x00000003 0x00000000 0x00000000
> 00000000785d8160: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8170: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8180: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8190: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d81a0: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d81b0: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d81c0: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d81d0: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d81e0: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d81f0: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8200: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8210: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8220: 0x00000000 0x00000000 0x00000000 0x00000000
> 00000000785d8230: 0x00000000 0x00000000 0x00000000 0x00000000
> (6) check the OSPM's ACK value(for example SEA)
> /* Before OSPM acknowledges the error, check the ACK value */
> (qemu) xp /1 0x00000000785D0098
> 00000000785d00f0: 0x00000000
>
> /* After OSPM acknowledges the error, check the ACK value */
> (qemu) xp /1 0x00000000785D0098
> 00000000785d00f0: 0x00000001
>
> Dongjiu Geng (6):
> ACPI: add APEI/HEST/CPER structures and macros
> ACPI: Add APEI GHES Table Generation support
> ACPI: build and enable APEI GHES in the Makefile and configuration
> target-arm: kvm64: detect guest RAS EXTENSION feature
> target-arm: kvm64: handle SIGBUS signal for synchronous External Abort
> target-arm: kvm64: Handle SError interrupt from the guest OS
>
> default-configs/arm-softmmu.mak | 1 +
> hw/acpi/Makefile.objs | 1 +
> hw/acpi/aml-build.c | 2 +
> hw/acpi/hest_ghes.c | 345 ++++++++++++++++++++++++++++++++++++++++
> hw/arm/virt-acpi-build.c | 6 +
> include/hw/acpi/acpi-defs.h | 193 ++++++++++++++++++++++
> include/hw/acpi/aml-build.h | 1 +
> include/hw/acpi/hest_ghes.h | 47 ++++++
> include/sysemu/kvm.h | 2 +-
> linux-headers/asm-arm64/kvm.h | 5 +
> linux-headers/linux/kvm.h | 2 +
> target/arm/cpu.h | 3 +
> target/arm/internals.h | 14 ++
> target/arm/kvm.c | 34 ++++
> target/arm/kvm64.c | 186 ++++++++++++++++++++++
> target/arm/kvm_arm.h | 1 +
> 16 files changed, 842 insertions(+), 1 deletion(-) create mode 100644 hw/acpi/hest_ghes.c create mode 100644
> include/hw/acpi/hest_ghes.h
>
> --
> 1.8.3.1
More information about the linux-arm-kernel
mailing list