[RFC PATCH v2 14/45] arm64: entry: Introduce entry specific exception masking helpers
Jinjie Ruan
ruanjinjie at huawei.com
Tue Aug 11 01:46:38 PDT 2026
在 2026/8/11 16:37, Vladimir Murzin 写道:
> On 8/11/26 09:13, Jinjie Ruan wrote:
>>
>> 在 2026/8/3 20:21, Vladimir Murzin 写道:
>>> On 7/28/26 10:18, Jinjie Ruan wrote:
>>>> 在 2026/7/28 0:34, Vladimir Murzin 写道:
>>>>> From: Ada Couprie Diaz <ada.coupriediaz at arm.com>
>>>>>
>>>>> The entry code handles interrupt masking differently from the rest of
>>>>> the kernel. Exception handlers enter and exit with all exceptions
>>>>> masked, but they must temporarily unmask the appropriate set of
>>>>> exceptions so that the rest of the handler executes with the expected
>>>>> exception state.
>>>>>
>>>>> For EL0 handlers, this means dropping to masking context appropriate
>>>>> for the work to be performed. For EL1 handlers, this means restoring
>>>>> the masking context of the interrupted task. In both cases, all
>>>>> exceptions must be masked again before returning from the exception
>>>>> handler.
>>>>>
>>>>> The rest of the kernel typically follows the opposite pattern: it
>>>>> raises the masking context to protect a critical section and later
>>>>> restores the previous context.
>>>>>
>>>>> Given these different usage patterns, introduce a dedicated set of
>>>>> exception masking helpers for the entry code. Keeping these helpers
>>>>> separate from the generic interrupt masking APIs makes the intended
>>>>> usage explicit and helps avoid mixing the two masking models.
>>>>>
>>>>> Signed-off-by: Ada Couprie Diaz <ada.coupriediaz at arm.com>
>>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin at arm.com>
>>>>> ---
>>>>> arch/arm64/include/asm/interrupts/entry.h | 113 ++++++++++++++++++++++
>>>>> 1 file changed, 113 insertions(+)
>>>>> create mode 100644 arch/arm64/include/asm/interrupts/entry.h
>>>>>
>>>>> diff --git a/arch/arm64/include/asm/interrupts/entry.h b/arch/arm64/include/asm/interrupts/entry.h
>>>>> new file mode 100644
>>>>> index 000000000000..d66eb5d633f0
>>>>> --- /dev/null
>>>>> +++ b/arch/arm64/include/asm/interrupts/entry.h
>>>>> @@ -0,0 +1,113 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>>> +/*
>>>>> + * Copyright (C) 2025 Arm Ltd.
>>>>> + */
>>>>> +#ifndef __ASM_INTERRUPTS_ENTRY_H
>>>>> +#define __ASM_INTERRUPTS_ENTRY_H
>>>>> +
>>>>> +#include <asm/arch_gicv3.h>
>>>>> +#include <asm/bug.h>
>>>>> +#include <asm/cpufeature.h>
>>>>> +#include <asm/interrupts/common_flags.h>
>>>>> +
>>>>> +
>>>>> +static __always_inline
>>>>> +arm64_exc_hwstate_t __arm64_switch_exc_hwstate_to(arm64_exc_hwstate_t prev,
>>>>> + arm64_exc_hwstate_t next)
>>>>> +{
>>>>> + bool irqs_disabled = arch_irqs_disabled_flags(next.flags);
>>>>> + bool force;
>>>>> +
>>>>> + arm64_debug_exc_hwstate(prev);
>>>>> +
>>>>> + if (prev.flags == next.flags)
>>>>> + return next;
>>>>> +
>>>>> + if (!irqs_disabled)
>>>>> + trace_hardirqs_on();
>>>>> +
>>>>> + force = system_uses_irq_prio_masking() && prev.pmr != next.pmr;
>>>>> +
>>>>> + __arm64_update_exc_hwstate(next, force);
>>>>> +
>>>>> + if (irqs_disabled)
>>>>> + trace_hardirqs_off();
>>>>> +
>>>>> + return next;
>>>>> +}
>>>>> +
>>>>> +static __always_inline
>>>>> +arm64_exc_hwstate_t arm64_inherit_exc_context(struct pt_regs *regs)
>>>>> +{
>>>>> + arm64_exc_hwstate_t prev = arm64_exc_hwstate_of_context(CRITICAL_CONTEXT);
>>>>> + arm64_exc_hwstate_t next = arm64_inherit_exc_hwstate(regs);
>>>>> +
>>>>> + return __arm64_switch_exc_hwstate_to(prev, next);
>>>>> +}
>>>>> +
>>>>> +static __always_inline
>>>>> +arm64_exc_hwstate_t arm64_drop_exc_context(arm64_exc_hwstate_t prev, arm64_exc_context_t context)
>>>>> +{
>>>>> + arm64_exc_hwstate_t next = arm64_exc_hwstate_of_context(context);
>>>>> +
>>>>> + if (IS_ENABLED(CONFIG_DEBUG_IRQFLAGS)) {
>>>>> + bool pnmi = system_uses_irq_prio_masking();
>>>>> +
>>>>> + WARN_ON_ONCE(context > ERROR_CONTEXT &&
>>>>> + prev.daif == DAIF_ERRCTX);
>>>>> +
>>>>> + WARN_ON_ONCE(context > NONMI_CONTEXT &&
>>>>> + prev.daif == DAIF_PROCCTX_NOIRQ);
>>>> For daif, we can directly compare next and prev because, as the context
>>>> drops, the value of daif decreases.
>>>>
>>>> This is also the opposite of the meanings of "drop" and "lift" in the
>>>> function names, which is easy to understand.
>>>>
>>>> WARN_ON_ONCE(prev.daif < next.daif);
>>> That's a very good point! I was too focused on checking the hardware
>>> state against the logical exception context, so I missed that we could
>>> perform the checks using the hardware state alone.
>>>
>>> What do you think about:
>>>
>>> if (IS_ENABLED(CONFIG_DEBUG_IRQFLAGS)) {
>>> WARN_ON_ONCE(prev.daif < next.daif);
>>>
>>> if (prev.daif == next.daif) {
>>> /*
>>> * GIC_PRIO_IRQON is larger that GIC_PRIO_IRQOFF so larger PMR value is weaker
>>> */
>>> WARN_ON_ONCE(system_uses_irq_prio_masking() && prev.pmr > next.pmr);
>>> WARN_ON_ONCE(system_uses_nmi() && prev.allint < next.allint);
>
> Hi Jinjie,
>
>> Hi Vladimir,
>>
>> I have come up with what I think is a more comprehensible debugging method.
>>
>> Could we define a function that is the reverse of
>> arm64_exc_hwstate_of_context(), such that we can derive the previous
>> context from the previous hardware state? That would allow us to
>> directly compare below enum types, making the logic significantly easier
>> to follow. Any thoughts?
>>
>
> I used to have such reverse mapping in my internal version and feedback I
> got off-list is to avoid that at all cost. Since I needed it only for debug
> I open-coded checks in relevant functions (and that the reason I was focused
> on logical exception context rather than using hardware state alone).
I believe it would be cleaner to define and use this only when
CONFIG_DEBUG_IRQFLAGS is enabled; this would not introduce any overhead
in production kernels, correct?
>
> Cheers
> Vladimir
>
>> This way, we don't even need to distinguish between pseudo-NMI and FEAT_NMI.
>>
>> +typedef enum arm64_exc_context {
>> + PROCESS_CONTEXT,
>> + NOIRQ_CONTEXT,
>> + NONMI_CONTEXT,
>> + ERROR_CONTEXT,
>> + CRITICAL_CONTEXT,
>> +} arm64_exc_context_t;
>>
>> Best regards,
>> Jinjie
>>
>>> }
>>> }
>>>
>>>>> +
>>>>> + WARN_ON_ONCE(context > NOIRQ_CONTEXT &&
>>>>> + pnmi && prev.pmr == GIC_PRIO_IRQOFF);
>>>>> +
>>>>> + WARN_ON_ONCE(context > PROCESS_CONTEXT &&
>>>>> + ((pnmi && prev.daif == DAIF_PROCCTX && prev.pmr == GIC_PRIO_IRQON) ||
>>>>> + (!pnmi && prev.daif == DAIF_PROCCTX)));
>>>>> + }
>>>>> +
>>>>> + return __arm64_switch_exc_hwstate_to(prev, next);
>>>>> +}
>>>>> +
>>>>> +static __always_inline
>>>>> +arm64_exc_hwstate_t arm64_lift_exc_context(arm64_exc_hwstate_t prev, arm64_exc_context_t context)
>>>>> +{
>>>>> + arm64_exc_hwstate_t next = arm64_exc_hwstate_of_context(context);
>>>>> +
>>>>> + if (IS_ENABLED(CONFIG_DEBUG_IRQFLAGS)) {
>>>>> + bool pnmi = system_uses_irq_prio_masking();
>>>>> +
>>>>> + WARN_ON_ONCE(context < CRITICAL_CONTEXT &&
>>>>> + prev.daif == DAIF_MASK);
>>>>> +
>>>>> + WARN_ON_ONCE(context < ERROR_CONTEXT &&
>>>>> + prev.daif == DAIF_ERRCTX);
>>>> WARN_ON_ONCE(prev.daif > next.daif);
>>>>
>>> ... and
>>>
>>> if (IS_ENABLED(CONFIG_DEBUG_IRQFLAGS)) {
>>> WARN_ON_ONCE(prev.daif > next.daif);
>>>
>>> if (prev.daif == next.daif) {
>>> /*
>>> * GIC_PRIO_IRQON is larger that GIC_PRIO_IRQOFF so smaller PMR value is stronger
>>> */
>>> WARN_ON_ONCE(system_uses_irq_prio_masking() && prev.pmr < next.pmr);
>>> WARN_ON_ONCE(system_uses_nmi() && prev.allint > next.allint);
>>> }
>>> }
>>>
>>>
>>> Cheers
>>> Vladimir
>>>
>>>>> +
>>>>> + WARN_ON_ONCE(context < NONMI_CONTEXT &&
>>>>> + pnmi && prev.daif == DAIF_PROCCTX_NOIRQ);
>>>>> +
>>>>> + WARN_ON_ONCE(context < NOIRQ_CONTEXT &&
>>>>> + ((pnmi && prev.pmr == GIC_PRIO_IRQOFF) ||
>>>>> + (!pnmi && prev.daif == DAIF_PROCCTX_NOIRQ)));
>>>>> + }
>>>>> +
>>>>> + return __arm64_switch_exc_hwstate_to(prev, next);
>>>>> +}
>>>>> +
>>>>> +
>>>>> +static __always_inline
>>>>> +arm64_exc_hwstate_t arm64_unmask_exc_context(arm64_exc_context_t context)
>>>>> +{
>>>>> + arm64_exc_hwstate_t prev = arm64_exc_hwstate_of_context(CRITICAL_CONTEXT);
>>>>> +
>>>>> + return arm64_drop_exc_context(prev, context);
>>>>> +}
>>>>> +
>>>>> +static __always_inline
>>>>> +arm64_exc_hwstate_t arm64_mask_exc_context(arm64_exc_hwstate_t prev)
>>>>> +{
>>>>> + return arm64_lift_exc_context(prev, CRITICAL_CONTEXT);
>>>>> +}
>>>>> +
>>>>> +#endif /* __ASM_INTERRUPTS_ENTRY_H */
>>>
>
>
More information about the linux-arm-kernel
mailing list