[RFC PATCH v2 14/45] arm64: entry: Introduce entry specific exception masking helpers
Vladimir Murzin
vladimir.murzin at arm.com
Tue Aug 11 01:37:42 PDT 2026
On 8/11/26 09:13, Jinjie Ruan wrote:
>
> 在 2026/8/3 20:21, Vladimir Murzin 写道:
>> On 7/28/26 10:18, Jinjie Ruan wrote:
>>> 在 2026/7/28 0:34, Vladimir Murzin 写道:
>>>> From: Ada Couprie Diaz <ada.coupriediaz at arm.com>
>>>>
>>>> The entry code handles interrupt masking differently from the rest of
>>>> the kernel. Exception handlers enter and exit with all exceptions
>>>> masked, but they must temporarily unmask the appropriate set of
>>>> exceptions so that the rest of the handler executes with the expected
>>>> exception state.
>>>>
>>>> For EL0 handlers, this means dropping to masking context appropriate
>>>> for the work to be performed. For EL1 handlers, this means restoring
>>>> the masking context of the interrupted task. In both cases, all
>>>> exceptions must be masked again before returning from the exception
>>>> handler.
>>>>
>>>> The rest of the kernel typically follows the opposite pattern: it
>>>> raises the masking context to protect a critical section and later
>>>> restores the previous context.
>>>>
>>>> Given these different usage patterns, introduce a dedicated set of
>>>> exception masking helpers for the entry code. Keeping these helpers
>>>> separate from the generic interrupt masking APIs makes the intended
>>>> usage explicit and helps avoid mixing the two masking models.
>>>>
>>>> Signed-off-by: Ada Couprie Diaz <ada.coupriediaz at arm.com>
>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin at arm.com>
>>>> ---
>>>> arch/arm64/include/asm/interrupts/entry.h | 113 ++++++++++++++++++++++
>>>> 1 file changed, 113 insertions(+)
>>>> create mode 100644 arch/arm64/include/asm/interrupts/entry.h
>>>>
>>>> diff --git a/arch/arm64/include/asm/interrupts/entry.h b/arch/arm64/include/asm/interrupts/entry.h
>>>> new file mode 100644
>>>> index 000000000000..d66eb5d633f0
>>>> --- /dev/null
>>>> +++ b/arch/arm64/include/asm/interrupts/entry.h
>>>> @@ -0,0 +1,113 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +/*
>>>> + * Copyright (C) 2025 Arm Ltd.
>>>> + */
>>>> +#ifndef __ASM_INTERRUPTS_ENTRY_H
>>>> +#define __ASM_INTERRUPTS_ENTRY_H
>>>> +
>>>> +#include <asm/arch_gicv3.h>
>>>> +#include <asm/bug.h>
>>>> +#include <asm/cpufeature.h>
>>>> +#include <asm/interrupts/common_flags.h>
>>>> +
>>>> +
>>>> +static __always_inline
>>>> +arm64_exc_hwstate_t __arm64_switch_exc_hwstate_to(arm64_exc_hwstate_t prev,
>>>> + arm64_exc_hwstate_t next)
>>>> +{
>>>> + bool irqs_disabled = arch_irqs_disabled_flags(next.flags);
>>>> + bool force;
>>>> +
>>>> + arm64_debug_exc_hwstate(prev);
>>>> +
>>>> + if (prev.flags == next.flags)
>>>> + return next;
>>>> +
>>>> + if (!irqs_disabled)
>>>> + trace_hardirqs_on();
>>>> +
>>>> + force = system_uses_irq_prio_masking() && prev.pmr != next.pmr;
>>>> +
>>>> + __arm64_update_exc_hwstate(next, force);
>>>> +
>>>> + if (irqs_disabled)
>>>> + trace_hardirqs_off();
>>>> +
>>>> + return next;
>>>> +}
>>>> +
>>>> +static __always_inline
>>>> +arm64_exc_hwstate_t arm64_inherit_exc_context(struct pt_regs *regs)
>>>> +{
>>>> + arm64_exc_hwstate_t prev = arm64_exc_hwstate_of_context(CRITICAL_CONTEXT);
>>>> + arm64_exc_hwstate_t next = arm64_inherit_exc_hwstate(regs);
>>>> +
>>>> + return __arm64_switch_exc_hwstate_to(prev, next);
>>>> +}
>>>> +
>>>> +static __always_inline
>>>> +arm64_exc_hwstate_t arm64_drop_exc_context(arm64_exc_hwstate_t prev, arm64_exc_context_t context)
>>>> +{
>>>> + arm64_exc_hwstate_t next = arm64_exc_hwstate_of_context(context);
>>>> +
>>>> + if (IS_ENABLED(CONFIG_DEBUG_IRQFLAGS)) {
>>>> + bool pnmi = system_uses_irq_prio_masking();
>>>> +
>>>> + WARN_ON_ONCE(context > ERROR_CONTEXT &&
>>>> + prev.daif == DAIF_ERRCTX);
>>>> +
>>>> + WARN_ON_ONCE(context > NONMI_CONTEXT &&
>>>> + prev.daif == DAIF_PROCCTX_NOIRQ);
>>> For daif, we can directly compare next and prev because, as the context
>>> drops, the value of daif decreases.
>>>
>>> This is also the opposite of the meanings of "drop" and "lift" in the
>>> function names, which is easy to understand.
>>>
>>> WARN_ON_ONCE(prev.daif < next.daif);
>> That's a very good point! I was too focused on checking the hardware
>> state against the logical exception context, so I missed that we could
>> perform the checks using the hardware state alone.
>>
>> What do you think about:
>>
>> if (IS_ENABLED(CONFIG_DEBUG_IRQFLAGS)) {
>> WARN_ON_ONCE(prev.daif < next.daif);
>>
>> if (prev.daif == next.daif) {
>> /*
>> * GIC_PRIO_IRQON is larger that GIC_PRIO_IRQOFF so larger PMR value is weaker
>> */
>> WARN_ON_ONCE(system_uses_irq_prio_masking() && prev.pmr > next.pmr);
>> WARN_ON_ONCE(system_uses_nmi() && prev.allint < next.allint);
Hi Jinjie,
> Hi Vladimir,
>
> I have come up with what I think is a more comprehensible debugging method.
>
> Could we define a function that is the reverse of
> arm64_exc_hwstate_of_context(), such that we can derive the previous
> context from the previous hardware state? That would allow us to
> directly compare below enum types, making the logic significantly easier
> to follow. Any thoughts?
>
I used to have such reverse mapping in my internal version and feedback I
got off-list is to avoid that at all cost. Since I needed it only for debug
I open-coded checks in relevant functions (and that the reason I was focused
on logical exception context rather than using hardware state alone).
Cheers
Vladimir
> This way, we don't even need to distinguish between pseudo-NMI and FEAT_NMI.
>
> +typedef enum arm64_exc_context {
> + PROCESS_CONTEXT,
> + NOIRQ_CONTEXT,
> + NONMI_CONTEXT,
> + ERROR_CONTEXT,
> + CRITICAL_CONTEXT,
> +} arm64_exc_context_t;
>
> Best regards,
> Jinjie
>
>> }
>> }
>>
>>>> +
>>>> + WARN_ON_ONCE(context > NOIRQ_CONTEXT &&
>>>> + pnmi && prev.pmr == GIC_PRIO_IRQOFF);
>>>> +
>>>> + WARN_ON_ONCE(context > PROCESS_CONTEXT &&
>>>> + ((pnmi && prev.daif == DAIF_PROCCTX && prev.pmr == GIC_PRIO_IRQON) ||
>>>> + (!pnmi && prev.daif == DAIF_PROCCTX)));
>>>> + }
>>>> +
>>>> + return __arm64_switch_exc_hwstate_to(prev, next);
>>>> +}
>>>> +
>>>> +static __always_inline
>>>> +arm64_exc_hwstate_t arm64_lift_exc_context(arm64_exc_hwstate_t prev, arm64_exc_context_t context)
>>>> +{
>>>> + arm64_exc_hwstate_t next = arm64_exc_hwstate_of_context(context);
>>>> +
>>>> + if (IS_ENABLED(CONFIG_DEBUG_IRQFLAGS)) {
>>>> + bool pnmi = system_uses_irq_prio_masking();
>>>> +
>>>> + WARN_ON_ONCE(context < CRITICAL_CONTEXT &&
>>>> + prev.daif == DAIF_MASK);
>>>> +
>>>> + WARN_ON_ONCE(context < ERROR_CONTEXT &&
>>>> + prev.daif == DAIF_ERRCTX);
>>> WARN_ON_ONCE(prev.daif > next.daif);
>>>
>> ... and
>>
>> if (IS_ENABLED(CONFIG_DEBUG_IRQFLAGS)) {
>> WARN_ON_ONCE(prev.daif > next.daif);
>>
>> if (prev.daif == next.daif) {
>> /*
>> * GIC_PRIO_IRQON is larger that GIC_PRIO_IRQOFF so smaller PMR value is stronger
>> */
>> WARN_ON_ONCE(system_uses_irq_prio_masking() && prev.pmr < next.pmr);
>> WARN_ON_ONCE(system_uses_nmi() && prev.allint > next.allint);
>> }
>> }
>>
>>
>> Cheers
>> Vladimir
>>
>>>> +
>>>> + WARN_ON_ONCE(context < NONMI_CONTEXT &&
>>>> + pnmi && prev.daif == DAIF_PROCCTX_NOIRQ);
>>>> +
>>>> + WARN_ON_ONCE(context < NOIRQ_CONTEXT &&
>>>> + ((pnmi && prev.pmr == GIC_PRIO_IRQOFF) ||
>>>> + (!pnmi && prev.daif == DAIF_PROCCTX_NOIRQ)));
>>>> + }
>>>> +
>>>> + return __arm64_switch_exc_hwstate_to(prev, next);
>>>> +}
>>>> +
>>>> +
>>>> +static __always_inline
>>>> +arm64_exc_hwstate_t arm64_unmask_exc_context(arm64_exc_context_t context)
>>>> +{
>>>> + arm64_exc_hwstate_t prev = arm64_exc_hwstate_of_context(CRITICAL_CONTEXT);
>>>> +
>>>> + return arm64_drop_exc_context(prev, context);
>>>> +}
>>>> +
>>>> +static __always_inline
>>>> +arm64_exc_hwstate_t arm64_mask_exc_context(arm64_exc_hwstate_t prev)
>>>> +{
>>>> + return arm64_lift_exc_context(prev, CRITICAL_CONTEXT);
>>>> +}
>>>> +
>>>> +#endif /* __ASM_INTERRUPTS_ENTRY_H */
>>
More information about the linux-arm-kernel
mailing list