armv7 R_ARM_PREL31 relocation out of range error when loading module
William Zhang
william.zhang at broadcom.com
Fri Sep 19 09:43:40 PDT 2025
On Fri, Sep 19, 2025 at 12:23 AM Ard Biesheuvel <ardb at kernel.org> wrote:
>
> On Fri, 19 Sept 2025 at 08:58, William Zhang <william.zhang at broadcom.com> wrote:
> >
> >
> >
> > On Thu, Sep 18, 2025 at 5:25 PM William Zhang <william.zhang at broadcom.com> wrote:
> >>
> >> On Thu, Sep 18, 2025 at 3:41 PM Ard Biesheuvel <ardb at kernel.org> wrote:
> >> >
> >> > On Thu, 18 Sept 2025 at 20:30, William Zhang <william.zhang at broadcom.com> wrote:
> >> > >
> >> > > On Wed, Sep 17, 2025 at 11:55 PM Ard Biesheuvel <ardb at kernel.org> wrote:
> >> > > >
> >> > > > On Thu, 18 Sept 2025 at 04:11, William Zhang <william.zhang at broadcom.com> wrote:
> >> > > > >
> >> > > > > On Wed, Sep 17, 2025 at 3:11 AM Ard Biesheuvel <ardb at kernel.org> wrote:
> >> > > > > >
> >> > > > > > On Wed, 17 Sept 2025 at 04:55, William Zhang <william.zhang at broadcom.com> wrote:
> >> > > > > > >
> >> > > > > > > Hi,
> >> > > > > > >
> >> > > > > > > We recently ran into the driver loading error resulting from
> >> > > > > > > apply_relocate function when it handle the R_ARM_PREL31 case in
> >> > > > > > > arch/arm/kernel/module.c:
> >> > > > > > > section 29 reloc 0 sym '': relocation 42 out of range (0xc2ab9be8 ->
> >> > > > > > > 0x7fad5998)
> >> > > > > > >
> >> > > > > > > After some dig, it seems this issue is caused by this change:
> >> > > > > > > ac3b43283923 ("module: replace module_layout with module_memory")
> >> > > > > > >
> >> > > > > > > I can see the necessity of this change but it seems it can cause this
> >> > > > > > > kind of error under certain configurations. In our particular case, we
> >> > > > > > > use the 2G/2G split and our armv7 board has 1GB ddr memory. This means
> >> > > > > > > the kernel lower memory is from 0x8000-0000 to 0xc000-0000 and vmalloc
> >> > > > > > > range is from 0xc000-0000 and above.
> >> > > > > > >
> >> > > > > > > Before this module layout change, the module is loaded through two
> >> > > > > > > layout sections: core and init. The driver in the test is fairly big
> >> > > > > > > and core module layout(text, data, rodata, sections) can not fit into
> >> > > > > > > the 16MB MODULE memory so both text, data and rodata fallback to
> >> > > > > > > vmalloc range. But with this change, it will text, data and rodata one
> >> > > > > > > by one. So it ends up text in the MODULES memory but rodata is
> >> > > > > > > allocated in VMALLOC and the distance is more than 1GB hence we
> >> > > > > > > believe it causes the above error when R_ARM_PREL31 relocation can not
> >> > > > > > > support more than 1GB offset.
> >> > > > > > >
> >> > > > > > > Since the default split is 1G/3G model, the distance between vmalloc
> >> > > > > > > and module memory is always below 1G so we won't see this issue. But
> >> > > > > > > it seems to us the problem is real when use 2G/2G split.
> >> > > > > > >
> >> > > > > > > It appears the sections containing R_ARM_PREL31 in our driver are
> >> > > > > > > mostly in .ARM.exidx section. Some search shows this is used by ARM
> >> > > > > > > C++ exception handling but our driver does not use C++. Is there a way
> >> > > > > > > to disable such section for R_ARM_PREL31? If not, is there any
> >> > > > > > > workaround or fix to handle this run-time R_ARM_PREL31 relocation
> >> > > > > > > issue? The same apply_relocate function can handle R_ARM_JUMP24 out of
> >> > > > > > > range using PLTS. Can this apply to R_ARM_PREL31?
> >> > > > > > >
> >> > > > > >
> >> > > > > > Hi William,
> >> > > > > >
> >> > > > > > Two options come to mind:
> >> > > > > >
> >> > > > > > 1) use the frame pointer for unwind information instead of the C++
> >> > > > > > exception metadata, i.e., enable CONFIG_UNWINDER_FRAME_POINTER
> >> > > > > >
> >> > > > > > 2) ensure that ARM.exidx sections are allocated from the same bucket
> >> > > > > > as the .text sections they annotate, e.g., something like the below
> >> > > > > >
> >> > > > > > --- a/arch/arm/kernel/module-plts.c
> >> > > > > > +++ b/arch/arm/kernel/module-plts.c
> >> > > > > > @@ -225,6 +225,11 @@
> >> > > > > > mod->arch.init.plt = s;
> >> > > > > > else if (s->sh_type == SHT_SYMTAB)
> >> > > > > > syms = (Elf32_Sym *)s->sh_addr;
> >> > > > > > +#if defined(CONFIG_ARM_UNWIND) && !defined(CONFIG_VMSPLIT_3G)
> >> > > > > > + else if (s->sh_type == ELF_SECTION_UNWIND)
> >> > > > > > + s->sh_flags |= SHF_EXECINSTR;
> >> > > > > > +#endif
> >> > > > > > +
> >> > > > > > }
> >> > > > > >
> >> > > > > > if (!mod->arch.core.plt || !mod->arch.init.plt) {
> >> > > > >
> >> > > > > Thank you Ard for your quick response! We tried the option 1 and
> >> > > > > also tried with gcc -fno-unwind-tables option just for the driver,
> >> > > > > both do not generate any PRREL31 relocation and the driver can load
> >> > > > > successfully.
> >> > > > > But we have some tool that needs to use the unwind tables so option 2
> >> > > > > will be the right direction for us.
> >> > > > >
> >> > > > > Applied your patch however it still reports the same error but from
> >> > > > > different section as we confirmed all the ARM.exidx sections are
> >> > > > > allocated from .text section.
> >> > > > > mod_mem_type 0 execmem_type 0 base 0x7f3e8000 size 0x6eb000
> >> > > > > mod_mem_type 1 execmem_type 4 base 0x7fad4000 size 0x9e000
> >> > > > > mod_mem_type 2 execmem_type 4 base 0xc21a4000 size 0x3dd000
> >> > > > > mod_mem_type 4 execmem_type 0 base 0x7f1ae000 size 0x1000
> >> > > > > mod_mem_type 5 execmem_type 4 base 0x7fb73000 size 0x10c000
> >> > > > > section 43 reloc 11360 sym '': relocation 42 out of range
> >> > > > > (0x7facca74 -> 0xc2580734)
> >> > > > >
> >> > > > > This loc address 0x7facca74 is from the MOD_DATA memory based on mod
> >> > > > > mem allocation log above.
> >> > > > > I also print some extra info about the dstsect with
> >> > > > > pr_err("dstsec name %s add %#lx size %u rel 0x%px rel offset %u\n",
> >> > > > > strtab + dstsec->sh_name, dstsec->sh_addr, dstsec->sh_size,
> >> > > > > rel, rel->r_offset)
> >> > > > > and got this print out
> >> > > > > dstsec name FUNCTION__.2 add 0x7fabd3c0 size 86912 rel 0xc33f931c
> >> > > > > rel offset 63156
> >> > > > >
> >> > > > > FUNCTION__.2 is in the rodata section from the symbol table.
> >> > > > >
> >> > > > > So looks like there is reference to ARM.exidx from the data/rodata
> >> > > > > section and that also need to be taken care of?
> >> > > >
> >> > > > It looks like the ARM.exidx, which is now in .text (the first entry in
> >> > > > your table), contains a reference to code that is emitted in .rodata
> >> > > > rather than .text, and .rodata (mod_mem_type 2) is emitted in the
> >> > > > vmalloc space, and is too far from .text.
> >> > > >
> >> > > This loc address 0x7facca74 is actually from the MOD_DATA mod_mem_type 1.
> >> > > Seems to me that it is some .data reference .rodata but how is this possible?
> >> > >
> >> >
> >> > How so? That MOD_DATA region starts at 0x7fad4000, no?
> >> >
> >> My bad. Wrong math.. That address is in the MOD_TEXT so it totally
> >> makes sense now.
> >>
> >> > > > If FUNCTION__.2 is really a function, why is it in .rodata and not in .text?
> >> > > >
> >> > > Not that symbol is not a function. I believe it is probably some
> >> > > string or fixed data
> >> > > that the compiler generated for a function so it is in .rodata. There
> >> > > are many of
> >> > > these instances from the objdump:
> >> > >
> >> > > 00001058 l O .rodata 0000000b __FUNCTION__.2
> >> > > 00001118 l O .rodata 0000001b __FUNCTION__.2
> >> > > ...
> >> > >
> >> >
> >> > So one thing you might consider is to find a way to emit these into
> >> > .text as well, or any other section that has the 'x' bit set.
> >> Thanks we will look more into this and see if it is possible.
> >
> >
> > It turns out that section .ARM.extab also contains R_ARM_PREL31 relocation but this section has a regular type of PROGBITS.
> > Below is the dump of the headers of the .ko module with .ARM.extab keyword:
> >
> > [27] .ARM.extab.t[...] PROGBITS 00000000 ac0e55 000000 00 A 0 0 1
> > [31] .ARM.extab.i[...] PROGBITS 00000000 ac17da 000000 00 A 0 0 1
> > [34] .ARM.extab.e[...] PROGBITS 00000000 ac17e4 000000 00 A 0 0 1
> > [41] .ARM.extab PROGBITS 00000000 ac1a60 000054 00 A 0 0 4
> > 000000fc 00001a2a R_ARM_PREL31 00000000 .ARM.extab
> > 00000a5c 00001a2a R_ARM_PREL31 00000000 .ARM.extab
> > 00003c24 00001a2a R_ARM_PREL31 00000000 .ARM.extab
> > 0000479c 00001a2a R_ARM_PREL31 00000000 .ARM.extab
> > 0000f6b4 00001a2a R_ARM_PREL31 00000000 .ARM.extab
> > 00011ba4 00001a2a R_ARM_PREL31 00000000 .ARM.extab
> > 000135cc 00001a2a R_ARM_PREL31 00000000 .ARM.extab
> > 17: 00000000 0 SECTION LOCAL DEFAULT 27 .ARM.extab.text.[...]
> > 20: 00000000 0 SECTION LOCAL DEFAULT 31 .ARM.extab.init.text
> > 22: 00000000 0 SECTION LOCAL DEFAULT 34 .ARM.extab.exit.text
> > 26: 00000000 0 SECTION LOCAL DEFAULT 41 .ARM.extab
> >
> > So with an extra check on this specific section, the driver can load successfully. Can this be a real fix for this issue?
>
> Yes, that seems reasonable. Apparently, both ARM.exidx and ARM.extab
> need to live within -/+ 1 GiB of the .text sections they annotate, and
> this is a reasonable and unintrusive way to achieve that.
>
> > But if the section type is PROGBITS, why do we need to set the SHF_EXECINSTR flag? Shouldn't it be treated as instruction by default?
> >
>
> PROGBITS just means the section contents in the file need to be copied
> into the image in memory in order to initialize it, not that the
> contents are code.
>
> E.g., NOBITS means that the associated memory region needs to be
> zeroed. Other types exist that are not part of the static memory image
> at all (i.e., relocations, ELF notes, debug data, symbol tables, etc)
>
Got you. Since both .ARM.extab and .ARM.extidx are used for unwind info, it
seems reasonable to mark.ARM.extab as ELF_SECTION_UNWIND type too
then the change will be much cleaner. But this will be the linker to solve...
> >
> > --- a/arch/arm/kernel/module-plts.c
> > +++ b/arch/arm/kernel/module-plts.c
> > @@ -225,6 +225,12 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
> > mod->arch.init.plt = s;
> > else if (s->sh_type == SHT_SYMTAB)
> > syms = (Elf32_Sym *)s->sh_addr;
> > +#if defined(CONFIG_ARM_UNWIND) && !defined(CONFIG_VMSPLIT_3G)
> > + else if (s->sh_type == ELF_SECTION_UNWIND ||
> > + (strncmp(".ARM.extab", secstrings + s->sh_name, 10) == 0)) {
> > + s->sh_flags |= SHF_EXECINSTR;
> > + }
> > +#endif
> > }
> >
> > if (!mod->arch.core.plt || !mod->arch.init.plt) {
> >
>
> You can send this out to the list as a patch. Please align the added
> line vertically with the 's' after 'else if(' and add
>
Will do.
> Co-developed-by: Ard Biesheuvel <ardb at kernel.org>
> Signed-off-by: Ard Biesheuvel <ardb at kernel.org>
More information about the linux-arm-kernel
mailing list