[LEDE-DEV] [RFC 0/1] fix segfault in make kernel_menuconfig

[Sergey Ryazanov]

Hello,

the folling patch fixes segfault in mconf when run "make
kernel_menuconfig" on my system (Gentoo with fresh portages).

This issue already was reported in FS#366 [1].

I tracked down this segfault to stranger linked mconf binary. It linked
with both ncurses and ncursesw libraries:

linux-4.4.61 $ ldd scripts/kconfig/mconf
        linux-gate.so.1
        libncurses.so.6 => /lib/libncurses.so.6
        libncursesw.so.6 => /lib/libncursesw.so.6
        libc.so.6 => /lib/libc.so.6
        /lib/ld-linux.so.2

When analyzing a core file with gdb, it looks like that mconf was
compiled with the headers of ncursesw, but it calls functions from
ncurses:

linux-4.4.61 $ gdb scripts/kconfig/mconf core
Core was generated by `scripts/kconfig/mconf Kconfig'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0xb770d2fd in waddch_literal (ch=<optimized out>, win=0x96a1f20)
385         line->text[x++] = ch;
(gdb) bt
#0  waddch_literal (ch, win)
#1  waddch_nosync (ch, win)
#2  waddch (win, ch)
#3  draw_box (win, y, x, height, width, box, border)
#4  dialog_menu (title, prompt, selected, s_scroll)
#5  conf (menu, active_menu)
#6  main (ac, av)
(gdb) print sizeof(*draw_box::win)
$1 = 108
(gdb) print sizeof(*waddch_literal::win)
$2 = 76

mconf itself was linked with the following command:

linux-4.4.61 $ grep -n '[-]lncurses' scripts/kconfig/.mconf.cmd
1:cmd_scripts/kconfig/mconf := gcc  -o scripts/kconfig/mconf
scripts/kconfig/mconf.o scripts/kconfig/zconf.tab.o
scripts/kconfig/lxdialog/checklist.o scripts/kconfig/lxdialog/util.o
scripts/kconfig/lxdialog/inputbox.o scripts/kconfig/lxdialog/textbox.o
scripts/kconfig/lxdialog/yesno.o scripts/kconfig/lxdialog/menubox.o
-L/tmp/lede/staging_dir/host/lib -lncurses -lncursesw

First linking instruction ('-lncurses') is comes from the
include/kernel-build.mk script in the HOST_LOADLIBES parameter, while
the second one ('-lncursesw') generated by the kernel itself and stored
in the HOSTLOADLIBES_mconf variable. Then both variables are joined to
get the final linking command.

Kernel uses script/kconfig/lxdialog/check-lxdialog.sh script to detect
compilation and linking options for ncurses[w]. And this script do the
everything possible to detects ncurses[w] header files and library on
the host system. This script have one more feature: it prefers the
ncursesw library (and widechars) to regular ncurses library.

Each kernel version in use (e.g 3.18, 4.4 and 4.9) uses this script and
behaves in a similar way.

So when we force to link with ncurses on the host system, which also
contains installed ncursesw library, we create prerequisites for
segfaults and other issues.

This forced linking was added in the commit 86c966a8ae9c4e (build: fix
regression on running make kernel_menuconfig). After I reverted this
commit the make kernel_menuconfig command became fully functional
without any issues.

Since mentioned commit should fix some issue (according to its subject)
and since I tested this reverting patch only on the few systems I could
miss something and segfault issue should have a more complex solution.
So I send this reverting patch as RFC.

1. https://bugs.lede-project.org/index.php?do=details&task_id=366

Sergey Ryazanov (1):
  Revert "build: fix regression on running make kernel_menuconfig"

 include/kernel-build.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
2.10.2