[OpenWrt-Devel] python ctypes.util.find_library cannot find libc

Alexandru Ardelean ardeleanalex at gmail.com
Fri Jan 20 02:24:21 EST 2017


Hey,

So, just to follow up, real quick.
There's another proposal here [that started in parallel and earlier than mine]:
https://patchwork.ozlabs.org/patch/713718/


On Thu, Jan 19, 2017 at 9:07 PM, Alexandru Ardelean
<ardeleanalex at gmail.com> wrote:
> Cool.
>
> Then, a fix I'm proposing to LEDE, and will also spin up a PR for OpenWrt:
> https://github.com/lede-project/source/pull/722/commits/6e0844540355c89e7f39504db374278d1c6cf05c
>
> Feel free to use it.
> It should apply on top of OpenWrt.
> Though, be aware ; it's not yet been discussed whether this method is
> completely sane.
>
> I'm hoping some sort of elegant solution will come out of that.
> In the worst case, you'll need to patch python-iptables [ add a patch
> in python-iptables' folder ] to use /usr/sbin/modprobe directly.
>
> Which may not be a bad idea either.
> I think it may come down to preferences.
>
> In any case, since it's not a issue with Python, I'll just keep an eye
> on that PR, and update OpenWrt if needed.
>
> Thanks
> Alex
>
>
>
> On Thu, Jan 19, 2017 at 8:20 PM, Andrew McConachie <andrew at depht.com> wrote:
>> Hi Alexandru,
>>
>> Yes, I'm sorry I didn't respond earlier. Having the correct binary location
>> in /proc/sys/kernel/modprobe resolves any problems with python-iptables.
>> It's really just a problem with the kernel not populating that file
>> correctly.
>>
>> Thanks,
>> Andrew
>>
>>
>> On 1/19/17 11:49, Alexandru Ardelean wrote:
>>>
>>> I'm pretty sure the driver was selected.
>>> I'll try to check a bit more in-depth.
>>>
>>> In the meantime, did you try the
>>>
>>>
>>> echo /usr/sbin/modprobe > /proc/sys/kernel/modprobe
>>>
>>>
>>> ?
>>>
>>>
>>> On Thu, Jan 19, 2017 at 4:07 PM, Andrew McConachie <andrew at depht.com>
>>> wrote:
>>>>
>>>> Hi Alexandru,
>>>>
>>>> You're missing some kernel hardware driver. I'm not sure if the default
>>>> compile options include kernel drivers for all HDDs, or for SATA/AHCI.
>>>> You
>>>> probably just need to include some extra kernel drivers in make
>>>> menuconfig.
>>>>
>>>> --Andrew
>>>>
>>>>
>>>> On 1/19/17 04:31, Alexandru Ardelean wrote:
>>>>>
>>>>> Hey,
>>>>>
>>>>> So, I've tried to run  an OpenWrt system, x86_64,
>>>>>
>>>>>
>>>>> https://github.com/openwrt/openwrt/commit/4e53a6e9c560b54361f9ed3639e8d12f9ab8939d
>>>>>
>>>>> It was hanging on boot.
>>>>> I've ran in Virtual Machine Manager with QEMU/KVM.
>>>>> HDD emulation is SATA/AHCI [ I checked it's SATA/AHCI ]
>>>>>
>>>>> It looks to me that it's hanging at  trying to mount root device
>>>>> /dev/mtdblock0
>>>>> Any thoughts ?
>>>>>
>>>>> [    1.213897] ata4: SATA link down (SStatus 0 SControl 300)
>>>>> [    1.214781] ata2: SATA link down (SStatus 0 SControl 300)
>>>>> [    1.215572] ata6: SATA link down (SStatus 0 SControl 300)
>>>>> [    1.216359] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
>>>>> [    1.217229] ata1.00: ATA-7: QEMU HARDDISK, 2.3.1, max UDMA/100
>>>>> [    1.217976] ata1.00: 31233 sectors, multi 16: LBA48 NCQ (depth 31/32)
>>>>> [    1.218787] ata1.00: applying bridge limits
>>>>> [    1.233284] ata1.00: configured for UDMA/100
>>>>> [    1.234233] ata5: SATA link down (SStatus 0 SControl 300)
>>>>> [    1.235915] ata3: SATA link down (SStatus 0 SControl 300)
>>>>> [    1.237371] scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK
>>>>>     1    PQ: 0 ANSI: 5
>>>>> [    1.239155] sd 0:0:0:0: [sda] 31233 512-byte logical blocks: (16.0
>>>>> MB/15.3 MiB)
>>>>> [    1.241487] sd 0:0:0:0: [sda] Write Protect is off
>>>>> [    1.242882] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
>>>>> enabled, doesn't support DPO or FUA
>>>>> [    1.245329]  sda: sda1 sda2
>>>>> [    1.245887] sda: p2 size 262144 extends beyond EOD, enabling native
>>>>> capacity
>>>>> [    1.247232]  sda: sda1 sda2
>>>>> [    1.247784] sda: p2 size 262144 extends beyond EOD, truncated
>>>>> [    1.248864] sd 0:0:0:0: [sda] Attached SCSI disk
>>>>> [    1.256721] block2mtd: erasesize must be a divisor of device size
>>>>> [    1.259958] rtc_cmos 00:00: setting system clock to 2017-01-19
>>>>> 09:27:33 UTC (1484818053)
>>>>> [    1.261388] Waiting for root device /dev/mtdblock0...
>>>>>
>>>>> On Wed, Jan 18, 2017 at 6:05 PM, Alexandru Ardelean
>>>>> <ardeleanalex at gmail.com> wrote:
>>>>>>
>>>>>> Hey,
>>>>>>
>>>>>> So, if you try on the system.
>>>>>>
>>>>>> echo /usr/sbin/modprobe > /proc/sys/kernel/modprobe
>>>>>>
>>>>>> Does it work ? I mean to just  import iptc ?
>>>>>> It worked for me, but I tried on a LEDE system (x86_64), which I'm
>>>>>> hoping may be similar to OpenWrt.
>>>>>>
>>>>>> For me, the part that interests me the most, is if this is a bug
>>>>>> within Python [ since I maintain it ], and it runs on both LEDE &
>>>>>> OpenWrt.
>>>>>>
>>>>>> Will try to spin up a OpenWrt [just cloned trunk from Github ].
>>>>>>
>>>>>> And I'll try to reproduce.
>>>>>> For reference ; Python is version 2.7.13
>>>>>> It's from here: https://github.com/openwrt/packages
>>>>>>
>>>>>> I could not find hash 5ba298c   in OpenWrt [
>>>>>> https://github.com/openwrt/openwrt ]  nor in packages [ link above ].
>>>>>>
>>>>>> Will come back with findings for OpenWrt.
>>>>>>
>>>>>> In the meantime, I will see about proposing a solution for updating
>>>>>> /proc/sys/kernel/modprobe   correctly for both LEDE & OpenWrt.
>>>>>>
>>>>>> Thanks
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 18, 2017 at 4:38 PM, Andrew McConachie <andrew at depht.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi 郭涛 and Alexandru,
>>>>>>>
>>>>>>> ldconfig depends on using eglibc to fulfill libc requirement.
>>>>>>>
>>>>>>> Symbol: PACKAGE_ldconfig [=n]
>>>>>>> Type  : tristate
>>>>>>> Prompt: ldconfig............................... Shared library path
>>>>>>> configuration
>>>>>>> Location:
>>>>>>> (3) -> Utilities
>>>>>>> Defined at tmp/.config-package.in:82365
>>>>>>> Depends on: !USE_MUSL [=y]
>>>>>>>
>>>>>>> If we make python depend on ldconfig, then are we saying python cannot
>>>>>>> be
>>>>>>> used with MUSL libc? I don't know what the default libc is for OpenWRT
>>>>>>> or
>>>>>>> whether one is considered experimental more than the other. But this
>>>>>>> is
>>>>>>> something to consider.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Andrew
>>>>>>>
>>>>>>>
>>>>>>> On 1/18/17 03:32, 郭涛 wrote:
>>>>>>>>
>>>>>>>> Hi Andrew & Alexandru
>>>>>>>>
>>>>>>>> Forget the patch in prev mail, use attached patch instead.
>>>>>>>>
>>>>>>>> To use ctypes.util.find_library, you need one of gcc, ldconfig or
>>>>>>>> objdump. I suggest you use ldconfig
>>>>>>>>
>>>>>>>> After install ldconfig,  run ldconfig first to update cache
>>>>>>>> then run ldconfig -p to show all of your libraries
>>>>>>>> in my case, it shows:
>>>>>>>>
>>>>>>>> 195 libs found in cache `/etc/ld.so.cache' (version 1.7.0)
>>>>>>>>            uhttpd_tls.so (libc0) => /usr/lib/uhttpd_tls.so
>>>>>>>>            rclibrary.so (libc0) => /usr/lib/rclibrary.so
>>>>>>>>            libz.so.1 (libc0) => /usr/lib/libz.so.1
>>>>>>>>            libz.so (libc0) => /usr/lib/libz.so
>>>>>>>>            libyaml-0.so.2 (libc0) => /usr/lib/libyaml-0.so.2
>>>>>>>>            ......
>>>>>>>>
>>>>>>>> All libraries are libc0, that's why ctypes.util.find_library does not
>>>>>>>> work on my platform
>>>>>>>>
>>>>>>>> You need to run 'uname -m' to get your matchine name and run
>>>>>>>> 'ldconfig
>>>>>>>> -p' to get library type.
>>>>>>>> Atter all, append  '$machine' : '$type'  to  mach_map list in
>>>>>>>> ctypes/util.py and try find_library('pthread')
>>>>>>>>
>>>>>>>>
>>>>>>>>>>> from ctypes.util import find_library
>>>>>>>>>>>
>>>>>>>>>>> find_library('pthread')
>>>>>>>>
>>>>>>>> 'libpthread.so.0'
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2017-01-17 22:22 GMT+08:00 Alexandru Ardelean
>>>>>>>> <ardeleanalex at gmail.com>:
>>>>>>>>>
>>>>>>>>> Will give it a try.
>>>>>>>>>
>>>>>>>>> On Mon, Jan 16, 2017 at 9:41 PM, Andrew McConachie
>>>>>>>>> <andrew at depht.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Alexandru and 郭涛,
>>>>>>>>>>
>>>>>>>>>> Attached is the Makefile I made for python-iptables. I can work
>>>>>>>>>> around
>>>>>>>>>> this
>>>>>>>>>> by hardwiring library locations in the source of python-iptables,
>>>>>>>>>> but
>>>>>>>>>> I'd
>>>>>>>>>> rather do it the correct way. To reproduce this build an OpenWrt
>>>>>>>>>> system
>>>>>>>>>> with
>>>>>>>>>> this Makefile and then just create a simple Python script with
>>>>>>>>>> 'import
>>>>>>>>>> iptc'.
>>>>>>>>>>
>>>>>>>>>> I am cloning OpenWrt from Github and running make menuconfig;make
>>>>>>>>>> to
>>>>>>>>>> build
>>>>>>>>>> everything. My Github version is about 6 days old with the last
>>>>>>>>>> commit
>>>>>>>>>> at
>>>>>>>>>> 5ba298c.
>>>>>>>>>>
>>>>>>>>>> I also found that /proc/sys/kernel/modprobe contains
>>>>>>>>>> /sbin/modprobe,
>>>>>>>>>> while
>>>>>>>>>> the modprobe binary is at /usr/sbin/modprobe. According to the
>>>>>>>>>> Debian
>>>>>>>>>> man
>>>>>>>>>> page on proc(5), /proc/sys/kernel/modprobe should point to the
>>>>>>>>>> modprobe
>>>>>>>>>> binary. Googling about seems also to suggest that this file should
>>>>>>>>>> contain
>>>>>>>>>> the location of the modprobe binary. So I would say this is also a
>>>>>>>>>> bug.
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>> --Andrew
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 1/16/17 07:23, Alexandru Ardelean wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hey Andrew & 郭涛
>>>>>>>>>>>
>>>>>>>>>>> Sorry I did not answer sooner.
>>>>>>>>>>>
>>>>>>>>>>> @Andrew: do you have a Makefile for the python-iptables packages ?
>>>>>>>>>>> I'd like to try to build it and see the issue. Or, are you just
>>>>>>>>>>> using
>>>>>>>>>>> that .py file ?
>>>>>>>>>>> Can you give a bit more input on which Python version you're
>>>>>>>>>>> using,
>>>>>>>>>>> and which OpenWrt version?
>>>>>>>>>>>
>>>>>>>>>>> If the issue is still present in the current packages trunk, I'd
>>>>>>>>>>> like
>>>>>>>>>>> to
>>>>>>>>>>> fix it.
>>>>>>>>>>> And if  郭涛's fix works, we can apply it to trunk.
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>> Alex
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jan 16, 2017 at 6:23 AM, 郭涛 <guotao945 at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I also meet this issue.
>>>>>>>>>>>> I fixed it using below change
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/gt945/Netgear-D7800-Openwrt-Packages/commit/fab71ca0ebf36d5f7b495b96f14d459e794b7224
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2017-01-13 0:43 GMT+08:00 Andrew McConachie <andrew at depht.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi OpenWRT Devs,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm building an OpenWRT package for python-iptables for a
>>>>>>>>>>>>> project
>>>>>>>>>>>>> I'm
>>>>>>>>>>>>> working on and getting this error message when attempting to use
>>>>>>>>>>>>> it.
>>>>>>>>>>>>>
>>>>>>>>>>>>>         import iptc
>>>>>>>>>>>>>       File "/usr/lib/python2.7/site-packages/iptc/__init__.py",
>>>>>>>>>>>>> line
>>>>>>>>>>>>> 10, in
>>>>>>>>>>>>> <module>
>>>>>>>>>>>>>         from ip4tc import (is_table_available, Table, Chain,
>>>>>>>>>>>>> Rule,
>>>>>>>>>>>>> Match,
>>>>>>>>>>>>> Target,
>>>>>>>>>>>>>       File "/usr/lib/python2.7/site-packages/iptc/ip4tc.py",
>>>>>>>>>>>>> line
>>>>>>>>>>>>> 13,
>>>>>>>>>>>>> in
>>>>>>>>>>>>> <module>
>>>>>>>>>>>>>         from xtables import (XT_INV_PROTO, NFPROTO_IPV4,
>>>>>>>>>>>>> XTablesError,
>>>>>>>>>>>>> xtables,
>>>>>>>>>>>>>       File "/usr/lib/python2.7/site-packages/iptc/xtables.py",
>>>>>>>>>>>>> line
>>>>>>>>>>>>> 677, in
>>>>>>>>>>>>> <module>
>>>>>>>>>>>>>         _optind = ct.c_long.in_dll(_libc, "optind")
>>>>>>>>>>>>> AttributeError: 'NoneType' object has no attribute '_handle'
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can view xtables.py here if you're curious.
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://github.com/ldx/python-iptables/blob/master/iptc/xtables.py
>>>>>>>>>>>>>
>>>>>>>>>>>>> The problem is that my python-iptables package cannot find libc
>>>>>>>>>>>>> functions
>>>>>>>>>>>>> using ctypes.util.find_library(). I've tried building OpenWRT
>>>>>>>>>>>>> using
>>>>>>>>>>>>> both
>>>>>>>>>>>>> musl and eglibc but neither work. I've also tried building
>>>>>>>>>>>>> OpenWRT
>>>>>>>>>>>>> with
>>>>>>>>>>>>> objdump and ldconfig. When I include ldconfig via 'make
>>>>>>>>>>>>> menuconfig'
>>>>>>>>>>>>> it
>>>>>>>>>>>>> doesn't actually populate my OpenWRT image with an ldconfig
>>>>>>>>>>>>> binary.
>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>> this is the problem?
>>>>>>>>>>>>>
>>>>>>>>>>>>> This bug report looks similar to my problem, but it's about MIPS
>>>>>>>>>>>>> and
>>>>>>>>>>>>> marked
>>>>>>>>>>>>> as closed.
>>>>>>>>>>>>> https://dev.openwrt.org/ticket/20123
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any help or pointers would be much appreciated.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Andrew
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> openwrt-devel mailing list
>>>>>>>>>>>>> openwrt-devel at lists.openwrt.org
>>>>>>>>>>>>> https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> openwrt-devel mailing list
>>>>>>>>>>>> openwrt-devel at lists.openwrt.org
>>>>>>>>>>>> https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
>>>>>>>>>>
>>>>>>>>>>
>>
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel


More information about the openwrt-devel mailing list