[PATCH v13 00/17] Introduce the Counter character device interface

[William Breathitt Gray]

On Tue, Jul 13, 2021 at 06:53:04PM +0900, William Breathitt Gray wrote:
> Suppose I open the chrdev from a userspace application and keep it open,
> but then remove the respective driver and Counter subsystem module from
> my system. The devm_counter_release() and counter_exit() functions will
> be called as expected; the counter_chrdev_release() function will not be
> called yet, but that is expected because the chrdev is still open by
> userspace. If I try to break out of my userspace application, I expect
> counter_chrdev_release() to finally be called, but this does not happen.
> Instead, my userspace application stalls and I see the following error
> in my dmesg:
> 
> [  172.859570] BUG: unable to handle page fault for address: ffffffffc09ae298
> [  172.859594] #PF: supervisor read access in kernel mode
> [  172.859598] #PF: error_code(0x0000) - not-present page
> [  172.859603] PGD 23615067 P4D 23615067 PUD 23617067 PMD 1029ad067 PTE 0
> [  172.859623] Oops: 0000 [#1] SMP NOPTI
> [  172.859629] CPU: 2 PID: 2485 Comm: counter_example Not tainted 5.13.0+ #1
> [  172.859640] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS d55cb5a 04/01/2014
> [  172.859645] RIP: 0010:filp_close+0x29/0x70
> [  172.859662] Code: 90 0f 1f 44 00 00 55 48 89 e5 41 56 41 55 41 54 48 8b 47 38 48 85 c0 0f 84 b7 40 86 00 48 8b 47 28 49 89 fc 49 89 f5 45 31 f6 <48> 8b 40 78 48 85 c0 74 08 ff d0 0f 1f 00 41 89 c6 41 f6 44 24 45
> [  172.859669] RSP: 0018:ffffad31c0ee7cb0 EFLAGS: 00010246
> [  172.859675] RAX: ffffffffc09ae220 RBX: 0000000000000001 RCX: 0000000000000001
> [  172.859680] RDX: ffff9a43829708e0 RSI: ffff9a4382970840 RDI: ffff9a43821f4f00
> [  172.859684] RBP: ffffad31c0ee7cc8 R08: 0000000000000001 R09: 0000000000000001
> [  172.859687] R10: ffffffffffff4d00 R11: ffff9a43933c6e10 R12: ffff9a43821f4f00
> [  172.859691] R13: ffff9a4382970840 R14: 0000000000000000 R15: 0000000000000003
> [  172.859694] FS:  0000000000000000(0000) GS:ffff9a44b7d00000(0000) knlGS:0000000000000000
> [  172.859699] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [  172.859704] CR2: ffffffffc09ae298 CR3: 0000000023610001 CR4: 0000000000370ee0
> [  172.859713] Call Trace:
> [  172.859730]  put_files_struct+0x73/0xd0
> [  172.859738]  exit_files+0x49/0x50
> [  172.859743]  do_exit+0x33b/0xa20
> [  172.859751]  do_group_exit+0x3b/0xb0
> [  172.859758]  get_signal+0x16f/0x8b0
> [  172.859766]  ? _copy_to_user+0x20/0x30
> [  172.859774]  ? put_timespec64+0x3d/0x60
> [  172.859784]  arch_do_signal_or_restart+0xf3/0x850
> [  172.859794]  ? hrtimer_nanosleep+0x9f/0x120
> [  172.859802]  ? __hrtimer_init+0xd0/0xd0
> [  172.859808]  exit_to_user_mode_prepare+0x122/0x1b0
> [  172.859816]  syscall_exit_to_user_mode+0x27/0x50
> [  172.859825]  do_syscall_64+0x48/0xc0
> [  172.859831]  entry_SYSCALL_64_after_hwframe+0x44/0xae
> [  172.859839] RIP: 0033:0x7f07f8b9951a
> [  172.859850] Code: Unable to access opcode bytes at RIP 0x7f07f8b994f0.
> [  172.859853] RSP: 002b:00007ffc0d12c230 EFLAGS: 00000246 ORIG_RAX: 00000000000000e6
> [  172.859860] RAX: fffffffffffffdfc RBX: ffffffffffffff01 RCX: 00007f07f8b9951a
> [  172.859863] RDX: 00007ffc0d12c2b0 RSI: 0000000000000000 RDI: 0000000000000000
> [  172.859867] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007ffc0d12c1c6
> [  172.859871] R10: 00007ffc0d12c2b0 R11: 0000000000000246 R12: 00007ffc0d12c2b0
> [  172.859874] R13: 00007ffc0d12c2b0 R14: 0000000000000000 R15: 0000000000000000
> [  172.859886] Modules linked in: intel_rapl_msr intel_rapl_common kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel nls_iso8859_1 aesni_intel crypto_simd cryptd rapl drm_ttm_helper ttm uvcvideo drm_kms_helper videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common input_leds syscopyarea videodev sysfillrect sysimgblt fb_sys_fops cec rc_core joydev mc drm serio_raw mac_hid qemu_fw_cfg sch_fq_codel msr parport_pc ppdev lp parport virtio_rng ip_tables x_tables autofs4 hid_generic usbhid hid virtio_net psmouse net_failover i2c_piix4 virtio_blk failover pata_acpi floppy [last unloaded: counter]
> [  172.859995] CR2: ffffffffc09ae298
> [  172.860009] ---[ end trace e7d3d7da1a73b8f4 ]---
> [  172.860013] RIP: 0010:filp_close+0x29/0x70
> [  172.860021] Code: 90 0f 1f 44 00 00 55 48 89 e5 41 56 41 55 41 54 48 8b 47 38 48 85 c0 0f 84 b7 40 86 00 48 8b 47 28 49 89 fc 49 89 f5 45 31 f6 <48> 8b 40 78 48 85 c0 74 08 ff d0 0f 1f 00 41 89 c6 41 f6 44 24 45
> [  172.860027] RSP: 0018:ffffad31c0ee7cb0 EFLAGS: 00010246
> [  172.860031] RAX: ffffffffc09ae220 RBX: 0000000000000001 RCX: 0000000000000001
> [  172.860034] RDX: ffff9a43829708e0 RSI: ffff9a4382970840 RDI: ffff9a43821f4f00
> [  172.860038] RBP: ffffad31c0ee7cc8 R08: 0000000000000001 R09: 0000000000000001
> [  172.860041] R10: ffffffffffff4d00 R11: ffff9a43933c6e10 R12: ffff9a43821f4f00
> [  172.860044] R13: ffff9a4382970840 R14: 0000000000000000 R15: 0000000000000003
> [  172.860047] FS:  0000000000000000(0000) GS:ffff9a44b7d00000(0000) knlGS:0000000000000000
> [  172.860052] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [  172.860056] CR2: ffffffffc09ae298 CR3: 0000000023610001 CR4: 0000000000370ee0
> [  172.860073] Fixing recursive fault but reboot is needed!
> 
> It looks like faults in filp_close() before counter_chrdev_release() is
> called. Is this issue manifesting because counter_exit() was called
> earlier while the chrdev was still open?

After giving this some more thought I realized that once the counter
module is unloaded, the counter_fops callbacks are lost and thus the
fops for the chrdev are no longer be valid. This means that
counter_chrdev_release will need to be called before the counter module
is unloaded. How can I guarantee this if a userspace application may
still have the chrdev indefinitely open? In counter_chrdev_remove(),
should I walk through the open chrdevs and release each one?

William Breathitt Gray
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20210728/871aced4/attachment.sig>