Multiple Qualcomm Hamilton cards on same host - none working

Tue Feb 6 23:08:08 PST 2024

Hi,

We need to run multiple wcn7850 (Hamilton) cards on the same host. 

A single wcn7850 works fine, more than one and all fail

Logs from a working setup with a single card using Ubuntu 23.10.1 desktop install

[    2.494236] ath12k_pci 0000:03:00.0: BAR 0: assigned [mem 0xfe200000-0xfe3fffff 64bit]
[    2.494256] ath12k_pci 0000:03:00.0: enabling device (0000 -> 0002)
[    2.496200] ath12k_pci 0000:03:00.0: Hardware name: wcn7850 hw2.0
[    4.993821] ath12k_pci 0000:03:00.0: chip_id 0x2 chip_family 0x4 board_id 0xff soc_id 0x40170200
[    4.993827] ath12k_pci 0000:03:00.0: fw_version 0x100301e1 fw_build_timestamp 2023-12-06 04:05 fw_build_id QC_IMAGE_VERSION_STRING=WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
[    5.003142] ath12k_pci 0000:03:00.0: failed to fetch board data for bus=pci,qmi-chip-id=2,qmi-board-id=255 from ath12k/WCN7850/hw2.0/board-2.bin
[    5.196430] ath12k_pci 0000:03:00.0 wlp3s0: renamed from wlan0
[    9.446866] UBSAN: array-index-out-of-bounds in /build/linux-9czVP0/linux-6.5.0/drivers/net/wireless/ath/ath12k/mac.c:2774:17
[    9.447067]  ath12k_mac_op_hw_scan+0x39c/0x3e0 [ath12k]

Logs form a similar setup with 2 cards

sudo dmesg | grep ath12k

gives me

[    3.570029] ath12k_pci 0000:01:00.0: BAR 0: assigned [mem 0xfc800000-0xfc9fffff 64bit]
[    3.570046] ath12k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    3.570347] ath12k_pci 0000:01:00.0: Hardware name: wcn7850 hw2.0
[    3.659620] ath12k_pci 0000:05:00.0: BAR 0: assigned [mem 0xfc400000-0xfc5fffff 64bit]
[    3.659648] ath12k_pci 0000:05:00.0: enabling device (0000 -> 0002)
[    3.660023] ath12k_pci 0000:05:00.0: Hardware name: wcn7850 hw2.0
[    4.910790] ath12k_pci 0000:01:00.0: Host capability request failed, result: 1, err: 90
[    4.910795] ath12k_pci 0000:01:00.0: qmi failed to send host cap QMI:-22
[    4.912112] ath12k_pci 0000:05:00.0: Host capability request failed, result: 1, err: 90
[    4.912116] ath12k_pci 0000:05:00.0: qmi failed to send host cap QMI:-22
[    4.913414] ath12k_pci 0000:01:00.0: Host capability request failed, result: 1, err: 90
[    4.913418] ath12k_pci 0000:01:00.0: qmi failed to send host cap QMI:-22

After running

sudo dmesg -n 8

then run

sudo dmesg | grep ath12k

again 

[49523.559674] ath12k_pci 0000:01:00.0: BAR 0: assigned [mem 0xfc800000-0xfc9fffff 64bit]
[49523.559965] ath12k_pci 0000:01:00.0: Hardware name: wcn7850 hw2.0
[49523.615645] ath12k_pci 0000:05:00.0: BAR 0: assigned [mem 0xfc400000-0xfc5fffff 64bit]
[49523.616109] ath12k_pci 0000:05:00.0: Hardware name: wcn7850 hw2.0
[49524.094457] ath12k_pci 0000:05:00.0: Host capability request failed, result: 1, err: 90
[49524.094469] ath12k_pci 0000:05:00.0: qmi failed to send host cap QMI:-22
[49524.095445] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000000 flags=0x0000]
[49524.095668] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000000 flags=0x0020]
[49524.096223] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000000 flags=0x0000]
[49524.096307] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7540000 flags=0x0020]
[49524.096316] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000040 flags=0x0000]
[49524.096327] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000080 flags=0x0000]
[49524.096335] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb70000c0 flags=0x0000]
[49524.096343] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000100 flags=0x0000]
[49524.096353] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000140 flags=0x0000]
[49524.096361] ath12k_pci 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xb7000180 flags=0x0000]
[49524.099587] ath12k_pci 0000:05:00.0: chip_id 0x2 chip_family 0x4 board_id 0x10e soc_id 0x40170200
[49524.099593] ath12k_pci 0000:05:00.0: fw_version 0x100301e1 fw_build_timestamp 2023-12-06 04:05 fw_build_id QC_IMAGE_VERSION_STRING=WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
[49524.110011] ath12k_pci 0000:05:00.0: failed to fetch board data for bus=pci,qmi-chip-id=2,qmi-board-id=270 from ath12k/WCN7850/hw2.0/board-2.bin
[49534.169968] ath12k_pci 0000:05:00.0: qmi failed set mode request, mode: 0, err = -110
[49534.169981] ath12k_pci 0000:05:00.0: qmi failed to send wlan fw mode:-110
[49534.169986] ath12k_pci 0000:05:00.0: failed to send firmware start: -110
[49534.169989] ath12k_pci 0000:05:00.0: failed to start firmware: -110

The error

[49524.094457] ath12k_pci 0000:05:00.0: Host capability request failed, result: 1, err: 90
[49524.094469] ath12k_pci 0000:05:00.0: qmi failed to send host cap QMI:-22

Would appear to originate in ath12k_qmi_host_cap_send(), I assume that all works but that the response read from qmi_txn_wait() is 90, which from qmi.h is QMI_ERR_INCOMPATIBLE_STATE_V01. No idea where to go on that!

Both the working and non working have this line

[    5.003142] ath12k_pci 0000:03:00.0: failed to fetch board data for bus=pci,qmi-chip-id=2,qmi-board-id=255 from ath12k/WCN7850/hw2.0/board-2.bin

I used firmware files from

https://github.com/kvalo/ath11k-firmware/tree/master/ath12k-testing

using the files from 

ath12k-testing/WCN7850/hw2.0/WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3

copied into the appropriate firmware file location

the last error

[49534.169968] ath12k_pci 0000:05:00.0: qmi failed set mode request, mode: 0, err = -110
[49534.169981] ath12k_pci 0000:05:00.0: qmi failed to send wlan fw mode:-110
[49534.169986] ath12k_pci 0000:05:00.0: failed to send firmware start: -110
[49534.169989] ath12k_pci 0000:05:00.0: failed to start firmware: -110

Appears to originate in ath12k_qmi_wlanfw_mode_send(), not able to find a meaning for the -110 code.

Not sure if the later error is relevant given the first has occurred though!

Any ideas?

David Atkins