ath11k and vfio-pci support

[Baochen Qiang]

On 1/17/2024 9:20 PM, James Prestwood wrote:
> Hi Baochen,
> 
> <snip>
> 
>>>>
>>>> While I don't think there is a way for qemu/ath11k to get the real 
>>>> MSI vector from host, I will try to read the vfio code to check 
>>>> further. Before that, to unblock you, a possible hack is to hard 
>>>> code the MSI vector in qemu to the same as in host, on condition 
>>>> that the MSI vector doesn't change. In my case, the change looks like
>>>>
>>>> diff --git a/drivers/net/wireless/ath/ath11k/pci.c 
>>>> b/drivers/net/wireless/ath/ath11k/pci.c
>>>> index 09e65c5e55c4..89a9bbe9e4d2 100644
>>>> --- a/drivers/net/wireless/ath/ath11k/pci.c
>>>> +++ b/drivers/net/wireless/ath/ath11k/pci.c
>>>> @@ -459,7 +459,12 @@ static int ath11k_pci_alloc_msi(struct 
>>>> ath11k_pci *ab_pci)
>>>>                 ab->pci.msi.addr_hi = 0;
>>>>         }
>>>>
>>>> -       ath11k_dbg(ab, ATH11K_DBG_PCI, "msi base data is %d\n", 
>>>> ab->pci.msi.ep_base_data);
>>>> +       ab->pci.msi.addr_hi = 0;
>>>> +       ab->pci.msi.addr_lo = 0xfee00578;
>>>> +       ath11k_dbg(ab, ATH11K_DBG_PCI, "msi addr hi 0x%x lo 0x%x 
>>>> base data is %d\n",
>>>> +                  ab->pci.msi.addr_hi,
>>>> +                  ab->pci.msi.addr_lo,
>>>> +                  ab->pci.msi.ep_base_data);
>>>>
>>>>         return 0;
>>>>
>>>> @@ -487,6 +492,7 @@ static int ath11k_pci_config_msi_data(struct 
>>>> ath11k_pci *ab_pci)
>>>>         }
>>>>
>>>>         ab_pci->ab->pci.msi.ep_base_data = msi_desc->msg.data;
>>>> +       ab_pci->ab->pci.msi.ep_base_data = 0;
>>>>
>>>>         ath11k_dbg(ab_pci->ab, ATH11K_DBG_PCI, "after request_irq 
>>>> msi_ep_base_data %d\n",
>>>>                    ab_pci->ab->pci.msi.ep_base_data);
>>>>
>>>>
>>>> This hack works on my setup.
>>>
>>> Progress! Thank you. This didn't work for me but its likely because 
>>> my host MSI vector is not fee00578. Where did you come up with this 
>>> value? 
>> It could, and most likely, be different from machine to machine.
>>
>>> I don't see anything in the dmesg logs, or in lspci etc.
>>>
>> fee00578 is the physical MSI vector so I got it using lspci in host, see
>> ...
>>         Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit-
>>                 Address: fee00578  Data: 0000
>>                 Masking: fffffffe  Pending: 00000000
>> ...
> 
> Mine looks like this:
> 
> ...
> 
>      Capabilities: [50] MSI: Enable- Count=1/32 Maskable+ 64bit-
>          Address: 00000000  Data: 0000
>          Masking: 00000000  Pending: 00000000
I guess you got this output when qemu/ath11k is not running, either qemu 
not started or ath11k module not loaded, right? Since MSI is not enabled 
in this output.

Sorry I should have made it clear:
step1: start qemu and load ath11k without changing anything, then you 
get those errors;
step2: keep qemu running and ath11k there, don't unload it. Then check 
lspci output in host to get the MSI vector.
step3: Back in qemu, hard code MSI vector to the one you get in step2, 
then rebuild and reload ath11k.

> ...
> 
> I've adjusted the patch:
> 
> diff --git a/drivers/net/wireless/ath/ath11k/pci.c 
> b/drivers/net/wireless/ath/ath11k/pci.c
> index 09e65c5e55..1cc7115582 100644
> --- a/drivers/net/wireless/ath/ath11k/pci.c
> +++ b/drivers/net/wireless/ath/ath11k/pci.c
> @@ -459,7 +459,12 @@ static int ath11k_pci_alloc_msi(struct ath11k_pci 
> *ab_pci)
>                  ab->pci.msi.addr_hi = 0;
>          }
> 
> -       ath11k_dbg(ab, ATH11K_DBG_PCI, "msi base data is %d\n", 
> ab->pci.msi.ep_base_data);
> +        ab->pci.msi.addr_hi = 0;
> +        ab->pci.msi.addr_lo = 0;
> +        ath11k_dbg(ab, ATH11K_DBG_PCI, "msi addr hi 0x%x lo 0x%x base 
> data is %d\n",
> +                   ab->pci.msi.addr_hi,
> +                   ab->pci.msi.addr_lo,
> +                   ab->pci.msi.ep_base_data);
> 
>          return 0;
> 
> @@ -487,6 +492,7 @@ static int ath11k_pci_config_msi_data(struct 
> ath11k_pci *ab_pci)
>          }
> 
>          ab_pci->ab->pci.msi.ep_base_data = msi_desc->msg.data;
> +       ab_pci->ab->pci.msi.ep_base_data = 0;
> 
>          ath11k_dbg(ab_pci->ab, ATH11K_DBG_PCI, "after request_irq 
> msi_ep_base_data %d\n",
>                     ab_pci->ab->pci.msi.ep_base_data);
> 
> But still getting the same errors:
> 
> [    3.563057] ath11k_pci 0000:00:06.0: Service connect timeout
> [    3.565044] ath11k_pci 0000:00:06.0: failed to connect to HTT: -110
> [    3.567031] ath11k_pci 0000:00:06.0: failed to start core: -110
> [    3.777514] ath11k_pci 0000:00:06.0: firmware crashed: MHI_CB_EE_RDDM
> [    3.777555] ath11k_pci 0000:00:06.0: ignore reset dev flags 0x4000
> [    3.885137] ath11k_pci 0000:00:06.0: firmware crashed: MHI_CB_EE_RDDM
> [    3.885178] ath11k_pci 0000:00:06.0: ignore reset dev flags 0x4000
> 
> I know this isn't a proper fix, so if its something that needs more 
> thought than just hard-coded values I understand.
> 
> Thanks,
> 
> James
> 
>>
>>> Thanks,
>>>
>>> James
>>>