[PATCH v4 00/32] Introduce QC USB SND audio offloading support
Wesley Cheng
quic_wcheng at quicinc.com
Mon Jul 24 19:38:25 PDT 2023
Apologies for the long delay in addressing some of your comments and
feedback. I understand it is difficult to refresh a lot of the material
after such a long gap, but hopefully I have addressed some of the
concerns in the latest revision. Appreciate all the suggestions/reviews!
Thanks
Wesley Cheng
On 7/24/2023 7:33 PM, Wesley Cheng wrote:
> Several Qualcomm based chipsets can support USB audio offloading to a
> dedicated audio DSP, which can take over issuing transfers to the USB
> host controller. The intention is to reduce the load on the main
> processors in the SoC, and allow them to be placed into lower power modes.
> There are several parts to this design:
> 1. Adding ASoC binding layer
> 2. Create a USB backend for Q6DSP
> 3. Introduce XHCI interrupter support
> 4. Create vendor ops for the USB SND driver
>
> USB | ASoC
> --------------------------------------------------------------------
> | _________________________
> | |sm8250 platform card |
> | |_________________________|
> | | |
> | ___V____ ____V____
> | |Q6USB | |Q6AFE |
> | |"codec" | |"cpu" |
> | |________| |_________|
> | ^ ^ ^
> | | |________|
> | ___V____ |
> | |SOC-USB | |
> ________ ________ | | |
> |USB SND |<--->|QC offld|<------------>|________| |
> |(card.c)| | |<---------- |
> |________| |________|___ | | |
> ^ ^ | | | ____________V_________
> | | | | | |APR/GLINK |
> __ V_______________V_____ | | | |______________________|
> |USB SND (endpoint.c) | | | | ^
> |_________________________| | | | |
> ^ | | | ___________V___________
> | | | |->|audio DSP |
> ___________V_____________ | | |_______________________|
> |XHCI HCD |<- |
> |_________________________| |
>
>
> Adding ASoC binding layer:
> soc-usb: Intention is to treat a USB port similar to a headphone jack.
> The port is always present on the device, but cable/pin status can be
> enabled/disabled. Expose mechanisms for USB backend ASoC drivers to
> communicate with USB SND.
>
> Create a USB backend for Q6DSP:
> q6usb: Basic backend driver that will be responsible for maintaining the
> resources needed to initiate a playback stream using the Q6DSP. Will
> be the entity that checks to make sure the connected USB audio device
> supports the requested PCM format. If it does not, the PCM open call will
> fail, and userpsace ALSA can take action accordingly.
>
> Introduce XHCI interrupter support:
> XHCI HCD supports multiple interrupters, which allows for events to be routed
> to different event rings. This is determined by "Interrupter Target" field
> specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
>
> Events in the offloading case will be routed to an event ring that is assigned
> to the audio DSP.
>
> Create vendor ops for the USB SND driver:
> qc_audio_offload: This particular driver has several components associated
> with it:
> - QMI stream request handler
> - XHCI interrupter and resource management
> - audio DSP memory management
>
> When the audio DSP wants to enable a playback stream, the request is first
> received by the ASoC platform sound card. Depending on the selected route,
> ASoC will bring up the individual DAIs in the path. The Q6USB backend DAI
> will send an AFE port start command (with enabling the USB playback path), and
> the audio DSP will handle the request accordingly.
>
> Part of the AFE USB port start handling will have an exchange of control
> messages using the QMI protocol. The qc_audio_offload driver will populate the
> buffer information:
> - Event ring base address
> - EP transfer ring base address
>
> and pass it along to the audio DSP. All endpoint management will now be handed
> over to the DSP, and the main processor is not involved in transfers.
>
> Overall, implementing this feature will still expose separate sound card and PCM
> devices for both the platorm card and USB audio device:
> 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
> SM8250-MTP-WCD9380-WSA8810-VA-DMIC
> 1 [Audio ]: USB-Audio - USB Audio
> Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
>
> This is to ensure that userspace ALSA entities can decide which route to take
> when executing the audio playback. In the above, if card#1 is selected, then
> USB audio data will take the legacy path over the USB PCM drivers, etc...
>
> This feature was validated using:
> - tinymix: set/enable the multimedia path to route to USB backend
> - tinyplay: issue playback on platform card
>
> Changelog
> --------------------------------------------
> Changes in v4:
> - Rebased to xhci/for-usb-next
> - Addressed some dt-bindings comments
>
> XHCI:
> - Pulled in latest changes from Mathias' feature_interrupters branch:
> https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=feature_interrupters
>
> - Fixed commit text and signage for the XHCI sideband/interrupter related changes
> - Added some logic to address the FIXME tags mentioned throughout the commits, such
> as handling multi segment rings and building the SGT, locking concerns, and ep
> cleanup operations.
> - Removed some fixme tags for conditions that may not be needed/addressed.
> - Repurposed the new endpoint stop sync API to be utilized in other places.
> - Fixed potential compile issue if XHCI sideband config is not defined.
>
> ASoC:
> - Added sound jack control into the Q6USB driver. Allows for userpsace to know when
> an offload capable device is connected.
>
> USB SND:
> - Avoided exporting _snd_pcm_hw_param_set based on Takashi's recommendation.
> - Split USB QMI packet header definitions into a separate commit. This is used to
> properly allow the QMI interface driver to parse and route QMI packets accordingly
> - Added a "depends on" entry when enabling QC audio offload to avoid compile time
> issues.
>
> Changes in v3:
> - Changed prefix from RFC to PATCH
> - Rebased entire series to usb-next
> - Updated copyright years
>
> XHCI:
> - Rebased changes on top of XHCI changes merged into usb-next, and only added
> changes that were still under discussion.
> - Added change to read in the "num-hc-interrupters" device property.
>
> ASoC:
> - qusb6 USB backend
> - Incorporated suggestions to fetch iommu information with existing APIs
> - Added two new sound kcontrols to fetch offload status and offload device
> selection.
> - offload status - will return the card and pcm device in use
> tinymix -D 0 get 1 --> 1, 0 (offload in progress on card#1 pcm#0)
>
> - device selection - set the card and pcm device to enable offload on. Ex.:
> tinymix -D 0 set 1 2 0 --> sets offload on card#2 pcm#0
> (this should be the USB card)
>
> USB SND:
> - Fixed up some locking related concerns for registering platform ops.
> - Moved callbacks under the register_mutex, so that
> - Modified APIs to properly pass more information about the USB SND device, so
> that the Q6USB backend can build a device list/map, in order to monitor offload
> status and device selection.
>
> Changes in v2:
>
> XHCI:
> - Replaced XHCI and HCD changes with Mathias' XHCI interrupter changes
> in his tree:
> https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=feature_interrupters
>
> Adjustments made to Mathias' changes:
> - Created xhci-intr.h to export/expose interrupter APIs versus exposing xhci.h.
> Moved dependent structures to this file as well. (so clients can parse out
> information from "struct xhci_interrupter")
> - Added some basic locking when requesting interrupters.
> - Fixed up some sanity checks.
> - Removed clearing of the ERSTBA during freeing of the interrupter. (pending
> issue where SMMU fault occurs if DMA addr returned is 64b - TODO)
>
> - Clean up pending events in the XHCI secondary interrupter. While testing USB
> bus suspend, it was seen that on bus resume, the xHCI HC would run into a command
> timeout.
> - Added offloading APIs to xHCI to fetch transfer and event ring information.
>
> ASoC:
> - Modified soc-usb to allow for multiple USB port additions. For this to work,
> the USB offload driver has to have a reference to the USB backend by adding
> a "usb-soc-be" DT entry to the device saved into XHCI sysdev.
> - Created separate dt-bindings for defining USB_RX port.
> - Increased APR timeout to accommodate the situation where the AFE port start
> command could be delayed due to having to issue a USB bus resume while
> handling the QMI stream start command.
>
> USB SND:
> - Added a platform ops during usb_audio_suspend(). This allows for the USB
> offload driver to halt the audio stream when system enters PM suspend. This
> ensures the audio DSP is not issuing transfers on the USB bus.
> - Do not override platform ops if they are already populated.
> - Introduce a shared status variable between the USB offload and USB SND layers,
> to ensure that only one path is active at a time. If the USB bus is occupied,
> then userspace is notified that the path is busy.
>
> Mathias Nyman (3):
> xhci: add support to allocate several interrupters
> xhci: add helper to stop endpoint and wait for completion
> xhci: sideband: add initial api to register a sideband entity
>
> Wesley Cheng (29):
> usb: host: xhci-mem: Cleanup pending secondary event ring events
> usb: host: xhci-mem: Allow for interrupter clients to choose specific
> index
> ASoC: Add SOC USB APIs for adding an USB backend
> ASoC: dt-bindings: qcom,q6dsp-lpass-ports: Add USB_RX port
> ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp
> ASoC: qdsp6: q6afe: Increase APR timeout
> ASoC: qcom: Add USB backend ASoC driver for Q6
> sound: usb: card: Introduce USB SND platform op callbacks
> sound: usb: Export USB SND APIs for modules
> dt-bindings: usb: dwc3: Add snps,num-hc-interrupters definition
> usb: dwc3: Add DT parameter to specify maximum number of interrupters
> usb: host: xhci-plat: Set XHCI max interrupters if property is present
> sound: usb: pcm: Export fixed rate check USB SND API
> sound: usb: qcom: Add USB QMI definitions
> sound: usb: Introduce QC USB SND offloading support
> sound: usb: card: Check for support for requested audio format
> sound: soc: soc-usb: Add PCM format check API for USB backend
> sound: soc: qcom: qusb6: Ensure PCM format is supported by USB audio
> device
> sound: usb: Prevent starting of audio stream if in use
> ASoC: dt-bindings: Add Q6USB backend bindings
> ASoC: dt-bindings: Update example for enabling USB offload on SM8250
> ASoC: qcom: qdsp6: q6afe: Split USB AFE dev_token param into separate
> API
> sound: Pass USB SND card and PCM information to SOC USB
> sound: soc: qdsp6: Add SND kcontrol to select offload device
> sound: soc: qdsp6: Add SND kcontrol for fetching offload status
> sound: soc: qcom: q6usb: Add headphone jack for offload connection
> status
> sound: usb: qc_audio_offload: Use card and PCM index from QMI request
> sound: usb: card: Allow for rediscovery of connected USB SND devices
> sound: soc: soc-usb: Rediscover USB SND devices on USB port add
>
> .../bindings/sound/qcom,q6usb-dais.yaml | 49 +
> .../bindings/sound/qcom,sm8250.yaml | 15 +
> .../devicetree/bindings/usb/snps,dwc3.yaml | 13 +
> arch/arm64/boot/dts/qcom/sm8350.dtsi.rej | 14 +
> arch/arm64/configs/defconfig.rej | 24 +
> drivers/usb/dwc3/core.c | 12 +
> drivers/usb/dwc3/core.h | 2 +
> drivers/usb/dwc3/host.c | 5 +-
> drivers/usb/host/Kconfig | 9 +
> drivers/usb/host/Makefile | 4 +
> drivers/usb/host/xhci-debugfs.c | 2 +-
> drivers/usb/host/xhci-hub.c | 29 +-
> drivers/usb/host/xhci-mem.c | 167 +-
> drivers/usb/host/xhci-plat.c | 2 +
> drivers/usb/host/xhci-ring.c | 2 +-
> drivers/usb/host/xhci-sideband.c | 195 ++
> drivers/usb/host/xhci.c | 109 +-
> drivers/usb/host/xhci.h | 102 +-
> .../sound/qcom,q6dsp-lpass-ports.h | 1 +
> include/linux/usb/xhci-intr.h | 86 +
> include/linux/usb/xhci-sideband.h | 90 +
> include/sound/q6usboffload.h | 20 +
> include/sound/soc-usb.h | 39 +
> sound/core/oss/pcm_oss.c | 4 +-
> sound/soc/Makefile | 2 +-
> sound/soc/qcom/Kconfig | 4 +
> sound/soc/qcom/qdsp6/Makefile | 1 +
> sound/soc/qcom/qdsp6/q6afe-dai.c | 50 +
> sound/soc/qcom/qdsp6/q6afe.c | 208 +-
> sound/soc/qcom/qdsp6/q6afe.h | 47 +-
> sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c | 23 +
> sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h | 1 +
> sound/soc/qcom/qdsp6/q6routing.c | 9 +
> sound/soc/qcom/qdsp6/q6usb.c | 458 +++++
> sound/soc/soc-usb.c | 197 ++
> sound/usb/Kconfig | 15 +
> sound/usb/Makefile | 2 +-
> sound/usb/card.c | 85 +
> sound/usb/card.h | 32 +
> sound/usb/endpoint.c | 2 +
> sound/usb/helper.c | 1 +
> sound/usb/pcm.c | 29 +-
> sound/usb/pcm.h | 12 +
> sound/usb/qcom/Makefile | 2 +
> sound/usb/qcom/qc_audio_offload.c | 1820 +++++++++++++++++
> sound/usb/qcom/usb_audio_qmi_v01.c | 892 ++++++++
> sound/usb/qcom/usb_audio_qmi_v01.h | 162 ++
> 47 files changed, 4870 insertions(+), 179 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
> create mode 100644 arch/arm64/boot/dts/qcom/sm8350.dtsi.rej
> create mode 100644 arch/arm64/configs/defconfig.rej
> create mode 100644 drivers/usb/host/xhci-sideband.c
> create mode 100644 include/linux/usb/xhci-intr.h
> create mode 100644 include/linux/usb/xhci-sideband.h
> create mode 100644 include/sound/q6usboffload.h
> create mode 100644 include/sound/soc-usb.h
> create mode 100644 sound/soc/qcom/qdsp6/q6usb.c
> create mode 100644 sound/soc/soc-usb.c
> create mode 100644 sound/usb/qcom/Makefile
> create mode 100644 sound/usb/qcom/qc_audio_offload.c
> create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.c
> create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.h
>
More information about the linux-arm-kernel
mailing list