Question: No audio when using USB hdmi capture device

[AllonisDave]

Using Raspberry PI4 8gb
webrtc-streamer version 0.6.3
Trying to stream using this hdmi capture device. https://www.amazon.com/gp/product/B088HBRM7T

Video streaming is great but cannot get any audio.

The message is "Cannot create capturer audio:audiocap://2"
How best to troubleshoot this?

This is the log
pi@myserver:/usr/local/Allonis/media $ ./webrtc-streamer
Version:v0.6.3/Linux-armv7l [email protected] webrtc@f075917cb0-dirty live555helper@897450c
nullLogger level:3
USB Video: USB Video=>audiocap://2
HTTP Listen at 0.0.0.0:8000
[000:013][1633] (PeerConnectionManager.h:235): virtual void PeerConnectionManager::PeerConnectionObserver::OnSignalingChange(webrtc::PeerConnectionInterface::SignalingState) state:0 peerid:0.5319288635652775
[000:099][1633] (PeerConnectionManager.h:144): virtual void PeerConnectionManager::DataChannelObserver::OnStateChange() channel:ServerDataChannel state:open
[000:100][1633] (PeerConnectionManager.h:225): virtual void PeerConnectionManager::PeerConnectionObserver::OnDataChannel(rtc::scoped_refptrwebrtc::DataChannelInterface)
[000:100][1633] (PeerConnectionManager.h:144): virtual void PeerConnectionManager::DataChannelObserver::OnStateChange() channel:ClientDataChannel state:open
[000:100][1633] (PeerConnectionManager.h:151): virtual void PeerConnectionManager::DataChannelObserver::OnMessage(const webrtc::DataBuffer&) channel:ClientDataChannel msg:local channel openned
[000:100][1633] (PeerConnectionManager.h:151): virtual void PeerConnectionManager::DataChannelObserver::OnMessage(const webrtc::DataBuffer&) channel:ServerDataChannel msg:remote channel openned
[002:172][1633] (PeerConnectionManager.h:229): virtual void PeerConnectionManager::PeerConnectionObserver::OnRenegotiationNeeded() peerid:0.3362833677760715
[002:178][1633] (PeerConnectionManager.h:235): virtual void PeerConnectionManager::PeerConnectionObserver::OnSignalingChange(webrtc::PeerConnectionInterface::SignalingState) state:3 peerid:0.3362833677760715
[002:180][1633] (webrtc_video_engine.cc:3394): Absent receive stream; ignoring clearing encoded frame sink for ssrc 0
[002:180][1633] (PeerConnectionManager.h:214): virtual void PeerConnectionManager::PeerConnectionObserver::OnAddStream(rtc::scoped_refptrwebrtc::MediaStreamInterface) nb video tracks:1
[002:229][1641] (CapturerFactory.h:246): audiourl:audiocap://2 idx_audioDevice:2/0
[002:231][1641] (PeerConnectionManager.cpp:1167): Cannot create capturer audio:audiocap://2
[002:231][1633] (PeerConnectionManager.h:229): virtual void PeerConnectionManager::PeerConnectionObserver::OnRenegotiationNeeded() peerid:0.3362833677760715
[002:244][1633] (PeerConnectionManager.h:235): virtual void PeerConnectionManager::PeerConnectionObserver::OnSignalingChange(webrtc::PeerConnectionInterface::SignalingState) state:0 peerid:0.3362833677760715
[002:391][1633] (PeerConnectionManager.h:144): virtual void PeerConnectionManager::DataChannelObserver::OnStateChange() channel:ServerDataChannel state:open
[002:393][1633] (PeerConnectionManager.h:225): virtual void PeerConnectionManager::PeerConnectionObserver::OnDataChannel(rtc::scoped_refptrwebrtc::DataChannelInterface)
[002:393][1633] (PeerConnectionManager.h:144): virtual void PeerConnectionManager::DataChannelObserver::OnStateChange() channel:ClientDataChannel state:open
[002:393][1633] (PeerConnectionManager.h:151): virtual void PeerConnectionManager::DataChannelObserver::OnMessage(const webrtc::DataBuffer&) channel:ClientDataChannel msg:local channel openned
[002:394][1633] (PeerConnectionManager.h:151): virtual void PeerConnectionManager::DataChannelObserver::OnMessage(const webrtc::DataBuffer&) channel:ServerDataChannel msg:remote channel openned

[mpromonet]

Hi,

In order to get more information you can increase verbosity level.
It may be possible than the guess to link audio with video doesnot work in your case, using /api/getAudioDeviceList you could get audio capture.

Best Regards,
Michel

[AllonisDave]

Sorry should have include that /api/getAudioDeviceList returns just an empty [] This capture device does work with other software I’ve tried. In UV4L video is at /dev/video0 and audio is at hw:2.0 Not sure how those translate to webrtc-streamer. Thanks,

[mpromonet]

Hi,

Till /api/getAudioDeviceList return empty arry, it could not work.
Then it may audio capture is not available from your WebRTC SDK build.

In the package https://github.com/mpromonet/webrtc-streamer/releases/download/v0.6.3/webrtc-streamer-v0.6.3-Linux-armv7l-Release.tar.gz audio is not enabled.
In the build log we can see :

CMAKE_FIND_ROOT_PATH = /webrtc-streamer/../webrtc/src/build/linux/debian_sid_arm-sysroot
ALSA_FOUND = FALSE
PulseAudio_FOUND = 0
GTK_FOUND = 
Done. Made 979 targets from 251 files in 1020ms

Best Regards,
Michel.

[AllonisDave]

So does this mean I should take the source and try to compile it on my pi? Or does it mean audio will not work on the pi?

Thanks

[mpromonet]

Hi,

It will not build on pi, webrtc SDK toolchain doesnot run on arm and the pi has not enouph resources to run the build.
I made a test on my PC and the crosscompilation detect sound, but not on CI pipeline (both docker build and CircleCI fails to detect alsa & pulse), surely something is different....

Best Regards,
Michel.

[AllonisDave]

Did your experiment to try to fix sound detection work? Is there anything you would like tested?
Thanks...

[mpromonet]

Hi,

I think it is working now, you could try with last build : https://7752-27565754-gh.circle-artifacts.com/0/distrib/webrtc-streamer-v0.6.3-18-g9ad49c2-Linux-armv7l-Release.tar.gz

Best Regards,
Michel.

[AllonisDave]

A lot of progress. I am getting audio now but I have to open two streams to get it.
First here is the output of api/getAudioDeviceList
[
"default: USB Video Analog Stereo",
"USB Video Analog Stereo"
]

and here is the output of api/getVideoDeviceList
[
"USB Video: USB Video",
"bcm2835-isp",
"videocap://2"
]

If I start the streamer using the default ./webrtc-streamer there is no audio like before.

In order to get audio I need to start the streamer like this
./webrtc-streamer -n test -u "videocap://2" -U "USB Video Analog Stereo"

I've tried several different permutations of commands but that was the only one that produced results.

With this option I can open the default usb video stream to get the video. but then I must also open the test stream to get the audio. Audio is good and in sync. It's just not with the video stream.

See the attached capture.png file for reference. The verbose log is in the test.log file
test.log

Thanks for working on this so quickly.
Best...

[AllonisDave]

Update: after looking closely at webrtcstreamer.html I see you can pass audio and video parameters. So if I try http://192.168.0.235:8000/webrtcstreamer.html?video=USB%20Video:%20USB%20Video&audio=USB%20Video%20Analog%20Stereo everything works! Excellent.

[AllonisDave]

Question: If I plug in a second hdmi capture device
/api/getVideoDeviceList reports
[
"USB Video: USB Video",
"videocap://1",
"bcm2835-isp",
"videocap://3"
]
and /api/getAudioDeviceList reports
[
"default: USB Video Analog Stereo",
"USB Video Analog Stereo",
"USB Video Analog Stereo"
]
How do you distinguish between the two audio devices named "USB Video Analog Stereo"?

[mpromonet]

Hi,

In case of video, if the name already exists it is replaced by videocap://<device number>. In case of audio this is not implemented.
You may try to use directly audiocap://<audio number> . To modify the enumeration the same as video, it needs to update the code.

Best Regards,
Michel.