by Patrik Höglund
This is the second in a series of articles about Chrome’s WebRTC Interop Test. See the first .
Verifying the Call
Now we can launch the two browsers, but how do we figure out the whether the call actually worked? If you try opening two apprtc.appspot.com tabs in the same room, you will notice the video feeds flip over using a CSS transform, your local video is relegated to a small frame and a new big video feed with the remote video shows up. For the first version of the test, I just looked at the page in the Chrome debugger and looked for some reliable signal. As it turns out, the remoteVideo.style.opacity bool WaitForCallToComeUp(content::WebContents* tab_contents) {
// Apprtc will set remoteVideo.style.opacity to 1 when the call comes up.
std::string javascript =
"window.domAutomationController.send(remoteVideo.style.opacity)" ;
return test::PollingWaitUntil(javascript, "1" , tab_contents);
}
Verifying Video is Playing
So getting a call up is good, but what if there is a bug where Firefox and Chrome cannot send correct video streams to each other? To check that, we needed to step up our game a bit. We decided to use our existing video detector , which looks at a video element and determines if the pixels are changing. This is a very basic check, but it’s better than nothing. To do this, we simply evaluate the .js file’s JavaScript in the context of the Chrome tab, making the functions in the file available to us. The implementation then becomes
bool DetectRemoteVideoPlaying(content::WebContents* tab_contents) {
if (!EvalInJavascriptFile(tab_contents, GetSourceDir().Append(
FILE_PATH_LITERAL(
"chrome/test/data/webrtc/test_functions.js" ))))
return false ;
if (!EvalInJavascriptFile(tab_contents, GetSourceDir().Append(
FILE_PATH_LITERAL(
"chrome/test/data/webrtc/video_detector.js" ))))
return false ;
// The remote video tag is called remoteVideo in the AppRTC code.
StartDetectingVideo(tab_contents, "remoteVideo" );
WaitForVideoToPlay(tab_contents);
return true ;
}StartDetectingVideo WaitForVideoToPlay video_detector.js . If the video feed is frozen and unchanging, the test will time out and fail.
What to Send in the Call
Now we can get a call up between the browsers and detect if video is playing. But what video should we send? For chrome, we have a convenient --use-fake-device-for-media-stream regular test infrastructure do not have either software or hardware webcams plugged into them, so this test must run on bots with webcams for Firefox to be able to acquire a camera. Fortunately, we have that in the WebRTC waterfalls in order to test that we can actually acquire hardware webcams on all platforms. We also added a check to just succeed the test when there’s no real webcam on the system since we don’t want it to fail when a dev runs it on a machine without a webcam:
if (!HasWebcamOnSystem())
return ;
Downloading all Code and Components
Now we have all we need to run the test and have it verify something useful. We just have the hard part left: how do we actually download all the resources we need to run this test? Recall that this is actually a three-way integration test between Chrome, Firefox and AppRTC, which require the following:
The AppEngine SDK in order to bring up the local AppRTC instance,
The AppRTC code itself,
Chrome (already present in the checkout), and
Firefox nightly.
thousands and thousands of machines , and while this test will only run on perhaps 5 at a time due to its webcam requirements, we don’t want manual maintenance work whenever we replace a machine. And for that matter, we definitely don’t want to download a new Firefox by hand every night and put it on the right location on the bots! So how do we automate this?
Downloading the AppEngine SDK authoritative source , but that’s a bad idea for a couple reasons. First, it updates outside our control. Second, there could be anti-robot measures on the page. Third, the download will likely be unreliable and fail the test occasionally.
Google storage bucket under our control and download it using the depot_tools script download_from_google_storage.py Downloading the AppRTC Code Chromium DEPS file (which is Chromium’s dependency provisioning framework).
Downloading Firefox in this script in order to download the Firefox nightly build. Unfortunately this fails every now and then so we would like to have some retry mechanism, or we could write some mechanism to “mirror” the Firefox nightly build in some location we control.
Putting it Together
With that, we have everything we need to deploy the test. You can see the final code here .
.gclient solution ” so that regular Chrome devs and bots are not burdened with downloading hundreds of megs of SDKs and code that they will not use. When this test runs, you will first see a Chrome browser pop up, which will ensure the local apprtc instance is up. Then a Firefox browser will pop up. They will each acquire the fake device and real camera, respectively, and after a short delay the AppRTC call will come up, proving that video interop is working.
Also run on Windows/Mac.
Also test Opera.
Interop between Chrome/Firefox mobile and desktop browsers.
Also ensure audio is playing.
Measure bandwidth stats, video quality, etc.
Hi, thanks for the article!
ReplyDeleteI was wondering if it's somehow possible to emulate fake device disconnection? Emulate case when you have some device that's used and then it gets disconnected.
You mean like a webcam? I jokingly suggested once we should build Lego Mindstorms robots to yank out USB webcams attached to build bots and plug them back in again to test we deal with it correctly in Chrome code. I suspect you mean getting a WebRTC call up and then severing the network connection here, though.
ReplyDeleteYou could write such a test; maybe force the WebRTC call to go over the machine's loopback interface, have the test issue some ipfw/iptables commands halfway through the test, and see what Chrome does. That is very difficult to implement however. A continuous integration machine is always going to be running a bunch of other tests, so tests that modify the global machine state are usually a bad idea. I'd recommend manual testing to establish it works and then write unit tests to exercise the code paths that deal with a lost connection.
Actually my question was about that robots ))
DeleteI need to simulate somehow microphone or webcam plug/unplug for my web app.
For now I'm thinking about OS-level virtual devices...