Google Testing Blog: Mobile

Android UI Automated Testing

Friday, March 20, 2015

by Mona El Mahdy Overview Robolectricgradle unit tests supportEspressotest support librariesStrategy 1: Using an End-To-End Test as a UI Test UI testend-to-end (E2E)

Very large and slow tests.

High flakiness rate due to timeouts and memory issues.

Hard to debug/investigate failures.

Authentication issues (ex: authentication from automated tests is very tricky).

Strategy 2: Hermetic UI Testing using Fake Serversserver hermetic testsblog

Strategy 3: Dependency Injection Design for Apps. Daggerinstrumentation testing

Strategy 4: Building Apps into Smaller Libraries

Conclusion

Don’t write E2E tests instead of UI tests. Instead write unit tests and integration tests beside the UI tests.

Hermetic tests are the way to go.

Use dependency injection while designing your app.

Build your application into small libraries/modules, and test each one in isolation. You can then have a few integration tests to verify integration between components is correct .

Componentized UI tests have proven to be much faster than E2E and 99%+ stable. Fast and stable tests have proven to drastically improve developer productivity.

8 comments

Google

Labels: Mobile , Mona El Mahdy

How the Google+ Team Tests Mobile Apps

Friday, August 30, 2013

by Eduardo Bravo Ortiz General

Understand the platform. Testing on Android is not the same as testing on iOS. The testing tools and frameworks available for each platform are significantly different. (e.g., Android uses Java while iOS uses Objective-C, UI layouts are built differently on each platform, UI testing frameworks also work very differently in both platforms.)

Stabilize your test suite and test environments. Flaky tests are worse than having no tests, because a flaky test pollutes your build health and decreases the credibility of your suite.

Break down testing into manageable pieces. There are too many complex pieces when testing on mobile (e.g., emulator/device state, actions triggered by the OS).

Provide a hermetic test environment for your tests. Mobile UI tests are flaky by nature; don’t add more flakiness to them by having external dependencies.

Unit tests are the backbone of your mobile test strategy. Try to separate the app code logic from the UI as much as possible. This separation will make unit tests more granular and faster.

Android Testing Unit Tests RobolectricHermetic UI Tests hermetic environmentEspresso Diagram: Non-Hermetic Flow vs. Hermetic Flow

Monkey Tests ANRspseudo-randomiOS Testing Unit Tests OCUnitGTMSenTestcaseHermetic UI Tests KIFMonkey Tests Backend Testing Guice Diagram: Normal flow vs Replay Tests flow

Conclusion

Unit tests: These should be your first priority in either Android or iOS. They run fast and are less flaky than any other type of tests.

Backend tests: Make sure your backend doesn’t break your mobile clients. Breakages are very likely to happen when the release cycle of mobile clients and backends are different.

UI tests: These are slower by nature and flaky. They also take more time to write and maintain. Make sure you provide coverage for at least the critical paths of your app.