by Mona El Mahdy
Overview
Robolectric and gradle unit tests support are great examples of unit test frameworks for Android. UI tests, on the other hand, are used to verify that your application returns the correct UI output in response to a sequence of user actions on a device. Espresso is a great framework for running UI actions and verifications in the same process. For more details on the Espresso and UI Automator tools, please see: test support libraries .
Strategy 1: Using an End-To-End Test as a UI Test
UI test ensures that your application returns the correct UI output in response to a sequence of user actions on a device. An end-to-end (E2E) test brings up the full system of your app including all backend servers and client app. E2E tests will guarantee that data is sent to the client app and that the entire system functions correctly.
Very large and slow tests.
High flakiness rate due to timeouts and memory issues.
Hard to debug/investigate failures.
Authentication issues (ex: authentication from automated tests is very tricky).
Strategy 2: Hermetic UI Testing using Fake Servers server hermetic tests . This way you split your E2E test into a server side test, a client side test, and an integration test to verify that the server and client are in sync (for more details on integration tests, see the backend testing section of blog ).
Strategy 3: Dependency Injection Design for Apps.
Dagger , or you can create your own dependency injection mechanism if needed.
instrumentation testing , the test apk and the app under test are loaded in the same process, so the test code has runtime access to the app code. Not only that, but you can also use classpath override (the fact that test classpath takes priority over app under test) to override a certain class and inject test fakes there. For example, To make your test hermetic, your app should support injection of the networking implementation. During testing, the test injects a fake networking implementation to your app, and this fake implementation will provide seeded data instead of communicating with backend servers.Strategy 4: Building Apps into Smaller Libraries
Conclusion
Don’t write E2E tests instead of UI tests. Instead write unit tests and integration tests beside the UI tests.
Hermetic tests are the way to go.
Use dependency injection while designing your app.
Build your application into small libraries/modules, and test each one in isolation. You can then have a few integration tests to verify integration between components is correct .
Componentized UI tests have proven to be much faster than E2E and 99%+ stable. Fast and stable tests have proven to drastically improve developer productivity.
Does the integration test mentioned in the article mean the test between many small libraries in the app?
ReplyDeleteHey Allen,
DeleteYou need 2 types of integration tests.
1) Integration test between client and server, for more details on integration tests, see the backend testing section of blog
2) As you mentioned, integration tests between small libraries in the app.
Hi Mona,
ReplyDeletethis is a great article! Thanks for writing it.
Have a great day.
Greets,
Daniel
Thanks Daniel!
DeleteGood article. I wonder whether this hermetic approach to client & server side testing would work for web applications as well, testing widgets & components of a site with the rest of the site and server back end mocked out. I've not come across it for web apps, so am curious on applying it there. Does Google+ team use this same hermetic client/server Android app testing approach to the Google+ website & Google+ for iOS as well?
ReplyDeleteThank you!
ReplyDeleteThis hermetic approach applies to IOS and web testing. You just need to use different tools for UI testing, data injection, etc.
Is hermetic testing similar to the idea of using stubs for services called by the app?
ReplyDeleteHi Mona,
ReplyDeletethis is a great article! Thanks for writing it.
Have a great day. you can check our blog which is based on mobile apps testing