Google Testing Blog: 2019

Testing on the Toilet: Tests Too DRY? Make Them DAMP!

Tuesday, December 03, 2019

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.
By Derek Snyder and Erik Kuefler
The test below follows the DRY principle (“Don’t Repeat Yourself”), a best practice that encourages code reuse rather than duplication, e.g., by extracting helper methods or by using loops. But is it a well-written test?

def setUp(self):
  self.users = [User('alice'), User('bob')]  # This field can be reused across tests.
  self.forum = Forum()

def testCanRegisterMultipleUsers(self):
  self._RegisterAllUsers()
  for user in self.users:  # Use a for-loop to verify that all users are registered.
    self.assertTrue(self.forum.HasRegisteredUser(user))

def _RegisterAllUsers(self):  # This method can be reused across tests.
  for user in self.users:
    self.forum.Register(user)

While the test body above is concise, the reader needs to do some mental computation to understand it, e.g., by following the flow of self.users from setUp() through _RegisterAllUsers(). Since tests don't have tests, it should be easy for humans to manually inspect them for correctness, even at the expense of greater code duplication. This means that the DRY principle often isn’t a good fit for unit tests, even though it is a best practice for production code.
In tests we can use the DAMP principle (“Descriptive and Meaningful Phrases”), which emphasizes readability over uniqueness

def setUp(self):
  self.forum = Forum()

def testCanRegisterMultipleUsers(self):
  # Create the users in the test instead of relying on users created in setUp.
  user1 = User('alice')
  user2 = User('bob')

  # Register the users in the test instead of in a helper method, and don't use a for-loop.
  self.forum.Register(user1)
  self.forum.Register(user2)

  # Assert each user individually instead of using a for-loop.
  self.assertTrue(self.forum.HasRegisteredUser(user1))
  self.assertTrue(self.forum.HasRegisteredUser(user2))

Note that the DRY principle is still relevant in tests; for example, using a helper function for creating value objects can increase clarity by removing redundant details from the test body. Ideally, test code should be both readable and unique, but sometimes there’s a trade-off. When writing unit tests and faced with a choice between the DRY and DAMP principles, lean more heavily toward DAMP.

11 comments

Code Health: Respectful Reviews == Useful Reviews

Wednesday, November 06, 2019

This is another post in our Code Health series. A version of this post originally appeared in Google bathrooms worldwide as a Google Testing on the Toilet episode. You can download a printer-friendly version to display in your office.
By Liz Kammer (Google), Maggie Hodges (UX research consultant), and Ambar Murillo (Google)
While code review is recognized as a valuable tool for improving the quality of software projects, code review comments that are perceived as being unclear or harsh can have unfavorable consequences: slow reviews, blocked dependent code reviews, negative emotions, or negative perceptions of other contributors or colleagues.
Consider these tips to resolve code review comments respectfully.
As a Reviewer or Author:

DO: Assume competence. An author’s implementation or a reviewer’s recommendation may be due to the other party having different context than you. Start by asking questions to gain understanding.

DO: Provide rationale or context, such as a best practices document, a style guide, or a design document. This can help others understand your decision or provide mentorship.

DO: Consider how comments may be interpreted. Be mindful of the differing ways hyperbole, jokes, and emojis may be perceived.

Author Don’t:
I prefer short names so I’d rather

not change this. Unless you make

me? :)

Author Do:
Best practice suggests omitting

obvious/generic terms. I’m not

sure how to reconcile that

advice with this request.

DON’T: Criticize the person. Instead, discuss the code. Even the perception that a comment is about a person (e.g., due to using “you” or “your”) distracts from the goal of improving the code.

Reviewer Don’t:
Why are you using this approach?

You’re adding unnecessary

complexity.

Reviewer Do:
This concurrency model appears to

be adding complexity to the

system without any visible

performance benefit.

DON’T: Use harsh language. Code review comments with a negative tone are less likely to be useful. For example, prior research found very negative comments were considered useful by authors 57% of the time, while more-neutral comments were useful 79% of the time.

As a Reviewer:

DO: Provide specific and actionable feedback. If you don’t have specific advice, sometimes it’s helpful to ask for clarification on why the author made a decision.
Reviewer Don’t: I don’t understand this.

Reviewer Do: If this is an optimization, can you

please add comments?
DO: Clearly mark nitpicks and optional comments by using prefixes such as ‘Nit’ or ‘Optional’. This allows the author to better gauge the reviewer’s expectations.

As an Author:

DO: Clarify code or reply to the reviewer’s comment in response to feedback. Failing to do so can signal a lack of receptiveness to implementing improvements to the code.
Author Don’t: That makes sense in some cases but

not here.

Author Do: I added a comment about why

it’s implemented that way.
DO: When disagreeing with feedback, explain the advantage of your approach. In cases where you can’t reach consensus, follow Google’s guidance for resolving conflicts in code review.

4 comments

Truth 1.0: Fluent Assertions for Java and Android Tests

Monday, July 08, 2019

(originally posted to the Google Open Source Blog)

assertEquals("Message has been sent", getString(notification, EXTRA_BIG_TEXT));
assertTrue(
    getString(notification, EXTRA_TEXT)
        .contains("Kurt Kluever <kak@google.com>"));

almoststructurewhat's being tested

A better way to structure these assertions is to use a fluent API:

assertThat(getString(notification, EXTRA_BIG_TEXT))
    .isEqualTo("Message has been sent");
assertThat(getString(notification, EXTRA_TEXT))
    .contains("Kurt Kluever <kak@google.com>");

IDE autocompletion can suggest assertions that fit the value under test, including rich operations like containsExactly(permission.SEND_SMS, permission.READ_SMS).

Failure messages can include the value under test and the expected result. Contrast this with the assertTrue call above, which lacks a failure message entirely.

Google's fluent assertion library for Java and Android is Truth. We're happy to announce that we've released Truth 1.0, which stabilizes our API after years of fine-tuning.

Truth started in 2011 as a Googler's personal open source project. Later, it was donated back to Google and cultivated by the Java Core Libraries team, the people who bring you Guava.

You might already be familiar with assertion libraries like Hamcrest and AssertJ, which provide similar features. We've designed Truth to have a simpler API and more readable failure messages. For example, here's a failure message from AssertJ:

java.lang.AssertionError:
Expecting:
  <[year: 2019
month: 7
day: 15
]>
to contain exactly in any order:
  <[year: 2019
month: 6
day: 30
]>
elements not found:
  <[year: 2019
month: 6
day: 30
]>
and elements not expected:
  <[year: 2019
month: 7
day: 15
]>

And here's the equivalent message from Truth:

value of:
    iterable.onlyElement()
expected:
    year: 2019
    month: 6
    day: 30

but was:
    year: 2019
    month: 7
    day: 15

For more details, read our comparison of the libraries, and try Truth for yourself.

Also, if you're developing for Android, try AndroidX Test. It includes Truth extensions that make assertions even easier to write and failure messages even clearer:

assertThat(notification).extras().string(EXTRA_BIG_TEXT)
    .isEqualTo("Message has been sent");
assertThat(notification).extras().string(EXTRA_TEXT)
    .contains("Kurt Kluever <kak@google.com>");

Coming soon: Kotlin users of Truth can look forward to Kotlin-specific enhancements.

No comments

Android Platform Testing Made Easy

Tuesday, January 15, 2019

Android Compatibility Test SuiteTrade Federation Test HarnessSoong build systemTest MappingAtest Configuring tests in Soong buildsSoong build systemAndroid.mksimple build configurationandroid_testAndroid.bp/platform_testing/tests/example/instrumentation/Android.bp

android_test {

name: "HelloWorldTests",

srcs: ["src/**/*.java"],

sdk_version: "current",

static_libs: ["android-support-test"],

certificate: "platform",

test_suites: ["device-tests"],

}

android_testandroid_appComplex test configuration Mapping tests in the source treeTest MappingTEST_MAPPINGtest grouppresubmitpostsubmitTEST_MAPPINGTEST_MAPPING{ "presubmit": [ { "name": "CtsAccessibilityServiceTestCases", "options": [ { "include-annotation": "android.platform.test.annotations.Presubmit" } ] } ], "postsubmit": [ { "name": "CtsWindowManagerDeviceTestCases" } ], "imports": [ { "path": "frameworks/base/services/core/java/com/android/server/am" } ]} Running tests locally with AtestAtestTrade Federation Test Harness
atest [optional-arguments] test-to-run
atest test-to-run-1 test-to-run-2
LOCAL_MODULELOCAL_PACKAGE_NAMEAndroid.mkAndroid.bp
For example:atest FrameworksServicesTests
atest CtsJankDeviceTestCases Discovering tests with Atest and TEST MAPPING

Go to the directory containing the TEST_MAPPING file.

Run the command: atest

All presubmit tests configured in the TEST_MAPPING files of the current directory and its parent directories are run. Atest will locate and run two tests for presubmit. Finding more testing documentationsource.android.com

Android Platform Testing

Test Development Workflow

Simple Test Configuration

Instrumentation Tests

Native Tests

JAR (Java) Host Tests

ci.android.comMore visibility into the Android Open Source ProjectContinuous Integration Dashboard
Google Testing BlogAndroid Developers Blogsource.android.com

2 comments

Testing Blog

Testing on the Toilet: Tests Too DRY? Make Them DAMP!

Code Health: Respectful Reviews == Useful Reviews

Truth 1.0: Fluent Assertions for Java and Android Tests

Android Platform Testing Made Easy

Labels

Archive

Feed