Google Testing Blog: October 2023

Include Only Relevant Details In Tests

Monday, October 30, 2023

@media only screen and (max-width: 600px) { .body { overflow-x: auto; } } This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.

By Dagang Wei

What problem in the code below makes the test hard to follow?

def test_get_balance(self):

settings = BankSettings(FDIC_INSURED, REGULATED, US_BASED)

account = Account(settings, ID, BALANCE, ADDRESS, NAME, EMAIL, PHONE)

self.assertEqual(account.GetBalance(), BALANCE)

The problem is that there is a lot of noise in the account creation code, which makes it hard to tell which details are relevant to the assert statement.

But going from one extreme to the other can also make the test hard to follow:

def test_get_balance(self):

account = _create_account()

self.assertEqual(account.GetBalance(), BALANCE)

Now the problem is that critical details are hidden in the _create_account() helper function, so it’s not obvious where the BALANCE field comes from. In order to understand the test, you need to switch context by diving into the helper function.

A good test should include only details relevant to the test, while hiding noise:

def test_get_balance():

account = _create_account(BALANCE)

self.assertEqual(account.GetBalance(), BALANCE)

By following this advice, it should be easy to see the flow of data throughout a test. For example:

Bad (flow of data is hidden):

Good (flow of data is clear):

def test_bank_account_overdraw_fails(self):

account = _create_account()

outcome = _overdraw(account)

self._assert_withdraw_failed(

outcome, account)

def _create_account():

settings = BankSettings(...)

return Account(settings, BALANCE, ...)

def _overdraw(account):

# Boilerplate code

...

return account.Withdraw(BALANCE + 1)

def _assert_withdraw_failed(

self, outcome, account):

self.assertEqual(outcome, FAILED)

self.assertEqual(

account.GetBalance(), BALANCE)

def test_bank_account_overdraw_fails(self):

account = _create_account(BALANCE)

outcome = _withdraw(account, BALANCE + 1)

self.assertEqual(outcome, FAILED)

self.assertEqual(

account.GetBalance(), BALANCE)

def _create_account(balance):

settings = BankSettings(...)

return Account(settings, balance, ...)

def _withdraw(account, amount):

# Boilerplate code

...

return account.Withdraw(amount)

No comments

Simplify Your Control Flows

Tuesday, October 24, 2023

@media only screen and (max-width: 600px) { .body { overflow-x: auto; } } This is another post in our Code Health series. A version of this post originally appeared in Google bathrooms worldwide as a Google Testing on the Toilet episode. You can download a printer-friendly version to display in your office.

By Jeff Hoy

When adding loops and conditionals, even simple code can become difficult to understand.
Consider this change:

if (commode.HasPreferredCustomer()) {	if (commode.HasPreferredCustomer()) {
commode.WarmSeat();	commode.WarmSeat();
	} else if (commode.CustomerOnPhone()) {
	commode.ChillSeat();
}	}

While the above change may seem simple, even adding a single else statement can make the code harder to follow since the complexity of code grows quickly with its size. Below we see the code surrounding the above snippet; the control flow on the right illustrates how much a reader needs to retain:

while (commode.StillOccupied()) {

if (commode.HasPreferredCustomer()) {

commode.WarmSeat();

} else if (commode.CustomerOnPhone()) {

commode.ChillSeat();

}

if (commode.ContainsKale()) {

commode.PrintHealthCertificate();

break;

}

Code Control Flow with 5 structures and 9 edges:

challenging for a reader to retain in memory.

In order to fully understand the code, the reader needs to keep the entire control flow in their head. However, the retention capacity of working memory is limited (source) Code path complexity will also challenge the reader, and can be measured using cyclomatic complexity.

To reduce cognitive overhead of complex code, push implementation logic down into functions and methods. For example, if the if/else structure in the above code is moved into an AdjustSeatTemp() method, the reviewer can review the two blocks independently, each having a much simpler control graph:

while (commode.StillOccupied()) {

commode.AdjustSeatTemp();

if (commode.ContainsKale()) {

commode.PrintHealthCertificate();

break;

}

3 control structures and 5 edges: easier to remember

Commode::AdjustSeatTemp()

with 2 structures and 4 edges

Avoiding complexity makes code easier to follow. In addition, code reviewers are more likely to identify logic errors, and maintainers are less likely to introduce complex code.

No comments

Improve Readability With Positive Booleans

Tuesday, October 17, 2023

@media only screen and (max-width: 600px) { .body { overflow-x: auto; } }

This is another post in our Code Health series. A version of this post originally appeared in Google bathrooms worldwide as a Google Testing on the Toilet episode. You can download a printer-friendly version to display in your office.

By Max Kanat-Alexander

Reading healthy code should be as easy as reading a book in your native language. You shouldn’t have to stop and puzzle over what a line of code is doing. One small trick that can assist with this is to make boolean checks about something positive rather than about something negative.

Here’s an extreme example:

if not nodisable_kryponite_shield:

devise_clever_escape_plan()

else:

engage_in_epic_battle()

What does that code do? Sure, you can figure it out, but healthy code is not a puzzle, it’s a simple communication. Let’s look at two principles we can use to simplify this code.

1. Name your flags and variables in such a way that they represent the positive check you wish to make (the presence of something, something being enabled, something being true) rather than the negative check you wish to make (the absence of something, something being disabled, something being false).

if not enable_kryponite_shield:

devise_clever_escape_plan()

else:

engage_in_epic_battle()

That is already easier to read and understand than the first example.

2. If your conditional looks like “if not … else …” then reverse it to put the positive case first.

if enable_kryponite_shield:

engage_in_epic_battle()

else:

devise_clever_escape_plan()

Now the intention of the code is immediately obvious.

There are many other contexts in which this gives improvements to readability. For example, the command foo --disable_feature=False is harder to read and think about than
foo --enable_feature=True, particularly when you change the default to enable the feature.

There are some exceptions (for example, in Python, if foo is not None could be considered a “positive check” even though it has a “not” in it), but in general checking the presence or absence of a positive is simpler for readers to understand than checking the presence or absence of a negative.

No comments

Shell Scripts: Stay Small & Simple

Wednesday, October 11, 2023

@media only screen and (max-width: 600px) { .body { overflow-x: auto; } } A version of this post originally appeared in Google bathrooms worldwide as a Google Testing on the Toilet episode. You can download a printer-friendly version to display in your office.

By David Mandelberg

Shell scripts (including Bash scripts) can be convenient for automating simple command line procedures, and they are often better than keeping complicated commands in a single developer's history. However, shell scripts can be hard to understand and maintain, and are typically not as well-supported as other programming languages. Shell scripts have less support for unit testing, and there is likely a lower chance that somebody reading one will be experienced with the language.

Python, Go, or other general-purpose languages are often better choices than shell. Shell is convenient for some simple use cases, and the Google shell style guide can help with writing better shell scripts. But it is difficult, even for experienced shell scripters, to mitigate the risks of its many surprising behaviors. So whenever possible, use shell scripts only for small, simple use cases, or avoid shell entirely.

Here are some examples of mistakes that are far too easy to make when writing a shell script (see Bash Pitfalls for many more):

Forgetting to quote something can have surprising results, due to shell's complicated evaluation rules. E.g., even if a wildcard is properly quoted, it can still be unexpectedly expanded elsewhere:

$ msg='Is using bash a pro? Or a con?'

$ echo $msg # Note that there's a subdirectory called 'proc' in the current directory.

Is using bash a proc Or a con? # ? was unexpectedly treated as a wildcard.

Many things that would be function arguments in other languages are command line arguments in shell. Command line arguments are world-readable, so they can leak secrets:

$ curl -H "Authorization: Bearer ${SECRET}" "$URL" &

$ ps aux # The current list of processes shows the secret.

By default, the shell ignores all errors from commands, which can cause severe bugs if code assumes that earlier commands succeeded. The command set -e can appear to force termination at the first error, but its behavior is inconsistent. For example, set -e does not affect some commands in pipelines (like false in false | cat), nor will it affect some command substitutions (such as the false in export FOO="$(false)"). Even worse, its behavior inside a function depends on how that function is called.

Many things run in subshells, which can (often unexpectedly) hide changes to variables from the main shell. It can also make manual error handling harder, compounding the issue above:

$ run_or_exit() { "$@" || exit $?; } # Executes the arguments then exits on failure.

$ foo="$(run_or_exit false)" # Exits the $() subshell, but the script continues.

No comments

Testing Blog

Include Only Relevant Details In Tests

Simplify Your Control Flows

Improve Readability With Positive Booleans

Shell Scripts: Stay Small & Simple

Labels

Archive

Feed