Isn't this style of coding automated GUI tests too grey or even white? As far as I understand, checking for the availability of GUI elements via the code itself prevents the tests from recognizing any delays or lags in the user interface.
In general, could you provide any pointers to what you would consider the best approach to automated GUI tests regarding the level of reliance on code internas? As a simple example, merely to illustrate the question, should one use the actual Strings to check for menu items or constants in the code?
"I'd be interested to hear if someone has done something similar, e.g. for an Ajax application"
Selenium has the waitForCondition() function. You pass in some javascript that will evaluate to true when the part of the page you're interested in is loaded.
Tyburn, a swing testing harness also works in a similar way. It adds an observer to the swing event queue and then listens for one of it's own messages. Once it receives that, it knows that all previous messages have been processed See: http://code.google.com/p/tyburn/
As eliot, the first think I though was in Selenium which enters in a loop checking if the desired condition is true and waiting a second for re-check if it isn't...
The approach of synchronising via the GUI event queue only works if the system under test has no other inputs but the GUI. However, if the system reacts to other events (network messages, for example), then using the event message queue won't work. You don't know how long it will take the system to process the event that will, eventually, cause paint/expose or other GUI events to appear on the client's queue.
Relying solely on the GUI queue is problematic in case there has other threads. Say a GUI operation ended, but a non-GUI thread is still running (calculating something). After some time, the non-GUI thread invokes a GUI operation (e.g. invokeLater() in swing). For a human, it will probably work since we can understands that we are suppose to wait, even if the GUI is not blocked (see excellent post http://googletesting.blogspot.com/2008/10/gui-testing-dont-sleep-without.html) But a testing machine will break.
In our company, we have implemented a mechanism that is indeed waiting for the GUI thread queue to be empty, but every once in a while the application is indeed violating and we had to add some waitFor() synchronizers.
Isn't this style of coding automated GUI tests too grey or even white? As far as I understand, checking for the availability of GUI elements via the code itself prevents the tests from recognizing any delays or lags in the user interface.
RispondiEliminaIn general, could you provide any pointers to what you would consider the best approach to automated GUI tests regarding the level of reliance on code internas? As a simple example, merely to illustrate the question, should one use the actual Strings to check for menu items or constants in the code?
Questo commento è stato eliminato dall'autore.
RispondiElimina"I'd be interested to hear if someone has done something similar, e.g. for an Ajax application"
RispondiEliminaSelenium has the waitForCondition() function. You pass in some javascript that will evaluate to true when the part of the page you're interested in is loaded.
Jemmy, a Java Swing GUI testing library, has a similar method to check if there are pending events in the event queue.
RispondiEliminaSee http://jemmy.netbeans.org/javadoc/org/netbeans/jemmy/QueueTool.html#waitEmpty(long)
Tyburn, a swing testing harness also works in a similar way.
RispondiEliminaIt adds an observer to the swing event queue and then listens for one of it's own messages.
Once it receives that, it knows that all previous messages have been processed
See: http://code.google.com/p/tyburn/
As eliot, the first think I though was in Selenium which enters in a loop checking if the desired condition is true and waiting a second for re-check if it isn't...
RispondiEliminaThe approach of synchronising via the GUI event queue only works if the system under test has no other inputs but the GUI. However, if the system reacts to other events (network messages, for example), then using the event message queue won't work. You don't know how long it will take the system to process the event that will, eventually, cause paint/expose or other GUI events to appear on the client's queue.
RispondiEliminanice post there, you touched upon the internals of how sync works...
RispondiEliminaIn case of automated tools like QTP and Robot we generally don't have to worry about these aspects since they are taken care by the tool itself.
Relying solely on the GUI queue is problematic in case there has other threads. Say a GUI operation ended, but a non-GUI thread is still running (calculating something). After some time, the non-GUI thread invokes a GUI operation (e.g. invokeLater() in swing).
RispondiEliminaFor a human, it will probably work since we can understands that we are suppose to wait, even if the GUI is not blocked (see excellent post http://googletesting.blogspot.com/2008/10/gui-testing-dont-sleep-without.html)
But a testing machine will break.
In our company, we have implemented a mechanism that is indeed waiting for the GUI thread queue to be empty, but every once in a while the application is indeed violating
and we had to add some waitFor() synchronizers.
The previous web address was cut.
RispondiEliminaHere it is in small pieces:
http://
googletesting.blogspot.com/
2008/
10/
gui-testing-dont-sleep-without.html
(it is worth reading)