Release 0.9.75 is now available to download and includes the following improvements:
Safari webdriver is now supported: just set the webdriver.driver property to “safari”. Make sure you follow the instructions here to install the Selenium SafariDriver extension first.
This release also brings the awesomeness of FluentLenium‘s fluent API to Thucydides. The best way to use FluentLenium within Thucydides is to use ThucydidesFluentAdapter which is available in PageObject. Here’s an example of the same PageObject written in the traditional style and with FluentLenium.
Traditional style:
import ch.lambdaj.function.convert.Converter;
import net.thucydides.core.annotations.DefaultUrl;
import org.openqa.selenium.By;
import org.openqa.selenium.Keys;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
import net.thucydides.core.pages.PageObject;
import java.util.List;
import static ch.lambdaj.Lambda.convert;
@DefaultUrl("http://en.wiktionary.org/wiki/Wiktionary:Main_Page")
public class DictionaryPage extends PageObject {
@FindBy(name="search")
private WebElement searchTerms;
@FindBy(name="go")
private WebElement lookupButton;
public DictionaryPage(WebDriver driver) {
super(driver);
}
public void enter_keywords(String keyword) {
element(searchTerms).type(keyword);
}
public void lookup_terms() {
element(lookupButton).click();
}
public List getDefinitions() {
WebElement definitionList = getDriver().findElement(By.tagName("ol"));
List results = definitionList.findElements(By.tagName("li"));
return convert(results, toStrings());
}
private Converter<WebElement, String> toStrings() {
return new Converter<WebElement, String>() {
public String convert(WebElement from) {
return from.getText();
}
};
}
}
and with FluentLenium…
import ch.lambdaj.function.convert.Converter;
import net.thucydides.core.annotations.DefaultUrl;
import net.thucydides.core.pages.PageObject;
import org.fluentlenium.core.domain.FluentList;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
import java.util.List;
import static ch.lambdaj.Lambda.convert;
import static org.fluentlenium.core.filter.FilterConstructor.withName;
@DefaultUrl("http://en.wiktionary.org/wiki/Wiktionary:Main_Page")
public class FluentDictionaryPage extends PageObject {
public FluentDictionaryPage(WebDriver driver) {
super(driver);
}
public void enter_keywords(String keyword) {
fluent().fill("input", withName("search")).with(keyword);
}
public void lookup_terms() {
fluent().click("input", withName("go"));
}
public List getDefinitions() {
FluentList results = fluent().findFirst("ol").find("li");
return results.getTexts();
}
}
A special shout-out to MathildeLemee for helping with the integration.
Another new experimental feature introduces the ability to replace the commonly-used element() method with ‘$’, as illustrated in the following examples:
...
@FindBy(name="search")
private WebElement searchTerms;
@FindBy(name="go")
private WebElement lookupButton;
public DictionaryPage(WebDriver driver) {
super(driver);
}
public void enter_keywords(String keyword) {
$(searchTerms).type(keyword);
}
public void lookup_terms() {
$(lookupButton).click();
}
public void click_on_article(int articleNumber) {
$("//section[@id='searchResults']/article[" + articleNumber + "]//a").click();
}
public String getHeading() {
return $("section>h1").getText()
}
}
PageObject.containsText() now returns true only if the text is visible on the page. Earlier this method returned true even if the text was not visible (for example, if CSS property display:none was set)As usual, we encourage you to get in touch via this blog, our mailing lists or Jira tracker for any queries or comments.
We have released another version of Thucydides today. Release 0.9.77 contains the following features and bug fixes:
Whenever a step is executed, Thucydides saves a screenshot which is displayed in the report. In order to manage disk space better, some users require finer control on when and how many screenshots are generated and stored. Thucydides provided some support for this in the past by allowing users to set the properties thucydides.only.save.failing.screenshots for saving screenshots for only failed steps and thucydides.verbose.screenshots for saving screenshots at every web element action (like click(), typeAndEnter(), type(), typeAndTab() etc.).
In the latest release, these properties are deprecated and replaced with a new property thucydides.take.screenshots which can have the following values:
thucydides.verbose.screenshotsthucydides.only.save.failing.screenshotsAn even more granular level of control is now possible using annotations. You can annotate any test or step method (or any method used by a step or test) with the @Screenshots annotation to override the number of screenshots taken within this step (or sub-step). Some sample uses are shown here:
@Step @Screenshots(onlyOnFailures=true) public void screenshots_will_only_be_taken_for_failures_from_here_on() {…} @Test @Screenshots(forEachStep=true) public void should_take_screenshots_for_each_step_in_this_test() {…} @Test @Screenshots(forEachAction=true) public void should_take_screenshots_for_each_action_in_this_test() {…}
Our latest release (0.9.90) introduces a more user friendly report format for data-driven (example-driven) jBehave tests.
The following screenshots will explain the changes in detail.
All examples are displayed in a table on top.
The top level step shows example data for easy reference.
Error Steps are highlighted in red for easy identification.
Currently, this feature is available only for jBehave tests but will soon be integrated with jUnit and easyb tests as well.
Basic support for iPhone and Android web drivers is now available in Thucydides. Set the property webdriver.driver to ‘iPhone’ or ‘Android’ when running your tests.
Release 0.9.92 is now available and contains the following enhancements:
The Requirements reporting tab has been enhanced to show test results at a requirements level. This provides an overall view of the progress of the project by making it easier to view not only what stories have been implemented, but also features and capabilities that remain to be done. Here’s a screenshot.
Requirements Reporting tab
You can read more about Thucydides’ approach to requirements reporting in this excellent blog post – Functional Test Coverage – Taking BDD Reporting To The Next Level
For security/privacy reasons, it may be required to blur sensitive screenshots in Thucydides reports. This is now possible by annotating the test methods or steps with the annotation @BlurScreenshots. When defined on a test, all screenshots for that test will be blurred. When defined on a step, only the screenshot for that step will be blurred. @BlurredScreenshot takes a string parameter with values LIGHT, MEDIUM or HEAVY to indicate the amount of blurring. For example,
@Test
@BlurScreenshots("HEAVY")
public void looking_up_the_definition_of_pineapple_should_display_the_corresponding_article() {
endUser.is_the_home_page();
endUser.looks_for("pineapple");
endUser.should_see_definition_containing_words("A thorny fruit");
}
This feature is currently available for junit stories but we’ll extend it shortly to jbehave and easyb.
Keep watching this space for more updates and announcements.
Conventional test reports, generated by tools such as JUnit or TestNG, naturally focus on what tests have been executed, and whether they passed or failed. While this is certainly useful from a testing perspective, these reports are far from telling the whole picture.
BDD reporting tools like Cucumber and JBehave take things a step further, introducing the concept of “pending” tests. A pending test is one that has been specified (for example, as an acceptance criteria for a user story), but which has not been implemented yet.
In BDD, we describe the expected behaviour of our application using concrete examples, that eventually form the basis of the “acceptance criteria” for the user stories we are implementing. BDD tools such as Cucumber and JBehave not only report on test results: they also report on the user stories that these tests validate.
However this reporting is still limited for large projects, where the numbers of user stories can become unwieldy. User stories are not created in isolation: rather, user stories help describe features, which support capabilities that need to be implemented to achieve the business goals of the application. So it makes sense to be able to report on test results not only at the user story level, but also at higher levels, for example in terms of features and capabilities. This makes it easier to report on not only what stories have been implemented, but also what features and capabilities remain to be done. An example of such a report is shown in Figure 1 (or see the full report here).
Figure 1: A test coverage report listing both tested and untested requirements.
In agile projects, it is generally considered that a user story is not complete until all of its automated acceptance tests pass. Similarly, a feature cannot be considered ready to deliver until all of the acceptance criteria for the underlying user stories have been specified and implemented. However, sensible teams shy away from trying to define all of the acceptance criteria up-front, leaving this until the “last responsible moment”, often shortly before the user story is scheduled to be implemented. For this reason, reports that relate project progress and status only in terms of test results are missing out on the big picture.
To get a more accurate idea of what features have been delivered, which ones are in progress, and what work remains to be done, we must think not in terms of test results, but in terms of the requirements as we currently understand them, matching the currently implemented tests to these requirements, but also pointing out what requirements currently have no acceptance criteria defined. And when graphs and reports illustrate how much progress has been made, the requirements with no acceptance criteria must also be part of the picture.
Thucydides puts some of these concepts into practice. Thucydides reports not only on how the tests did, but also fits them into the broader picture, showing what requirements have been tested and, just as importantly, what requirements haven’t.
During the rest of this article, we will see how to report on both your requirements and your test results using a very simple directory-based approach. You can follow along with this example by cloning the Github project at https://github.com/thucydides-webtests/thucydides-simple-demo
Thucydides can integrate with many different requirement management systems, and it is easy to write your own plugin to tailor the integration to suite your particular environment. A popular approach, for example, is to store requirements in JIRA and to use Thucydides to read the requirements hierarcy directly from the JIRA cards. However the simplest approach, which uses a directory-based approach, is probably the easiest to use to get started, and it is that approach that we will be looking at here.
Requirements can usually be organized in a hierarchial structure. By default, Thucydides uses a three-level hierarchy of requirements. At the top level, capabilities represent a high-level capacity that the application must provide to meet the application’s business goals. At the next level down, features help deliver these capabilities. To make implementation easier, a feature can be broken up into user stories, each of which in turn can contain a number of acceptance criteria.
Figure 2: JUnit test directories mirror the requirements hierarchy.
Of course, you don’t have to use this structure if it doesn’t suit you. You can override the thucydides.capability.types system property to provide your own hierarchy. For example, if you wanted a hierarchy with modules,epics, and features, you would just set thucydides.capability.types to “module,epic,feature”.
When we use the default directory-based requirements strategy in Thucydides, the requirements are stored in a hierarchial directory structure that matches the requirements hierarchy. At the lowest level, a user story is represented by a JBehave *.story file, an easyb story, or a JUnit test. All of the other requirements are represented as directories (see Figure 2 for an example of such a structure).
In each requirements directory, you can optionally place a file called narrative.txt, which contains a free-text summary of the requirement. This will appear in the reports, with the first line appearing as the requirement title. A typical narrative text is illustrated in the following example:
Learn the meaning of a word In order to learn the meaning of a word that I don't know As an online reader I want to be able to find out the meaning of the word
If you are implementing the acceptance criteria as JUnit tests, just place the JUnit tests in the package that matches the correspoinding requirement. You need to use the thucydides.test.root system property to specify the root package of your requirements. For the example in Figure 2, this value should be set to nz.govt.nzqa.lssu.stories.
Figure 3: The narrative.txt file appears in the reports to describe a requirement.
If you are using JBehave, just place the *.story files in the src/test/resources/storiesdirectory, again respecting a directory structure that corresponds to your requirement hierarchy. Thenarrative.txt files also work for JBehave requirements.
Progress is measured by the total number of passing, failing or pending acceptance criteria, either for the whole project (at the top level), or within a particular requirement as you drill down the requirements hierarchy. For the purposes of reporting, a requirement with no acceptance criteria is attributed an arbitrary number of “imaginary” pending acceptance criteria. Thucydides uses 4 per requirement by default, but you can override this value using the thucydides.estimated.tests.per.requirement system property.
Figure 3: For JBehave, everything goes under src/test/resources/stories.
BDD is an excellent approach for communicating with, and reporting back to, stakeholders. However, for accurate acceptance test reporting on real-world projects, you need to go beyond the story level, and cater for the whole requirements hierarchy. In particular, you need to not only report on tests that have been executed, but also allow for the tests that haven’t been written yet.
Thucydides puts these concepts into practice: using a simple directory-based convention, you can easily integrate your requirements hierarcy into your acceptance tests.
We released version 0.9.94 of Thucydides over the weekend.
Reports home page now shows two pie charts, one with the overall tests count and another with the test count weighted by number of steps.
The first pie chart gives an overall status of the tests as it shows how many tests have passed, failed or are still pending. The second pie-chart gives the test status weighted by number of steps which could be used to deduce the nature/criticality of test failures.
There are also some minor improvements to the requirements reporting page.
Selenium version has been updated to 2.29.0 (Selenium Release notes)
thucydides.verbose.steps property, that provides more detailed logging of WebElementFacade steps.Our latest release (0.9.95) brings the example reporting format changes (released last month for jBehave) to parametric junit tests. A few cosmetic changes have also been done for better organization of information and improved readability.
Let’s look at these changes in detail.
Tests table is reorganized to remove clutter.
As the screenshot above demonstrates, the test table on main report page reduces clutter by showing test results at a test-method level instead of a separate row for each parameter set.
Test details page is better organized for readability.
The details page will list all parameter sets in a table. By default, the parameters are names “Parameter 1″, “Parameter 2″ and so on. But this can be configured with a comma separated list of column headings passed as an attribute to the @TestData annotation.
@TestData(columnNames = "Word,Definition")
public static Collection<Object[]> testData() {
return Arrays.asList(new Object[][]{
{"apple", "A common, round fruit"},
{"banana", "An elongated curved fruit"}
});
}
Further, the steps section will show a collapsed view with a row for each parameter set. The 
icon can be clicked to expand the top level step to reveal all the steps and corresponding screenshots. Failed test examples will be highlighted in red.
A new property firefox.preferences can be used to supply a semicolon separated list of Firefox configuration settings. For ex.,
-Dfirefox.preferences="browser.download.folderList=2;browser.download.manager.showWhenStarting=false;browser.download.dir=c:\downloads"
Integer and boolean values will be converted to the corresponding types in the Firefox preferences; all other values will be treated as Strings. You can set a boolean value to true by simply specifying the property name, e.g. -Dfirefox.preferences=app.update.silent.
For the lazy, here’s a complete reference to Firefox’s configuration settings – http://kb.mozillazine.org/Firefox_:_FAQs_:_About:config_Entries
Thucydides makes it possible to split the web tests into smaller batches, and to run each batch on a different machine or different virtual display. This is achieved by specifying two parameters when you run each build: the total number of batches being run (thucydides.batch.count), and the number of the batch being run in this build (thucydides.batch.number).
For example, the following will divide the test cases into 3 batches (thucydides.batch.count), and only run the first test in each batch (thucydides.batch.number):
mvn verify -Dthucydides.batch.count=3 -Dthucydides.batch.number=1
However, this strategy divides the test cases equally among batches. This could be inefficient if some of your test cases have more tests than others. This release adds a new batching strategy based on number of tests in the test cases. The strategy is implemented with the help of a TestCountBasedBatchManager which evenly distributes test cases across batches based on number of test methods in each test case.
Thucydides Release 0.9.98 is now out.
Adding on to some recent enhancements to screenshots management, Thucydides now supports a method to give you even finer control on capturing screenshots in your tests. Using the newly added takeScreenshot method, you can now instruct Thucydides to take a screenshot at any arbitrary point in the step irrespective of the screenshot level set earlier.
Simply call Thucydides.takeScreenshot() in your step methods whenever you want a screenshot to be captured.
Thucydides now saves only rescaled screenshots by default. This is done to help reduce the disk space taken by reports. Should you require to save the original unscaled screenshots too, this default can be easily overridden by setting the property, thucydides.keep.unscaled.screenshots to true.
From this release onwards, html source files for the screenshots will also not be saved by default. Set the property, thucydides.store.html.source to true if you wish to override this behaviour.
This is not really a new feature but rather a hidden feature that one of our users, Alex Artukh, has been experimenting with. You can pass valid HTML text as parameter to @Step methods in the step library. This will show up as formatted text in the reports on the step details page. The following screenshot demonstrates this.
HTML formatted text, if passed to a step method will be displayed as shown. This can be useful for annotating or documenting the tests with helpful information.
This was achieved by creating a dummy @Step method called description that takes a String parameter. At runtime, the tests supply this method with formatted html text as parameter.
@Step
public void description(String html) {
//do nothing
}
public void about(String description, String...remarks) {
String html =
"<h2 style=\"font-style:italic;color:black\">" + description + "</h2>" +
"<div><p>Remarks:</p>" +
"<ul style=\"margin-left:5%; font-weight:200; color:#434343; font-size:10px;\">";
for (String li : remarks) html += "<li>" + li + "</li>";
html += "<ul></div>";
description(html);
}
Read here for more details.
Incidentally, this feature was broken in the last release due to some regression but has been fixed in the latest release.
getAttribute() method to return the value of a given attribute.thucydides.batch.strategy, has been added to configure the batch strategy. By default this property is set to the value DIVIDE_EQUALLY which divides the test cases equally among batches. Set this property to DIVIDE_BY_TEST_COUNT in order to distribute test cases across batches based on number of test methods in each test case.Keep watching this space for more.
Thucydides Release 0.9.103 adds some useful new features and enhancements.
Test reports now distinguish between test errors and failures. The following screenshot will make this clear.
Reports distinguish between Test Errors and Failures
Tests that do not have any screenshots will no longer show an empty screenshots column in the details page.
Instead of declaring WebElement variables in Page Objects and then calling element() or $() to wrap them in WebElementFacades, you can now declare WebElementFacade variables directly inside the Page Objects. This will make the Page Object code simpler more readable.
So, instead of writing
@DefaultUrl("http://en.wiktionary.org/wiki/Wiktionary:Main_Page")
public class DictionaryPage extends PageObject {
@FindBy(name="search")
private WebElement searchTerms;
private WebElement go; //variable name matches element id or name
public DictionaryPage(WebDriver driver) {
super(driver);
}
public void enter_keywords(String keyword) {
element(searchTerms).type(keyword);
}
public void lookup_terms() {
element(go).click();
}
public List getDefinitions() {
WebElement definitionList = getDriver().findElement(By.tagName("ol"));
List<WebElement> results = definitionList.findElements(By.tagName("li"));
return convert(results, toStrings());
}
private Converter<WebElement, String> toStrings() {
return new Converter<WebElement, String>() {
public String convert(WebElement from) {
return from.getText();
}
};
}
}
you can write
@DefaultUrl("http://en.wiktionary.org/wiki/Wiktionary:Main_Page")
public class DictionaryPage extends PageObject {
@FindBy(name="search")
private WebElementFacade searchTerms;
private WebElementFacade go; //variable name matches element id or name
public DictionaryPage(WebDriver driver) {
super(driver);
}
public void enter_keywords(String keyword) {
searchTerms.type(keyword); //directly use facade
}
public void lookup_terms() {
go.click(); //directly use facade
}
public List getDefinitions() {
WebElement definitionList = getDriver().findElement(By.tagName("ol"));
List<WebElement> results = definitionList.findElements(By.tagName("li"));
return convert(results, toStrings());
}
private Converter<WebElement, String> toStrings() {
return new Converter WebElement, String>() {
public String convert(WebElement from) {
return from.getText();
}
};
}
}
A new method, switchToPage() has been added to PageObject class to make it convenient to return a new PageObject after navigation from within a method of a PageObject class. For example,
@DefaultUrl("http://mail.acme.com/login.html")
public class EmailLoginPage extends PageObject {
...
public void forgotPassword() {
...
forgotPassword.click();
ForgotPasswordPage forgotPasswordPage = this.switchToPage(ForgotPasswordPage.class);
forgotPasswordPage.open();
...
}
...
}
containsOnlyText and shouldContainOnlyText methods to WebElementFacade. These methods are similar to containsText/shouldContainText methods but check for exact match.shouldContainText method in WebElementFacade so that error message shows both the expected text, and the text found in the element.WebElementFacade.shouldNotContainText.thucydides.native.events property to true or false.
