Migration to Arquillian - done

Or how the RichFaces functional tests suites were migrated to Arquillian framework.


Table of contents

1. Migration motivation
2. Arquillian Ajocado project set up
3. Writing tests
4. RichFaces Selenium vs. Ajocado API

Migration motivation

Initial reason for migrating was a problem with Maven cargo plugin and its support of JBoss AS 7. In a short time, we also realized how many additional advantages Arquillian would bring into our project. My task was to prove this concept by porting functional tests of RichFaces showcase app to the Arquillian framework.

Our former functional test suite was written as Selenium tests, more precisely we used our homemade framework (RichfFaces Selenium)
in top of Selenium 1, from which Arquillian Ajocado was born. You can read more about RichFaces Selenium on its author blog.

So the benefits of the new platform - Arquillian + Arquillian Ajocado - were pretty obvious:

• support for various containers (JBoss AS 6.0, JBoss AS 7.0, Tomcat 7 and many others, see this for more)
• some of them are managed by Arquillian, so starting, deploying etc. is done automatically, therefore they are suitable for CI tools like Jenkins for example
• Drone extension brings features of type safe Selenium 1.0 API by providing Ajocado, and also comes with Selenium 2.0 support and it's WebDriver API
• tests rapid development with Ajocado
• Ajocado best feature is not only the type safe API, but also it fills in Selenium gaps with very useful tools for testing Ajax requests, by it's waitAjax and guardXHR methods, which are so essential in AJAX frameworks like RichFaces
• Arquillian future support of mobile devices testing, and current WebDriver support of mobile devices testing with it's Android and iOS plugins.
• last but not least Arquillian is an opensource project with quite big community, it is quickly evolving, and as it is with opensource, when you do not have the feature, you can either easily develop it with support of community (which I found out for myself when I was developing Tomcat managed container for Arquillian) or you can file a feature request.

The only drawback, which we were aware of, was the API incompatibilities between RichFaces Selenium and Ajocado. I will return to them in the end.

Arquillian Ajocado project set up

The best way to set up Arquillian project is described in the documentation. As recommended it is good to configure it as Maven project. The first recommendation fulfilled from RichFaces side. 

In short, two configuration files need to be written or altered. Here are the examples of such from migrated RichFaces projects, pom.xml and arquillian.xml.

As you can read in docs, the only thing you need to add to your pom.xml is Arquillian dependencies and some profiles, which represents desired containers on which will be the testing application deployed. There is also an option to run tests from these containers, but in our project it is enough to run them on client.

An example of such dependencies are:

<dependency>
      <groupId>org.jboss.arquillian.ajocado</groupId>
      <artifactId>arquillian-ajocado-testng</artifactId>
      <version>1.0.0-SNAPSHOT</version>
      <type>pom</type>
</dependency>
<dependency>
      <groupId>org.jboss.arquillian.extension</groupId>
      <artifactId>arquillian-drone-webdriver</artifactId>
      <version>1.0.0.CR3</version>
</dependency>

With this, you bring to your project all required Ajocado dependencies, and also WebDriver object. Of course you have other options, like use instead of TestNG the JUnit test framework. For complete set up, again please see the corresponding docs.

Next xml snippet is required Maven profile, which represents container into which our application under test will be deployed. This is an example of JBoss AS 7.1.0.CR1b Arquillian container.
Note that 7.1.0.Final are going to be released soon (7 February 2012), and than it will not take much time to release also arquillian managed dependency. So in order to use newer versions of container, please checkout available maven dependencies (JBoss Nexus) or JBoss AS download page and change accordingly.

<profile>
   <id>jbossas-managed-7-1</id>
   <properties>
   </properties>
   <dependencies>
     <dependency>
       <groupId>org.jboss.as</groupId>
       <artifactId>jboss-as-arquillian-container-managed</artifactId>
       <version>7.1.0.CR1b</version>
       <scope>test</scope>
     </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-dependency-plugin</artifactId>
         <executions>
            <execution>
            <id>unpack</id>
            <phase>process-test-classes</phase>
            <goals>
              <goal>unpack</goal>
            </goals>
         <configuration>
            <artifactItems>
              <artifactItem>
              <groupId>org.jboss.as</groupId>
              <artifactId>jboss-as-dist</artifactId>
              <version>7.1.0.CR1b</version>
              <type>zip</type>
              <overWrite>false</overWrite>
              <outputDirectory>${project.build.directory}</outputDirectory>
             </artifactItem>
            </artifactItems>
         </configuration>
        </execution>
      </executions>
   </plugin>
   <plugin>
     <artifactId>maven-surefire-plugin</artifactId>
     <version>2.9</version>
     <configuration>
       <systemProperties>
         <arquillian.launch>jbossas-managed-7-1</arquillian.launch>
       </systemProperties>
       <environmentVariables>
         <JBOSS_HOME>${project.build.directory}/jboss-as-7.1.0.CR1b</JBOSS_HOME>
       </environmentVariables>
     </configuration>
  </plugin>
 </plugins>
</build>
</profile>

When this profile is executed in standard way, mvn -Pjbossas-managed-7-1, the distribution of JBoss AS is downloaded from Maven repo, and unziped to the target directory.

With help of surefire plugin, we then set system property arquillian.launch, which fires the right configuration from arquillian.xml. Indeed you can achieve this by -Darquillian.launch=[correspondingArquillianXMLQualifier]. And lastly, we set up JBOSS_HOME environmental variable,to say to Arquillian where our managed container is installed. Same thing can be achieved by setting up correctly jbossHome property in arquillian.xml.

The last required config file is arquillian.xml, placed on the classpath, so ideal place for it is src/test/resource. Example which set ups config for above mentioned JBoss AS, config for Ajocado, Selenium server and WebDriver would look like:

<arquillian xmlns="http://jboss.com/arquillian" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://jboss.org/schema/arquillian http://jboss.org/schema/arquillian/arquillian_1_0.xsd">

  <engine>
    <property name="maxTestClassesBeforeRestart">10</property>
  </engine>

  <container qualifier="jbossas-managed-7-1">
    <configuration>
      <property name="javaVmArguments">-Xms1024m -Xmx1024m -XX:MaxPermSize=512m</property>
      <property name="serverConfig">standalone-full.xml</property>
    </configuration>
    <protocol type="jmx-as7">
      <property name="executionType">REMOTE</property>
    </protocol>
  </container>

  <extension qualifier="selenium-server">
    <property name="browserSessionReuse">true</property>
    <property name="port">8444</property>
  </extension>

  <extension qualifier="ajocado">
    <property name="browser">*firefox</property>
    <property name="contextRoot">http://localhost:8080/</property>
    <property name="seleniumTimeoutAjax">7000</property>
    <property name="seleniumMaximize">true</property>
    <property name="seleniumPort">8444</property>
    <property name="seleniumHost">localhost</property>
  </extension>

  <extension qualifier="webdriver"> 
    <property name="implementationClass">org.openqa.selenium.firefox.FirefoxDriver</property>
  </extension> 

</arquillian>

In this file we defined that the maximum number of tests classes which will be executed is 10. Then the container will be restarted. This is a workaround for famous OutOfMemory exception: PERM_GEN thrown after multiple deployments on containers. The number 10 was chosen by multiple running the suite and it seems for now that this is the optimal number of tests, but you should test your test suite and you should choose your number. See also the MaxPermsSize, set for all containers for a quite big chunk of memory, this is also due to above mentioned error.

Next configuration is for JBoss AS managed container, there is JBOSS_HOME omitted since we are setting it up in pom.xml. Then there are additional JVM arguments, mainly increasing the permanent memory size and heap size. These setups seems to be the best effective, we can manage to run 10 test classes and run it quickly. 

Configuration for selenium server consists from property browserSessionReuse, which determines whether the same session of browser should be use, in other words whether there will be start of new browser after each test. The true value accelerates tests quite dramatically. Selenium server need to be run for Ajocado tests, for WebDriver not. For further configuration options, please see the docs.

There is also a need to alter your Java code, to run with Arquillian. If you will be following docs, you will be successful for sure. I am just providing our approach. 

We have one base class which is common for whole test suite. It contains method for deploying application under test. We are deploying whole application for all test classes, which will be probably replaced by deploying only what is needed, with help of ShrinkWrap project. This improvement should accelerate the testing.

Since we want to write both Ajocado and WebDriver tests we are providing two classes which particular tests will extends. It is not possible now to have Ajocado and WebDriver objects simultaneously accessible from the same test class.

Writing tests

Writing tests with Ajocado and Arquillian is really simple and fast. It is so because of Ajocado is targeting on rapid development with his OO API as much as possible. With these features and modern IDE code completing, it is more pleasure than struggle to write tests. This is an example of such test, lets examine it further.

So, as I mentioned above, to run successfully test, there should be:

Method for deploying application under test:

@Deployment(testable = false)

public static WebArchive createTestArchive() {

        WebArchive war = ShrinkWrap.createFromZipFile(WebArchive.class, new    File("target/showcase.war"));

        return war;
 }

We are deploying a war, which was copied into project build directory with help of Maven dependency plugin. This method is located in the parent of test, as it is the same for all tests. The argument in the annotations stands for running tests on the client rather than on server side.

Method for loading correct page on the browser:

@BeforeMethod(groups = { "arquillian" })
public void loadPage() {

  String addition = getAdditionToContextRoot();
  this.contextRoot = getContextRoot();

  selenium.open(URLUtils.buildUrl(contextRoot, "/showcase/", addition));
}

We have set our tests that it is loading the correct page according to the test class name, so the method getAdditionToContextRoot is dealing with it, the context root can be set in arquillian.xml and finally you just load that page as you are used to with Selenium 1, but again instead of String you are using higher object. Just note that if you are using testng.xml for including some test groups, you need to add you before, after ... methods to the group arquillian, and also to include this group in particular testng.xml

protected JQueryLocator commandButton = jq("input[type=submit]");
protected JQueryLocator input = jq("input[type=text]");
protected JQueryLocator outHello = jq("#out");

@Test
public void testTypeSomeCharactersAndClickOnTheButton() {

  /*
  * type a string and click on the button, check the outHello
  */
  String testString = "Test string";

  //write something to the input

  selenium.typeKeys(input, testString);

  //check whether after click an AJAX request was fired
  guardXhr(selenium).click(commandButton);

  String expectedOutput = "Hello " + testString + " !";
  assertEquals(selenium.getText(outHello), expectedOutput, "The output should be:   " + expectedOutput);
}

I think the test is pretty much self explanatory. Here you can also see the differences between Selenium 1 and Ajocado. Aim mainly your focus on the fact that Ajocado uses various objects instead of just String for everything. In this example it is JQyeryLocator, which provide convenient and fast way for locating page elements by JQuery selectors.

RichFaces Selenium vs. Ajocado API

The API differences where one of the last problems we had. As we were using RichFaces Selenium, which has very similar API to Ajocado, the migration could be done automatically. However, at first I had to migrate the whole suite manually to see exactly the differences. With the list of API changes I was able to develop small Java app to automate this migrating in future. It was created mainly for our purposes, as there were new tests to migrate each day, but you can accommodate that app for your purposes too. It is nothing big, it can be probably easily done in bash, but I am quite weak in bash scripting, so I did in Java.

For complete Ajocado vs. RichFaces Selenium differences, please visit the mention app sources, where you can find all. Here I am providing just the most important ones:

Richfaces Selenium	Ajoado
ElementLocator.getAsString	ElementLocator.getRawLocator
AjaxSeleniumProxy.getInstance()	AjaxSeleniumContext.getProxy()
SystemProperties	SystemPropertiesConfiguration, and its methods are no more static, for example seleniumDebug is retrieved in this way: AjocadoConfigurationContext.getProxy().isSeleniumDebug()
JQueryLocator.getNthOccurence	JQueryLocator.get
RetrieverFactory.RETRIEVE_TEXT	TextRetriever.getInstance()
removed getNthChildElement(i)	can be replaced by JQueryLocator(SimplifiedFormat.format("{0}:nth-child({1})", something.getRawLocator(), index));
RequestTypeGuard	RequestGuardInterceptor
RequestTypeGuardFactory	RequestGuardFactory
RequestInterceptor	RequestGuard
CommandInterceptionException	CommandInterceptorException
keyPress(String)	keyPress(char)
keyPressNative(String)	keyPressNative(int), so it is possible now to use KeyEvent static fields directly
isNotDisplayed	elementNotVisible, what is important about all displayed vs visible change is that visible methods will fail when the element is not present, displayed methods will return true, so keep it in mind while using visible methods, whether you need to use at first elementPresent
selenium.isDisplayed	selenium.isVisible
selenium.getRequestInterceptor()	selenium.getRequestGuard()
clearRequestTypeDone()	clearRequestDone()
waitXhr, waitHttp	guard(selenium, RequestType.XHR), guard(selenium, RequestType.HTTP)
FrameLocator	it has now two implementations FrameIndexLocator and FrameDOMLocator
ElementLocator methods	almost all mothods were removed, only few lasted, since now ElementLocator is implementing Iterator interface, and it is possible to replace them easily with it. Example of this is here (see the initializeStateDataFromRow method)