UI Testing With Xcode 7 and Swift – Part 1

This series of posts, will explore the new UI testing feature of Xcode 7 from an iOS developers perspective and try it out with some real-world scenarios. I will avoid using the new UI Recording feature where possible, and instead focus on assembling test scripts line by line. This is simply a matter of personal preference.

This post has been written with XCode 7 Beta 6. I will update this post as newer versions of Xcode 7 are announced.

Introduction:

User Interface testing (UI testing) is a new feature in Xcode 7 that allows you to write test methods that can launch an instance of your application, interact with UI elements of the application and validate the state and properties of these elements.

Xcode 7 also introduces a related feature called UI recording that can record all your interactions with your app and insert code into a test method to recreate these same sequence of interactions when the test is executed.

Adding UI Testing to an existing project

Adding UI testing support to an existing app is fairly straightforward. Open the project in XCode 7 and add a new build target to the project using the File > New > Target menu item. In the Target template options dialog box, select iOS UI Testing Bundle.

bp01_im02_augmented

In most cases, you will accept the default values in the target options dialog box.

bp01_im03_augmented

Some of the settings that you may wish to pay attention to are:

  • Language: The language in which UI test cases will be written (Objective-C/Swift). It is possible to have your application’s code in Objective-C and the UI tests in swift (or vice-versa), but doing so will require you to address some of the nuances of developing with multiple languages in the same Xcode project, which is perhaps the subject of another blog post.
  • Project: The project to which the target is being added
  • Target: The build target which is being tested by the new test target you are adding to the project.

Adding UI Testing to a new project.

If you are creating a new project, adding support for UI testing is a simple matter of enabling the Include UI Tests check box in the project options dialog box.

bp01_im01_augmented

The option to add UI tests is available for all new iOS, Watch OS, and OS X application templates. It is not available if you are creating a framework or library.

Changes to the Xcode project

When you add support for UI testing to a project, you will notice three changes to your project:

  1. A new group has been added to the project explorer. This group will be used to contain your ui test files.
  2. A new build target is added to the project settings. This new build target is called the test target.
  3. The test target is pre-configured to test the host application.

bp01_im04_augmented

Basics of UI Testing with Xcode 7.

UI tests are stored in their own group in the project explorer. UI tests are written as methods of classes that inherit from XCTestCase. UI Test classes can have three types of methods in them:

  • Setup method: This method is called setUp() and is called before each test method is executed in the test class.
  • Teardown method: This method is called tearDown() and is called after each test method is executed in the test class.
  • Test methods: These methods all begin with the word test. Each method encapsulates a single test.

When you add support for UI testing to a project, Xcode creates a single test class with stub methods:

To execute all unit tests across all test classes in a project, use the Product > Test menu item. Doing so will launch the app on the iOS Simulator or iOS device and execute all methods that begin with the word test in each test case sequentially.

Note: If your project has both unit tests and UI tests, UI tests will execute after all unit tests have completed, regardless of any unit test failures.

The result of the testing phase is visible in the Test Navigator, which can be accessed using the View -> Navigators -> Show Test Navigator menu item

bp01_im12

You will see a green tick box next to each test that has passed, and a red one next to each test that has failed. Keep in mind that test code is also code and must be able to compile for the tests to be executed. If your project has compilation errors you will need to fix these before the tests can run.

You can add additional test classes to your project by using the Add button (+) and the New UI Test Class command in the Test Navigator

bp01_im06

You could alternately, simply add a new file to your project using the File > New > File menu item and select the UI Test Case Class template located in the iOS Source group.

bp01_im07

 

New Classes, Protocols, Enumerations

To support UI Testing, Apple have added several new classes, protocols, and enumerations to XCTest.

Classes XCUIApplication, XCUIDevice, XCUIElement, XCUIElementQuery
Protocols XCUIElementAttributes
Enumerations XCUIElementType

The XCUIApplication class

An XCUIApplication instance is used to launch/terminate your application for testing on a device or simulator. Typically, you instantiate an XCUIApplication instance in your test class’ setup() method and call the launch method:

You can also set specific arguments of environment variables by setting the launchArguments property. For example the following snippet passes a launch argument USE_DEBUG_SERVER to the UIApplication instance:

The application can look out for this argument in application(application, didFinishLaunchingWithOptions) and take appropriate action (such as load web service end points for a staging server):

This snippet retrieves any launch arguments as an array of strings by calling Process.arguments. The first element in this array is always the full path to the application which is not of interest to us for this particular scenario, which is why it examines elements from index 1 onwards.

To terminate an app, you could call the terminate() method on an XCUIApplication instance. This is not strictly necessary as XCTest will terminate the application instance automatically every time a test finishes executing.

Note: You will need to call the terminate() method explicitly if you intend to run UI tests on Xcode server. This will be covered in a follow up post.

The XCUIDevice class

XCUIDevice is a singleton, an instance of this class represents the device on which the test is running. There is always only one instance of this device that can be accessed as follows:

As of writing this post, XCUIDevice has only one property called orientation that returns the orientation of the device. Setting this property changes the orientation of the device. This could be useful when you want to test your user interface in different orientations.

The following setup method passes a launch argument and changes the device to landscape orientation before running a UI test.

Note: You may be tempted to change the orientation property between test methods in the same class, in effect grouping a bunch of landscape mode tests and a bunch of portrait mode tests in a single .swift file. While there is no hard and fast rule against this, I prefer to keep separate files for each size class if I need to test at that level of detail.

The XCUIElementQuery class

An XCUIElementQuery instance encapsulates a UI query and is used to locate one or more UI elements in your applications user interface. This class, along with XCUIElement are the primary classes used while writing UI test cases.

In most situation, when it comes to create a query you will not instantiate an XCUIElementQuery explicitly. Instead you will use one of the properties implemented on the XCUIApplication instance to return an initial query which you will then refine.

The reason why you can get an initial high-level query object from XCUIApplication is because XCUIApplication implements the query provider protocol (XCUIElementTypeQueryProvider). The query provider protocol is discussed later in this post, but suffice to say it defines several properties that return usefull pre-configured query objects.

Bringing our focus back to the query class, you will notice that XCUIElementQuery defines several instance methods of which some of the more useful ones are summarized below:

Property/Method name Description
var count: UInt { get } Resolves the query and returns the number of elements matched by the query
func elementBoundByIndex(index: UInt) -> XCUIElement Resolves the query and returns an element at the specified index
func elementMatchingType(elementType: XCUIElementType, identifier: String?) -> XCUIElement Resolves the query and returns an element that matches a specific type and accessibility identifier.
func childrenMatchingType(type: XCUIElementType) -> XCUIElementQuery Returns a query that can be used to extract children of a specific type.

You will notice that some of these methods return an XCUIElement, whereas others return another XCUIElementQuery instance. In the latter case, the returned XCUIElementQuery instance is usually used to obtain a smaller subset of elements.

To get an idea of how queries work, consider a single view based application that has three buttons and a label on it:

bp01_im08

The following code will return the number of buttons in the view controller that are currently visible on the screen (which in this case will be 3).

Notice how we used application.buttons to get a query that retrieves all the buttons on the visisble screen. The buttons property is defined in XCUIElementTypeQueryProvider (more on that soon).

The XCUIElement class

An XCUIElement instance contains the information required to locate a user interface element in your application (but not the actual UIKit element). It can be thought of as a proxy for a UIKit element and is obtained by calling one of the methods on an XCUIElementQuery instance.

The information within an XCUIElement is only evaluated when a method is called on the XCUIElement. At the time of evaluation, if the XCUIElement does not resolve into an actual element an error will be raised.

It is important to keep in mind that the XCUIElement instance does not let you access the underlying user element directly. For instance if you had an XCUIElement that represents a text field instance on a view, you can not dereference the XCUIElement to arrive at the underlying UITextField object and then attempt to manipulate the underlying object.

However, an XCUIElement object does have a few methods that can be used to interact with the underlying element programmatically, but this interaction is restricted to taps, gestures, and text entry (as an end user would while using your app).

To achieve this, XCUIElement provides a number of properties and methods that you could call on a concrete instance, some of which are listed below:

 

Property/Method name Description
var exists: Bool { get } Returns true if the XCUIElement resolves into an actual UI element in the app.
func tap() Sends a tap event to the underlying UI element.
func doubleTap() Sends a double tap event to the underlying UI element.
func pressForDuration(duration: NSTimeInterval) Sends a long press gesture event to the underlying UI element.
pressForDuration(duration: NSTimeInterval, thenDragToElement otherElement: XCUIElement) Sends a press and hold gesture to the underlying UI element that then drags to another element.
func swipeUp() Sends a swipe up gesture to the underlying UI element.
func swipeDown() Sends a swipe down gesture to the underlying UI element.
func swipeLeft() Sends a swipe left gesture to the underlying UI element.
func swipeRight() Sends a swipe left gesture to the underlying UI element.

 

Interestingly enough, XCUIElement conforms to XCUIElementTypeQueryProvider (just like the XCUIApplication class), which means you could potentially query a UI element to obtain a list of child elements and so forth.

XCUIElement also conforms to the XCUIElementAttributes protocol which provides methods and properties that will let you access various attributes of an element such as its frame. XCUIElementAttributes will be covered later in this post.

Continuing with the three button sample introduced earlier, recall that the following snippet will result in a query that resolves to three buttons:

You could attempt to get an XCUIElement instance that represents the green button calling the elementBoundByIndex method on the query:

To verify you have indeed got the green button, simply inspect the value of the label property on XCUIElement instance (greenButton) :

In case you were wondering, the label property is defined in the XCUIElementAttributes protocol, which is implemented by XCUIElement.

This type of code is sensitive to the layout of the user interface. A far better approach is to setup an accessibility identifier for the buttons in the storyboard and use the accessibility identifier to retrieve the green button regardless of how the storyboard scene was laid out.

To setup an accessibility identifier for your user interface elements, select the user interface element in the storyboard and use the Identity Inspector.

bp01_im09_augmented

Once an element has an accessibility identifier set up, you could use the following snippet to return an XCUIElement instance that will resolve to the green button regardless of how the user interface is laid out.

The last line of the above snippet retrieves an XCUIElement of a specific type with a specific identifier. Alternately, you could have written the last line as:

This alternate statement uses the subscript operator ([ ]) to retrieve an element by accessibility identifier.

The XCUIElementAttributes protocol

The XCUIElementAttributes protocol defines several properties that return commonly used attributes, and is implemented by XCUIElement as one would expect. Some of the commonly used properties defined in XCUIElementAttributes are:

Property Name Description
var identifier: String { get } Returns the accessibility identifier of the element
var frame: CGRect { get } Returns the frame property of the element
var title: String { get } Returns the accessibility title of the element
var label: String { get } Returns the caption of the element (if applicable).
var elementType: XCUIElementType Returns an enum value that represents the type of the element.
var enabled: Bool { get } Returns true if the element is enabled for user interaction
func swipeDown() Sends a swipe down gesture to the underlying UI element.
func swipeLeft() Sends a swipe left gesture to the underlying UI element.
func swipeRight() Sends a swipe left gesture to the underlying UI element.

The XCUIElementTypeQueryProvider protocol

This protocol defines several properties that return pre-configured XCUIElementQuery instances. Both XCUIApplication and XCUIElement implement this protocol.

Some of the properties defined in this protocol are listed below. Typically you will use one of these methods on the XCUIApplication instance to return an initial query, and you will then use the methods defined in XCUIElementQuery to filter down to a specific element.

Property/Method name Description
var windows: XCUIElementQuery { get } Returns an query that provides access to all windows that are currently visible in app. iOS applications have just a single window.
var alerts: XCUIElementQuery { get } Returns an query that provides access to all alerts that are currently visible in app. Usually there is only one alert visible in an app at a time.
var buttons: XCUIElementQuery { get } Returns an query that provides access to all buttons that are currently visible in app
var navigationBars: XCUIElementQuery { get } Returns an query that provides access to all navigation bars that are currently visible in app
tables: XCUIElementQuery { get } Returns an query that provides access to all table views that are currently visible in app
var collectionViews: XCUIElementQuery { get } Returns an query that provides access to all collection views that are currently visible in app
var staticTexts: XCUIElementQuery { get } Returns an query that provides access to all labels that are currently visible in app
var textFields: XCUIElementQuery { get } Returns an query that provides access to all text fields that are currently visible in app
textViews: XCUIElementQuery { get } Returns an query that provides access to all text views that are currently visible in app
var maps: XCUIElementQuery { get } Returns an query that provides access to all map views that are currently visible in app
var otherElements: XCUIElementQuery { get } Returns an query that provides access to all view controllers that are currently visible in app

The XCUIElementType enumeration

The elementType property (in the XCUIElementAttribute protocol) is an enumerated value that represents the type of element. XCUIElementType is a very large enumeration and some of its members do not apply to iOS applications (they apply to MacOS X apps). Some of the more commonly used values are listed below:

  • Alert
  • Button
  • NavigationBar
  • TabBar
  • ToolBar
  • ActivityIndicator
  • SegmentedControl
  • Picker
  • Image
  • StaticText
  • TextField
  • DatePicker
  • TextView
  • WebView

Test Assertions

An assertion represents a failure of a unit test. Typically your UI test case will use one of the properties defined by the XCUIElementTypeQueryProvider protocol on the XCUIApplication instance to obtain an XCUIElementQuery instance.

It will them resolve the query into an XCUIElement and inspect some of the attributes of the underlying UI element.

If the value of the underlying attribute being tested does not match the expected value, the test will fail by firing an assertion. XCTest provides several macros to help you create assertions. Some of the more commonly used macros are listed below

Macro Description
XCTAssert(expression, String) Generates a failure if the expression evalutes to false. An optional string   message may be provided to indicate failure.
XCTAssertEqual(expression1, expression2, String) Generates a failure when expression1 is not equal to expression2. This test is for primitive data types
XCTAssertNil (expression, String) Generates a failure when the expression is not nil.
XCTAssertNotNil(expression, String) Generates a failure when the expression is nil.
XCTAssertTrue (expression) Generates a failure when the expression evaluates to false. Identical to XCTAssert()
XCTAssertFalse (expression) Generates a failure when the expression evaluates to true.

As an example, the following code snippet lists a UI test case that will try to locate a button with a specific accessibility identifier and assert if the button was not found:

It is worth noting that XCTAssert was used instead of XCTAssertNotNil. This is because greenButton is an XCUIElement instance.

Recall that XCUIElement is not the actual user element on the screen, it just represents the information needed by the testing framework to attempt to locate a user interface element. Only when you try to access the underlying element (by calling exists() on the XCUIElement) will the testing framework try to resolve the XCUIElement into an actual user interface element.

The following snippet builds on the previous test and asserts if the label on the button does not match a specific value.

UI Recording

One of the coolest new features added to Xcode 7 is UI Recording. With UI recording, you can launch an instance of your application and interact with it as normal. While you interact with your app, Xcode will record your taps, gestures, selections, and key strokes into a UI test script.

UI recording is tightly coupled with UI testing and to begin UI recording, simply place the text cursor within a UI test case and tap the red record button at the bottom of the Xcode editor
bp01_im10_augmented

To stop recording simply tap the stop button, which replaces the record button during a recording session. UI recording provides a good starting point to build your UI tests, you can then fine tune the code generated by UI recording and add appropriate XCTAssert statements.

Personally, I find that UIRecording can be useful to get a starting point for your UI test and then edit the results to fine tune it to your needs.

A concrete example

To put everything we have discussed so far in context, let me present a simple concrete example. Once again, building on the three-button, one-label example application

bp01_im08

In this app, when a user taps one of the three buttons, the text in the label changes to read the caption of the button that was tapped.

The code for the ViewController.swift file is presented below, as you can see it is fairly straightforward:

The buttons and label have their accessibility identifiers setup as redButton, greenButton, yellowButton, resultLabel

bp01_im11_augmented

Let us now see what some of the UI tests look like, in this simple example, all the tests are in a single class. The setup and teardown methods are standard boilerplate stuff:

For the first test, I would like to verify that the red button exists on the user interface:

As you can see, I have used app.buttons to get a query that enumerates all buttons, and then retrieve a specific element from the query by using the accessibility identifier of the red button (redButton)

A second test would naturally be to verify the other buttons exist on the user interface:

In these test methods I have used different methods to retrieve an element, most likely you will end up using the third option:

Another test scenario would involve tapping the red button and verify that the caption of the label updates correctly:

This test is only slightly more complicated than the earlier ones. First we get a XCUIElement instance for the red button and tap it:

Next we get the caption attribute of the XCUIElement that represents the label. We get this element by using the accessibility identifier of the label:

Last but not least, we use the XCTAssertEqual macro to assert if the caption is not what we expect.

Notice how I have hard coded the caption in the expectation. This of course implies that if I were to change the title of the button on the storyboard, I would need to come back and update this test.

Sometimes this is what you want, after all the UI test should break if something was changed on the storyboard. But, if you wanted, you could have written the assertion differently:

This assertion now verifies that the text on the label is the same as the caption on the button that was tapped, but not what the specific value of the caption is.

If you were to now run all the tests (Product -> Test), you can see that all of them pass:

bp01_im12

You can easily build on from these examples and add more test methods. That’s all for this post. I welcome your comments.

In the next post in this series, I will look at setting up test expectations and waiting for asynchronous operations with an example app.


Warning: count(): Parameter must be an array or an object that implements Countable in /homepages/5/d323418802/htdocs/runtimecrash/wp-includes/class-wp-comment-query.php on line 399

2 thoughts on “UI Testing With Xcode 7 and Swift – Part 1”

Leave a Reply

Your email address will not be published. Required fields are marked *