BDD, Gherkin, SpecFlow and Xamarin UITest

In this post I show how to use SpecFlow and Xamarin UITest to run automated tests written in Gherkin. The source of my demo solution is available here.

Why should you care?

The big benefit of this is that you can take a specification written by non technical team members and turn it into tests that prove that the app behaves corectly.

The super complicated demo app

BDD, Gherkin, and SpecFlow

Behaviour Driven Development (BDD) is an approach to software development that uses specifications written in natural language to verify that the product is behaving correctly. This enhances communication between more and less technical team members and ensures that the correct product gets built.

Gherkin is a language for describing softwares behaviour, without specifying how the behaviour is implemented. This means it can be written by non technical team members. A Gherkin file is made up of features, scenarios, and steps. Here’s the feature that will be tested in the demo app…

Feature: Log in to app

Scenario: Olivia logs in to app
    Given the login page is displayed
    When Olivia enters the username Olivia
    And she enters the password IHeartOwls
    And she presses the Logon button
    Then the home screen is displayed

Scenario: Olivia uses the wrong password
    Given the login page is displayed
    When Olivia enters the username Olivia
    And she enters the password IHateTrees
    And she presses the Logon button
    Then an invalid username or password message is displayed

They key thing here is anyone could have written it and it describes how the app behaves, not how it is implemented. Obviously in the real world there would be more scenarios!

SpecFlow is a .net tool to run the tests based on the features that are written in Gherkin.

Prerequisites

For Mac development in Visual Studio
– Straight8’s SpecFlow Integration

For Windows development in Visual Studio
– SpecFlow for Visual Studio 2017

Solution Setup

These instructions assume that you have a working app which isn’t set up for UITest. The first thing to do is add a UITest project to the solution.

The new UITest project has a Tests.cs file which should be deleted.

Next you need to make sure the appropriate automation ids are present in the apps interface. For example here’s the login page from the demo app. You can see that Automation Ids are present for the page, both entries and the login button.

Finally you need to add a reference to SpecFlow via NuGet. You need to be careful here because some of the SpecFlow packages use versions of Nunit which aren’t compatible with UITest or Test Cloud. The packages in the demo are…

Writing the Tests

Now we are ready to start writing the tests. Add a new SpecFlow Feature file to the test solution and add the appropriate Gherkin scenarios.

Now you can try to run the tests, but you’ll get a bunch of inconclusive results with the message ‘No matching step definition found for…’. You need to write the step definitions which are control the actions that occur for each step.

Add a SpecFlow Step Definition file called Steps.cs to the project. The example methods show how to bind a method to a step but they can all be replaced with steps based on the feature file. You can generate the code for the steps via a right click in the feature file.

Now you can define the steps. Here’s the file for the demo app…

using Xamarin.UITest;
using TechTalk.SpecFlow;

namespace SpecFlowDemo.UITests
{
    [Binding]
    public class Steps
    {
        private IApp app;

        [Given("the login page is displayed")]
        public void TheLoginPageIsDisplayed()
        {
            app = ConfigureApp.Android.StartApp();
            app.WaitForElement("LoginPage");
        }

        [When("Olivia enters the username Olivia")]
        public void OliviaEntersTheUsernameOlivia()
        {
            app.EnterText("UsernameEntry", "Olivia");
        }

        [When("she enters the password IHeartOwls")]
        public void WhenSheEntersThePasswordIHeartOwls()
        {
            app.EnterText("PasswordEntry", "IHeartOwls");
            app.DismissKeyboard();
        }

        [When("she enters the password IHateTrees")]
        public void WhenSheEntersThePasswordIHateTrees()
        {
            app.EnterText("PasswordEntry", "IHateTrees");
            app.DismissKeyboard();
        }

        [When("she presses the Logon button")]
        public void ShePressesTheLogonButton()
        {
            app.Tap("LoginButton");
        }

        [Then("the home screen is displayed")]
        public void TheHomeScreenIsDisplayed()
        {
            app.WaitForElement("MainPage");
        }

        [Then("an invalid username or password message is displayed")]
        public void AnInvalidUsernameOrPasswordMessageIsDisplayed()
        {
            app.WaitForElement("Logon Failed");
            app.WaitForElement("Invalid username or password");
        }
    }
}

Note that in the first step we set up the UITest application object. This works but it means that tests  can only run on either iOS or Android. Fortunately it’s easy to get it running on both…

Support for iOS and Android

To support both platforms we need to take advantage of the fact that SpecFlow is based around nunit, and that it creates partial classes for it’s test definitions. Add a new file called TestPartials.cs to the solution.

using System;
using NUnit.Framework;
using Xamarin.UITest;

namespace SpecFlowDemo.UITests
{
    public static class Global
    {
        public static Platform Platform { get; set; }
        public static IApp App { get; set; }
    }

    [TestFixture(Platform.iOS)]
    [TestFixture(Platform.Android)]
    public partial class LogInToAppFeature
    {
        public LogInToAppFeature(Platform platform)
        {
            Global.Platform = platform;
        }
    }
}

By adding a partial class for each feature like this you ensure they will run for both platforms. The Globals.Platform flag is set so that the app can be configured. The first step needs to be changed to pick this up…

[Given("the login page is displayed")]
public void TheLoginPageIsDisplayed()
{
    if (Global.Platform == Platform.iOS)
    {
        Global.App = ConfigureApp.iOS.StartApp();
    }
    else
    {
        Global.App = ConfigureApp.Android.StartApp();
    }

    Global.App.WaitForElement("LoginPage");
}

Code Improvements

So it’s all working nicely but there are a few things that can be improved. SpecFlow supports passing parameters to step definitions so we only need one definition to set the password. This is done by added a regular expression to capture the parameter to the definition’s attribute, and adding the parameter to the method’s signature. Here’s how it works for the password…

[When("she enters the password (.*)")]
public void WhenSheEntersThePassword(string password)
{
    Global.App.EnterText("PasswordEntry", password);
    Global.App.DismissKeyboard();
}

Another thing that’s worth considering is splitting the step definitions into separate files instead of big one. I have found that one file per screen is a good starting point. As long as the [Binding] attribute is present you can put the definitions wherever you want…

using Xamarin.UITest;
using TechTalk.SpecFlow;

namespace SpecFlowDemo.UITests
{
    [Binding]
    public class MainPageSteps
    {
        [Then("the main page is displayed")]
        public void TheMainPageIsDisplayed()
        {
            Global.App.WaitForElement("MainPage");
        }
    }
}

Typically you’ll want to run some steps at the start of each test to set up a clean environment, and start the app. Cucumber uses the Background keyword for this…

Feature: Log in to app

Background:
    Given a started app

Scenario: Olivia logs in to app
    Given the login page is displayed
    When Olivia enters the username Olivia
    ...

using Xamarin.UITest;
using TechTalk.SpecFlow;

namespace SpecFlowDemo.UITests
{
    [Binding]
    public class SetupSteps
    {
        [Given("a started app")]
        public void TheMainPageIsDisplayed()
        {
            if (Global.Platform == Platform.iOS)
            {
                Global.App = ConfigureApp.iOS.StartApp();
            }
            else
            {
                Global.App = ConfigureApp.Android.StartApp();
            }
        }
    }
}

So that’s it! BDD for mobile apps via Xamarin UITest and SpecFlow. Give it a go and let me know how you get on 🙂

Xamarin UITest – Streamlined Back Door Calls

Here’s a mini post – Making back door calls on iOS and Android is slightly different. Here’s an extension method that works on both…

 public static class AppExtensions
 {
    public static void CallBackdoorMethod(this IApp app, string method, object argument = null)
    {
        if (app is iOSApp)
        {
if (argument == null)
{
argument = "";
}
app.Invoke($"{method}:", argument);
        }
        else
        {
            app.Invoke(method, argument);
        }
    }
}

Xamarin Forms Cross Platform Libraries and Dependency Services

This post is about how to set up cross platform libraries for Xamarin Forms, and how to implement Dependency Services in those libraries. The source code for these examples are here.

The example app is a very simple page, which shows a time string that is taken from a cross platform library, and a culture that is taken from a platform specific library via a dependency service. It looks like this…

Cross Platform Libraries

Adding a cross platform library is easy – simply add a Xamarin.Forms class library project to your solution, and add a project reference to your cross platform application project. You can then use classes from the library as you would expect.

Platform Libraries & Dependency Services

Things get interesting when you want to add dependency services to the library. For this you need to add an iOS class library project, and an Android class library project to your solution. You also need to add the Xamarin.Forms to both projects.

Next you have to set up the references. These are the references to add…

Sample App References: Cross Platform Library
iOS App References: Cross Platform Library, iOS Library
Android App References: Cross Platform Library, Android Library
Cross Platform Library References: -
iOS Library References: Cross Platform Library, Xamarin Forms
Android Library References: Platform Library, Xamarin Forms

Now you can define the dependency service interface in the cross platform library…

using System;

namespace SampleLibrary
{
    public interface ICultureProvider
    {
        string GetCultureString();
    }
}

And implement the interface in the platform specific libraries, here’s the iOS implementation…

using System;
using SampleLibrary.iOS;
using Xamarin.Forms;
using System.Threading;

[assembly: Dependency(typeof(CultureProvider))]
namespace SampleLibrary.iOS
{
    public class CultureProvider : ICultureProvider
    {
        public CultureProvider()
        {
        }

        public string GetCultureString()
        {
            return Thread.CurrentThread.CurrentCulture.ToString();
        }
    }
}

Remember to add the Dependency attribute! This needs to reference the implementation class, not the interface.

Now you can use DependencyService.Get to get access the service…

var cultureProvider = DependencyService.Get();
var culture = cultureProvider.GetCultureString();

For a regular dependency service defined in the application projects this would be enough. However using libraries introduces an extra foible. At compile time the application specific library might not be included in the project because there’s no reference to it in code. If this DependencyService.Get returns null. To fix this you need to add some code to the platform specific application to reference the platform specific library. This can be as simple as creating an instance of one of the classes in the library…

[Register("AppDelegate")]
public partial class AppDelegate : global::Xamarin.Forms.Platform.iOS.FormsApplicationDelegate
{
    public override bool FinishedLaunching(UIApplication app, NSDictionary options)
    {
        global::Xamarin.Forms.Forms.Init();
        var foo = new SampleLibrary.iOS.CultureProvider();

        LoadApplication(new App());

        return base.FinishedLaunching(app, options);
    }
}

With the reference in place the dependency service will resolve correctly. This feels like a hack, but I haven’t found a better solution, if you have any ideas leave a comment!

Any better ideas??

Hyper-V and Haxm

Here’s a quick tip to make working with Hyper-V and Haxm in Windows a little easier. It’s not possible to run both at the same time which is a pain if, for example, you want to run Android emulators and Docker at the same time.

What you can do is set up a boot entry with Hyper-V turned off, then at start up you can choose to run with Hyper-V for Docker, or without to run your emulators.

To do this fire up an administrator command prompt and type

bcdedit /copy {current} /d "Hypervisor Off - Android Emulators On"

Next find and copy the identifier for the new boot entry by running

bcdedit

Then turn Hypervisor off with

bcdedit /set {newidentifier} hypervisorlaunchtype off

Restart the machine and you’ll be able to choose which one to boot 🙂

Xamarin UI Test – Testing State Management

State management, ensuring that an suspended app restarts to the same place, is one of the more challenging areas of mobile development. It’s a good idea to make sure it’s covered by your automated tests. Here’s how I managed this with Xamarin UITest…

The source code for this post is available here.

1 – Setup Tests to Run in Order

My example uses two tests, one to set the data up and one to make sure the same data is present in the restored app. The Order attribute in nunit ensures they run sequentially.

[Test, Order(1)]
public void StateTest()
{
    app = AppInitializer.StartApp(platform);
    app.Tap("Button");
    app.Tap("Button");
    app.EnterText("Text", "Some text");

    SaveAppState();
}

[Test, Order(2)]
public void StateTest2()
{
    app = AppInitializer.StartApp(platform, true);
    app.WaitForElement("Clicked 2 Times");
    app.WaitForElement(x => x.Marked("Text").Text("Some text"));
}

2 – Preserve Local Data Between Tests

In regular tests any local data will be wiped before the app is run, but here it needs to be preserved. A parameter added to the AppInitialiser controls this. This means that the local data from the first test is still in place for the second test.

public static IApp StartApp(Platform platform, bool preseveData = false)
{
    var dataMode = preseveData ? AppDataMode.DoNotClear : AppDataMode.Clear;

    if (platform == Platform.Android)
    {
        return ConfigureApp.Android
            .StartApp(dataMode);
    }
    else
    {
        return ConfigureApp.iOS
           .StartApp(dataMode);
    }
}

The documentation says that the AppDataMode attribute is only supported in test cloud, but I found it works with local tests as well.

3 – Suspend the App Programatically

When the app is stopped and restarted between tests it doesn’t get a chance to save state. To get round this the first test suspends the app programmatically. This is done via backdoor methods for each platform.

Here’s a way to suspend on iOS:

[Export("SuspendApp:")]
public void SuspendApp(NSString value)
{
    UIApplication.SharedApplication.PerformSelector(new ObjCRuntime.Selector("suspend"), null, 2.0);
}

Here’s a way to ‘suspend’ on Android (note this just triggers App.OnSleep, it doesn’t really suspend the app):

[Export("SuspendApp")]
public void SuspendApp()
{
    OnPause();
    OnStop();
}

Run the Tests!

With everything in place the last thing to do is run the tests – here’s how it looks on my machine 🙂

Running Xamarin UITest Repl Outside of Tests

The Xamarin UITest Repl is a handy little tool for inspecting and interacting with the interface for iOS and Android apps. Normally you run it by adding app.Repl() to your test code but it’s also easy to run Repl outside of a test…

Android on Windows

To run Repl on an Android device connecting to Windows you need to know:

• The location of the Repl executable (xut-repl.exe) – C:\Users\Username\AppData\Local\Temp\uitest\repl or similar
• The location of the UITest dll, something like – C:\Users\Username.nuget\packages\Xamarin.UITest\2.0.10\lib\Xamarin.UITest.dll
• The identifier for the connected device – you can get this by running adb devices

To start Repl open a command prompt and change to the executable folder. Then run xut-repl.exe android {uitestdllpath} {deviceid} after replacing the relevant values. For example:

xut-repl.exe android C:\Users\Username\.nuget\packages\Xamarin.UITest\2.0.10\lib\Xamarin.UITest.dll http://notused 15337521650191

Android and iOS on macOS

On macOS it’s a bit easier. Run Repl from a test then copy the title of the Repl window. Fire up a new terminal and enter the copied command.

Here’s an iOS example from my machine:

mono /var/folders/6l/bg1gb8v13mq5lgqgnj4_pdgw0000gn/T/uitest/repl/xut-repl.exe ios ~/Code/Trailmaker.Core/Trailmaker.Core.UITest/bin/Debug/Xamarin.UITest.dll http://127.0.0.1:37265 06C7B6BD-A7F2-436A-8BF9-A911F944A2CE true

And here’s an Android example:

mono /var/folders/6l/bg1gb8v13mq5lgqgnj4_pdgw0000gn/T/uitest/repl/xut-repl.exe android ~/Code/Trailmaker.Core/Trailmaker.Core.UITest/bin/Debug/Xamarin.UITest.dll http://127.0.0.1:60360 emulator-5554 ~Library/Developer/Xamarin/android-sdk-macosx

Xamarin – Build on iOS from Windows Command Prompt/MSBuild

A quick post to describe how to build a Xamarin iOS project from MSBuild on a Windows machine. You still need a mac available but this approach means you can create an iOS artefact from a Windows TFS build agent.

On the command line run:

msbuild YourProject.sln /p:Configuration=AppStore;Platform=iPhone /p:ServerAddress=MacIpAddress /p:ServerUser=MacLogin /p:ServerPassword=MacPassword

When this is running keep an eye on the mac as it might pop up a message box or two on the first run.

After the script completes you should have a new IPA file in the bin\iPhone\Ad-Hoc directory of your iOS project on the Windows machine.

 

Xamarin Forms Walkthrough UI

Here’s a walkthrough interface for use in Xamarin Forms apps. With a some work it’s easy to move beyond the default layouts and navigations patterns and make something a bit more exciting.

The video shows it in action. You can switch between pages using the buttons or by swiping. The content for the walkthrough control is defined by a list of views; it can be anything you want. The page indicators update every time you change a page.

The view provides properties to adjust the background color, back and next text, the image used for the progress indicators, and the amount of padding used for the control area.

Implementation

If you want to know how it works you can dive into the source here. In this post I’ll just run through the key parts.

The content view XAML is just a stack layout which contains two grids. The top grid expands to fill the screen and contains the child content pages when they are added. The bottom grid contains the controls to switch the page. There’s also a gesture recognizer on the whole view to pick up swipe gestures.

The key bit of code for the content view is the SetContent method. This takes a list of views and adds them into the grid. It offsets the children at intervals of a page width – so initially they extend horizontally out to right of the screen. When the page changes it animates each of the children to shift them right or left. The other thing it does is expose the configurable options.

I’ve split a lot of the logic for which pages andbuttons are displayed out into the WalkthroughController class for a couple of reasons. Firstly it’s easier to write unit tests, and secondly it nicely separates the processing and interaction logic.

That’s pretty much it – a bit of XAML and a couple of simple classes produce a nice, configurable interface 🙂

 

Xamarin Forms Repeater View

Here’s a quick demo and explanation of a repeater view I made for Xamarin Forms. It simply repeats a set of controls for a collection of items that it’s bound to. A ListView does this too but isn’t always what I want. It’s surprising something like this isn’t included in the framework but making your own is fairly simple.

Source code and a sample application are available here.

Here’s what it looks like in action. In this simple demo each item is represented by a Frame that contains a label and a button.

10 internet points if you can identify what’s in the background.

Let’s look at some key points in the code. The view is based on StackLayout which will take care of positioning the items.

public class Repeater : StackLayout

There are a couple of bindable properties – ItemTemplate for the item layout and ItemsSource which provides the items. Note that ItemsSource hooks an action to PropertyChanging – I’ll come back to this later on.

public static readonly BindableProperty ItemTemplateProperty = BindableProperty.Create("ItemTemplate", typeof(DataTemplate), typeof(Repeater), null);
public static readonly BindableProperty ItemsSourceProperty = BindableProperty.Create("ItemsSource", typeof(IEnumerable), typeof(Repeater), propertyChanging: ItemsSourceChanging);

Usage in XAML is similar to ListView, with an ItemTemplate, DataTemplate, and ViewCell.

<ui:Repeater x:Name="MainRepeater">
    <ui:Repeater.ItemTemplate>
        <DataTemplate>
            <ViewCell>
                <Frame HasShadow="True" Margin="10">
                    <StackLayout Orientation="Horizontal">
                        <Label Text="{Binding Title}" HorizontalOptions="StartAndExpand" VerticalTextAlignment="Center"/>
                        <Button Text="Select" />
                    </StackLayout>
                </Frame>
            </ViewCell>
        </DataTemplate>
    </ui:Repeater.ItemTemplate>
</ui:Repeater>

I populate the display items whenever the related properties change, or when the binding context changes.

protected override void OnPropertyChanged(string propertyName = null)
{
    base.OnPropertyChanged(propertyName);

    if (propertyName == ItemTemplateProperty.PropertyName || propertyName == ItemsSourceProperty.PropertyName)
    {
        this.Populate();
    }
}

protected override void OnBindingContextChanged()
{
    base.OnBindingContextChanged();
    this.Populate();
}

The Populate method clears out the layout, then populates the template for each item and adds it in.

public void Populate()
{
    if (this.ItemsSource != null)
    {
        this.Children.Clear();

        foreach (var item in this.ItemsSource)
        {
            var content = this.ItemTemplate.CreateContent();
            var viewCell = content as ViewCell;

            if (viewCell != null)
            {
                this.Children.Add(viewCell.View);
                viewCell.BindingContext = item;
            }
        }
    }
}

If we just wanted to bind to an IEnumerable we wouldn’t need anything more. However when binding to an ObservableCollection the list wouldn’t update when items are added or removed. ObservableCollection fires OnCollectionChanged, not OnPropertyChanged, when the list is updated. This is where the ItemsSourceChanging method we hooked up in the binding is used.

private static void ItemsSourceChanging(BindableObject bindable, object oldValue, object newValue)
{
    if (oldValue != null && oldValue is INotifyCollectionChanged)
    {
        ((INotifyCollectionChanged)oldValue).CollectionChanged -= ((Repeater)bindable).OnCollectionChanged;
    }

    if (newValue != null && newValue is INotifyCollectionChanged)
    {
        ((INotifyCollectionChanged)newValue).CollectionChanged += ((Repeater)bindable).OnCollectionChanged;
    }
}

private void OnCollectionChanged(object sender, NotifyCollectionChangedEventArgs notifyCollectionChangedEventArgs)
{
	Populate();
}

When the ItemsSource is changing we check to see if it implements INotifyCollectionChanged, if so we subscribe to the OnCollectionChanged event and populate the list when it fires. We also need to make sure we unsubscribe if the items source changes.

So that’s it, a simple alternative to ListView for repeating a set of controls.