BDD, Gherkin, SpecFlow and Xamarin UITest

In this post I show how to use SpecFlow and Xamarin UITest to run automated tests written in Gherkin. The source of my demo solution is available here.

Why should you care?

The big benefit of this is that you can take a specification written by non technical team members and turn it into tests that prove that the app behaves corectly.

Screen Shot 2018-03-18 at 17.36.45
The super complicated demo app

BDD, Gherkin, and SpecFlow

Behaviour Driven Development (BDD) is an approach to software development that uses specifications written in natural language to verify that the product is behaving correctly. This enhances communication between more and less technical team members and ensures that the correct product gets built.

Gherkin is a language for describing softwares behaviour, without specifying how the behaviour is implemented. This means it can be written by non technical team members. A Gherkin file is made up of features, scenarios, and steps. Here’s the feature that will be tested in the demo app…

Feature: Log in to app

Scenario: Olivia logs in to app
    Given the login page is displayed
    When Olivia enters the username Olivia
    And she enters the password IHeartOwls
    And she presses the Logon button
    Then the home screen is displayed

Scenario: Olivia uses the wrong password
    Given the login page is displayed
    When Olivia enters the username Olivia
    And she enters the password IHateTrees
    And she presses the Logon button
    Then an invalid username or password message is displayed

They key thing here is anyone could have written it and it describes how the app behaves, not how it is implemented. Obviously in the real world there would be more scenarios!

SpecFlow is a .net tool to run the tests based on the features that are written in Gherkin.

Prerequisites

For Mac development in Visual Studio
Straight8’s SpecFlow Integration

For Windows development in Visual Studio
SpecFlow for Visual Studio 2017

Solution Setup

These instructions assume that you have a working app which isn’t set up for UITest. The first thing to do is add a UITest project to the solution.

The new UITest project has a Tests.cs file which should be deleted.

Next you need to make sure the appropriate automation ids are present in the apps interface. For example here’s the login page from the demo app. You can see that Automation Ids are present for the page, both entries and the login button.



    
        
            
            
            
            
            
            
        
    

Finally you need to add a reference to SpecFlow via NuGet. You need to be careful here because some of the SpecFlow packages use versions of Nunit which aren’t compatible with UITest or Test Cloud. The packages in the demo are…



  
  
  
  
  

Writing the Tests

Now we are ready to start writing the tests. Add a new SpecFlow Feature file to the test solution and add the appropriate Gherkin scenarios.

Now you can try to run the tests, but you’ll get a bunch of inconclusive results with the message ‘No matching step definition found for…’. You need to write the step definitions which are control the actions that occur for each step.

Add a SpecFlow Step Definition file called Steps.cs to the project. The example methods show how to bind a method to a step but they can all be replaced with steps based on the feature file. You can generate the code for the steps via a right click in the feature file.

Now you can define the steps. Here’s the file for the demo app…

using Xamarin.UITest;
using TechTalk.SpecFlow;

namespace SpecFlowDemo.UITests
{
    [Binding]
    public class Steps
    {
        private IApp app;

        [Given("the login page is displayed")]
        public void TheLoginPageIsDisplayed()
        {
            app = ConfigureApp.Android.StartApp();
            app.WaitForElement("LoginPage");
        }

        [When("Olivia enters the username Olivia")]
        public void OliviaEntersTheUsernameOlivia()
        {
            app.EnterText("UsernameEntry", "Olivia");
        }

        [When("she enters the password IHeartOwls")]
        public void WhenSheEntersThePasswordIHeartOwls()
        {
            app.EnterText("PasswordEntry", "IHeartOwls");
            app.DismissKeyboard();
        }

        [When("she enters the password IHateTrees")]
        public void WhenSheEntersThePasswordIHateTrees()
        {
            app.EnterText("PasswordEntry", "IHateTrees");
            app.DismissKeyboard();
        }

        [When("she presses the Logon button")]
        public void ShePressesTheLogonButton()
        {
            app.Tap("LoginButton");
        }

        [Then("the home screen is displayed")]
        public void TheHomeScreenIsDisplayed()
        {
            app.WaitForElement("MainPage");
        }

        [Then("an invalid username or password message is displayed")]
        public void AnInvalidUsernameOrPasswordMessageIsDisplayed()
        {
            app.WaitForElement("Logon Failed");
            app.WaitForElement("Invalid username or password");
        }
    }
}

Note that in the first step we set up the UITest application object. This works but it means that tests  can only run on either iOS or Android. Fortunately it’s easy to get it running on both…

Support for iOS and Android

To support both platforms we need to take advantage of the fact that SpecFlow is based around nunit, and that it creates partial classes for it’s test definitions. Add a new file called TestPartials.cs to the solution.

using System;
using NUnit.Framework;
using Xamarin.UITest;

namespace SpecFlowDemo.UITests
{
    public static class Global
    {
        public static Platform Platform { get; set; }
        public static IApp App { get; set; }
    }

    [TestFixture(Platform.iOS)]
    [TestFixture(Platform.Android)]
    public partial class LogInToAppFeature
    {
        public LogInToAppFeature(Platform platform)
        {
            Global.Platform = platform;
        }
    }
}

By adding a partial class for each feature like this you ensure they will run for both platforms. The Globals.Platform flag is set so that the app can be configured. The first step needs to be changed to pick this up…

[Given("the login page is displayed")]
public void TheLoginPageIsDisplayed()
{
    if (Global.Platform == Platform.iOS)
    {
        Global.App = ConfigureApp.iOS.StartApp();
    }
    else
    {
        Global.App = ConfigureApp.Android.StartApp();
    }

    Global.App.WaitForElement("LoginPage");
}

Code Improvements

So it’s all working nicely but there are a few things that can be improved. SpecFlow supports passing parameters to step definitions so we only need one definition to set the password. This is done by added a regular expression to capture the parameter to the definition’s attribute, and adding the parameter to the method’s signature. Here’s how it works for the password…

[When("she enters the password (.*)")]
public void WhenSheEntersThePassword(string password)
{
    Global.App.EnterText("PasswordEntry", password);
    Global.App.DismissKeyboard();
}

Another thing that’s worth considering is splitting the step definitions into separate files instead of big one. I have found that one file per screen is a good starting point. As long as the [Binding] attribute is present you can put the definitions wherever you want…

using Xamarin.UITest;
using TechTalk.SpecFlow;

namespace SpecFlowDemo.UITests
{
    [Binding]
    public class MainPageSteps
    {
        [Then("the main page is displayed")]
        public void TheMainPageIsDisplayed()
        {
            Global.App.WaitForElement("MainPage");
        }
    }
}

Typically you’ll want to run some steps at the start of each test to set up a clean environment, and start the app. Cucumber uses the Background keyword for this…

Feature: Log in to app

Background:
    Given a started app

Scenario: Olivia logs in to app
    Given the login page is displayed
    When Olivia enters the username Olivia
    ...
using Xamarin.UITest;
using TechTalk.SpecFlow;

namespace SpecFlowDemo.UITests
{
    [Binding]
    public class SetupSteps
    {
        [Given("a started app")]
        public void TheMainPageIsDisplayed()
        {
            if (Global.Platform == Platform.iOS)
            {
                Global.App = ConfigureApp.iOS.StartApp();
            }
            else
            {
                Global.App = ConfigureApp.Android.StartApp();
            }
        }
    }
}

So that’s it! BDD for mobile apps via Xamarin UITest and SpecFlow. Give it a go and let me know how you get on 🙂

Advertisements
BDD, Gherkin, SpecFlow and Xamarin UITest

Xamarin UITest – Streamlined Back Door Calls

Here’s a mini post – Making back door calls on iOS and Android is slightly different. Here’s an extension method that works on both…

 public static class AppExtensions
 {
    public static void CallBackdoorMethod(this IApp app, string method, object argument = null)
    {
        if (app is iOSApp)
        {
if (argument == null)
{
argument = "";
}
app.Invoke($"{method}:", argument);
        }
        else
        {
            app.Invoke(method, argument);
        }
    }
}
Xamarin UITest – Streamlined Back Door Calls

Xamarin Forms Cross Platform Libraries and Dependency Services

This post is about how to set up cross platform libraries for Xamarin Forms, and how to implement Dependency Services in those libraries. The source code for these examples are here.

The example app is a very simple page, which shows a time string that is taken from a cross platform library, and a culture that is taken from a platform specific library via a dependency service. It looks like this…

Screen Shot 2018-02-10 at 14.02.43

Cross Platform Libraries

Adding a cross platform library is easy – simply add a Xamarin.Forms class library project to your solution, and add a project reference to your cross platform application project. You can then use classes from the library as you would expect.

Platform Libraries & Dependency Services

Things get interesting when you want to add dependency services to the library. For this you need to add an iOS class library project, and an Android class library project to your solution. You also need to add the Xamarin.Forms to both projects.

Next you have to set up the references. These are the references to add…

Sample App References: Cross Platform Library
iOS App References: Cross Platform Library, iOS Library
Android App References: Cross Platform Library, Android Library
Cross Platform Library References: -
iOS Library References: Cross Platform Library, Xamarin Forms
Android Library References: Platform Library, Xamarin Forms

Now you can define the dependency service interface in the cross platform library…

using System;

namespace SampleLibrary
{
    public interface ICultureProvider
    {
        string GetCultureString();
    }
}

And implement the interface in the platform specific libraries, here’s the iOS implementation…

using System;
using SampleLibrary.iOS;
using Xamarin.Forms;
using System.Threading;

[assembly: Dependency(typeof(CultureProvider))]
namespace SampleLibrary.iOS
{
    public class CultureProvider : ICultureProvider
    {
        public CultureProvider()
        {
        }

        public string GetCultureString()
        {
            return Thread.CurrentThread.CurrentCulture.ToString();
        }
    }
}

Remember to add the Dependency attribute! This needs to reference the implementation class, not the interface.

Now you can use DependencyService.Get to get access the service…

var cultureProvider = DependencyService.Get();
var culture = cultureProvider.GetCultureString();

For a regular dependency service defined in the application projects this would be enough. However using libraries introduces an extra foible. At compile time the application specific library might not be included in the project because there’s no reference to it in code. If this DependencyService.Get returns null. To fix this you need to add some code to the platform specific application to reference the platform specific library. This can be as simple as creating an instance of one of the classes in the library…

[Register("AppDelegate")]
public partial class AppDelegate : global::Xamarin.Forms.Platform.iOS.FormsApplicationDelegate
{
    public override bool FinishedLaunching(UIApplication app, NSDictionary options)
    {
        global::Xamarin.Forms.Forms.Init();
        var foo = new SampleLibrary.iOS.CultureProvider();

        LoadApplication(new App());

        return base.FinishedLaunching(app, options);
    }
}

With the reference in place the dependency service will resolve correctly. This feels like a hack, but I haven’t found a better solution, if you have any ideas leave a comment!

Any better ideas??

Xamarin Forms Cross Platform Libraries and Dependency Services

Hyper-V and Haxm

Here’s a quick tip to make working with Hyper-V and Haxm in Windows a little easier. It’s not possible to run both at the same time which is a pain if, for example, you want to run Android emulators and Docker at the same time.

What you can do is set up a boot entry with Hyper-V turned off, then at start up you can choose to run with Hyper-V for Docker, or without to run your emulators.

To do this fire up an administrator command prompt and type

bcdedit /copy {current} /d "Hypervisor Off - Android Emulators On"

Next find and copy the identifier for the new boot entry by running

bcdedit

Then turn Hypervisor off with

bcdedit /set {newidentifier} hypervisorlaunchtype off

Restart the machine and you’ll be able to choose which one to boot 🙂

Hyper-V and Haxm

Xamarin UI Test – Testing State Management

State management, ensuring that an suspended app restarts to the same place, is one of the more challenging areas of mobile development. It’s a good idea to make sure it’s covered by your automated tests. Here’s how I managed this with Xamarin UITest…

The source code for this post is available here.

1 – Setup Tests to Run in Order

My example uses two tests, one to set the data up and one to make sure the same data is present in the restored app. The Order attribute in nunit ensures they run sequentially.

[Test, Order(1)]
public void StateTest()
{
    app = AppInitializer.StartApp(platform);
    app.Tap("Button");
    app.Tap("Button");
    app.EnterText("Text", "Some text");

    SaveAppState();
}

[Test, Order(2)]
public void StateTest2()
{
    app = AppInitializer.StartApp(platform, true);
    app.WaitForElement("Clicked 2 Times");
    app.WaitForElement(x => x.Marked("Text").Text("Some text"));
}

2 – Preserve Local Data Between Tests

In regular tests any local data will be wiped before the app is run, but here it needs to be preserved. A parameter added to the AppInitialiser controls this. This means that the local data from the first test is still in place for the second test.

public static IApp StartApp(Platform platform, bool preseveData = false)
{
    var dataMode = preseveData ? AppDataMode.DoNotClear : AppDataMode.Clear;

    if (platform == Platform.Android)
    {
        return ConfigureApp.Android
            .StartApp(dataMode);
    }
    else
    {
        return ConfigureApp.iOS
           .StartApp(dataMode);
    }
}

The documentation says that the AppDataMode attribute is only supported in test cloud, but I found it works with local tests as well.

3 – Suspend the App Programatically

When the app is stopped and restarted between tests it doesn’t get a chance to save state. To get round this the first test suspends the app programmatically. This is done via backdoor methods for each platform.

Here’s a way to suspend on iOS:

[Export("SuspendApp:")]
public void SuspendApp(NSString value)
{
    UIApplication.SharedApplication.PerformSelector(new ObjCRuntime.Selector("suspend"), null, 2.0);
}

Here’s a way to ‘suspend’ on Android (note this just triggers App.OnSleep, it doesn’t really suspend the app):

[Export("SuspendApp")]
public void SuspendApp()
{
    OnPause();
    OnStop();
}

Run the Tests!

With everything in place the last thing to do is run the tests – here’s how it looks on my machine 🙂

Xamarin UI Test – Testing State Management

Running Xamarin UITest Repl Outside of Tests

The Xamarin UITest Repl is a handy little tool for inspecting and interacting with the interface for iOS and Android apps. Normally you run it by adding app.Repl() to your test code but it’s also easy to run Repl outside of a test…

Android on Windows

To run Repl on an Android device connecting to Windows you need to know:

  • The location of the Repl executable (xut-repl.exe) – C:\Users\Username\AppData\Local\Temp\uitest\repl or similar
  • The location of the UITest dll, something like – C:\Users\Username.nuget\packages\Xamarin.UITest\2.0.10\lib\Xamarin.UITest.dll
  • The identifier for the connected device – you can get this by running adb devices

To start Repl open a command prompt and change to the executable folder. Then run xut-repl.exe android {uitestdllpath} {deviceid} after replacing the relevant values. For example:

xut-repl.exe android C:\Users\Username\.nuget\packages\Xamarin.UITest\2.0.10\lib\Xamarin.UITest.dll http://notused 15337521650191

Android and iOS on macOS

On macOS it’s a bit easier. Run Repl from a test then copy the title of the Repl window. Fire up a new terminal and enter the copied command.

Here’s an iOS example from my machine:

mono /var/folders/6l/bg1gb8v13mq5lgqgnj4_pdgw0000gn/T/uitest/repl/xut-repl.exe ios ~/Code/Trailmaker.Core/Trailmaker.Core.UITest/bin/Debug/Xamarin.UITest.dll http://127.0.0.1:37265 06C7B6BD-A7F2-436A-8BF9-A911F944A2CE true

And here’s an Android example:

mono /var/folders/6l/bg1gb8v13mq5lgqgnj4_pdgw0000gn/T/uitest/repl/xut-repl.exe android ~/Code/Trailmaker.Core/Trailmaker.Core.UITest/bin/Debug/Xamarin.UITest.dll http://127.0.0.1:60360 emulator-5554 ~Library/Developer/Xamarin/android-sdk-macosx
Running Xamarin UITest Repl Outside of Tests

Xamarin – Build on iOS from Windows Command Prompt/MSBuild

A quick post to describe how to build a Xamarin iOS project from MSBuild on a Windows machine. You still need a mac available but this approach means you can create an iOS artefact from a Windows TFS build agent.

On the command line run:

msbuild YourProject.sln /p:Configuration=AppStore;Platform=iPhone /p:ServerAddress=MacIpAddress /p:ServerUser=MacLogin /p:ServerPassword=MacPassword

When this is running keep an eye on the mac as it might pop up a message box or two on the first run.

After the script completes you should have a new IPA file in the bin\iPhone\Ad-Hoc directory of your iOS project on the Windows machine.

 

Xamarin – Build on iOS from Windows Command Prompt/MSBuild