Streamline your mobile testing with Maestro

By Ludovico Besana

Comparing Maestro with other popular tools for mobile UI automation

In the ever-evolving landscape of mobile app development, efficient and effective testing is crucial. Mobile testing ensures that applications work seamlessly across different devices and platforms. By choosing the right tools, teams can significantly enhance their productivity and the quality of their applications.

This article explores the features of Maestro, a promising tool for mobile UI automation, and compares it with other popular tools such as Robot Framework Appium Library and WebdriverIO.

Real-world application of Maestro

A common challenge in mobile UI testing is managing the complexity of test setups and ensuring consistent results across different environments. Maestro offers a solution by enabling users to perform tests using a simple YAML syntax, which can be easily understood and written by anyone in the organisation. This flexibility allows teams to quickly adapt to changes and make reliable decisions promptly.

A simple Maestro test case

Here is an example of how a Maestro test case is structured using YAML:

appId: org.wikipedia
---
- tapOn: "ADD OR EDIT.*"
- tapOn: "ADD LANGUAGE"
- tapOn:
    id: ".*menu_search_language"
- inputText: "French"
- assertVisible: "Français"
- tapOn: "Français"
- tapOn: "Back"

This example demonstrates a simple yet comprehensive test case for a hypothetical Wikipedia application:

1. appId: Specifies the ID of the application to be tested.

2. tapOn: "ADD OR EDIT.*": Simulates a tap on any element whose text matches the pattern "ADD OR EDIT".

3. tapOn: "ADD LANGUAGE": Simulates a tap on the "ADD LANGUAGE" button.

4. tapOn (with id): Simulates a tap on an element whose ID matches the pattern "menu_search_language".

5. inputText: "French": Inputs the text "French" into the input field.

6. assertVisible: "Français": Verifies that the text "Français" is visible on the screen.

7. tapOn: "Français": Simulates a tap on the "Français" option.

8. tapOn: "Back": Simulates a tap on the "Back" button to return to the previous screen.

Localised testing challenges

Running tests in different languages can introduce complications, and this is true for any testing approach. With Maestro, this issue arises when the simulator's language settings do not match the language expected by the test scripts. For instance, if your simulator is set to Italian and your tests are designed for an English interface, the tests will fail because the locators (which depend on text strings) will not match.

A running test with Maestro

To address this, Maestro provides the ability to set up and configure simulators for different locales. Here are some solutions:

• Locale configuration: Ensure that the simulator's locale matches the language of your test scripts. You can configure the locale using Maestro's command-line options:

maestro start-device --platform ios --device-locale en_US --os-version=15

• Use of IDs and universal locators: Instead of relying solely on text strings, use element IDs and universal locators that remain consistent across different languages.

• Parameterised tests: Create parameterised test scripts that can adapt to different locales by using variables for text strings, making it easier to run the same tests in multiple languages.

By implementing these strategies, you can minimise discrepancies and ensure more reliable test results across various language settings with Maestro.

Emulator setup

Setting up emulators for different locales and operating system versions can be a challenge with Maestro. For instance, initiating a test run on an iOS simulator in British English requires specific commands and configurations:

maestro start-device --platform ios --device-locale en_GB --os-version=17

It’s important to note that Maestro supports specific OS versions, such as iOS 15, 16, and 17, at the time of writing. For the most current list of supported versions, refer to the Maestro documentation. Setting up the appropriate emulator can be tricky, especially for those not familiar with mobile testing environments.

Flaky tests and element inspection

A critical aspect of mobile testing is the accurate inspection of elements within the application. Maestro Studio is a tool within Maestro that helps testers design and execute tests through a visual interface. However, relying solely on Maestro Studio may not always allow for precise element inspection, leading to flaky tests if elements are not correctly identified.

Element IDs are unique identifiers for UI elements within an app, similar to how IDs are used in web applications. These IDs are crucial for ensuring tests interact with the correct elements. Without access to the app’s codebase, it can be challenging to inspect and identify these elements accurately.

To address this, tools like Appium Inspector can be used alongside Maestro. Appium Inspector provides a comprehensive solution for inspecting app elements, allowing testers to accurately identify and interact with elements even without direct access to the codebase. A common practice is to use such inspectors to identify elements and reference the app’s codebase when possible to retrieve IDs.

By combining these tools, testers can ensure more reliable and accurate test results, reducing the likelihood of flaky tests.

Maestro Studio Locators Inspector

AI integration in Maestro

One of Maestro’s main features is its integration of artificial intelligence (AI), which assists users in various ways:

• Automated element identification: The AI helps automatically identify UI elements, reducing the time needed to write test cases.

• Intelligent test recommendations: Based on the application’s structure and previous test cases, Maestro's AI can suggest test steps and improvements, ensuring more comprehensive test coverage.

Thanks to these AI capabilities, Maestro not only reduces setup time but also enhances the reliability of tests. Teams can then focus more on the core functionalities of their applications, ensuring higher quality and a faster time-to-market. This integration of AI into the testing process is relatively new, making Maestro an innovative tool in the mobile testing landscape.

Maestro Studio AI Feature

Parallel test execution

While Maestro simplifies many aspects of mobile testing, it does not support running tests in parallel across different simulators directly. Even with two simulators and two terminals open, parallel execution is not possible. There is, however, the option to run tests in the cloud within a mobile development environment, although this incurs additional costs.

Comparing mobile testing tools

Choosing the right tool for mobile test automation can significantly impact efficiency, accessibility, and test maintenance. Here, we compare three main tools: Maestro, Robot Framework Appium Library, and WebdriverIO.

Maestro

Maestro is designed for simplicity and ease of use, making it ideal for teams with limited coding experience. Its YAML-based syntax allows for quick test creation and execution, which is particularly useful for small projects or teams new to automation. The integration of AI in Maestro Studio provides intelligent test recommendations and automated element identification, enhancing productivity and test reliability.

Robot Framework Appium Library

Robot Framework Appium Library leverages the power of Python and the flexibility of Robot Framework. It is suited for teams with a solid coding background who need a robust and versatile testing solution. This tool offers comprehensive support for various mobile testing scenarios but requires more setup and a steeper learning curve compared to Maestro.

WebdriverIO

WebdriverIO is a JavaScript-based framework that supports both web and mobile testing. It is highly extensible and integrates well with various CI/CD pipelines, making it suitable for teams already familiar with JavaScript. WebdriverIO strikes a balance between usability and flexibility, offering powerful features for both beginner and advanced testers.

Feature comparison table

When to use each tool

• Maestro: Recommended for teams seeking quick setup and ease of use, especially when team members have limited coding knowledge. Ideal for small to medium projects where speed and simplicity are crucial.

• Robot Framework Appium Library: Best for teams with strong coding skills who require a highly versatile and robust testing solution. Suitable for complex projects that demand extensive customisation.

• WebdriverIO: Great for teams with JavaScript expertise looking for a flexible tool that can handle both web and mobile testing. Perfect for projects that benefit from extensive integrations and a balance of ease of use and power.

By understanding the strengths and appropriate use cases for each tool, teams can make informed decisions to enhance their mobile testing strategy.

Final considerations

Based on our experience, we consider Maestro an exceptionally valuable tool for those new to mobile UI automation. Its simplicity in setup and use makes it particularly suitable for both small projects and those looking to gain a basic understanding of UI automation without facing a steep learning curve. However, for more complex projects, it might be necessary to opt for a more powerful and versatile tool.

Maestro serves as a precious resource for organisations aiming to improve the quality of their mobile applications through efficient and manageable testing processes. By reducing the complexity and time required to configure and execute tests, Maestro enables teams to focus more on innovation, reducing time to market and enhancing the quality of the final products.