Test

This topic was updated due to the 2.2.0 MFTF release.

Test cases in the Magento Functional Testing Framework (MFTF) are defined in XML as <tests>. <tests> is a Codeception test container that contains multiple individual tests with test metadata and before and after actions.

MFTF <tests> is considered a sequence of actions with associated parameters. Any failed assertion within a test constitutes a failed test.

<before> and <after> hooks are not global within <tests> like in MFTF 1.0. They only apply to the <test> in which they are declared.

The steps in <after> are run in both successful and failed test runs.

The following diagram shows the structure of an MFTF test case:

Format

The format of <tests> is:

<?xml version="1.0" encoding="UTF-8"?>

<tests xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:noNamespaceSchemaLocation="../../../../../../vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Test/etc/testSchema.xsd">
        <test name="">
            <annotations>
                <!-- TEST ANNOTATIONS -->
            </annotations>
            <before>
                <!-- ACTIONS AND ACTION GROUPS PERFORMED BEFORE THE TEST -->
            </before>
            <after>
                <!-- ACTIONS AND ACTION GROUPS PERFORMED AFTER THE TEST -->
            </after>
            <!-- TEST ACTIONS, ACTION GROUPS, AND ASSERTIONS-->
        </test>
    </tests>
</config>

Principles

The following conventions apply to MFTF tests:

All names within the framework are in the CamelCase format.
<test> name must be alphanumeric.
Each action and action group has its own identifier <stepKey> for reference purposes.
A test may have any number of assertions, at any point within the <test>.
The file name and <test> name are equal.
If <test> is included in <suite>, it cannot be generated in isolation to the rest of the contents of the suite (see suites for details).

Multiple tags per XML file can make it hard to find and organize tags. To simplify, we generate one `test.php` file per tag provided, though we support both single and multiple tags per XML file.

Elements reference

There are several XML elements that are used in <tests> in the MFTF.

tests

<tests> is a container for multiple tests. It is a group of test methods that define test flows within a test case.

<tests> must contain at least one <test>.

test

<test> is a set of steps, including actions and assertions. It is a sequence of test steps that define test flow within a test method.

Attribute	Type	Use	Description
`name`	string	optional	A test identifier used while merging.
`remove`	boolean	optional	Set `true` to remove the test when merging.

<test> may also contain <annotations>, <before>, <after>, any action, or <actionGroup>.

annotations

Annotations are supported by both Codeception and Allure.

Codeception annotations typically provide metadata and are able to influence test selection. Allure annotations provide metadata for reporting.

See more about Annotations.

before

<before> wraps the steps to perform before the <test>.

<before> may contain these child elements:

Any <action>
<actionGroup>

after

<after> wraps the steps to perform after the <test>. The steps are run in both successful and failed test runs.

<after> may contain:

Any <action>
<actionGroup>

actionGroup

<actionGroup> calls a corresponding action group.

Attribute	Type	Use	Description
`ref`	string	required	References the required action group by its `name`.
`stepKey`	string	required	Identifies the element within `<test>`.
`before`	string	optional	`<stepKey>` of an action or action group that must be executed next while merging.
`after`	string	optional	`<stepKey>` of an action or action group that must be executed one step before the current one while merging.

<actionGroup> may contain <argument>.

argument

<argument> sets an argument that is used in the parent <actionGroup>.

Attribute	Type	Use
`name`	string	optional	Name of the argument.
`value`	string	optional	Value of the argument.

See more about Action groups.

Edit this page on GitHub

Give us feedback