SlideShare a Scribd company logo

WordPress Acceptance Testing, Solved!

•
•
Taylor Lovett
Taylor Lovett

Learn about WP Acceptance. A new framework that empowers developers and CI pipelines to test codebases using version controlled acceptance tests and sharable environments.

Related slideshows

Testing nightwatch, by David Torroija
Testing nightwatch, by David TorroijaTesting nightwatch, by David Torroija
Testing nightwatch, by David Torroija
 
Maven tutorial
Maven tutorialMaven tutorial
Maven tutorial
 
Getting up and running with selenium for automated Code palousa
Getting up and running with selenium for automated Code palousaGetting up and running with selenium for automated Code palousa
Getting up and running with selenium for automated Code palousa
 
1 of 27
Download to read offline
WordPress Acceptance Testing: Solved! https://github.com/10up/wpacceptance
What Is the Purpose of Automated Testing? • Testing helps us reduce bugs/regressions and ensures our application meets requirements. • Manual testing, while also useful and important, is slow, expensive, and prone to human error. • Automated testing can be integrated such that it is run continuously e.g. part of a development and delivery pipeline.
Types of Automated Testing • There are many different types of automated testing: unit, functional, performance, acceptance/end-to-end, integration, etc. • WordPress bundles integration/unit tests as well as end to end tests for certain features.
Automated Testing Tools • WordPress PHPUnit Test Suite - the framework used by core and one you can include in plugins/themes to write integration tests. Based on PHPUnit. • WP Mock - a project started by 10up for writing true unit tests for WordPress. Also based on PHPUnit. • A number of acceptance testing frameworks e.g. Codeception. Custom built ones e.g. Gutenberg’s end-to-end testing system. • JS testing frameworks
What Is Acceptance Testing? • Acceptance testing is testing that an application behaves as expected. • In the web application world, this usually means simulating a real user interacting with a web application via a browser.
WordPress Acceptance Testing, Solved!
The Problem • Acceptance testing is challenging because everyone and every system (CI) running the tests must be running the tests against the same codebase, environment, and database.
The Problem (cont.) • Codeception is a popular PHP framework for writing acceptance tests. The framework does not bundle any functionality around the environment or database sharing. • Different engineers on the team often run different local environments leading to test inconsistencies. • When an engineer runs the tests on their local, the tests are run against their local database. Different members of the engineering team have different local databases and it is tough to keep everyone in sync. • To make this work with CI you’d have to create a way for your CI tool to start a web environment and pass it predefined database exports. https://codeception.com
WP Acceptance Overview • WP Acceptance is a toolkit that empowers developers and CI pipelines to test codebases using version controlled acceptance tests and sharable environments. https://github.com/10up/wpacceptance
What Makes WP Acceptance Powerful? • WP Acceptance lets you create environment instructions to be version controlled with your tests. Every time you, a team member, or a CI pipeline run the tests, an environment will be created from those instructions guaranteeing the same results for everyone.
Environment Instructions • Environment instructions look like this:
 install wordpress where version is latest and url is http://url.test
 
 activate plugin where name is safe-redirect-manager
 install theme where name is twentynineteen
 
 create post where title is Test and content is “Test content” Read about supported instructions: https://github.com/10up/wpinstructions
wpacceptance.json • All WP Acceptance projects contain a file named wpacceptance.json. Here’s an example: { "name": "10up-experience", "exclude": [ "node_modules" ], "tests": [ "./tests/wpa/*.php" ], "enforce_clean_db": false, "project_path": "%WP_ROOT%/wp-content/plugins/10up-experience", [ "install wordpress where url is http://10upexperience.test", "activate plugin where name is 10up-experience", "install theme where name is twentynineteen" ] ] }
Snapshots • Sometimes you need to test in a complex environment e.g. you are building a site with 30 plugins and large database for a client. • For this, you will want to use a Snapshot. WP Acceptance lets you run your tests on Environment Instructions or Snapshot(s). • Snapshots are based on the WP Snapshots project: https://github.com/10up/wpsnapshots
Snapshots (cont.) • In the snapshot workflow you will need to prepare a “primary” snapshot for the development team to use. • You can run WPA against your local environment and save the snapshot:
 wpacceptance run --local --save_snapshot • This will update your wpacceptance.json file with the new snapshot ID so other developers/CI can use that snapshot. • When major changes are made to the site e.g. new content types, you will want to create a new snapshot.
wpacceptance.json • Here’s a wpacceptance.json file using WP Snapshots: { “name": "distributor", "exclude": [ "./node_modules" ], "tests": [ "tests/wpacceptance/*Test.php" ], "snapshots": [ { "snapshot_name": "WordPress 5.1.1", "snapshot_id": "fec81ac24227af6554c93d28d9af2bdf" } ], "enforce_clean_db": true, "bootstrap": "tests/wpacceptance/bootstrap.php", "repository": "10up" }
Install and Usage • WP Acceptance is added to a project via Composer. You can use it on a theme, plugin, or in any WordPress application. • Step 1: composer require 10up/wpacceptance • Step 2: Create your wpacceptance.json file (or run ./vendor/bin/wpacceptance init) • Step 3: ./vendor/bin/wpacceptance run
A Simple Test You can see a test lets us interact with the website as a user. If “Modal Title” is not visible after clicking .modal-link, the test will fail. $I = $this->openBrowserPage();
 $I->moveTo( '/' );
 $I->click( 'a.modal-link' );
 $I->seeText( 'Modal Title' );
Testing the Admin $I = $this->openBrowserPage();
 $I->login();
 $I->moveTo( '/wp-admin/profile.php' );
 $I->waitUntilElementVisible( '#wpadminbar' );
 $I->fillField( '#first_name', 'Test Name' );
 $I->click( '#submit' );
 $I->waitUntilElementVisible( '#wpadminbar' );
 $I->seeValueInAttribute( '#first_name', 'value', 'Test Name', 'Profile field did not update.' );
Saving a Post $I = $this->openBrowserPage(); $I->loginAs( 'admin' ); $I->moveTo( 'wp-admin/post-new.php?post_type=redirect_rule' ); $I->fillField( '#srm_redirect_rule_from', '/test' ); $I->fillField( '#srm_redirect_rule_to', '/test2' ); $I->fillField( '#srm_redirect_rule_notes', 'Notes' ); $I->click( '#publish' ); $I->waitUntilElementVisible( '.updated' ); $I->seeValueInProperty( '#srm_redirect_rule_from', 'value', '/test' ); $I->seeValueInProperty( '#srm_redirect_rule_to', 'value', '/test2' ); $I->seeValueInProperty( '#srm_redirect_rule_notes', 'value', 'Notes' ); (From Safe Redirect Manager)
Standard Test Suite • WP Acceptance includes standard tests that you can include in any test suite • The standard tests ensure basic functionality is working. This is useful as often times while building plugins and themes we can break things and not know about it.
Standard Test Suite • Test the front end loads properly. • Test that an admin can create a post. • Test the admin bar is showing. • Test users can update their profile. • Test users can update general settings. • Etc.
Standard Test Suite class MyTests extends WPAcceptancePHPUnitTestCase { /** * @testdox I can log in. */ public function testLogin() { parent::_testLogin(); } }
Continuous Integration • WP Acceptance works great in Travis. Here’s an example .travis.yml file: language: php php: - 7.2 before_script: - composer install script: - ./vendor/bin/wpacceptance run sudo: required services: docker
Debugging • WP Acceptance provides tools for debugging problematic tests locally. • Use verbose flag(s):
 wpacceptance run -v • Watch the browser as tests run:
 wpacceptance run --show_browser • Run the tests slower:
 wpacceptance run --slowmo=500
Examples • https://github.com/10up/distributor • https://github.com/10up/safe-redirect-manager • https://github.com/10up/10up-experience
Extensive Documentation https://wpacceptance.readthedocs.io/en/latest/
Questions? taylor.lovett@10up.com
 @tlovett12
 10up.com

More Related Content

WordPress Acceptance Testing, Solved!

  • 2. What Is the Purpose of Automated Testing? • Testing helps us reduce bugs/regressions and ensures our application meets requirements. • Manual testing, while also useful and important, is slow, expensive, and prone to human error. • Automated testing can be integrated such that it is run continuously e.g. part of a development and delivery pipeline.
  • 3. Types of Automated Testing • There are many different types of automated testing: unit, functional, performance, acceptance/end-to-end, integration, etc. • WordPress bundles integration/unit tests as well as end to end tests for certain features.
  • 4. Automated Testing Tools • WordPress PHPUnit Test Suite - the framework used by core and one you can include in plugins/themes to write integration tests. Based on PHPUnit. • WP Mock - a project started by 10up for writing true unit tests for WordPress. Also based on PHPUnit. • A number of acceptance testing frameworks e.g. Codeception. Custom built ones e.g. Gutenberg’s end-to-end testing system. • JS testing frameworks
  • 5. What Is Acceptance Testing? • Acceptance testing is testing that an application behaves as expected. • In the web application world, this usually means simulating a real user interacting with a web application via a browser.
  • 7. The Problem • Acceptance testing is challenging because everyone and every system (CI) running the tests must be running the tests against the same codebase, environment, and database.
  • 8. The Problem (cont.) • Codeception is a popular PHP framework for writing acceptance tests. The framework does not bundle any functionality around the environment or database sharing. • Different engineers on the team often run different local environments leading to test inconsistencies. • When an engineer runs the tests on their local, the tests are run against their local database. Different members of the engineering team have different local databases and it is tough to keep everyone in sync. • To make this work with CI you’d have to create a way for your CI tool to start a web environment and pass it predefined database exports. https://codeception.com
  • 9. WP Acceptance Overview • WP Acceptance is a toolkit that empowers developers and CI pipelines to test codebases using version controlled acceptance tests and sharable environments. https://github.com/10up/wpacceptance
  • 10. What Makes WP Acceptance Powerful? • WP Acceptance lets you create environment instructions to be version controlled with your tests. Every time you, a team member, or a CI pipeline run the tests, an environment will be created from those instructions guaranteeing the same results for everyone.
  • 11. Environment Instructions • Environment instructions look like this:
 install wordpress where version is latest and url is http://url.test
 
 activate plugin where name is safe-redirect-manager
 install theme where name is twentynineteen
 
 create post where title is Test and content is “Test content” Read about supported instructions: https://github.com/10up/wpinstructions
  • 12. wpacceptance.json • All WP Acceptance projects contain a file named wpacceptance.json. Here’s an example: { "name": "10up-experience", "exclude": [ "node_modules" ], "tests": [ "./tests/wpa/*.php" ], "enforce_clean_db": false, "project_path": "%WP_ROOT%/wp-content/plugins/10up-experience", [ "install wordpress where url is http://10upexperience.test", "activate plugin where name is 10up-experience", "install theme where name is twentynineteen" ] ] }
  • 13. Snapshots • Sometimes you need to test in a complex environment e.g. you are building a site with 30 plugins and large database for a client. • For this, you will want to use a Snapshot. WP Acceptance lets you run your tests on Environment Instructions or Snapshot(s). • Snapshots are based on the WP Snapshots project: https://github.com/10up/wpsnapshots
  • 14. Snapshots (cont.) • In the snapshot workflow you will need to prepare a “primary” snapshot for the development team to use. • You can run WPA against your local environment and save the snapshot:
 wpacceptance run --local --save_snapshot • This will update your wpacceptance.json file with the new snapshot ID so other developers/CI can use that snapshot. • When major changes are made to the site e.g. new content types, you will want to create a new snapshot.
  • 15. wpacceptance.json • Here’s a wpacceptance.json file using WP Snapshots: { “name": "distributor", "exclude": [ "./node_modules" ], "tests": [ "tests/wpacceptance/*Test.php" ], "snapshots": [ { "snapshot_name": "WordPress 5.1.1", "snapshot_id": "fec81ac24227af6554c93d28d9af2bdf" } ], "enforce_clean_db": true, "bootstrap": "tests/wpacceptance/bootstrap.php", "repository": "10up" }
  • 16. Install and Usage • WP Acceptance is added to a project via Composer. You can use it on a theme, plugin, or in any WordPress application. • Step 1: composer require 10up/wpacceptance • Step 2: Create your wpacceptance.json file (or run ./vendor/bin/wpacceptance init) • Step 3: ./vendor/bin/wpacceptance run
  • 17. A Simple Test You can see a test lets us interact with the website as a user. If “Modal Title” is not visible after clicking .modal-link, the test will fail. $I = $this->openBrowserPage();
 $I->moveTo( '/' );
 $I->click( 'a.modal-link' );
 $I->seeText( 'Modal Title' );
  • 18. Testing the Admin $I = $this->openBrowserPage();
 $I->login();
 $I->moveTo( '/wp-admin/profile.php' );
 $I->waitUntilElementVisible( '#wpadminbar' );
 $I->fillField( '#first_name', 'Test Name' );
 $I->click( '#submit' );
 $I->waitUntilElementVisible( '#wpadminbar' );
 $I->seeValueInAttribute( '#first_name', 'value', 'Test Name', 'Profile field did not update.' );
  • 19. Saving a Post $I = $this->openBrowserPage(); $I->loginAs( 'admin' ); $I->moveTo( 'wp-admin/post-new.php?post_type=redirect_rule' ); $I->fillField( '#srm_redirect_rule_from', '/test' ); $I->fillField( '#srm_redirect_rule_to', '/test2' ); $I->fillField( '#srm_redirect_rule_notes', 'Notes' ); $I->click( '#publish' ); $I->waitUntilElementVisible( '.updated' ); $I->seeValueInProperty( '#srm_redirect_rule_from', 'value', '/test' ); $I->seeValueInProperty( '#srm_redirect_rule_to', 'value', '/test2' ); $I->seeValueInProperty( '#srm_redirect_rule_notes', 'value', 'Notes' ); (From Safe Redirect Manager)
  • 20. Standard Test Suite • WP Acceptance includes standard tests that you can include in any test suite • The standard tests ensure basic functionality is working. This is useful as often times while building plugins and themes we can break things and not know about it.
  • 21. Standard Test Suite • Test the front end loads properly. • Test that an admin can create a post. • Test the admin bar is showing. • Test users can update their profile. • Test users can update general settings. • Etc.
  • 22. Standard Test Suite class MyTests extends WPAcceptancePHPUnitTestCase { /** * @testdox I can log in. */ public function testLogin() { parent::_testLogin(); } }
  • 23. Continuous Integration • WP Acceptance works great in Travis. Here’s an example .travis.yml file: language: php php: - 7.2 before_script: - composer install script: - ./vendor/bin/wpacceptance run sudo: required services: docker
  • 24. Debugging • WP Acceptance provides tools for debugging problematic tests locally. • Use verbose flag(s):
 wpacceptance run -v • Watch the browser as tests run:
 wpacceptance run --show_browser • Run the tests slower:
 wpacceptance run --slowmo=500