Building a Robust Automated Test Suite with Behave and Selenium

Automated testing is an essential aspect of modern software development, ensuring that applications function correctly and efficiently. In this article, we will explore how to create a robust automated test suite using Behave and Selenium. We will incorporate best practices to make our tests maintainable, efficient, and reliable.

Introduction to Behave, Selenium, and BDD

Behavior-Driven Development (BDD) is an approach that enhances collaboration between developers, testers, and business stakeholders. It encourages writing tests in plain language, making them easy to understand for everyone involved.

Behave is a popular BDD framework for Python that enables writing such tests. Selenium is a powerful tool for automating web browsers, allowing us to simulate user interactions with our web applications.

By combining Behave and Selenium, we can create an automated test suite that verifies the functionality of our web application in a user-friendly manner.

This article will guide you through setting up a project structure, writing feature files, implementing step definitions, and configuring the environment for running tests.

Creating a Simple Website for Testing

To effectively test the login and forgot password functionalities, a simple website is created. This website serves as the application under test and provides the necessary pages and elements to interact with.

The website consists of:

• login.html: A login page with fields for username and password, a login button, and a link to the forgot password page.
• forgot_password.html: A forgot password page with an email input field and a submit button.
• index.html: A homepage that users are redirected to upon successful login.

You can find the source code for this simple website in the GitHub repository here.

From Manual to Automated Testing

Before delving into the technical implementation, let’s understand the transformation from manual testing to automated testing and why this transition is beneficial.

Manual Testing Process

Previously, testing a login functionality manually would involve the following steps:

1. Navigate to the login page: Open the browser and type the URL of the login page.
2. Enter credentials: Manually input the username and password.
3. Click the login button: Use the mouse to click the login button.
4. Verify the result: Observe the resulting page to see if the login was successful or if an error message appeared.
5. Repeat for different scenarios: Change the credentials to test various scenarios such as valid login, invalid login, and locked account.

This manual process is time-consuming, prone to human error, and inefficient for regression testing.

Automated Testing Transformation

Automating this process involves writing scripts that perform the above steps programmatically. These scripts can be run repeatedly with consistent accuracy, saving time and effort.

Cucumber is a software tool that supports Behavior-Driven Development (BDD). It allows you to write tests in plain language that non-programmers can read and understand. Cucumber itself is language-agnostic, meaning it can be used with different programming languages through various implementations.

How Cucumber Helps in Automation

Cucumber plays a crucial role in transforming manual testing processes into automated workflows. Here’s how Cucumber aids in this transformation:

• Readable Syntax: Tests are written in a natural language style using Gherkin, making them accessible to all stakeholders.
• Executable Specifications: The plain text descriptions of application behavior serve as both documentation and executable tests. This means the scenarios you write can be directly executed by the Cucumber framework, ensuring the documentation and tests are always in sync.
• Integration with Different Languages: Cucumber can be used with multiple programming languages like Java, Ruby, JavaScript, and Python (via Behave or other libraries).
• Integration with Automation Tools: Cucumber integrates seamlessly with various automation tools and frameworks, such as Selenium for browser automation. This integration allows you to automate the steps defined in your scenarios, simulating real user interactions with the application.
• Regression Testing: Automated Cucumber tests can be run as part of a continuous integration pipeline, ensuring that any new changes do not break existing functionality.

Key Concepts in Cucumber

• Feature Files is a cornerstone of BDD. It contains the behaviours of the application described in a structured format called Gherkin. Each feature file describes a single feature of the application and contains multiple scenarios, each representing a particular use case.
• Gherkin Language: A plain-text language used to describe features, scenarios, and steps in a structured way.
• Step Definitions: Code that defines what each step in your Gherkin scenarios does.
• Hooks: Code blocks that run at certain points in the Cucumber test cycle, such as before or after a scenario or step.
• Tags: Labels that can be assigned to scenarios or features to categorize and selectively run them.

Best Practices for Designing Feature Files

Designing feature files is a critical aspect of Behavior-Driven Development (BDD). A well-designed feature file ensures clarity, maintainability, and effective communication among team members. Here are some best practices and things to consider when designing feature files:

1. Use Clear and Concise Language

• Simple Language: Write steps in plain English (or the language of your team) to ensure non-technical stakeholders can understand the scenarios.
• Avoid Jargon: Avoid technical terms or domain-specific jargon unless necessary. If you must use them, ensure they are well-defined.

2. Follow the Gherkin Syntax

• Given-When-Then Structure: Use the Given-When-Then structure to describe the steps of your scenario.
• Given: Sets up the initial context.
• When: Describes the action or event.
• Then: Defines the expected outcome.
• And/But: Use these keywords to add additional steps within Given, When, or Then to maintain readability.

3. Keep Scenarios Short and Focused

• Single Behavior: Each scenario should test a single behavior or functionality.
• Avoid Long Scenarios: Long scenarios are harder to read and maintain. Break them into smaller, more focused scenarios if necessary.

4. Use Background for Repeated Steps

• Background Section: Use the Background section to define steps that are common to all scenarios in a feature file. This reduces duplication and keeps scenarios clean.

Example:

Feature: User Login

  Background:
    Given the user is on the login page

  Scenario: Successful login
    When the user enters valid credentials
    And clicks the login button
    Then the user should be redirected to the homepage

  Scenario: Unsuccessful login with invalid credentials
    When the user enters invalid credentials
    And clicks the login button
    Then an error message should be displayed

5. Use Scenario Outlines for Data-Driven Testing

• Scenario Outline: Use Scenario Outline to run the same scenario with different sets of data. This reduces duplication and makes it easy to test multiple conditions.

Example:

Scenario Outline: User login with different credentials
  Given the user is on the login page
  When the user enters <username> and <password>
  And clicks the login button
  Then the user should see <message>

  Examples:
    | username      | password        | message                        |
    | valid_user    | valid_password  | Homepage                       |
    | invalid_user  | invalid_password| Invalid username or password.  |
    | locked_user   | locked_password | Your account is locked.        |

6. Make Scenarios Independent

• Isolated Scenarios: Ensure that each scenario is independent and can be run in any order. Do not assume any state left by a previous scenario.

7. Write Declarative Steps

• Focus on What, Not How: Write steps that describe what the system does rather than how it does it. This makes scenarios more readable and maintainable.

Example:

# Good
When the user logs in

# Bad
When the user enters username and password and clicks the login button

8. Keep Feature Files Organized

• One Feature per File: Each feature file should focus on a single feature of the application.
• Logical Grouping: Group related scenarios together within a feature file.

9. Use Tags for Organization

• Tagging: Use tags to categorize and filter scenarios. Tags can be used to run specific groups of tests.

Example:

@smoke
Feature: User Login

@positive
Scenario: Successful login
  ...

@negative
Scenario: Unsuccessful login with invalid credentials
  ...

8. Use Hooks for Setup and Teardown

• Before/After Hooks: Use hooks to set up and clean up before and after scenarios. This ensures that each test runs in a controlled environment.

Example:

from behave import fixture, use_fixture

@fixture
def browser(context):
    context.browser = webdriver.Chrome()
    yield context.browser
    context.browser.quit()

def before_scenario(context, scenario):
    use_fixture(browser, context)

def after_scenario(context, scenario):
    context.browser.quit()

9. Maintain Clean and DRY Step Definitions

• Don’t Repeat Yourself (DRY): Avoid duplicating step definitions. Reuse existing steps where possible.
• Clear Naming: Use clear and descriptive names for step definition methods.

Example:

from behave import given, when, then

@given('the user is on the login page')
def step_given_user_on_login_page(context):
    context.browser.get('http://localhost:8000/login.html')

@when('the user logs in with valid credentials')
def step_when_user_logs_in_with_valid_credentials(context):
    context.browser.find_element_by_name('username').send_keys('valid_user')
    context.browser.find_element_by_name('password').send_keys('valid_password')
    context.browser.find_element_by_name('login').click()

Automated Testing for a Simple Website Using Behave and Selenium

Let’s go through a detailed example of how to set up automated tests for a simple website using Behave and Selenium. We’ll cover the entire process, from project setup to running the tests.

Project Structure

Organizing your project structure is the first step towards maintainable tests.

my_cucumber_project/
├── features/
│   ├── steps/
│   │   └── step_definitions.py
│   ├── pages/
│   │   ├── login_page.py
│   │   └── forgot_password_page.py
│   ├── environment.py
│   ├── login.feature
│   └── forgot_password.feature
├── config/
│   ├── config.py
│   ├── logger.py
│   └── retry.py
├── test_data.json
├── behave.ini
├── requirements.txt
├── conftest.py
└── tests/
    └── test_login.py

Setting Up Feature Files

Feature files describe the application's behavior in a structured language known as Gherkin. Each feature file contains scenarios that define the steps to test specific functionality.

features/login.feature

Feature: Login functionality

  Scenario Outline: User logs in with different credentials
    Given I am on the login page
    When I enter <username> and <password>
    And I click on the login button
    Then I should see <message>

    Examples:
      | username      | password        | message                        |
      | valid_user    | valid_password  | Homepage                       |
      | invalid_user  | invalid_password| Invalid username or password.  |
      | locked_user   | locked_password | Your account is locked.        |

features/forgot_password.feature

Feature: Forgot password functionality

  Scenario: User requests a password reset
    Given I am on the login page
    When I click on the forgot password link
    And I enter my email address
    And I click on the submit button
    Then I should see a password reset confirmation message

Implementing Step Definitions

Step definitions map the steps in your feature files to Python functions. This is where Selenium comes into play, allowing us to interact with the web application.

features/steps/step_definitions.py

from behave import given, when, then
from features.pages.login_page import LoginPage
from features.pages.forgot_password_page import ForgotPasswordPage
from config.config import BASE_URL, EMAIL
from config.logger import get_logger
from config.retry import retry

logger = get_logger()

def before_scenario(context, scenario):
    context.browser.get(f'{BASE_URL}/login.html')
    context.login_page = LoginPage(context.browser)
    context.forgot_password_page = ForgotPasswordPage(context.browser)
    logger.info(f"Starting scenario: {scenario.name}")

def after_scenario(context, scenario):
    context.browser.quit()
    logger.info(f"Ending scenario: {scenario.name}")

@given('I am on the login page')
def step_given_on_login_page(context):
    context.browser.get(f'{BASE_URL}/login.html')
    logger.info("Navigated to login page")

@when('I enter {username} and {password}')
def step_when_enter_credentials(context, username, password):
    retry(lambda: context.login_page.enter_username(username))
    retry(lambda: context.login_page.enter_password(password))
    logger.info(f"Entered username: {username} and password: {password}")

@when('I click on the login button')
def step_when_click_login_button(context):
    retry(lambda: context.login_page.click_login())
    logger.info("Clicked on login button")

@then('I should see {message}')
def step_then_see_message(context, message):
    if message == "Homepage":
        assert context.browser.current_url == f'{BASE_URL}/index.html'
    else:
        error_message = context.login_page.get_error_message()
        assert error_message == message
    logger.info(f"Expected message: {message} displayed")

@when('I click on the forgot password link')
def step_when_click_forgot_password_link(context):
    retry(lambda: context.login_page.click_forgot_password())
    logger.info("Clicked on forgot password link")

@when('I enter my email address')
def step_when_enter_email_address(context):
    retry(lambda: context.forgot_password_page.enter_email(EMAIL))
    logger.info(f"Entered email address: {EMAIL}")

@when('I click on the submit button')
def step_when_click_submit_button(context):
    retry(lambda: context.forgot_password_page.click_submit())
    logger.info("Clicked on submit button")

@then('I should see a password reset confirmation message')
def step_then_see_password_reset_confirmation(context):
    confirmation_message = context.forgot_password_page.get_confirmation_message()
    assert confirmation_message == 'A password reset link has been sent to your email.'
    logger.info("Password reset confirmation message displayed")

Implementing Page Objects

The Page Object Model (POM) pattern helps in encapsulating the page elements and interactions, promoting code reuse and maintainability.

features/pages/login_page.py

from selenium.webdriver.common.by import By

class LoginPage:
    def __init__(self, browser):
        self.browser = browser

    def enter_username(self, username):
        self.browser.find_element(By.NAME, 'username').send_keys(username)

    def enter_password(self, password):
        self.browser.find_element(By.NAME, 'password').send_keys(password)

    def click_login(self):
        self.browser.find_element(By.NAME, 'login').click()

    def click_forgot_password(self):
        self.browser.find_element(By.ID, 'forgot-password').click()

    def get_error_message(self):
        return self.browser.find_element(By.ID, 'error').text

features/pages/forgot_password_page.py

from selenium.webdriver.common.by import By

class ForgotPasswordPage:
    def __init__(self, browser):
        self.browser = browser

    def enter_email(self, email):
        self.browser.find_element(By.NAME, 'email').send_keys(email)

    def click_submit(self):
        self.browser.find_element(By.NAME, 'submit').click()

    def get_confirmation_message(self):
        return self.browser.find_element(By.ID, 'confirmation').text

Configuring Environment

Configuration files help manage URLs, credentials, and other settings, while logging and retry mechanisms enhance test reliability.

config/config.py

import os

BASE_URL = 'http://localhost:8000'
EMAIL = os.getenv('EMAIL', 'user@example.com')

config/logger.py

import logging

def get_logger():
    logger = logging.getLogger("behave_logger")
    logger.setLevel(logging.DEBUG)
    handler = logging.StreamHandler()
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    return logger

config/retry.py

from time import sleep

def retry(func, retries=3, delay=1):
    for _ in range(retries):
        try:
            return func()
        except Exception as e:
            sleep(delay)
    raise e

Managing Test Data

Using external files for test data enhances data-driven testing.

test_data.json

[
    {"username": "valid_user", "password": "valid_password", "message": "Homepage"},
    {"username": "invalid_user", "password": "invalid_password", "message": "Invalid username or password."},
    {"username": "locked_user", "password": "locked_password", "message": "Your account is locked."}
]

Setting Up Behave and Selenium

behave.ini

[behave]
default_tags = ~@wip

conftest.py

import pytest
from selenium import webdriver

@pytest.fixture
def browser():
    driver = webdriver.Chrome()
    yield driver
    driver.quit()

Creating the Test File

We need to create a test file that will integrate with pytest-bdd and Behave. This file will define how pytest should run your feature files.

tests/test_login.py

import pytest
from pytest_bdd import scenarios, given, when, then, parsers
from selenium import webdriver
from features.pages.login_page import LoginPage
from features.pages.forgot_password_page import ForgotPasswordPage
from config.config import BASE_URL, EMAIL
from config.logger import get_logger
from config.retry import retry

logger = get_logger()

# Load scenarios from feature files
scenarios('../features/login.feature')
scenarios('../features/forgot_password.feature')

@pytest.fixture
def browser():
    driver = webdriver.Chrome()
    yield driver
    driver.quit()

@given('I am on the login page')
def step_given_on_login_page(browser):
    browser.get(f'{BASE_URL}/login.html')
    global login_page
    global forgot_password_page
    login_page = LoginPage(browser)
    forgot_password_page = ForgotPasswordPage(browser)
    logger.info("Navigated to login page")

@when(parsers.parse('I enter {username} and {password}'))
def step_when_enter_credentials(username, password):
    retry(lambda: login_page.enter_username(username))
    retry(lambda: login_page.enter_password(password))
    logger.info(f"Entered username: {username} and password: {password}")

@when('I click on the login button')
def step_when_click_login_button():
    retry(lambda: login_page.click_login())
    logger.info("Clicked on login button")

@then(parsers.parse('I should see {message}'))
def step_then_see_message(message):
    if message == "Homepage":
        assert login_page.browser.current_url == f'{BASE_URL}/index.html'
    elif message == "A password reset link has been sent to your email.":
        confirmation_message = forgot_password_page.get_confirmation_message()
        assert confirmation_message == message
    else:
        error_message = login_page.get_error_message()
        assert error_message == message
    logger.info(f"Expected message: {message} displayed")

@when('I click on the forgot password link')
def step_when_click_forgot_password_link():
    retry(lambda: login_page.click_forgot_password())
    logger.info("Clicked on forgot password link")

@when('I enter my email address')
def step_when_enter_email_address():
    retry(lambda: forgot_password_page.enter_email(EMAIL))
    logger.info(f"Entered email address: {EMAIL}")

@when('I click on the submit button')
def step_when_click_submit_button():
    retry(lambda: forgot_password_page.click_submit())
    logger.info("Clicked on submit button")

Running the Tests

Ensure the web server is running locally:

cd simple_website
python -m http.server 8000

Run the tests using Behave:

behave

Or using pytest:

pytest

Conclusion

By following these best practices, we can create a robust, maintainable, and efficient automated test suite using Behave and Selenium. This setup ensures that our tests are easy to understand, reliable, and capable of catching issues effectively.

Implementing behavior-driven development with tools like Behave and Selenium not only enhances test coverage but also bridges the gap between technical and non-technical stakeholders, promoting better collaboration and understanding of the application’s behavior. This comprehensive guide provides a solid foundation for setting up and running automated tests in a way that ensures your web applications remain reliable and user-friendly.

Transitioning from manual to automated testing significantly improves the efficiency and reliability of your testing process. Manual testing is labor-intensive and prone to human error, while automated testing with Behave and Selenium ensures consistency, speed, and accuracy, making it an indispensable part of modern software development.

The complete source code is available for you to download. You can try downloading the sample website and test cases from the links below to get hands-on experience:

1. Website To Test: The actual website to be tested
2. Tests : The test cases to be executed and tested.