Angular DevOps Series (PART 4)

Are you searching for a solution for automating the Angular application development process? In this series, I’m taking you along with me in my journey towards a fully automated process for my Angular application. The series includes 4 parts:

Part 1: Establish Workflows from Source Code to Staging Environment Using GitHub, CircleCI, Docker and Firebase.

Part 2: First Quality Gate: Static Testing.

Part 3: Second Quality Gate: Unit Testing.

Part 4: Third Quality Gate: E2E Testing.

Let’s go!


Angular DevOps – Part 4: Third Quality Gate – E2E Testing.

Overview

So far, we have gone together to establish and integrate the quality gates Static Testing and Unit Testing, which enforce a quality policy for source code and prevent code smells going inside the Auto Build and Auto Deploy pipelines. Today, we will finish by establishing the quality gate: E2E Testing.

Let’s move on.

Auto Test Pipeline

We talked about the Auto Test pipeline in Angular DevOps – Part 3 with a goal of triggering the Unit Testing gate.

Now, the source code, which passed the Unit Testing gate and compiled with Auto Build pipeline, is ready for deploying. So, the Auto Test pipeline is staying to focus on a goal, and that goal is to immediately trigger E2E Testing after the Auto Build pipeline is completed. Any errors will be reported to developers and update the status on the pull request.

E2E Testing

E2E Testing is a testing methodology used to test whether or not an application is performing from start to finish as expected. Goals of implementing E2E Testing in our Angular project are:

  • Building test environment using Docker and Docker Compose.
  • Executing E2E test scripts to test the ZeroToHero application.

The Angular CLI has already configured end-to-end testing on a new project with Protractor and Jasmine framework. You can use default E2E framework from the Angular CLI to implement test scripts. However, in this article, I’m going to use another framework: Protractor Cucumber Framework, which is integrated to a Node.js project.

It doesn’t matter what is used, but we are going to make some test scripts to test the Login feature of ZeroToHero application together. I use Gherkin syntax and Cucumber framework to describe 2 test scenarios. To learn more about Gherkin syntax and Cucumber framework, refer to Cucumber Docs.

Feature: To login ZTH application

@LoginFeature
@OutlineScenario
Scenario Outline: Login successfully with a valid username and password
  Given I am on login page
  When I do login with email is "<email>" and password is "<password>"
  Then I am on landing page after login successfully

  Examples:
  | email | password |
  | test@gmail.com | mwd@12345jdfd |


@LoginFeature
@OutlineScenario
Scenario Outline: Login unsuccessfully with a wrong password
  Given I am on login page
  When I do login with email is "<email>" and password is "<wrong_password>"
  Then I am still on login page with an error message

  Examples:
  | email | wrong_password |
  | test@gmail.com | mwd@12345wrong |

Figure 1: login.feature file (A cucumber feature file for list of login test scenarios)

Next, we need to implement test steps for these scenarios using Typescript language and Cucumber framework. Below is a step example.

Given("I am on login page", async () => {
    await loginPage.openLoginPage();
});

Figure 2: An example for step definition method

By using Protractor to run tests, we always create a configuration file, protractor.config.ts, to setup testing parameters and integrate external plugins, such as Cucumber HTML Report. In order to help the Protractor framework understand tests and configuration files, we have to compile Typescript to Javascript, using the command tsc. I wrapped this command with an npm script build in package.json file to easily extend it to the building process and execute with NPM.

npm run build

After that, you can run tests using the protractor command with the first parameter as a protractor.config.js file and view a test report. Please ensure that the Selenium Server is always started with commands webdriver-manager update and webdriver-manager start.

Figure 3: Start a Selenium Server on a local machine

protractor build/protractor.config.js

Figure 4: Execute E2E test scripts and generate an E2E Allure HTML report

E2E Testing Gate in CircleCI

Everything seems simple when the E2E Testing is setup and executed from a local environment with test environment, Selenium Server, Web Browser, AUT, and ready test scripts. However, to establish the quality gate E2E Testing in CircleCI, a CI system that supports Docker executor, we need to answer the questions below:

  • How do we establish test environment inside Docker containers?
  • How do we run test scripts on the established test environment?

The answer for the first question is Docker ComposeDocker Compose is a wrapper around the Docker CLI in order to define and run multi-container Docker applications. It uses docker-compose.yml file in order to retrieve parameters. Let’s open the docker-compose.yml file to define services, which will be built as Docker containers for Selenium Hub, Selenium Node, and AUT.

As you see, I’m planning to build Selenium Grid with hub and nodes. By doing it this way, you can run multiple tests at the same time against different browsers running on different machines using different operating systems.

Now, you are able to use content below and save as a docker-compose.yml file. The docker-compose.yml file will be executed in CI pipelines of the ZeroToHero source code. So, please put it inside the root folder of application source code.

version: '2'
services:
  hub:
    image: selenium/hub
    container_name: hub
    expose:
      - 4444
    ports:
      - 4444:4444
    networks:
      - backend
  chrome:
    image: selenium/node-chrome
    volumes:
      - /dev/shm:/dev/shm
    expose:
      - 5901
    ports:
      - 5901:5900
    links:
      - hub
    environment:
      - HUB_HOST=hub
      - HUB_PORT=4444
    networks:
      - backend
  aut:
    build:
      context: .
    expose:
      - 8080
    ports:
      - 8080:8080
    networks:
      - backend
  e2e:
    build:
      context: ./e2e-angular
    networks:
      - backend
    links:
      - hub
      - aut
      - chrome
networks:
  backend:

Figure 5: docker-compose.yml file (Define services for building and starting Docker containers by Docker Compose)

Focusing on the aut service definition, I don’t use a Docker image to build an aut container. Instead, the aut container is built from a Dockerfile file.

FROM webratio/nodejs-http-server:0.9.0
WORKDIR /opt/www

COPY ./dist/zero-to-hero /opt/www
EXPOSE 8080

Figure 6: Dockerfile file (A Dockerfile file for building an AUT container)

In order to execute the Dockerfile file, compiled source code will be deployed and run on a HTTP server; all are wrapped inside a Docker container. The port 8080 is exposed to external; you’ll need to allow access to the application from local or another container with a same network.

The e2e service is similar to aut service, with building e2e container from a Dockerfile file in E2E source code. The main reason to build an e2e container is all containers have to be joined in a network. This way, the Protractor can use containers, Selenium Hub, Selenium Node, and AUT to execute test scripts.

Now, you can try to build and run Docker containers on a local machine, by executing the docker-compose file with the command.

docker-compose -p angulare2e up -d

docker run -itd --rm --name angulare2e_container --network=angulare2e_backend angulare2e_e2e

Figure 7: Building and starting Docker containers (Preparing test environment with Selenium Hub, Selenium Chrome Node, AUT, and E2E containers)

Use the command docker ps to see built containers and docker network ls for the list of networks. With the parameter -p angulare2e, I created a virtual network angulare2e_backend and joined all containers to this network. Now, you can navigate to localhost:8080 to open the ZTH application and use this end point for E2E testing.

Figure 8: ZeroToHero local site (Access to the ZeroToHero site from an AUT container)

Awesome! Everything is ready to establish the E2E Testing gate in CircleCI. Let’s open the config.yml file and define a new job e2e-testing with tasks below:

  1. Install Docker Client and Docker Compose on a Circle container.
  2. Pull E2E source code from a Git repository.
  3. Install dependencies and build E2E source code.
  4. Build and start Docker containers from docker-compose.yml file.
  5. Run tests inside the E2E container.
  6. Collect test report from E2E container and save as a CircleCI build artifact.
  7. Notify status to developers.
e2e-testing:
executor: my-executor
parameters:
target-path:
description: Path for report directory
type: string
default: e2e-angular/allure-report
results-path:
description: Path to directory with test results
type: string
default: e2e-angular/allure-result
results-container-path:
description: Path to directory with test results inside E2E container
type: string
default: allure-result
artifact-path:
description: Path that will be used when storing result as artifact
type: string
default: Report/Allure
steps:
- attach_workspace:
at: ~/mwd-angular-zerotohero-v7
- setup_remote_docker
- run:
name: Install Docker client
command: |
set -x
VER="17.12.0-ce"
curl -L -o /tmp/docker-$VER.tgz https://download.docker.com/linux/static/stable/x86_64/docker-$VER.tgz
tar -xz -C /tmp -f /tmp/docker-$VER.tgz
sudo mv /tmp/docker/* /usr/bin
- run:
name: Install Docker Compose
command: |
set -x
curl -L https://github.com/docker/compose/releases/download/1.11.2/docker-compose-`uname -s`-`uname -m` > ~/docker-compose
chmod +x ~/docker-compose
sudo mv ~/docker-compose /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
- run:
name: Avoid hosts unknown for github
command: mkdir -p ~/.ssh/ &amp;&amp; echo -e "Host github.com\n\tStrictHostKeyChecking no\n" > ~/.ssh/config
- run:
name: Checkout E2E source
command: |
git clone $GH_E2E_REPO
cd e2e-angular
git fetch origin
git checkout master
- restore_cache:
keys:
- v3-e2e-dependencies-{{ checksum "./e2e-angular/package.json" }}
- v3-e2e-dependencies-
- run:
name: Install local dependencies
command: |
cd e2e-angular
sudo npm i npm@6.9.0 -g
npm install
npm install --unsafe-perm node-sass
- run:
name: Install Allure
command: sudo npm install -g allure-commandline --save-dev
- save_cache:
key: v3-e2e-dependencies-{{ checksum "./e2e-angular/package.json" }}
paths:
- node_modules
- run:
name: Build and start containers with Docker Compose
command: |
docker login -u $DOCKER_LOGIN -p $DOCKER_PASSWORD
set -x
docker-compose -p ${DOCKER_COMPOSE_PROJ_NAME} up -d
docker run -itd --rm \
--name angulare2e_container \
--network=${DOCKER_COMPOSE_PROJ_NAME}_backend ${DOCKER_COMPOSE_PROJ_NAME}_e2e
- run:
name: E2E Testing
command: docker exec -it angulare2e_container ./wait-for-grid.sh sudo npm run smoke-test:1.0.1
- run:
name: Save test results
command: docker cp angulare2e_container:/opt/www/<< parameters.results-container-path >> ~/mwd-angular-zerotohero-v7/e2e-angular
when: always
- run:
name: Allure report generation (<< parameters.results-path >> -> << parameters.target-path >>)
command: |
allure generate \
--report-dir << parameters.target-path >> \
<< parameters.results-path >>
when: always
- store_artifacts:
path: << parameters.target-path >>
destination: << parameters.artifact-path >>
- slack/notify:
message: 'E2E Testing for AUT Build $AUT_IMG_BUILD is completed.'
webhook: $DEV_SLACK_WEBHOOK
- slack/status:
webhook: $DEV_SLACK_WEBHOOK

Figure 9: config.yml file (Define an e2e-testing CircleCI job)

Integrate the E2E Testing Gate to CI Workflows

An application build should not be released until it PASSED the quality gate E2E Testing, because its quality is not good enough. To do that, you need to setup the E2E Testing gate after the Auto Build pipeline in CI workflows.

Let’s open the config.yml file and edit the section workflows. The CircleCI needs to know about the e2e-testing job, which requires this job in deployment build-docker and deploy jobs.

workflows:
version: 2
commit:
jobs:
- setup-environment
- static-testing:
requires:
- setup-environment
- unit-testing:
requires:
- setup-environment
- build:
requires:
- setup-environment
- static-testing
- unit-testing
- e2e-testing:
requires:
- build
filters:
branches:
only:
- /develop-.*/
- /staging-.*/
- master
- build-docker:
requires:
- e2e-testing
filters:
branches:
only:
- /staging-.*/
- master
- deploy:
requires:
- e2e-testing
filters:
branches:
only:
- master

Figure 10:config.yml file (Integrate the e2e-testing job to CI workflows)

Integrate the E2E Testing Gate to Git Branches

Open the Settings panel, and be sure to require quality checks for your staging branches.

Figure 11: GitHub Branches Settings (Configure quality checks for staging branches)

Before new feature source codes are allowed to merge into a staging branch, it has to be worked as expected without any side effects. So, we need to prepare a test suite for Smoke Testing combined with test cases for new features. This way, a new build is verified to ensure that the most important functions work.

With Protractor and Cucumber frameworks, you can use tags option to specify a test suite for Smoke Testing and feature test cases.

protractor build/_config/config.js --cucumberOpts.tags @SmokeTest:1.0.1 @LoginFeature

A helpful tip in NodeJS projects is setting up deployment scripts with the package.json file. We can wrap the long command above with a short script smoke-test:1.0.1.

"scripts": {
...
"smoke-test:1.0.1": "protractor build/_config/config.js --cucumberOpts.tags @SmokeTest:1.0.1 @LoginFeature"
},

After that, execute the script with NPM.

npm run smoke-test:1.0.1

Figure 12: smoke-test:1.0.1 script (Define a script for executing E2E smoke tests)


Verify Auto Test Pipeline

We are going to verify our configuration for the quality gate E2E Testing, by triggering the Auto Test pipeline from a pull request on staging branches.

Our story starts with a pull request to merge a development branch develop-1.0.0 into the staging branch staging-1.0.1.

Figure 13: Creating a GitHub pull request (A pull request for publish the AUT build 1.0.1)

To follow the branch policy, which is applied on staging branches, the pull request requires at least 2 approvals and all required checks have to be “PASSED.”

Figure 14: All required checks are triggered on a new pull request

I updated the feature login to make the test scenario Login successfully with a valid username and password a FAILED status. Now, let’s look at the screenshot below. The workflow has stopped at the Auto Test pipeline with the e2e-testing check “FAILED” as expected. So, this pull request needs to be fixed.

Figure 15: e2e-testing check is failed (A pull request is not able to merge with a failed check)

Figure 16: A CircleCI workflow is stopped (A CircleCI workflow is stopped by a failed e2e-testing job)

You can investigate and fix the feature from report details, which is shown in the Artifact section of the e2e-testing CI build.

Figure 17: An E2E Allure HTML Report (View E2E test details from the Allure report in the artifact on CircleCI build)

Submit a new commit for fixing the Login feature, and the Auto Test pipeline is triggered once again with required checks. You need to wait for all checks to be completed with “PASSED” results, in order to be able to merge the pull request.

Figure 18: All required checks are passed (A pull request is able to merge with all required checks are PASSED)

Figure 19: E2E Allure HTML Report

Now, the source code from the develop-1.0.0 branch is ready to merge to the staging-1.0.1 branch and to be published in the next AUT build for QA Testing.


Summary

Let’s take a look at all we’ve accomplished in this part of the series.

In this article, Angular DevOps – Part 4: Third Quality Gate – E2E Testing, we implemented the E2E Testing gate on an Angular project and integrated this quality gate into our CI workflows. The E2E Testing gate joined with Unit Testing gate in the Auto Test pipeline in order to verify code quality before and after the source code is complied.

Below is our CI workflows chart.

Figure 20: CI workflows chart (CI workflows chart with the quality gate E2E Testing)

Done! We finished the Angular DevOps series with 4 parts. I hope that this series has been helpful and that is has given some insight on how to automate Angular development process. Feedback or questions are very welcome! Happy testing!

Luong Mai
Luong Mai joined LogiGear in 2012 as an Automation Engineer. Currently, he works as a Software Developer for MOWEDE. He is responsible for developing front end applications (mobile, web) and DevOps in web development. Mai loves to automate everything in software development in order to maximize the productivity of development teams and is actively involved in developing software framework in MOWEDE
Luong Mai
Luong Mai joined LogiGear in 2012 as an Automation Engineer. Currently, he works as a Software Developer for MOWEDE. He is responsible for developing front end applications (mobile, web) and DevOps in web development. Mai loves to automate everything in software development in order to maximize the productivity of development teams and is actively involved in developing software framework in MOWEDE