Separate Unit, Integration, and Functional Tests for Continuous Delivery

Working Women at Computers —

I’ve recently read the article “JavaScript Testing: Unit vs Functional vs Integration Tests” written by Eric Elliott and found it is very helpful to understand the differences between these 3 types of tests: Unit, Integration, and Functional that you might used to see on this test pyramid:

Source: Test Automation Basics — Levels, Pyramids & Quadrants

GUI Tests in the top of this pyramid is the Functional test in my meaning. It is sometime called End-to-End Tests or System Tests. In addition, let’s treat API, Integration, and Component Tests in the pyramid as Integration Tests. Typically, most apps require Unit and Functional Tests. For complex app, it may also require Integration Test as well. We can call all of them as Test Suite.

In practical, you might feel that these 3 main layers are not totally clear-cut and kind of overlapped with each other. One big unit (which call another smaller units) can be a component or an API. Testing via UI could be either functional or API test.

However, when you want to automate these tests (e.g. for continuous delivery). It is very important to separate them especially the Unit Tests because of their different behaviors and purposes you expect from their test results.

First, let’s learn the differences of them:

Unit Tests

  • Validate individual component of the app
  • Assertions test the component’s input and output parameters
  • Testing is done from developer’s perspective
  • Real-time developer feedback
  • Simple, fast, and has characteristics of F.I.R.S.T. Principle
  • No uncontrollable dependencies or interfaces to other component e.g. database access, file/log read/write, remote/API calls, etc.
  • Always give the same consistent result when executing anywhere (DEV/QA/Prod servers), anytime (next day, next month, next year, next 5 years).
  • Generate good bug report which could answer these 4 questions:
  1. Which component is under test?
  2. What is the expected behavior?
  3. What was the actual result?
  4. What is the expected result?

Integration Tests

  • Validate components collaboration
  • Assertions may test component’s input and output parameters (API), UI, or side-effects (e.g. database, file, log, etc.)
  • Testing may be done from developer’s (e.g. API) or end-user’s perspective (e.g. UI)
  • They are simply Unit Tests (no matter how big it is) with uncontrollable dependencies or interfaces e.g. database access, file/log read/write, remote/API calls, etc.
  • Give unconsistent result when executing due to their unpredictable dependencies

Unit vs. Integration Test

To understand more the difference between these twos, let’s take a look into this example:

Suppose you have several functions (in a modular program) with calling relationships as shown in below diagram. Function E needs to read a table from database to fulfill its operation so it has dependency as well as B and A which calling E.

Components with dependencies

In this case, you can only write Unit Tests for C, D, F, and G. If you write tests for E, B, or A, then they are not Unit Tests but Integration Tests which are not the purpose of xUnit framework which is the tool for writing Unit Tests. The most dangerous thing was you mix Unit and Integration Tests together and how do you trust the bug report?

So if you want E, B, and A to be “testable” by Unit Tests, there are ways to workaround. One way is using a mock/fake database if available in your platform to ensure that E will always get the same result.

Another way is to apply Inversion of Control (IoC) by implementing a Dependency Injection as show in this diagram:

Dependency Injection

You may create another function H which only does selecting data from the database which is required by E. You let the caller calls H first to get the required data and then pass (inject) this data (dependency) into E (through B and A). Now, you can write Unit Tests for E, B, and A.

Integration Tests should always be kept separate from Unit Tests.

Functional Tests

  • Ensure the whole app or system works as expected
  • Also known as End-to-End Tests (from Front-end UI to the back-end database) or System Tests (set of Functional Tests)
  • Assertions primarily test the actual user interface output
  • Testing is done from end-user’s perspective
  • Typically have thorough tests for “happy paths” — ensuring the critical app capabilities, such as user logins, signups, purchase work flows, and all the critical user workflows all behave as expected.

There is another kind of test which is also a Functional Tests that I would like to mentioned i.e. Smoke Tests

Smoke Tests

  • A subset of production-safe functional tests
  • Run to ensure that none of the critical functionality was broken during the deploy process.
  • The purpose is “ You don’t want your users to find the bugs before you do”
  • You should also maintain this kind of functional test separated from others in your automation test suite so you can run them in production after you deploy.