Why is Unit Testing Embedded Software really Difficult?

UT is simple by the definition; a test case shall be clear, simple, testing one requirement. But, when it comes to testing C code for embedded software, we have to run an extra mile to do the job properly compared to testing Java or C++ code. This article will illustrate some known difficulties in testing software for embedded systems.hisham.jpg

I.Problems & Solutions

1.Static Variables

Problem

In OOP languages, the methods and the variables are encapsulated with the class. So, when you create a new object of the class you are testing, it will be a clean fresh copy.OOP and JAVA.gifBut in In C, we can’t create an instance of the code under test. This means that we can’t easily get a fresh object with initialized data for every test.

Why is it a problem?

This leads us to problem with “Static initializations” inside your module under test as each test case will be dependent on the values of static variables set by the previous test case. Remember that a good test case must be reliable; it yields the same result when executed.

Capture
Proposed Solution

Simply, before each test case you must re-initialize all of your static data for each test case. You can use setup functions to help you do initialization sequence before each test case. You must be able to reset your data to a known state before running each test case.

 

2.Module under Test Dependencies

Problem

There is always dependency between module under test and external modules.

Why is it a problem?

To be able to properly test a module you need to abstract it from all external components so that all the input states are controllable and all the outputs are observable.stub.PNG

In OOP we can use Polymorphism to break the dependencies between module under test and external modules (dependency injection & inversion control).IVIdB.png
Proposed Solution

Our only solution for dependency-breaking in C is the preprocessor and the linker; we need to compile the implemented stub source files instead of the original implementation.sty

3.Global variables

If the system under test is accessing a global variable, you need to provide a fake (stub) implementation for this variable clearly.

4.Hardware Access

Problem

In embedded systems we often have memory mapped hardware register access that affects the behavior of the system under test.

Why is it a problem?

If you are using a host compiler for testing, you definitely don’t want to be dereferencing from random memory addresses in your tests.  Also, you want to be able to control the return data of this register from your test case.

Proposed solution

There are two ways we can deal with dilemma:

  1. change the address we dereference,
  2. change the function we call

5.Compiler Extensions

If you are using a host compiler, the non-ansi C extension will cause compilation problems in the SUT. Simply you need to remove these extensions using the preprocessor.

II.Problems with No Solution:

There are some problems that we can’t solve by UT efficiently. Remember that your test cases must be fully automated, easily executed after regression and reliable. Also your test cases must not pollute your source code. So there are some problems that we can’t address them in the scope of UT. Here are some points listed:

  • Timing Problems: Unit testing can’t magically simulate the runtime properties of your system.
  • Interrupts: This is a special case of timing problems, but it is the same issue all developers come across when going multi-threaded.
  • Bit-correct operations: If you are running 24-bit code on a 32-bit architecture you will not see the exact same behavior for various overflow, underflow, bit-shifting and arithmetic operations.

III.General Testing Pattern

  1. Define fake functions for the dependencies you want to stub out
  2. If the module depends on a global, you need to define your fake one
  3. Replace external modules with the stub implementation during linking
  4. Define a method to reset all the static data to a known state.
  5. Define your tests

steps.PNG

IV.Summary

Testing C code is hard and testing legacy C code is even harder.  Although the limited dependency-breaking language features we have in C (the linker and the preprocessor) we can accomplish quite a lot. The blocking issues in UT are an extreme minority of most code bases.

Waiting for your comments, feedback and questions !     View Hisham El Meligi's LinkedIn profile View Hisham El Meligi's profile

 References:

https://meekrosoft.wordpress.com/2009/11/09/unit-testing-c-code-with-the-googletest-framework/

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s