Stubs vs. mocks in software testing

Stubs in software testing

A stub is a basic placeholder used to simulate the behavior of a dependency in a controlled manner. Often it is an implementation that returns simple predefined responses to function calls without executing any actual logic. A test stub's primary goal is to provide specific data or responses that let the test focus on the behavior of the component under test, while isolating it.

Example of a test stub

Consider a function that fetches data from an API. One could use a stub to return a hardcoded response instead of making an actual network request. This is useful when testing how a component processes the data without depending on the availability or speed of the external API. It also enables a focus on testing the actual app code and not the API code.

Advantages of a test stub

There are three basic advantages of using test stubs vs. mocks.

Simplicity. Stubs are simple to create. Because they only return hardcoded values, they require minimal setup and are easy to manage.

Consistency and control. Stubs provide consistent results, ensuring that tests are predictable and reliable. By returning predefined values, they eliminate variability and allow the tester to control the environment, which is particularly useful for edge-case testing.

Stability. Stubs don't depend on external components or complex logic, which means stubs are stable.

Disadvantages of a test stub

There are several drawbacks of using test stubs rather than mocks.

Limited functionality. Stubs are simplistic by nature and cannot simulate complex behavior. They are mostly useful for straightforward unit tests, but their lack of flexibility makes them unsuitable for tests that require a more realistic interaction between components.

Maintenance overhead. As the codebase evolves, keeping stubs in sync with the actual implementation can become a maintenance burden. If the behavior of the dependency changes, one must update all the stubs that rely on it, which increases the upkeep.

Lack of verification. Stubs are designed to provide predefined outputs but do not validate how they are called. This means they do not verify whether the function or component under test interacts with them correctly, and this limits their effectiveness for tests that require interaction validation.

Mocks in software testing

A mock is a more sophisticated test replacement than a stub. It not only simulates the behavior of a dependency, but also records how it was interacted with during the test. Mocks are dynamic and can be configured to return different responses based on input parameters. They are designed for situations where it's important to validate that certain methods were called with the correct parameters or that specific interactions occurred during the test.

Example of a test mock

Consider a function that sends a notification email. A mock can verify that the email-sending method was called with the expected recipient and message content. This enables a tester to validate that the logic not only runs but also behaves as expected when integrated with other components.

Advantages of a test mock

Mocks have some important benefits when used in software testing, including the following:

Interaction verification. Mocks can be configured to verify interactions, such as to ensure a method was called a specific number of times or with specific arguments. This is the primary difference between a mock and a stub, and it is important for tests where the interaction is a factor.

Dynamic behavior. Mocks can return different values based on the inputs they receive, which enables them to simulate more complex scenarios. This flexibility makes mocks suitable for testing cases where the behavior varies with the inputs.

Integration testing support. Mocks are valuable to test how components integrate with each other. They verify that the correct methods are called with the expected parameters, and thus ensure that the components interact properly. This also means they can help detect issues early in development.

Disadvantages of a test mock

Like stubs, mocks have their cons, such as the following:

Complexity. Mocks are inherently more complex than stubs. To set up a mock one must configure its behavior and define the expected interactions, which can be time-consuming, especially for larger systems with numerous dependencies.

Brittleness. Tests that rely heavily on mocks can become brittle if the underlying code changes frequently. Mocks can validate specific interactions, so any change in method signatures or interaction patterns could break these and increase the maintenance burden.

Overuse. Mocks are powerful, but overusing them can lead to tests that are too tightly coupled to the implementation details of the code. This reduces the test flexibility and can hinder or make refactoring code more time-consuming (and expensive), as changes to the code might require substantial updates to the mock configurations.

Verdict: Stubs for functionality, mocks for verification

Choosing between stubs vs. mocks for software testing largely depends on the specific testing requirements and the type of test to be performed.

Stubs are ideal for simple tests where the focus is on testing the functionality of a specific module without involving complex interactions. They provide consistency and stability, which makes them great for isolated testing scenarios. However, they lack the capability to verify behavior, which limits their usefulness for more complex tests.

Mocks are better suited for tests where interaction verification is necessary. Testers can check that methods are called correctly and that the proper behavior occurs, which makes them invaluable for more complex tests. On the other hand, the complexity and brittleness associated with mocks mean that they should be used judiciously. Over-reliance on mocks can lead to fragile tests that require frequent updates as the codebase evolves.

Neither stubs nor mocks are inherently better -- they serve different purposes in testing. For comprehensive testing, a balanced approach that combines the simplicity and control of stubs with the dynamic behavior and interaction verification capabilities of mocks is often the most effective strategy.

In the words of all senior developers: It depends.

David "Walker" Aldridge is a programmer with 40 years of experience in multiple languages and remote programming. He is also an experienced systems admin and infosec blue team member with interest in retrocomputing.