Regression Tests and Smoke Tests: The Gate Before Release
- date
- category
- Testing
- also in
- CI/CD · Reliability Engineering
- reading
- 3 min / 572 words
Not every test is named after a place in the architecture.
Unit test says how small the unit is.
Integration test says we check a boundary between parts.
Regression test says something else:
this bug has already happened once
It is named after the reason the test exists.
Regression tests
Regression test appears after a bug or where old behavior may return.
First there is a symptom:
a user without permissions sees a button
an invoice calculates VAT twice
retry creates a duplicate order
cache returns another tenant's data
Then there is a fix.
Then there should be a test that closes that path.
A regression test can be a unit test, integration test, API test, or E2E test.
The name regression does not say which level the test runs at.
It says the test protects against a bug returning.
What happens when they are missing
Without regression tests, the team fixes the same problem more than once.
The bug returns after a refactor.
It returns after a dependency change.
It returns after a migration.
It returns because nobody wrote down in code what the system must never do again.
The worst part is that the second occurrence is more expensive psychologically.
Not only did the system fail.
The learning process after the incident failed too.
Smoke tests
Smoke test answers a simpler question:
is the system alive at all?
It is not a full quality test.
It is a fast gate after a build, deployment, or traffic switch.
Examples:
application starts
health endpoint responds
home page renders
login does not return 500
worker reads a message from the queue
new version can access the database
A smoke test should be short.
If it runs for a long time and checks many cases, it stops being a smoke test.
Sanity tests
Sanity test is close to a smoke test, but usually tied to a specific change.
After a payment fix, it can check the minimal payment path.
After a routing change, it can check one request through the new route.
After a configuration migration, it can check whether the application reads the right variables.
A sanity test does not replace full regression.
It answers whether the basic assumption after the change still makes sense.
Where they sit in the pipeline
Smoke tests often sit early after build or right after deployment.
Regression tests are spread across levels.
unit regression test
integration regression test
API regression test
E2E regression test
That matters because E2E is not automatically the right choice just because the bug was production.
If the production bug came from one function, the best regression test may be a unit test.
The test should close the shortest path for the bug to return, not replay the whole incident.
Tool from the stack
For smoke tests in this stack I would use GitHub Actions as the gate and a small suite built with Supertest or Playwright.
For the backend, a smoke test can start the Node application and check:
health endpoint
database connection
basic API endpoint
For the frontend, a Playwright smoke test can check whether the application renders and whether login does not end with an error.
For regression tests I would use the tool that closes the bug at the shortest level: Vitest, Supertest, or Playwright.
This is not a separate runner, but a decision about the level.
When they are placed poorly
A smoke test is placed poorly when it tries to be a full system test.
A regression test is placed poorly when nobody knows which bug it closes.
In both cases the test becomes a ritual.
It is in the pipeline, but nobody remembers what it protects.
A good review question is:
if we remove this test, which bug can pass again?
If the answer is concrete, the test makes sense.
If the answer is "because coverage", the test probably needs rethinking.
The next test group concerns the place where rollback is often the hardest: data.