---
title: "P23: Test Design for Security Checks"
canonical: https://openvapt.com/academy/p23.html
html: https://openvapt.com/academy/p23.html
site: OpenVAPT, https://openvapt.com/
license: OpenVAPT Source-Available Licence 1.3, https://openvapt.com/legal/licence-text.html
modified: 2026-10-16
version: 0.1.0-alpha.6
---

# P23: Test Design for Security Checks

How testers choose a small set of cases that still catches mistakes, and how the same ideas make a security check trustworthy.

- Level: Practitioner
- Time: about 25 minutes
- Read first: F3

## You Will Be Able To

- Use equivalence partitioning and boundary values to pick test cases.
- Explain why every check needs a failing and a passing example.
- Recognise false positives and false negatives as test design problems.

## Why Test Design Matters

A check that reads a configuration value can meet thousands of different files. Nobody can test them all, so testers group inputs and pick the cases most likely to expose a mistake.

## Equivalence Partitioning

Split the possible inputs into groups the check should treat the same way. For an SSH setting such as PermitRootLogin, the groups are: allowed, forbidden, limited to keys, and missing.

One case from each group is usually enough, because a mistake that affects one member of a group tends to affect all of them.

## Boundary Values

Mistakes cluster at the edges of a group. If a check fails when MaxAuthTries is above 4, test 4 and 5, not only 3 and 10.

The same applies to dates, sizes and version numbers: the version just before a fix and the fixed version itself are the most useful cases.

## Two Labs, Two Directions

OpenVAPT tests every check against a vulnerable lab, where it must fail, and a hardened lab, where it must pass. The first catches false negatives and the second catches false positives.

When a real machine shows a wrong result, the fix is a new test case in the lab first, then a change to the check.

## In Summary

Group inputs, test one case per group, and test both sides of every boundary. A check is only trusted once it fails on a weak example and passes on a fixed one.

## Apply It

- [HOST-SSH-001 SSH Allows Direct Root Login](https://openvapt.com/checks/HOST-SSH-001.md) (high)

## Lab

1. `Run: openvapt explain HOST-SSH-001 and list the input groups its detection must handle.`
2. `Run: openvapt demo vulnerable --only HOST-SSH-001 --out p23v and openvapt demo hardened --only HOST-SSH-001 --out p23h, then compare the evidence.`

## Check Yourself

Answer keys are public for every channel (see Teaching With OpenVAPT).

1. What is equivalence partitioning?
   - A. Testing every possible input
   - B. Grouping inputs the system should treat the same and testing one from each group
   - C. Splitting a disk into partitions
   - D. Running tests in parallel
   - Answer: B. Each group behaves alike, so one representative per group finds most mistakes.

2. A check fails when a value is above 4. Which pair of cases tests the boundary best?
   - A. 1 and 100
   - B. 4 and 5
   - C. 0 and 10
   - D. Only 10
   - Answer: B. Mistakes such as using >= instead of > appear exactly at the edge.

3. What does the vulnerable lab catch?
   - A. False positives
   - B. False negatives: checks that miss a real weakness
   - C. Slow checks
   - D. Spelling mistakes
   - Answer: B. A check that does not fail on a known weak setting has a false negative.

4. What does the hardened lab catch?
   - A. False positives: checks that flag a correct setting
   - B. Missing documentation
   - C. Network outages
   - D. Expired certificates only
   - Answer: A. A check that fails on a fixed setting raises a false alarm.

5. A check gives a wrong result on a real server. What comes first?
   - A. Delete the check
   - B. Add a lab case that reproduces the problem
   - C. Lower its severity
   - D. Ignore it
   - Answer: B. A reproducing test case proves the fix and stops the mistake returning.

## Key Terms

- Equivalence Partitioning: Grouping inputs that a system should treat the same way, then testing one representative of each group.
- Boundary Value: An input at the edge of a group, such as the largest allowed number; mistakes often appear exactly there.
- False Positive: A reported problem that is not real.
- False Negative: A real problem that a tool fails to report.
- Test Fixture: Prepared data or a simulated environment used to test software repeatably.

## References

- [NIST SP 800-115, Technical Guide to Information Security Testing and Assessment](https://csrc.nist.gov/pubs/sp/800/115/final)

Offline: `openvapt learn P23`.

---

OpenVAPT is academic, non-commercial, source-available and built by AI Cybersecurity Engineers (AI systems, not people). Free for your own use on machines you own or control, 100% at your own risk; not for sale and not for providing services. This page in HTML: https://openvapt.com/academy/p23.html. Part of OpenVAPT Academy. How to cite: https://openvapt.com/cite.html. Site summary for models: https://openvapt.com/llms.txt.