Engineering Reference Library
Menu
Eng Calculators

Reference

Statistical Power For Engineering Tests

Plan an engineering test around the probability of detecting a practically important change instead of relying on significance level alone.

Referenceengineering & statisticsLast reviewed: 2026-07-29

Use statistical power when a test must have a defined chance of detecting a change that matters in engineering terms, such as a performance loss, dimensional shift, or increase in nonconformance.

What This Means

Power is the probability that a statistical test rejects the null hypothesis when a specified alternative is true. It is written as 1 - beta, where beta is the risk of missing that specified change.

Power is not a single permanent property of a test method. It depends on the alternative being considered. A study may have high power to detect a large shift and poor power to detect a smaller shift using the same sample size and significance level.

The engineering decision therefore comes before the calculation: define the smallest difference that would change a design, process, acceptance, or investigation decision. That difference becomes the effect the test should be capable of detecting.

Key Relationships

power = 1 - beta

one-sided mean test, known sigma:
n >= (z_(1-alpha) + z_(1-beta))^2 (sigma / delta)^2

two-sided mean test, known sigma:
n >= (z_(1-alpha/2) + z_(1-beta))^2 (sigma / delta)^2
  • alpha is the chosen Type I error risk for rejecting a true null hypothesis.
  • beta is the Type II error risk at the specified alternative.
  • delta is the change in the population mean that the plan is intended to detect.
  • sigma is the assumed population standard deviation.
  • n is the planned number of independent observations.

These normal-approximation relationships illustrate the tradeoffs; the correct calculation depends on the response type, test direction, design, variance treatment, and statistical model.

Use This When

  • Planning a test intended to detect a minimum meaningful shift rather than estimate a parameter within a margin of error.
  • Comparing sample-size options before an expensive or destructive engineering test.
  • Evaluating whether a nonsignificant result could simply reflect low sensitivity.
  • Setting alpha, target power, and a minimum detectable difference before data collection.
  • Communicating the false-negative risk associated with a validation or process-change decision.

Assumptions

  • The null and alternative hypotheses describe the actual engineering decision.
  • The target difference is selected for practical significance before looking at the final test data.
  • The variability estimate is credible for the process, measurement system, and test conditions.
  • Observations are independent enough for the planned method.
  • Distributional and model assumptions match the selected power calculation.

Limitations

  • The displayed sample-size relationships apply to a one-sample mean test with known standard deviation and a normal approximation.
  • Unknown variance, paired tests, two-sample comparisons, proportions, reliability demonstrations, equivalence tests, and sequential plans require different methods.
  • A powered test does not correct biased sampling, poor measurement capability, process instability, or an irrelevant effect threshold.
  • Observed post-test power generally adds little beyond the estimate, uncertainty interval, and original design assumptions.
  • High power to detect a trivial difference does not make that difference important.

Common Mistakes

  • Treating alpha = 0.05 as proof that the test has adequate power.
  • Choosing the detectable difference after seeing the data.
  • Using an optimistic standard deviation that understates real process and measurement variation.
  • Interpreting failure to reject the null hypothesis as proof of equivalence.
  • Using a margin-of-error sample-size calculator for a shift-detection objective without recognizing the different question.
  • Increasing sample size without addressing dependence among repeated measurements from the same part, batch, operator, or setup.

These calculators address estimation precision, not power-based detection. Use them only when the engineering objective is a confidence-interval margin of error.

Sources

This reference uses the NIST/SEMATECH Engineering Statistics Handbook for the definition of power as 1 - beta, the relationship among significance level, beta risk, detectable shift, variability, and sample size, and the distinction between statistical and practical significance.

  1. National Institute of Standards and Technology. NIST/SEMATECH e-Handbook of Statistical Methods (NIST Engineering Statistics Handbook), National Institute of Standards and Technology, 2003. DOI: 10.18434/M32189. Source page.