Design Verification tested if the product was built correctly. Design Validation asks a harder question: was the product being built the correct one in the first place? The first proof is technical: does the product consistently do what it is supposed to do for the people who use it? The second proof is narrative: can you convince an informed, external, skeptic? That shift in audience is what makes validation different from Design Verification. Every method must be pre-specified and every assumption justified in writing. And the statistical framework must hold together across several types of validation, each with its own rules, methods, and expectations. Design Validation, Process Validation, and Clinical Validation each ask fundamentally different questions, use different statistical tools, and answer to different regulatory frameworks. A confidence/reliability interval that works for a device performance requirement may have no equivalent in a clinical bridging study, and the process capability metrics that govern manufacturing have no direct parallel in human factors testing.

With that complexity as the backdrop, the threats multiply. Standing in the middle of it all, generating more data than anyone can reasonably process while obscuring the signal beneath sheer volume, is Dr. Data himself. When we last left Stat-Woman, she had closed out a demanding Design Verification campaign. The Outlier had been confronted honestly, acceptance criteria had held up, and sample sizes had been justified with evidence. But Mr. Ledger offered no congratulations. He simply said, “You proved it once. Now you have to prove it again to someone whose job is to not believe you.”

Exhibiting Discipline

Regardless of which validation front you are on, pre-specification is the single most important discipline, and it is also the one most frequently treated as a formality. Teams know they are supposed to define their statistical methods and acceptance criteria before testing begins with methods and acceptance criteria clearly appearing in the protocol. On paper, the requirement is met, but pre-specification is not about filling in protocol blanks. It is about making real decisions, in advance, about how data will be analyzed and what conclusions will be drawn before anyone knows what the data looks like.

The distinction matters because validation is the one phase where the appearance of rigor is not sufficient. A regulatory reviewer will not simply check whether a method was listed in the protocol. They will ask whether the method was appropriate to answer the safety and/or efficacy question, whether the acceptance criteria were justified for the intended use, and whether the sample size was sufficient to support the claims being made. If the answer to any of these is “we chose it because it was in the protocol template” or “it was the same method we used in Design Verification,” that is not pre-specification. That is a copy-paste shortcut Dr. Data feasts on.

True pre-specification means that every statistical decision has a documented independent rationale. Why was this method chosen? Why was this sample size selected? Why was a non-parametric approach used for this requirement but not that one? Why were these choices made for this test? Each answer should trace back through the lifecycle: the distributional assumptions tested in the Training Grounds, the variability characterized during Design Verification, and the risk classification that drives the level of rigor required. These decisions were not made in the validation protocol as they were built across every phase that preceded it. The protocol is where they are formalized, not where they originate.

Three Battles, Three Fronts: What Validation Actually Looks Like

One of the reasons validation feels like a continuation of Design Verification is that teams often treat it as one monolithic activity. In practice, validation for a combination product is fought on at least three distinct fronts: Design Validation (does the product meet user needs?), Process Validation (does the manufacturing process consistently produce conforming product?), and Clinical Validation (does the product perform as intended in the patient population?). Each front has its own statistical language and regulatory expectations. Software validation is its own discipline as well, but the statistical methods there are sufficiently distinct that they warrant separate treatment and are outside the scope of this article.

Design Validation asks whether the product meets the user’s needs. That question is broader than any single study or method. Human factors validation is one of the most prominent examples for combination products, but it is not the entirety of Design Validation and it only answers use-related questions. Other Design Validation activities may include clinical performance studies, simulated use testing, or patient preference assessments, each with its own statistical approach. For device-specific endpoints, there is often risk-based tolerance in how acceptance criteria are structured, meaning the rigor scales with the severity of potential harm. For drug-related endpoints on the same product, the focus typically shifts to impediments to delivery regardless of risk classification, because any interference with the drug’s pharmacokinetic profile is treated as a critical concern. The statistical methods across these activities look nothing like Design Verification. Instead of continuous-variable tolerance intervals and capability metrics, the focus may shift to task completion rates, use error analysis, categorical data, or clinical endpoints. Sample sizes are driven by use-related risk analysis, the number of distinct user populations, or clinical power calculations rather than standard deviation estimates and K-factor tables. A team that walks into Design Validation expecting the same statistical framework they used in Design Verification will find themselves unprepared for the questions being asked.

Process Validation asks whether the manufacturing process consistently produces conforming product. Process Performance Qualification (PPQ) is one of the most visible components, but Process Validation also includes process design, process qualification beyond PPQ, and continued process verification. The statistical questions across all of these are fundamentally about stability and capability over time as opposed to single-point design performance. This is where Cpk and Ppk earn their real weight, not as one-time snapshots, but as time-based metrics that must be sustained across production lots, shifts, and operators. Control chart design, process capability trending, and the statistical criteria for what constitutes a meaningful process shift are the tools of this front. The connection to lifecycle monitoring is direct: the process validation data becomes the baseline against which every future manufacturing lot will be judged.

For nearly all combination products, and certainly any Class III or drug-led program, there is a clinical or regulatory validation front that demands its own statistical framework entirely. A drug-device combination may require clinical bridging studies to demonstrate that the device does not alter the drug’s pharmacokinetic profile. A biosimilar in a pre-filled syringe may need bioequivalence data with statistical methods governed by ICH and FDA guidance that have no parallel in the device world: equivalence margins, 90% confidence intervals on geometric mean ratios, and crossover study designs. Essential Drug Delivery Outputs (EDDOs) play a critical role here, as they must be traced through clinical development to demonstrate that device performance directly supports the clinical claims being made. This is also where Design Verification and Design Validation can have competing or overlapping priorities for combination products, because the device performance data that satisfies Design Verification requirements may need to be re-evaluated or supplemented to address Clinical Validation questions. Stability data under ICH protocols may require statistical modeling of degradation over time, with shelf-life estimates derived from regression analysis against specification limits. These are their own discipline entirely, and treating them as an afterthought is one of Dr. Data’s most reliable strategies for creating confusion.

The Audience Has Changed: What Regulators Actually Look For

Across all three fronts, the single biggest shift from Design Verification to validation is the audience, though the nature of that audience differs by validation type. Design Validation may involve end users, human factors specialists, or usability study participants. Process Validation answers to manufacturing quality and process engineers. Clinical Validation faces clinical investigators and, ultimately, the most demanding external reviewers of all: regulatory agencies evaluating submissions. In each case, the person evaluating the work has never attended the design reviews, has never seen the exploratory data, and has no knowledge of the engineering rationale behind each decision. They are reading a document that must make the statistical logic self-evident without supplementary explanation.

Regulators do not evaluate statistical methods in isolation. Instead, they evaluate the logic that connects them. A perfectly executed analysis attached to an unjustified method is worse than a simple analysis attached to a well-reasoned one, because it creates the illusion of rigor without the substance. This is Dr. Data’s operating model: overwhelm the reviewer with volume, complexity, and sophisticated-looking outputs while hoping that nobody asks whether the underlying approach was appropriate for the specific claim being made. The antidote is traceability: requirement -> risk -> test -> method -> result -> decision. If any link in that chain is missing or borrowed from a different context without justification, the entire chain is weakened.

Mr. Ledger’s simple test applies here more than anywhere: can you explain every statistical decision in one sentence without referencing what the results turned out to be? If the justification only makes sense in light of the outcome, it is not a justification. It is a rationalization.

Midstream Changes and the Temptation of Rationalization

Even the best-planned validation campaigns encounter surprises. A test result falls outside the expected range. A measurement system shows more variability than anticipated based off what was previously understood in Measurement System Analysis (MSA). An acceptance criterion that seemed appropriate during planning now appears too tight based on actual data. These moments are where validation discipline is truly tested.

The temptation to adjust midstream is powerful. Widening an acceptance criterion after seeing the data, switching from a parametric to a non-parametric method because the normality assumption did not hold, adding additional samples to strengthen a marginal result, or reframing a requirement to better fit the outcome are all forms of post-hoc rationalization. Any change to a pre-specified plan requires a documented justification that is independent of the results already obtained. The justification cannot reference the data it is trying to accommodate, because that is precisely the circular reasoning that pre-specification exists to prevent.

This does not mean validation protocols are irreversible. Legitimate protocol amendments and deviations happen and are expected, but there is a clear line between a protocol change driven by new technical understanding, such as discovering a previously unknown failure mode that requires additional testing, and one driven by the desire to achieve a passing result. Mr. Ledger has a simple test for this as well: “Would you have made this change if the results had been passing?” If the answer is no, the change is not a scientific decision.

Surviving the Scrutiny

Validation is not where you build your statistical story. It is where you defend it across three distinct fronts, to an audience that does not share your context, under conditions that demand every decision be self-evidently justified. Dr. Data loses his power the moment the statistical narrative becomes traceable, pre-specified, and honest about its own limitations. Stat-Woman wins this battle not with more sophisticated tools, but with a story that holds together from the first question asked in the Stat-Cave to the last result documented in the validation report.

The next time we visit Variance City, Stat-Woman will turn her attention to what happens after validation is complete. The product launches, the team disperses, and the statistical methods that carried the program through development get archived. But Mr. Ledger knows the fight does not end there. It simply changes form. Lifecycle monitoring, post-market surveillance, customer complaints and adverse events, and the discipline of statistical continuity will determine whether yesterday’s victories hold, or whether the assumptions that underpin them quietly erode when nobody is watching.

AUTHOR

Alex Spivak, Principal Consultant, Suttons Creek – Alex is a highly strategic Biomedical Engineer with 13 years of MedTech industry experience, specializing in solving complex R&D challenges and ensuring strict Quality Assurance and Compliance. Alex transforms product development through rigorous statistical analysis and process optimization, consistently delivering substantial quality enhancements and guaranteeing minimal Post Market Surveillance complaints. As the lead Quality Engineer for numerous global commercial product launches, he has successfully operated in diverse environments, from large corporations to small startups, and across multiple domestic and international locations, including Japan.