Training Course on Statistics for Method Validation

I attended a training course entitled "Statistics for Method Validation" which was organized by Hong Kong Accreditation Service (HKAS) on 27 Oct 2011. It was a very informative seminar not only theories review, but also experience sharing.

The trainer was Mr. Dan Tholen (Statistical Consulting, A2LA Lead Auditor (PT) and Professionals in Proficiency Test) and his presentation mainly focused on method validation. There were two main standard series stated during the course. One is Clinical and Laboratory Standards Institute (CLSI) EP series for medical and the other is ISO series for general.

Mr. Dan Tholen explained method validation based on the following standards:

- ISO 5725: Accuracy (trueness and precision) of measurement methods

- Medical applications, CLSI EP5: Precision of quantitative methods and EP17: Limits of detection and quantitation

- Food microbiology: ISO 16140 - Microbiology of food and animal feeding stuffs - Protocol for the validation of alternative methods

Then Mr. Dan Tholen stated the definition of Verification and Validation from VIM3: International Vocabulary of Metrology.

- 2.44 verification: provision of objective evidence that a given item fulfills specified requirements, taking any measurement uncertainty into consideration

- 2.45 validation: verification, where the specified requirements are adequate for a stated use

He said that 'validation' is subordinate to (or is a type of) 'verification'.

However, this interpretation was found different to ISO 9001 definition in Design and Development process. Verification (7.3.5) is checking that the results at the end of the design process meet the requirements identified as necessary at the beginning of the design process. It may be carried out on a stage-by-stage basis. Validation (7.3.6) is the process of checking that the final product or service will be capable of meeting or does meet the customers' and end-users' needs when used in the intended environment.

One of participants said the meaning of verification and validation in terms of Chinese are no confusion. They are "驗證" for verification and "確認" for validation.

After that Mr. Dan Tholen introduced the ISO 5725 series as follows.

Part 1: General principles and definitions

Part 2: Basic method for the determination of repeatability and reproducibility of a standard measurement method

Part 3: Intermediate measures of the precision of a standard measurement method

Part 4: Basic methods for the determination of the trueness of a standard measurement method

Part 5: Alternative methods for the determination of the precision of a standard measurement method

Part 6: Use in practice of accuracy values

Repeatability (r) and Reproducibility (R) are the general principles of ISO 5725 which represent the extremes of precision (smallest and largest).

The definition of r conditions and R conditions, as well as intermediate precision condition are given in ISO 3534-2 (Statistics - Vocabulary and symbols - Part 2: Applied statistics).

After that he introduced the concept of Limit of Blank (LoB), Limit of Detection (LoD) and Limit of Quantitation (LoQ).

Based on CLS EP17, the definition of LoB, LoD and LoQ as below.

LoB - Highest measurement result that is likely to be observed (with a stated probability) for a blank sample.

LoD - Lowest amount of analyte in a sample that can be detected with (stated) probability, although perhaps not quantified as an exact value.

LoQ - Lowest amount of analyte in a sample that can be quantitatively determined with stated acceptable precision and trueness, under stated experimental conditions.

The following diagram explained how to use LoB, LoD and LoQ for reporting.

If the result is below LoB, it reports "Not detected".

If the result is between LoB and LoD, it reports "Detected but below LoQ". It is because the result distribution between LoB and LoQ is not quantified.

If the result is between LoD and LoQ, it also reports "Detected but below LoQ".

Only the result is above LoQ, it can report "As measured value".

Since patient's sample is usually individual, nonparametric statistic will be used in this approach.

CLSI EP 17 protocol is for all in one laboratory in which interlaboratory comparison study is not required. It could be manufacturer or test laboratory.

Q&A session

Reference:

HKAS Seminar: http://www.itc.gov.hk/en/quality/hkas/seminar.htm

CLSI EP5: Precision of quantitative methods

CLSI EP9: Method comparison and bias estimation using patient samples

CLSI EP17: Limits of detection and quantitation

ISO 5725: Accuracy (trueness and precision) of measurement methods

ISO 16140: Microbiology of food and animal feeding stuffs - Protocol for the validation of alternative methods

ISO 3534-2: Statistics - Vocabulary and symbols - Part 2: Applied statistics

ISO 11843 series: Capability of detection