INTRODUCTIONOpen Access icon OPENOpen Access license

Focus Issue Articles on Diagnostic Assay Development and Validation: The Science of Getting It Right

    Authors and Affiliations
    • Kitty F. Cardwell1
    • Carrie L. Harmon2
    • James P. Stack3
    • Poonam Sharma4
    1. 1Oklahoma State University Stillwater, Stillwater, OK
    2. 2Department of Plant Pathology, University of Florida, Gainesville, FL
    3. 3Department of Plant Pathology, Kansas State University, Manhattan, KS
    4. 4Institute for Biosecurity and Microbial Forensics, Oklahoma State University, Stillwater, OK

    This focus issue is the result of years of a growing awareness of a gap in coordination and resources for plant disease diagnostic assay development and validation. The existing and emerging plant health problems on a vast array of plant hosts mean that we are often responding to an outbreak even before there is a well-validated diagnostic test. With risk and pathway analyses, which provide intelligence about the likelihood of a serious potential problem, there is the opportunity to develop and validate diagnostic assays in a timely manner. The problems that we do not see coming tax the ability to quickly develop and validate a diagnostic assay. The development and validation of new tests or assays, and further evaluation of the performance of the test or assay under new conditions, are not coordinated within the U.S. biosecurity system. An American Phytopathological Society (APS) symposium in 2017 explored aspects of assay validation. That symposium was the first time that many of us were introduced to concepts such as fitness for purpose, assay validation research, and the vocabulary that refers to the performance metrics of a diagnostic assay. At the same APS meeting, we met with European colleagues who were beginning their VALITEST project (Trontin et al. 2021 and 2023 in this focus issue). Additionally, with the intent to harmonize the language and practice of diagnostic test validation in the United States, APS published a glossary of diagnostic assay validation terminology (Cardwell et al. 2018), followed by a Plant Health Progress article on the Principles of Diagnostic Assay Validation for Plant Pathogens (Cardwell et al. 2019). In 2020, a group of research collaborators launched a National Institute of Food and Agriculture seed grant (NIFA 2020) to gather as much information as possible about how diagnostic assays were being developed within the United States and to what degree validation research was being conducted. A desk study published by Sharma and Luster (2023) in this focus issue confirmed that there was little consensus on what metrics were needed, and few publications about diagnostic assays contained more than cursory descriptions of validation. Summaries of the outcomes of the 2020 NIFA seed grant deliberations appear in six perspective articles in this focus issue, which are summarized in the article by Cardwell et al. (2023).

    This PhytoFrontiers focus issue was developed to house the perspective articles derived from discussions among a large group of experts over several years during the NIFA seed grant and to create a space for other perspectives and research papers specific to plant disease diagnostic assay validation. The primary objective of the focus issue was to raise awareness about the importance of assay validation for diagnostic accuracy. Reviewers were instructed to pay attention to assay validation parameters. Authors were asked to explicitly state the purpose of the assay (fitness for purpose) and to describe which of the following validation parameters were addressed and how: analytical and diagnostic specificity, analytical and diagnostic sensitivity, repeatability, and reproducibility. Not all purposes for an assay require all parameters of validation, but, if parameters were not addressed in the manuscript, authors were encouraged to state the potential consequences of not knowing the omitted performance characteristics.

    The purpose is to provide the diagnostician user with what they should expect of the assay and to determine whether other confirmatory methods might be needed to assure a correct diagnosis.


    The first perspective articles in this focus issue (Cardwell et al. 2023; Geiser et al. 2023; Groth-Helms et al. 2023; Harmon et al. 2023; Sharma and Luster 2023; Stack and Cardwell 2023) are described and outlined in the need and vision for developing a diagnostic assay validation network in Cardwell et al. (2023). A concept note by Harrison et al. (2023) describes the basic concepts underlying test performance and validation, as well as the consensus interpretation by key stakeholder groups developed during the EU VALITEST project. The project produced a prototype mathematical framework for optimal sampling programs aiming to streamline collaboration between stakeholder groups. They recommend improvements to the collection and use of validation data, specifically with regard to test performance studies, which include everything from sample selection and testing to interpreting and communicating test results. Hiddink et al. (2023) recommend the use of indirect assays as prescreen assays, followed by direct assays. Trontin et al. (2023) present the outcomes of VALITEST, an EU-funded project on diagnostic assay validation. They discuss how this project improved diagnostic procedures such as organization of test performance studies, statistical analysis, and high-throughput technologies. In addition, it strengthened interaction between stakeholders in plant health, including public and private organizations.


    Validation of high-throughput sequencing for diagnostics

    High-throughput sequencing (HTS) is one of the newest molecular tools being brought into the arsenal of diagnostic assays for plant pathogens. Five papers appear in this focus issue on developing and validating HTS methods to detect and identify viroids, viruses, oomycetes, nematodes, fungi, and bacteria.

    Roman-Reyna et al. (2023) demonstrated the application of metagenomics as a valuable tool in identifying specific pathogens or pathogen strains of interest and informing the industry about emerging pathogen threats. In their concept note, they discuss adopting a metagenomic approach to detect and identify all possible pathogens in a sample, offering great potential for improved diagnostic capabilities in potato seed certification and related systems. Rong et al. (2023) discuss how to validate HTS for virus testing Musa germplasm and evaluating performance criteria by assessing the impact of contamination using an alien control. Their research demonstrated the impacts of cross-reacting contaminant virus sequences present in sample data on sensitivity and specificity metrics when using bioinformatic programs to detect viruses within a Musa metagenomic sequence. They report that HTS provides better analytical sensitivity than current molecular testing standards while allowing more inclusivity than classical indexing, but they point out that viral sequences integrated into the host genome could give a false positive signal. The authors also tout the technology to further virus discovery in metagenomic sequence data. van de Vossenberg et al. (2023) present the development and validation of an HTS test for mitogenome and rDNA assembly and annotation and its use in support of identification of nematodes of regulatory concern.

    Three papers demonstrate the development, validation, and use of e-probe diagnostic nucleic acid analysis (EDNA) that was originally described in Stobbe et al. (2013). Dang et al. (2023) demonstrate methods to validate e-probes for citrus viroids, viruses, and bacteria. Proaño-Cuenca et al. (2023) demonstrate how analytical and diagnostic sensitivity can be calculated for e-probes designed for oomycetes. They implemented the EDNA pipeline of HTS and analysis through the MiFi web application to design e-probes for the detection of oomycete pathogens. The e-probes were subsequently validated with metagenomic Nanopore sequencing to detect Phytophthora nicotianae, demonstrating the potential implementation of EDNA and MiFi for sensitive and specific detection of oomycetes in diseased plant tissue without the need for culturing or complex bioinformatics. Bocsanczy et al. (2023) needed a diagnostic tool specific enough to differentiate new strains of Ralstonia solanacearum that appeared in Florida highbush blueberries from endemic background R. solanacearum. The EDNA pipeline was used to develop e-probes that could differentiate three blueberry strains from a complex population of R. solanacearum isolated from blueberry. The e-probes were validated in silico, generating 21 simulated metagenomes in vitro by using blueberry DNA spiked with pathogen DNA and 22 in vivo soil and stem samples. Data for several methods of DNA extraction for soil and blueberry stems are presented, and several MinION libraries with or without barcodes were analyzed to evaluate sensitivity in multiplexed sequencing runs. MiFi blueberry e-probes were found to be highly specific and sensitive enough for reliable detection.

    Validation of PCR and recombinase polymerase amplification plant disease diagnostic assay methods

    Fungi and stramenopiles.

    Heller et al. (2023) redesigned and validated two qPCR assays to support sensitive detection of ohia tree pathogens, Ceratocystis lukuohia and C. huliohia, in soil and water samples. Based on earlier assays that are specific for the pathogens but not sensitive enough to deploy for environmental sampling, the increased sensitivity of the assays enables detection of windblown and soilborne pathogen dispersal to support disease mitigation decisions. Rebello et al. (2023) developed and validated a highly sensitive high-resolution melting assay to accurately detect and differentiate strawberry-infecting Neopestalotiopsis spp., separating them from other diseases with similar symptoms. The older strains cannot be distinguished morphologically from the newer, more aggressive strains that require early and intensive intervention, so this new assay can be used in a clinical setting to quickly identify causal organisms and recommend management actions. A multiplex real-time PCR was designed for rapid and sensitive detection of citrus fruit Elsinoë pathogens, which are difficult to culture or distinguish morphologically. Because the regulated disease can be confused with others early in symptom development, this multiplexed assay can increase throughput of diagnostic testing in symptomatic fruit (Elliott et al. 2023). SODplex, a suite of multiplex real-time PCR tools, was developed to streamline the detection and identification of P. ramorum mating types, crucial to informing mitigation and eradication efforts. The assays have high levels of accuracy, reduce false positives and negatives, and are robust across different instruments, operators, and temperatures, improving the toolbox available for the detection of P. ramorum and its lineages (Capron et al. 2023).

    Insects and nematodes.

    Taddei et al. (2023) describe a novel process for harmonized arthropod diagnosis in their approach to validation of identification protocols for Bactrocera dorsalis, considered the most destructive and economically important fruit fly worldwide. The morphological and molecular tests are validated in the same study, based on the same panel of samples, allowing a two-way control of the assigned values of samples. Potential critical issues that could represent weaknesses of the protocols are also discussed in detail, supporting the use of this system as a model for diagnostic assay validation.

    Hu et al. (2023) developed molecular markers to differentiate Meloidogyne chitwoodi races known to occur in the Pacific Northwest, enabling sensitive detection and differentiation of these pests that reduce quality and yield. The different nematode races infest potato and some rotation crops, so the specificity of the simplex PCR diagnosis supports cropping system management decisions.


    Three commercial assays were developed and subsequently validated for field screening or high-throughput diagnostic laboratory detection of tomato brown rugose fruit virus (ToBRFV), which has been documented to break resistance developed for tobacco mosaic virus and tomato mosaic virus (Eads et al. 2023). Although many commercial diagnostic tests had the capability to detect ToBRFV as a serological cross-reaction, no diagnostic methods in 2015 were able to detect ToBRFV specifically. As ToBRFV was unique in its resistance-breaking phenotype, a test with greater specificity was needed.


    Wang and Turechek (2020) developed a viable-cell detection assay (propidium 21 monoazide [PMA]-qPCR) for specific detection of viable (living) cells of Xanthomonas fragariae, the pathogen causing angular leaf spot of strawberry. The technology differs from qPCR as it detects only viable cells. In this issue, Turechek and Wang (2023) demonstrate latent class analysis (LCA) for validation of the new assay in the absence of a truly comparative gold standard. PMA-qPCR is not readily comparable to qPCR because detection of a nucleic acid signal from living cells is a subset of total nucleic acid, which includes both viable and nonviable cell signatures. LCA assumes that the underlying disease status in the populations is unknown or unobservable (latent) and uses the cross-classified results of two or more diagnostic tests to obtain information on test performance. This article describes an elegant statistical alternative for diagnostic assay validation in the absence of a comparative technology. Cesbron et al. (2023) evaluated a new rapid commercial detection test called AmplifyRP based on molecular recombinase polymerase amplification (RPA) for the detection of Xylella fastidiosa. Their results showed that sensitivity varies depending on the plant matrix, and the test limits performance to specific plant species.

    A multiplex conventional PCR (cPCR) assay was developed for detection and sub-pathotype determination of Xanthomonas citri pv. citri. Assay specificity was assessed by four different labs on a total of 146 X. citri pv. citri and 58 other Xanthomonas isolates. The assay demonstrated high analytical sensitivity, specificity, and selectivity for differentiating Asiatic citrus canker sub-pathotypes from symptomatic citrus tissue at the USDA-APHIS-PPQ Plant Pathogen Confirmatory Diagnostics Laboratory (Yasuhara-Bell et al. 2023). A real-time multiplex qPCR assay was designed and validated for the simultaneous detection of three ‘Candidatus Liberibacter’ species associated with citrus greening for use in a variety of citrus surveys, germplasm, or nursery stock programs that require different pathogen detection tools for their successful operation. The assay was successfully deployed on a large number of diverse samples at independent laboratories in four countries, thus demonstrating its transferability, applicability, practicability, and robustness (Osman et al. 2023).


    Because of the multitude of diagnostic needs and the recognition that the consequences of diagnostic outcomes can significantly affect our producers and stakeholders, a growing awareness has been given to harmonizing diagnostics in plant health. Seed and germplasm production industries send living materials to and from all parts of the world. Their need for rapid and highly multiplexed diagnostic tests demands constant development and resources. Examples provided by the European VALITEST project reveal how their endeavor builds the assay development and validation system that influences up to 52 countries engaged in the European and Mediterranean Plant Protection Organizations. Recognizing the need for a similar structure, a concept and details are presented to establish a Diagnostic Assay Validation Network in the United States, with the desired outcome of faster, more coordinated diagnostic assay development and an assembly of important tools and resources for assay validation.

    The author(s) declare no conflict of interest.


    • Bocsanczy, A. M., Espindola, A. S., Cardwell, K., and Norman, D. J. 2023. Development and validation of e-probes with the MiFi system for detection of Ralstonia solanacearum species complex in blueberries. PhytoFrontiers 3:137-147. AbstractGoogle Scholar
    • Capron, A., Herath, P., Alayon, D. I. O., Cervantes, S., Day, B., Brar, A., Bilodeau, G. J., Shamoun, S. F., Webber, J., Brasier, C., Feau, N., and Hamelin, R. C. 2023. SODplex, a series of hierarchical multiplexed real-time PCR assays for the detection and lineage identification of Phytophthora ramorum, the causal agent of sudden oak death and sudden larch death. PhytoFrontiers 3:173-185. Google Scholar
    • Cardwell, K., Dennis, G., Flannery, A., Fletcher, J., Luster, D., Nakhla, M., Rice, A., Shiel, P., Stack, J., Walsh, C., and Levy, L. 2018. Diagnostic assay validation terminology. The Plant Health Instructor. Google Scholar
    • Cardwell, K., Dennis, G., Flannery, A. R., Fletcher, J., Luster, D., Nakhla, M., Rice, A., Shiel, P., Stack, J., Walsh, C., and Levy, L. 2019. Principles of diagnostic assay validation for plant pathogens: A basic review of concepts. Plant Health Prog. 19:272-278. LinkGoogle Scholar
    • Cardwell, K. F., Harmon, C. L., Luster, D. G., Stack, J. P., Hyten, A. M., Sharma, P., and Nakhla, M. K. 2023. The need and a vision for a diagnostic assay validation network. PhytoFrontiers 3:9-17. AbstractGoogle Scholar
    • Cesbron, S., Dupas, E., and Jacques, M.-A. 2023. Evaluation of the AmplifyRP XRT+ kit for the detection of Xylella fastidiosa by recombinase polymerase amplification. PhytoFrontiers 3:225-234. LinkGoogle Scholar
    • Dang, T., Wang, H., Espindola, A. S., Habiger, J., Vidalakis, G., and Cardwell, K. 2023. Development and statistical validation of e-probe diagnostic nucleic acid analysis (EDNA) assays for the detection of citrus pathogens from raw high-throughput sequencing data. PhytoFrontiers 3:113-123. AbstractGoogle Scholar
    • Eads, A., Groth-Helms, D., Davenport, B., Cha, X., Li, R., Walsh, C., and Schuetz, K. 2023. The commercial validation of three tomato brown rugose fruit virus assays. PhytoFrontiers 3:206-213. AbstractGoogle Scholar
    • Elliott, A. J., van Raak, M. M. J. P., Barnes, A. V., Field, C. J., van Duijnhoven, A. A. L. A. M., Webb, K., and van de Vossenberg, B. T. L. H. 2023. Real-time PCR detection of Elsinoë spp. on Citrus. PhytoFrontiers 3:164-172. AbstractGoogle Scholar
    • Geiser, D. M., Martin, F. N., Espindola, A. S., Brown, J. K., Bell, T. H., Yang, Y., and Kang, S. 2023. Knowledge gaps, research needs, and opportunities in plant disease diagnostic assay development and validation. PhytoFrontiers 3:51-63. LinkGoogle Scholar
    • Groth-Helms, D., Rivera, Y., Martin, F. N., Arif, M., Sharma, P., and Castlebury, L. A. 2023. Terminology and guidelines for diagnostic assay development and validation: Best practices for molecular tests. PhytoFrontiers 3:23-35. LinkGoogle Scholar
    • Harmon, C. L., Castlebury, L. A., Boundy-Mills, K., Broders, K. D., Hyten, A. M., Jacobs, J. L., Knight-Connoni, V. K., Mollov, D., Riojas, M. A., and Sharma, P. 2023. Standards of diagnostic validation: Recommendations for reference collections. PhytoFrontiers 3:43-50. LinkGoogle Scholar
    • Harrison, C., Agstner, B., Tomlinson, J., Macarthur, R., and van den Berg, F. 2023. Understanding and improving the collection and use of diagnostic test validation data: Reflections and next steps. PhytoFrontiers 3:64-70. AbstractGoogle Scholar
    • Heller, W. P., Harrington, T. C., Brill, E., and Keith, L. M. 2023. High-sensitivity ITS real-time PCR assays for detection of Ceratocystis lukuohia and Ceratocystis huliohia in soil and air samples. PhytoFrontiers 3:148-155. AbstractGoogle Scholar
    • Hiddink, G. A., Willmann, R., Woudenberg, J. H. C., and Souza-Richards, R. 2023. Seed health testing: Doing things right. PhytoFrontiers 3:71-74. LinkGoogle Scholar
    • Hu, S., Franco, J., Bali, S., Chavoshi, S., Brown, C., Mojtahedi, H., Quick, R., Cimrhakl, L., Ingrham, R., Gleason, C., and Sathuvalli, V. 2023. Diagnostic molecular markers for identification of different races and a pathotype of Columbia root knot nematode. PhytoFrontiers 3:199-205. AbstractGoogle Scholar
    • Osman, F., Dang, T., Bodaghi, S., Haq, R., Lavagi-Craddock, I., Rawstern, A., Rodriguez, E., Polek, M., Wulff, N. A., Roberts, R., Pietersen, G., Englezou, A., Donovan, N., Folimonova, S. Y., and Vidalakis, G. 2023. Update and validation of the 16S rDNA qPCR assay for the detection of three ‘Candidatus Liberibacter species’ following current MIQE guidelines and workflow. PhytoFrontiers 3:246-258. AbstractGoogle Scholar
    • Proaño-Cuenca, F., Espindola, A. S., and Garzon, C. D. 2023. Detection of Phytophthora, Pythium, Globisporangium, Hyaloperonospora, and Plasmopara species in high-throughput sequencing data by in silico and in vitro analysis using Microbe Finder (MiFi). PhytoFrontiers 3:124-136. AbstractGoogle Scholar
    • Rebello, C. S., Wang, N.-Y., Marin, M. V., Baggio, J. S., and Peres, N. A. 2023. Detection and species differentiation of Neopestalotiopsis spp. from strawberry (Fragaria × ananassa) in Florida using a high-resolution melting analysis. PhytoFrontiers 3:156-163. LinkGoogle Scholar
    • Roman-Reyna, V., Rioux, R., Babler, B., Klass, T., and Jacobs, J. 2023. Concept note: Toward metagenomic sequencing for rapid, sensitive, and accurate detection of bacterial pathogens in potato seed production. PhytoFrontiers 3:82-90. AbstractGoogle Scholar
    • Rong, W., Rollin, J., Hanafi, M., Roux, N., and Massart, S. 2023. Validation of high-throughput sequencing as virus indexing test for Musa germplasm: Performance criteria evaluation and contamination monitoring using an alien control. PhytoFrontiers 3:91-102. AbstractGoogle Scholar
    • Sharma, P., and Luster, D. G. 2023. State of the field of plant pathogen diagnostic assay development and validation. PhytoFrontiers 3:18-22. AbstractGoogle Scholar
    • Stack, J. P., and Cardwell, K. F. 2023. Communications ecosystem to support the assay validation community: A concept. PhytoFrontiers 3:36-42. AbstractGoogle Scholar
    • Stobbe, A. H., Daniels, J., Espindola, A. S., Verma, R., Melcher, U., Ochoa-Corona, F., Garzon, C., Fletcher, J., and Schneider, W. 2013. E-probe Diagnostic Nucleic acid Analysis (EDNA): A theoretical approach for handling of next generation sequencing data for diagnostics. J. Microbiol. Methods 94:356-366. CrossrefGoogle Scholar
    • Taddei, A., Reisenzein, H., Mouttet, R., Lethmayer, C., Egartner, A., Gottsberger, R. A., Blümel, S., Heiss, C., Pohn, C., and Reynaud, P. 2023. Morphological and molecular identification protocols for Bactrocera dorsalis: A joint validation study. PhytoFrontiers 3:186-198. AbstractGoogle Scholar
    • Trontin, C., Agstner, B., Altenbach, D., Anthoine, G., Bagińska, H., Brittain, I., Chabirand, A., Chappé, A. M., Dahlin, P., Dreo, T., Freye-Minks, C., Gianinazzi, C., Harrison, C., Jones, G., Luigi, M., Massart, S., Mehle, N., Mezzalama, M., Mouaziz, H., Petter, F., Ravnikar, M., Raaymakers, T. M., Renvoisé, J. P., Rolland, M., Santos Paiva, M., Seddas, S., van der Vlugt, R., and Vučurović, A. 2021. VALITEST: Validation of diagnostic tests to support plant health. EPPO Bull. 51:198-206. CrossrefGoogle Scholar
    • Trontin, C., Agstner, B., Altenbach, D., Anthoine, G., Bagińska, H., Brittain, I., Chabirand, A., Chappé, A.-M., Dahlin, P., Dreo, T., Freye, C., Gianinazzi, C., Harrison, C., Jones, G., Kaiser, M. S., Luigi, M., Massart, S., Mehle, N., Mezzalama, M., Mouaziz, H., Ravnikar, M., Raaymakers, T., Renvoisé, J.-P., Rolland, M., Santos-Paiva, M., Seddas, S., van der Vlugt, R. A., Vučurović, A., and Petter, F. 2023. What did we achieve through VALITEST, an EU project on validation in plant pest diagnostics? PhytoFrontiers 3:75-81. AbstractGoogle Scholar
    • Turechek, W. W., and Wang, H. 2023. Evaluation of a viable-cell detection assay for Xanthomonas fragariae with latent class analysis. PhytoFrontiers 3:214-224. AbstractGoogle Scholar
    • van de Vossenberg, B. T. L. H., van Kempen, T. L., van Duijnhoven, A. A. L. A. M., Warbroek, T., van der Gouw, L. P., Silva De Oliveira, D. A., van Heese, E. Y. J., and Karssen, G. 2023. Development and validation of a high-throughput sequencing test for mitogenome and rDNA assembly and annotation, and its use in support of nematode identification of regulatory concern. PhytoFrontiers 3:103-112. LinkGoogle Scholar
    • Wang, H., and Turechek, W. W. 2020. Detection of viable Xanthomonas fragariae cells in strawberry using propidium monoazide and long-amplicon quantitative PCR. Plant Dis. 104:1105-1112. LinkGoogle Scholar
    • Yasuhara-Bell, J., Santillana, G., Robène, I., Pruvost, O., Nakhla, M., and Mavrodieva, V. 2023. Genome-informed multiplex conventional PCR for identification and differentiation of Xanthomonas citri pv. citri subpathotypes, the causal agents of Asiatic citrus canker. PhytoFrontiers 3:235-245. AbstractGoogle Scholar

    Funding: Funding was provided by the USDA National Institute of Food and Agriculture, initially as a seed grant to explore diagnostic assay validation in the U.S. (Grant #2020-67014-30972) and then a full grant, Establishing the Diagnostic Assay Validation Network (Grant #2022-68013-36537).

    The author(s) declare no conflict of interest.

    Copyright © 2023 The Author(s). This is an open access article distributed under the CC BY-NC-ND 4.0 International license.