Extending the discussion on inconsistency in forensic decisions and results.

Publisher: Blackwell Pub Country of Publication: United States NLM ID: 0375370 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1556-4029 (Electronic) Linking ISSN: 00221198 NLM ISO Abbreviation: J Forensic Sci Subsets: MEDLINE

Publication: 2006- : Malden, MA : Blackwell Pub.
Original Publication: [Chicago, Ill.] : Callaghan and Co., 1956-

Software*; Humans ; Likelihood Functions ; Monte Carlo Method ; DNA Fingerprinting ; Genotype ; Laboratories/standards ; Decision Making ; Forensic Genetics/methods

The subject of inter- and intra-laboratory inconsistency was recently raised in a commentary by Itiel Dror. We re-visit an inter-laboratory trial, with which some of the authors of this current discussion were associated, to diagnose the causes of any differences in the likelihood ratios (LRs) assigned using probabilistic genotyping software. Some of the variation was due to different decisions that would be made on a case-by-case basis, some due to laboratory policy and would hence differ between laboratories, and the final and smallest part was the run-to-run difference caused by the Monte Carlo aspect of the software used. However, the net variation in LRs was considerable. We believe that most laboratories will self-diagnose the cause of their difference from the majority answer and in some, but not all instances will take corrective action. An inter-laboratory exercise consisting of raw data files for relatively straightforward mixtures, such as two mixtures of three or four persons, would allow laboratories to calibrate their procedures and findings.
(© 2024 American Academy of Forensic Sciences.)

Dror IE. The most consistent finding in forensic science is inconsistency. J Forensic Sci. 2023;68(6):1851–1855. https://doi.org/10.1111/1556‐4029.15369.
National Academies of Sciences Engineering and Medicine. Reproducibility and Replicability in Science. Washington, DC: National Academies Press; 2019. https://doi.org/10.17226/25303.
Organization of Scientific Area Committees. OSAC preferred terms. 2023 July. https://www.nist.gov/system/files/documents/2024/01/18/OSAC%20Preferred%20Terms_July%202023%20Revised.pdf. Accessed 10 May 2024.
Hill CR, Duewer DL, Kline MC, Coble MD, Butler JM. US population data for 29 autosomal STR loci. Forensic Sci Int Genet. 2013;7(3):e82–e83. https://doi.org/10.1016/j.fsigen.2012.12.004.
Moretti TR, Moreno LI, Smerick JB, Pignone ML, Hizon R, Buckleton JS, et al. Population data on the expanded CODIS core STR loci for eleven populations of significance for forensic DNA analyses in the United States. Forensic Sci Int Genet. 2016;25:175–181. https://doi.org/10.1016/j.fsigen.2016.07.022.
Bright J‐A, Cheng K, Kerr Z, McGovern C, Kelly H, Moretti TR, et al. STRmix™ collaborative exercise on DNA mixture interpretation. Forensic Sci Int Genet. 2019;40:1–8. https://doi.org/10.1016/j.fsigen.2019.01.006.
Alfonse LE, Garrett AD, Lun DS, Duffy KR, Grgicak CM. A large‐scale dataset of single and mixed‐source short tandem repeat profiles to inform human identification strategies: PROVEDIt. Forensic Sci Int Genet. 2018;32:62–70. https://doi.org/10.1016/j.fsigen.2017.10.006.
Applied Biosystems Life Technologies. GeneMapper®ID‐X software, Version 1.5 Publication Number 100031707 Revision A. 2015 https://assets.thermofisher.com/TFS‐Assets/LSG/manuals/100031707_GeneMapIDX_ver1_5_ReferenceGuide.pdf. Accessed 30 May 2024.
Applied Biosystems. GeneMapper® ID‐X software, Version 1.2. 2009 https://assets.thermofisher.com/TFS‐Assets/LSG/manuals/cms_072557.pdf. Accessed 30 May 2024.
Applied Biosystems. GlobalFiler™ and GlobalFiler™ IQC PCR amplification kits: user guide. https://assets.thermofisher.com/TFS‐Assets/LSG/manuals/4477604.pdf. Accessed 30 May 2024.
Taylor D, Balding D. How can courts take into account the uncertainty in a likelihood ratio? Forensic Sci Int Genet. 2020;48:102361. https://doi.org/10.1016/j.fsigen.2020.102361.
Thompson WC. Uncertainty in probabilistic genotyping of low template DNA: a case study comparing STRMix and TrueAllele. J Forensic Sci. 2023;68(3):1049–1063. https://doi.org/10.1111/1556‐4029.15225.
Taylor D, Bright J‐A, Buckleton J. Interpreting forensic DNA profiling evidence without specifying the number of contributors. Forensic Sci Int Genet. 2014;13:269–280. https://doi.org/10.1016/j.fsigen.2014.08.014.
Butler J, Iyer H, Press R, Taylor MK, Vallone PM, Willis S. DNA mixture interpretation: a NIST scientific foundation review. Gaithersburg, MD: National Institute of Standards and Technology (NIST); 2021.
Buckleton JS, Pugh SN, Bright J‐A, Taylor DA, Curran JM, Kruijver M, et al. Are low LRs reliable? Forensic Sci Int Genet. 2020;49:102350. https://doi.org/10.1016/j.fsigen.2020.102350.
Bright J‐A, Stevenson KE, Curran JM, Buckleton JS. The variability in likelihood ratios due to different mechanisms. Forensic Sci Int Genet. 2015;14:187–190. https://doi.org/10.1016/j.fsigen.2014.10.013.
Berger CEH, Slooten K. The LR does not exist. Sci Justice. 2016;56(5):388–391. https://doi.org/10.1016/j.scijus.2016.06.005.
Ramos D, Meuwly D, Haraksim R, Berger CEH. Validation of forensic automatic likelihood ratio methods. In: Banks D, Kafadar K, Kaye D, Tackett M, editors. Handbook of forensic statistics. Boca Raton, FL: Chapman and Hall/CRC; 2020.
Taylor D, Bright JA, Buckleton J, Curran J. An illustration of the effect of various sources of uncertainty on DNA likelihood ratio calculations. Forensic Sci Int Genet. 2014;11:56–63. https://doi.org/10.1016/j.fsigen.2014.02.003.
Vul E, Pashler H. Measuring the crowd within: probabilistic representations within individuals. Psychol Sci. 2008;19(7):645–647. https://doi.org/10.1111/j.1467‐9280.2008.02136.x.
Kahneman D, Sibony O, Sunstein CR. Judgment. Noise: a flaw in human judgment. New York, NY: Little, Brown; 2021.
ANSI National Accrediation Board (ANAB). ANAB guidance on proficiency testing/inter‐laboratory comparisons. Milwaukee, MI: ANAB; 2016.
Hahn M, Anslinger K, Eckert M, Fimmers R, Grethe S, Hohoff C, et al. Gemeinsame empfehlungen der projektgruppe “Biostatistische DNA‐Berechnungen” und der spurenkommission zur biostatistischen bewertung forensischer DNA‐analytischer befunde mit vollkontinuierlichen modellen (VKM) [common recommendations of the project group “biostatistical DNA calculations” and the trace commission for the biostatistical assessment of forensic DNA‐analytical findings with full continuous models (VKM)]. Dent Rec. 2023;33(1):3–12. https://doi.org/10.1007/s00194‐022‐00599‐5.
Templin M, Zimmermann P, Kranz S, Eckert M, Leuker C, Razbin S, et al. Einsatz vollkontinuierlicher Modelle zur biostatistischen Bewertung forensischer DNA‐analytischer Befunde [use of fully continuous models for the biostatistical assessment of forensic DNA‐analytical findings]. Dent Rec. 2023;33(1):13–29. https://doi.org/10.1007/s00194‐022‐00600‐1.
Hahn M, Courts C, Eckert M, Fimmers R, Grethe S, Kranz S, et al. Authors' response. J Forensic Sci. 2024;69(2):736–738. https://doi.org/10.1111/1556‐4029.15426.
Buckleton J, Susik M, Curran JM, Cheng K, Taylor D, Bright J‐A, et al. A diagnosis of the primary difference between EuroForMix and STRmix™. J Forensic Sci. 2024;69(1):40–51. https://doi.org/10.1111/1556‐4029.15387.
Hopwood AJ, Puch‐Solis R, Tucker VC, Curran JM, Skerrett J, Pope S, et al. Consideration of the probative value of single donor 15‐plex STR profiles in UK populations and its presentation in UK courts. Sci Justice. 2012;52(3):185–190. https://doi.org/10.1016/j.scijus.2012.05.005.
Kruijver M, Kelly H, Taylor D, Buckleton J. Addressing uncertain assumptions in DNA evidence evaluation. Forensic Sci Int Genet. 2023;66:102913. https://doi.org/10.1016/j.fsigen.2023.102913.
Buckleton J, Taylor D, Bright J‐A, Hicks T, Curran J. When evaluating DNA evidence within a likelihood ratio framework, should the propositions be exhaustive? Forensic Sci Int Genet. 2021;50:102406. https://doi.org/10.1016/j.fsigen.2020.102406.
Taylor D, Buckleton J. Combining artificial neural network classification with fully continuous probabilistic genotyping to remove the need for an analytical threshold and electropherogram reading. Forensic Sci Int Genet. 2023;62:102787. https://doi.org/10.1016/j.fsigen.2022.102787.

15PNIJ-21-GG-02710-SLFO National Institute of Justice

Keywords: forensic DNA analysis; inter‐laboratory consistency; quality assurance

Date Created: 20240610 Date Completed: 20240626 Latest Revision: 20240626

20240627

10.1111/1556-4029.15558

38853374