Sensitivity is not an intrinsic property of a diagnostic test: empirical evidence from histological diagnosis of Helicobacter pylori infection
© Lunet et al; licensee BioMed Central Ltd. 2009
Received: 18 May 2009
Accepted: 24 December 2009
Published: 24 December 2009
We aimed to provide empirical evidence of how spectrum effects can affect the sensitivity of histological assessment of Helicobacter pylori infection, which may contribute to explain the heterogeneity in prevalence estimates across populations with expectedly similar prevalence.
Cross-sectional evaluation of dyspeptic subjects undergoing upper digestive endoscopy, including collection of biopsy specimens from the greater curvature of the antrum for assessment of H. pylori infection by histopathological study and polymerase chain reaction (PCR), from Portugal (n = 106) and Mozambique (n = 102) following the same standardized protocol.
In the Portuguese sample the prevalence of infection was 95.3% by histological assessment and 98.1% by PCR. In the Mozambican sample the prevalence was 63.7% and 93.1%, respectively. Among those classified as infected by PCR, the sensitivity of histological assessment was 96.2% among the Portuguese and 66.3% among the Mozambican. Among those testing positive by both methods, 5.0% of the Portuguese and 20.6% of the Mozambican had mild density of colonization.
This study shows a lower sensitivity of histological assessment of H. pylori infection in Mozambican dyspeptic patients compared to the Portuguese, which may be explained by differences in the density of colonization, and may contribute to explain the heterogeneity in prevalence estimates across African settings.
The dependence of positive and negative predictive values of diagnostic tests on the frequency of the target condition in the population under study is widely recognized and easy to quantify, as predictive values are seen as post-test probabilities of disease, determined by the pre-test probability which is changed by knowledge of the results of the test. Sensitivity and specificity, despite being stable across the settings in which the test is used, actually also depend on the characteristics of patients in whom those parameters were assessed. There is much evidence for varying specificity, depending on alternative diagnoses and comorbid conditions that affect patients without the target disease and which may affect the test. Sensitivity is conditioned by the severity of the disease that is being diagnosed and comorbid conditions that may lead to false negative results . However, this possibility is often not acknowledged, since these situations are much less frequently reported .
The external validity of the results of studies evaluating the accuracy of a diagnostic test may be affected by variations in case mix - spectrum effects - either as a consequence of flawed study designs , or resulting from true variation across populations. Examples of the latter, however, are seldom described.
We provide empirical evidence of how spectrum effects may affect the sensitivity of histological assessment of Helicobacter pylori infection, in two different settings, Portugal and Mozambique, in which high prevalences of infection in the general population are to be expected .
Dyspeptic subjects from Portugal and Mozambique underwent upper endoscopy evaluation and gastric biopsy samples were collected following the same standardized protocol, including the collection of biopsy specimens from the greater curvature of the antrum for assessment of H. pylori infection by histopathological study and polymerase chain reaction (PCR), as previously described [5, 6].
The ethics committees of involved institutions approved the study, and all participants provided written informed consent.
Portuguese sample 
All workers from the Viana do Castelo shipyard, north of Portugal, were invited for a gastric pathology survey in 1998. Four hundred and sixty volunteered to the study and completed a physician-administered questionnaire on digestive symptoms. An upper digestive endoscopy was performed to 107 participants reporting dyspeptic symptoms and biopsy samples from the antrum were available from 106 participants (median age: 44 years, interquartile range: 39-48 years; 96.2% men; 100% white).
Mozambican sample 
Among patients observed at the outpatient department of Gastroenterology at the Maputo Central Hospital, adults with dyspeptic symptoms and clinical criteria for upper digestive endoscopy were consecutively invited to this study between August 2005 and May 2006. One gastric cancer patient was excluded and the present series includes 102 participants for whom antrum biopsies with adequate size were considered for histopathological study (median age: 37 years, interquartile range: 29-46 years; 34.3% men; 93.1% black).
Collection of biopsy specimens
The protocol included the collection of four biopsy specimens--one from the greater curvature of the antrum, two from the incisura angularis, and one from the anterior wall of the corpus, for the histopathological study. In the Mozambican sample only 57 subjects had biopsies from the corpus available due to a high frequency of erroneous labelling as corpus of samples collected from antrum or incisura.
In the Portuguese participants an additional antral biopsy specimen was collected for PCR analysis and for the Mozambican subjects an antral biopsy used for histopathological study was also was used for PCR.
Biopsy specimens were fixed in 10% formalin and embedded in paraffin and cut into 4-μm sections. Modified Giemsa stained sections were used to identify H. pylori. Histological evaluation of samples from both settings was performed according to the Modified Sydney system  by two experienced pathologists, using a semi-quantitative scoring (0 = absent; 1 = mild; 2 = moderate; 3 = marked) for density of colonization with H. pylori.
Assessment of H. pylori infection by PCR
In Portuguese patients, total DNA was extracted from frozen antral biopsy specimens using the method described by Boom et al . Briefly, biopsies were homogenized in guanidinium isothiocyanate, using a sterile micropestle. DNA was captured onto silica particles, washed, and eluted in 100 ml of 10 mmol/L Tris-HCl, pH 8.3. Two microliters were used for each PCR. H. pylori detection was performed by multiplex amplification of vacA and cagA genes followed by reverse hybridization on a line probe assay (LiPA) as described earlier .
In Mozambican samples, DNA was extracted from formalin-fixed, paraffin-embedded antral biopsy specimens after digestion with Proteinase K for at least 12 hours at 55°C. Proteinase K was inactivated by incubation at 95°C for 10 minutes. Ten microliters of the lysate were used for PCR. Detection of H. pylori was performed with the same primers used for the Portuguese samples. Amplified products were visualised after electrophoresis in 2% agarose gels.
Since only one biopsy (from the antrum) was used for PCR, we compared the results of PCR assessment of infection with those from histological assessment using a biopsy collected from a similar location, to ensure the comparability of the two results obtained with the two methods. PCR was defined as the reference standard and sensitivity of histological assessment was computed with respective 95% Confidence Intervals (95%CI). Proportions were compared using the χ 2 test.
Sensitivity analyses were also conducted comparing the results from the histological assessment of infection in four biopsies collected from different stomach locations with those from PCR evaluation of the antral specimen, and restricting the analysis to subjects not reporting previous eradication and/or proton pump inhibitors (PPI)/H2 receptor blockers treatment.
In the Portuguese sample the prevalence of infection was 95.3% by histological assessment and 98.1% by PCR. In the Mozambican sample the prevalence was 63.7% and 93.1%, respectively.
Sensitivity of histological assessment of Helicobacter pylori infection using PCR analysis as the reference standard, in Portuguese and Mozambican populations of dyspeptic patients.
PCR (reference standard)
Histological assessment (index text)
Participants not reporting previous eradication treatment
Participants not reporting previous PPI/H2 receptor blockers treatment
Excluding participants reporting previous eradication and PPI/H2 receptor blockers treatments
Among the Mozambicans, sensitivity was not improved when defining infection by histopathological analysis of 4 biopsy samples from corpus, antrum and incisura angularis (66.7%, 95%CI: 52.9%-78.5%).
Thirty percent of the Mozambican subjects reported a previous eradication treatment, with a similar prevalence of infection and H. pylori density score regardless of treatment (previous eradication therapy vs. no previous treatment: no infection - 34.5% vs. 33.9%; mild - 17.2% vs. 12.9%; moderate - 10.3% vs. 29.0%; marked - 37.9% vs. 24.2%, p = 0.195). Fifty-seven percent of the Mozambican subjects reported a previous treatment with PPI/H2 receptor blockers with a similar prevalence of infection and H. pylori density score regardless of treatment (previous treatment with PPI/H2 receptor blockers vs. no previous treatment: no infection - 29.6% vs. 40.5%; mild - 11.1% vs. 16.2%; moderate - 22.2% vs. 24.3%; marked - 37.0% vs. 18.9%, p = 0.304). Among the Portuguese participants, 6% reported previous eradication therapy, and 25% declared to have used PPI/H2 receptor blockers.
The sensitivity of the histological assessment of H. pylori infection was considerably lower in Mozambique than in Portugal, which may be explained by a lower density of colonization among Mozambican patients.
In Portuguese samples, DNA was extracted from frozen antral biopsies and used for PCR, followed by reverse hybridization onto specific probes, whereas in Mozambican samples, DNA was extracted from formalin-fixed, paraffin-embedded biopsies and used for PCR, followed by electrophoresis in agarose gels stained with ethidium bromide. Visual inspection of PCR products on agarose gels provides only limited reliability, as compared to hybridization to allele-specific probes. However, it has been previously shown that H. pylori genotypes obtained from paraffin sections matched those corresponding cultured strains . Nevertheless, even if PCR results for Mozambican samples were underestimated, this would mean that the sensitivity of histological assessment for this population would even be lower, and more discrepant than those obtained for the Portuguese samples.
Co-infections with other Helicobacter species have been described in African populations , but the PCR analysis that we have used is specific for H. pylori (we have used primers directed to the vacA gene which is only present in this species). Primer sequences used in this study were analyzed for specificity by using the Blast program at the National Institute of Health Data libraries .
The histological identification of H. pylori in tissue sections may be affected by biopsy and observer-related factors. Sensitivity is higher when at least samples from the antrum and the body are analyzed, but the proportion of infected subjects did not vary meaningfully when we considered the histological assessment of infection status using biopsies from the three locations, which is in accordance with the fact that the antrum biopsies tend to yield a higher prevalence of infection than the other locations , strengthening the validity of our conclusions. Moreover, no meaningful increase in the prevalence of infection when assessed by PCR was to be expected, as it was well above 90% when only one biopsy is used for this purpose.
Staining methods such as modified Giemsa are highly sensitive, but histological examination relies on the experience of the histopathologist to recognize the typical morphology of the organism, and small numbers of organisms may be missed [14, 15], while PCR methods have been described as more sensitive and not dependent on the level of experience of the pathologist [16, 17].
In our study, samples from both settings were collected and processed following similar methods and were evaluated by the same experienced pathologists. It is unlikely that differences in technical aspects or inter-observer and laboratory variations explain the differences in the results obtained in Portuguese and Mozambican samples. The same applies to the grading of the density of colonization, which was conducted following the same criteria in both samples.
Although coccoid forms (suggesting possible degenerative forms of H. pylori) could have been found in the negative specimens, and would contribute to the low perceived sensitivity of histological assessment of infection, the pathologists where aware of this possibility. Even considering the difficulties in classifying correctly these uncharacteristic forms in the absence of findings of structures with the characteristic H. pylori shape, these were seldom observed and the differences observed in sensitivity or in the density of H. pylori colonization cannot be ascribed to such phenomenon.
A lower sensitivity of histology for the assessment of H. pylori infection has been recognized after partially effective eradication therapy [15, 16], as low levels of recurrent infection can be easily missed by biopsy, leading to overestimation of therapeutic efficacy . The proportion of Mozambican subjects reporting a previous eradication treatment was surprisingly high, which could explain the lower density of colonization among these patients, but no information was obtained regarding the success of eradication or time since eradication. No significant differences were found in infection status or density of H. pylori colonization according to self-reported eradication treatment, probably reflecting the large potential for erroneous reporting of such a specific therapy, and sensitivity was similar when only subjects not reporting an eradication treatment were considered. Similarly, the previous use of PPI/H2 receptor blockers was also self-reported and no information was collected regarding current treatment or time since last treatment course, but sensitivity was similar when only subjects not reporting the use of PPI/H2 receptor blockers were considered.
Gastric atrophy and intestinal metaplasia lead to a pH increase in the stomach, which can create an unfavorable environment for H. pylori survival, and could contribute to lower density of colonization [19, 20]. The density of H. pylori may be increased in macroscopic erosions . However, only 14.7% of the Mozambican patients had chronic atrophic gastritis (5.9%) or intestinal metaplasia (8.8%), in comparison with 43.9% in the Portuguese subjects (chronic atrophic gastritis: 6.5%; intestinal metaplasia: 37.4%), and erosions/ulcerations were observed in one Portuguese and four Mozambicans. Therefore, the unequal distribution of these lesions could not explain the differences in the prevalence of infection detected by histology.
Although the samples were evaluated in different time periods and present different characteristics regarding the participants' ethnicity and gender, our analysis only included subjects testing positive for H. pylori infection by PCR and the above differences between the Portuguese and the Mozambican sample are not likely to account for our conclusions.
In studies conducted in dyspeptic patients from different African populations the overall prevalence of H. pylori infection, assessed by histology, was approximately 72%, varying from 25% in Uganda to 97% in Ghana , although a high prevalence of infection is expected given its strong association with low socioeconomic status. The heterogeneity of estimates across African settings, as well as unexpectedly low figures, is likely to be explained both by observer-related factors and differences in the density of colonization by H. pylori.
Mozambique and other African countries have a low frequency of gastric cancer and gastric precancerous lesions, despite the high prevalence of infection [23, 24], reflecting what has been called the African enigma. In addition to providing an example of how spectrum effects can affect the sensitivity of histological assessment of H. pylori infection, our results add a piece of information to our current understanding of the "enigma", suggesting that despite the absence of differences in the frequency of infection with the more virulent strains in Portugal and Mozambique , the higher prevalence of mild infections in Africa may be associated with a lower incidence of cancer.
This study provides empirical evidence of how subtle spectrum effects, not induced by a flawed study design, may be responsible for large differences on the sensitivity of histological assessment of H. pylori infection, and may contribute to explain the heterogeneity in prevalence estimates across African settings.
Grants from the Fundação Calouste Gulbenkian (projects FC-54918 and FC-68697) and Fundação para a Ciência e a Tecnologia (JNICT UI&D 51/94) are gratefully acknowledged.
- Ransohoff DF, Feinstein AR: Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978, 299: 926-930. 10.1056/NEJM197810262991705.PubMedGoogle Scholar
- Leeflang MM, Bossuyt PM, Irwig L: Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis. J Clin Epidemiol. 2009, 62: 5-12. 10.1016/j.jclinepi.2008.04.007.PubMedGoogle Scholar
- Rutjes AW, Reitsma JB, Vandenbroucke JP, Glas AS, Bossuyt PM: Case-control and two-gate designs in diagnostic accuracy studies. Clin Chem. 2005, 51: 1335-1341. 10.1373/clinchem.2005.048595.PubMedGoogle Scholar
- Lunet N, Barros H: Helicobacter pylori infection and gastric cancer in developing countries: revisiting the "enigmas". About cancer in Africa: from epidemiology to biomedical applications and perspectives. Edited by: Ly A, Khayat D. 2006, Paris: INCAGoogle Scholar
- Peleteiro B, Lunet N, Figueiredo C, Carneiro F, David L, Barros H: Smoking, Helicobacter pylori virulence, and type of intestinal metaplasia in Portuguese males. Cancer Epidemiol Biomarkers Prev. 2007, 16: 322-326. 10.1158/1055-9965.EPI-06-0885.PubMedGoogle Scholar
- Carrilho C, Modcoicar P, Cunha L, Ismail M, Guisseve A, Lorenzoni C, Fernandes F, Peleteiro B, Almeida R, Figueiredo C, David L, Lunet N: Prevalence of Helicobacter pylori infection, chronic gastritis, and intestinal metaplasia in Mozambican dyspeptic patients. Virchows Arch. 2009, 454: 153-160. 10.1007/s00428-008-0713-7.PubMedGoogle Scholar
- Dixon MF, Genta RM, Yardley JH, Correa P: Classification and grading of gastritis. The updated Sydney System. International Workshop on the Histopathology of Gastritis, Houston 1994. Am J Surg Pathol. 1996, 20: 1161-1181. 10.1097/00000478-199610000-00001.PubMedGoogle Scholar
- Boom R, Sol CJ, Salimans MM, Jansen CL, Wertheim-van Dillen PM, Noordaa van der J: Rapid and simple method for purification of nucleic acids. J Clin Microbiol. 1990, 28: 495-503.PubMedPubMed CentralGoogle Scholar
- van Doorn LJ, Figueiredo C, Sanna R, Pena S, Midolo P, Ng EK, Atherton JC, Blaser MJ, Quint WG: Expanding allelic diversity of Helicobacter pylori vacA. J Clin Microbiol. 1998, 36: 2597-2603.PubMedPubMed CentralGoogle Scholar
- Scholte GH, van Doorn LJ, Cats A, Bloemena E, Lindeman J, Quint WG, Meuwissen SG, Kuipers EJ: Genotyping of Helicobacter pylori in paraffin-embedded gastric biopsy specimens: relation to histological parameters and effects on therapy. Am J Gastroenterol. 2002, 97: 1687-1695. 10.1111/j.1572-0241.2002.05775.x.PubMedGoogle Scholar
- Fritz EL, Slavik T, Delport W, Olivier B, Merwe van der SW: Incidence of Helicobacter felis and the effect of coinfection with Helicobacter pylori on the gastric mucosa in the African population. J Clin Microbiol. 2006, 44: 1692-1696. 10.1128/JCM.44.5.1692-1696.2006.PubMedPubMed CentralGoogle Scholar
- van Doorn LJ, Figueiredo C, Rossau R, Jannes G, van Asbroek M, Sousa JC, Carneiro F, Quint WG: Typing of Helicobacter pylori vacA gene and detection of cagA gene by PCR and reverse hybridization. J Clin Microbiol. 1998, 36: 1271-1276.PubMedPubMed CentralGoogle Scholar
- Bayerdorffer E, Lehn N, Hatz R, Mannes GA, Oertel H, Sauerbruch T, Stolte M: Difference in expression of Helicobacter pylori gastritis in antrum and body. Gastroenterology. 1992, 102: 1575-1582.PubMedGoogle Scholar
- Vaira D, Gatta L, Ricci C, Miglioli M: Review article: diagnosis of Helicobacter pylori infection. Aliment Pharmacol Ther. 2002, 16 (Suppl 1): 16-23. 10.1046/j.1365-2036.2002.0160s1016.x.PubMedGoogle Scholar
- Saad R, Chey WD: A clinician's guide to managing Helicobacter pylori infection. Cleve Clin J Med. 2005, 72: 109-110. 10.3949/ccjm.72.2.109.PubMedGoogle Scholar
- Megraud F, Lehours P: Helicobacter pylori detection and antimicrobial susceptibility testing. Clin Microbiol Rev. 2007, 20: 280-322. 10.1128/CMR.00033-06.PubMedPubMed CentralGoogle Scholar
- Ricci C, Holton J, Vaira D: Diagnosis of Helicobacter pylori: invasive and non-invasive tests. Best Pract Res Clin Gastroenterol. 2007, 21: 299-313. 10.1016/j.bpg.2006.11.002.PubMedGoogle Scholar
- Goddard AF, Logan RP: Diagnostic methods for Helicobacter pylori detection and eradication. Br J Clin Pharmacol. 2003, 56: 273-283. 10.1046/j.1365-2125.2003.01941.x.PubMedPubMed CentralGoogle Scholar
- Correa P, Haenszel W, Cuello C, Tannenbaum S, Archer M: A model for gastric cancer epidemiology. Lancet. 1975, 2: 58-60. 10.1016/S0140-6736(75)90498-5.PubMedGoogle Scholar
- Kuipers EJ: Review article: exploring the link between Helicobacter pylori and gastric cancer. Aliment Pharmacol Ther. 1999, 13 (Suppl 1): 3-11. 10.1046/j.1365-2036.1999.00002.x.PubMedGoogle Scholar
- Molnar B, Szoke D, Ruzsovics A, Tulassay Z: Significantly elevated Helicobacter pylori density and different genotype distribution in erosions as compared with normal gastric biopsy specimen detected by quantitative real-time PCR. Eur J Gastroenterol Hepatol. 2008, 20: 305-313. 10.1097/MEG.0b013e3282f2fda4.PubMedGoogle Scholar
- Kidd M, Louw JA, Marks IN: Helicobacter pylori in Africa: observations on an 'enigma within an enigma'. J Gastroenterol Hepatol. 1999, 14: 851-858. 10.1046/j.1440-1746.1999.01975.x.PubMedGoogle Scholar
- Lunet N, Barros H: Helicobacter pylori infection and gastric cancer: facing the enigmas. Int J Cancer. 2003, 106: 953-960. 10.1002/ijc.11306.PubMedGoogle Scholar
- Peleteiro B, Bastos J, Barros H, Lunet N: Systematic review of the prevalence of gastric intestinal metaplasia and its area-level association with smoking. Gac Sanit. 2008, 22: 236-247. 10.1157/13123970. discussion 246-237PubMedGoogle Scholar
- Peleteiro B, Carrilho C, Modcoicar P, Cunha L, Ismail M, Guisseve A, Lorenzoni C, Fernandes F, Almeida R, Figueiredo C, et al: Chronic Atrophic Gastritis, Intestinal Metaplasia, Helicobacter pylori Virulence, IL1RN Polymorphisms, and Smoking in Dyspeptic Patients from Mozambique and Portugal. Helicobacter. 2009, 14: 306-308. 10.1111/j.1523-5378.2009.00687.x.Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-230X/9/98/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.