This article has Open Peer Review reports available.
Perspectives on gastroesophageal reflux disease in primary care: the REFLEX study of patient-physician agreement
© Dorval et al; licensee BioMed Central Ltd. 2011
Received: 19 May 2010
Accepted: 24 March 2011
Published: 24 March 2011
Physicians may be unaware of the severity and extent of gastroesophageal reflux disease (GERD) in their patients. The aim of this study was to evaluate patient-physician agreement concerning proton pump inhibitor (PPI) treatment.
1818 French primary-care physicians and 5174 adult patients with GERD who were taking PPIs answered questions regarding symptoms and treatment satisfaction. Patient-physician agreement was scored using the Kappa (κ) method.
There was moderate patient-physician agreement for PPI treatment satisfaction (κ = 0.60), PPI prescription adherence (κ = 0.57) and use of over-the-counter gastrointestinal medications (κ = 0.44-0.51). Patient satisfaction with PPI therapy and PPI treatment adherence rates were both ~90%. There was poor patient-physician agreement concerning PPI therapy expectations (κ = 0.22-0.33). Residual reflux symptoms occurred in 61% of patients. Physicians underestimated residual symptom severity compared with their patients (κ = 0.43-0.47), though there was good agreement regarding the presence (κ = 0.62-0.78) and frequency (κ = 0.61-0.66) of these symptoms and their effect on patients' daily life (κ = 0.64).
Patient-physician agreement regarding PPI therapy for GERD was moderate or good for the presence of residual symptoms and moderate for treatment satisfaction, but poor for treatment expectations. PPI treatment resulted in high satisfaction rates, but residual symptoms were fairly common and their severity was underestimated by physicians.
Gastroesophageal reflux disease (GERD) develops when reflux of the stomach contents into the esophagus causes troublesome symptoms such as heartburn or acid regurgitation, and/or complications such as esophagitis . The prevalence of GERD is thought to be 10-20% in Western countries based on the presence of heartburn and/or regurgitation at least once per week in general population surveys . In France, 8% of the population experience typical symptoms of GERD at least once per week .
GERD is a chronic disease and disrupts many aspects of patients' everyday lives. At least two-thirds of patients still have GERD 10 years after their initial diagnosis, and almost half of adults with GERD have had their symptoms for 10 years or more [2, 4]. Surveys of patients and the general population using validated generic health questionnaires show that GERD disrupts patients' lives in many ways, primarily by causing pain [5, 6], but also through interference with normal activities such as eating and drinking, work, sleep and enjoyment of social occasions .
Consultation rates for GERD are low: only 5-30% of individuals with gastroesophageal reflux consult a physician about their symptoms each year [8–10]. Symptom burden and anxiety about serious underlying disease are the major reasons for consultation [3, 11, 12]. Only 1.7% of primary-care consultations are the result of GERD , but consultation rates are increasing, probably because of an apparently increasing prevalence of GERD in the community [13, 14].
Physicians are not always aware of the full burden of illness of patients who consult them with reflux symptoms. A study of patient and physician ratings of reflux symptoms in clinical trials showed that physicians tend to underestimate the prevalence and severity of such symptoms . Agreement between patients and physicians was better after treatment than before, possibly because physicians' ratings were more likely to agree with those of patients when symptoms were absent . A study in primary care has shown that patients do not provide their physicians with full information on their reflux symptoms unless they have the aid of a questionnaire , and that physicians find such information very useful .
Recent surveys in primary care indicate that approximately a quarter of patients are not satisfied with their prescription treatment for GERD [18, 19]. Patients with GERD continue to experience residual reflux symptoms despite acid-suppressive therapy, and these symptoms can substantially reduce patients' quality of life . This leads to use of over-the-counter (OTC) medications such as antacids, alginates and histamine (H2)-receptor antagonists, repeated physician consultation and treatment dissatisfaction [3, 11, 20, 21]. Unsurprisingly, patients with the greatest burden of illness during treatment are least likely to be satisfied with their treatment .
The aim of the present study was to evaluate patient-physician agreement concerning treatment satisfaction in a large population of adult patients with GERD treated with proton pump inhibitors (PPIs) in primary care. This study also assessed the presence, frequency and severity of reflux symptoms, as well as their effect on patients' daily life and the need for self-medication with OTC gastrointestinal drugs.
The study was conducted from 1 September 2003 to 22 June 2004. A representative sample of 2500 primary-care physicians working in metropolitan France was selected by drawing lots from an independent database with regional stratification. The following information was collected about the physicians: sex, age, average number of adult GERD patients, number of patients seen each week and GERD treatment patterns. The patients who were asked to participate were the first adult patients coming to consult their physician (for any reason) after enrolment of the centre in the study, who had reflux symptoms, a PPI prescription covering the previous month and who had taken their PPI on at least 14 days of that month.
Patients were excluded from the study if they were under 18 years old, were unable to fill in the questionnaire, did not have reflux symptoms or esophagitis, were not treated in the last month with a PPI prescribed in a continuous way (or had not taken a PPI for at least 14 days in the preceding month), or were enrolled before the designated inclusion period.
The patient questionnaire collected information including demographic data (age, sex, weight, height, and alcohol and tobacco consumption); recent and current PPI treatment (agent, dosage, start date and mode [continuous, intermittent or on-demand]); upper gastrointestinal endoscopy in the previous 3 years (presence or absence of esophagitis); specialist consultations about GERD in the previous 3 years; concurrent treatment with non-steroidal anti-inflammatory drugs (NSAIDs), aspirin, anticoagulants and corticosteroids.
Assessment of symptoms and satisfaction
The patient and the physician each answered questions about the symptoms and treatment satisfaction. In the case of the patient questionnaire, questions were asked regarding patients' own experiences and satisfaction. With regard to the physician questionnaire, doctors were asked about their degree of satisfaction with the PPI treatment of their patient, and their own knowledge of their patients' quality of life and residual symptoms. The patient and physician questionnaires both asked for details on the following items (with the symptom descriptors used in the patient questionnaire given in brackets):
• the presence/absence (at any time), frequency (number of days per week the symptoms were experienced) and severity (mild, moderate or severe) of daytime heartburn (a burning feeling rising from the stomach or lower chest up towards the neck), night-time heartburn and acid regurgitation (a sour or bitter liquid in the mouth) during treatment with PPIs
• the presence/absence of other gastrointestinal symptoms (at any time), including dysphagia (difficulty swallowing), epigastric burning (burning in the stomach), abdominal pain (pain in the stomach), nausea (desire to vomit), vomiting, eructation (belching) and bloating (distension) during treatment with PPIs
• current expectations with respect to treatment, the current impact of reflux symptoms on everyday life and current overall patient satisfaction
• adherence to PPI therapy (yes/no)
• self-medication for reflux symptoms during treatment with PPIs (agents and frequency).
Patients received an information form concerning the study and were free to choose not to participate. Formal written informed consent was not required because the investigation did not affect normal medical care or modify the doctor-patient relationship. Each doctor saw their patients within the usual framework of consultations, which were not convened specifically for the purposes of the investigation. Each patient completed his or her questionnaire anonymously and sealed it in an envelope, which along with the physician-completed questionnaire, was sent for analysis to Axonal S.A. (Nanterre, France), a contract research organization responsible for the study logistics. Additional assessment and explorations were not required by this protocol. The investigation was the subject of a declaration specific to the Advisory Committee for Data Processing in Health Research (Comité consultatif sur le traitement de l'information en matière de recherche dans le domaine de la santé) and the approval of the National Commission of Data Processing and Freedoms (Commission Nationale de l'Informatique et des Libertés). The study also complied with epidemiological good practice according to the recommendations of the Association of French-Speaking Epidemiologists (ADELF). The study was approved by the National Physician Council in July 2003. Ethical approval was not required.
The presence, frequency and severity of reflux symptoms were defined on the basis of patient rather than physician reports . The effect of GERD on everyday life, overall satisfaction and expectations with respect to treatment (symptoms, quality of life and self-medication) was assessed through a five-grade Likert scale. Physician-patient answers were analysed by the Kappa (κ) method , using the following grades: very poor (0.00-0.20), poor (0.21-0.40), moderate (0.41-0.60), good (0.61-0.80) and excellent (0.81-1.00).
In total, 1818 primary-care physicians participated in the study, giving a response rate of 72.7%. The physicians were predominantly male (86.2%), and their mean age (± SD) was 47.0 ± 7.1 years. They saw 122.1 ± 50.4 adult patients each week, of whom 13.1 ± 9.4 were consulting them about gastroesophageal reflux symptoms. With respect to prescribing practice, most physicians prescribed PPIs often (68.5%) or always (29.1%) for adult patients with GERD, whereas H2-receptor antagonists were prescribed rarely (46.8%) or never (17.2%).
Patient sociodemographic and clinical characteristics (N = 5326).
Mean ± SD or %
Age (years, mean ± SD)
53.0 ± 14.6
Sex (% male)
Weight (kg, mean ± SD)
74.8 ± 13.6
Height (cm, mean ± SD)
169.4 ± 8.4
BMI (kg/m2, mean ± SD)
26.0 ± 4.07
Alcohol consumers* (%)
Use (often or always) of OTC medication according to the physician
Antacids and alginates (%)
H2-receptor antagonists (%)
Upper GI endoscopy in previous 3 years (%)
Esophagitis detected on endoscopy (%)
Patient expectation, adherence and satisfaction
Overall, 92.2% of patients reported taking their PPI medication as prescribed, compared with a physician rating of adherence of 89.3%. This corresponded to moderate agreement between patients and physicians (κ = 0.57). There was widespread use of OTC medications for GERD. According to the physicians, 51.4% of patients with a PPI prescription also used an OTC medication for their gastrointestinal symptoms. The physicians reported that, of the 5326 patients who were recruited, 44.6% of GERD patients with a PPI prescription used antacids/alginates (8.2% often or always), 21.0% used prokinetics (3.4% often or always) and 7.1% used H2-receptor antagonists (0.8% often or always). There was moderate agreement between patients and physicians with respect to use of antacids or local alginates (κ = 0.44) and prokinetic agents (κ = 0.51).
Residual symptoms and impact
Prevalence of gastrointestinal symptoms and patient-physician agreement.
κ (95% CI)
This study shows that there is a high level of satisfaction associated with the treatment of GERD with PPIs. Patients and their physicians rated their treatment satisfaction levels as excellent or good in more than 70% of cases, and excellent, good or moderate in over 90% of cases. This compares well with previously published treatment satisfaction rates of 70-94% for GERD medical therapies . Adherence to PPI prescription, as assessed by physicians and patients, was also very high at approximately 90%, although this was a simple response to a question that asked if the prescription had been followed.
Despite high satisfaction and compliance rates, just over half of all patients with GERD in the present study experienced at least one residual gastrointestinal symptom. This is consistent with previously published survey data in which 46% of daily PPI users experienced breakthrough symptoms, which occurred on 28% of treatment days . Although it is difficult to determine how many patients were responders to PPI therapy in the present study it is interesting to note that many patients experiencing residual heartburn and acid regurgitation had these symptoms frequently (five or more days per week in 26-35% of cases). Furthermore, residual reflux symptoms were moderate or severe in 48-63% of cases.
There is a moderate or good degree of agreement between patients and physicians for most parameters examined in the present study. Physicians had a moderate-to-good understanding of their patients' symptom burden and satisfaction with treatment. Physicians tended to underestimate the severity of reflux symptoms by underestimating the prevalence of moderate and severe symptoms and overestimating the prevalence of mild symptoms (Figure 4). There was a poor level of agreement between patients and physicians concerning expectations of PPI treatment. In general, physicians had greater expectations than patients concerning the benefits of PPI therapy.
There are several clinical implications arising from physicians' underestimation of the severity of GERD, their high expectations of PPI therapy and the level of residual gastrointestinal symptoms observed in PPI-treated patients in this study. It may be beneficial to use patient-completed questionnaires to help improve the management of newly diagnosed and existing GERD patients in primary care . Several studies show that satisfaction with treatment is associated with meeting patient expectations and good patient-physician communication [26–28]. Although satisfaction rates were high in the present study, satisfaction may be improved if physicians are able to manage patients' expectations through more effective communication. Response to therapy may be increased by educating patients about the correct way in which to take their medication to improve patient adherence . A proportion of patients with GERD receiving prescription PPIs also need to move to a more effective therapy to prevent residual gastrointestinal symptoms. Additional self-medication or switching to another PPI may be helpful . In addition, a therapeutic option other than acid-suppressive drugs (including surgical treatment) may prove useful, particularly in severe cases.
The data collected in the present study for treatment expectations, residual symptoms and satisfaction are similar to those of the US PUNS Study , the international Burning Questions survey  and two French treatment-satisfaction surveys conducted in primary and secondary care [18, 19]. However, very few other studies have paired answers from physicians and patients within the same study. McColl and colleagues showed that before treatment, physicians tended to underestimate the severity of their patients' reflux symptoms . However, after treatment, agreement between physicians and patients improved to reach a moderate to substantial level. This is similar to the findings of the present study in which all patients were receiving treatment with PPIs before the start of the study and agreement between patients and physicians was moderate. Similarly, correlations between physician and patient assessments of the severity of reflux symptoms in another study were low at baseline and stronger when assessed after treatment had begun .
Although the present study was a large survey of paired responses in primary care with an excellent response rate, it suffers from several disadvantages and limitations. While patients were neither specially selected nor specially invited for the purpose of this study, they may have been more likely to consult their physician if they had more severe reflux symptoms. Thus, this study may have a selection bias towards individuals with more severe symptoms. Furthermore, the high prevalence of esophagitis among patients who underwent an endoscopy in the current study could reflect a referral bias. Adherence was self-reported and so may be underestimated (particularly unintentional non-adherence such as taking the medication at the wrong time of day). Information on treatment expectations was based on patients recalling how they felt before the start of treatment. No information was available concerning patients' pre-treatment symptom burden or consultations, which may have influenced patients' satisfaction. A further possible limitation is that the study did not use a validated questionnaire, since the most appropriate tools (e.g. the Treatment Satisfaction Questionnaire [TSQ] in GERD) were not available in French. The Overall Treatment Effect (OTE) questionnaire (adapted from the Global Ratings of Change Questionnaire [GRCQ]) is validated in French, but does not capture the dimension of satisfaction [32, 33].
In conclusion, this study shows there were very high rates of patient-physician satisfaction and treatment adherence among GERD patients taking a PPI (all ~90%), and that there was moderate patient-physician agreement for these parameters. However, agreement levels were lower concerning therapeutic expectations, with physicians having greater expectations of PPI therapy than their patients. Residual gastrointestinal symptoms were fairly common and physicians underestimated residual symptom severity compared with their patients, though there was good agreement regarding the presence and frequency of these symptoms and their effect on patients' daily life. These results suggest the potential for better communication between physicians and their patients with GERD, possibly through structured symptom questionnaires, and a need for physicians to be open to the option of switching PPI, or choosing an alternative or additional GERD medication in selected patients with difficult-to-treat GERD.
Initial data analyses were undertaken by Corinne Artigues-Saugnac who is an employee of Axonal and received funding from AstraZeneca. Writing support was provided by Dr Richard Clark and Dr Christopher Winchester of Oxford PharmaGenesis Ltd and funded by AstraZeneca R&D Mölndal. This study was funded by AstraZeneca France, study code: REFLEX 3.
Data presented previously at Digestive Disease Week 2006 and published as abstract: Dorval E, Rey J, Barthelemy P, Caekaert A, Soufflet C. Assessment of satisfaction following the treatment of gastro-oesophageal reflux by proton pump inhibitors in primary care: the « REFLEX » physician-patient agreement study [abstract]. Gastroenterology 2006; 130 (4 Suppl 2): T1019.
- Vakil N, Veldhuyzen van Zanten S, Kahrilas P, Dent J, Jones R: The Montreal definition and classification of gastro-esophageal reflux disease (GERD) - a global evidence-based consensus. Am J Gastroenterol. 2006, 101: 1900-1920. 10.1111/j.1572-0241.2006.00630.x.View ArticlePubMedGoogle Scholar
- Dent J, El-Serag HB, Wallander MA, Johansson S: Epidemiology of gastro-oesophageal reflux disease: a systematic review. Gut. 2005, 54: 710-717. 10.1136/gut.2004.051821.View ArticlePubMedPubMed CentralGoogle Scholar
- Bretagne JF, Honnorat C, Richard-Molard B, Caekaert A, Barthelemy P: Comparative study of characteristics and disease management between subjects with frequent and occasional gastro-oesophageal reflux symptoms. Aliment Pharmacol Ther. 2006, 23: 607-616. 10.1111/j.1365-2036.2006.02811.x.View ArticlePubMedGoogle Scholar
- Armstrong D: Systematic review: persistence and severity in gastroesophageal reflux disease. Aliment Pharmacol Ther. 2008, 28: 841-853.PubMedGoogle Scholar
- Wiklund I, Talley NJ: Update on health-related quality of life in patients with gastroesophageal reflux disease. Expert Rev Pharmacoeconomics Outcomes Res. 2003, 3: 341-350. 10.1586/1473722.214.171.1241.View ArticleGoogle Scholar
- Wiklund I: Review of the quality of life and burden of illness in gastroesophageal reflux disease. Dig Dis. 2004, 22: 108-114. 10.1159/000080308.View ArticlePubMedGoogle Scholar
- Liker H, Hungin AP, Wiklund I: Management of reflux disease in primary care: the patient perspective. J Am Board Fam Pract. 2005, 18: 393-400. 10.3122/jabfm.18.5.393.View ArticlePubMedGoogle Scholar
- Kennedy T, Jones R: The prevalence of gastro-oesophageal reflux symptoms in a UK population and the consultation behaviour of patients with these symptoms. Aliment Pharmacol Ther. 2000, 14: 1589-1594. 10.1046/j.1365-2036.2000.00884.x.View ArticlePubMedGoogle Scholar
- Locke GR, Talley NJ, Fett SL, Zinsmeister AR, Melton LJ: Prevalence and clinical spectrum of gastroesophageal reflux: a population-based study in Olmsted County, Minnesota. Gastroenterology. 1997, 112: 1448-1456. 10.1016/S0016-5085(97)70025-8.View ArticlePubMedGoogle Scholar
- Haque M, Wyeth JW, Stace NH, Talley NJ, Green R: Prevalence, severity and associated features of gastro-oesophageal reflux and dyspepsia: a population-based study. N Z Med J. 2000, 113: 178-181.PubMedGoogle Scholar
- Jones R, Armstrong D, Malfertheiner P, Ducrotté P: Does the treatment of gastroesophageal reflux disease (GERD) meet patients' needs? A survey-based study. Curr Med Res Opin. 2006, 22: 657-662. 10.1185/030079906X100032.View ArticlePubMedGoogle Scholar
- Crawley JA, Hamelin B, Gallagher E: Does it matter why heartburn sufferers seek health care?. Gastroenterology. 2000, 118: A209-10.1016/S0016-5085(00)82911-X.View ArticleGoogle Scholar
- Morbidity statistics from general practice: fourth national study 1991-1992. [http://www.statistics.gov.uk/downloads/theme_health/MB5No3.pdf]
- El-Serag HB: Time trends of gastroesophageal reflux disease: a systematic review. Clin Gastroenterol Hepatol. 2007, 5: 17-26. 10.1016/j.cgh.2006.09.016.View ArticlePubMedGoogle Scholar
- McColl E, Junghard O, Wiklund I, Revicki D: Assessing symptoms in gastroesophageal reflux disease: how well do clinicians' assessments agree with those of their patients?. Am J Gastroenterol. 2005, 100: 11-18. 10.1111/j.1572-0241.2005.40945.x.View ArticlePubMedGoogle Scholar
- Jones RH, Hungin AP, Phillips J, Mills JG: Gastro-oesophageal reflux disease in primary care in Europe: clinical presentation and endoscopic findings. Eur J Gen Pract. 1995, 1: 149-154. 10.3109/13814789509161628.View ArticleGoogle Scholar
- Jones R, Coyne K, Wiklund I: The gastro-oesophageal reflux disease impact scale: a patient management tool for primary care. Aliment Pharmacol Ther. 2007, 25: 1451-1459. 10.1111/j.1365-2036.2007.03343.x.View ArticlePubMedGoogle Scholar
- Bommelaer G, Caekaert A, Barthelemy P: Management of gastro-oesophageal reflux in primary care at the time of diagnosis [abstract]. Gastroenterology. 2005, 128 (4 Suppl 2): A382-Google Scholar
- Bretagne JF, Rey JF, Caekaert A, Barthelemy P: Routine management of gastro-oesophageal reflux disease by gastroenterologists in France: a prospective observational study. Dig Liver Dis. 2005, 37: 566-570. 10.1016/j.dld.2005.02.008.View ArticlePubMedGoogle Scholar
- Crawley JA, Schmitt CM: How satisfied are chronic heartburn sufferers with their prescription medications? Results of the patient unmet needs survey. J Clin Outcomes Manage. 2000, 7: 29-34.Google Scholar
- Robinson M, Shaw K: Proton pump inhibitor attitudes and usage: a patient survey. Pharm Ther J. 2002, 27: 202-206.Google Scholar
- Degl' Innocenti A, Guyatt GH, Wiklund I, Heels-Ansdell D, Armstrong D, Fallone CA, Tanser L, van Zanten SV, El-Dika S, Chiba N, Barkun AN, Austin P, Schunemann HJ: The influence of demographic factors and health-related quality of life on treatment satisfaction in patients with gastroesophageal reflux disease treated with esomeprazole. Health Qual Life Outcomes. 2005, 3: 4-View ArticlePubMedPubMed CentralGoogle Scholar
- Dent J, Armstrong D, Delaney B, Moayyedi P, Talley NJ, Vakil N: Symptom evaluation in reflux disease: workshop background, processes, terminology, recommendations, and discussion outputs. Gut. 2004, 53 (Suppl 4): iv1-24. 10.1136/gut.2003.034272.PubMedPubMed CentralGoogle Scholar
- Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174. 10.2307/2529310.View ArticlePubMedGoogle Scholar
- Revicki DA: Patient assessment of treatment satisfaction: methods and practical issues. Gut. 2004, 53 (Suppl 4): iv40-44. 10.1136/gut.2003.034322.PubMedPubMed CentralGoogle Scholar
- Jackson JL, Chamberlin J, Kroenke K: Predictors of patient satisfaction. Soc Sci Med. 2001, 52: 609-620. 10.1016/S0277-9536(00)00164-7.View ArticlePubMedGoogle Scholar
- Lewin SA, Skea ZC, Entwistle V, Zwarenstein M, Dick J: Interventions for providers to promote a patient-centred approach in clinical consultations. Cochrane Database Syst Rev. 2001, CD003267-Google Scholar
- Jackson JL, Kroenke K: The effect of unmet expectations among adults presenting with physical symptoms. Ann Intern Med. 2001, 134: 889-897.View ArticlePubMedGoogle Scholar
- Britten N, Stevenson FA, Barry CA, Barber N, Bradley CP: Misunderstandings in prescribing decisions in general practice: qualitative study. BMJ. 2000, 320: 484-488. 10.1136/bmj.320.7233.484.View ArticlePubMedPubMed CentralGoogle Scholar
- Fass R, Sontag SJ, Traxler B, Sostek M: Treatment of patients with persistent heartburn symptoms: a double-blind, randomized trial. Clin Gastroenterol Hepatol. 2006, 4: 50-56. 10.1016/S1542-3565(05)00860-8.View ArticlePubMedGoogle Scholar
- Fallone CA, Guyatt GH, Armstrong D, Wiklund I, Degl'Innocenti A, Heels-Ansdell D, Barkun AN, Chiba N, Zanten SJ, El-Dika S, Austin P, Tanser L, Schunemann HJ: Do physicians correctly assess patient symptom severity in gastro-oesophageal reflux disease?. Aliment Pharmacol Ther. 2004, 20: 1161-1169. 10.1111/j.1365-2036.2004.02257.x.View ArticlePubMedGoogle Scholar
- Jaeschke R, Singer J, Guyatt GH: Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989, 10: 407-415. 10.1016/0197-2456(89)90005-6.View ArticlePubMedGoogle Scholar
- Juniper EF, Guyatt GH, Willan A, Griffith LE: Determining a minimal important change in a disease-specific quality of life questionnaire. J Clin Epidemiol. 1994, 47: 81-87. 10.1016/0895-4356(94)90036-1.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-230X/11/25/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.