Instrument to evaluate the knowledge of patient with cirrhosis on his disease: construction and validity
BMC Gastroenterology volume 21, Article number: 83 (2021)
The application of measurement instruments is a strategy to evaluate the patient's knowledge about the disease. This study aims to build an instrument that evaluates the patient's knowledge about liver cirrhosis.
This study includes three phases. The first one was the construction of the instrument based on a literature review. In the second phase, five experts were participated in the evaluation of the instrument, to check the validity of the content. Quantitative and qualitative analyzes were made. The tool used was the CVI (Content Validity Index) and it was used the semantic study of the questions. The third phase was the process of the restructuring the instrument.
The final version of the instrument consisted of 36 questions. The instrument was evaluated in 91.7 by the average CVI and 94.4% by the universal CVI.
The questions are properly structured and clear, therefore, understandable. Thus, the final instrument presented satisfactory content validity, so that, it reached the aim of this study.
Liver cirrhosis (LC) is a pathological condition that leads to the formation of permanent scars in the liver and impairment of blood flow, which causes loss of its normal function. The main etiological factors are alcohol abuse, viral hepatitis, and autoimmune diseases. In the last decade there was a higher incidence of causes related to cryptogenic disease and non-alcoholic fatty liver disease [1, 2].
In 2018, LC was the 11th main cause of mortality worldwide, responsible for approximately 2 million deaths . Since it is a chronic disease and often related to lifestyle, it requires the patient to change habits, from dietary changes to commitment to medication use and need for continuous care, which can affect quality of life. If we include transplantation to this chronic disease, the only curative therapy, it becomes an even more complex process .
It is imperative that health professionals understand the importance of information transmission as a relevant point of their work and develop ways to make information accessible to the population. Low levels of formal education may interfere negatively in the disease prognosis, since the understanding of the medical guidelines is hindered, which may increase mortality rates, visits to emergency centers and hospital readmission . In the case of LC, since it is a chronic disease, it requires changes in lifestyle from the patient, from changes in the diet to commitment to the use of medications and the need for continuous care. That is, there are countless variables that need to be understood by the patient so that the disease does not affect their quality of life. This shows the complexity of factors influencing patients´ adherence. Within this multifaceted context, it becomes fundamental for health professionals to develop strategies to better identify the needs of each patient. From this knowledge, it is possible to perform more tailored interventions, making the patient a protagonist of his own care, and thus, more active in his treatment. Patients who have more information communicate with the healthcare team more effectively and produce better results in their treatments .
The patient's understanding of his own physical condition will lead to the development of self-care. In the case of patients who need a solid organ transplant, the need for an educational process becomes even more relevant. In this case, the patient needs to learn how to administer the new medication that he will use for the rest of his life, in addition to adhering to changes in lifestyle, such as hygiene practices, monitoring the new organ, preventing infection and social and professional readjustment. .
The use of standard scales as evaluation instruments may contribute to the work of health professionals, helping to track patients in need of follow-up or further intervention. Measurement scales are standardized tools that require a strict process of construction and validation to reach their objectives. When fully validated, they are useful in clinical practice .
Some studies have developed and validated instruments to understand the patient's knowledge of his disease. Specifically, in Brazil, we can mention Padilha et. al  who investigated beliefs and attitudes in patients with heart valve disease. The work of Bonin et. al , built a questionnaire to measure the knowledge of patients with heart failure. The research by Zulianello et. al , built and validated a psychometric scale to assess the knowledge of hypertensive patients. In addition, referring to international studies, there is Benhamou et. al  who developed and validated an instrument to assess fears and beliefs in patients with knee osteoarthritis; Siklosi et. al  who developed and validated a questionnaire to assess patients' knowledge about Cystic Fibrosis; Bardazzi et. al  who validated a questionnaire on the awareness of patients with psoriasis; Zschocke et. al  who developed a questionnaire to assess adherence factors in patients receiving topical therapy; and Webb et. al , who developed an instrument to assess quality of life in patients with primary hyperparathyroidism.
In Brazil, there are some instruments developed for the population of patients with LC. For example, the Chronic Liver Disease Questionnaire (CLDQ) instrument that went through the process of translation and cross-cultural adaptation by Mucci et. al , and the Liver Disease Quality of Life (LDQOL), which was also translated and adapted into Portuguese by Teixeira et. al . However, it is worth noting that both instruments aim to assess patients' quality of life.
This study aims to construct and perform the criterion validity of an instrument that is able to assess the knowledge that a patient with cirrhosis has about his disease and its treatment and help health professional have a better understanding of their patients need.
The construction of the instrument´s items (second stage), was carried out based on an extensive literature review, with relevant publications on the theme from the previous five years . The instrument was divided into topics according to a psychometrician for optimization and according to the educational needs of the target population: signs and symptoms, diagnosis, treatment, and medication.
The Likert scale is the most widely used to survey opinions, attitudes, and assessments. In this type of scale, the individual is asked to evaluate the phenomenon (the question) on a 5-point scale. [19, 20] The idea is that the evaluated person chooses for each question presented one of the answer categories Still, according to Gorenstein , a Likert scale must strictly have 5 points; any other type of punctuation and / or category must be referred to as “Likert type”. The selection and organization of the items (third stage) should be formulated to present questions according to the principles of clarity, coherence, and neutrality. The drafting of the questions should be done logically to minimize the mental effort of the respondent, using clear and easy to understand language, avoiding complex, ambiguous and excessively long questions .
Criterion validity (fourth stage) evaluates the extent to which the test (instrument) measures what it was designed to. In other words, the validity refers to a concordance index between what it measures and what it was designed to measure .
Construct and criterion validity was carried out by a committee of specialists. A group of professionals from the Liver Transplant Service Outpatient Clinic of the University of Sao Paulo School of Medicine, was invited to participate in the evaluation, with the requirement that they had at least one year’s experience working with patients with LC. Seven healthcare professionals from different areas were recruited (one in nursing, three doctors, two psychologists, one of them a psychometrician, and a social worker). The literature points to controversies about the number of specialists that should participate in content validation, therefore the minimum parameter cited of five specialists was used, where there was a maximum of ten specialists in the literature [18, 19, 23].
Data analysis and processing
According to the literature , there is no specific statistical test to confirm content validity. Quantitative and qualitative analyses were carried out.
Quantitative analysis: the tool used was the Content Validity Index (CVI). The CVI of the question, the mean CVI and the universal CVI were calculated for each topic and for the instrument. The criteria for inclusion of a question were the same proposed by the literature, of 80% agreement between the specialists [18, 19]. The parameters used to calculate each type of CVI are exposed on Table 1.
Qualitative Analysis: At the end of each question, the specialists could make suggestions about the content, clarity, and semantics. Additionally, space was left to make general comments and suggest issues or pertinent content that had not been approached.
After the first content validation was performed, a second version of the instrument was made following the specialist’s advices and item’s exclusion criteria. This second version was submitted to a new content validity (stage 5), similar to the first one except for a change in qualitative analysis. Then the instrument would be evaluated based on the CVI of each item in the following criteria: clarity, objectivity, appropriateness for the medical context, for Brazilian culture and for the population evaluated.
Of the 60 questions that composed the initial version of the instrument, 31 were excluded, of which seven were from the topic “signs and symptoms”, seven from the topic “diagnosis”, eight from the topic “treatment” and nine from the topic “medication”. The exclusion criteria were the question not reaching the CVI of 80% or having repetitive contents in some topics according to the specialists.
Questions were added based on the suggestions of the specialists. The final version of the instrument (in annex 01) had a total of 36 distributed between the topics signs and symptoms, diagnosis, treatment and the subcategory medication. Each topic with 10 and the subcategory with 6 questions.
Questions were formulated based on important information that can help patient make better decision and be aware of the disease, such potential short-term and long-term complications. Common misconceptions, such as that having no symptoms indicates that liver cirrhosis is at the beginning of the disease were also address. Cirrhosis treatment and patient prognosis were also evaluated. The complete questionnaire can be found in supplement material.
Quantitative analysis—mean and universal CVIs.
Qualitative analysis was based on the mean and universal CVI of each topic and the whole instrument. Information is summarized on Table 2.
Qualitative analysis—instrument semantics
Semantic analysis presented the unanimous suggestion to change the words “hepatic cirrhosis” for “cirrhosis of the liver”. In general, the specialists suggested using simpler and more accessible language for the target population (patients with cirrhosis). The points were pertinent and were followed in the restructuring of the instrument for the second version.
During this analysis, only one question on the “signs and symptoms” topic and on the “medication” topic did not obtain 80% acceptance as seen on Table 3. The remaining questions were included with a CVI of more than 80%, showing that they were assessed as adequate. The specialists did not make new semantic suggestions. There were no changes in the instrument following this analysis, therefore the second and final versions were equal.
Liver cirrhosis is a prevalent condition worldwide, with potentially fatal complications and requiring a complex care support network. As a carrier of a severe chronic disease, it is necessary for a patient to undergo various changes in lifestyle, from dietary changes to the use of immunosuppressors. Among the many factors that help patients to adhere to these changes, knowledge about the disease is a variable that can be relayed by the healthcare team.
According to Saberifiroozi , the rate of adherence to treatment can greatly increase if the health team is closer, adapting the information to the needs of the patient. Educational groups, continuous contact, and increased capacity for self-management of the patient are examples of actions that the team can perform reflecting on a better prognosis and quality of care.
The creation of an instrument aims to assess the extent of the knowledge that the patient has about the disease. This information can help better understand the patients’ needs aiming to improve adherence and prognosis. The development of the instrument followed the stages described in the literature [18, 19, 24]. Two evaluation approaches were used: one qualitative and one quantitative. The instrument was assessed quantitatively using the CVI. The mean and universal CVIs of each topic were analyzed (signs and symptoms, diagnosis, treatment, and medication). At the end of the evaluation of the first version of the instrument, an average of seven questions were eliminated from each topic for not obtaining the score required for inclusion. This raised the result of the CVI of each topic in the second version of the instrument to a mean higher than 90%, showing an expressive improvement in the results.
The result of the CVI shows that the instrument reached values of concordance among the specialists proposed in the literature. The mean CVI increased from 70.3% to 91.7%. The universal CVI went from 56.7% to 94.4%. Both CVIs obtained over 80% concordance, reaching 90% in the final version, considered ideal by the literature [24, 25]. These results reveal that, with the considerations of the specialists, the instrument was reformulated and significantly improved. Therefore, it achieved satisfactory results for the study proposal.
Similar results were found in other studies using CVI for the evaluation of measuring instruments. An instrument developed about blood transfusion by Bezerra et al.  showed a universal CVI of 87%. A study carried out by Fan et al.  also aiming to construct instruments for tuberculosis presented a CVI of 96%. Another study focusing on osteoporosis by Rodrigues et al.  had a CVI of 87%. These studies showed a convergence in the literature (of around 80%) and presented similar results to the instrument developed in this study, which was 94%.
The qualitative approach was carried out via semantic analysis. The spaces for comments allowed the specialists to express their opinions on each question in a directive and timely manner. This evaluation approach was extremely effective, allowing the transformation of some content into questions from their comments. It is possible to infer that the inclusion of the qualitative analysis allowed the specialists more space and autonomy to express their opinions, making the instrument more appropriate for its purpose. Mostly, the lower rated questions did not use clear language, relying on words that patients would find difficult to understand. Care with the use of clear language is a fundamental requirement for the construction of instruments, as suggested by the literature .
The development of an instrument is a complex process that requires the following of several steps to become reliable. One of the largest challenges is that there is no single standard model. We did not find agreement in the literature regarding various issues: the number of questions necessary to make the instrument reliable, the number of specialists to consult, or even a single model to follow for the steps in construction of the model. Each instrument studied had a particular process of development, which required rigorous investigation before choosing the construction stages of this instrument.
We found similar complications regarding the construction of instruments as other authors have previously. For example, the literature highlights the importance of instruments, and warns of the need for a critical evaluation of their psychometric properties. The quality of results that an instrument should aim to reach is only possible with precise and appropriate parameters .
In this sense, the need to continue the assessment and psychometric processes of the instrument is reiterated. An important stage was completed: the construction of the instrument. With the completion of the construction of the questions of the instrument, a proposal to be added in later stages is related to the elaboration of the pilot instrument to be applied to the target population [21, 30]. The advantages of including this step are to obtain a more accurate analysis of the understanding of the questions and to evaluate the average time to complete the instrument in the population of patients with cirrhosis. Based on these data acquired with the application, it is possible to make changes to the instrument's structure, regarding the number or semantics of the questions, for example, before developing the final version. Having information about the instrument's performance when using it with the population can make it even more useful and adapted.
The instrument we developed is a valid tool for assisting healthcare professionals and is an important step towards developing strategies that can contribute to professional healthcare. The greater the knowledge and comprehension of the patient, the more effective the treatment will be, which leads to lower costs for healthcare and better results.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Brazilian Association of Organ Transplantation
Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo
Content Validity Index
Saberifiroozi M. Improving Quality of Care in Patients with Liver Cirrhosis. Middle East J Dig Dis. 2017;9(4):189–200. https://doi.org/10.15171/mejdd.2017.73.
Muir AJ. Understanding the Complexities of cirrhosis. Clin Ther. 2015;37(8):1822–36.
Bittencourt PL, Farias AQ, Couto CA. Liver Transplantation in Brazil. Liver Transplant. 2016;22(9):1254–8.
Wright JP, Edwards GC, Goggins K, Tiwari V, Maiga A, Moses K, et al. Association of Health Literacy With Postoperative Outcomes in Patients Undergoing Major Abdominal Surgery. JAMA Surg. 2017;2391(2):137–42.
da Taddeo P, S, Gomes, K.W.L., Caprara, A., Gomes, A.M. de A., Oliveira, G.C. de, & Moreira, T.M.M. . Access, educational practice and empowerment of patients with chronic diseases. Cien Saude Colet. 2012;17(11):2923–30. https://doi.org/10.1590/S1413-81232012001100009.
Mendes KD, Sasso SJ, de Castro O, e, Ziviani, Luciana da Costa, Rossin, Fabiana Murad, Zago, Márcia Maria Fontão, & Galvão, Cristina Maria. . Intervenção educativa para candidatos ao transplante de fígado. Revista Latino-Americana de Enfermagem. 2013;21(1):419–25. https://doi.org/10.1590/S0104-11692013000100018.
Gorenstein C, Wang Y. Fundamentos de mensuração em saúde mental. Instrumentos avaliação em saúde Ment. 2016. https://doi.org/10.1590/0047-2085000000136.
Padilha KM, Gallani MCBJ, Colombo RCR. Development of an instrument to measure beliefs and attitudes from heart valve disease patients. Revista Latino-Americana De Enfermagem. 2004;12(3):453–9. https://doi.org/10.1590/S0104-11692004000300002.
Bonin, Christiani Decker Batista, Santos, Rafaella Zulianello dos, Ghisi, Gabriela Lima de Melo, Vieira, Ariany Marques, Amboni, Ricardo, & Benetti, Magnus. (2014). Construction and Validation of a Questionnaire about Heart Failure Patients' Knowledge of Their Disease. Arquivos Brasileiros de Cardiologia, 102(4), 364–373. Epub February 17, 2014. https://doi.org/10.5935/abc.20140032
Zulianello R, Decker C, Bonin B, Ten E, Martins C, Junior MP, et al. (2018) Construção e Validação Psicométrica do HIPER-Q para Avaliar o Conhecimento de Pacientes Hipertensos em Reabilitação Cardíaca. Arq Bras Cardiol.
Benhamou M, Baron G, Dalichampt M, Boutron I, Alami S, Rannou F, et al. (2013) Development and Validation of a Questionnaire Assessing Fears and Beliefs of Patients with Knee Osteoarthritis: The Knee Osteoarthritis Fears and Beliefs Questionnaire (KOFBeQ). Milanese S, editor. PLoS One. Jan 21;8(1):e53886.
Siklosi KR, Gallagher CG, Mckone EF. Development, validation, and implementation of a questionnaire assessing disease knowledge and understanding in adult cystic fibrosis patients. J Cyst Fibrose. 2010;9:400–5.
Bardazzi F, Amerio P, Amoruso G, Campanati A, Conti A, Simone CDE, et al (2014). Investigating psoriasis awareness among patients in Italy: validation of a questionnaire. Eur Rev Med Pharmacol Sci ;3435–52.
Zschocke I, Mrowietz U, Lotzin A. Assessing adherence factors in patients under topical treatment: development of the Topical Therapy Adherence Questionnaire (TTAQ). Arch Dermatol Res. 2014;306:287–97.
Webb SM, Puig-domingo M, Villabona C, Muñoz-torres M, Farrerons J, Badia X. Development of a new tool for assessing Health-Related Quality of Life in patients with primary hyperparathyroidism. Heal Qual live outcomes. 2013;11(97):1–8.
Muci, S. (2010). Adaptação cultural do Chronic Liver Disease Questionnaire (CLDQ) para população Brasileira. Caderno de Saúde Pública. Rio de Janeiro, 26(1):199–205. https://doi.org/10.1590/S0102-311X2010000100021.
Teixeira MCD, Ribeiro MFS, Strauss E. A new insight into difference among non-cirrhotic patients using the liver Disease Quality of Life Instrument (LDQOL). Ann Hepatol. 2005;4:264–71.
Alexandre NMC, Coluci MZO. Content validity in the development and adaptation processes of measurement instruments. Cien Saude Colet. 2011;16(7):3061–8. https://doi.org/10.1590/S1413-81232011000800006.
Coluci MZO, Alexandre NMC, Milani D. Construction of measurement instruments in the area of health. Cien Saude Colet. 2015;20(3):925–36. https://doi.org/10.1590/1413-81232015203.04332013.
Gunther, H. (2003). Como Elaborar um Questionário. In Gunther, H. (1ed). Planejamento de Pesquisa nas Ciências Sociais. (pp. 37–54). Brasília, DF: Laboratório de Psicologia Ambiental.
Gorenstein C, Wang Y (2016). Fundamentos de mensuração em saúde mental. Instrumentos avaliação em saúde Ment;1–52.
Rodrigues IB, Adachi JD, Beattie KA, MacDermid JC. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC Musculoskelet Disord. 2017;18(1):540. https://doi.org/10.1186/s12891-017-1914-5.
Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar A-R. Design and Implementation Content Validity Study: Development of an instrument for measuring Patient-Centered Communication. J Caring Sci. 2015;4(2):165–78.
de Souza AC, Alexandre NMC, de Guirardello E, B., Souza, A.C. de, Alexandre, N.M.C., & Guirardello, E. de B. . Psychometric properties in instruments evaluation of reliability and validity. Epidemiol e Serviços Saúde. 2017;26(3):649–59. https://doi.org/10.5123/S1679-49742017000300022.
Almanasreh E, Moles R, Chen TF. Evaluation of methods used for estimating content validity. Res Soc Adm Pharm. 2018. https://doi.org/10.1016/j.sapharm.2018.03.066.
Bezerra CM, Cardoso MVLML, Silva GRFD, Rodrigues EDC. Creation and validation of a checklist for blood transfusion in children. Rev Bras Enferm. 2018;71(6):3020–6. https://doi.org/10.1590/0034-7167-2018-0098.
Fan Y, Zhang S, Li Y, Li Y, Zhang T, Liu W, Jiang H. Development and psychometric testing of the Knowledge, Attitudes and Practices (KAP) questionnaire among student Tuberculosis (TB) Patients (STBP-KAPQ) in China. BMC Infect Dis. 2018;18(1):213. https://doi.org/10.1186/s12879-018-3122-9.
Rodrigues RSR, da Mendes CR, S., Alves F.M., Sandro, A., & Grangeiro, D.M. . Head and neck cancer: validation of a data collection instrument. Rev Bras Enferm. 2018;71(4):2010–8. https://doi.org/10.1590/0034-7167-2017-0227.
Keszei AP, Novak M, Streiner DL. Introduction to health measurement scales. J Psychosom Res. 2010;68(4):319–23. https://doi.org/10.1016/j.jpsychores.2010.01.006.
Pasquali L. Princípios de elaboração de escalas psicológicas (1998). Rev Psiquiatria Clinica; 25(5):206–13.
Processo no. 2018/11686-4, Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP). The authors are thankful to João Italo Dias França for his expert statistics advice and Justin Axel-Berg for English corrections.
This research did not receive any specific grant from any funding agency in the public, commercial, or not-for-profit sector.
Ethics approval and consent to participate
All procedures performed in this study were approved by the Research Ethics Committee Hospital das Clinicas, Faculty of Medicine, University of São Paulo (HCFMUSP) (opinion No. 1.790.107) and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Written Informed consent was obtained from all individual adult participants included in this study.
Consent for publish
All participants gave written consent for their personal or clinical details along with any identifying images. All named authors have agreed to the paper publication.
The authors have no conflicts of interest to declare.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Stelmach, M., de Almeida Medeiros, K.A., Carvalho, B.J. et al. Instrument to evaluate the knowledge of patient with cirrhosis on his disease: construction and validity. BMC Gastroenterol 21, 83 (2021). https://doi.org/10.1186/s12876-021-01665-0