Article Type: Review Article

 

Article Citation: Professor Ishtiaq Ahmed. (2021). HOW TO CRITICALLY APPRAISE A RESEARCH PAPER?. International Journal of Research -GRANTHAALAYAH, 9(3), 229-240. https://doi.org/10.29121/granthaalayah.v9.i3.2021.3779

 

Received Date: 08 March 2021

 

Accepted Date: 31 March 2021

 

Keywords:

Critical Appraisal

Research Paper

Evidence Based Practice

Research Methodology

Study Validity

Study Reliability
ABSTRACT

Background: Critical appraisal of research paper is a fundamental skill in modern medical practice, which is skills-set and developed throughout the professional career. The professional experience facilitates this and through integration with clinical experience and patient preference, permits high quality evidence-based medicine practice in patient care. These skills to be mastered not only by academic medical professionals but also by the clinicians involved in clinical practice.

Objective: To provide a simple and robust method for assessing the trustworthiness of a research paper and its value in clinical practice.

Methodology: Through detailed literature search, All essential sections and subsection mandatory for a research paper were identified followed by the necessary steps or information required in each section or questions which may arise or needs to addressed were identified. The important questions or steps which are integral in assessing the reliability and validity of a research are gathered during critical review of a research paper.  

Results: Out of 128 full text articles, 49 full-text articles containing robust and pertinent information as per objective were short listed for review.

Conclusion: Critical appraisal of a research paper or project is a fundamental skill in modern medical practice for assessing the worth of clinical research and in providing a guideline of its relevance to the profession.

1.     INTRODUCTION

 

1.1. SEARCH METHODOLOGY

 

The literature for this review is searched from MEDLINE (Ovid), Medscape, Pub Med, EMBASE, Scopus,  HMIC, CINAHL+ and the Journal of Research Evaluation along with other sources (i.e. Google Scholar, Research Gate and Expert communication) by using Critical appraisal, Research paper, Evidence based practice, Research methodology and Study validity as key words. All these databases contain archives of majority of biomedical journals from all over the world. Among a total of 128 full text articles retrieved, through meticulous analysis, 49 full-text articles containing robust and pertinent information mentioned above were short listed for review. All relevant scientific papers, written in English were included and non-scientific articles, non-scientific commentary and reports were excluded from the review.

Through detailed literature search, for writing this review, all important sections necessary for manuscript writing like title page, abstract, introduction, methodology, results, data analysis, discussion etc along with subsections were identified. All necessary steps or information required for review in each section or questions which may arise or needs to addressed during a review were also identified. Among all shortlisted papers the pertinent information’s or data retrieved, analyzed and reproduced in each section or sub-section in own description. Through detailed literature search, all essential sections and subsection mandatory for a research paper were identified

 

2.     BACKGROUND

The objective of publishing a research paper is to provide an accurate, impartial medical information, authenticated by robust scientific evidences which are helpful in enhancing the patient care [1]. The Critical appraisal of a research is “The process of carefully and systematically examining the research to judge its trustworthiness, value and relevance in a particular context” [2] . With an ever-increasing volume of scientific literature (i.e. only in MEDLINE database around more than 12,000 new papers were added weekly[3]), it  can be arduous for a clinician to keep themselves abreast with this massive load of information’s from current literature while working with their patients in hospital [4], [5]. One of the keystones of high-quality health care is Evidence-based medicine and by its judicious, explicit and conscientious use, we can provide an effective and efficient patient care and make right decisions in best interest of our patients [6], [7]. Therefore, the health professionals require a good quality, comprehensive, well structured, easily accessible while interacting with patients [7], [8]. To achieve this, the critical appraisal of a research paper is essential to judge its reliability, relevance and value of clinical research regarding future implications and also as a part of Continuing Professional Development (CPD) [9].

A good research work always builds the basis for future research and also determines its values and further course. Whereas, an inaccurate data or results leads to inaccurate outcomes, which end up in further misconceptions in scientific knowledge. To set a concrete basis for research in medical sciences, it is imperative to prevent the fabrication or dishonestly multiplication of information by critically appraising the scientific evidence presented and to decide that which information can be considered as plausible [1], [10].

It is also very helpful in identifying the flaws in study and to guide the researcher, how to overcome these observations because flawed published studies most likely leads to faulty clinical decisions which may end up with worst outcome or no benefit in patient care [11], [12]. Moreover, by knowing how to critically analyze a research paper, readers can make their informed judgement about the reliability and validity of research produced. In addition, the critical appraisal of research projects or paper also plays a key role in institutional research policies like promotions, hiring, firing, and funding schemes [6], [9].

The importance of the quality of research even emphasized by Cicero [10] in the beginning of last century, by emphasizing that “number does not matter but the quality does”. But so, for, there is no or little consensus on accepted standards on how to measure research quality objectively? [5], [8] Therefore, the individual research evaluation remains a notoriously difficult practice with no standard solution [13], [14]. Different guidelines were followed by different medical journals or institutions for critical appraisal of research papers. For example, the guideline for reporting and evaluating randomized controlled trials is CONSORT (Consolidated Standard for Reporting Trial) statement which is mandatory for publication of manuscript in majority of international medical journals [15]. Similarly, some other scientific societies like Scottish Intercollegiate Guidelines Network, has also devised checklists which are helpful in the critical appraisal of manuscripts depending upon the type of study methodology [16], [17].

Another major problem in evaluating or considering a research hypothesis or an idea is that we are bound to rely on findings on that topic given in the literature in past [18]. So, what is the criteria to conclude whether the results and outcome of a research can be trusted or not? Moreover, as research evaluator one should kept in mind that any type of errors in research may emerge from any stage of the research process. Therefore, the appraisal is all to boils down to how the research has been conducted, results interpreted, what is outcome, should be considered meticulously when reading as articles. This review is written with an aim to give an outline of how to conduct a critical appraisal, especially in the main domains of a research paper.

3.     ASSESSING RESEARCH PAPER QUALITY: [1], [4], [19], [20], [21]

 

The quality of medical research published in literature can vary considerably. It is very important that a reviewer or reader should keep this in mind the quality of that paper when reading the outcome or findings from a research study or in deciding that whether he can use these finding or results for secondary analysis or not.

Before starting a critical review or reading a paper one should keep in mind the following aspects or questions.

1)     Is this a peer reviewed research?

The research studies or projects which are peer reviewed, have been evaluated by the experienced scholars, having relevant experience in their field. Similarly, the research papers published in peer viewed professional journals have been reviewed or evaluated by the professional researchers who are experts in their relevant field and who can vouch for the reliability and validity of the methodology and the analysis applied. Therefore,  peer-reviewed research or publication is considered a high-quality research or paper.

2)     Is the study's quality to be evaluated with the information provided?

The quickest way is, by going through the abstract, one should critically analyze or evaluate the study's objective, methodology, results and conclusions along with precise description of the sample, main variables, data collection procedures and analytic techniques. The research consumers should be critical when reading or analyzing the research paper that whether sufficient information is provided about the above-mentioned key quality research components or not.

3)     Are there any potential threats to the validity of the study?

The research questions should be answered scientifically and in a rigorous manner in a valid study. The potential threats to the validity of a study are found in three domains mainly i.e. threat to Internal, external, and construct validity which needs to be assessed.

4)     Any ethical concerns about this research?

All ethical issues or concerns which could be raised during this study to be considered meticulously and pointed out or asked to justify.

 

4.     TITLE PAGE: [21], [22], [23]

 

The title page of a manuscript or research proposal should contain the information regarding title of subject, authors information (qualification, designation, institute, contact details), correspondence author with full details.

While critical analyzing this, one should keep in mind the following questions:

Is title meaningful, catchy and in line with objective?

It should not be the repetition of objective?

It should not contain any abbreviation.

Are authors/ participants competent enough to conduct this study and justified?

 

5.     ABSTRACT: [9], [21], [22], [24]

 

Abstract reading is a quickest way of analyzing a manuscript or research article regarding its objective, study design, purpose of research, methodology carried, important results and conclusions. It should be written in a way that one can easily judge what are the key learning point, clinical relevance of study and whether all domains are in line with each other or not. In case of any serious doubts regarding any of the domains, the appraiser or assessor should skip the manuscript or synopsis at this stage. While analyzing abstract one should look for:

1)     Is the objective or aim of the study are significantly interesting, important and are written clearly?

2)     Is study design, place of study and exact duration (DD/MM/YY) given?

3)     Is the type of groups, randomization process if any, sample size, sample selection, variables to be measured and measuring tool(s) evaluated properly and mentioned briefly or not?

4)     Is the sample size adequate to withdraw a meaningful and reliable difference or effect?

5)     Are important results as per objective with their statistical analysis and significance given or not?

6)     Is there any significant reason to doubt the validity of results (i.e. poor methodology, insufficient sample size or large confounders)?

7)     Conclusion has clearly answered the question of interest as per objective and not the repetition of results?

6.     INTRODUCTION/ BACKGROUND SECTION: [21], [22], [24], [25]

 

The introduction should thoroughly include literature evidence or references from the earlier work which is related to the present research under discussion and also include the limitations and importance of the research which has been acknowledged previously. If research hypothesis and objective are not clearly justified or study variables or findings varies from objectives, the outcome of study becomes questionable. Therefore, the hypothesis and objective should be clearly explained in this section along with rationale or justification of conducting this study in a scientific way. While critically analyzing one should look whether following information are included or not:

1)     Any historical back ground if relevant.

2)     Any brief epidemiology if necessary.

3)     What is the hypothesis or study question?

4)     Why this study is considered necessary or rationale of this study?

5)     What has been already achieved (as a brief literature review) and how does this study will achieve at variance?

6)     Any impact on existing knowledge, or clinical practice?

7)     Does the scientific approach adopted, outlines the benefits along with possible disadvantages which may be related with the observations or interventions?

8)     What is the objective or purpose of this study?

 

Research question: It is imperative for a research to be considered valuable, if it addresses an important or significant health care issue, give meaningful results and appropriate conclusion. For this, the hypothesis and objective of the research must be clear and self-explanatory.  The different terms should be defined clearly, even more so if they’re new or used in specific non-spread ways. As a reader one should pay particular attention to the errors in logic, especially those regarding association, relationship or causation [1], [10].

 

7.     METHODOLOGY: [1], [22], [25], [26], [27], [28]

 

A steppingstone to conduct a meaningful research, is a good workable scientific hypothesis supported by the strong and well-designed methodology. In a research protocol or paper all questions related to study design, clinical query, the sample or subjects, methodology used to collect data and different correlated measures to minimize bias and confounding factors should be explored and addressed thoroughly and adequately. It needs to be written in order, in a precise way containing important and full details of how the study was conducted. Precise information should be included about the study design, place of study, exact duration, studied population, sample selection procedure, sample size, interventions, data collection should be mentioned briefly and precisely.

Research methodology assessment in the study is a major and prime step in critical appraisal which is carried out by using checklists specific to study design. The commonest questions in check list are:

1)     Does suitable study design or type is mentioned?

2)     Does the study place (where carried out, not places of data collection) is mentioned?

3)     Is the study type appropriate to the research question?

4)     What is sample size and how it’s selected?

5)     What are the outcome variables or parameters to be measured as per objective of study?

6)     How these variables or parameters to be measured?

7)     What are the important potential confounders to be considered?

8)     Are all ethical issues are considered?

9)     How data analyzed and which statistical analysis method used on each variables or parameters in the study?

10) What are the Statistical results?

11) What conclusions did the authors reach about the research question?

 

Research designs: The research or study design is fundamental to the usefulness of study and should be appropriate as per objective and methodology given and suitable to address the research question [26].

 

Sample size and selection: [1], [21], [27], [29] To achieve trustworthy outcome or conclusions, it’s important that the sample selection and size should be representative and adequate respectively. The researchers should clearly identify the targeted subject or population, which they are interested in. The representativeness of sample depends upon the selection method and assignment.  For example, a random assignment has better advantages as compared to systematic assignment in establishing a group equivalence. Similarly, a sample can be biased when one use selective or volunteer’s attrition. The adequate sample size should be calculated by using power analysis before start of research. The Knowledge of the samples baseline characteristics is important because it allows to assess how closely the subjects match their own patients.

The sample size should be adequate to have a high probability of identifying a worthwhile results or outcome if it exists.  All research studies should report the total number of specimens or subjects at the beginning along with details of how many have completed the study and justification or reasons for incomplete follow up if there is any. In this regard, the assistance from statistician about sample calculation and size should be looked.

To critically analyze the information’s regarding sample the following information should be included in methodology and analyzed during critical reviews:

1)     Is the sample clearly defined or which subject it represents (i.e. type Human, Animals etc.)?

2)     Is the eligibility criteria mention with reasons?

3)     Does it give information that how and from where the sample recruited or selected and assessed?

4)     Is the sample size justified or calculated rightly and adequate to give clinical and statistically significant results?

5)     Are the samples uniform or alike at the baseline and any attrition in sample is mentioned or not?

6)     How the subjects were selected or recruited? If not random, then are they representative of the population?

7)     Are there any selection biases? If yes than how possible selection biases controlled (like volunteer bias, admission rate bias, prevalence bias, detection bias, recall bias, lead Time bias etc.)?

 

Study variable/ parameter assessment: [21], [27], [30] All study variables as per objective should be mentioned clearly and defined along with their assessment or data collection techniques. It should be mentioned precisely, how data measured or collected and who collected the data or performed intervention. The used data collection or measuring techniques or instruments used in research should be reliable and valid and the criteria measures must demonstrate reliability and validity for both, the independent and dependent variable. The following issues should be considered at the time of critical evaluation:

1)     What are the study variables/ parameters as per objective?

2)     Are all relevant variables/ parameters as per objective are included in study?

3)     How these variable/parameters measured?

4)     Have the factors been measured using appropriate tools?

5)     Are any data collection technique or measurements taken likely to be reliable and valid? – effectiveness, soundness of the measuring instrument.

6)     Does it measure, what it is supposed to be measured?

7)     What does the test measure?

8)     What are the outcome factors and how are they measured?

9)     Are all relevant study outcomes are included and assessed?

10) Is measurement error an important source of bias?

 

A suitable and convenient structure for the assessment of the health care problem addressed is the Problem Intervention Comparison Outcome (PICO) method [30] which is comprises of:

Patient or problem (P) - needs to be identified if research has a focused question like what is the chief complaint? This is to inquire about disease status, previous ailments, current medications etc.,

Intervention (I) - it should state clearly and appropriately the management strategies e.g. to assess a new diagnostic test, treatment intervention or any adjunctive therapy etc.

Comparison (C ) - to see any appropriate control or alternative which is available e.g. any specific choice which is limited to one alternative.

Outcomes (O) - to assess or identify the desired results or patient related outcome to be identified. e.g. symptom elimination, functional improvement, esthetics outcome etc.

 

Intervention: [20], [21], [23], [24], [25], [28] The researchers use a wide variety of robust techniques to make methodology more effective and reliable like blinding, randomization, restriction matching etc. Any shortcoming in intervention in methodology most likely leads to collection data or results which that do not reflect the truth. Any change in clinical practice on the basis of these false results can leads to harm to the patients. The following questions needs attention and analyzed whatever is applicable according to study design and type:

1)    In blind study –

·        What are the types of Blinding (Single, Double, Triple) and is it used properly?

·        Does it mention who was blinded?

·        Are the assessors and participants blind to the interventions received?

2)    Is there any control group and how it was selected?

3)    How the follow up is done? Who is the study drop outs and what is reason and number?

4)    In a multi-center study - what measure employed for quality assurance to get consistency in results across all centers?

5)    Are the dependent (outcome) and independent (predictor) study variables clearly defined, identified and measured properly?

6)    Does type of randomization process if any is mentioned?

7)    In a case-control study:

·        Is this study is controlled adequately?

·        Is the use of controls appropriate?

·        Were the records of control and case reviewed blindly?

8)    In a Cross-Sectional Study:

·        is sample selected in an appropriate way (i.e. convenience sampling etc.)?

·        are any efforts made to guarantee a good response rate or to minimize missing data occurrence?

·        any validity and reliability (reproducibility) is reported?

9)    In intervention study- how the subjects recruited and assigned to groups?

10) In a cohort study:

·        Are subject’s representatives of population to which the outcome is applied?

·        any evidence of volunteer bias and appropriate follow-up period was given?

·        how many subjects have reached final follow-up?

·        What was the subject drop-out rate and why?

 

Bias is the term used to designate an error which can occur at any stage of research and it is not due to chance. Bias in research most likely leads to results having systematic deviation from the actual results. It is not possible to measure the bias so in order to minimize the researchers should rely on a good study design. An example in this is that in a study of disease prevalence in a population, the sample population should be representative of the population. Moreover, it is imperative to consider the study sample size and proper statistical analysis to produce statistically significant results like <0.05 in p-values etc.[31], [32].

Control of confounding variables: A confounder is having a triangular relationship with both exposure and the outcome but it is not on the causal pathway. It shows a direct relationship between the outcome and exposure or it may even mask an association that would have been present otherwise in methodology, effective methods should be applied to control the relevant confounding variables which are expected to influence the findings. The extraneous variation can influence research findings, therefore methods to control relevant confounding variables should be applied [33]. During critical evaluation one should look for information’s regarding [21],[28].

1)     Which potential important confounders are considered?

2)     Are any potential confounders examined and controlled for?

3)     Is confounding an important source of bias?

 

Ethics of Research: [8],[25],[34],[35],[36],[37] It is important to one should assess that whether the researchers have followed the principles of research ethics. This is the commonly neglect domain by the reviewer or reader during critical review of a paper or research project. Like many other the Belmont Report [3] has given a detailed outline for the research ethics involving human subjects. Three basic ethical principles should be looked for are:

1)     Respect for human subject: —the research subjects should not persuade to participate in a research and requires protection who have diminished autonomy.

2)     Beneficence: the research should not harm the subjects, and its responsibility of the researchers to minimize the risks and maximizing the benefits for subjects.

3)     Justice: all forms of differential treatments or diagnostic modalities offered to the subjects should be justified.

 

Inclusion of human subjects to research should have given consideration to:

Informed Consent: [34],[35] It is mandatory that informed consent should be given by the participants before participating in research. In order to get informed consent.

1)     The purpose, duration, contents, potential benefits and risk of the research should be conveyed to the participants by the researcher.

2)     The participant has all right to quit research at any time.

3)     In research having questionnaire or survey, the participants are not bound to respond to all items.

4)     In case of children under legal age, consent from parents or guardian must be taken.

 

Confidentiality: [34], [36]  It is imperative that the identity of the participants should not be disclosed unless the consent is given otherwise. The confidentiality means, in any way, the participants cannot be identified. Any of the participants' identifiers like name, telephone numbers, addresses, national ID etc., must be safeguarded.

Anonymity: [35], [37]  Anonymity of personal identity or data is an even stronger safeguard of the privacy of participant. If the anonymity is ensured by the researcher in study, it means that it is not possible for researcher to link the participants' names to the data or information collected from them.

While critically analyzing the research paper, the emphasis should be given to following:

1)     Are all ethical issues in study considered properly as per guideline?

2)     If a study involves human subjects or tissues or animals, were approval from appropriate institution or regulatory body obtained?

3)     If consent is required, whether all aspects were addressed properly?

4)     Does the paper indicate ethics approval?

5)     How confidentiality and anonymity of the subjects were maintained?

6)     Can any potential ethical issues have identified?

7)     Any conflict of interest mentioned by author or identified by reviewer?

 

8.     RESULTS: [1],[10],[21],[25],[28]

 

This section should clearly contain information’s regarding what actually occur to the studied subjects. The results may contain raw data and explained in the statistical analysis. The results should be comprehensive and given in the form of tables, graphs or diagrams.  The tables or graphs should be labelled properly giving n, % or other values in an understandable way. Any Irrelevant data beyond the scope of objective or repetition of data should be avoided. During critical analysis of the results the following quarries should be answers:

1)     Does results clearly gave the main finding(s) as per objective and presented data support them?

2)     Are all the relevant outcomes mentioned in objective are assessed?

3)     How strong is the association between outcome and intervention?

4)     Is the sample size suitable to perceive a clinically significant result?

5)     How accurately it measured the results and is there any measurement error in data?

6)     Does clinical significance of results mentioned?

7)     Is any adverse event or lack of it mentioned if applicable?

8)     Are the data/results presented in a way to help in health policy decisions?

9)     Do the results apply to the population in which the researchers are interested?

10) Can we use these results to clinical governance or other studies?

 

Data analysis: [21],[38],[39],[40],[41] A correct statistical analysis of the data is very crucial in the reliability of the conclusions which is drawn at the end of research. It is important that on each variable or data obtained an appropriate statistical test should be applied and mentioned clearly in methodology or statistical analysis. For example, Post hoc tests on multiple comparisons needs to be performed. Percentages, n value should be carefully calculated and all figures and tables should be labelled clearly showing which type of data is presented. Ideally, effect sizes of data should be incorporated throughout by giving a precise indication of the variables’ impact.

p-value interpretation: [38], [39] It refers to the probability of any specific outcome which could have occurred by chance. A p-value of <0.05 or >1 in 20 is significant statistically. The null hypothesis is rejected and results are considered statistically significant, when the p-value is less than significance level (>0.05). Conversely, in more than 0.05 p-value, the result is considered statistically insignificant and null hypothesis is accepted.

Confidence interval interpretation: [38], [39] The repeated trials of the same study, unlikely to yield exactly the same results every time but on average the results should be within a certain range.  Therefore, confidence interval of 95% denoted that there are 95% probability that the results will lie within this range.

During critical analysis of data, the following quarries should be answers in manuscript:

1)     How data is analyzed?

2)     Were the statistical tests appropriate for the variables and data?

3)     Is assessment of the statistical significance evaluated?

4)     Are p-value or confidence intervals given?

5)     Is any measurement error considered as an important source of bias?

6)     Is there any risk estimation and how precise is the estimation?

7)     Do the statistical analysis or tests answer the research question?

 

9.     DISCUSSION AND CONCLUSIONS: [10],[12],[14],[21],[25],[28],[40]

Discussion should be given in a scientific and rational way. All variables or parameters of the study should be discussed separately in separate paragraphs. Findings of the study to be compared with literature and reasons or justifications should be given if they vary, in a scientific way with an outcome in last sentence of that paragraph. The objective, discussion and conclusions should be in line and consistent with the study’s data and results. It is a common mistake that researcher emphasizes the findings in discussion which are consistent with their expectations and not focusing on the findings which are not. The limitations of the study should be mentioned if any.

In discussion a rationale and absolute comparison of what is already published or identified in literature and its clinical relevance with the newly observed outcome or results should be included meticulously and precisely. In addition, any possible related limitations of the study, any necessity for further research and any suggestions proposed should also be indicated.

The conclusions of a study should ensure that it’s not the repetition of results and is in line with the objective and results attained within the scope of study.

While critically analyzing this section, we should look for these quarries:

1)     Does the main findings summarized and any deficiencies in the study design or methodology of study were mentioned? (This is called intention to treat analysis).

2)     Are the interpretations in discussion consistent with results?

3)     Does any comparison of results with the previous work in same domain were done?

4)     Can the results suitable to be generalized (i.e. external validity)?

5)     Does any clinical implications or applicability of study is mentioned?

6)     Which of the outcome/results/findings are applicable to the existing clinical practice and how will they affect it?

7)     Is there any discussion out of the scope of objective or data?

8)     Any source of potential bias addressed and how null findings are interpreted if any?

9)     Any shortcomings or limitation of the study mentioned and constructive suggestions for future research are given?

10) Are the study question or hypothesis posed, addressed adequately in discussion?

11) Which conclusions did the researcher reach about study question?

12) Does the conclusion of study amicably answer the study question and convincing?

13) Are the conclusions given, justified by the findings or data?

14) Do the conclusions match the objective, data and statistical significance?

10.  META-ANALYSIS AND SYSTEMIC REVIEWS [42],[43],[44],[45],[46] :

 

In evidence-based medicine pyramid, the meta-analysis and systematic reviews are considered the preferred source with highest level of reliability for practice guidelines and professional recommendations. The study level Meta-analysis (MAs)are often considered as an attractive initial step in publication especially for young researchers as primary data collection from subjects and ethical approval is not required. Moreover, MA can be performed relatively within available data from literature.  Due to this, given importance of MA in making clinical decisions and in analysis of available evidences, the factors affecting the quality of a MA are considered of paramount significance.

Similarly, to answer a wide variety of clinical quarries about an etiology, epidemiology, diagnosis, prognosis and adverse effects other than interventional studies systematic review of randomized clinical trials is also appropriate. For example, the randomized trials can provide good estimates of effects of treatment or intervention but are poor estimates of overall outcome or prognosis.  With rare exception, the Systematic reviews are always preferred because in isolation, no study, whatever the type is, can be interpreted. Therefore, the systematic reviews are considered the best study type available in answering the clinical question posed.

Several guidelines available in literature for reviewers and authors for standardized reporting and conduction systemic reviews and meta-analysis. Among those, PRISMA guidelines for Systematic review and Meta-Analysis of randomized controlled trials [46], [47] and for network meta-analysis [48] are preferred usually. Similarly, the MOOSE (Meta-analysis of Observational Studies in Epidemiology) Guidelines are also available to reviewers to conduct meta-analysis of observational studies [49].

In the critical analysis of Meta-analysis and Systemic review, the following queries must be looked for [43],[44],[45]:

1)     Is the research question clearly focused and specific?

2)     Is the inclusion and exclusion criteria for studies is specific and mentioned?

3)     are studies included with homogeneous, appropriate and proper study design?

4)     Did it address a clearly focused issue?

5)     Was there enough information on population studied, intervention given and outcome considered?

6)     Did the authors do enough to assess the quality of the included studies?

7)     How the publications bias issue is addressed (Funnel Plot Test is used usually)?

8)     Is the valid and clear definition of treatment or exposure is given in a priori fashion?

9)     Is the valid and clear outcome definition is given in a priori way?

10) Is the results presentation, clear, consistent and precise?

11) Is comprehensive or appropriate literature search strategy with an aim to include all relevant studies given?

12) Is method of the quality assessment of each study individually with standardized tools given?

13) Are appropriate mathematical modeling techniques used?

14) How assessment of heterogeneity and attempt to explain it done?

15) Is the appropriate conclusion given?

 

11.  DISCUSSION

 

Critical appraisal of a research paper or project is a fundamental skill in modern medical practice for assessing the worth of clinical research and in providing a guideline of its relevance to the profession. Its skills to be mastered not only by academic medical professionals but also by the doctors involved in clinical practice. A meticulous evaluation of the published or current literature is an important step in evidence‑based medicine practice, before incorporating changes in patient’s management.

This review presents an overall, generic and multi-disciplinary outline to identify the quality of research and how to evaluate each domain  of a research paper presented. First, all possible and essential sections and subsection of a paper were identified followed by the important steps and information required in each section or questions which may arise or needs to addressed were identified and mentioned. The purpose is to offer a comprehensive, systematic and a broad outline necessary for an appraisal of a research paper or project.  We also argue that this will provide a comprehensive and a useful approach for discussing research projects and their quality from different perspectives, and also helpful in advance discussions on research quality within and across disciplines.

 

SOURCES OF FUNDING

 

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

 

CONFLICT OF INTEREST

 

The author have declared that no competing interests exist.

 

ACKNOWLEDGMENT

 

None.

 

REFERENCES

 

    [1]       Umesh G, Karippacheril JG, Magazine R.. Critical appraisal of published literature. Indian J Anaest. 2016;60( 9): 670-673.

    [2]       Burls A. What is Critical Appraisal? 2nd ed. Oxford, UK:University of Oxford. (What is…? Series).2009. Available from website: http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/what_is_critical_appraisal.pdf. Accessed on June 20, 2020

    [3]       Audit of Submissions: July 2018-June 2019. J Health Serv Res Policy. 2020;25(1):4. doi: 10.1177/1355819619899554.

    [4]       Glasziou PP. Information overload: What’s behind it, what’s beyond it? Med J Aust 2008;189:84‑85.

    [5]       Sadeghi-Bazargani H, Tabrizi JS, Azami-Aghdash S. Barriers to evidence-based medicine: a systematic review. J Eval Clin Pract 2014;20(6):793-802. doi: 10.1111/jep.12222

    [6]       Banzi R, Liberati A, Moschetti I, Tagliabue L, Moja L. A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res 2010;12(3):e26. doi: 10.2196/jmir.1288

    [7]       Kwag KH, González-Lorenzo M, Banzi R, Bonovas S, Moja L. Providing Doctors with High-Quality Information: An Updated Evaluation of Web-Based Point-of-Care Information Summaries. J Med Internet Res 2016;18(1):e15. doi: 10.2196/jmir.5234.

    [8]       Lenaerts G, Bekkering GE, Goossens M, De Coninck L, Delvaux N, Cordyn S, Adriaenssens J, Vankrunkelsven P. Tools to Assess the Trustworthiness of Evidence-Based Point-of-Care Information for Health Care Professionals: Systematic Review J Med Internet Res 2020;22(1):e15415. URL: https://www.jmir.org/2020/1/e15415. doi: 10.2196/15415.

    [9]       Burls A. What is critical appraisal?  London, Hayward Medical Communications. 2016. Website: [ http://www.whatisseries.co.uk/what-is-critical-appraisal/.] Accessed on June 18, 2020

  [10]     Hill MD. How to Review a Clinical Research Paper. Stroke. 2018;49(5): e204-e206. https://doi.org/10.1161/STROKEAHA.118.021286.

  [11]     Cruz Rivera S, Kyte DG, Aiyegbusi OL, Keeley TJ, Calvert MJ. Assessing the impact of healthcare research: A systematic review of methodological frameworks. PLoS Med. 2017;14(8):e1002370. https://doi.org/10.1371/journal.pmed.1002370.

  [12]     Mårtenssona P, Forsb U, Wallinc S, Zanderd U, Nilssone G H. Evaluating research: A multidisciplinary approach to assessing research practice and quality. Res Policy;2016: 45 (2016) 593–603. http://dx.doi.org/10.1016/j.respol.2015.11.009.

  [13]     Guinea J, Sela E, GoÂmez-Nu ñez AJ, Mangwende T, Ambali A, Ngum N, et al. Impact oriented monitoring: A new methodology for monitoring and evaluation of international public health research projects. Res Eval. 2015; 24(2):131±45. https://doi.org/10.1093/reseval/rvu034.

  [14]     Belcher BM, Rasmussen KE, Kemshaw MR, Zornes DA.  Defining and assessing research quality in a transdisciplinary context. Res Eval. 2015; 25:(2016):1–17.  doi:10.1093/reseval/rvv025.

  [15]     Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomized trials. BMJ 2010;340:698‑702.

  [16]     Scottish Intercollegiate Guidelines Network. 2020 website: [http://www.sign.ac.uk/methodology/checklists.html]. Accessed on 2016 Jun 19.

  [17]     Strobe Statement. 2020 Available from: [http://www.strobe‑statement.org/index.php?id=available‑checklists]. Accessed on 2016 Jun 19.

  [18]     Sorensen HT, Rothman KJ. The prognosis for research. BMJ. 2010;340:c703. doi: 10.1136/bmj.c703.

  [19]     Meltzoff J. Critical Thinking About Research. Washington, DC: Am Psychol Assoc. 2007;23(5):234-243.

  [20]     Roever L, Resende ES, Diniz ALD, Penha-Silva N, Biondi-Zoccai G, et al.  Critical Analysis of Clinical Research Articles: A Guide for Evaluation. Evid Based Med and Pract. 2016; 2(1): e116. Doi: 10.4172/2471-9919.1000e116.

  [21]     Al-Jundi A, Sakka S.   Critical Appraisal of Clinical Research. J of Clin and Diag Res. 2017;11(5): JE01-JE05. DOI: 10.7860/JCDR/2017/26047.9942.

  [22]     Al-Jundi A, Sakka S. Protocol writing in clinical research. J Clin Diagn Res. 2016;10(11):ZE10-ZE13j.

  [23]     Young JA, Solomon MJ. How to Critically Appraise an Article. Nat Clin Pract Gastroenterol Hepatol.2009; 6: 82-91.

  [24]     Gibson CJ. Critical appraisal: a template to evaluate scientific literature. Dent Update. 2008; 35: 414-417.

  [25]     MacInnes A, Lamont T. Critical appraisal of a research paper. Scott Uni Med J. 2014;3 (1):10-17.

  [26]     Röhrig B, du Prel JB, Blettner M. Study design in medical research: part 2 of a series on the evaluation of scientific publications. Dtsch Arztebl Int. 2009;106: 184-189.

  [27]     Röhrig B, du Prel JB, Wachtlin D, Kwiecien R, Blettner M. Sample size calculation in clinical trials: part 13 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2010; 107: 552-556.

  [28]     Micallef C. Critical analysis: a vital element in healthcare research.  Critical analysis: a vital element in healthcare research. Int. J. Behav and Healthcare Res. 2015;5(1/2):105-110.      

  [29]     Greenhalgh, T. How to read a paper: The basics of evidence-based medicine. 5th ed, New York, United States; John Wiley & Sons: 2014. Pp134-167.

  [30]     Boudin F, Nie J, Bartlett JC. Combining classifiers for robust PICO element detection. BMC Med Inform Decis Mak. 2010;10:29. https://doi.org/10.1186/1472-6947-10-29.

  [31]      Galdas P. Revisiting Bias in Qualitative Research: Reflections on Its Relationship with Funding and Impact. (Editorial). Int J qualitative Methods. 2017: 16:1-2. https://doi.org/10.1177/1609406917748992

  [32]     Mackieson P, Shlonsky A,  Connolly M. Increasing rigor and reducing bias in qualitative research: A document analysis of parliamentary debates using applied thematic analysis. 2018;18:176-180. https://doi.org/10.1177/1473325018786996

  [33]     Thomas L. Understanding confounding variables. Scribber 2020; website: [https://www.scribbr.com/methodology/confounding-variables/]. Accessed on June 10, 2020

  [34]     The Belmont Report. 1979 Ethical Principles and Guidelines for the Protection of Human Subjects of Research The National Commission for the Protection of Human Subjects of Biomedical. Retrieved from Website: [https://science.education.nih.gov/supplements/webversions/bioethics/guide/teacher/Mod5_Belmont.pdf] Accessed on June 11, 2020.

  [35]     American Psychological Association (2010, June 1). American Psychological Association Ethical Principles of Psychologists and Code of Conduct. 2011. Retrieved from [ http://www.apa.org/ethics/code/index.aspx]. Accessed on June 7, 2020

  [36]     Ethical standards and procedures for research with human beings. Ethics and Research. WHO Manual (Section XV.2)  2020. Retrieved from Website:[ https://www.who.int/ethics/research/en/#:~:text=Research%20ethics%20govern%20the%20standards,and%20welfare%20of%20research%20participants.]. Accessed on June 28, 2020

  [37]     Ensuring your research is ethical: A guide for Extended Project Qualification students. EPQs ethics guide. Welcome Trust. 2020. Retrieve from website:[ https://wellcome.ac.uk/sites/default/files/wtp057673_0.pdf]. Accessed on June 28, 2020

  [38]     du Prel JB, Hommel G, Röhrig B, Blettner M. Confidence interval or p-value?: part 4 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2009;106: 335-339.

  [39]     du Prel JB, Röhrig B, Hommel G, Blettner M. Choosing statistical tests: part 12 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2010; 107: 343-348

  [40]     Bialocerkowski A, Klupp N, Bragge P. How to read and critically appraise a reliability article? Int J Ther Rehab 2010;17:114‑20.

  [41]     Young J M, Solomon MJ. How to Critically Appraise an Article. Nat Clin Pract Gastroenterol Hepatol. 2009;6(2):82-91.

  [42]     Hameed A, Demetres M, Tam D Y, Rahouma M, Khan F M, Wright D N et al.  An assessment of the quality of current clinical meta-analyses BMC Med Res Methodol. 2020;20:105-113. https://doi.org/10.1186/s12874-020-00999-9.

  [43]     Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both. BMJ. 2017;358:j4008. https://doi.org/10. 1136/bmj.j4008.

  [44]     Roever L, Zoccai GB (2015) Critical Appraisal of Systematic Reviews and Meta-analyses. Evid Based Med and Prac. 2015;1:1000e106. doi:10.4172/2471-9919.1000e106.

  [45]     Patel N, Bajaj NS. Meta-analyses: How to critically appraise them? J of Nuclear Cardio. 2017; 25(5);1598–1600. doi:10.1007/s12350-017-0898-8

  [46]     Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1.

  [47]     Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and explanation. BMJ 2015;349:g7647.

  [48]     Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: Checklist and explanations. Ann Intern Med 2015;162:777-784.

  [49]     Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: A proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008-2012.