1.
INTRODUCTION
1.1. SEARCH METHODOLOGY
The literature for
this review is searched from MEDLINE (Ovid), Medscape, Pub Med, EMBASE,
Scopus, HMIC, CINAHL+ and the Journal of
Research Evaluation along with other sources (i.e. Google Scholar, Research
Gate and Expert communication) by using Critical appraisal, Research paper,
Evidence based practice, Research methodology and Study validity as key words.
All these databases contain archives of majority of biomedical journals from
all over the world. Among a total of 128 full text articles retrieved, through
meticulous analysis, 49 full-text articles containing robust and pertinent
information mentioned above were short listed for review. All relevant
scientific papers, written in English were included and non-scientific
articles, non-scientific commentary and reports were excluded from the review.
Through
detailed literature search, for writing this review, all important sections
necessary for manuscript writing like title page, abstract, introduction,
methodology, results, data analysis, discussion etc along with subsections were
identified. All necessary steps or information required for review in each
section or questions which may arise or needs to addressed during a review were
also identified. Among all shortlisted papers the pertinent information’s or
data retrieved, analyzed and reproduced in each
section or sub-section in own description. Through detailed literature search,
all essential sections and subsection mandatory for a research paper were
identified
2.
BACKGROUND
The objective of publishing a research
paper is to provide an accurate, impartial medical information, authenticated
by robust scientific evidences which are helpful in enhancing the patient care [1]. The Critical appraisal of a research is “The process of carefully
and systematically examining the research to judge its trustworthiness, value
and relevance in a particular context” [2] . With an ever-increasing volume of scientific literature (i.e.
only in MEDLINE database around more than 12,000 new papers were added weekly[3]), it can be arduous for a
clinician to keep themselves abreast with this massive load of information’s
from current literature while working with their patients in hospital [4], [5]. One of the keystones of high-quality health care is Evidence-based
medicine and by its judicious, explicit and conscientious use, we can provide
an effective and efficient patient care and make right decisions in best
interest of our patients [6], [7]. Therefore, the health
professionals require a good quality, comprehensive, well structured, easily
accessible while interacting with patients [7], [8]. To achieve this, the critical appraisal of a research paper is
essential to judge its reliability, relevance and value of clinical research
regarding future implications and also as a part of Continuing Professional
Development (CPD) [9].
A good research work always builds the
basis for future research and also determines its values and further course.
Whereas, an inaccurate data or results leads to inaccurate outcomes, which
end up in further misconceptions in scientific knowledge. To set a concrete
basis for research in medical sciences, it is imperative to prevent the
fabrication or dishonestly multiplication of information by critically
appraising the scientific evidence presented and to decide that which
information can be considered as plausible [1], [10].
It is also very helpful in identifying the
flaws in study and to guide the researcher, how to overcome these observations
because flawed published studies most likely leads to faulty clinical decisions
which may end up with worst outcome or no benefit in patient care [11], [12]. Moreover, by knowing how to critically analyze a research paper,
readers can make their informed judgement about the reliability and validity of
research produced. In addition, the critical appraisal of research projects or
paper also plays a key role in institutional research policies like promotions,
hiring, firing, and funding schemes [6], [9].
The importance of the quality of research
even emphasized by Cicero [10] in the beginning of last century, by emphasizing that “number does
not matter but the quality does”. But so, for, there is no or little consensus
on accepted standards on how to measure research quality objectively? [5], [8] Therefore, the individual research evaluation remains a notoriously
difficult practice with no standard solution [13], [14]. Different guidelines were followed by different medical journals or
institutions for critical appraisal of research papers. For example, the
guideline for reporting and evaluating randomized controlled trials is CONSORT
(Consolidated Standard for Reporting Trial) statement which is mandatory for
publication of manuscript in majority of international medical journals [15]. Similarly, some other
scientific societies like Scottish Intercollegiate Guidelines Network, has also
devised checklists which are helpful in the critical appraisal of manuscripts
depending upon the type of study methodology [16], [17].
Another major problem in evaluating or
considering a research hypothesis or an idea is that we are bound to rely on
findings on that topic given in the literature in past [18]. So, what is the
criteria to conclude whether the results and outcome of a research can be
trusted or not? Moreover, as research evaluator one should kept in mind that
any type of errors in research may emerge from any stage of the research
process. Therefore, the appraisal is all to boils down to how the research has
been conducted, results interpreted, what is outcome, should be considered
meticulously when reading as articles. This review is written with an aim to
give an outline of how to conduct a critical appraisal, especially in the main
domains of a research paper.
3.
ASSESSING RESEARCH PAPER
QUALITY: [1], [4], [19], [20], [21]
The quality of medical research published
in literature can vary considerably. It is very important that a reviewer or
reader should keep this in mind the quality of that paper when reading the
outcome or findings from a research study or in deciding that whether he can
use these finding or results for secondary analysis or not.
Before starting a critical review or
reading a paper one should keep in mind the following aspects or questions.
1)
Is this a peer reviewed
research?
The research studies or
projects which are peer reviewed, have been evaluated by the experienced
scholars, having relevant experience in their field. Similarly, the research
papers published in peer viewed professional journals have been reviewed or
evaluated by the professional researchers who are experts in their relevant
field and who can vouch for the reliability and validity of the methodology and
the analysis applied. Therefore,
peer-reviewed research or publication is considered a high-quality
research or paper.
2)
Is the study's quality to be
evaluated with the information provided?
The quickest way is, by
going through the abstract, one should critically analyze or evaluate the
study's objective, methodology, results and conclusions along with precise
description of the sample, main variables, data collection procedures and
analytic techniques. The research consumers should be critical when reading or
analyzing the research paper that whether sufficient information is provided
about the above-mentioned key quality research components or not.
3)
Are there any potential threats
to the validity of the study?
The research questions should
be answered scientifically and in a rigorous manner in a valid study. The
potential threats to the validity of a study are found in three domains mainly
i.e. threat to Internal, external, and construct validity which needs to be
assessed.
4)
Any ethical concerns about this
research?
All ethical issues or
concerns which could be raised during this study to be considered meticulously
and pointed out or asked to justify.
4.
TITLE PAGE: [21], [22], [23]
The title page of a manuscript or research
proposal should contain the information regarding title of subject, authors
information (qualification, designation, institute, contact details),
correspondence author with full details.
While critical analyzing this, one should
keep in mind the following questions:
Is title meaningful, catchy and in line
with objective?
It should not be the repetition of
objective?
It should not contain any abbreviation.
Are authors/ participants competent enough to
conduct this study and justified?
5.
ABSTRACT: [9], [21], [22], [24]
Abstract reading is a quickest way of
analyzing a manuscript or research article regarding its objective, study
design, purpose of research, methodology carried, important results and
conclusions. It should be written in a way that one can easily judge what are
the key learning point, clinical relevance of study and whether all domains are
in line with each other or not. In case of any serious doubts regarding any of
the domains, the appraiser or assessor should skip the manuscript or synopsis
at this stage. While analyzing abstract one should look for:
1)
Is the objective or aim of the
study are significantly interesting, important and are written clearly?
2)
Is study design, place of study
and exact duration (DD/MM/YY) given?
3)
Is the type of groups,
randomization process if any, sample size, sample selection, variables to be
measured and measuring tool(s) evaluated properly and mentioned briefly or not?
4)
Is the sample size adequate to
withdraw a meaningful and reliable difference or effect?
5)
Are important results as per
objective with their statistical analysis and significance given or not?
6)
Is there any significant reason
to doubt the validity of results (i.e. poor methodology, insufficient sample
size or large confounders)?
7)
Conclusion has clearly answered
the question of interest as per objective and not the repetition of results?
6.
INTRODUCTION/ BACKGROUND
SECTION: [21], [22], [24], [25]
The introduction should thoroughly include
literature evidence or references from the earlier work which is related to the
present research under discussion and also include the limitations and
importance of the research which has been acknowledged previously. If research
hypothesis and objective are not clearly justified or study variables or
findings varies from objectives, the outcome of study becomes questionable.
Therefore, the hypothesis and objective should be clearly explained in this
section along with rationale or justification of conducting this study in a
scientific way. While critically analyzing one should look whether following
information are included or not:
1)
Any historical back ground if
relevant.
2)
Any brief epidemiology if
necessary.
3)
What is the hypothesis or study
question?
4)
Why this study is considered
necessary or rationale of this study?
5)
What has been already achieved
(as a brief literature review) and how does this study will achieve at
variance?
6)
Any impact on existing
knowledge, or clinical practice?
7)
Does the scientific approach
adopted, outlines the benefits along with possible disadvantages which may be
related with the observations or interventions?
8)
What is the objective or
purpose of this study?
Research question: It is imperative for a research to be considered valuable, if it
addresses an important or significant health care issue, give meaningful
results and appropriate conclusion. For this, the hypothesis and objective of
the research must be clear and self-explanatory. The different terms should be defined
clearly, even more so if they’re new or used in specific non-spread ways. As a
reader one should pay particular attention to the errors in logic, especially
those regarding association, relationship or causation [1], [10].
7.
METHODOLOGY: [1], [22], [25], [26], [27], [28]
A steppingstone to conduct a meaningful
research, is a good workable scientific hypothesis supported by the strong and
well-designed methodology. In a research protocol or paper all questions
related to study design, clinical query, the sample or subjects, methodology
used to collect data and different correlated measures to minimize bias and
confounding factors should be explored and addressed thoroughly and adequately.
It needs to be written in order, in a precise way containing important and full
details of how the study was conducted. Precise information should be included
about the study design, place of study, exact duration, studied population,
sample selection procedure, sample size, interventions, data collection should
be mentioned briefly and precisely.
Research methodology assessment in the
study is a major and prime step in critical appraisal which is carried out by
using checklists specific to study design. The commonest questions in check
list are:
1)
Does suitable study design or
type is mentioned?
2)
Does the study place (where
carried out, not places of data collection) is mentioned?
3)
Is the study type appropriate
to the research question?
4)
What is sample size and how
it’s selected?
5)
What are the outcome variables
or parameters to be measured as per objective of study?
6)
How these variables or
parameters to be measured?
7)
What are the important
potential confounders to be considered?
8)
Are all ethical issues are
considered?
9)
How data analyzed and which
statistical analysis method used on each variables or parameters in the study?
10) What are the Statistical results?
11) What conclusions did the authors reach about the research question?
Research designs: The research or study design is fundamental to the usefulness of
study and should be appropriate as per objective and methodology given and
suitable to address the research question [26].
Sample size and selection: [1], [21], [27], [29] To achieve trustworthy outcome or conclusions, it’s important that
the sample selection and size should be representative and adequate
respectively. The researchers should clearly identify the targeted subject or
population, which they are interested in. The representativeness of sample
depends upon the selection method and assignment. For example, a random
assignment has better advantages as compared to systematic assignment in
establishing a group equivalence. Similarly, a sample can be biased when one
use selective or volunteer’s attrition. The adequate sample size should be
calculated by using power analysis before start of research. The Knowledge of
the samples baseline characteristics is important because it allows to assess
how closely the subjects match their own patients.
The sample size should be adequate to have
a high probability of identifying a worthwhile results or outcome if it
exists. All research studies should
report the total number of specimens or subjects at the beginning along with
details of how many have completed the study and justification or reasons for
incomplete follow up if there is any. In this regard, the assistance from
statistician about sample calculation and size should be looked.
To critically analyze the information’s
regarding sample the following information should be included in methodology
and analyzed during critical reviews:
1)
Is the sample clearly defined
or which subject it represents (i.e. type Human, Animals etc.)?
2)
Is the eligibility criteria
mention with reasons?
3)
Does it give information that
how and from where the sample recruited or selected and assessed?
4)
Is the sample size justified or
calculated rightly and adequate to give clinical and statistically significant
results?
5)
Are the samples uniform or
alike at the baseline and any attrition in sample is mentioned or not?
6)
How the subjects were selected
or recruited? If not random, then are they representative of the population?
7)
Are there any selection biases?
If yes than how possible selection biases controlled (like volunteer bias,
admission rate bias, prevalence bias, detection bias, recall bias, lead Time
bias etc.)?
Study variable/ parameter assessment: [21], [27], [30] All study variables as per objective should be mentioned clearly and
defined along with their assessment or data collection techniques. It should be
mentioned precisely, how data measured or collected and who collected the data
or performed intervention. The used data collection or measuring techniques or
instruments used in research should be reliable and valid and the criteria
measures must demonstrate reliability and validity for both, the independent
and dependent variable. The following issues should be considered at the time
of critical evaluation:
1)
What are the study variables/
parameters as per objective?
2)
Are all relevant variables/
parameters as per objective are included in study?
3)
How these variable/parameters
measured?
4)
Have the factors been measured
using appropriate tools?
5)
Are any data collection
technique or measurements taken likely to be reliable and valid? –
effectiveness, soundness of the measuring instrument.
6)
Does it measure, what it is
supposed to be measured?
7)
What does the test measure?
8)
What are the outcome factors
and how are they measured?
9)
Are all relevant study outcomes
are included and assessed?
10) Is measurement error an important source of bias?
A suitable and convenient structure for the
assessment of the health care problem addressed is the Problem Intervention
Comparison Outcome (PICO) method
[30] which is comprises of:
Patient or problem (P) - needs to be identified if research has a focused question like what
is the chief complaint? This is to inquire about disease status, previous
ailments, current medications etc.,
Intervention (I) - it should state clearly and appropriately the management strategies
e.g. to assess a new diagnostic test, treatment intervention or any adjunctive
therapy etc.
Comparison (C ) - to see any appropriate control or alternative which is available
e.g. any specific choice which is limited to one alternative.
Outcomes (O) - to assess or identify the desired results or patient related outcome
to be identified. e.g. symptom elimination, functional improvement, esthetics
outcome etc.
Intervention: [20], [21], [23], [24], [25], [28] The researchers use a wide variety of robust techniques to make
methodology more effective and reliable like blinding, randomization,
restriction matching etc. Any shortcoming in intervention in methodology most
likely leads to collection data or results which that do not reflect the truth.
Any change in clinical practice on the basis of these false results can leads
to harm to the patients. The following questions needs attention and analyzed
whatever is applicable according to study design and type:
1)
In blind study –
·
What are the types of Blinding
(Single, Double, Triple) and is it used properly?
·
Does it mention who was
blinded?
·
Are the assessors and
participants blind to the interventions received?
2)
Is there any control group
and how it was selected?
3)
How the follow up is done?
Who is the study drop outs and what is reason and number?
4)
In a multi-center study -
what measure employed for quality assurance to get consistency in results
across all centers?
5)
Are the dependent (outcome)
and independent (predictor) study variables clearly defined, identified and
measured properly?
6)
Does type of randomization
process if any is mentioned?
7)
In a case-control study:
·
Is this study is controlled
adequately?
·
Is the use of controls
appropriate?
·
Were the records of control and
case reviewed blindly?
8)
In a Cross-Sectional
Study:
·
is sample selected in an
appropriate way (i.e. convenience sampling etc.)?
·
are any efforts made to guarantee
a good response rate or to minimize missing data occurrence?
·
any validity and reliability
(reproducibility) is reported?
9)
In intervention study- how the subjects recruited and assigned to groups?
10) In a cohort study:
·
Are subject’s representatives
of population to which the outcome is applied?
·
any evidence of volunteer bias
and appropriate follow-up period was given?
·
how many subjects have reached
final follow-up?
·
What was the subject drop-out
rate and why?
Bias is the term used to designate an error which can occur at any stage of research
and it is not due to chance. Bias in research most likely leads to results
having systematic deviation from the actual results. It is not possible to
measure the bias so in order to minimize the researchers should rely on a good
study design. An example in this is that in a study of disease prevalence in a
population, the sample population should be representative of the population.
Moreover, it is imperative to consider the study sample size and proper
statistical analysis to produce statistically significant results like <0.05
in p-values etc.[31], [32].
Control of confounding variables: A confounder is having a triangular relationship with both exposure
and the outcome but it is not on the causal pathway. It shows a direct
relationship between the outcome and exposure or it may even mask an
association that would have been present otherwise in methodology, effective
methods should be applied to control the relevant confounding variables which
are expected to influence the findings. The extraneous variation can influence
research findings, therefore methods to control relevant confounding
variables should be applied [33]. During critical evaluation one should look for information’s
regarding [21],[28].
1)
Which potential important
confounders are considered?
2)
Are any potential confounders
examined and controlled for?
3)
Is confounding an important
source of bias?
Ethics
of Research: [8],[25],[34],[35],[36],[37] It is important to one should assess that whether the researchers
have followed the principles of research ethics. This is the commonly neglect
domain by the reviewer or reader during critical review of a paper or research
project. Like many other the Belmont Report [3] has
given a detailed outline for the research ethics involving human subjects.
Three basic ethical principles should be looked for are:
1)
Respect for human subject: —the
research subjects should not persuade to participate in a research and requires
protection who have diminished autonomy.
2)
Beneficence: the research
should not harm the subjects, and its responsibility of the researchers to
minimize the risks and maximizing the benefits for subjects.
3)
Justice: all forms of
differential treatments or diagnostic modalities offered to the subjects should
be justified.
Inclusion of human subjects to research
should have given consideration to:
Informed Consent: [34],[35] It is mandatory that informed consent should be given by the
participants before participating in research. In order to get informed
consent.
1)
The purpose, duration,
contents, potential benefits and risk of the research should be conveyed to the
participants by the researcher.
2)
The participant has all right
to quit research at any time.
3)
In research having
questionnaire or survey, the participants are not bound to respond to all
items.
4)
In case of children under legal
age, consent from parents or guardian must be taken.
Confidentiality: [34], [36] It is imperative that the
identity of the participants should not be disclosed unless the consent is
given otherwise. The confidentiality means, in any way, the participants cannot
be identified. Any of the participants' identifiers like name, telephone
numbers, addresses, national ID etc., must be safeguarded.
Anonymity: [35], [37] Anonymity of personal
identity or data is an even stronger safeguard of the privacy of participant.
If the anonymity is ensured by the researcher in study, it means that it is not
possible for researcher to link the participants' names to the data or
information collected from them.
While critically analyzing the research
paper, the emphasis should be given to following:
1)
Are all ethical issues in study
considered properly as per guideline?
2)
If a study involves human
subjects or tissues or animals, were approval from appropriate institution or
regulatory body obtained?
3)
If consent is required, whether
all aspects were addressed properly?
4)
Does the paper indicate ethics
approval?
5)
How confidentiality and
anonymity of the subjects were maintained?
6)
Can any potential ethical
issues have identified?
7)
Any conflict of interest
mentioned by author or identified by reviewer?
8.
RESULTS: [1],[10],[21],[25],[28]
This section should clearly contain
information’s regarding what actually occur to the studied subjects. The
results may contain raw data and explained in the statistical analysis. The
results should be comprehensive and given in the form of tables, graphs or
diagrams. The tables or graphs should be
labelled properly giving n, % or other values in an understandable way. Any
Irrelevant data beyond the scope of objective or repetition of data should be
avoided. During critical analysis of the results the following quarries should
be answers:
1)
Does results clearly gave the
main finding(s) as per objective and presented data support them?
2)
Are all the relevant outcomes
mentioned in objective are assessed?
3)
How strong is the association
between outcome and intervention?
4)
Is the sample size suitable to
perceive a clinically significant result?
5)
How accurately it measured the
results and is there any measurement error in data?
6)
Does clinical significance of
results mentioned?
7)
Is any adverse event or lack of
it mentioned if applicable?
8)
Are the data/results presented
in a way to help in health policy decisions?
9)
Do the results apply to the
population in which the researchers are interested?
10) Can we use these results to clinical governance or other studies?
Data analysis: [21],[38],[39],[40],[41] A correct statistical analysis of the data is very crucial in the
reliability of the conclusions which is drawn at the end of research. It is
important that on each variable or data obtained an appropriate statistical
test should be applied and mentioned clearly in methodology or statistical
analysis. For example, Post hoc tests on multiple comparisons needs to be
performed. Percentages, n value should be carefully calculated and all figures
and tables should be labelled clearly showing which type of data is
presented. Ideally, effect sizes of data should be incorporated throughout by
giving a precise indication of the variables’ impact.
p-value interpretation: [38], [39] It refers to the probability of any specific outcome which could
have occurred by chance. A p-value of <0.05 or >1 in 20 is significant
statistically. The null hypothesis is rejected and results are considered
statistically significant, when the p-value is less than significance level
(>0.05). Conversely, in more than 0.05 p-value, the result is considered
statistically insignificant and null hypothesis is accepted.
Confidence interval interpretation: [38], [39] The repeated trials of the same study, unlikely to yield exactly
the same results every time but on average the results should be within a
certain range. Therefore, confidence
interval of 95% denoted that there are 95% probability that the results will
lie within this range.
During critical analysis of data, the
following quarries should be answers in manuscript:
1)
How data is analyzed?
2)
Were the statistical tests
appropriate for the variables and data?
3)
Is assessment of the
statistical significance evaluated?
4)
Are p-value or confidence
intervals given?
5)
Is any measurement error
considered as an important source of bias?
6)
Is there any risk estimation
and how precise is the estimation?
7)
Do the statistical analysis or
tests answer the research question?
9.
DISCUSSION AND CONCLUSIONS: [10],[12],[14],[21],[25],[28],[40]
Discussion should be given in a scientific and rational way. All variables or
parameters of the study should be discussed separately in separate paragraphs.
Findings of the study to be compared with literature and reasons or
justifications should be given if they vary, in a scientific way with an
outcome in last sentence of that paragraph. The objective, discussion and
conclusions should be in line and consistent with the study’s data and results.
It is a common mistake that researcher emphasizes the findings in discussion
which are consistent with their expectations and not focusing on the findings which
are not. The limitations of the study should be mentioned if any.
In discussion a rationale and absolute
comparison of what is already published or identified in literature and its
clinical relevance with the newly observed outcome or results should be
included meticulously and precisely. In addition, any possible related
limitations of the study, any necessity for further research and any
suggestions proposed should also be indicated.
The conclusions of a study should ensure
that it’s not the repetition of results and is in line with the objective and
results attained within the scope of study.
While critically analyzing this section, we
should look for these quarries:
1)
Does the main findings
summarized and any deficiencies in the study design or methodology of study
were mentioned? (This is called intention to treat analysis).
2)
Are the interpretations in
discussion consistent with results?
3)
Does any comparison of results
with the previous work in same domain were done?
4)
Can the results suitable to be
generalized (i.e. external validity)?
5)
Does any clinical implications
or applicability of study is mentioned?
6)
Which of the
outcome/results/findings are applicable to the existing clinical practice and
how will they affect it?
7)
Is there any discussion out of
the scope of objective or data?
8)
Any source of potential bias
addressed and how null findings are interpreted if any?
9)
Any shortcomings or limitation
of the study mentioned and constructive suggestions for future research are
given?
10) Are the study question or hypothesis posed, addressed adequately in
discussion?
11) Which conclusions did the researcher reach about study question?
12) Does the conclusion of study amicably answer the study question and
convincing?
13) Are the conclusions given, justified by the findings or data?
14) Do the conclusions match the objective, data and statistical
significance?
10. META-ANALYSIS AND SYSTEMIC REVIEWS [42],[43],[44],[45],[46] :
In evidence-based medicine pyramid, the
meta-analysis and systematic reviews are considered the preferred source with
highest level of reliability for practice guidelines and professional
recommendations. The study level Meta-analysis (MAs)are often considered as an
attractive initial step in publication especially for young researchers as
primary data collection from subjects and ethical approval is not required.
Moreover, MA can be performed relatively within available data from
literature. Due to this, given
importance of MA in making clinical decisions and in analysis of available
evidences, the factors affecting the quality of a MA are considered of
paramount significance.
Similarly, to answer a wide variety of
clinical quarries about an etiology, epidemiology, diagnosis, prognosis and
adverse effects other than interventional studies systematic review of
randomized clinical trials is also appropriate. For example, the randomized
trials can provide good estimates of effects of treatment or intervention but
are poor estimates of overall outcome or prognosis. With rare exception, the Systematic reviews are
always preferred because in isolation, no study, whatever the type is, can be
interpreted. Therefore, the systematic reviews are considered the best study
type available in answering the clinical question posed.
Several guidelines available in literature
for reviewers and authors for standardized reporting and conduction systemic
reviews and meta-analysis. Among those, PRISMA guidelines for Systematic review
and Meta-Analysis of randomized controlled trials [46], [47] and for network meta-analysis [48] are preferred usually. Similarly, the MOOSE (Meta-analysis of
Observational Studies in Epidemiology) Guidelines are also available to
reviewers to conduct meta-analysis of observational studies [49].
In the critical analysis of Meta-analysis
and Systemic review, the following queries must be looked for [43],[44],[45]:
1)
Is the research question
clearly focused and specific?
2)
Is the inclusion and exclusion
criteria for studies is specific and mentioned?
3)
are studies included with
homogeneous, appropriate and proper study design?
4)
Did it address a clearly
focused issue?
5)
Was there enough information on
population studied, intervention given and outcome considered?
6)
Did the authors do enough to
assess the quality of the included studies?
7)
How the publications bias issue
is addressed (Funnel Plot Test is used usually)?
8)
Is the valid and clear
definition of treatment or exposure is given in a priori fashion?
9)
Is the valid and clear outcome
definition is given in a priori way?
10) Is the results presentation, clear, consistent and precise?
11) Is comprehensive or appropriate literature search strategy with an
aim to include all relevant studies given?
12) Is method of the quality assessment of each study individually with
standardized tools given?
13) Are appropriate mathematical modeling techniques used?
14) How assessment of heterogeneity and attempt to explain it done?
15) Is the appropriate conclusion given?
11. DISCUSSION
Critical appraisal of a research paper or
project is a fundamental skill in modern medical practice for assessing the
worth of clinical research and in providing a guideline of its relevance to the
profession. Its skills to be mastered not only by academic medical
professionals but also by the doctors involved in clinical practice. A
meticulous evaluation of the published or current literature is an important
step in evidence‑based medicine practice, before incorporating changes in
patient’s management.
This review presents an overall, generic
and multi-disciplinary outline to identify the quality of research and how to
evaluate each domain of a research paper
presented. First, all possible and essential sections and subsection of a paper
were identified followed by the important steps and information required in
each section or questions which may arise or needs to addressed were identified
and mentioned. The purpose is to offer a comprehensive, systematic and a broad
outline necessary for an appraisal of a research paper or project. We also argue that this will provide a
comprehensive and a useful approach for discussing research projects and their
quality from different perspectives, and also helpful in advance discussions on
research quality within and across disciplines.
SOURCES
OF FUNDING
This research received no specific grant
from any funding agency in the public, commercial, or not-for-profit sectors.
CONFLICT
OF INTEREST
The author have declared that no competing
interests exist.
ACKNOWLEDGMENT
None.
REFERENCES
[1]
Umesh
G, Karippacheril JG, Magazine R.. Critical appraisal of published literature.
Indian J Anaest. 2016;60( 9): 670-673.
[2]
Burls
A. What is Critical Appraisal? 2nd ed. Oxford, UK:University of Oxford. (What
is…? Series).2009. Available from website:
http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/what_is_critical_appraisal.pdf.
Accessed on June 20, 2020
[3]
Audit
of Submissions: July 2018-June 2019. J Health Serv Res Policy. 2020;25(1):4.
doi: 10.1177/1355819619899554.
[4]
Glasziou
PP. Information overload: What’s behind it, what’s beyond it? Med J Aust
2008;189:84‑85.
[5]
Sadeghi-Bazargani
H, Tabrizi JS, Azami-Aghdash S. Barriers to evidence-based medicine: a
systematic review. J Eval Clin Pract 2014;20(6):793-802. doi: 10.1111/jep.12222
[6]
Banzi
R, Liberati A, Moschetti I, Tagliabue L, Moja L. A review of online
evidence-based practice point-of-care information summary providers. J Med
Internet Res 2010;12(3):e26. doi: 10.2196/jmir.1288
[7]
Kwag
KH, González-Lorenzo M, Banzi R, Bonovas S, Moja L. Providing Doctors with
High-Quality Information: An Updated Evaluation of Web-Based Point-of-Care
Information Summaries. J Med Internet Res 2016;18(1):e15. doi:
10.2196/jmir.5234.
[8]
Lenaerts
G, Bekkering GE, Goossens M, De Coninck L, Delvaux N, Cordyn S, Adriaenssens J,
Vankrunkelsven P. Tools to Assess the Trustworthiness of Evidence-Based
Point-of-Care Information for Health Care Professionals: Systematic Review J
Med Internet Res 2020;22(1):e15415. URL: https://www.jmir.org/2020/1/e15415.
doi: 10.2196/15415.
[9]
Burls
A. What is critical appraisal? London,
Hayward Medical Communications. 2016. Website: [
http://www.whatisseries.co.uk/what-is-critical-appraisal/.] Accessed on June
18, 2020
[10] Hill MD. How to Review a Clinical
Research Paper. Stroke. 2018;49(5): e204-e206.
https://doi.org/10.1161/STROKEAHA.118.021286.
[11] Cruz Rivera S, Kyte DG, Aiyegbusi
OL, Keeley TJ, Calvert MJ. Assessing the impact of healthcare research: A
systematic review of methodological frameworks. PLoS Med. 2017;14(8):e1002370.
https://doi.org/10.1371/journal.pmed.1002370.
[12] Mårtenssona P, Forsb U, Wallinc S,
Zanderd U, Nilssone G H. Evaluating research: A multidisciplinary approach to
assessing research practice and quality. Res Policy;2016: 45 (2016) 593–603.
http://dx.doi.org/10.1016/j.respol.2015.11.009.
[13] Guinea J, Sela E, GoÂmez-Nu ñez
AJ, Mangwende T, Ambali A, Ngum N, et al. Impact oriented monitoring: A new
methodology for monitoring and evaluation of international public health
research projects. Res Eval. 2015; 24(2):131±45.
https://doi.org/10.1093/reseval/rvu034.
[14] Belcher BM, Rasmussen KE, Kemshaw
MR, Zornes DA. Defining and assessing
research quality in a transdisciplinary context. Res Eval. 2015;
25:(2016):1–17.
doi:10.1093/reseval/rvv025.
[15] Schulz KF, Altman DG, Moher D,
CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting
parallel group randomized trials. BMJ 2010;340:698‑702.
[16] Scottish Intercollegiate
Guidelines Network. 2020 website:
[http://www.sign.ac.uk/methodology/checklists.html]. Accessed on 2016 Jun 19.
[17] Strobe Statement. 2020 Available
from: [http://www.strobe‑statement.org/index.php?id=available‑checklists].
Accessed on 2016 Jun 19.
[18] Sorensen HT, Rothman KJ. The
prognosis for research. BMJ. 2010;340:c703. doi: 10.1136/bmj.c703.
[19] Meltzoff J. Critical Thinking
About Research. Washington, DC: Am Psychol Assoc. 2007;23(5):234-243.
[20] Roever L, Resende ES, Diniz ALD,
Penha-Silva N, Biondi-Zoccai G, et al.
Critical Analysis of Clinical Research Articles: A Guide for Evaluation.
Evid Based Med and Pract. 2016; 2(1): e116. Doi: 10.4172/2471-9919.1000e116.
[21] Al-Jundi A, Sakka S. Critical Appraisal of Clinical Research. J
of Clin and Diag Res. 2017;11(5): JE01-JE05. DOI: 10.7860/JCDR/2017/26047.9942.
[22] Al-Jundi A, Sakka S. Protocol
writing in clinical research. J Clin Diagn Res. 2016;10(11):ZE10-ZE13j.
[23] Young JA, Solomon MJ. How to
Critically Appraise an Article. Nat Clin Pract Gastroenterol Hepatol.2009; 6:
82-91.
[24] Gibson CJ. Critical appraisal: a
template to evaluate scientific literature. Dent Update. 2008; 35: 414-417.
[25] MacInnes A, Lamont T. Critical
appraisal of a research paper. Scott Uni Med J. 2014;3 (1):10-17.
[26] Röhrig B, du Prel JB, Blettner M.
Study design in medical research: part 2 of a series on the evaluation of
scientific publications. Dtsch Arztebl Int. 2009;106: 184-189.
[27] Röhrig B, du Prel JB, Wachtlin D,
Kwiecien R, Blettner M. Sample size calculation in clinical trials: part 13 of
a series on evaluation of scientific publications. Dtsch Arztebl Int. 2010;
107: 552-556.
[28] Micallef C. Critical analysis: a
vital element in healthcare research.
Critical analysis: a vital element in healthcare research. Int. J. Behav
and Healthcare Res. 2015;5(1/2):105-110.
[29] Greenhalgh, T. How to read a
paper: The basics of evidence-based medicine. 5th ed, New York, United States;
John Wiley & Sons: 2014. Pp134-167.
[30] Boudin F, Nie J, Bartlett JC.
Combining classifiers for robust PICO element detection. BMC Med Inform Decis
Mak. 2010;10:29. https://doi.org/10.1186/1472-6947-10-29.
[31]
Galdas P. Revisiting
Bias in Qualitative Research: Reflections on Its Relationship with Funding and
Impact. (Editorial). Int J qualitative Methods. 2017: 16:1-2.
https://doi.org/10.1177/1609406917748992
[32] Mackieson P, Shlonsky A, Connolly M. Increasing rigor and reducing
bias in qualitative research: A document analysis of parliamentary debates
using applied thematic analysis. 2018;18:176-180.
https://doi.org/10.1177/1473325018786996
[33] Thomas L. Understanding
confounding variables. Scribber 2020; website:
[https://www.scribbr.com/methodology/confounding-variables/]. Accessed on June
10, 2020
[34] The Belmont Report. 1979 Ethical
Principles and Guidelines for the Protection of Human Subjects of Research The
National Commission for the Protection of Human Subjects of Biomedical.
Retrieved from Website: [https://science.education.nih.gov/supplements/webversions/bioethics/guide/teacher/Mod5_Belmont.pdf]
Accessed on June 11, 2020.
[35] American Psychological Association
(2010, June 1). American Psychological Association Ethical Principles of
Psychologists and Code of Conduct. 2011. Retrieved from [ http://www.apa.org/ethics/code/index.aspx].
Accessed on June 7, 2020
[36] Ethical standards and procedures
for research with human beings. Ethics and Research. WHO Manual (Section
XV.2) 2020. Retrieved from Website:[
https://www.who.int/ethics/research/en/#:~:text=Research%20ethics%20govern%20the%20standards,and%20welfare%20of%20research%20participants.].
Accessed on June 28, 2020
[37] Ensuring your research is ethical:
A guide for Extended Project Qualification students. EPQs ethics guide. Welcome
Trust. 2020. Retrieve from website:[
https://wellcome.ac.uk/sites/default/files/wtp057673_0.pdf]. Accessed on June
28, 2020
[38] du Prel JB, Hommel G, Röhrig B,
Blettner M. Confidence interval or p-value?: part 4 of a series on evaluation
of scientific publications. Dtsch Arztebl Int. 2009;106: 335-339.
[39] du Prel JB, Röhrig B, Hommel G,
Blettner M. Choosing statistical tests: part 12 of a series on evaluation of
scientific publications. Dtsch Arztebl Int. 2010; 107: 343-348
[40] Bialocerkowski A, Klupp N, Bragge
P. How to read and critically appraise a reliability article? Int J Ther Rehab
2010;17:114‑20.
[41] Young J M, Solomon MJ. How to
Critically Appraise an Article. Nat Clin Pract Gastroenterol Hepatol.
2009;6(2):82-91.
[42] Hameed A, Demetres M, Tam D Y,
Rahouma M, Khan F M, Wright D N et al.
An assessment of the quality of current clinical meta-analyses BMC Med
Res Methodol. 2020;20:105-113. https://doi.org/10.1186/s12874-020-00999-9.
[43] Shea BJ, Reeves BC, Wells G, Thuku
M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic
reviews that include randomized or non-randomized studies of healthcare
interventions, or both. BMJ. 2017;358:j4008. https://doi.org/10.
1136/bmj.j4008.
[44] Roever L, Zoccai GB (2015)
Critical Appraisal of Systematic Reviews and Meta-analyses. Evid Based Med and
Prac. 2015;1:1000e106. doi:10.4172/2471-9919.1000e106.
[45] Patel N, Bajaj NS. Meta-analyses:
How to critically appraise them? J of Nuclear Cardio. 2017; 25(5);1598–1600.
doi:10.1007/s12350-017-0898-8
[46] Moher D, Shamseer L, Clarke M,
Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for
systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst
Rev 2015;4:1.
[47] Shamseer L, Moher D, Clarke M,
Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for
systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and
explanation. BMJ 2015;349:g7647.
[48] Hutton B, Salanti G, Caldwell DM,
Chaimani A, Schmid CH, Cameron C, et al. The PRISMA extension statement for
reporting of systematic reviews incorporating network meta-analyses of health
care interventions: Checklist and explanations. Ann Intern Med
2015;162:777-784.
[49] Stroup DF, Berlin JA, Morton SC,
Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies
in epidemiology: A proposal for reporting. Meta-analysis Of Observational
Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008-2012.