Assessment protocol

Wikis > Research > Appraising Research > Assessment protocol

Assessment protocol:

Title:
• Is the title clear and a concise statement of the research and informative?
• Is it relevant?

Abstract:
• Is the abstract clear about the aims, methods, results, conclusions and implications?
• Does the abstract make clear what the study was all about?
• What is the research question being answered?
• Does the study add anything to current knowledge?

Common problems:
• abstract too long
• too many unnecessary words

Introduction:
• Does the introduction provide an adequate context for the study?
• Is the review of the relevant literature used appropriately to provide an argument that is structured leading to the reasons for conducting the research? Does the review justify the aims?
• Is the research problem narrowed down to a researchable problem?
• Why is the problem important
• Are the aims or hypothesis or research question of the study clear?
• At the end of the introduction, do you know what its all about?

Common problems:
• Excessive amount of background information
• Need for study poorly justified
• nothing new
• too long
• insignificant research question
• off little clinical interest
• Lack of a clear statement of the problem
• Inadequate description of the aims/objectives

Methods:
• What type of design is being used?
• Has the design been properly described – are they designed to answer the research question?
• Is it made clear why the particular design was chosen?
• Are sources of error controlled for or acknowledged?
• Is the sample size appropriate?
• How are subjects are selected?
• Who are the subjects?
• Is the sample properly selected and described?
• Who is or is not included and why? – What is the inclusion and exclusion criteria?
• Are there any sources of bias or error in the sample or in the method by which they were selected? – Could this impact on the studies outcome?
• What is evidence for true random selection rather than systematic or volunteer self selection?
• What are the possible sources of bias in the selection of subjects?
• Is any mechanical apparatus used clearly described?
• Are any questionnaires, scales etc that were used adequately described?
• Were questionnaires adequately tested and suitable for the purpose that they were used?
• Is the description of procedures absolutely clear?
• Are the validity and reliability of tests or instruments used mentioned?
• Are instructions given to subjects clearly described?
• Was the data collection method clearly described and appropriate?
• Can this study be replicated based on the description?
• Method of randomisation described? Blindness of subjects?
• Outcomes blindly assessed?
• Is enough detail given so the study can be replicated?

Common problems:
• Insufficient detail or too much unnecessary detail
• poor study design
• biased data collection
• non-random sample
• Inadequate description of procedures
• Inappropriate statistical test (eg parametric test on data not normally distributed)

Results:
• Are graphs and tables clear, self explanatory, clearly labelled and useful?
• Are the statistical tests used the correct ones?
• Does the text of this section state the facts without interpretation?
• Is the p value clearly stated?
• Are confident intervals used?
• Information provided about subjects loss to follow up
• Is there sufficient data provided that the treatment and control group was similar before treatments were applied?
• Is there any information on the reliability of the measurements used?

Common problems:
• Presenting data in an ambiguous manner
• poorly organised
• Presenting insufficient data for reader to make own conclusions
• Results not answering research question
• exact p values not provided
• using standard error of the mean rather than standard deviation for variability of the data
• no statement that the assumption of the statistical test used where met
• confidence intervals not used when appropriate
• interpretive comments made (should be in discussion)

Discussion/Conclusion:
• Any confounding or bias that may have influenced the results?
• Are the results and conclusions clearly stated?
• Are the conclusions logical and valid?
• Are the interpretations and conclusions based on the results?
• Are the results related to other research in the area?
• Are any statements of fact properly referenced from prior research?
• Is the discussion related sufficiently to the introduction and research question?
• Are any assumptions reasonable?
• How generalisable are the results?
• Is there a coherent theoretical explanation for the studies results?
• Are the results interpreted fully/correctly/selectively/extravagantly?
• Are any flaws or shortcomings in the design of the research highlighted?
• Are the practical ramifications of the results discussed?

Common problems:
• Lack conciseness and organisation
• Results may be compared to other publications that are favourable to the authors interpretation
• All of the factors that may explain results may not have been considered
• Too many irrelevant issues discussed
• unexplained inconsistencies
• not acknowledging methodological flaws
• inflated view of the importance of the findings
• unsupported/unfounded conclusions
• over interpretation of data
• Failure to place the results in the context of the relevant literature and the Introduction
• Recommendations for further research not related to outcome of current project
• too long/’wordy’
• uses colloquialisms, cliches and euphemisms

References:
Is each reference quoted listed in the reference section?
Do the references give all the information necessary to check them?

Common problems:
• References missing

Overall:
• Is the write up clear and appropriately structured?
• Is the writing style appropriate?
• Was the research question worth asking in the first place?
• Has an adequate attempt been made to answer the research question?

Comments are closed.