ICrESAI - IMeCI: Tools for Selecting and Evaluating Scientific Articles for Research and Evidence-based Practice

Abstract

Objective: The increase in the production of scientific knowledge is of concern to scientists, who must decide daily on selecting literature of utmost scientific merit to support decision- making on research and practice The purpose of this study is to construct two registered instruments5 with psychometric properties, face validity and content validity. Materials and Methods: The Research Article Eligibility Criteria Instrument (ICrESAI) proposes standards to be assessed to determine which research papers should be eligible for a systematic review (SR) or meta-analysis (MA). Inter-reevaluator reliability was calculated as well. The second instrument involves Criteria for Evaluating the Scientific Merit of Articles Selected for Meta-analysis (IMeCI) , which proposes standards to be evaluated to rate the scientific merit and methodological quality of the articles selected for a SR. The estimates for face and content validity were made with three researchers who are experts in research methodology and measurement. Content validity was calculated using the Modified Lawshe Index (Tristán -López, 2008) and inter-codifier reliability was measured for qualitative variables using Cohen’s Kappa Index. Results: The ICrESAI instrument obtained a content validity index (CVI) of 1.0 . The CVI for the IMeCI instrument was 0.75. With the ICrESAI instru- ment, the values for the inter-codifier reliability or concordance index were between 0.547 and 0.973, which is considered good according to the interpretation proposed by Cohen. Conclusions: Both instruments are valid (CVI and Lawshe) for selecting and evaluating the scientific merit of research articles. These two tools are useful to the study of any subject area and to guide the use of information for decision-making in research and evidence-based practice.