Skip to main content

Scholarly Communication Toolkit: Evaluating Journals

Evaluating Scholarly Journals

Changing publishing models, including the rise of open access journals, have reshaped the ways in which scholars share and use journal articles. The author-pays model of some open access publications did not give rise to predatory or vanity publishing as is often claimed nor are such problems exclusive to open access publishing. Even traditional subscription journals should be carefully analyzed for quality. Librarians should be aware of the various quality indicators and tools that are available for use by their constituents in evaluating potential places of publication.

Determining Journal Quality

When reviewing an open access publisher or journal for quality and legitimatcy -- the following should be considered:

1. Peer review process: All of a journal’s content, apart from any editorial material that is clearly marked as such, shall be subjected to peer review. Peer review is defined as obtaining advice on individual manuscripts from reviewers expert in the field who are not part of the journal’s editorial staff. This process, as well as any policies related to the journal’s peer review procedures, shall be clearly described on the journal’s Web site.

2. Governing Body: Journals shall have editorial boards or other governing bodies whose members are recognized experts in the subject areas included within the journal’s scope. The full names and affiliations of the journal’s editors shall be provided on the journal’s Web site.

3. Editorial team/contact information Journals shall provide the full names and affiliations of the journal’s editors on the journal’s Web site as well as contact information for the editorial office.

4. Author fees: Any fees or charges that are required for manuscript processing and/or publishing materials in the journal shall be clearly stated in a place that is easy for potential authors to find prior to submitting their manuscripts for review or explained to authors before they begin preparing their manuscript for submission.

5. Copyright: Copyright and licensing information shall be clearly described on the journal’s Web site, and licensing terms shall be indicated on all published articles, both HTML and PDFs.

6. Identification of and dealing with allegations of research misconduct: Publishers and editors shall take reasonable steps to identify and prevent the publication of papers where research misconduct has occurred, including plagiarism, citation manipulation, and data falsification/fabrication, among others. In no case shall a journal or its editors encourage such misconduct, or knowingly allow such misconduct to take place. In the event that a journal’s publisher or editors are made aware of any allegation of research misconduct relating to a published article in their journal – the publisher or editor shall follow COPE’s guidelines (or equivalent) in dealing with allegations.

7. Ownership and management: Information about the ownership and/or management of a journal shall be clearly indicated on the journal’s Web site. Publishers shall not use organizational names that would mislead potential authors and editors about the nature of the journal’s owner.

8. Web site: A journal’s Web site, including the text that it contains, shall demonstrate that care has been taken to ensure high ethical and professional standards.

9. Name of journal: The Journal name shall be unique and not be one that is easily confused with another journal or that might mislead potential authors and readers about the Journal’s origin or association with other journals.

10. Conflicts of interest: A journal shall have clear policies on handling potential conflicts of interest of editors, authors, and reviewers and the policies should be clearly stated.

11. Access: The way(s) in which the journal and individual articles are available to readers and whether there are associated subscription or pay per view fees shall be stated.

12. Revenue sources: Business models or revenue sources (eg, author fees, subscriptions, advertising, reprints, institutional support, and organizational support) shall be clearly stated or otherwise evident on the journal’s Web site.

13. Advertising: Journals shall state their advertising policy if relevant, including what types of ads will be considered, who makes decisions regarding accepting ads and whether they are linked to content or reader behavior (online only) or are displayed at random.

14. Publishing schedule: The periodicity at which a journal publishes shall be clearly indicated.

15. Archiving: A journal’s plan for electronic backup and preservation of access to the journal content (for example, access to main articles via CLOCKSS or PubMedCentral) in the event a journal is no longer published shall be clearly indicated.

16. Direct marketing: Any direct marketing activities, including solicitation of manuscripts that are conducted on behalf of the journal, shall be appropriate, well targeted, and unobtrusive.

From Principles of Transparency and Best Practices in Scholarly Publishing.

Think. Check. Submit.

think check submit logo A coalition of scholarly publishers and associations collaborated to create this short checklist for authors to refer to when evaluating a journal as a possible place of publication for his research. By asking a few short questions and evaluating the journal according to the checklist, authors can be assured that the journal they are considering, whether subscription based or open access, will be one of quality, rigor, and respect.

 

ThinkAsk yourself, can you trust this journal with your research? Does the journal publish research you would read yourself?

Check: Is the organization or publisher of the journal identifiable? Can you contact them easily? 

For journals with publication fees (color charges, Open Access) - are the fees clearly listed on the publisher's website? Reputable publishers should list their fees clearly and publicly. 

Do you know the names or reputations of any of the editorial board members? 

Are the articles indexed in services you use within your subject area?

Submit: If you can answer yes to these questions, then submit!

 

Additional Resources for Evaluating Journal Quality

  • Open Access Journal Quality Indicators - from Grand Valley State University, these guidelines can help one evaluate open access publications as one considers appropriate publication venues, or invitations to serve as reviewers or editors. See also this article published in the Journal of Librarianship and Scholarly Communication about the development of these guidelines.
  • Journal Evaluation Tool - from Loyola Marymount University, created by Shilpa Rele, Marie Kennedy, and Nataly Blas. This PDF download from the LMU institutional repository includes steps for evaluating journals, a detailed rubric of criteria, and a scoring table to accompany the rubric.
  • JournalGuide - JournalGuide uses a “whitelist” approach, indexing known journals of quality and providing information on scope, how quickly the journal reviews and publishes papers, where the journal is indexed, open access options, page charges and more. JournalGuide also provides links directly to a journals “instructions for authors” page.
  • Directory of Open Access Journalsa community-curated list of open access journals and a good starting point for identifying quality, peer reviewed open access material. DOAJ also provides a "Best Practices" for publishers, which can be useful when evaluating journals for quality.
  • Harzing's Journal Quality List - this collated list of quality journals is based upon nearly 20 different rankings of more than 900 journals
  • "Dear Esteemed Author:" Spotting a Predatory Publisher in 10 Easy Steps - blog post from the Society of Teachers of Family Medicine that describes 10 things to consider when evaluating a scholarly journal

Open Access Spectrum & Evaluation Tool

The Open Access Spectrum was developed by SPARC in collaboration with PLOS and OASPA. This tool displays the core components of open access across a spectrum indicating how closed or open a journal is based upon its policies on reader rights, reuse rights, copyrights, author posting rights, automatic posting, and machine readability. The How Open Is It? guide can be downloaded and shared with faculty and students who have questions about how to evaluate open access journals. The Open Access Spectrum Evaluation Tool can also be used to filter and locate journals based upon publisher policies on these key components.