Rayyan.ai is web-based software that makes it easy to conduct systematic reviews, and other types of reviews, by yourself or with a team of collaborators. The base version of Rayyan is free to use. Premium features, such as the ability to automatically generate PRISMA diagrams, are included with a premium subscription. Free accounts can have up to three active reviews at a time; premium accounts have unlimited active reviews. Rayyan is browser-based online software. Before using Rayyan, check your browser compatibility here: https://rayyan.ai/check
To start a systematic review, log in and click New Review on the home page. Enter a title, research field, review domain, and optional description. Select Systematic Review as the review type and click the Create button.
When you create a new review, Rayyan will prompt you to upload references. You can import references directly from Mendeley through their interface. For other databases and citation managers, Rayyan has several migration guides with examples linked. To import your references, first export them from your citation manager in a supported file format (e.g. BibTeX). Choose the Select Files… button and navigate to the file you just exported. Upload this file to Rayyan.
Once you have uploaded your references, they will appear in the middle pane of your review interface. Clicking on a reference shows the details and abstract for that reference. The left panel will populate with keywords from the references that you just uploaded. You can also filter by decision, label, reason, search method, and topic from this pane.
Rayyan allows researchers to add an unlimited number of reviewers to an active review. To add a reviewer, click All Reviews, choose your active review, and click Invite. Enter your collaborators’ email addresses and a message.
Collaborators will need their own Rayyan account to participate in your review. Once they log in, the reviews that they have been invited to appear under their Collaboration Reviews tab on the home page.
Clicking the Detect Duplicates button at the top will scan the uploaded references for duplicates.
A facet titled Possible Duplicates will show on the left pane. Clicking on a set of possible duplicates will bring up more detail in the bottom pane.
After reviewing the set of possible duplicates, you can choose to delete the duplicate(s) or mark them as not duplicates.
Premium accounts have a feature called Auto Deduplication, which automatically detects and deletes 100% matches. These deleted matches are still available for users to see under the Auto Deduplicated tab. Possible duplicates that are not a 100% match will still need to be screened by a reviewer.
Blinding is turned on by default in collaborative reviews. Collaborators will not be able to see each others’ inclusion/exclusion decisions until blinding is turned off. This can be done on the home page by clicking on the active review. Click the Blind On/Off button to toggle this option. Only the review owner can change the blind options.
Under My Reviews, click Show under your active reviews to start screening. You can view references individually or select multiple references at the same time using the shift key and the up/down arrows. To include references, select the reference(s) and click Include or press I on the keyboard. To exclude, click Exclude or press E. If you’re not sure whether to include or exclude this reference, click Maybe or press the M or U keys. For a full list of keyboard shortcuts in Rayyan, see this link: https://help.rayyan.ai/hc/en-us/articles/4414328743953-Shortcut-Keys
You and your collaborators can add reason and label tags to your references so that you can remember and share your reasons for inclusion or exclusion of a particular article. The Reason dropdown menu comes pre-populated with common reasons for inclusion/exclusion. To add a custom reason, type your reason in the Reason box and press enter.
Labels can be added in the same way, by typing into the Label box.
After making 50 decisions, the Compute Ratings option becomes available.
Rayyan will rate the likelihood of inclusion for the remaining references based on the decisions you have made so far. This is displayed next to the reference name on a one-to-five-star scale. You can recompute ratings after making more decisions. The more decisions made, the more accurate the ratings should become.
The review chat allows you to leave messages for your collaborators. You do not have to be online at the same time as your collaborators to leave messages in the chat. Your messages will be visible the next time your collaborators login and view this review.
Any criterion that populates in the left pane can be used to filter your references. This includes filtering by year, author, journal, type of reference, topic, labels, keywords for inclusion or exclusion, duplicate status, and inclusion.
Selecting multiple filters from the same facet (e.g. “exclusion reasons”) is treated as an “or”– selecting “nonrandomized” and “observational” will include references that include either of those labels.
Selecting multiple filters from difference facets (e.g. “exclusion reasons” and “labels”) is treated as an “and”– selecting “nonrandomized” and “2018” will filter for references that include both of those criteria.
The “at most” filter can be used to display references based on how many collaborators have screened the reference. Choose “at most 0” to see references that have not been screened by any reviewers.
Once you have finished screening and want to compare the decisions, turn the blind off by clicking the Blind On/Off button at the top of the My Reviews tab on the home page. To resolve conflicts with your collaborators, filter by “conflict” (under the Inclusion Decisions facet) to see references where there is disagreement about whether to include or exclude that reference. Once you have reached consensus, one of the collaborators will need to change their decision. After refreshing the page, resolved conflicts will no longer show under the “conflict” filter.
Once you have finished title or abstract reviews and are ready to move on to full-text reviews, you can continue with your current review or make a new review. Making a new review is often easier, as you will not need to filter out the excluded abstracts.
To start a new full-text review, make a new review from the home page. Return to your abstract review, set the filter to “included”, and click the Copy button on the top right of the page.
A new box will appear. Choose “filtered” from the drop-down menu and copy the results to the (currently empty) full-text review. Re-invite collaborators to the new full-text review.
Select your reference(s) and choose “upload PDF full texts”. Choose the PDF files on your disk and map the PDFs to the correct reference. The full text PDFs will now be associated with the references that you chose to include earlier.
Once finished with your full-text review, you can choose to export the references. The export link will be sent to you via email. This link will contain the included articles and a .csv log file which lists the actions taken during the review.
Rayyan has a help chatbot and human support available for questions. Their Help and FAQ pages can be found here:
Help Center: https://help.rayyan.ai/hc/en-us
FAQ Page: https://help.rayyan.ai/hc/en-us/categories/360003671417-FAQ
