Systematic reviews and meta-analyses, which involve thorough literature searches and synthesis, are essential for evidence-based decision-making. With the growing number of publications, thorough review takes more time. A systematic review often takes more than 15 months to complete, and the long conception-completion interval may render a systematic review outdated by the time they are ready to be submitted and published.1,2

Artificial intelligence (AI)-based quick approaches are being developed in response to the growing demand for screening and data extraction techniques that are more effective. These techniques automate decision-making by utilising machine learning (ML) algorithms and natural language processing (NLP) to extract pertinent information from textual material. These AI-based methods have the potential to accelerate the processes of systematic reviews and evidence synthesis while requiring less human labour.3,4

ChatGPT is a Large Language Model (LLM) developed by OpenAI (OpenAI, L.L.C., San Francisco, CA, USA) which has tremendous potential to revolutionise the healthcare industry. ChatGPT has been effectively used for a variety of tasks in healthcare, including conducting literature reviews, analysing datasets, writing academic papers, documentation, and enhancing clinical workflow. ChatGPT is a significant advancement in the field of NLP. It generates human-like text by mimicking human language processing abilities using deep learning techniques and neural networks. Owing to its training on diverse datasets, it is able to provide pertinent responses to human prompts by processing large amounts of information. The easy accessibility and free availability have contributed to the success ChatGPT version 3.5 platform.5

We tried to use ChatGPT for conducting Risk of Bias analysis and data extraction from a randomised controlled trial. The response of ChatGPT was surprising, to say the least. Conducting such processes through ChatGPT is simple and straightforward.

Firstly, to conduct a ROB analysis effectively using AI, obtaining the full-text version of the article under evaluation is crucial. This ensures that all relevant information and details necessary for the assessment are available. As an AI tool, it does not have real-time browsing capabilities, so it cannot search the internet or access the latest research articles. Additionally, it being a text-based AI, we cannot upload or import text/pdf files into the chatbox. However, ChatGPT can still try to assist us with general questions or provide guidance based on the information we provide from the study. If we have any specific questions or excerpts from the study that we would like ChatGPT to analyse or discuss, we can paste them into the chat box, and ChatGPT will do its best to assist.

AI provides validation to the information obtained through human effort and may even catch some details that are missed by the human eye.

Secondly, we need to provide clear and specific instructions to the tool for it to provide an appropriate response. Hence when prompted 'Can you use the ROBINS-I tool to evaluate "(Materials and methods from the full-text article)"', it generated a detailed response highlighting the various lacunae in the methodology in context of the Biases included in the ROBINS-I tool (Fig. 1 and Fig. 2). This assessment does not eliminate the need for a human expert; however, it considerably reduces the time and effort required for the process. Also, AI provides validation to the information obtained through human effort and may even catch some details that are missed by the human eye. Additionally, ChatGPT is extremely helpful in the interpretation of studies for those who are not so fluent in the language. While ChatGPT can comprehend and generate text in languages other than English to some extent, its proficiency may vary across different languages. It's worth noting that ChatGPT's performance is generally more robust in English, as the majority of its training data is in English.

Fig. 1
figure 1

Screenshot of query for RoB analyis

Full size image
Fig. 2
figure 2

Screenshot of ChatGPT's response to RoB analysis

Full size image

Next, we tried to use ChatGPT for data extraction. By sharing the complete text and specifying the desired information or data to be extracted, AI can be of assistance. Hence, following various prompts from us, ChatGPT extracted data like mean values, standard deviation and sample size for study groups and sub-groups and presented it. Also, it was able to extract information from the tables provided, though it is not possible to paste a table in the chatbox. We can paste the data in the table en masse into the chatbox and it was able to interpret and extract information from it (Fig. 3 and Fig. 4). Needless to say, this will reduce the time required for data extraction as well as ensure accuracy of the values, thereby reducing human errors.

Fig. 3
figure 3

Screenshot of query for data extraction from text

Full size image
Fig. 4
figure 4

Response from ChatGPT for various queries for data extraction

Full size image

AI methods are increasingly integrated into healthcare reviews to streamline processes. A recent systematic review by Blaizot et al. identified and examined automated tools and platforms that utilise artificial intelligence (AI) techniques. It assessed the pros and cons associated with the use of these methods in conducting systematic reviews. They identified various tools like Rayyan, Robot Reviewer, EPPI-reviewer, SWIFT-review, and Abstrackr which were employed for different review stages. Most included reviews acknowledged that screening with AI methods generally demonstrates low risks and enhances efficiency. However, possibility of missing relevant articles and issues in data extraction required human intervention.4

ChatGPT possesses several distinct advantages in comparison to other AI tools. Its ability to retain context and coherence during conversations equips it to comprehend and respond to complex queries. It is extremely versatile which is demonstrated through its proficiency in handling various tasks, including factual inquiries, explanatory responses, creative content generation, and problem-solving assistance. ChatGPT is continuously evolving through regular updates and fine-tuning based on user feedback and new training data, enhancing its performance and accuracy over time.5

It is crucial to acknowledge that ChatGPT also has limitations. These include the potential for generating incorrect or biased responses and its reliance on pre-existing data. Users should exercise critical thinking and independently verify information obtained from any AI tool. Currently, it is unable to process visual information; however, this issue has been addressed in ChatGPT 4.0. Additionally, the model's output heavily relies on user input; minor changes in the query can lead to significant alterations in the response.5

This was the first attempt to test the feasibility of using ChatGPT in the tedious task of conducting systematic reviews and meta-analyses. It shows promise in reducing workload and time, but careful implementation and validation are necessary. We as researchers need to interact more with this tool to completely understand its applicability in generating evidence.