{"id":57450,"date":"2023-05-22T16:03:06","date_gmt":"2023-05-22T14:03:06","guid":{"rendered":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/"},"modified":"2023-05-22T16:03:06","modified_gmt":"2023-05-22T14:03:06","slug":"chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report","status":"publish","type":"post","link":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/","title":{"rendered":"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report"},"content":{"rendered":"
\n

\nResearchers asked the artificial intelligence tool to take 2021 and 2022 multiple-choice self-assessment tests; results were published in the American Journal of Gastroenterology<\/i><\/p>\n

MANHASSET, N.Y.–(BUSINESS WIRE)–While the popular artificial intelligence (AI) ChatGPT is seen as a potential educational tool, it won\u2019t be getting its medical specialty certification anytime soon. To test its abilities and accuracy, investigators at The Feinstein Institutes for Medical Research<\/a> asked the consumer-facing ChatGPT (Chat Generative Pre-trained Transformer, OpenAI) to take the 2021 and 2022 multiple-choice self-assessment tests for the American College of Gastroenterology. ChatGPT failed to make the grade, scoring 65.1 percent and 62.4 percent compared to the required 70 percent to pass the exams. Full details of the study were published today in the American Journal of Gastroenterology<\/i><\/a>.<\/i><\/p>\n

<\/a>
<\/a><\/p>\n

\nChatGPT is a 175-billion-parameter natural language processing model that generates human-like text in response to user prompts. The tool is a large language model (LLM) trained to predict word sequences based on context. ChatGPT has been tested before, even passing the United States Medical Licensing Exam. In this study, the Feinstein Institutes\u2019 researchers wanted to challenge ChatGPT\u2019s (versions 3 and 4) ability to pass the ACG assessment, which is supposed to gauge how one would fare on the actual American Board of Internal Medicine (ABIM) Gastroenterology board examination.<\/p>\n

\n\u201cRecently, there has been a lot of attention on ChatGPT and the use of AI across various industries. When it comes to medical education, there is a lack of research around this potential ground-breaking tool,\u201d said Arvind Trindade, MD,<\/a> associate professor at the Feinstein Institutes\u2019 Institute of Heath System Science<\/a> and senior author on the paper. \u201cBased on our research, ChatGPT should not be used for medical education in gastroenterology at this time and has a ways to go before it should be implemented into the health care field.\u201d<\/p>\n

\nEach ACG test consists of 300 multiple-choice questions with real-time feedback. Each question and answer was copied and pasted directly into the ChatGPT versions 3 and 4. Overall, ChatGPT answered 455 questions (145 questions were excluded because of an image requirement). Chat GPT-3 answered 296 of 455 questions correctly (65.1 percent) across the two exams, and Chat GPT-4 answered 284 questions correctly (62.4 percent).<\/p>\n

\n\u201cChatGPT has sparked enthusiasm, but with that enthusiasm comes skepticism around the accuracy and validity of AI\u2019s current role in health care and education,\u201d Andrew C. Yacht, MD,<\/a> senior vice president, academic affairs and chief academic officer at Northwell Health. \u201cDr. Trindade\u2019s fascinating study is a reminder that, at least for now, nothing beats hitting time-tested resources like books, journals and traditional studying to pass those all-important medical exams.\u201d<\/p>\n

\nChatGPT does not have any intrinsic understanding of a topic or issue. Potential explanations for ChatGPT\u2019s failing grade could be the lack of access to paid subscription medical journals or ChatGPT\u2019s sourcing of questionable outdated or non-medical sources, with more research needed before it is used reliably.<\/p>\n

\nAbout the Feinstein Institutes<\/i><\/b><\/p>\n

\nThe Feinstein Institutes for Medical Research<\/i><\/a> is the home of the research institutes <\/i>of Northwell Health, the largest health care provider and private employer in New York State.<\/i> Encompassing <\/i>50 research labs, 3,000 clinical research studies and 5,000 researchers and staff, the Feinstein Institutes raises the standard of medical innovation through its five institutes of behavioral science, bioelectronic medicine, cancer, health system science, and molecular medicine. We make breakthroughs in genetics, oncology, brain research, mental health, autoimmunity, and are the global scientific leader in bioelectronic medicine \u2013 a new field of science that has the potential to revolutionize medicine. For more information about how we produce knowledge to cure disease, visit<\/i> <\/i>http:\/\/feinstein.northwell.edu<\/i><\/a> <\/i>and follow us on<\/i> <\/i>LinkedIn<\/i><\/a>.<\/i><\/p>\n

Contacts<\/b> <\/p>\n

\nMatthew Libassi
\n
631-793-5325
\n
mlibassi@northwell.edu<\/a><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Researchers asked the artificial intelligence tool to take 2021 and 2022 multiple-choice self-assessment tests; results were published in the American Journal of Gastroenterology MANHASSET, N.Y.–(BUSINESS WIRE)–While the popular artificial intelligence (AI) ChatGPT is seen as a potential educational tool, it won\u2019t be getting its medical specialty certification anytime soon. To test its abilities and accuracy, … [Read more…]<\/a><\/span><\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13],"tags":[],"class_list":{"0":"entry","1":"post","2":"publish","3":"author-business","4":"post-57450","6":"format-standard","7":"category-industry"},"yoast_head":"\nChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report - Pharma Trend<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report - Pharma Trend\" \/>\n<meta property=\"og:description\" content=\"Researchers asked the artificial intelligence tool to take 2021 and 2022 multiple-choice self-assessment tests; results were published in the American Journal of Gastroenterology MANHASSET, N.Y.–(BUSINESS WIRE)–While the popular artificial intelligence (AI) ChatGPT is seen as a potential educational tool, it won\u2019t be getting its medical specialty certification anytime soon. To test its abilities and accuracy, ... [Read more...]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/\" \/>\n<meta property=\"og:site_name\" content=\"Pharma Trend\" \/>\n<meta property=\"article:published_time\" content=\"2023-05-22T14:03:06+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mms.businesswire.com\/media\/20230522005470\/en\/1799470\/21\/Trindade_GPT.jpg\" \/>\n<meta name=\"author\" content=\"Business Wire\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Business Wire\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/\"},\"author\":{\"name\":\"Business Wire\",\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/#\\\/schema\\\/person\\\/02d41342c7a74fa7f0032bb35ef0bb24\"},\"headline\":\"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report\",\"datePublished\":\"2023-05-22T14:03:06+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/\"},\"wordCount\":609,\"publisher\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mms.businesswire.com\\\/media\\\/20230522005470\\\/en\\\/1799470\\\/21\\\/Trindade_GPT.jpg\",\"articleSection\":[\"Industry\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/\",\"url\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/\",\"name\":\"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report - Pharma Trend\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mms.businesswire.com\\\/media\\\/20230522005470\\\/en\\\/1799470\\\/21\\\/Trindade_GPT.jpg\",\"datePublished\":\"2023-05-22T14:03:06+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mms.businesswire.com\\\/media\\\/20230522005470\\\/en\\\/1799470\\\/21\\\/Trindade_GPT.jpg\",\"contentUrl\":\"https:\\\/\\\/mms.businesswire.com\\\/media\\\/20230522005470\\\/en\\\/1799470\\\/21\\\/Trindade_GPT.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/\",\"name\":\"Pharma Trend\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/#organization\",\"name\":\"Pharma Trend\",\"url\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"\",\"contentUrl\":\"\",\"caption\":\"Pharma Trend\"},\"image\":{\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/pharma-trend.com\\\/en\\\/#\\\/schema\\\/person\\\/02d41342c7a74fa7f0032bb35ef0bb24\",\"name\":\"Business Wire\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report - Pharma Trend","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/","og_locale":"en_US","og_type":"article","og_title":"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report - Pharma Trend","og_description":"Researchers asked the artificial intelligence tool to take 2021 and 2022 multiple-choice self-assessment tests; results were published in the American Journal of Gastroenterology MANHASSET, N.Y.–(BUSINESS WIRE)–While the popular artificial intelligence (AI) ChatGPT is seen as a potential educational tool, it won\u2019t be getting its medical specialty certification anytime soon. To test its abilities and accuracy, ... [Read more...]","og_url":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/","og_site_name":"Pharma Trend","article_published_time":"2023-05-22T14:03:06+00:00","og_image":[{"url":"https:\/\/mms.businesswire.com\/media\/20230522005470\/en\/1799470\/21\/Trindade_GPT.jpg","type":"","width":"","height":""}],"author":"Business Wire","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Business Wire","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/#article","isPartOf":{"@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/"},"author":{"name":"Business Wire","@id":"https:\/\/pharma-trend.com\/en\/#\/schema\/person\/02d41342c7a74fa7f0032bb35ef0bb24"},"headline":"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report","datePublished":"2023-05-22T14:03:06+00:00","mainEntityOfPage":{"@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/"},"wordCount":609,"publisher":{"@id":"https:\/\/pharma-trend.com\/en\/#organization"},"image":{"@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/#primaryimage"},"thumbnailUrl":"https:\/\/mms.businesswire.com\/media\/20230522005470\/en\/1799470\/21\/Trindade_GPT.jpg","articleSection":["Industry"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/","url":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/","name":"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report - Pharma Trend","isPartOf":{"@id":"https:\/\/pharma-trend.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/#primaryimage"},"image":{"@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/#primaryimage"},"thumbnailUrl":"https:\/\/mms.businesswire.com\/media\/20230522005470\/en\/1799470\/21\/Trindade_GPT.jpg","datePublished":"2023-05-22T14:03:06+00:00","breadcrumb":{"@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/#primaryimage","url":"https:\/\/mms.businesswire.com\/media\/20230522005470\/en\/1799470\/21\/Trindade_GPT.jpg","contentUrl":"https:\/\/mms.businesswire.com\/media\/20230522005470\/en\/1799470\/21\/Trindade_GPT.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/pharma-trend.com\/en\/chatgpt-flunks-american-college-of-gastroenterology-exams-feinstein-institutes-report\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/pharma-trend.com\/en\/"},{"@type":"ListItem","position":2,"name":"ChatGPT flunks American College of Gastroenterology exams, Feinstein Institutes report"}]},{"@type":"WebSite","@id":"https:\/\/pharma-trend.com\/en\/#website","url":"https:\/\/pharma-trend.com\/en\/","name":"Pharma Trend","description":"","publisher":{"@id":"https:\/\/pharma-trend.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/pharma-trend.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/pharma-trend.com\/en\/#organization","name":"Pharma Trend","url":"https:\/\/pharma-trend.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/pharma-trend.com\/en\/#\/schema\/logo\/image\/","url":"","contentUrl":"","caption":"Pharma Trend"},"image":{"@id":"https:\/\/pharma-trend.com\/en\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/pharma-trend.com\/en\/#\/schema\/person\/02d41342c7a74fa7f0032bb35ef0bb24","name":"Business Wire"}]}},"_links":{"self":[{"href":"https:\/\/pharma-trend.com\/en\/wp-json\/wp\/v2\/posts\/57450","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pharma-trend.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/pharma-trend.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/pharma-trend.com\/en\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/pharma-trend.com\/en\/wp-json\/wp\/v2\/comments?post=57450"}],"version-history":[{"count":0,"href":"https:\/\/pharma-trend.com\/en\/wp-json\/wp\/v2\/posts\/57450\/revisions"}],"wp:attachment":[{"href":"https:\/\/pharma-trend.com\/en\/wp-json\/wp\/v2\/media?parent=57450"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/pharma-trend.com\/en\/wp-json\/wp\/v2\/categories?post=57450"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/pharma-trend.com\/en\/wp-json\/wp\/v2\/tags?post=57450"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}