Nearly a year ago, I started using OpenAI and ChatGPT, both in my work as a physician and researcher, and also outside of work. The first time I used an AI interface on my home computer, my jaw hit the floor. I was blown away with the power of the tool, but also a little disturbed by some aspects of it. OpenAI was able to answer some interesting medical questions I asked it. But it also lied to me, as I wrote
Fascinating article, and wondered "could AI pass oral boards too? Or just written?" but had a longer question/idea I decided to email you instead at insidemedicine@substack.com - not sure if you check that much but give it a look (very limited) time permitting.
At least in my field, ChatGPT’s penchant for just making things up rendered old versions fatally flawed. But I wonder if its improvement — but not to 100% — is actually more problematic, because the fake papers and incorrect info it supplies will be harder to pick out of the mostly correct responses. I haven’t found it to be all that helpful anyway — its understanding of most concepts I care about seems to be on the level of introductory undergraduate coursework — but regardless I will keep trying to steer colleagues away from it for literature searches until it is much more reliable. That goes double for inexperienced ones who might not be prepared to sift the fake from the real.
It is helpful for coding though. It rarely provides actual working code, but can provide good guidance to get a project started or identify a particular library that’s useful that I might not have known about.
Thanks, Dan. Great point on the paradox of "the better it gets, the more easily we might get fooled." It's true that I'll catch mistakes it makes. The same tool in less experienced hands could really screw some stuff up.
Thank you for the Case Report Doctor but, my ER Doc ... fyi, she is faster than AI.
:) I still have job security...but I enjoy the extra help!
Nice, thank you.
You always keep us informed and on our toes. Considering I have to call my “tech guy”(17 year old son) all the time, AI has all the feels for me😂
Fascinating article, and wondered "could AI pass oral boards too? Or just written?" but had a longer question/idea I decided to email you instead at insidemedicine@substack.com - not sure if you check that much but give it a look (very limited) time permitting.
Replied:)
Very interesting read.
At least in my field, ChatGPT’s penchant for just making things up rendered old versions fatally flawed. But I wonder if its improvement — but not to 100% — is actually more problematic, because the fake papers and incorrect info it supplies will be harder to pick out of the mostly correct responses. I haven’t found it to be all that helpful anyway — its understanding of most concepts I care about seems to be on the level of introductory undergraduate coursework — but regardless I will keep trying to steer colleagues away from it for literature searches until it is much more reliable. That goes double for inexperienced ones who might not be prepared to sift the fake from the real.
It is helpful for coding though. It rarely provides actual working code, but can provide good guidance to get a project started or identify a particular library that’s useful that I might not have known about.
Thanks, Dan. Great point on the paradox of "the better it gets, the more easily we might get fooled." It's true that I'll catch mistakes it makes. The same tool in less experienced hands could really screw some stuff up.