Last month, CRIO teamed up with Clinical Leader for a live panel, “The Site AI Playbook: Vendor Tools, Homegrown Solutions – What’s Working.” The conversation brought together site operators who are actively using AI in their day-to-day operations for an honest look at what’s actually working, moderated by CRIO Chief Innovation Officer Mike Wenger. Joining him were Aneesh Vaze, Managing Director of Clinical Research Philadelphia, Sam Stein, CEO of ALSA Research, and Nick Spittal, COO of Velocity Clinical Research.
The session generated far more questions than we could answer live, so we followed up with attendees individually. A few themes came up again and again, so we’re sharing generalized versions here for anyone who registered or watched the recording. If you’d like the full conversation, you can access the on-demand webinar here.
Should we build our own AI tools, or buy from a vendor?
This was probably the most common question of the day, and it’s really a “where are you in your journey” question rather than a “which is better” one. The panel described AI showing up at sites in three ways: established platform vendors (like CRIO) building AI into the tools you already use, point-solution vendors solving one specific problem (recruitment, feasibility, scheduling), and sites building their own tools on top of their existing systems.
The honest answer is that all three have a place. If you’re early in your AI journey and don’t yet have the data infrastructure or in-house comfort level to build, leaning on vendor-provided tools is the natural, lower-risk starting point. Homegrown solutions tend to make more sense once you have clean, accessible data and a team that’s already comfortable experimenting with AI day-to-day.
For sites that are still fully paper-based and haven’t adopted eSource or eReg, the realistic path looks something like this: move to an electronic eSource/eReg system first, put a basic AI governance policy in place, give staff enterprise LLM accounts so they can experiment safely, build small internal tools around your most painful bottlenecks, and only then explore backend data access and more sophisticated workflows. Skipping straight to “build it ourselves” before your data is organized tends to create more work than it saves.
How do you actually connect AI tools to your existing data?
Several attendees asked variations of “we have an eSource platform, so how do we get AI working with that data?” For CRIO customers specifically, backend access is available through Google BigQuery and Looker, which opens the door to custom dashboards, financial forecasting, and business intelligence layered directly on top of your operational data, without needing to export or re-enter anything.
Connecting a platform like Claude to that data does take a bit of setup. The main recommendation from the panel: use an enterprise account rather than a personal license, since enterprise accounts come with data-training protections that matter for compliance. If you’re not sure where to start, your platform vendor’s customer success or support team should be able to walk you through the connection and any permissions involved.
What tools are people actually using for recruitment and other point solutions?
On the recruitment side, several attendees asked about AI-powered outreach (calls, texting, and similar) and whether anyone had wired AI agents directly into their patient data. The consistent theme from the panel was that most sites aren’t building recruitment AI from scratch. They’re plugging in point solutions through their existing vendor ecosystems for patient acquisition, retention, and pre-screening, then layering their own review and follow-up on top.
As for which LLMs people are actually using day to day: Claude came up most often, though a few panelists also use other major frontier models, Gemini, and NotebookLM depending on the task. There wasn’t a single “right” platform. The more useful takeaway was how people are using them: for things like drafting and updating investigator CVs, an enterprise LLM account, a clear description of the current manual process, and iterative back-and-forth with the tool, starting rough and refining, produced faster results than trying to automate the whole thing perfectly on the first attempt. A human stayed in the loop reviewing outputs the whole way through.
Where does AI still fall short?
Not every use case is a good fit yet, and the panel was candid about that. AI tends to struggle most with unstructured, high-variability information, things like inclusion/exclusion criteria or prohibited medications, where the details differ from study to study and don’t follow a consistent pattern. Feed that kind of inconsistency into a model and you’re more likely to get incorrect interpretations than useful output.
The other pattern the panel flagged: automations that try to do too much in one step tend to underperform. When a single workflow has to take in many different kinds of input (documents, chat logs, system data) and produce many different kinds of output (emails, reports, dashboards), it’s harder to trust and harder to fix when something goes wrong. Narrower, well-scoped use cases have generally been easier to get right.
Keep exploring
If you want a deeper dive into making your own site’s data AI-ready, we’d point you to this recent CRIO blog post. And if you’re evaluating AI-enabled recruitment, eSource, or other point solutions, our Partners page is a good place to see what’s already integrated with CRIO, including several of the tools referenced during the panel.
Thanks again to everyone who joined us live and to those of you catching up now. If you have a question we didn’t cover here, feel free to reach out to your CRIO customer success manager directly.