Skip to content
6 Minute Read

How the World’s Leading Vaccine Site Network Decreased Protocol Deviations by Almost 40%


With 9 locations in Texas, California and Louisiana, Benchmark Research is the world’s leading vaccine network. Benchmark has performed over 10,000 studies on over 40,000 subjects and has received numerous awards, including being a five-time recipient of the Best Clinical Trial Site/Network award from the World Vaccine Congress.


Historically, Benchmark used paper charts to collect source data. In 2019, the management team concluded that they should move to electronic source but had not set a date for transition. When the pandemic hit in spring 2020, this desire became an urgency.


Benchmark put together a team of end users to review all the competitive options. The team selected CRIO for its functionality and ease of use. The network rolled out CRIO on a couple pilot studies, then determined that all new studies would occur on the CRIO eSource system.

“The biggest obstacle was fear. You had coordinators who had been collecting source data on paper for 20 years,” explained Deborah Copeland, the network’s Chief Operating Officer. “Fortunately, CRIO is very intuitive, so once the coordinators got used to it, they didn’t want to go back.We were wasting hours scanning and uploading documents for remote monitoring. It was untenable and slowed us down. We needed to move to eSource.”


The Benefits of eSource

The network quickly realized several benefits from the transition. From an efficiency standpoint, eSource freed up significant space and made it easier to find and organize study data. It facilitated PI oversight because it made the source data available at the PI’s fingertips at all times.

“Several monitors are thrilled that they have the information available to them at any time in real time,” said Charie Hynes, Clinical Research Coordinator. “It has helped for coordinator questions as well because if there is a question about a specific subject or regulatory document, the monitor can review the same information with them in real time. All information is available at the click of a button – with a more transparent identification function, as opposed to PI needing to rifle through paper source in order to find the information they need.”

It also made remote monitoring much easier. For site staff, it freed up considerable time from not having to scan and upload paper charts; instead, the site could simply invite the study monitors to log in and review the source data directly. For study monitors, it made direct access immediate and frictionless.

The biggest benefit to Benchmark Research was the gain in quality. Unlike paper charts, the system has built-in edit checks that enforce quality at the point of data collection. Study designers make use of the system’s flexible instructions, formatting, alerts and branching logic to prevent coordinators from entering incomplete or wrong information. The system also has built-in workflows to help sites keep visits within window, track required re-consents, and facilitate contemporaneous PI review and sign-off of source.

Quality Assurance Findings – 38% Reduction in Protocol Deviations

The network’s Quality Assurance department was able to quantify these gains. The team reviewed the number of protocol deviations in their CRIO-based studies compared to their paper-based studies. They sampled all visits that occurred on all studies at the network’s three largest sites from 01 January 2022 through 22 March 2022. Because only new studies were put into CRIO, these sites still had considerable legacy studies occurring on paper charts during this time, thus providing a control group. Since the same investigators and staff worked on both types of studies, the QA team is confident there is no effect from differences in staffing quality or training.

The QA team used, as their main endpoint, the number of site-generated and source-specific protocol deviations divided by the number of visits conducted. Site-generated deviations are protocol deviations attributable to site error, as opposed to patient noncompliance, which is beyond the site’s control. Source-specific deviations are deviations that stem from data collected in either the paper charts or the CRIO eSource system; examples of other deviations excluded are those stemming from lab collection or informed consents, which are done outside of both paper and CRIO eSource.

The team found a 38% reduction in the incidence of site-generated and source-specific protocol deviations, as measured by ratio of deviations to site visits. The following table illustrates the findings by site and in total:

A nearly 40% reduction in protocol deviations would be a significant win for the industry as a whole. It represents a material improvement in patient safety and data quality. It also creates significant time savings because it reduces the need to develop and maintain corrective action plans, time that can be re-invested in recruiting and processing more patients.

The Final Frontier: Eliminating Secondary Data Entry and SDV

For Benchmark, the next leap for both sites and sponsors is to make CRIO’s eSource the eCRF data, either through use of CRIO’s fully integrated Reviewer EDC platform, or through CRIO’s ODM-compatible API to third-party EDC systems. This would eliminate secondary data entry into the EDC and the attendant clean-up from initial data entry errors

Data entry is a significant drag on site efficiency. Benchmark staffs at least 1 full-time data entry specialist for every 9 coordinators who collect data, implying a 10-12% workload inefficiency for Benchmark. For smaller sites, who lack Benchmark’s scale efficiencies, the inefficiency is likely even higher, perhaps in the 15-20% range.

What would it mean if sponsors could eliminate secondary data entry for the sites on their trials? For Benchmark, Ms. Copeland explains that it would mean they could see more patients with the same overall level of resources, meaning they could achieve their enrollment target faster. For other sites, where coordinators play a role in recruiting, it means more time could be devoted to recruitment. Either way, sponsors would see faster enrollment, improved data quality and greater site engagement.

“For many sites, EDC entry is very difficult, and any sponsor that could eliminate that would become the sponsor of choice for so many sites,” said Ms. Copeland.

Maximizing the Benefits of eSource for Sponsors

Benchmark has some advice for how sponsors can utilize CRIO on their trials.

“First, design the templates in a way that works for sites,” explains Tara Pratt, the network’s Technology Manager who designs most of the eSource templates. That means tailoring the template to be a real-time workflow tool for sites, while still capturing the critical eCRF fields required by the sponsor. Ms. Pratt has designed CRIO eSource on over 40 studies and has never had a problem designing them in a way to capture sponsor-required eCRF fields.

Second, Ms. Copeland does not anticipate sites will resist or struggle with the technology, especially since the sponsor would be furnishing the templates. “CRIO is so intuitive, and so obviously built for sites that even our least technically savvy coordinators could use it,” explained Ms. Copeland.

Third, sponsors can take advantage of the site configurability feature. In CRIO, the centrally designed source templates cannot be modified by receiving sites; however, sites may add their own custom visits, forms or questions on top of the main template. This ensures each site can configure the template to reflect their specific workflows.

“Site configurability has been critical to adoption within Benchmark,” explained Ms. Pratt. “Even though we’re one network, individual sites may have slightly different ways of doing things, so we find that there are small differences across the site templates on the same study.”

The biggest reason to believe in site adoption, of course, is that the system costs would be free to the site and the sites would not have to do data entry. “Any system that is user friendly and that eliminates EDC entry would be adopted by the vast majority of sites in a heartbeat,” said Ms. Copeland. “It’s such an obvious benefit to sites.”

CRIO favicon by Team CRIO
Share this post
You may also find interesting
Explore our Blog
JSCDM Header
CRIO Company News
1 Minute Read -

JSCDM: Comparison of electronic health records and electronic source data in clinical research trials

The Journal of the Society for Clinical Data Management (JSCDM) recently published the original research article “Comparison of electronic health records and electronic source data in clinical research trials: a retrospective review” which was co-authored by Amelia Tian, CRIO Solutions Consultant, and Olivia Dennis, CRIO Analytics and Data Migration Manager, alongside Dr. Elena Christofides (Endocrinology...

21 cfr compliance update
Running a Site
5 Minute Read -

21 CFR Part 11 Regulation Compliance Update

Compliance Update:  The 21 CFR Part 11 Regulation is a cornerstone of conducting clinical trials in today’s world.  The release of the regulation 1997 established guidelines for the use of electronic records and electronic signatures in FDA-regulated industries and had a significant impact on the pharmaceutical and medical device industries. The FDA began working on...

Get articles delivered to your inbox, every week