This document describes CRIO’s integrated eSource/EDC model, in which source data is collected electronically by the site using CRIO’s eSource system, then transmitted automatically to CRIO’s Reviewer EDC module or to legacy EDC systems, which allows sponsors to review, query, code, lock and extract the data. This model unlocks significant savings for the sponsor/CRO, mainly by eliminating source data verification and enabling real-time, continuous access to site data. When implementing CRIO’s system on trials, sponsors should revise their monitoring and data management processes to take advantage of these efficiencies. That’s because incumbent monitoring and data management processes have been optimized for a double-data-entry system that CRIO replaces. To support sponsors and CROs in their deployment of CRIO’s system, CRIO has developed templates for a Clinical Monitoring Plan and a Data Management Plan which operationalize the principles outlined here. Download this article here.
THE CURRENT MODEL
To understand the benefits of CRIO’s model, it’s important to describe the current model in clinical trials.
In the current model, clinical research sites collect source data, which is the data collected contemporaneously with the protocol-prescribed procedures, and then transcribe this data into the electronic case report forms (eCRF) housed in an Electronic Data Collection (EDC) system provisioned by the sponsor. This transcription is not contemporaneous and can lag weeks and even months. The source data is then reviewed and confirmed directly by the Clinical Research Associates (CRAs aka “monitor”), and the EDC data is reviewed and queried by the Data Managers. After the EDC is locked, DMs will extract the data for statistical analysis.
The two data sets (source & EDC) are separate data sets. Data is entered through separate workflows and exist separately from each other. It’s true that the EDC data set is a representation of the source data set – but that’s enforced only through manual entry by the site with manual source data verification by the CRA. We should therefore think of each data set as having a differing workflow for review by the CRA and DM. The following is a schematic:
For source data collection, the site develops their own source templates to collect data per the protocol and eCRF guidelines. Currently, the vast majority of sites utilize paper templates, not electronic medical record (EMR) templates, for reasons we’ve explained here. These paper templates will vary from site to site, and the templates are written as workflow tools to enable the site to collect data properly, For example, if the protocol specifies that vitals must be taken after 5 minutes in the sitting position, and the arm must be the same as at Visit 1, the site may write the template this way:
With paper, the site’s data collection process is highly error prone; for instance, in the above template the site might mis-record the date of the procedure, record the wrong arm, leave a field blank, forget to initial and date, or record a signature (attribution) date inconsistent with the vitals date. All of these are potential findings that the CRA will flag for corrective action, as part of the sponsor’s obligations to ensure quality and patient safety across the trial.
Typically, after the patient visit, the site transcribes the data from the templates into the eCRF, which often involves a simplified summary version of the data. In the above example, the equivalent eCRF might look like this:
The eCRF fields usually represent the critical data points required for analysis. Often, as in this example, sites collect more data points with the purpose of documenting not just the target eCRF data, but also the data necessary to demonstrate that the procedure was performed consistent with protocol requirements. In this example, the eCRF simply asks for the number of minutes in the position as opposed to the sitting time and vitals time – which is a more appropriate way of demonstrating protocol compliance. It also leaves off the arm used – a protocol requirement that is left to the CRA to enforce.
When the site enters the data, this will be the first time that Data Management sees the data. However, because the EDC is a secondary entry point, when data is blank, DM cannot tell if it’s because the data wasn’t collected or was collected but not yet entered. Indeed, DM often waits a long time for data to appear in the EDC – industry studies show an average EDC entry time of at least 8 days, and many times data entry can be delayed by a month or more. At times, DM may see discrepancies across the systems, such as that a visit was recorded in the IXRS but not in the EDC. Furthermore, DM may see some forms in a visit completed but not others, leaving open the question of whether the visit is a “split” visit to be completed later, or the site simply didn’t get around to full data entry. And when data is entered, DM has no assurance that it was entered accurately, especially if the data is outside an expected range.
Because of the lack of visibility into the source data, CRAs must perform periodic monitoring visits to review the source data. CRAs perform two interrelated processes at these site visits. First, CRAs perform full source data review (SDR) to ensure that the source data evidences compliance with the protocol and ICH-GCP. This means that the CRA will review the completed templates to make sure that they are complete, internally consistent, and fulfill the ICH-GCP requirements of being ALCOA+ – attributable, legible, contemporaneous, original and accurate along with complete, consistent, enduring, and available. Second, CRAs perform source data verification (SDV) by confirming that the data entry in EDC is consistent with the source. For this latter workflow, most EDC systems support a process specifically for the CRA to indicate SDV was completed, as a prerequisite to DM review.
Recently, risk-based monitoring (RBM) permits the CRA to perform SDV on a select subset of the data. However, this has translated into minimal cost savings since targeted SDV can be difficult to understand and set up, much less enforce. It is also not supported in all EDC systems. In many cases, CRAs end up performing 100% SDV since that is the workflow they are accustomed to, and it may be quicker simply to confirm all data points than to first identify the data points to confirm, and then to confirm. Also, the CRA still must perform full SDR on the data set. Thus, RBM has not lived up to its promise of reduced savings. If you are a sponsor and you have any doubts about this proposition, ask yourself a simple question: Have my monitoring costs gone down since the introduction of RBM?
After SDV, DM will review the data on a form-by-form basis, marking each form as reviewed and issuing queries as needed. In addition to review at the form level, DM sometimes supplements the review with listings-based review to identify outliers or trends across sites. Many EDC systems do not support this type of review, so DM may extract the required datasets, sort and analyze off-line typically in SAS, then go back into the EDC to open and query the flagged forms. For interim or final data locks, DM needs to ensure that all forms have been reviewed and queries responded to. When the site’s data set is ready to be locked, the CRAs usually ask the PI’s to enter the EDC database and input their signature. Since PI’s rarely interact with the EDC system (usually delegating data entry to coordinators), this process can be time consuming and require multiple PI follow-ups.
THE CRIO MODEL
In CRIO, there is only one template, not two – the source and the eCRF are one and the same. This single data collection template is built by the sponsor/CRO for publication to the sites. The eSource template should be written in a site-friendly manner, similar to the way sites are used to capturing data. To this end, CRIO has developed a study design guide explaining the way CRIO’s eSource templates should be configured, and how the design approach differs from more traditional eCRF approaches. Once published to a site, the site cannot edit the core template, but may add site-specific custom procedures or questions on top – thus accommodating potential differences in workflow across sites.
Unlike paper source, CRIO’s eSource is interactive, guiding the sites along the appropriate workflow path. With built in edit checks and branching logic, the template ensures that sites are adhering to the protocol and collecting data accurately and completely. An automated audit log ensures full attribution. Thus, CRIO’s eSource builds quality into the front-end data collection process – unlike the eCRF, where the edit checks only trigger after the secondary data is entered, which is many days, weeks if not months, after the patient visit is complete. And when the site disregards the system’s built-in alerts, CRIO converts the alert into an auto-query.
THE CRA WORKFLOW
Upon visit completion, the visit is placed into the Reviewer EDC queue, teed up for the CRA to review within minutes of visit completion. There is no need to do SDV whatsoever. Instead, the CRA simply does remote SDR, but this time, the CRA can review the visits as they are completed, instead of having to batch review into discrete “monitoring visits” that can be weeks or months apart. With continuous monitoring, the CRA can identify and course-correct site mistakes proactively in the process, and Central Monitoring can more quickly identify data collection trends across sites. Because the visit templates are standardized, a cross-site, real-time review process is extremely efficient.
When reviewing the visits, the CRA will see the questions, answers and audit trails. The CRA can query the data at the form or field level. The CRA will also view the auto queries generated by the system and can close them out if the source adequately explains the query generated. When done, the CRA can mark the visit as Reviewed. Any changes to source will put the visit back in the CRA’s queue with an indicator of what changed.
This remote process generates significant monitoring savings:
- No need to travel to the site
- No need to perform SDV
- SDR is faster because the data is cleaner and the presentation standardized
Plus, CRAs can course correct sites sooner, thus preventing further errors.
Not only does the CRIO system place the visit in the CRAs queue, it also places the visit in the PI’s queue for signature simultaneously. This is because the CRIO system is an eSource system, and PI’s traditionally evidence trial oversight by signing the completed source shortly after the visit is completed. This contrasts with the traditional model, where the PI’s signature is solicited right before the interim and/or final data lock, on a bulk basis. CRIO’s system is superior in that it leverages existing site workflows, ensures continuous oversight by the PI, and avoids the cumbersome practice of soliciting PI signature in an EDC system they rarely interact with.
When the CRA has reviewed the visit, all queries have been resolved and the PI has signed off, the visit is eligible for locking. CRIO encourages the CRA to lock the visits as they are eligible. This creates finality and enables much more rapid interim and final database analyses. The CRA can always unlock a visit if the site needs to make further changes.
THE DM WORKFLOW
After the CRA review is complete, the DM can perform a second review. However, in contrast to the current model, the data comes to DM in a much more reliable state. By the time the data has been locked, the data has already gone through three quality filters:
- The system itself, during data collection, uses real-time edit checks to ensure completeness and accuracy – and then there is no secondary data transcription;
- The CRA, in short order, has already completed full SDR and resolved all queries;
- The PI, through the PI sign-off process, has reviewed and signed off on the data.
Because of these built-in reliability filters, CRIO recommends that DM not perform traditional form-by-form review. That extra level of review is simply redundant.
An analogy to electronic patient report outcomes (ePRO) may be helpful. Before the widespread use of ePRO, patients were given patient diaries by the site. Patients completed those entries off-site, then returned them to the site at the next visit. The site then transcribed the data points into the EDC. Because of this process, sponsors and CROs had to treat PRO data the same way they treat site-initiated source data: The CRA had to SDR and SDV of the patient diaries, and DM had to review and query the data. With the advent of ePRO, CRA and DM workflows are much more streamlined. Neither party scrutinizes the ePRO data for reliability or data integrity the same way they do site-transcribed eCRF data. For instance, most DM teams do not manually review every daily entry by every patient for completeness or accuracy.
Given the significantly reduced risk associated with data quality, CRIO recommends DM move to a risk-based review of the data. To start with, CRIO recommends that DM only review and interrogate those data fields that will be used in the biostatistical analysis. Typically, these data points are captured in these eCRF forms:
- Medical history
- Medications and procedures
- Adverse events
- Subject disposition
- Primary and secondary endpoint data
CRIO allows the sponsor to designate certain fields in the template as “Core” – these can be used to designate the fields that will be submitted to the statistician for the clinical study report. In the DM review workflow, CRIO takes these Core forms and presents a cross-site, cross-subject listing view of the data, such as this:
This list is filterable and sortable. Individual rows hyperlink to the underlying visit record. This review queue is designed to facilitate rapid review of outliers and patterns and facilitates the type of listings-based review many DM’s perform today to complement form-level review.
Using this listing view, DM’s can perform individual or bulk-level review; if there are questions about the data, DM’s can generate a query to the site or an internal query to the CRA to follow up with the site. Because CRIO recommends that CRAs lock the visits as they go, DM or the CRA can unlock the visit if in fact the site needs to modify data. Any changes in site data will re-queue the visit into the PI, CRA and DM queues.
As mentioned above, data is continuously being reviewed, queried, and locked, so interim and final dataset analyses can be done quickly. CRIO lets authorized users export the data at any time in CSV format; DM can filter only for Core fields as an option.
IMPACT ON SITES
CRIO’s approach greatly enhances efficiency for the sites. First, sites do not have the burden of creating their own source templates, which is time consuming and often inaccurate, although each site does have the ability to add specific forms or procedures to meet site specific workflow requirements. This ensures consistency of the eCRF fields across all sites while allowing the flexibility for each site to meet their own study conduct requirements. Second, sites do not have to perform EDC entry – saving anywhere from 30 minutes to 2 hours per visit completed. Third, sites will spend a lot less time cleaning up data, responding to queries or preparing their source for on-site or remote monitoring review. With this saved time, sites can focus on patient recruitment and retention.
CRIO’s eSource system has been voluntarily adopted by more than 1,000 sites around the world. They have adopted the CRIO eSource system because they themselves experience the benefits of time savings (up to 25%), quality improvements (up to 70%), and remote collaboration and PI oversight. Unlike other systems, CRIO’s system was built and deployed at sites first, ensuring that the end users – those actually charged with data collection – embrace the technology.
The CRIO Model represents the next generation of clinical trial operations. It addresses one of the last remaining areas of paper-based data capture and extends and amplifies the move toward Decentralized Clinical Trials. It leverages existing research site workflows, builds in quality in the front end, and brings those efficiencies to the sponsor. To take full advantage of this model, CRIO recommends sponsors update legacy processes to optimize for a direct data capture paradigm. CRIO supports sponsors and CROs in this transition with recommendations on how to design eSource templates and how to approach monitoring and data management.