Part three: 15 pragmatic guidelines to handle data quality issues

In the first article in this series, published Wednesday 14 August, we discussed the importance that data quality plays in the effectiveness of data analyses by Machine Learning (ML) and AI. In part two, we looked at the impact that poor data quality can have on ML models. Here, we outline 15 pragmatic guidelines to ensure better data quality in early drug discovery and to identify potential issues with compromised data.

1. Data collection and entry

Standardisation, by implementing standardised operating procedures (SOPs) for data collection and entry, is a key consideration. This includes consistent use of units, naming conventions, and data formats. The team involved in data collection should be trained on the importance of data quality and the specific protocols to follow in order to minimise errors.

2. Data validation

Automated checks should be carried out with automated validation scripts, which check for common issues such as missing values, duplicates, outliers and inconsistencies in units or formats. Furthermore, manual reviews of a subset of the data should be performed periodically to identify any issues that automated checks might miss.

3. Data cleaning

A strategy should be developed for handling missing data, such as deciding when to use imputation, exclude data points, or flag datasets for further investigation. Also, methods should be implemented to identify and investigate outliers, determining whether they represent true variability or errors.

4. Data integration

It is essential to ensure that data from different sources or experiments are harmonised before integration. This includes reconciling different naming conventions, units and formats. Consistency and correctness can be checked by cross validation, in which cross-referencing methods validate integrated datasets.

5. Data documentation

Researchers should maintain detailed metadata for each dataset, including information about the origin, collection method, and any preprocessing steps. This helps in tracking data provenance and understanding the context. Also, version control systems for datasets should be used to track changes and ensure that any modifications are well documented and reversible.

6. Data monitoring

Data should be monitored continuously, recording quality metrics like completeness, accuracy, and consistency, throughout the data lifecycle. Moreover, automated alerts can be set up to notify relevant personnel if data quality metrics fall below predefined thresholds.

7. Data auditing

Conduct regular data audits to assess the overall quality of your datasets. This involves checking for adherence to data quality standards and identifying any systemic issues. Also, maintain audit trails that log all data processing steps, transformations, and any changes made to the data to ensure traceability and accountability.

8. Bias and variability checks

Regularly assess your datasets for potential biases, such as over-representation of certain chemical scaffolds or biological targets. Statistical techniques should be used to quantify bias and take corrective actions. Analyse the variability in your data, particularly in biological assays, to understand the level of noise and its impact on model performance.

9. Data redundancy and duplication checks

Implement robust mechanisms for detecting and removing duplicate records, to prevent skewing of the data. Then, techniques such as correlation analysis can be used to identify and eliminate redundant features that do not contribute new information.

10. Data imbalance handling

Continuously monitor the balance of different classes in your data (eg, active versus inactive compounds). Address imbalances through methods such as over-sampling, under-sampling, or synthetic data generation. If data imbalance is unavoidable, consider using model algorithms that are better suited to handle imbalanced data.

11. Data security and access control

Security measures can be implemented to protect data from corruption, loss or unauthorised access, ensuring that data integrity is maintained. With this, access to data can be restricted based on roles and responsibilities to prevent unauthorised modifications or data entry errors.

12. Communication and collaboration

Foster collaboration between data scientists, domain experts and IT professionals to ensure that data quality requirements are clearly understood and addressed. A feedback loop should be established, whereby issues identified by data scientists or model results are communicated back to the experimental team to refine data collection processes.

13. Use of quality control samples

Include quality control samples (eg, known standards or replicates) in experimental runs to monitor and ensure consistency in assay performance. Regularly analyse the results of quality control samples to identify any drifts or deviations in experimental conditions that could impact data quality.

14. Data quality metrics and reporting

Define specific data quality metrics such as accuracy, completeness, consistency, timeliness, and uniqueness. Use these metrics to evaluate and report on the quality of your data regularly.

When reporting data quality metrics to stakeholders, ensure transparency to facilitate continuous improvement.

15. Continuous improvement

When data quality issues are identified, perform a root cause analysis to understand the underlying reasons and implement corrective actions. You should treat data quality improvement as an iterative process, continuously refining and enhancing your strategies as new challenges and technologies emerge.

In the next article, which will be published Tuesday 24 September, we will present views from an industry veteran on the topic of data quality.