Lessons from spreadsheet data horror stories
2022TL; DR
Lessons from published stories of mismanagement of data using Excel leading to data loss and corruption.
Session Details
There are many pitfalls in using Microsoft Excel to processing important data. I present published reports of horror stories and draw lessons from them. Firstly, how unskilled Excel users corrupted gene name records; researchers decided to rename genes to suit Excel! The lesson is to learn how to import CSVs into Excel correctly. Then I describe the Public Health England debacle, where they lost Covid lab results. That illustrates the risks of automated data imports without safety controls and the lesson is a basic reconciliation technique.