Why spreadsheets aren’t always the best solution

Spreadsheets are often used within organisations because it is what many employees are familiar with using, not because they are the right tool for the job.

Recently, as reported by Leo Kelion, Technology desk editor of the BBC (Read the article here) we have seen the reliance on spreadsheets cause major problems for Public Health England (PHE) with nearly 16,000 coronavirus cases left unreported.

In this article we will look at the reasons behind the missing cases for PHE and how organisations can avoid this type of error with more scalable solutions.

Under-reported figures

The scale of the issue is plain to see when you look at the figures. 15,841 unreported cases missed due to an ‘IT error’. Further investigation suggests that the error lies with PHE and their choice to merge the data submitted by commercial testing firms, supplied in CSV files, into an excel spreadsheet file.

The main problem was that the file format that they chose was the older XLS file format which can only handle approximately 65,000 rows of data. That may sound like a lot of room for many cases, but in practice it meant that only roughly 1,400 cases could be stored on each file. This is due to each test result having many rows of data per case.

This could have been avoided by utilising the newer XLSX file format which would allow for 16 times the amount of cases, but that would still be a short-sighted solution with cases and testing set to rise over the coming months.

As stated in an article by Alex Hern, UK technology editor of the Guardian (Read the article here), another problem with the approach taken at PHE  is that it is largely a manual process, especially in the early days. Manual processes will always be error-prone, further compounding the risks involved with dealing with such volumes of data, with such high levels of importance, such as data outlining positive cases of a highly infectious disease.

These errors are not exclusive to PHE’s handling of the coronavirus data and have haunted many other companies in the past. For example, as Laurence Dodds pointed out, in his article for The Telegraph (Read on the article here), large financial organisations have also fallen foul of using Excel spreadsheets to manage their financial risk assessment system.

Free Guide Download our free guide to learn the top 5 digital transformation trends for 2021

5 DX Trends Download

• Email Address
• Phone
  This field is for validation purposes and should be left unchanged.

Δ

Low-Code. An alternative solution?

So, what else could PHE have done differently? When it comes to large amounts of data, especially in the volumes expected later in the year for coronavirus testing, it is apparent that Excel isn’t the right solution. This is where Low-Code applications, such as PhixFlow, really shine, especially when the need for automation becomes increasingly critical.

For example, applications that we have built on the PhixFlow platform process in excess of 1.5 billion records on a monthly basis, with each record benefitting from data enrichment and additional calculations, and tens of millions of records daily. And with no practical limits on connecting to data sources and systems the end solution is future proofed for scalability.

The benefits of Low-Code

Not only can Low-Code applications handle large amounts of data, but there are many other benefits too.

1)      Ease of adoption

With many users already used to working with spreadsheet programs such as Excel, adoption of Low-Code applications is made easier through the ability to view and filter data in grid views. Users also can produce reports, export data and perform analytics on the data using excel-like functions to provide further insight.

2)      Reduced Development Time

Low-code development allows teams with little development experience to create and deploy solutions using easy to use drag and drop visual design screens, pre-built templates and a library of connectors to seamlessly connect with any data source.

3)      Lower Costs

With Low-Code being accessible to more people, smaller organisations lacking deep pockets and large budgets can create applications without hiring, training and onboarding expensive developers.

4)      Improved agility

With software development times greatly improved and in-house teams available to develop solutions you will see a vast change in your organisational agility. Employees will be able to spend more time on what matters most to your business and should any changes be required to your applications, there will be no delay in waiting for third-party development companies or scare development resources to make the changes.

5)      Data security

Low-Code platforms also provide excellent levels of security, allowing complex data policies to be adhered to. These platforms also have strict user access control, automatic back-ups and audit trails to ensure that any action performed on the system is logged and analysed, with any errors being reported as they happen.

Summary

“Data is the new oil,” as coined by Clive Humby, the architect of the Tesco Clubcard, and with the rise in the volume of data that is available for companies to process and analyse it is clear that many companies are overly reliant on spreadsheets which causes issues such as those seen at PHE.

Therefore, to ensure that your data is processed efficiently, accurately and reliably, it is imperative that businesses choose a solution that is fit for purpose and can handle the ever-increasing amounts of data.

For more information on how PhixFlow can help your business, please request a demo.