Those cheating the public sector should be afraid of the same technology that led to the disgraced cyclist's downfall, writes Steven Totman, data integration business unit executive at Syncsort.
I was over in the US when Lance Armstrong's interview with Oprah Winfrey was broadcast. During the interview Armstrong explained how drug testing has evolved in sport and why he was now afraid of getting caught.
Previously, he told Winfrey, drugs testing took places only at races, but now the emphasis is on out-of-competition testing – and on testing over a much longer period of time.
"The shift to out-of-competition testing and the biological passport really worked," he told the chat show host.
As a big data geek, I was curious about how this biological passport effectively put an end to what can only be described as pathological mass cheating.
A biological passport is like the average speed cameras on motorways. It tracks what's happening in an athlete's blood over time. Previously, speed cameras were fixed at certain points and people would generally drive over the speed limit, slamming on the brakes just before the camera. This is pretty much what Armstrong and others were doing.
But with an average speed camera you log and track the car over time and distance, forcing drivers to maintain a steady speed. Similarly, the biological passport uses biomarkers that will remain valid for several decades.
There's a clear link here with using big data to prevent fraud in the public sector. To build and maintain a long-term profile, you need to gather and process a lot of data – in this case biometrics. The ability to collect big data and the technology to allow any organisation to collect and process the ever-expanding data avalanche is already having a huge impact in many areas. Armstrong made the point that what changed his sport was tracking key indicators over time. The same applies to fraud: the better the data passport built up on a user, the easier it is to spot fraudulent behaviour.
As I mentioned in the Public Leaders Network live discussion on public sector fraud on 1 February, new big data technologies completely change the cost of collecting data. One financial institution we have worked with, for instance, was formerly able to keep only two years' customer data online, due to the costs of storage and processing.. Now, big data enables them to collect analyse their entire customer history.
That means they can use a much broader analysis of customer spending habits when looking for patterns of fraud. I have previously moved credit card companies because several could not figure out that I travelled constantly with work and did not want to spent 20 minutes on the phone every time I bought something in a country I visited frequently.
This has huge implications for detecting public sector fraud. As with speed cameras, if you look only at a small, focused data point, people can brake just before it. If you are looking over a much longer period, it's much easier to spot the scammers.
It isn't just Lance Armstrong that's worried about big data – it's anyone trying to cheat the system.
(Column first published as a guest blog on The Guardian)