They say the definition of madness is doing the same thing over and over again and expecting different results. So what, then, are we to make of the Australian Government’s fervour for data retention? Is it not the government taking something other countries have done and found to be ineffective, doing it anyway, yet expecting different results?

Mandatory data retention is in the news again in Australia, as an incredibly unpopular and incompetent Prime Minister scrambles back to his comfort zone of national security, and attempts to scare us into forgetting about his failings as a leader. So once again, Tony Abbott and his ministers are in the media saying silly things about technology they don’t understand, in an attempt to sell us the complete falsehood that data retention is required to fight crime and prevent terrorism.

Which is not to say terrorism is not a problem. It might not be as big a threat as the government and media make it out to be, but it is a threat. Which is why it requires sensible, evidence based approaches. As opposed to data retention, which represents an anecdotal, fear based approach.

Australia is not the first country to try mass surveillance of its citizens in an attempt to combat crime. Others countries have had data retention in place for a number of years already, which means we can learn from their experience. The evidence is fairly damning. The international experience shows quite conclusively that data retention doesn’t work. Not only is it completely ineffective, but it’s incredibly expensive, undemocratic, and open to abuse. It’s for those reasons many countries that have adopted data retention measures are now abolishing them.

But Australian authorities are determined to learn nothing from this, and are persisting in their misguided quest to spy on all of us.

While there’s been a backlash on social media to the data retention proposal, out in the real world people don’t really care, unfortunately.

For a country that’s somewhat cynical towards its politicians, we’re still remarkably willing to give government the benefit of the doubt. We shouldn’t.

Perhaps part of the apathy also comes from living in a world where organisations collecting data about us has become somewhat normalised. Whether it’s government or business, we know our actions are being tracked and stored. We know that social media companies track and store a huge amount of data about our online activities. We know that retail companies track our shopping habits through reward card programs.

But it’s worth noting there’s a big difference in the potential downside of a private company collecting data on users, and a government collecting data on its citizens.

For starters we have a choice whether to use the services of a private company, so we can always opt out. But if we do decided to “opt in” to a business and use its services, it becomes a value proposition. We accept that a social media website will track our usage, in exchange for the (free) service that social media website provides. We accept the retailer tracking our purchases, in exchange for whatever perceived benefit we may get (whether real or illusory) from the rewards program. If we decide the value provided by the service is no longer worthwhile, or the company displeases us enough in some way, we have the option of no longer doing business with them.

And usually the worst thing a private company can (legally) do is try to sell you something. Or sell data about you to some other company that tries to sell you something.

In contrast, the worst thing a government can do is take all your money and put you in jail for something you didn’t do — and maybe even execute you.

While the worst thing that can happen when Google’s advertising algorithm goes wrong is you will see an ad that has zero relevance to you. The worst thing that can happen if the government’s data retention goes wrong is you end up in jail for the rest of your life even if you didn’t do anything wrong. Or hackers get hold of your metadata and use it to steal your identity. Or you end up stalked by horny ASIO agents who have been illicitly reading your emails and looking at your private photos in the cloud. The possibilities for abuse are endless, but the benefits nonexistent.

There is no opt out from government mass surveillance, and no value proposition. What do we as citizens get out of it? Apart from a completely false sense of security? Apart from the illusion of government doing something about crime? Data retention has never worked to prevent crime or terrorism anywhere in the world where it’s been tried, so any perceived value is a mirage.

The benefits of data retention are illusory, but the downside is huge, especially considering the false positives problem. Terrorism is subject to the same principles of randomness and probability as everything else, and the false positives problem is the statistical reason data retention is ineffective in catching or preventing criminals.

With any kind of data analysis, the larger and broader the dataset, the more statistical noise, and the harder it is to analyse the data to find rare patterns or occurrences. When you have a large untargeted dataset, of the sort generated by mandatory data retention, meaningless patterns will occur out of sheer randomness. Just like when tossing a coin, it’s not unusual to get series of heads or tails in a row, but those consecutive heads or tails runs are just random noise and don’t tell us anything about the long term probabilities of the coin toss. This kind of randomness makes using a huge collection of data to find a very small number of criminals or terrorists pretty much impossible. Of particular concern is that any kind of analysis will generate a huge number of false positives — totally innocent people the algorithm has flagged due to randomness. No matter how good the algorithms may be, not matter how advanced the analysis, there will still be a huge number of false positives, because that’s how the probabilities play out.

Given that the worst thing a government can do if it’s wrong about an individual is take away everything they own and deprive them of their freedom for the rest of their life, mandatory data retention should be of concern to everyone. Especially innocent, law-abiding citizens. Those thinking they have nothing to hide because they haven’t done anything wrong are in for a rude awakening if they’re unlucky enough to end up being one of false positives.

Mandatory data retention will increase the chances of innocent people being convicted of crimes they didn’t commit, all the while diverting police resources from pursuing actual guilty criminals.

Any kind of analysis on the metadata database will flag so many genuinely innocent people as potential criminals worthy of investigation, it will be impossible to sift through them all to find the truly guilty. Police and intelligence resources will be wasted on wild metadata driven goose chases, trying to establish that innocent people are actually innocent. Or worse, prosecuting the innocent.

Authorities already have considerable powers for targeted data collection on specific suspects. It is extreme overkill, not to mention extremely expensive and counterproductive, to use mass surveillance to collect data on everyone. Especially the innocent.