Predicting the Brexit referendum

I'm proud to announce that last month I became an entrepreneur! Together with two of my friends and colleges, physicist Dejan Vinkovic and computer scientist Mile Sikic, we started a company called Oraclum Intelligence Systems Ltd., based in Cambridge, UK. The name of the game? Electoral forecasting. But also data visualization, big data analysis, and market research.

Essentially, we are a start-up in its R&D phase, and are at this point mostly concerned with experimental testing of our prediction models on real life data. And what better way to test a prediction method than elections!  

I already wrote about our earlier efforts in predicting the Croatian 2015 general election, but now our focus is on the international stage where the first big election coming up is the UK EU membership referendum (the popular 'Brexit'). After that we will focus our efforts and attention on the US 2016 Presidential election (and what is most certainly going to be a duel between Clinton and Trump - as I predicted back in January - I also gave Hillary a slight advantage over Trump at the time. We'll see how that goes). 

Anyway, over the next month my blogs will be mostly focused around Brexit and electoral predictions of the referendum. These will include a series of introductory texts on our methodology, the benchmarks we will be using for comparison (which will be regularly updated), the ranking of pollsters based on their historical performance, etc. And of course we will provide day-to-day coverage of our results two weeks prior to the referendum (which is held on June 23rd).

All these texts will be available on our official Oraclum Blog, plus I will also create a separate page on this blog where I will keep track of all these texts (similar to the page I have on the Eurozone crisis).

How Britain got to the Brexit referendum

A brief introduction into the political landscape so far. Back in January 2013, in response to mounting pressure from his own party and the upsurge in popularity of the eurosceptic UK Independence Party (UKIP), British Prime Minister David Cameron promised to his voters that if they re-elect his Conservative government he will give the citizens a chance to vote on the in-out EU referendum for the first time since 1975: "It is time for the British people to have their say. It is time to settle this European question in British politics." The date was set to be in 2017 at the latest. 

As a prelude to the 2015 general election campaign, he emphasized on numerous occasions his willingness to keep his referendum pledge, even announcing that the referendum might take place earlier than initially conceived. 

The campaign strategy worked. Conservatives were sworn in by a landslide electoral victory, much to the complete surprise of almost all UK pundits and almost every pollster. While everyone was predicting a very close election and a virtual tie between Labour and Conservatives, the Conservatives picked up 100 seats more than Labour which was enough for them to form a single-party government. Reinvigorated by this success the party quickly moved forward the EU Referendum Act. It was introduced to the House of Commons already in May 2015 (a few weeks after the elections), approved in December, with the official date (23rd June 2016) announced in February. The referendum question itself was designed to be quite clear, leaving no room for ambiguity: 

“Should the United Kingdom remain a member of the European Union or leave the European Union? 

• Remain a member of the European Union 
• Leave the European Union”

It is hard to say whether or not the pledge of an EU in-out referendum helped the Conservative party (there were certainly other things that led them to such an impressive and unexpected result, primarily the dismal performance of Labour and Liberal Democrats across the country), but it was a gamble the PM was willing to take. He kept his promise, even allowing individual party members to form their own opinions on the Brexit, not necessarily along official party lines. 

Cameron’s plan was to renegotiate Britain’s deal with the EU, primarily concerning immigration and welfare policies. In February he did just that, although many would disagree on the extent of his success, calling it a lukewarm deal at most, falling short of many of his promises. The deal is set to grant Britain a “special status” within the EU if they vote Remain. It ensures that Britain will not be a part of the path towards “an ever closer union”, that the financial sector is protected from further EU regulations, and that Britain is exempt from further bailouts of troubled eurozone nations (and is even to be reimbursed for funds used so far). Where it fell short of expectations was the migrant welfare payments and child benefits. Migrant workers will still be able to send child benefits back to their home countries, while new arrivals will gradually be able to get more benefits, the longer they stay. Some compromises were made by both sides of the bargaining table, however this hardly satisfied the eurosceptics back home. 

Today, the Conservatives are divided. The party leadership as well as the majority of government ministers support the Remain campaign. However, roughly half of Conservative MPs, 5 government ministers, and the former Mayor of London and prominent Conservative figure, Boris Johnson, all support the Leave campaign. It would seem that the Conservative voters accurately depict their party’s split – YouGov reports that the distinction is 44%-56% towards Leave. 

On the other hand the Labour party’s new leadership under Jeremy Corbyn has expressed its official position to support the Remain campaign, although political pundits have noted a slight reservation of Corbyn towards the EU (primarily based on his previous opinions of the EU). However, Labour voters are much more inclined towards the EU than their current party leader. 75% of them support Remain, while only 25% support Leave. LibDem voters are even more pro-EU (79-21), while on the other hand of the spectrum, UKIP voters are perfectly aligned with their party’s position (97% support Leave). 

Usually, when political party leaderships in a country announce their position towards a referendum (particularly on EU membership), the outcome is very often predictable – voters listen to their parties and vote accordingly. The same can actually be said of the current division regarding Brexit – voters do listen to their parties. The Conservative party is split (its official position is to be – neutral) and their voters act accordingly. UKIP, LibDems, and Labour are all relatively united towards the referendum question, so their voters also vote accordingly. It is this interesting dynamic operating within the Conservative party and the electorate in general that makes this referendum a difficult one to predict. After all, the majority of the polls are predicting a very close result, within the margin of error. 

The role of Oraclum

What is our stake at these elections? We are, above all, a non-partisan venture in its start-up R&D phase, interested in experimental testing of our models on real-life electoral data. We aim to use a Facebook survey of UK voters (more on that in the next text), along with our unique set of Bayesian forecasting methods to try and pick out the best and most precise prediction method. Essentially our motivation in this initial stage is purely scientific. We wish to uncover a successful prediction method using the power of social networks. After the Brexit referendum, we will apply the same methods on the forthcoming US Presidential elections in November 2016. 

In our Facebook survey we will not use any user data from Facebook directly or indirectly, only the data the users provide us in the survey itself. We will have no knowledge of voter preferences of any individual user, nor will we be able to find that out. 

The Facebook survey will be kick-started 10 days prior to the referendum, on 13th June, and will run up until the very last day when we will provide our final forecast. Our forecasts will show both the total predicted percentages and the probability distributions for both options. They will also show the distribution of preferences for the friends of each user (so that the user could see how his or her social network is behaving and who they, as a group, are voting for), as well as the aggregate predictions the survey respondents will be giving. 

Furthermore we will present our predictions in a map format, based on UK regions, where we will show the actual polling numbers, and our Bayesian adjusted version. 

We welcome all suggestions, comments, and criticism. 

In the next blog post, I will introduce you briefly to our method and the number of benchmarking methods which we will use for comparison.