Election Forecasting Accuracy

FiveThirtyEight isn't the only news source I know of that tries to check themselves for errors and talk about why they got things wrong (or why they got them right), but they do tend to be more comprehensive about it than most places are. The specific example which I'm writing about is this very detailed article from last week about their midterm forecasts.

One of the most counter-intuitive things I got from that article is that their probabilistic forecasts mean that they actually don't want to be right all of the time. For example, out of all the races they say are fifty-fifty splits, they'd want to get half of them right - if they were capable of picking the winner more often than that, it would imply that the odds weren't actually even to begin with. So they're actually worried if they're calling too many of the winners accurately, which is a rather amusing problem to have.

They also offered a number of concerns they might want to take a harder look at as they go on to forecast the 2020 election. One of them was underestimating the effects that stronger partisanship was going to have, which is rather depressing but certainly not surprising. The other big one was them discounting polls conducted by partisan groups a bit too much in some cases. It will be interesting to see how they change their forecasts going forward... although we might not figure out how useful any changes are until it's time for the 2020 postmortem.

And then I probably have to admit that one of my favorite parts was the closing portion, which I will just quote rather than trying to paraphrase:

We actually do think there’s enough of a track record now to show that our method basically “works.”... We probably won’t be as accurate-slash-lucky in 2020 as we were in 2018, especially in the primaries, which are always kind of a mess. But our way of covering elections is a good way to cover them, and it’s here to stay.

I'm looking forward to plenty more great reporting from them.