By Vann R Newkirk II, THE ATLANTIC
Donald Trump’s surprise victory begs the question: How did we get this thing thiswrong? From the myriad polls and poll aggregators, to the vaunted oracles at Nate Silver’s FiveThirtyEight and the New York Times’s shiny forecasting interface, most serious predictors completely misjudged Trump’s chances of victory.
Though election night had the appearance of an unlikely come-from-behind victory by Trump, that narrative only exists because virtually all predictions—perhaps even from the Trump camp—started with the assumption that Trump was an underdog. In reality, when viewed with proper perspective, Trump sailed to a rather easy victory, challenged Clinton in several stronghold states, and realistically wrapped the election well before midnight. That kind of result doesn’t come out of nowhere, but few pre-election polls even began to pick up such large effects.
Forecasting sites and models have keyed into this discrepancy and had success over the past few election cycles by aggregating smaller state and county-level polls, and then forecasting actual Electoral College votes from those aggregates. That approach has obvious advantages, but suffers sometimes from lack of available and reliable data. As a rule, many state and local polls are newer and more volatile than national polls, and several rely necessarily on unorthodox methods to achieve enough proper sample sizes, which are also often much lower than national polls. Also, the baseline statistics from Census products and other large surveys used for “weighting” state and local results become less reliable as they drill down.
Long story short: Statistical power is important, and any misrepresentation of the population in the sample or weights can lead to unusable results.
But the rise of cell phones and the demographic differences of their adoption meant that random samples of landlines became increasingly inadequate in finding good samples. The problem with moving to cell phones or even attempting a hybrid approach is that cell phones are not usually publicly-listed, making it harder and harder to find representative samples. Various online survey methods have been used to supplement or supplant more expensive and less expansive phone methods, but they often also suffer from bias and are generally considered of lower quality than other polls.
The most recent poll in that model came from the mixed landline and online Gravis Marketing poll, and featured results with a whopping 3 percentage-point margin of error and a sample that was weighted not to Pennsylvania demographics, but to national demographics. One other poll in the aggregate is the SurveyMonkey poll, which is likely limited by its reliance on a largely skewed group of voters—people who respond to SurveyMonkey polls. Each of these showed Clinton leads in the state that Donald Trump eventually won.
Namely, if polls tend to weight Democratic or Republican likely-voters and demographics based on 2012 elections patterns or older demographic distributions, they will naturally miss out on big shifts in the composition of likely-voters or where they live. If high numbers of the wealthy, white, educated pieces of the Obama coalition turned out for Trump, and he also picked up unprecedented turnout from rural voters, models that weight data to recent past elections might understate those effects. Many of these polls might be ill-suited to understanding sudden changes in the electorate or the way the electorate votes.
There are some solutions to this “likely-voter” problem in polls, but many of them involve methods that might make several cheap and accessible polls less so. Utilizing advanced statistics, analyzing previous similar election events, using machine-learning, and creating “kitchen-sink” models based on voter rolls are established ways to improve the underlying assumptions of polls. But those methods might be a bit too costly and time-intensive for polls that use online surveys and publicly-available annual Census data precisely because they tend to be cheaper than deep research.
Bad models happen, and the very nature of what appears to be the Trump constituency probably made most models worse. Forecasts are best at telling us what old data tells us about new data, and the thing about using existing data is that large deviations in the underlying assumptions of those data may go unnoticed. Those deviations are especially dangerous when they bolster existing confirmation bias among analysts and journalists, but the directionality of that bias is often unclear. Did we all believe Clinton would win because of bad data, or did we ignore bad data because we believed Clinton would win? There’s the question for the ages.
Perhaps the lesson here about the Trump presidency is that it was truly unpredictable. Good models often fail to accommodate events outside of the bounds of their sensitivity, and sounding the alarm on their flaws would necessarily involve knowing or suspecting more about elections than the data we fed the polls.
For many unfortunate Cassandras like Silver himself, caution was roundly ridiculed from this lack of perspective. But if this is the new normal, pollsters will have to adapt in order to maintain relevance.
Edgar G. Said:
The above would make great handle or password.
@ honeybee:
I was originally Edgar G. for many years, until I tried to log in to Israpundit. WordPress just went crazy, refusing it, inducing me to apply for a new password.
I obediently did this about 10-12-20 times over the space of a few months. All were rejected. So I have a collection or rejected passwords which in a 100 years, will be valuable to a collector of same.
Then, by some unexplained “glitch” maybe, my “Austin” password was accepted and that’s how it remained. But since the presently ongoing non-stop Malware attack, the “Austin” password was removed, and rejected on attempted re-installation. So, purely by chance I decided to see if any of the past rejected passwords would do. And the “Edgar G.” password was accepted for the past few days. When I tried to reply to you a few minutes ago, Edgar G. was rejected, so I tried it again and it was just now accepted. “Who knows why, not I..” as a poet might have said.
So after all that tzorrus, I can’t let good old Edgar G’ down so will stick with him. Especially because there is a real story behind my choice of name.
Edgar G. Said:
The truth comes out !!!!!!!!!! How do you wish to be addressed.??
@ Max:
My wife Stefi Harris and I have functioning brains, Max. It has been a long time since we even have had a working television set in our home. The only television we ever see are those accessible through the little Samsung monitors available on the exercise machines at the big fitness gymnasium where we work out every morning. And I use those monitors mainly to listen to the better rock bands of the 1960s and 1970s.
In any case, Stefi and I, along with just about everyone else in our family, pay little or no attention to the mainstresm news media opinioneers.
Among other things about me is that my bachelor’s degree was in journalism and communications, from the University of Illinois at Champaign-Urbana in 1962. My actual experiences was as a working reporter for the Southtown Economist Newspaper chain in the southernmost reaches of Chicago, and as a bureau reporter for United Press International (UPI) in Des Moines, Iowa.
Along with nearly everyone else whom I worked with, I learned to cross-check and verify the validity of everything that I wrote.
Mainstream news media has lost its influence because the news gatherers have become public relations hacks.
Simple as that.
Arnold Harris, Outspeaker
@ Sebastien Zorn:
The Talmudic saying has nothing about “Trust”…or “verify”. It says “Respect (them) but Suspect (them)…” It comes from a story told about Rabbi Yehoshua which I learned in Chaidar so long ago…before I was barmitzvah..
“Trust and Verify”, do not mean anything like “Respect and Suspect”-except maybe very superficially. They have different connotations.
“In Trump we Trust”
…and Breitbart and individual user youtube channels.
Please people, for your own good, unplug your TV’s .
It’s not news , it’s OLD’s. They got nothing worth watching.
Get your social confluence elsewhere.
Get a real friend..
NATO is the fascist ruling class oligarchy’s war lord. They can’t stand an independent Russia that wouldn’t let them loot and pillage wantonly. They tried to but ran into the Russian Mafia. I’m not calling them hero patriots or anything, but here we are. The invisible hand machinations of the ruling class to control global prices, raw materials, finances as nothing more than collateral for their rapacious debt policies and punishments to those who don’t fall in line, are being revealed on a daily basis and are accessible to anyone with an Internet connection. Their game must be conducted in secrecy, but in the electronic global village, there is no secrecy, information is being spilled minute by minute.
The numbers didn’t lie, the press lied. The poll numbers were oversampled among Democrats, period. Then the spin jockeys rode their ideological pogo sticks up the delusional stairway to heaven. Reporters are the lowest form of life in the social petri dish and should be spit upon whenever in public. The smarter ones of us now get our news from independent online sources such as IsraPundit, YouTube and Alex Jones’ InfoWars.
addendum: and “Trust but verify.”
Some pollsters or predictors had it correct. Congrats to:
The analytical team Trump used which saw rural areas would turn out in larger numbers for Trump. They noticed this in the primaries.
The SUNY professor who bases election predictions on change of direction if people do not like the direction.
LA Times/USC Poll (LA Times a really lefty paper who endorsed Hillary) with their larger number of rural voters who used the same 3000 people all the time. It showed Trump 3 at the end. Yet he did not win the popular vote but eked out fairly narrow wins in FL., WI., MI., PA accounting for a large electoral college win(306).
IBD/Tipp poll also had it right in direction TRump plus 2. Yet again TRump did not win the popular vote but the electoral college.
Last but not least the Zohar and Yamit gut poll (Greps or belch poll had it corrrrecttttttttttt.
Well, aside from the deliberate fraud that was revealed here in polling — Thank you, Ted — I recall taking a political science class in college in which we examined the failure of the polls to predict Truman’s victory over Dewey. The consensus was — similar to the explanation of mobile vs. landline here — that in 1948, most working-class homes did not have phones yet and pollsters failed to take that into account.
The article says that it is difficult or impossible to predict electoral outcomes. I am not sure why this is so, since the poll would just have to be in each state, state by state — but the point is moot because I am so glad that they had a hard time predicting the outcome; Made it that much harder to rig.
The Left has used the Global Warming/Climate Change scam to force us to weaken ourselves by de-industrializing our economy.
While I oppose this, I favor strengthening our democratic process by de-industrializing the way we vote and count the votes.
The more decentralized and labor-intensive it is — i.e., no electronics or even machines, everything by hand, results to be counted by hand and checked and double checked by teams consisting of representatives of all the parties and individuals on the ballot, each district phoning in the results and posting hard paper copies in multiple places — the harder to rig.
Redundancy and verifiability is key.
Like what I could have sworn originated as a satirical sign in a 19th Century American cartoon of the sign over the cracker-barrel in the dry good store:
“In God we trust. All others pay cash.”*
*wikipedia.org/wiki/In_God_We_Trust:
Everything about this whichI am finding says it comes from a book by a delightful but forgotten humorist I used to listen to religiously on the radio every night when I was little: Jean Shephard (could I have inadvertently slid** into an alternate reality here?)
https://en.wikipedia.org/wiki/In_God_We_Trust:_All_Others_Pay_Cash
** https://en.wikipedia.org/wiki/Sliders
My old friend WordPress has changed my user-name from Austin to Edgar G. my original name that it refused about a dozen times months ago…So now I’m Edgar G.
I actually found 538 pretty good after a shaky beginning. It was consistently predicting a huge percentage margin for Trump to win, and I spent most of the night alternating between the sad emittances of CNBC and Nate Silver. Although his personal notes displayed certain negativism and surprise, his figures plainly laid out were the exact opposite.
By the way, for the article writer, Clinton did not win the popular vote by a few points. She won by 1/5th of 1%…that is .2 of 1%. According to several reports i looked at especially to find the amount of the lead. It was well within the margin of error and really means nothing. the writer should have just said that the popular vote was pretty evenly split between them. Trump could just as easily have been ahead in the popular count 2-3% also.
Spit-Take!
Why are you even printing this nonsense?
..
THEY LIED. THE POLLS WERE TACTICAL LIES.
NOTHING WENT WRONG WITH THEM!
THEY LIED.
THEY LIED.
THEY LIED.
THEY LIED.
THEY LIED.
THEY LIED.
Does anyone here have a functioning brain?
Wow, what a comprehensive list of possibilities as to how the multitude of polls could all be wrong.. What about the Main Stream Media, selectively publishing polls that supported their bias toward Clinton and against Trump in order to mislead the public into believing that Trump is a loser and a vote for him is a waste.
Even their (MSM’s) interpretation of polls that showed Trump ahead were rationalised away by carefully selected “Expert Political Analysts”.
Not to mention the possibility that the figures were fudged by the MSM exercising their arrogant “moral relativity”. After all Media people consider themselves to be smarter than everyone else and know it all including what’s best for us poor ignorant non journalists.
Nothing they publish in MSM can ever be trusted, true and factual reporting is almost dead except for sites like this one so we can expect them to try and undermine everything Trump tries to do.