Thursday, November 6, 2014

Massachusetts gubernatorial polls were less biased than national midterm polls

A look at the polling average and a comparison of each pollster's last poll to the final result

While a comprehensive look at the polls of the 2014 midterm races show that the polls were biased by several points towards Democrats, that did not seem to be the case for the totality of the Massachusetts gubernatorial polls of the race between Martha Coakley and Charlie Baker. The final Mass. Numbers time-weighted polling average showed Baker up by two points, essentially equalling the unofficial election result of a 1.88 point Baker win, and a simple average of the last poll from each pollster shows a similar result of Baker +2.3.

While the average was very close to the election result, the final polls of the Massachusetts gubernatorial race showed a reasonably wide range from a 4 point lead for Martha Coakley, to a 7 point lead for Charlie Baker.

The following table gives details of each pollster's final poll sorted by difference from the election result. The polling methods included internet panel, live phone interviews, automated phone calls (IVR), and IVR supplemented with an internet panel to reach non-landline respondents.

MA Governor polls vs. result (chart)

The YouGov poll showing Coakley up 4 was the biggest miss of the final polls with a difference of -5.88 points from the actual result. The Globe/SocialSphere poll had Baker up 7 (a 5.12 point difference), the UMass poll missed by -4.88 points, and the Emerson College Polling Society poll missed by 4.02 points.

The closest poll to the final result was the Rasmussen IVR/Internet poll 22 days before the election which had an almost perfect 2 point Baker margin. MassINC Polling was also close to perfect with Baker +1, closely followed by Suffolk with Baker +3. UMass Lowell was just over 2 points off (Baker +4), and WNE had Baker +5 for a miss of 3.12 points.

While some of the polls of the Massachusetts gubernatorial race were remarkably accurate and others missed the mark by 5 or 6 points, the overall picture provided by the polls in the form of an average gave us a good idea of the eventual result.

Wednesday, November 5, 2014

Money from Massachusetts GOP shadow group fails to unseat targeted state representatives

Massachusetts Fiscal Alliance spending on immigrants-over-veterans mailings had little affect on race outcomes

An outside spending political action committee run by Republican State Committee member Rick Green and called the Massachusetts Fiscal Alliance portrayed itself as an issues-based committee concerned with fiscal discipline. But mailings undertaken by the committee against a group of 19 Democratic State Representatives made it clear that the goal of the group was to campaign against those house members and support of GOP challengers using a debunked immigrants-over-veterans attack.

The mailings seem to have failed as 17 of the 19 representatives were re-elected, several in very GOP-leaning red districts. A loss by Rep. Rhonda Nyman to GOP challenger David DeCoste was in the 5th Plymouth District where President Obama lost by 8 points, Senator Warren lost by 26 points, and is ranked the third most GOP-leaning Rep. district in the Commonwealth. This is a hard district for any Democrat to hold.

The other loss came in the more Democratic 2nd Franklin District. Rep. Denise Andrews of Orange lost to two-time challenger Susannah Whips Lee. Lee outraised Rep. Andrews in the pre-primary and pre-election reporting periods, raising $44,698 compared to $16,960 for Andrews.

Election results for MFA targets

The success of Democratic incumbents Josh Cutler, Colleen Garry, Jim Cantwell, Ted Speliotis, and Jim Arciero in Republican-leaning districts shows that the PAC mailings were not an effective campaign tool for the GOP.

Monday, November 3, 2014

A data supplement to legislative races to watch article

District partisan lean and pre-election fundraising data for competitive legislative races

Gintautas Dumcius and Mike Deehan put together a great overview of some of the interesting Massachusetts races for Massachusetts State Senate and Representative available on WBUR Poll Vault.

While Dumcius and Deehan touched on fundraising and district partisanship in the article, I put together a table that has each candidates fundraising total for the OCPF pre-election period, and the Obama and Warren margins for each district for your analyzing pleasure.

Legislative races to watch

Here is an online spreadsheet with the same data.

Not all survey methods are equal in 2014 midterm race polling

Many midterm races have seen a similar method-based bias as the Massachusetts gubernatorial race (internet+Dem, IVR+GOP), but others show the opposite

There have been significant and consistent differences between IVR, live operator, and internet polls of the race for Massachusetts governor with internet polls showing a Coakley +4 lead, live operator polls showing Baker +2, and automated phone (IVR) polls showing Baker +5. But is this internet+Dem/IVR+GOP bias seen across other 2014 midterm polling?

In order to answer this question I downloaded all of the October polls for a large sample of reasonably competitive midterm Governor and Senate races from the Huffington Post Pollster site, computed a polling average for each race, and then compared the race-wide average to the average for each survey method, recording a difference as positive number for Democratic bias and a negative for a Republican bias.

The results are in the following table with polling methods including live phone calls, automated phone calls (IVR by itself, or with supplements live phone calls or internet panels to reach non-landline respondents), and internet-based survey panels.

2014 Midterm survey bias by method (chart)

The results show an internet+Dem/IVR+GOP bias similar to MA-Gov in the Colorado, Maryland, Maine, and Michigan gubernatorial races, and also in the Colorado and New Hampshire Senate races.

However, some races show exactly the opposite, including Alaska, Georgia, and Michigan Senate, and Arkansas, Arizona, Florida, and Kansas Governor. Michigan is particularly strange with the Governor and Senate race on opposite sides.

A look at the distribution of differences for all of the races for each survey method provides some insight into the race-by-race biases.

2014 Midterm survey bias distribution by method (graph)

The histogram for live phone polling sows a nicely shaped normal distribution centered at the average margin, with a similar probability of differences above and below the mean.

Internet polls are skewed to the Democratic side, but shifted to the left past the mean of zero, possibly a result of the sample weighting. On the other side, IVR polls are somewhat skewed to the GOP side, but shifted to the right past the mean of zero, possible due to weighting.

The IVR/Online polls look closer to a normal distribution, but the IVR/Live Phone polls have very irregular-looking distribution shape.

A possible explanation differences in bias for some of the races could be weighting overcompensation for Democratic bias in the internet polling and for GOP bias in the IVR polling.

After Tuesday's election results it will be possible to determine which survey method type was most accurate in the midterm polling and look for explanations of the affect of survey methods on poll bias.

Friday, October 31, 2014

There are significant differences between IVR, live operator, and internet polls of the race for Massachusetts governor

Which polling methodology is seeing the right set of voters won't be known until after Tuesday's election

As the polls of the race for Massachusetts Governor have steadily moved from a Martha Coakley lead in early September to a Charlie Baker lead in late October, a complicating factor has arisen: each of the three polling methodologies used to survey the race are showing different results. Interactive Voice Response (IVR) or robopolling firms show a reasonably large 5 point lead for Baker. Traditional live operator telephone polling shows Baker with a 2 point lead. And internet panel surveys show Coakley with a 4 point lead in the race.

MA Gov. polls by type (graph)

MA Gov. polls by type (chart)

Which of the methodologies are seeing the appropriate set of voters? IVR-only firms like Emerson are not able to reach large swaths of the population without landlines, meaning fewer younger, urban, and non-white voters. Live operator telephone polling suffers from severe non-response effects as more and more people screen their phone calls. Internet surveys are the new kids on the block and have their own (probably smaller) set people who cannot be reached.

Pollsters of each persuasion use best-practices and weighting to compensate for these effects, but it seems that each of the methodologies is getting a different view on the Coakley/Baker race. If the IVR firms and/or the live operator pollsters are having trouble seeing all of the Democratic voters, Baker could be in for a Tuesday shocker. If the internet pollsters are getting it wrong, Martha Coakley will suffer another tough loss and Charlie Baker will be the next in a long line of Republican governors of Massachusetts.

Wednesday, October 29, 2014

A comprehensive look at polling of all four ballot questions provides an indicator of likely results

Baker disagrees with voter consensus on gas tax and earned sick time while Coakley and the average voter disagree on bottle bill expansion

While the occupancy Massachusetts' corner office is still up for grabs—the race between Martha Coakley and Charlie Baker remains too close to call—a comprehensive look at the last two months of polling shows a strong enough voter consensus on the four ballot questions that a likely result can be predicted for each initiative.

The following summary table shows a polling average for each question, whether the measure is likely to pass or fail, and the position of the two major gubernatorial candidates. The remainder of the article gives a comprehensive look at the last two months of polling on each question.

Ballot question summary

A plurality of survey respondents and candidate Coakley are against gas tax indexing repeal (by a relatively small average margin of four points) while Baker supports indexing repeal. Voters, on average, are against bottle bill expansion as is GOP candidate Charlie Baker, while Democrat Martha Coakley sides with environmental groups in favor of bottle bill expansion. Most voters oppose repeal of the casino law, as do both gubernatorial candidates. Question 4 on earned sick time is very popular among voters (with an average margin of 25 points) and is also supported by Coakley, while Baker opposes the earned sick time initiative.

Question 1 - Eliminate gas tax indexing

Ballot Question 1 on repeal of automatic gas tax indexing is the only initiative that has shown some polling inconsistency. That being said, only one public poll in the last two months has shown a positive margin for repeal, and the most recent polls have 14 and 7 point margins against repeal. There is some evidence that survey respondents have been confused by the meaning of "Yes" or "No" in the context of this question in earlier surveys, but there is also evidence that voters are starting to figure it out, based on the fact that Republicans are more likely to be for repeal, and Democrats against. This is the only question where there is a reasonable probability of a result that goes against the polling consensus.

Ballot Q1 polling

Question 2 - Expand bottle bill

The polling of Ballot Question 2 on expanding the types of containers with a 5 cent deposit is definitive. Expansion proponents have not led in a public poll in the last two months and the margin seems to be increasing with a whopping 46 point margin in the latest poll from UML. One reason may be the difference in the money spent by environmental proponents and the large retail stores who oppose the question.

Ballot Q2 polling

Question 3 - Repeal casino law

It does not seem that voters are in favor of repealing the Massachusetts casino law that allows for the creation of several resort-style casinos in the Commonwealth. Ballot Question 3 has been behind in every public poll in the last two months, by an average margin of 15 points.

Ballot Q3 polling

Question 4 - Require earned sick time for most employees

Polling of likely voters has shown consistent support for Ballot Question 4, which would require employers to provide earned sick time for most Massachusetts employees. The initiative has led in every public poll in the last two months, with an average margin of 25 points. It would seem reasonable for Coakley to use Baker's opposition to this popular measure against him, which she started to do in debates, but there has been no large ad campaign drawing the distinction.

Ballot Q4 polling

Thursday, October 16, 2014

Polling methodology seems to be playing a roll in Massachusetts gubernatorial survey results

IVR polls are 6 points better for Baker/GOP

Polling averages of the Massachusetts gubernatorial race between Republican Charlie Baker and Democrat Martha Coakley show Coakley with a small 2 point lead, but there seems to be a major difference between the polls conducted using IVR (Interactive Voice Response) methods, and the polls that use traditional polling methods of operator-based dialing, or newer Internet-based survey techniques.

IVR polling has the significant disadvantage of only being able to reach respondents with land-line telephones—federal law disallows robocalls to cell phone numbers. IVR pollsters like Rassmussen and Massachusetts-based Emerson College Polling Society use various sampling and weighting techniques to attempt to compensate for their limited reach.

Breaking down all of the Massachusetts gubernatorial polls conducted since the September 9 primary by polling methodology (Internet, IVR, and Traditional) shows a large gap between the non-cell-phone IVR polls, which have Republican Charlie Baker up 3 points, and the traditional and internet-based polls, which have Democrat Coakley up by 3 points. The boxplot below gives a visual representation of the poll distributions.

There is some chance that these variations are due to sampling error or other causes, but the variations could very well be due to the IVR pollsters not reaching cell-phone-only voters who tend to vote more for Democratic candidates.
MA Gov Polls by polling method (graph)

MA Gov Polls by polling method (chart)