Friday, March 21, 2014

Polling of Massachusetts Governor's race still a story of mostly unknown candidates

Democratic ballot test performance driven by name recognition of candidate

On Thursday, March 20 the MassINC Polling Group released a new poll of the Massachusetts Governor's race commissioned by WBUR. While WBUR led with Attorney General Martha Coakley maintaining a large lead in the race, the most important aspect of the poll—the continuing low name recognition of the non-Coakley candidates—was only covered in passing in comments by pollster Steve Koczela in the on-air interview.

In fact, the performance of the Democratic candidates on the Democratic Primary ballot test is defined, and drastically limited, by low name recognition, as the following charts and graph will show. These early polls give us very little information about how these candidates will fair in the September primary or November general election after voters begin to pay attention to the race and get to know and differentiate between the top-tier candidates.

MA Governor name recognition chart

The key number is the 75% of respondents that have an opinion of Martha Coakley, which is the driver for her performance in the primary ballot test against the other Democrats, and in her 41% to 26% lead over Charlie Baker in the general election ballot test. Charlie Baker and Steve Grossman are the next closest candidates with 44% and 29% respectively. The remaining candidates are largely unknown to Massachusetts voters.

The next chart shows the share of the respondents that said they would vote for each of the candidates in the Democratic primary.

MA Governor Dem. ballot test chart

There is a strong correlation between the Democratic Primary ballot test percentages and the Has opinion column in the name recognition chart. A linear regression between the ballot test percentage and the name recognition percentage shows the strength of this relationship. While the amount of data is small, you would expect candidates to be well above or below the line if they were over- or under-performing within the pool of voters who knew them.

Ballot test vs. name recognition

Martha Coakley's name recognition, combined with her favorable/unfavorable numbers are a positive sign for her prospects in the fall elections, and at some point low name recognition will be a problem for the lesser-known candidates. However, history points towards an tightening of the race as candidates who build statewide organizations have an opportunity to make their case to the voters when the election moves into its next stage and voters are actually paying attention.

Monday, March 17, 2014

A comprehensive history of NCAA Men's Basketball Tournament Final Four seeds from 1985 through 2013

Only 16 out of 116 Final Four teams have been seeded greater than four

I am taking my once-a-year break from writing about Massachusetts politics to write a sports analytics column, this one looking at how often teams with various seeds end up in the NCAA Men's Basketball Final Four and go on to win the championship.

While working on your bracket you may be asking yourself questions like, "Should I pick all number one seeds to go to the Final Four?," or "Should I only pick long shots to have a better chance differentiating myself from my bracket pool?" While I don't have definitive answers to these questions, it is instructive to look at how many teams of each seed have made appearances in the Final Four, have gone on to the final, and have won the national championship. This is a comprehensive history from 1985—the year the NCAA moved to a 64 team field—through last year's 2013 tournament.

There are a lot of 1s, 2s, 3s, and 4s in this table. In fact, out of the 116 teams that have made the Final Four since 1985, 100 have been teams with seeds 1 through 4, with only 16 teams having higher seeds. There has never been a team with a seed above 11 in the Final Four. It is important to note that this is likely to change as more games are played within this system—events with low probability do happen. However, teams with the lowest seeds are much more likely to make the Final Four.

Let's break this down by round. Here is a table of appearances in the Final Four by seed.

NCAA Final Four appearances by seed

Of the 116 teams that have made the Final Four since 1985, 47 (41%) have held the top seed, 25 (22%) the 2 seed, 14 (12%) have been 3 seeds, and 14 (12%) have been 4 seeds. The remaining 16 teams (14%) have had seeds from 5 through 11. While it may seem mysterious that a 7 seed or a 10 seed has never made the Final Four, we are dealing with a sample size of only 116 and with a percentage that would likely be in the very low single digits—it is likely just a matter of chance. Later we perform a regression that gives us a feel for what expected frequencies for each of the seeds might be.

The median seed for this set of 116 Final Four seeds is 2, the mean of the seeds is 2.68, with a standard deviation of 2.26.

NCAA Finals appearances by seed

If we consider the seeds of the teams that have made the final championship game, the highest seed is 8 (compared to 11 in the Final Four). There is a drop off from the 1 seed to the 2 seed similar to that in the Final Four round, and only 10% of the teams had a seed below 4. The two 8-seeded teams that made the final game were Villanova in 1985—who went on to win the championship game in spectacular fashion against top-seeded Georgetown—and Butler in 2011. The 2011 Butler team lost to 3-seed Connecticut 53-41.

The median seed of this set is again 2, and the mean of the seeds has inched down to 2.26, with a standard deviation of 1.69.

NCAA Championships by seed

The seeds of the teams that have won the NCAA championship since 1985 are an exclusive set, with only the 1985 Villanova team having a seed greater than four. Well over half the teams that have won the championship (62%) have been 1 seeds. There have been 4 teams (14%) that were 2 seeds, and 4 teams (14%) that were 3 seeds.

The two teams with 4 seeds that won the championship were Danny Manning's 1988 Kansas team, and the 1997 Arizona Wildcats who came back twice from 10 point deficits in earlier rounds. The median of the championship seeds is 1, the mean is 1.86, and the standard deviation is 1.53.

The seed data from all of the rounds comes remarkably close to fitting an exponential decay regression of the form (1/2)^seed. Here are the same data with a fitted regression line. The regression lines indicate some possible outliers like over-performing 4 seeds in the Final Four, and underperforming 2 seeds for the final championship.

Final Four appearances by seed
Finals appearances by seed
Final Four appearances by seed
The 28 years of NCAA 64-team-bracket historical data show clearly that choosing a team with a seed greater than 4 to go all the way and win the championship is hoping for a true long shot. However, chances are that a 5 or 6 seed will win a championship if the tournament continues in its current form.

Sunday, March 9, 2014

Fifth Middlesex Senate seat likely to remain blue

Redistricting changes removed GOP stronghold of Lynnfield and added Democratic precincts in Winchester and Melrose

On Tuesday, March 4, Winchester State Representative Jason Lewis won the Democratic special state primary for the 5th Middlesex State Senate seat, which had previously been held by Congresswoman Katherine Clark. Lewis will face Melrose Republican and Alderman-at-Large Monica Medeiros in the general election—all signs point towards a win by Representative Lewis on April 1.

That is not to say the 5th Middlesex Senate District is a bastion of liberalism. A strong GOP candidate who out-worked and out-raised a Democratic candidate could very well take this seat. However, the lean of the district has moved to the left since it was held by GOP State Senator Richard Tisei, Lewis showed strength in the Democratic primary, and Lewis shows no signs of being out-worked or out-raised.

Much of the pushback on my piece predicting a win by Lewis in the primary and in the general was based on an assertion that the district has a Republican lean and that the previous incarnation of the district was help by Republican Richard Tisei for 20 years. I would like to present some specifics about district's shift to the left in the 2011 redistricting that makes it a tougher win for the GOP.

The Middlesex and Essex District

Congresswoman Clark was originally elected to the State Senate in the Middlesex and Essex State Senate District, the previous incarnation of the 5th Middlesex. Clark was elected to replace Republican Richard Tisei who had served the district for 20 years and who left to run for Lieutenant Governor as Charlie Baker's running mate in 2010.

Before redistricting, the Middlesex and Essex State Senate District consisted of Lynnfield, Malden, Wards 1 through 5 of Melrose, Reading, Stoneham, and Wakefield. We can look at the 2008 performance of President Obama in the precincts of the district to see how they compare to the rest of Massachusetts. The first column shows President Obama's 2008 margin in district precincts in each of the municipalities; the second column subtract's Obama's statewide margin, normalizing it for the Commonwealth; and the last column shows Katherine Clark's margin against Republican Craig Spadafora in the 2010 election to replace Tisei.

Middlesex and Essex Margins 2008, 2010

Lynnfield is one of the most Republican towns in Massachusetts. Reading, Stoneham, and Wakefield are split between more liberal and more conservative precincts. Melrose could be considered about average for the Commonwealth, and Malden is fairly liberal. The district as a whole had an Obama margin of 18.7, which was 7.7 points below the state average of 26.4 points.

In 2010 Clark outperformed Obama in her hometown of Melrose and came out 4.6 points above her Republican opponent Craig Spadafora district-wide. Clark's 4.6 point margin was 12.3 points above the normalized district score of -7.7, a margin we can project into the new district to judge how a Democrat might fair post-redistricting.

Middlesex and Essex precinct-by-precinct Clark margins
(click map for interactive version)

The Lynnfield portion of the map is the darkest red—Clark lost Precinct 4 of Lynnfield by a whopping 35 points. Clark lost Reading and Stoneham by small margins, and won the remaining municipalities.

Redistricting and the new 5th Middlesex Senate District

Each of the United States are required to redraw election district boundaries every 10 years based on population changes shown by the U.S. Census. A holistic view of Massachusetts redistricting shows few signs of gerrymandering, but population shifts did require changes making some districts more conservative, and some more liberal. The Middlesex and Essex State Senate District lost all of Lynnfield, its most conservative piece, and picked up more liberal precincts in the remainder of Melrose and half of Winchester, and was renamed the 5th Middlesex.

While Katherine Clark did not have an opponent in the first post-redistricting election in November 2012, we can look at President Obama's performance in the 5th Middlesex compared to the state average to evaluate the changes. The Projected Dem Margin column in the chart adds Clark's 2010 12.3 point differential to Obama's normalized performance, giving an idea how a strong Democratic candidate might fare.

5th Middlesex Margins 2012

In 2012 President Obama's Massachusetts statewide margin was down to 23.6 points (from 2008's 26.4 points). The 5th Middlesex redistricting added more precincts close to the statewide average—the rest of Melrose and half of Winchester—bringing the district's Obama margin within 3.5 points of the quite-liberal state average. A simple projection of Clark's 12.3 points above the state average gives a ballpark of a Democratic win of 8.8 points for the district, while projecting Democratic losses in Reading, Stoneham, and Wakefield.

It is unlikely that this projection will be realized exactly, resulting in an 8.8 point win by Rep. Lewis. However, the 4.2 point difference between the Middlesex and Essex District's score of -7.7 and the 5th Middlesex District's score of -3.5 can be seen as an approximate measure of the district's move to the left in redistricting.

Had Clark not run for Congress, she would have greatly benefited from the additional precincts in her hometown of Melrose (in addition to the loss of conservative Lynnfield). GOP candidate Monica Medeiros is from Melrose and will likely benefit from having all of her home town in the district. Jason Lewis is likely to get a hometown boost and perform on par with Obama in Winchester.

The following map leaves out these municipality-specific differences and simply gives the precinct-by-precinct projection based on Obama's performance, giving a feel for a generic Democrat's performance in each precinct. While the map contains a lot of red, it will be hard for the GOP to overcome the deep blue of Malden, the left-leaning precincts in Melrose, Winchester, Stoneham, and Reading. One recipe for a Medeiros win would be to win her hometown of Melrose and hold down Lewis's margins in Malden.

5th Middlesex precinct-by-precinct projected margins
(click map for interactive version)

The 5th Middlesex State Senate seat will be won by the candidate who does the best of job of connecting with voters in the next few weeks and getting them to the polls on April 1st. The electoral makeup of the district makes this a readily manageable task for the well-organized Rep. Lewis.

Sunday, March 2, 2014

A rundown of the five March 4 special state primaries

One primary is inconsequential, the remainder will likely determine winner of general election

On Tuesday, March 4 there will be five special state primaries: one to fill Katherine Clark's State Senate seat, and four to fill the State Representative seats of Donald Humason (recently elected to the State Senate), Kathi-Anne Reinstein (left for the private sector), Marty Walsh (recently elected Mayor of Boston), and Eugene O'Flaherty (serving as Boston Corporate Council under Walsh). The primaries are uncontested for the Humason seat, but the winner of the remaining Democratic primaries will likely win the general election.

5th Middlesex Senate

5th Middlesex Senate District Map

The 5th Middlesex State Senate seat was held by Katherine Clark, who was elected to the U.S. Congress to replace Ed Markey, who was elected to the U.S. Senate to replace Secretary of State John Kerry. The ripple effects continue as State Representatives Christopher Fallon and Jason Lewis face HomeStart non-profit director Anthony Guardia in the Democratic primary. Alderman Monica Medeiros is unopposed in the Republican primary.

Christopher FallonDMaldenState Representative$61,350
Anthony GuardiaDWakefieldDirector of HomeStart non-profit$21,795
Jason LewisDWinchesterState Representative$146,908
Monica MedeirosRMelroseAlderman$6,618

President Obama carried the 5th Middlesex District by 20 points and Elizabeth Warren defeated Scott Brown by 2 points in the district, making this seat unlikely to fall into Republican hands.

Rep. Lewis has a strong fundraising advantage over Rep. Fallon. In addition, the Democratic primary voting electorate will likely be more favorable to Lewis's ideology. Fallon had the dubious distinction of supporting Scott Brown over Elizabeth Warren for Senate and Fallon has a much more conservative voting record than Lewis. The one disadvantage for Lewis is the small overlap of his hometown of Winchester—only about 7% of the district votes—while Fallon's home town of Malden has about 27% of the 5th Middlesex votes.

Looking at all of the factors, Jason Lewis is the favorite. The winner of the Democratic primary should not have a problem winning the general election unless there is some strange and unexpected revelation.

13 Suffolk State Representative

13th Suffolk House District Map

There are five Democratic candidates and no Republican candidates for the 13 Suffolk primary to replace newly-elected Boston Mayor Marty Walsh, virtually guaranteeing that Tuesday's winner will be the general election winner.

Liam CurranDBostonCity of Boston Attorney$22,387
Gene GormanDBostonEmerson College Professor$9,795
Daniel HuntDBostonMSBA Project Analyst$104,568
Paul McCann Jr.DBostonBoston Public Health Attorney$15,070
John O'TooleDBostonCity Council Candidate, Plumber$25,245

While this is a particularly strong field of candidates, Daniel Hunt has a big enough fundraising advantage to be considered the favorite. The Dorchester district, like much of Boston, is incredibly Democratic with Obama winning the district by 53 points, and Elizabeth Warren by 40 points.

4th Hampden State Representative

4th Hampden House District Map

There is only one candidate in each of the Democratic and Republican primaries to fill Republican Donald Humason's seat who was elected to the State Senate in November. Democratic candidate John Velis is a recently retired Army Captain and attorney who will face off against Republican City Councilor Dan Allie. President Obama won the 4th Hampden District by 6 points, but Elizabeth Warren lost it by 13 points, making this seat winnable by either party.

John VelisDWestfieldArmy Captain (Ret.), Attorney$9,740
Dan AllieRWestfieldCity Councilor$10,960

16th Suffolk State Representative

16th Suffolk House District Map

Revere State Representative and Assistant Majority Leader Kathi-Anne Reinstein resigned in January to take a position with the Boston Beer Company. There are three Democratic candidates running in the primary and Reinstein Chief-of-Staff Roselee Vincent has shown fundraising power and has received endorsements from groups like the Massachusetts Women's Political Caucus—she can be considered the favorite.

Joshua MonahanDChelseaAttorney$6,718
Linda RosaDRevereFormer Bob Travaglini staffer$5,100
Roselee VincentDRevereReinstein staffer$42,598
Todd TaylorRChelseaStaffing company owner$7,115

The 16th Suffolk District went for Obama by 28 points and Warren by 14 points, so the Democratic winner should prevail in the general election on April 1.

2nd Suffolk State Representative

2nd Suffolk House District Map

Chelsea State Representative and Judiciary Committee Chair Eugene O'Flaherty left the State House for City Hall, becoming the chief lawyer for Boston Mayor Marty Walsh. There are three strong candidates to replace O'Flaherty.

Roy AvellanedaDChelseaFormer City Councilor$28,460
Christopher RemmesDBostonRealtor$49,897
Daniel RyanDBostonCapuano District Aide$47,360

It is not at all clear who will win this primary and become the next State Representative for the 2nd Suffolk District. While Avellaneda had the smallest fundraising haul, he received a ringing endorsement from the Boston Globe.