Selection Bias and Voluntary Drug Testing Part II

In August of 2011, a post of mine addressed the policy of drug testing welfare recipients. At that time, much was made of the fact that Florida’s mandatory drug testing policy produced just a 2% failure rate.  From this statistic, Tampa Bay Online concluded:

The initiative may save the state a few dollars anyway, bearing out one of Gov. Rick Scott’s arguments for implementing it. But the low test fail-rate undercuts another of his arguments: that people on welfare are more likely to use drugs.

Irrespective of whether the policy of drug testing welfare recipients makes sense, we just cannot draw these kinds of conclusions from the data.  Indeed, the data suffers from a problem known as “selection bias.”  Simply put, individuals most likely to fail a drug test are not going to take the test.  We just can’t generalize about the entire population of welfare recipients from this severely self-selected sample.

In the current version of this story, Utah has just published it’s version of the same data. Upworthy has the results in a not-so-subtle pie chart.  Here are the findings:

 

 

 

 

 

 

 

 

 

 

 

Yeah.  Wow.  Again, it’s reasonable to conclude that this policy may be “unfair” or a “poor use of taxpayer dollars” given the low rate of positive results.  But unfortunately the author of the post draws the same faulty conclusion as in the Florida case.  For us to conclude that welfare recipients (the population of interest) have a lower incidence of drug use compared to the state, we would need a random sample where every individual in the population has an equal, non-zero chance of being selected.  We don’t have that here.

| Leave a comment

Quantifying Bill’s Value to Hillary’s Campaign (Part I)

In presidential elections, relationships matter. 

For example, political scientists know that the relationship between economic conditions, the number of causalities in war, and the incumbent’s party affiliation explain the bulk of presidential election outcomes.

In the 2016 presidential election, however, there is another “relationship” worth keeping an eye on.  But rather than the correlation between two variables, this relationship is of the social variety.  I’m referring, of course, to the marriage of Bill and Hillary Clinton.

Pundits on the left claim that Bill Clinton is an asset to Hillary because he brings legions of faithful supporters and has a high approval rating.  Pundits on the right claim that Bill Clinton is a liability because he reminds voters of the Clintons’ personal affairs.

But when we strip this rhetoric down to its core, both sides are making the same empirical point: As goes Bill, so goes Hillary. 

But is it true?  Are the Clinton’s’ “married” in the minds of voters such that the opinion of one affects opinions of the other? 

In the figure bellow, I plotted approval ratings from Gallup.com (here and here) and Pollster.com (here) from 1992 to 2014 and created smooth trend lines for both Clintons.  According to the figure, the answer would seem to be very clearly “yes.”  We can see that Bill’s approval rating moves up and down alongside Hillary’s approval rating. 

Approval

Surprised?  No?  Let’s get a little deeper into the data.

For starters, two variables can move up and down together without being causally related.  In fact, causal relationships are notoriously hard to identify in non-experimental data (see an old post of mine on the relationship between Nickleback, Herpes, and Obama’s vote share in 2012).  Indeed, other factors could be causing the above patterns.  In other words, it’s possible the above patterns are a “spurious relationship.”  But also, the question here is about an individual-level relationship (what happens in the minds of voters).  Inferring an individual-level relationship from aggregate data can lead to what’s known as an “ecological fallacy.” 

In short, we need better data.

Fortunately, the American National Elections Studies (ANES) has been conducting surveys for every presidential election from 1948-2012.  We can easily download the ANES dataset and quickly produce some answers to this question.

Let’s explore the relationship between opinions of Bill and Hillary Clinton in 2000.  Respondents’ opinions are measured using a “feeling thermometer” where a score of “100” indicates the highest possible approval of the Clintons while a score of “0” indicates the lowest possible approval.  A simple regression analysis will tell us if these two variables are indeed related and whether that relationship is statistically meaningful or not.  Here are the results:

Reg1

I’ll skip the boring statistical details, but basically the regression model confirms what we see in the approval data.  It would seem that that there is a very strong positive relationship between opinions of Bill and Hillary Clinton (indicated by the number 0.77 in the column “Coef”).  We can also see that this relationship is statistically meaningful (indicated by the number 38.95 in the column “t”).  In sum, as Bill’s approval goes up, Hillary’s increases, and as Bill’s approval goes down, Hillary’s declines as well.

But what about all the “other factors” that could be causing opinions of both simultaneously?  For example, perhaps both are caused by views of Democrats in general, raw party identification, or the performance of the economy.  In the social sciences, we call these “control” variables.  

In the model below, I added five control variables (opinions of the Democratic Party, opinions of the economy, a respondent’s party identification, ideology, and gender).  Here are the results:

Reg2

Among the control variables, Democrats, respondents with a favorable opinion of Democrats, and women all have a higher opinions of Hillary Clinton.  None of this is surprising, but again, it’s important to account for these relationships.

But what’s most notable about the results is the magnitude of Bill’s effect.  Indeed, from the above results we can quantify Bill’s value to Hillary’s campaign (as the title of this post suggests)  In particular, because the coefficient on the “Bill” variable is 0.465, the model indicates that as Bill’s approval rating increases by 1 unit, Hillary’s approval increases by just under ½ in the same direction

So while it’s not a 1-to-1 relationship, Bill has a sizable effect on how people view Hillary.  Moreover, we when look at the last column on the right (labeled “beta”), what we see is that Bill’s effect is larger in magnitude than any other variable in the model.  So not only does Bill matter, but he matters quite a bit.

Interested in one more model?  Actually, you don’t have a choice. 

For some additional context on the magnitude of Bill and Hillary’s “statistical” relationship, I wanted to see what happened if we used the same model to predict opinions Al Gore.  Here are the results:

Reg3

We would expect Bill Clinton to have an effect on Al Gore’s approval rating given that they shared the White House together (remember, these data were collected in 2000).  And indeed, that’s what we see in the results.  However, what’s notable is that the magnitude of this relationship has decreased by about 30% from what the same model predicts regarding Bill’s effect on Hillary (from 0.465 to 0.320).  In short, the statistical relationship between Bill and Hillary’s approval ratings is larger in magnitude than the statistical relationship between Bill and his Vice President. 

What’s the big takeaway?  In short, yes, opinions of Bill Clinton seem to sway opinions of Hillary.  While we can’t say definitively this is causal (for example, causality could go the other way, with opinions of Hillary could be affecting Bill’s), this relationship persists even when we control for various factors.  Perhaps most importantly, the effect is surprisingly large in magnitude.  It would seem that Hillary earns about a ½ increase or decrease in her approval rating for every 1 unit change in Bill’s approval rating.  In the 2016 campaign, the so-called “Bill factor” would matter quite a lot.  

For next week’s post:

What happens after Bill left office?  Does this relationship exist in, say, 2008?

Does this statistical relationship translate into actual votes?

Is Bill’s effect strongest for Democrats, Independents, or Republicans? 

Posted in Political Behavior, The Presidency | 2 Comments

No, the Senate did not block the House Border Bills – Yet

The Washington Examiner reported today, “in a matter of minutes, Senate Democrats on Tuesday blocked a pair of House bills that would have provided nearly $700 million to deal with the surge of immigrants on the southern border and blocked President Obama from expanding an anti-deportation program.”

Problem is Senate Democrats didn’t block the bill. In fact, they haven’t yet even delayed the bill. What occurred today was a routine Senate scheduling process.

Here’s why: Senate rules require all bills (and J.R.s) be read twice before further proceedings. In almost all cases, further proceedings would be to refer the bill to its committee of jurisdiction.

However, there is a procedural trick that allows the Senate to bypass committees and place the bills directly on the calendar where the full Senate may consider it. If a Senator (normally the majority leader or a Senator acting on his behalf) asks unanimous consent for a second reading and objects to further proceedings, the bill bypasses referral and moves directly to the calendar. This process actually expedites consideration of bills. In today’s Senate, most big bills are considered under this process. Rather than work through committee, senators reach agreements on legislation and bring it straight to the floor.

In other words, Democrats (in this case, Senator Coons) have not yet blocked the House border supplemental and deferred action bills. Rather, they have begun the process will allow those bills to come before the full Senate more quickly than if it was referred to committee. So both bills will be available for Senate consideration on the first day the Senate returns from recess.

Of course, this process in no way guarantees the Senate will actually consider either of these pieces of legislation. But it is important to note the Senate has not blocked the House bills – at least not yet.

| Leave a comment

Yes, Democrats can win the House (Though Probably not This Year)

Chris Cillizza wrote a piece titled, “Why it’s going to be hard for Democrats to win back the House this decade.” He makes the argument that with a declining share of competitive seats it will make it harder for Democrats to retake the House. The declining number of competitive seats is a big problem. However, it’s not the reason Democrats won’t have the House majority in 2015.

There are a few points to discuss. First, Cillizza argues the decline in competitive seats is due to the redistricting process. He particularly points out 2001 and 2011 as important moments when seat competition declined. It’s important to note that redistricting does have an effect on House elections. The Monkey Cage estimated that redistricting cost Democrats roughly 7 to 10 seats in the 2012 election. Gerrymandering can make some seats safer and it can also alter seat totals in the House.

That said it is probably not the major reason competitive seats are in decline. For one, the decline in competitive districts goes back several decades. David Mayhew first pointed out the case of the “vanishing marginals” in 1974, noting that competitive districts had been on the decline since at least 1956. While it’s possible gerrymandering has had a role in this development, most political scientists believe the overall effect of gerrymandering is a wash. While it may make some seats safer, it often makes others more competitive.

Second, partisan gerrymandering can’t explain low electoral competition across the U.S. This argument is not explicitly made in the piece but it is often assumed partisan gerrymandering hurts competition. However, even in states without partisan gerrymandering congressional competition is weak. For example, California uses a nonpartisan commission to draw its congressional districts. In the 2012 election, after congressional district lines were redrawn, party control changed hands in only three of California’s 53 House districts. In other words, 94% of California’s House districts remained unchanged. Both partisan and nonpartisan redistricting practices suffer from lacking party competition.

Lastly, Democrats will not lose the 2014 House election due to lack of competitive races. The Cook Political Report and the Rothenberg Political Report estimate there are roughly 43 to 36 competitive races in this election cycle, respectively. Democrats are 19 seats short of a majority in the House. Numerically speaking, there are more than enough competitive races for Democrats to have a fighting chance at the Speaker’s gavel in the 114th Congress.

In other words, Democrats’ problem is not a lack of competitive seats. It is the electoral conditions they face in 2014. A sluggish economy, low presidential approval, the mysterious 6-year itch, when historically voters punish incumbent presidents at the polls, and low-turnout will likely plague Democrats in 2014. Under a good economy, high presidential approval, and high turnout their chances would be much better. Democrats almost certainly won’t win the 2014 House election. But their future chances are still very much up for grabs.

Bottom line, Democrats almost certainly will not win the 2014 House election but that won’t be because there are not enough competitive seats.

Posted in Electoral Institutions, Political Behavior, Political Parties | Leave a comment

Seven Numbers to Remember About the VA Compromise

According to multiple sources, Representative Jeff Miller (R-FL) and Senator Bernie Sanders (I-VT) have reached a tentative agreement on a bill to overhaul the Veterans Affairs health care system.  A news conference is scheduled for 1:30 today.

[edit: Confirmed.  Details of the compromise bill can be found here.]

For the agreement to become law, it will need to be approved by a conference committee and subsequently passed by the full House and Senate.  Even though reforming the VA is widely considered a political no-brainer, up until this morning a compromise seemed unlikely.    But while various hurdles remain, in my view there are reasons to be cautiously optimistic about the odds of enactment.

Here are 7 numbers worth remembering:

5 days: the time remaining before Congress adjourns for its August recess.  Simply put, if the VA compromise isn’t sitting on the president’s desk by Friday, further action will be delayed until mid-September.  While this may seem like a bad thing, there are two important points to remember.  First, the deadline is probably good news for getting the compromise enacted.  Congress regularly faces deadlines (such as adjournment and expiring legislation) and empirical studies have shown that such deadlines can actually increase the likelihood of bill passage (see for example here).  Second, while most people respond negatively upon hearing that Congress is on “recess,” it’s important to keep in mind that lawmakers have two core jobs.  While policy creation is most visible, lawmakers also meet with constituents and fulfill their representative responsibilities during these so-called “breaks.”  So even though Congress is dysfunctional, legislative recesses are not a reason why.

3 day rule: a parliamentary rule requiring legislation to be available for three calender days before it can be considered by the entire House.  So even though the looming recess may compel lawmakers to act quickly on the compromise, the three day rule could be a significant barrier.  Of course, the rules of both chambers can be waived, so this isn’t an insurmountable hurdle.  Expect a vote late in the legislative week.

3 votes: number of votes short of adopting a motion “instructing” House conference committee members to pass the Senate’s bill.  Quick background: When the House and Senate pass competing bills, as they have on the VA issue, a conference committee is tasked with merging the competing proposals (think of it as a smaller “super committee“).  Last Thursday,  House Democrats came within three votes of passing a non-binding motion telling the conference committee to simply pass the Senate’s bill.  Given the narrowness of this vote, where thirteen House Republicans joined all Democrats in supporting the other chambers plan, its clear lawmakers in both parties want to get a deal done.

128 laws: number of bills enacted into law in the 113th Congress.  At the same point two years ago, the total was 150 law.  And just a decade ago, the 107th Congress (which also had split chambers) had passed 203 laws at the end of July.  In sum, while the specifics of the compromise plan make me optimistic about the reform proposal’s fate, the larger historical trends suggest greater caution is needed.  If the reform proposal somehow fails, it will be yet another bill in the graveyard of the “do nothing Congress.”  Which bring us to the next number…

80% disapproval: the percentage of Americans who hold an unfavorable opinion of Congress.  Yes, there are many reasons for Congress’s low approval rating (which we detail here, here, and here), including the inability to pass major legislation.  And though each party is viewed unfavorably, both the net favorability and the generic ballot favor Democrats.  In sum, while both sides might object to certain elements of the Sanders-Miller compromise, there is significant external pressure on members of both parties (but particularity Republicans) to pass a bill before the August recess.

$17 billion: the total cost of the package.  In their negotiations, Sanders requested $25 billion while Miller was asking for $10 billion.  So while this is certainly a compromise, the financial aspects of the agreement are closer to Miller’s proposal.  However, the most important detail may be the fact that the $15 billion is “capped.”   In other words, once the allocated money is spent, the VA will have to request further funding from Congress.  According to one CBO report, the actual cost of expanded care (including the provision allowing veterans to use private medical facilities) will be around $50 billion.  So like the debt ceiling, we could be right back here a year from now.  [edit: the original post, published before the announcement, listed the price tag at $15 billion.]

99 days: the number of days before the 2014 midterm election.  How will the passage (or failure) of VA compromise proposal affect the balance of power in Washington?  Stay tuned…

Posted in Bicameralism, Legislative Politics, Political Parties | 1 Comment

An Ideological Mapping of South Carolina’s Senate Candidates

Flag-map_of_South_CarolinaWhen it comes to politics, South Carolina is full of intrigue. From Lee Atwater’s Southern Strategy and the 2000 Republican primary to Joe Wilson’s “You Lie!” and Stephen Colbert’s rally with Herman Cain, the Palmetto State routinely produces compelling political storylines.

In the most recent iteration, South Carolina has not one, but two (!) Senate contests this fall. Lindsey Graham’s race has garnered the most attention.  Until last week, Graham had just two challengers: Democrat Brad Hutto and Libertarian Victor Kocher.  But on July 14th, Thomas Ravenel submitted 17,000 signatures in his attempt to qualify as an independent candidate.  Ravenel is perhaps best known for three things: (1) for being the son of Arthur Ravenel Jr., a well connected state politician from whom the bridge in Charleston is named (2) for being the star of the Bravo reality series “Southern Charm” and (3) for his 2007 arrest on federal cocaine distribution charges.

Needless to say, Ravenel’s legal troubles have received the lion’s share of attention.  But as an old post document, research has shown that political scandals don’t matter as much in congressional elections as most people assume.  If Ravenel loses, it’s not because of his legal past.  Rather, election outcomes are often easily explained by the fundamentals: things like name recognition, party identification, the economy, and the ideological positioning of candidates relative to voters.

Which brings me to the topic of today’s post…

Given South Carolina’s deep-red hue, candidates regularly position themselves as the “true conservative” and label their opponents as “moderates” or worse in Lindsey Graham’s case, “liberals.”

Who’s “conservative” and who’s “liberal” in South Carolina’s Senate election?

I feel a chart coming on…

Graph

In the above chart we can see the relative ideological position of each candidate in South Carolina’s general Senate election.  At “zero” a candidate is considered a perfect moderate while liberals and conservatives are located in their familiar left-to-right orientation. Both distributions represent the ideological positioning of all Democratic Senate candidates (blue) and all Republican Senate candidates (red) from 1980 to 2012. Each candidate’s initials (and the corresponding vertical line) denote their position in the spectrum (BH=Hutto, LG=Graham, TR=Ravenel, and VK=Kocher).  Finally, because ideology is a tenuous construct, the chart includes Tim Scott (TS), Jim DeMint (JD) and Fritz Hollings (FH) for comparison purposes.

Some notable findings:

  • Lindsey Graham is indeed the most liberal Republican.  Still, it’s important to keep in mind that his positioning is firmly in the middle of all Republican candidates.
  • Ravenel is more conservative than Graham.  In fact, he’s roughly as conservative as both Tim Scott and Jim DeMint (though still just to their left).  Ostensibly, you could say that Ravenel would be a good ideological fit for state-wide office.
  • While Brad Hutto is a Democrat, he’s actually on the conservative side of the aisle.  In fact, we can see that Hutto is almost a mirror image of Fritz Hollings.  While both Hollings and Hutto would be considered “moderates” in the above spectrum, as they are closer to “zero” than either party’s median, Hutto is equally conservative as Hollings is liberal.
  • Tim Scott and Jim DeMint occupy almost exactly the same ideological position, which is interesting given that Scott was appointed to replace DeMint when he suddenly retired.
  • Finally, Kocher is located in the far right of the distribution, which makes him exceptionally conservative.  In particular, Kocher is more conservative than 98.7% of all candidates.

 

Methodological Details:

The data for this figure come from the Database on Ideology, Money in Politics, and Elections (also known as “DIME”) and have been made available by Stanford political scientists Adam Bonica.  In brief, Bonica’s dataset uses campaign donations to “scale” all candidates in the same policy space.  You can access the data and further information here.

Posted in Elections, Political Behavior, Political Parties, Voting Behavior | Leave a comment

Who’s the Worst President? Evaluating the Quinnipiac Poll

Quinnipiac University’s “worst president” poll got a lot of press. Washington Post, the New York Times, CNN, Fox, and virtually every other news outlet have carried the headline, “Obama is the Worst President since WWII.” This particular survey question is press-chum. The survey’s designers likely knew it would get the press’s attention and included it for that purpose. Mission accomplished. The problem is it tells us practically nothing.

For one, this question lacks consistent measurement. Without consistent measures it is difficult to trust the “marginals” (raw percentages of agree/disagree, approve/disapprove, etc). The reason is that any single survey is subject to a variety of errors and biases. The sample can be biased. The survey’s respondents may not represent the actual population. Question wording can bias results. For example, research by Tom Smith (1987) shows questions using the words “welfare” rather than “poor” elicit more negative responses on surveys.

Similarly, question ordering can bias results. For example, offering respondents a question about unrest in the Middle East followed later by a question asking the respondent to judge the president on foreign affairs can bias responses. In this case, the surveyor has primed respondents with hostility and violence overseas. This can lead to more negative responses than if the survey leads with questions about foreign policy successes. (I haven’t looked through the rest of the poll, but the “worst president” is question 36 in this survey.) This can make interpreting the marginals without a track-record difficult for researchers or, in this case, the public.

However, the real crux is the nature of the question itself. The question effectively asks respondents to compare current events to past events. This flies in the face of what we know about survey response. John Zaller, often in collaboration with Stanley Feldman, illustrated that respondents answer questions based on what is at the “top-of-their-head.” Recent stimuli affect the attitudes respondents use to answer a question. For example, if you ask somebody today about their thoughts on politics, you are likely to receive an opinion on issues most recently in the news cycle, like unaccompanied children crossing the border into the US. If you asked the same question a month ago, the individual likely talked about another issue entirely, one that was popular at that time. And if you ask it five months from now, these issues may not register at all.

This makes retrospective questions like the one in the Quinnipiac poll biased against sitting presidents. Obama’s rank as the “worst president since WWII” is currently plagued by lost emails at the IRS, House Republicans’ accusations of executive overreach, and a sluggish economy, among other things. Previous presidents are free from current events. Bush’s approval is no longer burdened by the Iraq War, which helped lower his approval to 25% at one point. Harry Truman’s faults in office have evidently been forgiven. He tops the list as the least-worst president despite having the lowest recorded job approval in Gallup history (22%). Reagan’s legacy is no longer burdened by the early-1980s recession, which dragged his approval into the mid-30s. Nixon’s memory has evidently escaped Watergate. He fairs better than Obama and Bush II despite leaving office with only 24% approval. Clinton’s memory has certainly escaped his improprieties in the Oval Office.

It’s unsurprising that the two times this question has been asked (in 2014 and 2006) the sitting president has been recorded as “the worst president.” This isn’t to say it will always be the case. However, our memory of previous presidents fairs much better than their actual job approval during their presidencies. The events dragging down their job approval are long over.

Ultimately, this question wasn’t necessary and it is a prime example of when not to trust the marginals. Simple job approval/disapproval ratings are far more accurate at capturing presidential disapproval. And further, they are far less misleading.

Originally posted to reviseandextend.com.

Posted in Political Behavior | Leave a comment