Monday, January 26, 2015

How America Responded to All 339 Lines in the State of the Union

For non-Americans: the State of the Union is an annual address in which the president outlines how the country’s doing and the agenda for the future.

Which parts of Obama’s address particularly resonated with the public? One can judge the congressional response to each line by listening to applause in the chamber, but it's harder to know what the country as a whole thought. One way to find out is to look up every single sentence in the speech on Twitter and study each response. Yes, this took a while, and yes, I should probably find other hobbies, but the results were worth it.

Each point in the graph below represents a single sentence, ordered from left to right by order in the speech; the height of the point represents how many people retweeted it.

What is that point in the upper-right corner, the quote that is shared 10 times as widely as any other line in the speech? Surely it must be some rhetorical masterstroke, a particularly inspired policy proposal? No: it's the ad-libbed moment in the speech where Obama puts down the Republicans.

We criticize our politicians for lack of civility, but how do we expect them to behave when we pay more attention to their snappy retorts than their policy proposals?

Zooming in on the graph (click here and mouse over points to see which quotes they represent) we see that many of the most widely shared lines reprise the messages of hope and unity that so resonated when Obama originally took office: "I still believe we are one people", "I still believe that together, we can do great things", and "Let's begin this new chapter -- together -- and let's start the work right now". (More substantive lines, like those about gay marriage, community college, middle-class economics, and climate change, were also shared widely).

The only problem with these pleas for unity? For the most part, only the Democrats found them compelling. Self-identified liberal tweeters outnumbered self-identified conservative tweeters more than two to one for every single one of the top 25 most retweeted lines in the speech. Among those sharing "My fellow Americans, we too are a strong, tight-knit family", I could identify 96 liberal tweeters and only 5 conservative tweeters [1]: perhaps not such a tight-knit family after all.

So what lines were shared among Republicans? Here are the lines whose retweeters skewed most conservative (among lines shared by at least five tweeters with identified political affiliation).

Republicans / Total
Already, we've made strides towards ensuring that every veteran has access to the highest quality care.
100% (14/14)
Helping hardworking families make ends meet.
99% (74/75)
In Iraq and Syria, American leadership -- including our military power -- is stopping ISIL's advance.
96% (22/23)
There are no guarantees that negotiations will succeed, and I keep all options on the table to prevent a nuclear Iran.
91% (10/11)
We need to do more than just do no harm.
86% (32/37)
That's what middle-class economics is -- the idea that this country does best when everyone gets their fair shot, everyone does their fair share, and everyone plays by the same set of rules.
84% (42/50)

Of course, just because conservatives were sharing Obama's words doesn't mean they agreed with them. For example, "Already, we've made strides towards ensuring that every veteran has access to the highest quality care" was shared by conservatives with the addendum "This one difficult to take". "We need to do more than just do no harm" was shared by conservatives who replied, "PLEASE DON'T".

Sometimes differences between the political parties emerged even in consecutive lines of the speech. For example, when Obama said "And today, America is number one in oil and gas. America is number one in wind power," 68% of people sharing the line about oil were conservative; 82% of people sharing the line about wind power were liberal.

Of course, political party isn't the only characteristic that affects how we react to a speech. Men and women [2] reacted in different ways as well.

Lines Whose Resharers Were Most Likely to Be Female
Lines Whose Resharers Were Most Likely to Be Male
That's why this Congress still needs to pass a law that makes sure a woman is paid the same as a man for doing the same work.
Instead of getting dragged into another ground war in the Middle East, we are leading a broad coalition, including Arab nations, to degrade and ultimately destroy this terrorist group.
I want our actions to tell every child, in every neighborhood: your life matters, and we are as committed to improving your life chances as we are for our own kids.
We can't slow down businesses or put our economy at risk with government shutdowns or fiscal showdowns.
It's time we stop treating childcare as a side issue, or a women's issue, and treat it like the national economic priority that it is for all of us.
We believed we could reverse the tide of outsourcing, and draw new jobs to our shores.
[Child care]'s not a nice-to-have -- it's a must-have.
Members of both parties have told me so.
Today, we're the only advanced country on Earth that doesn't guarantee paid sick leave or paid maternity leave to our workers.
We still need to make sure employees get the overtime they've earned.
That's why we defend free speech, and advocate for political prisoners, and condemn the persecution of women, or religious minorities, or people who are lesbian, gay, bisexual, or transgender.
Today, we have new tools to stop taxpayer-funded bailouts, and a new consumer watchdog to protect us from predatory lending and abusive credit card practices.
We still may not agree on a woman's right to choose, but surely we can agree it's a good thing that teen pregnancies and abortions are nearing all-time lows, and that every woman should have access to the health care she needs.
If we don't act, we'll leave our nation and our economy vulnerable.

Beyond gender and political party, people’s backgrounds guide their reactions in many subtler ways. Tweeters who reshare the speech’s quote from Pope Francis are more likely to describe themselves as Catholic; those who reshare figures on Iraq and Afghanistan are more likely to be veterans, and those who reshare figures on graduation rates and test scores are more likely to be educators.

After I completed this analysis, I could see why Obama’s speechwriter needs a stiff drink. He had to please a thousand different constituencies in a single speech, half the country wasn’t going to be happy no matter what he said, and the line that people liked best wasn’t even in his draft. I’ll stick to writing code.

[1] Most tweeters do not identify their political affiliation in their profile, so this approach works only for a small subset of tweeters.
[2] I used the name of the tweeter to identify their gender. This does not work for all tweeters, and it does not account for people whose gender does not fit a binary description or does not match that implied by their name, but I still think it’s worth doing on balance because it gives a useful (if noisy) signal.

Thursday, January 15, 2015

The Illusion of Agreement: #CrimingWhileWhite, #AliveWhileBlack, and #LeelahAlcorn

This post discusses Leelah Alcorn, a transgender girl who committed suicide.

The Atlantic’s Quartz just published an analysis I worked on with Gilad Lotan [1] about how people discuss race on Twitter using the hashtags #CrimingWhileWhite and #AliveWhileBlack. You should read the piece (cmon -- I was writing this over Christmas) but in brief, the results are more hopeful than my earlier, depressing analysis of how people discussed Ferguson. When studying Ferguson, I found that different social groups expressed opposite views, rarely talked, and were often hostile when they did; studying #CrimingWhileWhite/#AliveWhileBlack revealed that while different social groups expressed different views, they weren’t opposite views, and information sometimes spread between groups in a constructive way.

Good signs! Unfortunately, this apparent improvement may be somewhat illusory. In the Ferguson discussion, conservatives and liberals often clashed; in the #CrimingWhileWhite and #AliveWhileBlack discussion, conservatives appear to have mostly disappeared. So it’s not that people all agree; it’s that those who disagree are silent. It’s not that the schisms that scar Twitter have healed; we’ve just zoomed in on part of the landscape.

Here’s another example of this. On December 28, Leelah Alcorn, a teenager who was born Josh Alcorn but identified as female, committed suicide and posted a note blaming her parents’ intolerance. The story spread rapidly on Twitter, where people overwhelmingly supported Leelah and blamed her parents. One of the clearest ways to see this is in the gender people used when referring to Leelah: while her mother used male pronouns, Leelah used female pronouns. Tweeters were hundreds of times more likely to use “Leelah” than “Josh”, and pronouns also show a dramatic skew. In the graph below, the green line shows the proportion of tweets that use “Josh”; the blue line shows the proportion of tweets that use “Leelah” [2].

The green spike at the beginning is due to local tweeters who were offering condolences before the suicide note had been published. Their use of “Josh” implies that Leelah’s preferred identity was not widely known (or, perhaps, ignored) among those who knew her. (One tweeter who used male pronouns defended himself by saying, “She went to my school but I didn’t know she was trans”).  

It would, of course, be wrong to conclude from this that there is overwhelming acceptance of transgender Americans. A survey of LGBT Americans found that just 3% believed there was a lot of acceptance of transgender adults. Two explanations seem more plausible: the people who would refer to Leelah with male pronouns either aren’t on Twitter, or aren’t choosing to comment in the aftermath of her suicide.

As I write in the Quartz piece, we need more than this illusory uniformity; we need fierce but productive dissent. The internet tantalizes us with its ability to create a single human overmind, but at the moment that mind’s had its corpus callosum severed. Please write to me if you have thoughts on

  1. how to get people with very different worldviews to have productive discussions of controversial issues
  2. how to search Twitter (or other areas of the internet) for signs that these discussions are occurring.


[1] A Twitter expert. He does professionally what I only dabble in, and has studied everything from ISIS to Israel.

Wednesday, December 31, 2014

Chasing Sunlight

Sunlight is a rare gift in Oxford for three reasons: we’re very far north, it rains a lot, and even if it’s noon and sunny, the sun is often too low to make it into the narrow stone alleys [1]. So when I do see sunlight, my first instinct is that there’s something wrong with my eyes, and my second instinct is to soak up as much as possible. Others have shared this impulse, according to this very old cartoon which I swear I am not making up; note the caption at bottom:

I wanted to write about my Oxford experiences chasing sunlight and what they’ve taught me about happiness. Recently I was walking down Oxford’s main street when the sun suddenly came out from behind clouds. I found a patch of sunlight and took out the book I’d been reading: The Opposite of Loneliness, by Marina Keegan. Marina was killed in a car crash when she was 22 years old, five days after graduating from Yale, and her parents compiled her unfinished pieces into The Opposite of Loneliness. The story I opened to was about a bunch of people in a submarine that breaks, leaving them trapped at the bottom of the ocean in total darkness. They have food for six months and spend most of the story arguing about whether to commit suicide. I finished this story about all this darkness by an author who died too young and stood there in my tiny patch of sun. I’m a stats nerd so metaphors aren’t my speciality, but even I couldn’t miss the overlapping images here.
The following day, I was sitting in a cafe with my tea studying some data on the Ferguson protests, which was of course depressing, when the sun hit my cheek. Instinctively, I turned and closed my eyes to soak it in. It was so peaceful there, with the tea and the babbling British, thousands of miles from the epicenter of Ferguson’s chaos. I felt very lucky.
These experiences made me wonder: if sunlight makes me so happy, why don’t I go back to California? Two reasons. The first is the famous paper showing that, although people expect the Californian sunshine to make them happier, Californians do not in fact report higher life satisfaction. But the second reason is more interesting: even if I would experience more moment-to-moment happiness in California, that isn’t necessarily a reason to return, because I don’t live to maximize happiness. I should clarify that by “happiness” I’m referring to an emotional state -- “feeling warm and fuzzy”, maybe -- as opposed to a broader philosophical notion of fulfillment/flourishing/eudaimonia. Here are four reasons I don’t think I maximize happiness.
  1. There are other desirable emotions. At Oxford I frequently argue with people who force me to see the world in a new way, and while this feeling isn’t exactly happiness, it’s definitely something I want. I go to cocktail parties at which I’m uncomfortable, but the nervousness I feel preparing to walk into those glittering rooms is also something I want to experience. Ditto with hearing music so beautiful it makes me cry or falling down exhausted after a long hike or getting really angry over something I want to change in the world. There are also positive feelings which require longer-term effort to achieve, like self-respect or a sense of purpose.
  2. Happiness is hard to control. My happiness comes in random bursts, small pieces of glittering mica in a long stretch of cement. My happiest moment today was when I was playing piano and my kitten suddenly jumped up and started playing with the strings (also, when he jumped into my lap and we danced for a while to “Sweet Caroline”. Maybe happiness isn’t that unpredictable: the rule is, get a kitten.) In general it seems frustratingly difficult to manipulate happiness, since things you might think would do it, like winning the lottery or becoming paraplegic, often don’t produce long-term effects.
  3. I work too hard on things which have too small an effect on my happiness. I put hundreds of hours into perfecting a paper, and when it gets published (or rejected) I’m on to the next thing within 5 minutes. I have always been driven not by happiness, but desire -- to know/build/win something new. As an often romantically unsuccessful teenager, I realized that, while I was happier if my crush returned my interest, even unrequited desire could give me something exciting to think about and a reason to wake up in the morning. (Requited desire gets you into bed, but unrequited desire gets you out of it. Jane Austen said this less crudely.) Now that I’m older, I apparently still approach my science like a love-starved teenager.
But just because I am always chasing new things doesn’t mean I should be. Perhaps I push forward to avoid confronting the fact that I’m not happy where I am. My high school had a lot of hypercompetitive, overstressed kids who tried to earn perfect grades so they could go to Harvard so they could go to med school so they could ... and I imagine you’d see a similar effect among Goldman Sachs investment bankers working 90-hour weeks. For my own part, I’ve definitely been lonely on a Friday night and written some code to distract myself. But I’ve also walked out on parties, including several I hosted, because I had some code I wanted to finish; you’re judging me, whatever, but the point is that my desire to code is sincere and not merely a means of denying existential angst. (To continue the love metaphor above, the person in bed next to you, like my habit of coding at parties, may be totally crazy and annoying all your friends, but as long as you sincerely desire them it’s okay.)
Maybe, you say, you’re working really hard so you can maximize your long-term happiness: you’ll achieve some goal and be satisfied. But I’m pretty sure if you gave me a Nobel Prize tomorrow (I prefer medicine but I’ll take peace, physics and, if you really have to, econ) I would wake up the next day and write code.
  1. While I spend almost all my time coding and writing, I feel just as much (or more) happiness doing other things. A few months ago I drove down the California coastline with some friends; by day, we drank beers by the breaking waves, and at night we drank strawberry moonshine and stargazed among the redwoods. (Strawberry moonshine is better than it sounds, and I’m not an alcoholic.) Budgeting $100/day for such an epicurean lifestyle, an average tech worker could spend like eight months a year on vacation. So if I want to maximize my moment-to-moment happiness, there’s no need to go to grad school. A few years ago I did semi-seriously consider this path: I had just eaten a spectacular chocolate chip cookie and had a lovely time with someone I knew I could not seriously date, and heart attacks and heartbreak aside, the pleasures of the moment were seeming pretty good. My plan was to buy a keyboard and drive up and down California playing for tips -- “In the redwoods I'll play Rachmaninoff; by the sea, Debussy. I will eat chocolate chip cookies and foie gras, and at night i'll sleep under the stars...this is a world to be explored by tongue and fingertip, eyes closed, cerebrum sleeping.”
There are pretty obvious responses to this, like “you would get bored” or “just eating cookies and ignoring the world’s problems makes you a bit of an asshole”. The former is just true [2], so let’s talk about the latter. If I can do work that touches the lives of thousands of people, that seems more important than my moment-to-moment happiness, or maybe even the happiness of those close to me. (I have a friend who, on this basis, once vowed not to have children because it would take time from his research.) The most frequent disagreement my boyfriend and I have concerns the fact that I wake up too early; he wants me to stay in bed and talk, and I want to get up and work. But if I can make an essay a little bit better, and thousands or millions of people read it, surely that matters more than whatever conversation we were going to have.
Maybe. But this sounds self-important. If my average reader is like me, they’re reading quickly, so they may totally misinterpret what I say, ignore it even if they understand it, or quickly forget it even if they don’t ignore it. The internet in theory connects you to tons of people, but your capacity to affect each of them is tiny. You really can make a million stars tremble by shaking your fist at the sky (gravity) but we don’t fancy ourselves galactic overlords. Given our impotence at the center of this spiderweb, maybe a little humility is warranted, and some preferential attention to those whose happiness we really do have great power to affect: ourselves and those close to us.
My code always calls me, but I doubt anyone dies wishing they had spent more time coding. And if I stay in bed a little longer, I can walk with my boyfriend to get iced coffee with cream and fresh mint. The air will smell like honeysuckle and the sun will make diamonds through the leaves. In a world where a 22-year-old girl who wrote better than I do can be killed in a car crash, such pleasures do not feel like small things.

[1] On average there are 1.7 hours of sunlight / day in December.

[2] I was initially going to argue that lots of people really do just want to drink and relax, as evidenced by their retirement plans, but the suggestions I found for “things to do when you retire” suggest that even in old age, people want more than Miami and Mai Tais.

Friday, December 26, 2014

Response To The Comments On The Sexual Assault Model

Content warning: this post discusses sexual assault in some detail.  

A few weeks ago I created a statistical model which implied that someone who has been accused of sexual assault by multiple people is much less likely to be innocent than someone who has been accused of assault once. This ended up on the front page of Hacker News, which Wikipedia describes as a “social news website that caters to programmers and entrepreneurs”, and got as many views in two hours as this blog usually gets in two months. (Granted, that’s not really that much traffic, but we math nerds take what social interaction we can get.)

While many of the comments were critical, almost all of them were civil, thoughtful, and free from the kind of trolling that women worry about when discussing controversial gender issues. All this was tremendously welcome and we need more of it on the internet, so I thought I’d take the time to respond to the most interesting criticisms.

To briefly recap (skip to the next paragraph if you remember all this) the model computes the probability that someone who has been accused of sexual assault by k people has never actually committed assault. The model assumes that there are serial predators, who have a high probability of assaulting people, and non-serial predators, who have a low probability of assaulting people. If someone has been assaulted, they accuse the other person of assault with probability pag; if they have not, they accuse with lower probability pai. You can play with the model here (move the sliders).

As I note in the post, the model is a simplification of the real world, as all models are. That said, it is possible to build a model which is too simple. For example, after Thomas Duncan’s nurse became infected with Ebola, it would’ve been irresponsible to say, “Assuming the current rate of weekly doubling of the Ebola epidemic in the United States continues, we expect every US citizen to be infected 28 weeks from now.” So I will discuss three assumptions the model makes and you can judge their validity and usefulness for yourself. Overall my view is that, while the model simplifies out of necessity, its basic conclusions probably hold even under more complicated assumptions. But I also see why there is room for disagreement because of the uncertainty about basic facts about assault, and trying to build a model has definitely reinforced my belief that we need better data. There also are clear tradeoffs between reducing the number of assaults and not punishing innocent people; your background and life experiences differ from mine, so you may worry more about victims of assault and less about the wrongfully accused, or vice versa.

  1. Objection 1: the model assumes that each accuser comes forward independently, but in fact each accuser may be influenced by previous accusers. This was the main concern people on Hacker News raised [1], and I think it is definitely important in high-profile cases like Bill Cosby’s, where accusers may well know about and be influenced by previous accusations. But I do not think cases like Cosby’s are the most representative or important to focus on, because the vast majority of accused and actual rapists are not famous. In these more representative cases, I do not think we should worry too much about accusers influencing each other for several reasons.

  1. In my post I discuss a tool which allows accusers to file accusations with a third party who will keep the accusations confidential unless multiple accusations are levied at the same person. If this tool were used, it seems quite likely that accusations would be essentially independent because they would be kept secret. (I guess it is possible that accusers might collude and all submit accusations together, but this John Tucker Must Die scenario stretches the bounds of plausibility.)
  2. Even if this tool is not used, accusers have to know about previous accusations to be influenced. This is by no means certain: how many people do you know who are currently being accused of assault? How often do you compare notes with someone who has had a sexual encounter with the same person?
  3. Even if accusers are aware of previous accusations, either i) they were actually assaulted, in which case it’s a good thing if they’re more likely to come forward or ii) they were not actually assaulted, in which case it’s a little unclear why they’re now more likely to come forward. The thought process might go something like, “Well, I’m not sure exactly what happened...I thought it was consensual at the time, but if they attacked other people, I guess they attacked me too.” This seems a bit unlikely to me, but feel free to suggest more plausible reasoning.
  4. Even if you disagree with all of the above speculations about probabilities, I built you another model that allows you to adjust them (you might have to zoom out to see all the sliders) and account for accusers who influence each other. I describe the mathematical details here [2]. Even if accusers can influence each other quite strongly, this model yields the same conclusion as before: someone accused of sexual assault multiple times is much less likely to be innocent. For example, I still see the same effect if I make an accuser ten times more likely to accuse an innocent person of assault if they learn they’ve been previously accused.

2. Objection 2: the model assumes that everyone is equally likely to be accused of assault. Several people pointed out that there are factors that influence how likely a person is to be accused of assault besides whether they actually assaulted someone. To take one example, someone who has sex with their partner of ten years may be less likely to be accused than someone who has sex with a person they met for the first time at a frat party, even if both encounters are consensual. More generally, we can imagine two groups of people, A and B, who have different sorts of sexual encounters, and thus different probabilities of being innocent:

I think this only alters our basic conclusion, though, if we think of a group who is a) reasonably large and b) a lot more likely to be accused of assault even when they are innocent. (Merely identifying group differences is not enough: you would have to identify a group for whom we can set the parameters of the original model such that the innocence curve no longer plunges sharply.) None of the people I spoke to about this advanced plausible candidates for such a group, but let me know if you have thoughts.
3. Objection 3: the model computes the probability that the accused is guilty of at least one assault, but that is not the probability we are interested in. In no court would a judge say, “Well, I don’t know which of these six alleged assaults the accused is guilty of, but there’s a very high probability he’s guilty of at least one, so I’m throwing him in jail.”

Still, when we’re not throwing someone in jail, we often do care about the probability the model computes. Maybe I’m in charge of student housing, someone’s been accused of assault by two of their dorm mates, and I’m trying to decide whether to move them to another house (or out of housing altogether). We frequently apply statistical reasoning that isn’t allowed in a courtroom.

Also, it turns out we can use a similar model to compute the probability that someone is guilty in a particular case (rather than in any case) given that they have k previous accusations of sexual assault, and it produces similar curves [3].

If you’ve made it this far, especially if you disagreed with me at the outset, thanks so much for reading, because I wrote this mainly for you. I imagine we still have disagreements and would welcome your comments below or via email.

[1] Some made comparisons to witchcraft trials, where a “mob mentality” takes hold and many people accuse one person of witchcraft. I will say, first, that I think witch trials are so different as to make this comparison pretty useless. A large number of people are truly assaulted; no one is a true victim of witchcraft. I do not believe there are mobs of potential assault accusers who, having all had sex with the same person, are suddenly all possessed by groupthink and decide that their sexual encounters constituted assault.

[2] The model assumes that if someone’s been previously accused of assault, a potential accuser knows about it with probability pk; if they know about the previous accusations, they accuse with probability pag1  if the person assaulted them and pai1  if the person did not; if the potential accuser does not know about the previous accusations, they accuse with probability pag0  if the person assaulted them and pai0  if the person did not. As pk  goes to zero -- no accuser knows about the others -- we recover the old model, where every accuser acts independently. On the other hand, if we increase pk  and pai1  to 1 -- accusers are certain to know about the first accusation, and certain to accuse if they know about it -- every accusation after the first becomes uninformative, as we would expect. We see the same effect as in the old model even for quite high settings of pk  and pai1 -- which can’t be all that high by simple virtue of the fact that very few people are accused of assault -- someone who is accused multiple times is much less likely to be innocent.

[3] The lawyers I discussed this with are still uncomfortable with this kind of reasoning, where past accusations can influence present guilt. My general sense is that the legal system disallows a lot of reasoning that statisticians deem kosher -- which is probably, given the weight we place on not convicting the innocent, a good thing.