Monday, September 15, 2014

Four Tips For Avoiding Fist Fights at 35,000 Feet

At least three planes have been forced to land in recent weeks because passengers have gotten into fights over a question of cosmic importance: is it acceptable to lean back a plane seat? One might question the rationality of forcing a plane to land because you feel so passionately about this issue; luckily, like all problems stemming from human irrationality, this can be cured with statistics. (That’s a joke. Calm down.)

To this end, I designed a survey to study people’s views on leaning seats back. Huge thanks to the people who shared it on Facebook and Twitter -- Maria Mateen, Danielle Rossoni, Sarah Weston, David Ryan, Nat Roth, and Scott McWilliams, among others, as well as all their friends who then reposted it -- this would never have worked without you. (I should probably have mentioned this earlier, but it’s really tremendously helpful and gratifying when people repost content -- thanks if you have.) We collected 216 full survey responses, and from analysis of those responses I drew four lessons which will hopefully keep you flying the friendlier skies.

Lesson 1: Know what is socially acceptable. Here are five things you might do in economy class on a plane ranked in descending order of how okay they are.

“Agreed” or “Strongly Agreed” that this is acceptable
“Disagreed” or “Strongly Disagreed” that this is acceptable
Leaning your seat back


Saying no if the person in front of you asks if they can lean their seat back
Asking the person in front of you not to lean their seat back
If you ask if you can lean the seat back, and the person says no, doing it anyway
Purchasing a device, like a Knee Defender, to prevent the person in front of you from leaning back

People are clearly decided on three of the actions: it’s okay to lean your seat back, but it’s very not okay to ignore the person’s request that you don’t, and it’s even less okay to purchase a Knee Defender. People are split, however, on whether it’s okay to say no yourself -- whether by asking the person unsolicited, or even by saying “no” if they ask your permission.
Confession: I ran this survey expecting to find that tall people were less okay with leaning seats back. This is because my boyfriend and my dad, two mild and lovely people who are 6’ 0” and 6’ 3” respectively, rarely express annoyance at anything -- the exception being people who lean their seats back. If even these gentle giants get irritated, I reasoned, surely we have to fear the less even-tempered ones.

But! In my data, there is no statistically significant correlation between height and how acceptable a person thinks it is to lean a seat back, or with any of the other acceptability questions, regardless of whether I control for other factors [1]. The males in my life have misled me, but statistics has opened my eyes. The moral, I guess, is to not become emotionally close to anyone because that will bias you as a statistician. You will be pleased to know that, in my quest for statistical purity, I have now severed all ties with my boyfriend and father [2].

Granted, it’s not a huge sample. But an effect that doesn’t emerge in a sample of 200 people is maybe not an effect you want to risk getting in a fist fight over. (The 95% confidence interval for my height coefficient is [-0.045, -0.046], which means that my model estimates that adding a foot to someone’s height moves them at most halfway from “Agree” to “Strongly Agree”. That’s just not a very big effect.) Which brings me to

Lesson 2: Do not make assumptions based on the person behind you.

It wasn’t just height that didn’t predict how someone would feel about leaning seats back: it was also their gender, age, American citizenship, whether they enjoyed flying, how comfortable they found airline seats, how often they flew economy, and how often they flew first or business. So, my children, do not judge the person behind you by such superficial features. Instead, take the time to stare deeply into their soul, or else retweet this survey so we can get enough data to stereotype in a statistically significant way.

On the one hand, it’s kind of cool that we can identify something that people feel passionately and unpredictably about. On the other hand, not being able to predict things sucks. But there is a bright side from the prediction standpoint.

Lesson 3: It’s often safe to make assumptions based on the situation.

I asked people whether various situational nuances increased or decreased the acceptability of leaning one’s seat back. Here are the nuances; green denotes the fraction of people saying it made it more acceptable to lean your seat back, and red, less acceptable.
So, while you shouldn’t make assumptions based on the person, you can sometimes make assumptions based on the situation. Translation: anyone who keeps you from leaning your seat back on a redeye is by social consensus a horrible person and deserves to be pelted with pretzels.

Here, again, the characteristics of the person fail to predict their views. For example, tall people respond no differently to the questions about being tall, and women respond no differently to the question about children.

The thing is, you could follow the first three rules and still get punched, because

Lesson 4: Some people are just weird.

Sorry, I really appreciate everyone filling out this survey and I don’t mean to hate. But there are certain answer combinations that don’t really make sense to me. For example, if I say that it’s acceptable to buy a Knee Defender, I should also think it’s acceptable to ask someone not to lean their seat back, or to say no if they ask if they can; both are way less provocative. And yet there are people who say it’s okay to buy the Knee Defender but not to just talk to the person; to these people I would recommend a less passive-aggressive form of conflict resolution. I’m also confused by the people who say it’s okay to lean your seat back but also say it’s acceptable to use a Knee Defender -- these things cannot occur simultaneously, guys. Also, watch out for the people who think it’s okay both to use the Knee Defender and to ignore the person who asks them not to lean back -- these people are the Nietzsches of economy class, and will rely on brute force both to get into your lap and keep you out of theirs.

All of these groups were quite small, but the lesson is: there are outliers in every sample, and one of them might be sitting right behind you. Also, if people do weird things when sitting comfortably at their computers, they probably do even weirder things when they’ve been squeezed into a tiny coffin for four hours breathing deoxygenated and virus-laden air. God, I hate flying.

Two important caveats here. First, this is a weird sample -- my Facebook friends, and the friends of my Facebook friends -- which does not represent the whole population. One obvious difference is that 84% of the respondents were between 18 and 25. So if there is, for example, a crotchety old person demographic or an angry baby demographic, they will not be captured here. (There is also, incidentally, a suspiciously small number of men in the survey reporting that they are 5’ 11” -- probably they’re rounding up to 6 feet.) Second, asking people what they find “acceptable” leaves room for a lot of interpretation -- does that mean acceptable for me to do, or acceptable in general? If I say something is unacceptable, does that mean I’ll disembowel you if you do it, or just that I find it slightly rude? Etc. Please bear these caveats in mind, be civil, treat this as a fun experiment as opposed to a scientifically rigorous paper, and don’t write me angry emails if you get assaulted with a tray table. But on the other hand, if the person behind you insists on buying a Knee Defender, feel free to cite this survey as evidence that society condemns them to hell.


[1] All factors examined: your gender, your age, whether you enjoy flying, how comfortable you find airline seats, whether you’re an American citizen, how often you fly economy, and how often you fly first or business.

[2] This is, obviously, a joke, but there is a serious lesson here -- we learn about how people work from a very small and biased sample, and while personal experience is evocative and compelling, it’s an unreliable way to draw general conclusions.

Monday, September 8, 2014

Death Threats and Cramped Planes

This week I was watching The Dark Knight and I came to that wonderful scene where the Joker describes his life philosophy: “Do I really look like a guy with a plan? You know what I am? I’m a dog chasing cars. I wouldn’t know what to do with one if I caught it. I just do things.” Which often describes my approach to scientific research. Some random investigation seems interesting, so I try it [1].

So here are three experiments in ascending order of seriousness. If enough people participate and we get interesting results, I’ll describe them in a future post.  

1. Recently the New York Times has written at least 3 stories about leaning your seat back on a plane, which I’m pretty sure is more than it’s written in the same time period about, say, global warming. I would love to get your opinions on the acceptability of leaning your seat back -- please fill out this short survey! (And don’t go and read all the articles prior to filling it out, please.)

2. I went into an Apple store to purchase a new laptop and got a recommendation from an employee. I would love it if, the next time you are near an Apple store, you could drop in and take careful note of your experiences so we see if we get the same recommendation. (I promise there is a motivation for this, but I have to tell you what it is later to avoid biasing you. Because clearly this is a very rigorous experiment.) Here are your marching orders:

a) Go into the store, tell the employee you want to buy a new laptop and that your main requirement is that it have 16 gigabytes of RAM (say it like the sheep).
b) After that, be pretty vague and suggestible. For example, if they ask you what you use the computer for, try not to say things like “I am a professional data scientist” or “Mainly for watching porn” which will probably bias the employee in one direction or another; instead say something like “Work, email and watching movies.” Do try to ask the employee for details like “So do I want the fastest chip?” or “Do I want a ton of hard disk space?” or “Do I want the 15-inch or the 13-inch?” If they try to talk you out of the 16 gigabytes of RAM thing, don’t be too pushy, and let yourself be persuaded.
c) Take careful note of what computer they recommend you buy -- remember as many details as possible, particularly its price. Record your experiences here.

3. Death threats on Twitter -- they’re a thing. I’m wondering if it’s worth monitoring them so we can attempt to determine something about the people who send death threats and when they come in. Probably this is a lost cause, but if you have received threats of violence via Twitter or other means, please shoot me an email.

Because one should never take data without giving some in return, here’s a link to all the datasets I have collected that I would be happy to share. You are welcome to use them -- all I ask is that you a) shoot me an email describing what you find and b) provide a citation or link to this blog!


[1] To keep from being kicked out of grad school, I’ll point out that I’m separated from the Joker by a conscience and lack of psychopathy. Also, I have a tendency to see connections to research in every movie I watch. Recently I saw Iron Man, and when he puts on his suit I thought yeah! That’s how I feel when I open my laptop -- and after that I watched Sex Tape, and when they say, “Nobody understands the cloud!” I thought true that. These experiences indicate either that I need to think less about research or watch less TV.

Saturday, September 6, 2014

Outnumbered but Well-Spoken

The Atlantic’s Quartz just published my analysis of how women often stay silent, and why it’s important that they speak up. The analysis uses data from the New York Times, which you can download here; you can also read the academic paper, which will be presented at CSCW 2015. Huge thanks to Suzanna Fritzberg, J. Nathan Matias, Nat Roth, Jacob Steinhardt, and Seth Stephens-Davidowitz for their help on this.

Thursday, August 28, 2014

The Dark Side of Viral Rage

You may have seen the stories about the Stanford student who aroused internet outrage when he was quoted comparing rape to bicycle theft. But, if you don't read the Stanford Daily, you almost certainly haven't heard the true story, which I describe here (see also the student's response). Unfortunately, the media outlets which smeared the student's name in the first place have thus far been uninterested in correcting the story, which has now propagated internationally. Please shoot me an email (emmap1 at alumni dot stanford dot edu) if you have thoughts on how to help this story reach a wider audience.

Tuesday, August 19, 2014

How to Study the Rage of Millions of People

Dear Twitter: in this post, I offer to give away your data. I do this completely in good faith and for no monetary gain because I am a researcher, I think your data is fascinating, and I hope to help people make sense of it. I have reviewed your Terms of Service, read a number of research papers written on your data, and contacted employees at Twitter, and to my knowledge I am not in violation of any of your rules. But if I have misunderstood please contact me at emmap1 at alumni dot stanford dot edu and I am more than happy to comply with your requests.
I am very excited about Twitter because it combines two qualities.

1. People actually use it. Famous people -- it’s become standard for celebrities to say “Follow me on Twitter!” -- and more importantly, lots of people.

2. It makes massive amounts of data available in a way you can process with a computer. 500,000,000 tweets are sent every day and Twitter will give you up to 1% of those. And if I know what 1% I want -- for example, only Tweets containing the word “Spock” -- it will give me all of them, which means I can actually hear everything that’s being said on a topic by millions of people worldwide. And not just what’s being said, but who’s saying it -- how they describe themselves, where they live, who their friends are, and the last few thousand things they said [1].

[Pause so we can all process how incredibly cool this is.]

If you still don’t think this is incredibly cool, you’re either not paying attention or dead on the inside. Twitter is enabling new research on everything from the Mexican drug war to the Israel-Palestine conflict to earthquakes to the stock market. Just this week, it easily provided enough data for research papers on three topics I can think of off the top of my head: societal reactions to suicide using Robin-Williams-related tweets, altruistic behavior using #icebucketchallenge, and protests against racism using Ferguson-related tweets. I’ll come back to the last one in a second.

I want to study Twitter with you. Consequently I am making three things available. (If you like working with data you should read about the first two; if you just like reading about data, you should skip to the third.) The first is a tool that makes it easy to collect all the Tweets (and all the data for the Tweeters) that contain sets of words or phrases. Important caveat: this program will only collect Tweets live -- it cannot search for Tweets in the past, because Twitter makes it very hard to get these -- so you need to be quick on the draw. This tool may be slightly buggy -- let me know if you find weird things! -- but it’s probably not seriously buggy because I have been using it more or less without incident for the last few months; you can turn it on, forget about it, and come back later to get your data.

So the first tool gives you raw data. The second thing is a tool that infers cool things from this raw data and returns it in a table which is easy to analyze. For example, you can sometimes use a Tweeter’s name to get their gender and race, as I describe here. You can use a technique called sentiment analysis to analyze the emotions in the Tweets, and watch how levels of sadness, anger, profanity, and so on change over time, or by group. You can often figure out the Tweeter’s location from their timezone, and you can also get the local time, which is important if the phenomenon you’re studying has daily cycles. The documentation for this tool is here. Unfortunately, I cannot make the code for the tool publicly available because it relies on a sentiment analysis library which is proprietary, although I may cut it down and release a less complicated version when I have time. But if you have a dataset which you would like to use it to analyze, shoot me an email!

The third thing, to illustrate the utility of the first two things, is an actual dataset of Tweets relating to the Ferguson shooting. I’ve been monitoring Twitter for about a week for hashtags like ferguson, iftheygunnedmedown, and handsupdontshoot, and I initially was collecting so many Tweets that I ended up keeping only a tenth; even so, it’s a few hundred thousand Tweets. It’s a very rich dataset, and I’ll probably do some more analysis on it myself after events play out, but email me if you’re interested in looking at it and we can discuss possibilities for collaboration. (Twitter’s Terms of Service prohibit me from just making the dataset publicly available.)

I’ve barely glanced at the data, but one thing I did do was take the most common hashtags and connect the ones which tended to appear together [2]. At first it’s a little hard to see what’s going on, but when we look closer we can see evidence of a rich and complex conversation:

1. There’s a purple cluster talking about the many other unjust police shootings, often in connection with the lastwords hashtag.
2. There’s a red cluster of Anonymous users -- a group of online activists who conducted cyberattacks against the Ferguson police department.

3. There’s a yellow cluster of Tea Party members and gun rights activists, who I’m sure have been made much less paranoid about abuses of government power because of this whole episode. Close to them is a more liberal group that includes hashtags like “p2” (Progressives 2.0), “libcrib”, “stoprush”, “ows” (Occupy Wall Street) and “civilrights”. Oddly, some military hashtags (“military” and “vets”) appear to be more connected to the liberals than the conservatives.
4. There’s a red-purple group of people who are advocating peaceful protests with hashtags like “love”, “unity”, “equality”, and “MLK”.
5. There’s a purple group of people drawing connections to Gaza, and close to them there’s another group of people drawing connections to other international events (“egypt”, “syria”, “ukraine”, “iraq”, “isis”).

If you want to explore further, zoom in and click on the circles. Clearly there’s a complicated and interesting conversation going on here, and even if there’s a lot of dirt in the data, there’s a lot of gold there as well; let me know if you’re interested in digging deeper! Here’s one question that occurred to me: there’s been a daily pattern with peaceful protests by day and more violence and anger at night. Can we see evidence of that cycle on Twitter? And, if we can, is it because a) the same people get angrier at night or b) different groups of people tweet by day and by night?

Do me a favor and if you do end up using any of this, or if you have thoughts for new or improved tools, please:

a) Shoot me an email and let me know what you find / ideas you have! I’d be happy to publish cool analyses here or collaborate to find other audiences for them as well. And if you’re not a computer person but you think of some cool societal trend, or you notice something important happening, let me know quickly and maybe we can track it!
b) Feel free to point people to the tools or this blog!


[1] Contrast this to other big data companies -- Google would never make individual level data like this available, and even when they make grouped data available (say, how many people are searching a certain term) they make it very hard to use a computer to get it quickly. (Google Trends is cool, but I could never use it to get, say, the volumes of 10,000 different searches over time.) Facebook requires me to get something called “user consent” (what?) to get most interesting data. I’m not criticizing Google and Facebook for keeping their data hidden, by the way; their users expect privacy. But the whole point of Twitter is that you’re a Twit in public, and users have no expectation of privacy. Twitter does conceal some information, like the user’s location, if the user chooses this in their privacy settings.
[2] This was created, incidentally, using NetworkX + Gephi, because I got excited about Gilad Lotan’s excellent talk on the combination.

Monday, July 14, 2014

A Tale of Two Cities: The Twitter Reaction to the Return of Lebron James

At 9:31 AM PST on July 11, LeBron James announced that he was returning to Cleveland, and Twitter exploded. (If you don’t know who LeBron James is, see [1] for backstory.) The frenzy was such that the New York Times ran a front page story purely about the tweets. I collected more than 2 million of them, and learned some things about forgiveness, race, and fangirls.

One obvious question: did people on balance approve of James’ decision? The NYT did not attempt to figure this out -- come on, NYT! -- probably because they didn’t have the data and it’s hard to measure approval. One standard way to do it is to count words with positive and negative associations using a word list, but this is a bit dicey in this data; words like “fan” are usually positive, but here you have tweets like “LeBron fans suck”. Instead, I came up with customized phrases. For example, I recorded 1,392 tweets containing the phrase “I love LeBron” and 1,549 tweets containing the phrase “I hate LeBron”. But the latter group contained tweets like “Do I hate LeBron still? Nope” -- some people might loathe James for his prior mistakes, but admire this decision. Indeed, the data supports this idea:  
Phrase Pair
Number of Tweets
“good decision” compared to “bad decision”
547 to 94
“good move” compared to “bad move”
954 to 123
“smart move” compared to “stupid move”
390 to 20

Overall, the Twitter data indicates that while James is still polarizing, this decision was popular. Obviously, however, not everyone was thrilled. I recorded 25k tweets from Tweeters who listed “Miami” in their location and compared those to the 21k tweets from Tweeters in Cleveland. This was a little sad. Miami fans used twice as many words expressing negative emotion, three times as many words expressing anger, twice as much profanity. Interestingly, though, they were about four times as likely to express respect: they were sad and angry but also reluctantly impressed. We can also look at the hashtags which were particularly common in each city: some of them were obvious (191/191 “northeastohio” hashtags were from Cleveland) but some were interesting:

Miami Hashtags
Fraction From Miami
Cleveland Hashtags
Fraction from Cleveland
smh (“shake my head”)
tfm (“total frat move”)

(“Lift the ban” and “I’m sorry” turn out to relate to this crazy Cleveland fan who got himself banned from Cleveland games for a year for running onto the basketball court while James was playing and begging him to return. James patted him on the head as he was dragged away by security.)

Let’s talk about race. Twitter doesn’t provide race data, but I wanted to see if I could infer it for a few reasons:

1. Racial dynamics in professional basketball are often interesting: 76% of the players are black, as compared to 43% of coaches and 2% of owners. There have been a lot of race-related episodes: see the owner who was banned for life for racist remarks; Jesse Jackson’s allegations that Lebron James was being treated like a runaway slave; the differential popularity of Lebron James among different races; the racism against Jeremy Lin.
2. I’ve done a fair bit of work on gender dynamics, and women and racial minorities share many problems; studying race seems a natural extension.
3. It’s an interesting problem.

Obviously, race is very complicated -- at 23andMe, I’ve learned from our ancestry experts just how tangled the relationship between biological ancestry and self-identified race is -- and so any inference from Twitter data is going to be highly imperfect. Please keep this in mind before writing me blistering emails. I tried to identify Tweeters as black, white, Hispanic, or Asian, and used three methods to do so:

1. Tweeter self-description. Someone who uses the word “Asian” in their self-description is usually Asian, although obviously there are some false positives (people who use the word black but are saying they like black dresses, etc).
2. Tweeter last name. See here. This turns out to be very useful for Asian and Hispanic names, not so much for white vs black names.
3. Tweeter first name. Freyer and Levitt wrote a nice article about the consequences of having a distinctively black name; we can supplement their list of black and white names with data on baby names from NYC, which gives us Asian and Hispanic names as well.

People have been trying to get race from name for many years and it’s a lot more dicey than getting gender from name. The most basic problem is this: while someone who names their kid “Alabaster Snowflake” is probably white, they’re also probably not representative of the general white population. The people for whom you can identify race from name are going to be unusual. Similarly, someone who identifies herself as Asian on her Twitter profile may not be representative of Asians generally. So we’re not really comparing white people to Asian people, we’re comparing people with distinctively white names to people with distinctively Asian names [2]; similarly for profiles. To emphasize this distinction, I'm going to refer to tweeters not as "Asian" but as "d-Asian" -- ie, distinctively Asian.

I was able to identify 124k tweets from d-White tweeters, 32k from d-Hispanic tweeters, 12k from d-black tweeters, and 7k from d-Asian tweeters (in North America).  I could not identify clear racial differences in whether Tweeters approved of James’ decision, but I found other interesting differences. d-Asian tweeters do, in fact, tend to tweet about Jeremy Lin; 56% of tweets containing “jlin” come from d-Asians. d-Hispanic tweeters are especially likely to use hashtags supporting teams in Los Angeles, San Antonio, and Miami -- all cities with large Hispanic populations -- and, unsurprisingly, tend to use Spanish words. d-black tweeters also tended to use different language: “finna”, “ima”, “tryna”, and “yall” were among the words that increased in frequency most among d-black Tweeters, as were various versions of n*****. (d-Black tweeters were about four times as likely as d-white tweeters to use n***a, with d-Asians and d-Hispanics falling in the middle.)

I also looked at gender. Only about 17% of tweets came from women, and some of the male tweeters complained about how female tweeters were just tweeting “I looooove LeBron!” But the stereotype of the sweet-spoken fangirls turns out to be wrong: the girls tweeting about James express more anger and use more profanity than the guys, and while they are indeed more likely to say they love him, they’re more likely to say they hate him, too. And forget about the welcoming female domestic stereotype: female tweeters are actually slightly (but statistically significantly) less likely to use variants of “welcome home”. These results surprised me enough that I checked whether my filters were broken (I don’t think they are); one explanation is that interest in basketball is somewhat unusual for women, and that women who tweet about LeBron James are unusual in other ways as well. (Alternately, there might be some weird correlation between gender and another variable, like location.)

This is about as much time as I'm willing to spend studying LeBron James; on the other hand, if you could infer race in a way that doesn't introduce weird biases, that would be exciting and powerful, so let me know if you have ideas about that. Also, I realize that race (like gender) is a fraught topic, so please let me know if anything I've written seems insensitive or inaccurate.

[1] LeBron James is one of the greatest and most polarizing basketball players of all time. At 18, he began his career playing for Cleveland, a sad sports city that hasn’t won a championship since 1964; then he broke their hearts and drew widespread disgust by announcing in a graceless press conference that he was leaving to join two superstars on Miami’s team.
[2] I initially thought I could get around this problem by looking at all names and simply assigning each name a score for each race depending on how frequently it was used for that race (rather than just looking at names with >90% confidence for a particular race); this would incorporate data for all Tweeters rather than just the distinctive name ones, and then you could just run a regression on the name race score. I think this runs into a similar problem, though, because you find that for black last names, for example, very few Tweeters have names which strongly indicate that they are black, which may mean that whatever signal you get is predominantly driven by these distinctive Tweeters.