Designers, world changers, we have a problem.
Nobody has been happier than me about the recent uptick in pop-psychology books. I’m known among my friends and coworkers as that guy who tries to read everything – an I’m relatively successful – but I’m starting to wonder if I’ve been consuming too much junk-food.
You know the books I’m talking about. They’re written by an academic – usually a behavioral psychologist, a behavioral economist, or maybe someone in sociology – that has recently discovered that she can make WAY more money selling books than writing white papers. They have catchy, one-word names like Habit, Nudge, Switch, Influence, or slightly cryptic, but clever titles like Thinking Fast and Slow, The Righteous Mind, or Predictably Irrational (I could go on, the list is long and growing). You can knock out their 300 pages or so in 3-4 days. They’ve got enough psychology in them to make us feel smart for knowing things, but not enough to scare us away. We also get that familiar rise when we read them because we feel like we’re in on some super-secret about humanity that our users/customers/clients aren’t in on.
I love these books. I would love to work for my personal heroes, Richard Thaler or Jonathan Haight (hit me up guys, I know a lot about contextual, qualitative research).
But we may be in danger
Behavioral Psychology has given a lot to us, but we really need to think hard about the quality of what they’re selling. In the marketplace of ideas, they’ll give us what we ask for, but are we asking for junk? Is this going to come back to bite us?
Specifically, I’m talking about something a little scary. We’ve heard from several guests on the Econtalk podcast that there is a replication crisis in academia. Academic researchers are rewarded (though book deals, talk show appearances, tenure, and other prestigious perks) for discovering new things – not replicating old discoveries. We can all understand the appeal of discovering new things, but as professionals, we can only use the science if we can trust it. We can only trust it if there is a strong incentive for replication, and a demand for follow-up studies that are statistically significant.
This problem is starting to gain a little popularity. Slate Star Codex’s pen-named writer ‘Scott Alexander’ mentioned the problem in an excellent essay about “Devoodooifying” Psychology, and John Oliver dedicated a recent rant to a similar problem with the way the media discusses Scientific Studies. Most interesting to me was a recent Medium article in which Nassim Taleb, the author of Black Swan, called out a number of Intellectual Yet Idiots by name – accusing them of using sloppy science to try to shape the current cultural zeitgeist.
These critics aren’t unintelligent pundits. They’re thoughtful, educated, and likely correct to a certain degree. The problem is, we don’t really have a way to separate the good from the bad.
Psychological research in the current day is already stunted by an over-reliance on sterile studies that are devoid of context and sample populations that are almost ubiquitously western college students. Because these are published by scientists, we at least need to be able to believe that they’re using good statistics practices and statistically significant sample sizes!
I’m afraid.
But as professionals, we take action on these things. When we feel that ‘Nudging’ can cause big changes, we devote resources to understanding how to execute things. We sell this as wisdom to our clients. Sometimes, we can A/B test our hunches and logic to fit a solution to a problem, but sometimes the costs are higher.
For instance, activists have adopted the notion of implicit racism with a gusto. We try to convince people – and ourselves – that we’re actually secretly racist (or sexist) and there is nothing we can do about it. It is a TOUGH thing to convince people of, but we try because we think that we’re standing on solid science.
This idea isn’t contained to the lab. There are real social movements that try to use this research to achieve their political motives. They have no reason to question it because it fits with their political narrative. Scientists have no reason to question it either – these results are likely to resonate with the political sympathies of their colleagues.
Can we trust this science? I don’t know. I’m certainly unconvinced by some of the methodologies that I’ve seen that ‘proves’ these notions, but who am I to say? Implicit bias, however, is an extremely nuanced idea that is studied to change complex systemic problems. If there is sloppy science, it has the potential to completely undermine strides that advocates for these causes are trying to make.
I’m afraid because most of us don’t have PhD’s. I would also be likely to “mistake the ensemble for the linear aggregation of its components*” if I knew what that phrase even meant. The only way we know if Richard Thaler and Cass Sunstein are lying is if Nassim Taleb tells me so – and if he calls them out on it. But then, I’m powerless to know what Nassim Taleb has missed. We get this deep reflection from some more introspective academics, but it is rare, and almost never comes out in pop-psychology business books: they’re here to sell us answers, not speculation.
*Lucky for me, intro to statistics is free on Coursera.
What are we going to do about it?
I don’t really know, but here are my thoughts at the moment.
One could argue that science should be a national investment, and that might be true. One thing that is certain, however, is that certain fields – Design being one of them – directly benefit from this research’s accuracy. We don’t all have the time or qualifications to make this judgement for ourselves, so we need some way to compare notes.
We may also need to discuss funding a little bit. As an academic, it is more profitable (not just in money, but in reputation) to publish a book full of compelling psychological short-cuts that may or may not be replicable. Is there a way that – as an industry – we can design a solution to this problem?
Note on the books mentioned
I mentioned a lot of books in the introduction. I chose them on purpose: it is a list of my favorite books, and I don’t want to accuse any of them of having bad science. That said – I don’t know if the science is good or bad. They told me things that I wanted to hear in a way that I found useful. What do you guys think?
At the time of this writing, I haven’t read Nassim Taleb’s book, Black Swan, yet, but I’ve heard wonderful things.
