Annie Duke seems like a surprising person to give advice about making better decisions. She’s a poker champion turned author of a bestselling book, “Thinking in Bets,”. I’ expected that her ideas are about how to take high risk bets and do them well. I recently listened to an interview of hers^[1], which was refreshingly different. There was an early aha moments for me that made me go on (more shortly). Then, she said she’s now a ‘decision facilitator’ to some of the world’s top Venture capitalists. What’s a decision facilitator? And why is a poker champion advising venture capitalists about decision making.

Here are my reflections on listening to her. At the outset, there’s a practical toolkit of tactics that are both fun and actionable. Then, some sage observations on why we make poor decisions. Finally, there was a provoking idea, which make me think a lot. One of the reasons I write is to help clarify such provocations to myself. Let’s dive in.

What we are most prone to

Annie’s sage observations about what makes us poor decision makers. She says we:

1. have overconfidence in our intuition. We place too much faith in our gut instinct
2. are too present-focused, with intense but fleeting feelings that inhibit rationality
3. carry illusions of unique insight. Although in reality, other people see our situation more clearly
4. are reluctant to use de-biasing techniques. Wouldn’t that be admitting our poor judgement?

“We think a lot more highly of our intuition than we really ought to….of our ability to notice things in the moment and act rationally toward”

Annie Duke, author of “Thinking in Bets” and “How to Decide”

Over-confident, present-biased, egocentric and generally unwilling to do something about it. That’s us.
Sobering.

Adversarial collaboration

Annie speaks of an interesting story from the late legendary Daniel Kahneman’s life.

“Kahneman actively sought out strong disagreement or contradictory views, not to argue against them, but to collaborate with those adversaries in hopes of getting closer to the truth through joint empirical investigation. It was an eagerness to prove himself wrong if the evidence pointed that way.

One of Daniel’s defining works (along with Angus Deaton) is a famous study in 2010 that got a lot of press^[2]. It said that money doesn’t buy happiness beyond $75,000 annual income. The idea was that once your basic needs are met and you don’t have to worry about paying rent and so on, more money doesn’t increase happiness by much.
About a decade later, Matthew Killingsworth’s found this to be not true. Money and happiness are correlated at all income levels, he found. The more money you make, the happier you tend to be. Our normally to this is trying to dismiss the other view as wrong. “But instead, it was vintage Danny – he said, ‘Alright, let’s do this together!’ He joined forces with Killingsworth and Barbara Mellers to basically try to resolve that issue through collaborative research. He was just always seeking out situations like ‘Why am I wrong?’ He was eager and excited to prove himself wrong if the evidence pointed that way.”

This is the idea of adversarial collaboration, my aha moment. It reminded me of an interview of Daniel Kahneman I’d heard on Shane Parrish’s knowledge podcast^[4]. I’d read his book “Thinking Fast and Slow” laboriously. I was a bit shocked when, in the interview, he said, some of the ideas in his book “didn’t replicate”. For example the priming effect I found fascinating was found to be untrue^[4]. I remember feeling a tad angry then, having to “unlearn it”. But he’d spent years discovering those. Years. Yet, he wasn’t attached to what he found (more on what they jointly found coming up).

Daniel Kahneman did have the luxury of being a researcher. He could afford to go back and spend time on something that turns out to be wrong. Some of us practitioners need to decide on the fly and then live with how things turn out. Our “adversaries” are simply too opinionated, poorly researched counter views. You can’t collaborate and resolve on most occasions. Yet, I have to admit that, on several occasions I could have practically embraced “adversarial collaboration”, but didn’t.

Two Tips for instantly better decisions

These two tactics from her tactical tool-kits are so simple, that I’m already trying them out:

1. Mental Time Travel: “One thing I used to do with my kids when they were really upset about something, I had to impose on them. I’d say ‘this is going to be so great for you when you’re grown up. At Thanksgiving you’re going to be able to tell these stories to your children and it’s going to be the best. You should be thanking me, so you can talk about your crazy mother.’ It allowed them to get some time and space to realise ‘this is going to be funny at some point’ no matter how horrible it seems now.”
   With the benefit of hindsight, I see this working really well for us adults too.
2. “Nevertheless”: “Let’s say my children are arguing about why something I’m imposing on them is unfair. I’d say ‘I hear you, nevertheless you’re grounded for two weeks.’ In essence, I’m saying ‘I hear what you’re saying and I understand, nevertheless this is what’s going to happen.’ The key here is to genuinely make people feel heard (which is not hard, although we rarely do it). In the workplace you can say ‘I heard you and your input. Trust me it was incorporated into the decision. Nevertheless this is the path we’re going to take.'”
   If they feel heard, people will accept it, even if they don’t like it.

Avoid decision meetings

“Too often in meetings, we try to do discovery (finding out what everyone thinks), discussion (debating those views), and decision-making all together. That’s extremely flawed.”

The only thing we should do in a meeting is discussion… No discovery, no decision-making.

Annie Duke

When people voice their opinions live in a shared space, the loudest or most confident person tends to have an outsized influence, regardless of whether they are actually right. Social pressures like that undermine sound judgment. Instead, in the discovery phase – solicit everyone’s true independent opinions and reasoning, says Annie. It’s quite easy to do in an asynchronous manner, where people can’t influence each other. For example, before a product roadmap meeting, send out a list of potential features to the team. Have everyone individually rank and rationale their priorities, without seeing others’ responses first.

Only once you’ve independently gathered this raw information should you convene to discuss the differing views and rationales out in the open. The meeting itself should focus solely on areas of disagreement. Then the final decision should be made independently. Not as a live group in the meeting. People can either vote privately on their personal leanings, or the decision-maker(s) can take the call.

The key is separating those three phases – discover, discuss, decide. This preserves true independent thought in the begin and end stages.

Driving alignment is toxic

This provoked me. Yet, I see the wisdom in it.

Says Annie, “drop the misguided pursuit of alignment coming out of meetings. It’s naive to expect everyone to agree, nor should that be the goal. It breeds coercion. Disagreement is healthy and captures the full decision landscape for leadership. If you can implement this independent model of getting raw unbiased opinions first, an open discussion in the middle, and a final decision independently, it greatly improves decision-making. It’s a game-changer I’ve seen work.”

Here’s why I feel provoked. First, this feels hard when I’m at the receiving end of this. Second, it requires exercising power, which breeds the fear (and possibility) of making enemies. If they’re not aligned, how will we execute? What will I do when I hear, ‘which is why I told you this is not the right decision’.

When I reflect on this more, we often allow our decisions to degrade to the level of alignment. If we need high quality decisions, we need to raise the bar on culture. We need to embrace taking unpopular calls. People need to truly embrace disagreeing and committing. Great decisions and need great culture.

On quitting

Stewart Butterfield shut down his startup Glitch at an unexpected juncture. Stewart was the creator of a product called Glitch, a massive multiplayer online world-building cooperative game. When Glitch launched, it received tons of great reviews and word-of-mouth praise – it was described as “Monty Python meets Dr. Seuss.” Prestigious investors like Andreessen Horowitz and Accel invested $6 million. And Glitch had 5,000 diehard users who played over 20 hours per week.

However, the customer acquisition numbers revealed a major issue. For every 1 user playing 20+ hours weekly, there were 95-99 users who just tried it briefly for 5 minutes before leaving. It was a customer acquisition nightmare. To combat this, in 2012 they committed to 6 weeks of paid marketing. During those 6 weeks, new user acquisition grew an impressive 6-8% week-over-week. But at the end of that 6 week period, on a Monday morning in November 2012, Butterfield woke up with “the dead certainty that Glitch was over.” He still had most of the investor money in the bank. He did some back-of-the-envelope modeling their growth and customer acquisition costs going forward. His calculation was that it would take 31 weeks until they broke even. But he realized that was an absurdly optimistic assumption, as their acquisition costs would inevitably rise once they saturated the core gaming market.

Butterfield realised Glitch was not going to be a venture-scale business model. And even though his team didn’t yet see it, he decided he needed to shut it down. Not just because it was a financial drain, but because it wasn’t fair to keep stringing along employees who had taken equity, when that equity likely had no future value.

So against all conventions, he sent an email to his investors and co-founders admitting defeat and declaring his intention to shut down Glitch and return the remaining money. Who actually does that? But he knew it was the right call, though he admits he probably should have made that call even before running the paid marketing push.

Now for the most interesting postcript. Two days later, Butterfield realised the internal communication tool his team had been using and loved could actually be a product itself. He pitched that idea to his investors, who rolled their Glitch investment into this new communication platform – which would eventually be named Slack.

The key learning, says Annie, is “we get so focused on ‘but what about everything that I’ve put into it’. Sunk cost fallacy. What we forget is there’s also the cost of not pursuing other opportunities that might be available to you,” she said.

“And as smart as Stuart Butterfield is, he couldn’t see Slack until he quit Glitch. And that is a true cost that he would have borne of continuing with Glitch … Slack would not be something that we’re all using today.”

How to quit objectively?

Her simple adivce – set up a “kill criteria” in advance. Kill criteria are specific scenarios or trigger events which signals that it’s time to pivot, cancel, or kill the initiative. You pre-commit to the action if they occur. This is, says Annie, the antidote to our irrational persistence bias.

“For example, when working with a sales team, I had them do a premortem exercise imagining they spent 6 months pursuing a lead through an RFP process, only for the deal to fall through in the end. I asked them to identify warning signals they wish they had heeded early on that foreshadowed the failed outcome. Some kill criteria they identified included:
1) If the RFP seemed tailored for a competitor
2) If the customer refused to do a demo and only wanted to discuss pricing, and
3) If they couldn’t get an actual decision-maker in the meeting room after initial conversations.

For each of those flags, they pre-committed to a specific action. For the pricing-only request, the kill criteria was to walk away immediately without pursuing further. For the competitor-tailored RFP, they’d directly ask about the competitor’s involvement and how advanced they were, then killing the deal or proceeding based on the response. For lack of decision-makers, they’d offer to elevate to an executive meeting and kill the deal if the customer declined.

“That’s actually the best use of a pre-mortem – to create kill criteria. It’s an upfront exercise to combat our human tendencies toward irrational persistence once we’re emotionally invested in an initiative.”

I’m now tempted to pick up Annie’s next book. “How to Decide”, although I’ve already read quite some literature on this.

Sources and more interesting notes

1. Interview link
2. High income improves evaluation of life but not emotional well-being
3. Kahneman resolves conflict on income-wellbeing study
4. Don’t read the book “Thinking fast & Slow”
5. Disclosure, I’ve lightly edited her quotes for brevity
6. Illusion of unique insight:
   “We tend to think that we have insight that other people don’t have when actually the other people probably have more insight into our situation than we do.”
7. Reluctance to use debiasing techniques:
   “I think that’s where the issue comes in…if I’m willing to do it think about what an edge I’m going to have over people who aren’t willing to do it.”