Decision-based evidence making

Evidence-based need for change in early phase...

All decisions are not equal. Qualitatively, quantitatively, they vary in many ways, not least in terms of import, impact and information.

Decisions also have 'quality': decision quality is something to gauge before you make a decision, not something that can be wound back in the presence of new information. The question: 'was it a good decision, at the time?' is critical, not 'does it now seem a good decision?'

So, given that pretty much everything we do in pharma is decision-based, one might imagine the emphasis would be on improving decision quality. One, however, would be very wrong.

In order to do 'evidence-based decision making', which is the mantra in most places, you need evidence upon which to make that decision. If all you have collected is narrow and focused, your decision quality is inevitably impacted: simply, you have too few alternatives to consider when you make your decision.

The decision to 'find out' is not the same as the decision to 'do'. As Annie Duke, author of Thinking In Bets, points out, the early 'bets' in poker are not trying to win, they're trying to learn - small, meaningful investments in a future decision. Likewise, in many chess metaphors, you'll hear 'you can't win the game in the early rounds, but you can lose it.' If early moves are investment-based, and exploratory, in games of decision-making, why not in pharma?

Decision quality begets the need for evidence quality. Even if you set aside how low validity most translational models are (let's not pretend that AI has changed that by more than a decimal behind the 0), the evidence behind early stage decisions is both weak and suffers from 'absence of evidence' problems more than 'evidence of absence' problems.

'Was it a good decision, at the time?' then becomes a function of how you chose to spend your time pre-decision. Spend time, as Annie Duke, David Epstein (and every military strategist, and every philosopher ever), would say, in planning to make a decision in the future, and you change your mindset.

Evidence-based decision making needs decision-focused evidence making. Knowing that you have a decision to make, you can generate an evidence plan (a Plan To Learn as we call it). It should, if you plan to optimise Decision Quality, have wide range, alternatives and a healthy reading of import and impact, as well as depth and quality of evidence.

That, in a paragraph, is why we're so focused on early phase becoming exploratory, rather than confirmatory - because it certainly is rather narrow and focused now. Picking and running into phase I is like the Barnes move in chess, apparently the 'worst possible opening move' (I loved this para from Wikipedia):

The move does exert influence over the central square e4, but the same or more ambitious goals can be achieved with almost any other first move. The move 1.f3 does not develop a piece, opens no lines for pieces, and actually hinders the development of White's king knight by denying it its most natural square, f3. It also weakens White's kingside pawn structure, opens the e1–h4 diagonal against White's uncastled king, and opens the g1–a7 diagonal against White's potential kingside castling position.[4]

Decision quality stands or falls on 'things you do know, and things you do not.' Pretending that 'things you do not' will be revealed by a narrow and focused phase I is a key reason for low phase II decision quality (and guess what, for phase III and beyond, as a consequence...).

We recognise this is not a small change from where things are. But it is logically, rationally, critical.