How to prioritize as a PM

Prioritisation of features is a key pillar in Product Management. It’s part art, part science. It’s being out there in the field, gathering data, metrics and insights on one hand, and having a feel for where the market is moving and staying true to your vision on the other. Picture the PM here as some kind of dual-wielding product superhero.

Now while we’re imagining, picture yourself on Day 1 as PM for a new CRM company. You are handed 3 feature proposals, which all sound quite good:

  • Addition of a new revenue dashboard, which marketing team are very excited about
  • Two-factor authentication (TFA) for administrators, which a customer just signed a conditional contract on
  • Ability for CRM to send customised SMS to customers, which is a top feature request amongst users

Which do you build first?

Thankfully, there are many tools out there which will give you a framework to find out.

Weighted Shortest Job First & Cost of Delay

WSJF takes an economics view to produce the maximum economic benefit. It does this by calculating the Cost of Delay (CoD) and dividing by job size.

The theory borrows from operations and the changing view of how software is delivered. In a continuous flow system, job sequencing produces the best result over theoretical return on individual jobs.

If you only quantify one thing, quantify the Cost of Delay.

Don Reinersten (Scaledagileframework.com, 2019)

WSJF allows you to continuously update backlog priorities based on user and business value, time factors, risk, opportunity enablement and effort. It also ignores sunk costs.

How is it calculated?

First, start by calculating the Cost of Delay by determining three variables:

  • User-business value: What is the impact on users and/or our business?
  • Time criticality: How urgent is this? Are there regulatory or contractual reasons to do this now?
  • Risk-reduction/opportunity enablement: Does this reduce the risk the business is exposed to? Does it open up future business?

However to complete our calculations, we need a small segway on relative estimation.

Relative Estimation

Relative estimation is the technique of comparing items either to each other, or to past known efforts, and applying a value.

Imaging you were evaluating the effort of preparing different fruit to eat. You had some experience chopping fruit already:

  • Cutting an apple took 2 minutes (includes removing core)
  • Cutting strawberries took 1 minute
  • Cutting a mango took 5 minutes

Now you have to evaluate more work; a cucumber, an orange, and a pineapple.

Despite not having timed yourself cutting a cucumber before, you know that there is little preparation involved. You could approximate it to slicing the heads off strawberries, and estimate 1 minute.

Cutting an orange and removing skin might be more work than cutting an apple, but less than a mango. So you estimate that at 3 minutes.

Finally, cutting a pineapple is quite complex, given the spiky exterior and effort involved, yet less work than cutting two mangos; so you estimate 8 minutes.

The cool thing about relative estimation is that the metric itself is not important; only in how it relates to other values.

These values could have been 10 (cucumber), 30 (orange), and 80 (pineapple), with no ‘concept’ of the actual minutes or seconds involved. It is enough to say that cutting the orange is 3x as much work as cucumber.

Some common approaches in applying these relative values include using number systems such as:

  • Numbers 1 through 10
  • Order of magnitude (1, 3, 9)
  • Fibonacci sequence (1,2,3,5,8,13, etc)

Cost of Delay

Let’s apply relative estimation to our proposed features:

  • Addition of new revenue dashboard
  • Two-factor authentication (TFA) for administrators
  • Ability for CRM to send customised SMS to customers

Now we need to rate the items against each other on each of the COD dimensions.

User-Business Value

In terms of user-business value, the revenue dashboard would help a subset of users (Managers). We’ll give it a rating of 3.

TFA is associated with increased security – however doesn’t add a lot of value to an individual user or the business. So we’ll rate this a 1.

Recall that this ‘3’ & ‘1’ do not stand for or convert to anything else; it simply signals that the dashboard would provide higher value to users.

The ability to send customised SMS is something that every customer could potentially use. As the vendor, it’s also worth more to us, since we charge for SMS. With both user & business value, it then scores an 8 relative to the dashboard and TFA features.

FeatureUser-Business
value score
Addition of new revenue dashboard 3
Two-factor authentication1
Ability for CRM to send customised SMS to customers8

However, we’re not done yet.

Time Criticality

The new revenue dashboard has been proposed by a senior manager and the marketing group. However there is no contractual obligation, and also a work around; the same data can be extracted from reports. We’ll give it a value of 3.

Speaking to the sales team, they actually sold a contract to a corporate customer that specified TFA as a requirement. We therefore rate that as an 8 on time criticality.

The sending of customised SMS is a common request, so that also gets a 3.

FeatureUser-Business
value score
Time
Criticality
Addition of new revenue dashboard 33
Two-factor authentication 18
Ability for CRM to send customised SMS83

Risk Reduction / Opportunity Enablement

Marketing believe that the new revenue dashboard could be a big differentiator against competing CRM’s, and want to plan a campaign around it. Thus the opportunity enablement is a 5.

A larger rollout of licenses from our corporate customer is hinging on TFA being delivered. TFA also reduces the risk of hackers into the system. With less risk and increased opportunity, we’ll rate this higher at an 8.

Customised SMS is a nice to have feature however wouldn’t necessarily reduce risk or enable opportunity. We rate that a 1.

FeatureUser-Business
value score
Time
Criticality
Risk-reduction /
Opportunity enablement
Addition of new revenue dashboard 335
Two-factor authentication 188
Ability for CRM to send customised SMS831

Now we can calculate our Cost of Delay.

FeatureUser-Business
value score
Time
Criticality
Risk-reduction /
Opportunity enablement
Cost of Delay
Addition of new revenue dashboard 33545
Two-factor authentication 18864
Ability for CRM to send customised SMS83124

We now have our Cost of Delay calculated. You can see that everything else being equal, we should work on the highest COD item first, and so on.

To calculate WSJF, we need to add a relative estimation of work effort from the Development team, and divide it by COD.

FeatureUser-Business
value score
Time
Criticality
Risk-reduction /
Opportunity enablement
Cost of DelayDev EffortWSJF 
(COD / Effort)
Addition of new revenue dashboard 33545133.46
Two-factor authentication 1886488
Ability for CRM to send customised SMS83124212

Interesting. Whilst based on COD alone, we would have worked on the TFA, look how WSJF has changed things. Based on development estimates, the customised SMS has moved from 3rd place into 1st place. With a high value/effort ratio some call this a ‘quick win’. We should then follow with TFA, with the new dashboards taking a backseat.

It’s worth seeing visually how the sequencing of items affects the value delivered over time.

Imagine we follow COD’s recommended approach and spend the first 8 weeks delivering TFA, followed by the new dashboards and then finally SMS. This is represented below (in blue), with each milestone representing the value delivered.

We then compare that to using the order prescribed by WSJF. You can see the relative value delivered over time below (in orange):

They both end up in the same place, as we eventually delivered the same features in both scenarios.

However if you measure the area of the orange vs the blue, you can see that WSJF has delivered 20% more value over time than CoD alone.

Can we do even better?

WSJF is great at providing clarity and insights in terms of prioritisation. It does have one Achilles Heel though; it is predicated on your relative estimations being accurate.

If we refer back to our ‘features as bets’ analogy, then WSJF provides more detail on how much we stand to win. The chance of winning becomes key in determining where to bet.

This is where the ICE model comes in.

The ICE Model

ICE stands for Impact vs Ease of Implementation *
Confidence. These are all rated on a 1-10 scale. With some simple maths, we can convert our previous WSJF scores into ICE terms:

  • Set the feature with the highest CoD @ 10, adjust other scores in relation
    • TFA (64) becomes 10
    • Dashboard (45) becomes 45 / 64 * 10 = 7.03
    • Custom SMS (24) becomes 24 / 64 * 10 = 3.75
  • Setting the feature with the lowest Dev Effort at 10, adjust other scores in relation
    • Custom SMS (2) becomes 10
    • Dashboard (13) becomes 2/13 * 10 = 1.54
    • TFA (8) becomes 2/8 * 10 = 2.5
FeatureImpactEase of ImplementationI*E
Addition of new revenue dashboard 7.031.5410.83
Two-factor authentication 102.5

25
Ability for CRM to send customised SMS3.751037.5

You can see Impact * Ease gives us the same priority of features as WSJF did; which we would expect, since we simply transposed those values.

Now let’s add a confidence variable. How do we become confident in any one bet delivering on the promised value?

The Confidence Model


Source: Gilad, 2019

As it turns out, gut-feel, pitch decks, fun with spreadsheets and buzzwords rank at near-zero on this model. Even business models, timelines and senior management’s opinions only get us to a ‘very low’ level of confidence.

The reason is that all of these are totally subjective and subject to bias. What you need is more data.

Let’s assess the confidence of our three features.

  • Dashboard: with only internal opinions and a business model document, this rates very low (0.4).
  • TFA: contractual agreements don’t appear on the model above, however for our purposes we’ll label this as a medium (5).
  • SMS: being a top customer request would give this a medium-low (3) score.

The team became concerned their estimate for the Ease of Implementation of SMS may have been optimistic, so a technical spike was scheduled. During this, their fears were confirmed; the team found out this wasn’t nearly as easy as they originally thought. The SMS back-end code is littered with technical debt they would need to refactor. Ease of Implementation needed to be adjusted down to a 5.

FeatureImpactEase of ImplementationConfidenceI * C * E
Addition of new revenue dashboard 7.031.540.44.33
Two-factor authentication 102.5

5125
Ability for CRM to send customised SMS3.755356.25

Interesting! The lack of confidence on the dashboards and overall ICE score means that project won’t be greenlit for development anytime soon.

However, it doesn’t mean it’s a bad idea. The team sets out to gather more data.

Due to the revised estimate of customised SMS, it also falls into 2nd place.

Which leaves TFA going into development in 1st place.

The great thing about this approach is you’re not required to dial it all the way up to 10. In some cases, doing so would be more effort than just developing the feature! For features that have a large pay-off, it makes sense that you could bet ‘earlier’ than those with a lower pay-off.

After taking a step back, the team set out to gather more information on dashboards and how real customers would use them. After a series of interviews and design mock-ups, the team has a new level of understanding. Rather than an entire dashboard full of metrics, the majority of customers really just wanted a widget to track late payments. This was their number #1 problem.

The team then did some market research to find that customers would be willing to solve this problem, over and above their standard subscription.

New ICE metrics were then calculated for the remaining two features.

FeatureImpactEase of ImplementationConfidenceI * C * E
New ‘late payment’ widget103390
Ability for CRM to send customised SMS3.755356.25

Turns out customised SMS was actually a dog to avoid building straight away. Recall that way back, this was something we previously had in 1st place!

By using CoD, WSFJ & ICE, we navigated a tricky situation and delivered maximum value to our users. Product superhero indeed.

References:

Gilad, I. (2019). The Tool that Will Help You Choose Better Product Ideas. [online] Hacker Noon. Available at: https://hackernoon.com/finding-winning-ideas-using-the-confidence-tool-d8f2d8cc2c15 [Accessed 19 Jan. 2019].

Scaledagileframework.com. (2019). WSJF – Scaled Agile Framework. [online] Available at: https://www.scaledagileframework.com/wsjf/ [Accessed 19 Jan. 2019].

Leave a Comment