Analytics Cases 2026: Why Precision Decides the Money

A business question before a method in the analytics case interview

The failure looks competent from the outside. A candidate hears churn, proposes a gradient boosted classifier, discusses feature engineering and cross validation, and answers a question nobody asked. The interviewer wanted to know what the client should do differently on Monday.

So begin with the decision. What action becomes available if the analysis succeeds, who takes it, and what does it cost to take. Until those three are on the table, no model specification can be evaluated, because there is nothing to evaluate it against.

Oliver Wyman publishes criteria under each question of its practice cases, and the opening question's criteria are the "Ability to identify the key question and pertinent information", the "Ability to structure a problem in a clear and effective manner", and the ability to communicate that structure succinctly, checked 29 September 2026 at oliverwyman.com. On the quantitative step of the same firm's other cases the criteria ask for the ability to "identify and ask for key information needed" before the ability to perform calculations. Two published criteria lists, both putting the question ahead of the arithmetic.

Analytics also arrives inside ordinary cases, which is worth knowing because the skill is tested whether or not the word appears. McKinsey's transformation case ends by handing over metrics for four competitor products and asking "What does this information tell us about the potential benefits", checked 30 September 2026 at mckinsey.com. That is an analytics question wearing a standard case label.

The roles are real and named, which helps when a candidate wants to know what the job is. ZS advertises a "Decision Analytics Associate" whose work is to "Design custom analyses in R, Tableau, SAS, Visual Basic and Excel to investigate and inform client needs", checked 29 September 2026 at jobs.zs.com. The tooling is listed and the purpose is to inform a decision.

Define the metric so two people compute it the same way

Most analytics disagreements are definitional, and they get discovered after the work is done. The discipline is to write the definition tightly enough that two analysts working separately would produce identical numbers.

Take churn at a subscription retailer with 200,000 customers. Definition one is a cancelled subscription. Definition two is no purchase in 90 days. Those are different populations and they support different programs.

Assume the true annual loss is 18 percent, so 18 percent of 200,000 is 36,000 customers who genuinely leave. Now assume that 12 percent of the base goes 90 days without a purchase and then returns, so 12 percent of 200,000 is 24,000 customers who look lost and are not.

Under definition two the labeled population is 36,000 plus 24,000, which is 60,000. That inflates the target list by 24,000 against the real 36,000, which is 66.7 percent more names. Every downstream number moves with it.

Three questions settle a metric definition, and they take a minute.

1. What is the unit? Customer, account, subscription or household. A family with three accounts churns three times under one definition and zero times under another.
2. What is the window? A rolling 90 days and a calendar quarter give different counts on the same data, and the difference is largest at period boundaries.
3. What is the event? An explicit cancellation, a payment failure, or an inferred lapse. Payment failure is often reversible, so counting it as churn overstates the problem and understates the fix.

Say the definition out loud before doing anything else. It costs twenty seconds and it prevents the whole analysis from being rebuilt.

Costing the campaign the model would trigger

Now cost the decision. The retailer wants a model that predicts which customers will leave, so it can send them a retention offer.

The offer costs $40 to deliver, including the discount and the handling. It persuades 25 percent of the genuine leavers who receive it. A saved customer is worth $260 of annual gross profit.

The data science team delivers a model at 60 percent recall and 20 percent precision. Recall of 60 percent means it catches 60 percent of the 36,000 leavers, which is 21,600. Precision of 20 percent means those 21,600 are a fifth of everyone it flags, so the flagged list is 21,600 divided by 0.20, which is 108,000 customers.

The campaign goes to everyone flagged, because the model cannot tell you which of its flags are right. So 108,000 contacts at $40 is $4.32 million of cost.

The benefit reaches only the genuine leavers. 25 percent of 21,600 is 5,400 customers saved, and 5,400 saved customers at $260 is $1.404 million.

The model is accurate in the sense that it finds most of the leavers, and acting on it destroys $2.916 million. That result is the content of the case, and it cannot be reached without costing the action.

Precision decides the money, recall decides the size

Write the net value as an expression and something useful falls out.

Let the flagged list be F and precision be p. True leavers inside the list are p times F. Benefit is p times F times 0.25 times $260, which is $65 times p times F. Cost is $40 times F. So net value is F times the quantity 65p minus 40.

Recall does not appear. It sets how large F can become for a given precision, so it sets the scale of the program, and it has no influence on whether each contact is worth making. A team that improves recall on a negative program makes the loss bigger.

The sign is decided where 65p equals 40, so at a precision of $40 divided by $65, which is 61.5 percent. Below that, every additional flagged customer loses money. Above it, every additional flagged customer makes money and recall becomes the lever that matters.

Check it at the threshold. At 61.5 percent precision and 60 percent recall, the flagged list is 21,600 divided by 0.615, which is 35,122. Cost is 35,122 at $40 is $1.405 million, and benefit is still 5,400 saved at $260, which is $1.404 million. Net is approximately zero, as the formula says.

So the answer to give is a number and a direction. The business case needs 61.5 percent precision, the model delivers 20 percent, and the gap is a factor of just over three that no amount of recall tuning closes. Say that before anyone spends another sprint on features.

The definition alone can cap precision below breakeven

Now bring the metric definition back, because it interacts with the threshold in a way that ends the project.

Suppose the team trains on definition two, no purchase in 90 days. The label marks 60,000 customers as churners while only 36,000 actually leave. A perfect model on that label flags exactly the 60,000 labeled customers.

Against the business outcome, its precision is 36,000 divided by 60,000, which is 60 percent. That is the ceiling. No model, however good, can exceed 60 percent precision against real churn while being trained to predict that label.

And 60 percent sits below the 61.5 percent the business case requires. So the program cannot pay under this definition, at this offer cost, with a perfect model. The definition alone has settled the question.

That is the single most valuable finding available in an analytics case, and it takes two divisions. It also tells the client precisely what to do first: tighten the label, or change the economics. Retraining on cancellations recovers the headroom, because the ceiling on precision then comes from the model rather than from the label.

The general lesson transfers. A label that is broader than the outcome you care about puts a hard cap on precision equal to the ratio between them. Checking that ratio against the breakeven precision is a two minute test that can save a year of work, and nobody runs it because it belongs to neither the analytics team nor the commercial team.

The cheapest lever is usually not the model

With the threshold expression in hand, every input can be ranked, and the model comes out low.

The required precision is the cost of contact divided by the value of a successful save. Halve the offer cost from $40 to $20 and the required precision becomes $20 divided by $65, which is 30.8 percent. Halving the cost halves the bar.

Raise the persuasion rate instead. If the offer works on 40 percent of leavers rather than 25 percent, the value per true positive becomes 0.40 times $260, which is $104, and the required precision becomes $40 divided by $104, which is 38.5 percent.

Do both and the bar falls to $20 divided by $104, which is 19.2 percent, which is below the precision the existing model already delivers. The program flips from losing $2.916 million to being viable without touching the model at all.

Read the fourth row against the model's 20 percent precision and the recommendation writes itself. Redesign the offer before retraining the model, because the offer is cheaper to change and it moves the threshold further.

This is what an analytics recommendation should look like. It names the binding constraint, prices two alternatives to relaxing it, and puts the model last because the arithmetic puts it last. A candidate who arrives at that ordering has demonstrated the judgment the case exists to test.

What would change the decision in a data analytics case

Close on the value of the analysis itself, which is the question a partner asks when a candidate finishes.

An analysis is worth at most the value of the decision it can change. Here the decision is a $4.32 million campaign that currently destroys $2.916 million, so an analysis that prevents it is worth $2.916 million and an analysis that cannot change the answer is worth nothing regardless of its rigor.

That gives a clean test to apply to any proposed piece of work. Name the two possible findings, say what the client would do under each, and if the answer is the same under both, the work should not be commissioned. Most requested dashboards fail that test.

It also gives an order for the next steps, and all three are cheap.

1. Restate the label. Retrain against cancellations rather than 90 day inactivity, which lifts the precision ceiling off 60 percent.
2. Reprice the offer. Test a $20 version, which drops the required precision to 30.8 percent.
3. Measure persuasion with a holdout. Randomly withhold the offer from a share of the flagged list, which is the only way to learn the 25 percent figure rather than assume it.

The tracks that firms hire for reflect this split. OC&C publishes an analytics route with an "Online Python Coding Test" and a first round with its "Analytics Team", and still runs "2 x 45-minute in-person case study interviews" at the second round, checked 29 September 2026 at occstrategy.com. The technical filter is real and it does not remove the business case.

Where an analytics answer loses its points

The weights below sit in our own grading code in src/lib/grading. They are ours, no consulting firm publishes anything equivalent, and the argument for each is on how an answer is assessed.