The Problem with Using Average Settlement Value as Your Only Case Quality Metric

Average settlement value is the most commonly cited metric for evaluating case quality by source. It is also one of the most misleading. A single high-value outlier can mask a vendor that consistently produces low-value cases — and a single low-value outlier can make an excellent vendor look mediocre.

If you're using average settlement value as your primary or only measure of case quality, you are almost certainly making budget decisions on distorted data. Here is why — and what to use instead.

How One Outlier Distorts the Entire Picture

Vendor X sends your firm 80 leads per month. Over the past 12 months, 24 of those leads became signed cases and 18 have settled. Here are the settlement values:

• 14 cases settled between $35,000 and $65,000
• 2 cases settled between $80,000 and $95,000
• 1 case settled at $120,000
• 1 case settled at $780,000

The average settlement value across those 18 cases is $102,000. That sounds solid. But remove the single $780,000 case — which was a multi-vehicle accident with surgical intervention, an unusual case type for this vendor — and the average drops to $62,000.

The median tells the real story: $52,000. Half of the settled cases came in below $52,000. The $102,000 average is nearly double the median — a distortion caused by a single outlier representing 5.5% of the settled cases.

The Budget Decision This Distortion Creates

Based on the $102,000 average, a marketing director might conclude Vendor X is producing high-quality cases and increase their budget. Meanwhile, Vendor Y — with an average settlement of $88,000 but a median of $82,000 and no outliers — gets flagged as underperforming.

In reality, Vendor Y is producing consistently higher-value cases. Its tight distribution around $82,000 means you can predict case value reliably. Vendor X is producing predominantly low-value cases with an occasional windfall that inflates the average. These are fundamentally different vendor profiles — and the average settlement metric treats them as nearly identical.

For a firm spending $150,000 per month across vendors, a misguided budget shift of $30,000 from Vendor Y to Vendor X based on averages alone could reduce annual settlement revenue by $180,000 to $250,000. The wrong metric drives the wrong decision.

Why Average Fails: The Statistical Problem

Settlement values are not normally distributed. They follow a right-skewed distribution — most cases cluster in a lower range, with a long tail of high-value outliers. In any right-skewed distribution, the mean (average) is pulled upward by the tail, making it a poor measure of central tendency.

This is not an academic distinction. It has direct budget implications:

• A vendor with one $500K case and nine $40K cases has a $86,000 average — but nine out of ten cases are below $50,000. The average describes almost none of the actual cases.
• A vendor with ten cases between $70K and $100K has an $85,000 average — nearly identical. But every single case falls in a predictable, high-value range. This vendor is objectively more reliable.
• Both vendors show ~$86K average settlement — but they are entirely different in quality, predictability, and the kind of cases they produce.

What to Measure Instead

Average settlement value should be one of several metrics — not the only one. Here are the metrics that, together, give you an accurate picture of case quality by source:

Median Settlement Value

The median is the midpoint — half of cases settle above it, half below. It is resistant to outliers and gives you a more representative picture of “typical” case value from a source. When the median is significantly lower than the average, you have an outlier problem.

Settlement Value Distribution

Break settlements into ranges: under $50K, $50K-$100K, $100K-$200K, over $200K. Looking at what percentage of cases fall in each range tells you what a source actually produces — not what a lucky outlier suggests it produces.

Standard Deviation (or Interquartile Range)

How much variation exists in the settlement values? A low standard deviation means the vendor produces predictable case values — you can forecast revenue reliably. A high standard deviation means wide variance — some great cases, some poor ones, harder to plan around.

Settlement Rate

What percentage of signed cases from this source actually settle? A vendor with a $120,000 average but a 40% dismissal rate is very different from one with a $95,000 average and a 75% settlement rate. Average settlement value ignores dismissed cases entirely — creating a survivorship bias that overstates vendor quality.

Revenue Per Lead

Total settlement revenue divided by total leads from the source. This metric accounts for conversion rate, settlement rate, and settlement value in a single number. A vendor that sends 200 leads producing $1.2M in settlements ($6,000 per lead) is more valuable than one sending 50 leads producing $400,000 ($8,000 per lead) — if you have the intake capacity to handle the volume.

Segmenting Averages: When the Average Is More Useful

Average settlement value becomes more useful when you segment it properly. The problems described above are worst when you calculate a single average across all case types and all time periods for a given vendor. Segment the data and the outlier effect diminishes:

• Average by case type. Compare auto accident averages to auto accident averages — not auto accident to trucking. Case type is the primary driver of settlement range, so controlling for it removes the biggest source of variation.
• Average by time period.A vendor's case quality may change over time. Calculating separate averages by quarter reveals trends that a rolling 12-month average hides. If Q4 cases settle 30% lower than Q1 cases from the same vendor, that is an actionable signal.
• Average with outlier flags. Calculate the average both with and without outliers (cases more than 2 standard deviations from the mean). If the two numbers differ by more than 20%, the average is being distorted and the median is the better reference point.

Building a Case Quality Scorecard

The most effective approach is a vendor scorecard that combines multiple metrics into a holistic view. For each vendor, track:

This scorecard approach prevents any single metric from driving the entire evaluation. A vendor needs to perform well across multiple dimensions — not just have one impressive number that might be driven by outliers.

The Survivorship Bias Problem

Average settlement value has another subtle but significant flaw: it only includes cases that settled. Dismissed cases, cases that went to trial and lost, and cases the firm had to refer out are invisible in the average.

This creates a survivorship bias. A vendor whose leads produce 10 signed cases but only 4 settle is evaluated on those 4 settlements — ignoring the 6 cases that consumed attorney time, produced no revenue, and represent a 60% failure rate. The average settlement of the 4 successful cases might look fine. The overall return on marketing spend to that vendor is terrible.

Always pair settlement value metrics with settlement rate and case attrition data. The full picture includes the cases that did not work out — because you paid to acquire those too.

Making the Shift

If your firm currently evaluates vendors primarily on average settlement value, here is the path forward:

• Add median to every report. Start showing median alongside average in all vendor performance reports. When partners and stakeholders see both numbers, the conversations naturally become more nuanced.
• Flag outlier cases.Any case that settles at more than 3x the vendor's median should be flagged as an outlier. Calculate metrics with and without these cases to show both pictures.
• Add settlement distribution. Show the percentage of cases in each value range. This visual makes it immediately obvious when an average is being distorted by the tail.
• Include settlement rate. Report how many signed cases actually settle — not just what the settled cases are worth. This closes the survivorship bias gap.

A revenue intelligence platform calculates all of these metrics automatically, flagging when averages diverge from medians and surfacing the distribution data that spreadsheet-based reporting typically omits. The goal is not more data — it is the right data to make vendor decisions that actually connect to revenue.

Average settlement value is a useful data point. It is a dangerous decision point. The firms that recognize the difference — and build their vendor evaluation around a fuller set of metrics — make better budget decisions, avoid outlier-driven mistakes, and allocate marketing spend based on what vendors actually produce, not what one lucky case suggests they might.