Two vendors published cold email reply rates for the same calendar year, using the same stated denominator. One said 0.45 per cent. The other said 3.43 per cent.

Neither is lying. Both are internally consistent. They are measuring different self-selected populations of their own customers, and because no standards body defines a reply rate for prospecting, both can be right about their own data and useless to you.

That is the whole problem with channel benchmarks in one example, and this piece is about how deep it goes. We set out to test a claim rather than assert it: that no neutral United States source publishes comparable performance data across prospecting channels. The claim survives, with three qualifications that turn out to be more interesting than the claim.

What the government actually measures

The federal statistical system is good at some things and completely silent on others, and the boundary is sharp.

We read the annual business survey questionnaire rather than a summary of it. The closest it comes to a channel question is asking which types of customer accounted for ten per cent or more of sales: federal government, state and local government, other businesses, other organisations, individuals. It asks who you sell to. It never asks how you found them. There is no question on marketing spend, advertising, prospecting or customer acquisition anywhere in the instrument.

The economic census does collect a purchased advertising and promotional services expense line, which is a genuine positive. It is one of eleven detailed operating expense items, collected only for construction, manufacturing and mining, as a single aggregate dollar figure. It tells you what an industry spent, never what it bought.

We grepped the small business advocacy office’s flagship statistical publication for “marketing”, “advertising”, “referral”, “word of mouth” and “customer acquisition”. Zero hits across all five. The federal office whose statutory job is researching small business concerns publishes nothing on how small businesses get customers.

The central bank’s small business survey has an entire chapter titled “Customers”. It contains two questions: which customer types account for ten per cent or more of sales, and how far customers are from headquarters.

What federal programmes measure about business sellingTwo column comparison of what United States federal statistical programmes measure about business selling and prospecting against what they do not measure. The left column records what is measured. The labour statistics agency publishes a median annual wage of seventy six thousand four hundred and sixty dollars, equivalent to thirty six dollars and seventy six cents per hour, across one million five hundred and seventy one thousand four hundred wholesale and manufacturing sales representative jobs in two thousand twenty five, with a projected employment change of zero per cent and a decline of seven thousand seven hundred positions over the decade to two thousand thirty five, alongside median pay for related occupations including sales engineers at one hundred and twenty four thousand nine hundred dollars and advertising sales agents at sixty four thousand eight hundred and twenty dollars. The economic census collects a single aggregate purchased advertising and promotional services expense figure, for construction, manufacturing and mining establishments only. The annual business survey and the central bank small business survey both record which customer types account for ten per cent or more of sales, and the latter also records the distance of customers from headquarters. The right column records what no federal programme measures: response rates by channel, connect rates, cost per lead, cost per acquisition, the share of revenue arising from referrals or word of mouth, and the time to first result on any channel. The comparison notes that the central bank survey states plainly that it is not a random sample and that results should be analysed with awareness of biases associated with convenience samples, publishes its weighting method and names its academic collaborator, and concludes that this disclosure is the standard against which commercial benchmark publishers should be judged.Measured precisely, and not measured at allFederal programmes measureSales rep median pay: $76,460Jobs in the occupation: 1,571,400Projected change to 2035: 0 %Advertising expense, in aggregate,for three sectors onlyWhich customer types buy from youThe cost of a sales function, well.No programme measuresResponse rates by channelConnect ratesCost per leadCost per acquisitionShare of revenue from referralsTime to first resultHow you found a customer: never asked.
Institutions measure the cost of a sales function with real precision. None of them measures what any channel returns.

Note what that leaves you with. The only figure in this entire field that comes from a probability-based federal survey is the cost of a salesperson: a median wage of $76,460 across roughly 1.57 million jobs, with projected employment change of zero per cent over the decade. You can cost the input precisely. You cannot source the output from anyone disinterested.

The honest counter-example, and why it matters

Before taking the vendors apart, it is worth naming what good disclosure looks like, because one institution in this area does it and the contrast is the real argument.

The central bank’s small business survey states in print that it “is not a random sample; results should be analyzed with awareness of potential biases that are associated with convenience samples”. It names the commercial databases it drew its sample from. It publishes its weighting method and names the academic organisation that developed it.

Even the direct mail trade association, whose benchmark we are about to criticise, prints twice in its own report that the data “should not be considered benchmark data”.

Institutions and honest trade bodies tell you their limits. The software vendors do not. That is the distinction worth carrying, and it is sharper than a blanket accusation of bias.

Tracing the numbers

Prospecting benchmarks traced to publisher and commercial interestTable tracing the most widely circulated performance benchmark for each business to business prospecting channel back to the organisation that publishes it and to what that organisation sells. For cold email, two vendors published reply rates for the same calendar year using the same stated denominator of replies divided by total emails sent, one reporting zero point four five per cent across seven and a half million emails and thirty four thousand three hundred and ninety three replies, the other reporting three point four three per cent across an undisclosed volume described as billions of interactions across thousands of workspaces; the first sells outsourced appointment setting, for which a low industry average is an argument, and the second sells the sending platform, lead database and deliverability tooling, for which a workable average is validation. For direct mail, the benchmark is produced by the trade association of the channel with a commercial research partner, sits behind a member login, and rested its headline house file response rate of fifteen point six per cent on twenty six self selected respondents. For cold calling, the most quoted average connect rate of five point four per cent with a top quartile of thirteen point three per cent is published by a revenue intelligence platform whose dataset consists of its own customers recorded calls, with no time period, sampling method or customer set disclosed. For trade shows, the figures are attributed to the research division of the exhibition organisers trade association, and four mutually incompatible cost per lead figures circulate under that single attribution without any traceable report title, year or sample. For social selling, the claim that social selling leaders create forty five per cent more opportunities and are fifty one per cent more likely to reach quota originates in an internal study by the platform itself, whose primary page returns a not found error, with no sample size, date or definition of opportunity published anywhere; the platform defines the index, computes each user score, publishes the validating research and sells the subscription that raises the score.Every benchmark, and who owns itChannelThe quoted figurePublished by someone who sellsCold email0.45 % and 3.43 % reply,same year, same denominatorOutsourced SDRs, and sending softwareDirect mail15.6 % response,from 26 respondentsThe channel’s own trade associationCold calling5.4 % connect rate,no period or method givenSoftware that records the calls measuredTrade showsFour incompatible figuresThe organisers’ trade associationSocial selling“45 % more opportunities”The platform that computes your score
For each channel, the most quoted figure and what the organisation publishing it sells. The measurement and the commercial interest are the same party throughout.

The social selling case is the purest, so it is worth stating fully. The platform defines the index, computes each user’s score, publishes the study validating the index, and sells the subscription that raises the score. The primary page for that study now returns a not-found error, and across the pages citing it, not one gives a sample size, a date, or a definition of “opportunity”.

Four figures to refuse outright, because we tried to trace them and could not:

The claim that the great majority of small businesses get customers by word of mouth comes from a vendor-commissioned survey of small business owners in one city, conducted in 2014.

The trust-in-recommendations figure everyone quotes is from fieldwork conducted in 2011, in a 56-country global survey of consumers with home internet access, about advertising formats. It is not US, not B2B, and measures a stated attitude rather than any behaviour. The same publisher’s later edition reports a materially lower number, and the two circulate interchangeably.

The most cited cold email response rate outside the vendor reports comes from a study of link building, guest posting and PR outreach to bloggers and editors, a population that expects to be pitched. It states no time period. It is routinely quoted as a sales benchmark.

And the trade show cost-per-lead comparison against a field sales call has the shape of a legacy advertising research number from decades ago, recirculated as current.

The channel everyone names first, and nobody measures

Referrals are the channel business owners cite before any other, and they are the one with no institutional data at all. We checked the small business advocacy office, the annual business survey, the central bank survey and the business trends survey. The word “referral” appears once across all of them, and it is about how a borrower chose a lender, not how a company found a customer.

That absence is itself worth stating. The most-cited channel in the country is measured by no federal statistical programme.

What fills the gap is worse than nothing, because it looks like evidence.

The two most quoted referral statistics traced to sourceAnalysis tracing the provenance of the two statistics most frequently cited in support of referral marketing to business audiences in the United States. The first claim, that the large majority of small businesses say word of mouth is the principal way new customers find them, originates in a survey conducted by a telecommunications company in conjunction with a small business publication in May two thousand fourteen, among small business owners in a single city. It is therefore a single city, vendor commissioned survey more than a decade old, circulating as a current national fact, published by a company that sells small business connectivity and marketing services. The second claim, that ninety two per cent of people trust recommendations from people they know above all forms of advertising, originates in global trust in advertising research whose fieldwork was conducted in August and September two thousand eleven, among more than twenty eight thousand consumers across fifty six countries, restricted to respondents with home internet access, weighted to be representative of internet consumers rather than of the population. The question asked to what extent respondents trust various forms of advertising, and the reported figure aggregates the responses trust completely and trust somewhat. It is therefore not United States specific, not business to business, and measures a stated attitude towards advertising formats rather than any purchasing behaviour. The same publisher later edition reports a materially lower figure of eighty three per cent, and the two numbers circulate interchangeably. The analysis contrasts both with the one rigorous source available, a peer reviewed study published in a marketing journal in two thousand eleven which tracked approximately ten thousand customers of a large German retail bank for nearly three years and found referred customers to be worth at least sixteen per cent more than matched non referred customers, while noting that this study is German, retail banking, business to consumer, single firm, and measures the value of a formal referral programme rather than the share of revenue arising from organic word of mouth.Two famous numbers, traced“Most small businesses get customers by word of mouth”A vendor-commissioned survey of small business owners in one city, May 2014.Circulating as a current national fact.“92 % trust recommendations from people they know”Fieldwork in 2011, 56 countries, consumers with home internet, about advertising formats.Not US, not B2B, an attitude not a behaviour. A later edition reports 83 %.The one rigorous source~10,000 bank customers tracked for three years: referred customers worth at least 16 % more. German, retail, B2C, one firm.
A single-city vendor survey from 2014, and consumer fieldwork from 2011 about advertising formats. Neither is B2B, and neither is current.

There is one piece of serious work here, and it is worth knowing precisely because it is so much narrower than the claims it gets used to support. A peer-reviewed study tracked roughly ten thousand customers of a large bank for nearly three years and found that referred customers were worth at least 16 per cent more than non-referred customers matched on demographics and acquisition timing, with higher margin and higher retention.

Note the caveats, because they are the point: it is German, it is retail banking, it is B2C, it is a single firm, and it measures the value of a formal referral programme rather than the share of revenue produced by organic word of mouth. It is also, by a distance, the best-evidenced statement anyone can make about referrals.

The channel nobody sells software for, and its free outcome data

There is one exception to everything above, and it is the channel with the least marketing around it.

Federal award data is published openly, and we queried it rather than reading about it. Per award, it returns the recipient, the amount, the awarding agency and sub-agency, start and end dates, the classification code for what was bought and a plain description. Ask for one award in detail and it also returns the solicitation identifier, the number of offers received, whether the work was competed at all and on what authority, the pricing type, the parent company, and named executives with compensation.

What free federal award data exposesSummary of the information a small business can obtain free of charge from published United States federal award data, contrasted with what is available for commercial prospecting channels. A search of award data returns for each contract the recipient name, the award amount, the awarding agency and sub agency, the period of performance, the industry classification code and description of what was purchased, and a plain text description of the work. Retrieving an individual award in detail additionally returns the solicitation identifier, the number of offers received, the extent to which the contract was competed and the statutory authority for any exception, the type of contract pricing, the subcontracting plan status, the parent company name and identifier, and the names and compensation of company executives. Awards can be filtered by set aside type and by industry classification code, so a company in a given classification can list by name and dollar value every competitor that won a small business set aside contract in its own field. Registration to bid is free, and the registering agency confirms that obtaining an identifier and maintaining the annual registration carries no charge, notwithstanding the existence of a paid third party registration industry. The summary contrasts this with commercial channels, where no equivalent record of who won which customer at what price exists at all, and notes that the relevant federal small business contracting goal is twenty three per cent of prime contracting dollars while the most recent reported achievement was nearly twenty eight per cent, representing one hundred and seventy nine billion dollars in prime contracts.The one channel with audited outcome dataWhat you can see, freeWho won it, and for how muchWhich agency bought itWhat was bought, and for how longHow many competitors bidWhether it was competed at allFilterable by set-aside and by sector code.What it means in practiceYou can list every competitor thatwon in your own sector codeYou can see when a contract endsRegistration to bid is free. Never payanyone to register you.No commercial channel offers any of this.
For one channel, the outcome data is public, free, granular and audited. It is the one nobody builds prospecting software for.

We ran a filtered query on small business set-aside awards within a single marketing services classification code and got back named companies with contract values in the millions, by agency. A firm in that sector can list, by name and dollar amount, every competitor that won a set-aside in its own code. There is no equivalent for any private channel, at any price.

Two further things make this channel structurally different rather than merely transparent.

Payment terms are set in regulation. Payment is due within a defined period after a proper invoice or after acceptance, interest runs automatically from the day after the due date without any claim being filed, and the rate is set by the treasury and published every six months. A further penalty applies if the interest itself is paid late.

Small business access is a statutory target, and it is exceeded. The government-wide goal is 23 per cent of prime contracting dollars. The most recent reported achievement is nearly 28 per cent, representing $179 billion in prime contracts.

None of which makes public bidding easy. It demands administrative competence and a long horizon, and it is the least improvisable channel in this article. But it is the only one where the targeting data is audited, the outcomes are public and the buyer’s payment behaviour is regulated.

The finding that reframes the whole argument

Here is where the bias framing stops being the point.

A peer-reviewed economics study examined twenty-five large randomised field experiments run with major retailers and brokerages, most reaching millions of customers, together representing millions of dollars of advertising spend. Its conclusions are uncomfortable for everyone in this discussion.

The median confidence interval on return on investment was over 100 percentage points wide. Individual sales are so volatile that, in the authors’ framing, an informative advertising experiment “can easily require more than 10 million person-weeks”, making such experiments costly and potentially infeasible for many firms. And because advertising is targeted, selection bias is described as a crippling concern for the observational methods everyone actually uses.

Why a reliable channel return on investment number is expensive for anyone to produceSummary of a peer reviewed economics study of twenty five large randomised field experiments in advertising, conducted with major retailers and brokerages, most reaching millions of customers. The study found that the median confidence interval on return on investment across those experiments was more than one hundred percentage points wide, meaning that even large randomised experiments frequently could not distinguish a substantial positive return from a substantial negative one. It attributes this to the volatility of individual level sales, noting that a coefficient of variation of ten is common, and concludes that informative advertising experiments can easily require more than ten million person weeks of exposure, which makes them costly and potentially infeasible for many firms. It further concludes that selection bias arising from the targeted nature of advertising is a crippling concern for the widely employed observational methods that firms and vendors use in place of experiments. The summary draws two consequences. First, the vendor benchmarks criticised elsewhere in this article are observational estimates drawn from self selected populations, which is precisely the method the study identifies as unreliable. Second, and more importantly for a small business reader, the same constraint applies to that reader attempting to measure their own channel return, so the appropriate response is not to search for a better published benchmark but to stop treating a precise return on investment figure as obtainable and to manage instead on measures that are directly observable.Twenty-five randomised experiments, and the resultMedian confidence interval on ROI: over 100 points wideLarge randomised experiments often could not separate a strong positive return from a strong negative one.An informative experiment: more than 10 million person-weeksBecause individual sales are extremely volatile. Costly, and infeasible for most firms.Observational methods: selection bias is cripplingWhich is exactly the method behind every vendor benchmark in the table above.
The bias framing suggests a better source would fix it. This finding says the number is not cheaply producible, by a vendor or by you.

Read that twice, because it changes what you should do. The bias framing invites you to think a better source would fix the problem. This says a better source is not economically producible, and that the same constraint applies to you measuring your own channels. The vendors are not withholding a number they have. Nobody has it.

There is genuine cross-channel causal work, and it looks nothing like a benchmark table: a peer-reviewed study of two retailers, using hierarchical models plus a randomised field experiment for causal support, concluding that direct mail drove acquisition while email drove sales to existing customers, with a modest and conditional reallocation gain. Two firms, two channels, a named funder, a published method, a careful result. Set that beside a 15.6 per cent response rate from twenty-six survey respondents.

What to do with this

Stop shopping for a benchmark. Whatever number you find will come from a company selling the channel, and the study above says the number they would need to produce honestly costs more than it is worth to them. Ask instead who published it and what they sell, and if you cannot answer in one sentence, do not put it in a plan.

Manage on what you can observe. Not modelled return, but counts a human recorded: contacts worked, conversations held, meetings booked, opportunities opened. They are crude and they are yours.

Cost the input precisely, because that part is knowable. A salesperson’s fully loaded cost is the one figure in this field with a federal probability survey behind it. Put your own version of it against the meetings actually produced, and you have a ratio that means something in your business even though it means nothing in anyone else’s.

Ask one question at the start of every discovery call: how did you come across us. Recorded by the person in the room rather than reconstructed from a model afterwards, it is the only attribution that survives a long sales cycle.

And if you sell anything a public agency buys, look at the award data before you buy a list. It is free, it names your competitors and their prices, and it tells you when the incumbent’s contract runs out. No commercial channel will give you that, because no commercial channel has it.

The related pieces are running a cold email campaign, cold calling rules and reality and what a trade show booth actually costs.