Measurement
Part of Women founder funding measurement: stages, metrics and impact limits
How to audit women founder funding measurement for mistakes
Audit women founder funding measurement for mistakes: sample every case outcome, rebuild the figures, test claims and subgroups, then log and repair defects.
An audit of women founder funding measurement starts from the cases, not from the dashboard. A figure is checked by rebuilding it from source records for a sample of founders, then by testing each public claim, subgroup table and series against the measure record. Defects are logged, repaired and tested again.
What to take away
- Sample funded, rejected, withdrawn, pending and unreachable founders, not only the successes.
- Rebuild each stage, denominator contribution and finance amount from source records.
- Flag claims that say enabled, delivered or generated when the evidence shows only sequence or association.
- Inspect small groups and combined years before trusting a subgroup table.
- Give every defect an owner, then repeat the audit after repair using different cases.
Step 1: draw a sample that includes the failures
Select a sample that includes funded, rejected, withdrawn, pending and unreachable founders. A sample of successes alone will reproduce the report's flattering picture and test nothing.
Reproduce each founder's stage, denominator contribution and finance amount from source records. Separate requested, offered, accepted, received and usable money, along with the observation date. Where one business appears more than once, check whether the dashboard would change if duplicate applications were handled under a different plausible rule.
Retain unsuccessful and abandoned cases where lawful, because removing incomplete journeys exaggerates completion and can hide access barriers.
Step 2: test each public claim against the record
Review public claims against the underlying dictionary. Flag words such as enabled, delivered or generated when the evidence shows only sequence or association. Require the claim owner to cite the design that supports attribution.
An observed outcome and responsibility for it are separate questions. HM Treasury's impact-evaluation quality guidance explains this, and revenue, jobs or investment may reflect markets and other support rather than the service.
Use the Treasury's current Magenta Book to separate process, impact and value-for-money questions. Engage an evaluator before promising causal results, then publish uncertainty and limitations with the finding. A claim about return on investment needs the same design behind it.
Step 3: inspect subgroup tables and series
Inspect subgroup tables for tiny counts, wide uncertainty and selective missingness. Test whether combining years, routes or regions has hidden a definition change. Where a stable estimate is impossible, report counts or a qualitative finding with its method rather than manufacturing precision.
Detailed region, ethnicity, disability and route combinations can identify people or produce unstable percentages. Apply statistical and data-protection review, then combine or suppress where needed. Distinguish administrative data, self-report, forecast and independent verification, and keep the uncertainty checks beside each table.
Step 4: check what the figure is compared with
A national figure is context, not a service target. The Investing in Women Code has over 300 signatories, and the Code's 2026 annual report reflects participating UK finance providers and agreed reporting. It is not a like-for-like benchmark for every programme, location or applicant mix.
A pound of grant, personal debt and equity does not carry the same cost or obligation. Report route-specific outcomes before any responsible aggregate, and settle the comparison basis first, including the mixed-team rule beside the figure.
Step 5: log, repair and test again
Give every defect an affected period, severity, owner and correction route. Notify previous recipients when an error could have changed a decision. Record why the control failed, then alter the collection, validation or approval process.
| Defect found | How the audit finds it | Repair |
|---|---|---|
| Women-led left undefined | No definition or mixed-team rule sits beside the figure | State the definition, the evidence and the rule |
| Hidden denominator | Numerator, denominator and unresolved cases cannot be reproduced | Publish all three together |
| Applications counted as applicants | A different duplicate rule changes the result | Choose one unit and deduplicate under a recorded method |
| Offer treated as cash | Offered and received amounts differ in the sample | Separate requested, offered, accepted, received and usable money |
| Withdrawals dropped | Completion differs between funded cases and the whole sample | Report attrition at each stage |
| Volume rewarded | Enquiries rise while responses slow or records go stale | Pair reach with source errors, founder effort and complaints |
Repeat the audit after repair using different cases. Closing a ticket without confirming the new output leaves the measurement risk in place.
Before you act
- Draw a sample that includes funded, rejected, withdrawn, pending and unreachable founders.
- Rebuild stage, denominator contribution and finance amount from source records.
- Check each public claim against the dictionary and the design behind it.
- Inspect subgroup tables for tiny counts and hidden definition changes.
- Log each defect with period, severity, owner and correction route.
- Repeat the audit with different cases after repair.
Common questions
What should an audit sample include?
Funded, rejected, withdrawn, pending and unreachable founders. Reproduce their stage, denominator contribution and finance amount from source records, then check whether the dashboard would change under a different plausible duplicate rule.
What should happen when an audit finds an error?
Record its affected period, severity, owner and correction route, and notify previous recipients when it could have changed a decision. Then alter the collection, validation or approval process that let it through.
Why repeat the audit after repair?
Because closing a ticket without confirming the new output leaves the measurement risk in place. Use different cases the second time so the repair is tested rather than the old sample.
