The challenge
Construction pricing demands reliable, up-to-the-minute data on commodities and activities. For cost consultants and quantity surveyors, competitive advantage often comes down to who has the best benchmarks - and who can update them fastest.
A leading international cost, project management and advisory consultancy had built exactly this advantage: a proprietary reference catalogue of material and construction rates, extracted from historic, approved bills of quantities (BQs). But as this data-driven capability matured, so did the volume of BQs requiring analysis, and the team's capacity risked becoming a bottleneck. Meanwhile, customers were asking whether the firm could help them extract similar value from their own historic data.
The firm approached Hoppa to explore how AI could scale their benchmarking capabilities without compromising accuracy.
Hoppa’s solution
Hoppa developed a data mining workflow for the client that automates the extraction of rates from Excel and PDF documents, classifying them according to RICS NRM (New Rules of Measurement) and POMI (Principles of Measurement) taxonomies. The workflow was able to extract and structure 200x more benchmark rates per day than a human operator.
Addressing the technical complexities of the use case required close attention to detail and collaboration between Hoppa and the client’s subject-matter-experts. This included:
Varied document structure
BQ document layout and breakdown varied considerably between projects, and even engineering disciplines. Estimates were often prepared by other suppliers, and the client had little control over how information was presented.
Hoppa’s workflow was designed to be able to handle either PDFs or Excel documents, ensuring the client could process the information as it was received from other suppliers. Hoppa developed specialist AI content extractors capable of handling non-standard input formats, combined with intermediate schema validation and automated retry mechanisms to ensure outputs conformed to the expected structure.
Long context windows
BQs can often run into the tens of thousands of rows, detailing rich supporting information alongside quantities estimates. When chunking BQs up for analysis there was a risk that quantity estimates would become separated from headings/sub-headings earlier in the document.
Hoppa developed a multi-step workflow to scan through the BQ and maintain a context log whilst isolating chunks containing estimates. This semantically dense context log allowed for workflow parallelisation of estimate schema normalisation steps – speeding up BQ analysis without impacting performance. This allowed Hoppa to tackle BQs up to 800 pages in length.
Technical jargon
BQs commonly contained technical and project-specific jargon that meant benchmark rates were unexplainable and of very limited value in the reference dataset.
Hoppa classified all rates to the Principles of Measurement International (POMI) system, an industry standard taxonomy. The client quantity surveyors could use this widely recognised taxonomy to search and find rates, irrespective of their source description. The Hoppa workflow also exported other contextual metadata such as references to construction specifications and drawings, and the type of asset the rate applied to so that downstream users could verify the suitability of the benchmark.
Customer outcomes
Through engaging Hoppa, the client opened up new strategic business opportunities:
- Slash database ingest times for near real-time rates insights
- Scale existing team without headcount constraints
- Free-up key personnel to re-focus on benchmark quality assurance
- Free-up key personnel to deliver new client-facing services for rates benchmarking
- Quickly establish rates benchmarks when expanding into new regions
“Out of the solutions we evaluated Hoppa was the clear winner for cost data structuring. We were impressed by their ability to automate what until now has been an intensive manual exercise: pulling cost line items, quantities and rates out of complex BQs, and classifying against our taxonomy. This automation is key, as we look to take new data-driven services to our clients the ability to process data near real-time has never been more important, and Hoppa helps us to close the gap.”
Looking ahead
Blending AI and industry expertise has ensured the client can realise near-term value return from AI investment while positioning itself as the project and cost management consultancy of the future.
As the automated rates benchmarking capability moves into scaled deployment, Hoppa and the client will continue to apply the methodology to other project / cost management and advisory use cases.
Feeling inspired?
See Hoppa in action and learn how it can make the difference to your workflows.
