Leveraging AI to Enhance Receipt Processing in the Household Expenditure Survey

In conducting the "Household Expenditure Survey" (HES), the Census and Statistics Department (C&SD) processes a large volume of receipts uploaded or submitted by households to analyse their spending patterns. These receipts come from various sources (such as supermarkets, restaurants, and e-commerce platforms), vary in format, and often contain a mixture of Chinese and English. Traditionally, staff manually identify and input key information such as merchant name, purchase date, item description, quantity, and amount. This manual process is time-consuming and prone to human error.

To address this challenge, C&SD launched a proof-of-concept initiative earlier this year, leveraging the Big Data Analytics Platform to apply Vision Language Models for the automated extraction of key information from receipts. Evaluation results confirmed that the approach is both feasible and effective. Through the proven structured workflow: receipt image ingestion → automated parsing → structured data output → sample-based human verification, this initiative automates highly repetitive and rule-based data extraction tasks.

This AI model was introduced in the latter part of the 2024/25 HES, and initial results have shown its capability to reduce manual data entry time. Upon full implementation in the next round of the HES (with further improvements in the accuracy and reliability of image-based data extraction and interface establishment with the computer system), the overall processing time for receipt images is projected to be reduced significantly by over 90%.

Looking ahead, C&SD will continue to leverage data science and AI to sustain the delivery of high-quality statistical services that meet the needs of society.