AI-Based Detection of Manipulated Receipt Date Fields Using Object Detection and Semantic Segmentation

Sammanfattning

Receipt fraud has become an increasing problem in financial systems due to thea vailability of digital editing tools that enable realistic document manipulation. This thesis investigates whether computer vision can be used to detect localized copy-move manipulations in receipt date fields. A prototype system was developed using a two-stage pipeline. First, a YOLO-based oriented bounding box model is used to locate the date field in a receipt image. Second, the detected date field is extracted and analyzed using a U-Net++ semantic segmentation model to localize possible manipulated pixels.The manipulation detection model is formulated as a binary semantic segmentation task, where each pixel in the detected date field is predicted as either manipulated or non-manipulated. The original receipt images are real-world samples, while manipulated examples are created through copy-move data augmentation by modifying existing receipt images. The segmentation output is then converted into a rule-based image-level verdict using a predefined probability threshold and connected component analysis. The final verdict is reported as low, medium, or high probability of manipulation.The work was carried out in collaboration with Fortnox and was constrained by limitations in dataset size, variation, and project time. The results show that the proposed pipeline can detect and localize small manipulated regions in receipt date fields under controlled experimental conditions. However, the system should be viewed as a proof-of-concept for copy-move manipulation detection rather than a complete receipt fraud detection solution.