\u201cWould you like the receipt?\u201c\u2014It\u2019s hard to say no to that. Not because you actually want it (you may even throw it in the trash before exiting the store), but because doing otherwise might hurt the feelings of the cashier. But if you take a closer look, you will discover that a receipt carries all kinds of wonderful information. The most obvious data point is the total amount, which is useful in tracking your monthly expenses. Close to that you\u2019ll typically find a list of individual items. This you could use to inform your flatmate or spouse that you already bought the milk (but forgot the toilet paper) they wanted to pick up after work. The date of purchase might be useful if you need to decide if the eggs in the fridge are still safe to eat. Tracking receipts over a longer period of time might reveal trends in item prices and maybe even tell stories about your life, like that time you decided to eat more healthy food.<\/p>\n
Alas, all these use-cases require the information to be digital instead of being printed on thermal paper! Until retail decides to start using digital receipts, that means that you\u2019ll have to manually type the information into a computer\u2014and we can probably agree that while you may record some of that information (like the total amount), transcribing all of it is simply not realistic. But isn\u2019t this the age of automation where anything is possible? If we can put self-driving cars into space, we can surely digitize the information on a receipt, can\u2019t we? Well yes, we can. Google Lens, Evernote, Expensify, PaperScan and taggun.io are just some of the many apps and services you could use. So problem solved, right? OK, thanks, bye!<\/p>\n
But wait! Don\u2019t you want to know how<\/em> these apps do it? Then you have come to the right place, because in the following I will describe our take on recreating the basic functions of the aforementioned apps, show the engineering involved in tackling this task, describe what did not work the way we thought and why that may have been the case. And yes, there will be code so you can try it out for yourself!<\/p>\n Besides, receipt digitization is actually a great example for computer vision in general, where the goal is to extract information from raw pixels. The approaches and techniques outlined here can certainly be modified to cater to other use-cases as well. And believe it or not: receipt understanding is actually an active area of research\u2014take a look at this 2017 paper by Raoui-Outach et al.<\/a> or the ICDAR 2019 Robust Reading Challenge<\/a>.<\/p>\n <\/p>\n<\/div>