Belgian ID Cards: Issue with Document Numbers Exceeding 9 Characters in TD1

[Removed-5an]

Hi there, thank you for making this library!

I have an issue with TD1, specifically scanning Belgian ID cards. If the document_number_hash digit is "<" the document will not verify.

I have checked this with 3 different Belgian ID cards and they all have "<" on index 14 of line 0.

After a ton of googling and reading specs I found an issue with the way you check document_number_hash...

Normally a document number starts at position 5 and ends at position 13 but sometimes a document number exceeds the size of it's slot and optional fields will be used, let's take a look at this example:

IDBEL123456789<1233<<<<<<<<<<<

In this case the document number check is < when we have a scenario like that we need to look at the optional numbers (1233). So when the document number check is < we need to look at the last none empty value: 3. This is the actual hash number. After that we simply verify the hash of:

Document Number: 123456789<123
Hash: 3

And this should verify as True using your verify function.

from string import ascii_uppercase, digits

def hash_string(string: str) -> str:
    """
    >>> hash_string("ABCDEFGHIJKLMNOPQRSTUVWXYZ")
    '7'
    >>> hash_string("0123456789")
    '7'
    >>> hash_string("0123456789ABCDEF")
    '0'
    >>> hash_string("0")
    '0'
    """
    printable = digits + ascii_uppercase
    string = string.upper().replace("<", "0")
    weight = [7, 3, 1]
    summation = 0
    for i in range(len(string)):
        c = string[i]
        if c not in printable:
            raise ValueError("%s contains invalid characters" % string, c)
        summation += printable.index(c) * weight[i % 3]
    return str(summation % 10)

print(hash_string("123456789<123"))

[Arg0s1080]

Hi @5an1ty !

It seems that Belgium has 3 types of ID Cards:

The first one complies with ICAO specs (Belgian Citizens). The other two (Kids and Foreigners) work as you have explained, so It seems that Belgium is "twisting" the ICAO specifications.

According ICAO 9303-5 (TD1) 4.2.2.1, MRZ chars position will be 6 to 14 in line 1 for document number and 15 for document number hash, so this case is outside the scope of mrz. However, these problems can usually be solved simply by overwriting some property (see issue #3). In this case, overwriting document_number property.

A possible solution could be:

#!/usr/bin/python3
# -*- coding: utf-8 -*-

from mrz.checker.td1 import *
from mrz.base.functions import hash_is_ok


class TD1BELCodeChecker(TD1CodeChecker):
    @property
    def document_number_hash(self) -> bool:
        """Return True if the hash of the document number is validated, False otherwise."""
        if self._document_number_hash == "<":
            doc_number_fin = self._optional_data.rstrip("<")
            self._document_number = self._document_number + "<" + doc_number_fin[:-1]
            self._document_number_hash = doc_number_fin[-1]
        return self._report("document number hash", hash_is_ok(self._document_number, self._document_number_hash))

Usage:

from mrz.checker.td1_belgian import TD1BELCodeChecker

# CASE 1
code_citizens = ("IDBEL590330101085020100200<<<<\n"
                 "8502016F0901015BEL<<<<<<<<<<<8\n"
                 "VAN<DER<VELDEN<<GREET<HILDE<<<")

# CASE 2
mrz_code_kids = ("IDBEL000610035<7017<<<<<<<<<<<\n"
                 "0002015F0910190BEL000201002003\n"
                 "MAES<<SOPHIE<ANN<G<<<<<<<<<<<<")


td1_check_citz = TD1BELCodeChecker(code_citizens)
print("CASE 1:%s" % td1_check_citz)

td1_check_kids = TD1BELCodeChecker(mrz_code_kids)
print("CASE 2:%s" % td1_check_kids)

# CASE 3: Let's change document number hash
mrz_code_kids = ("IDBEL000610035<7010<<<<<<<<<<<\n"
                 "0002015F0910190BEL000201002003\n"
                 "MAES<<SOPHIE<ANN<G<<<<<<<<<<<<")

td1_check_kids = TD1BELCodeChecker(mrz_code_kids)
print("CASE 3:%s" % td1_check_kids)
print("FALSES CASE 3:")
print(td1_check_kids.report_falses)

Output:

CASE 1:True
CASE 2:True
CASE 3:False
FALSES CASE 3:
[('final hash', False), ('document number hash', False)]

This solution is valid for the 3 types of Belgian ID Cards. It's a very quick solution, so, I'm sure it can be improved. For example, if you want to report children and foreigners id cards as a warning:

    @property
    def document_number_hash1(self) -> bool:
        """Return True if the hash of the document number is validated, False otherwise."""
        ok = True
        if self._document_number_hash == "<":
            doc_number_fin = self._optional_data.rstrip("<")
            self._document_number = self._document_number + "<" + doc_number_fin[:-1]
            self._document_number_hash = doc_number_fin[-1]
            self._report("Possible Kids or Foreigners ID Card", kind=1)
            ok = not self._compute_warnings
        return self._report("document number hash",
                            ok and hash_is_ok(self._document_number, self._document_number_hash))

Output:

CASE 2:True
WARNINGS CASE 2:
['Possible Kid or Foreigner ID Card']

I hope I've helped.

Regards.

PS: I'm thinking that maybe it could be a good idea to create a folder to store all these special cases outside of ICAO specs

[Removed-5an]

Hi, thank you for replying!

It helps me a lot! However your explanation is not fully correct.

I have verified 3 full Belgian eID cards (not kids or foreigners) and they also don't follow the actual ICAO 9303-5 (TD1) 4.2.2.1 spec. They have the same exception as the kids and foreigners cards like you describe above. I guess it's mostly newer cards that have a high enough document number.

It would be nice indeed to also support special cases and have them in another folder.

By the way:
Unrelated to this issue, but it would be great if there was a function in your library that returns a dict of the parsed mrz.

[Arg0s1080]

Hi again!

I understand.. I'm from Spain and we also have 2 types of cards. In the old cards the national identification number is the document_number field, in the new cards that number is assigned to optional_data field and the document_number field is occupied by the number of the physical support of the cards (a real mess!)

I dont know if it's what you're looking for, but the library has several methods to report the result. For example, continuing with the previous example:

# CASE 3: Let's change document number hash
mrz_code = ("IDBEL000610035<7010<<<<<<<<<<<\n"
            "0002015F0910190BEL000201002003\n"
            "MAES<<SOPHIE<ANN<G<<<<<<<<<<<<")
td1_check = TD1BELCodeChecker(mrz_code)

print("CASE 3:%s" % td1_check)
print("\nList of tuples with all the fields analyzed:")
print(td1_check.report)

if bool(td1_check) == False:
    print("\nList of tuples (same as above but only returns Falses):")
    print(td1_check.report_falses)
    print("\nList with errors:")  # I've never liked it (it's possible that I can change or eliminate it)
    print(td1_check.report_errors)
    print("\nList with warnings:")  # same as above
    print(td1_check.report_warnings)

for field, result in td1_check.report:
    print(field.title().ljust(30, "."), result)

Output:

CASE 3:False

List of tuples with all the fields analyzed:
[('final hash', False), ('document number hash', False), ('birth date hash', True), ('expiry date hash', True), ('document type format', True), ('valid country code', True), ('valid nationality code', True), ('birth date', True), ('expiry date', True), ('valid genre format', True), ('identifier', True), ('document number format', True), ('optional data format', True), ('optional data 2 format', True)]

List of tuples (same as above but only returns Falses):
[('final hash', False), ('document number hash', False)]

List with errors:
['false final hash', 'false document number hash']

List with warnings:
['Possible Kid or Foreigner ID Card']

Final Hash.................... False
Document Number Hash.......... False
Birth Date Hash............... True
Expiry Date Hash.............. True
Document Type Format.......... True
Valid Country Code............ True
Valid Nationality Code........ True
Birth Date.................... True
Expiry Date................... True
Valid Genre Format............ True
Identifier.................... True
Document Number Format........ True
Optional Data Format.......... True
Optional Data 2 Format........ True

[imanenter]

hi Arg0s1080
thank u for ur nice code
i have same problem in generating belguim id card mrz
as u know Document number is 12 numbers and this app doesent accept it for ex 000590448 301
i tried to put ">301" in first optional data but first check number will go next to 8 ( it should be next to 1)
IDBEL0005904480<301<<<<<<<<<<< this is check number
whats ur idea about it how to generate belgium id card mrz code

[Arg0s1080]

hi Arg0s1080
thank u for ur nice code
i have same problem in generating belguim id card mrz
as u know Document number is 12 numbers and this app doesent accept it for ex 000590448 301
i tried to put ">301" in first optional data but first check number will go next to 8 ( it should be next to 1)
IDBEL000590448_0_<301<<<<<<<<<<< this is check number
whats ur idea about it how to generate belgium id card mrz code

Hi, whats up!

This issue was solved with a "special case". It´s possible to check Belgian id cards with this class, but I think there is nothing to generate its mrz code.

I'm very busy right now. However let me re-study this issue again and when I have a little free time I will try to find a solution.

BR