Invoke OCR on a Sensitive Document using Python and Enclaves

There are many cases in which a customer might be required to submit proof of identity — for example, banks and lenders often require customers to provide a passport or driver's licence in order to apply for a loan or a mortgage. This is often to prevent people from opening accounts using false identities.

One way to speed up verification is through the use of Optical Character Recognition (OCR), a process that converts an image of text into a machine-readable text format. There are many available stable OCR libraries and projects that can help with this process in a variety of programming languages.

However, because these documents contain Personally Identifiable Information (PII), they pose a risk to the customer and the institution if a threat actor were to capture them in the processing flow. The check is to make sure the customers are not fraudulent — so the last thing you'd want to do is make them vulnerable to those trying to do just that.

In this guide you will see how you can invoke the OCR process inside of an Evervault Enclave: a Docker container with your app that runs inside a cloud based Trusted Execution Environment (TEE), ensuring that nobody can access the document or any of the extracted PII data. This app will take an uploaded passport document and send it into a Enclave, where you will use a Python OCR library to extract the data and let the customer know whether their application is approved, denied, or requires more information.

Prerequisites

• Python version 3.6 or higher (install here)
• An Evervault account (create a free account here)
• Docker installed (get it here)

Set up

Clone the working GitHub repo.

To run Enclaves you will need to install the Evervault CLI by running the following command.

The Python app

The back end of the app that will run protected inside of the Enclave uses the PassportEye library to identify machine readable zones (MRZ) within an identity document and the pycountry to obtain the name of the country and nationality.

Once you have extracted the data from the document, you will format it into an object with all of the data.

PassportEye offers a measure called validity score where:

• If the score is less than 60 it is likely the document is fradulent
• If the score is moreo than 60 but less than 80, the document could be valid but is suspicious
• If the score is more than 80, the document is considered valid

In this example, you will define that a check has been approved if the validity score is more than 80, requires further review if between 60 and 80, and is denied if less than 60

For security reasons, you will only send back whether the document was approved, denied, or requires further review — you can alter the code if you would like to send back additional data.

The Dockerfile

Open up the Dockerfile. You will use a virtual environment to install the required libraries needed to run the app. You’ll also tell Docker your webserver will listen on port 8008, which matches the Flask server port defined in app.py.

Initialize the Enclave

First, make sure that you have Docker running. Then, get your Evervault API Key from your account.

In your terminal run the following command to initalize the Enclave. You can use the suggested name below or change it to one of your choosing.

You should see that a cert.pem, key.pem and enclave.toml are generated. Open up the enclave.toml. You can see that important details are generated that will help with the deployment of your Enclave.

Build the Enclave

Now build the Enclave using the following command. This will also generate a Dockerfile used to generate the EIF file which will run inside the Enclave (it may take a few minutes).

It will also provide a list of PCRs, cryptographic measurements that are required for the attestation process for the Enclave. These PCRs will be injected into your enclave.toml.

Deploy the Enclave

Finally, deploy the Enclave.

Setting up the Client

Copy the PCR values generated and add them into the empty JSON values that they correspond with in client/app.py.

The code will take the passport image that you upload and pass it into the Enclave. It will also calculate the time elapsed to run the Enclave so that you can monitor it to evaluate for performance.

Add Environment Variables

In order to upload the image as an encrypted file in the client/templates/form.html file, you will need to provide your app ID and team ID. Add these to the .env.example file, and rename the file to .env.

Run the Client

Now you can start the Flask server that will run the client/app.py code and serve the front-end so you can upload a passport image and test the app.

Run the following commands.

Now you can visit http://127.0.0.1:5000/form and try uploading a passport image to test the app. When working, you should recieve a browser alert with the status of the application.

Conclusion

In this guide, you used Python to invoke OCR on a sensitive identity document, and kept it safe throughout the process. If you ran into an issue or have any questions about this guide, feel free to raise them on GitHub. What other use cases will you use confidential OCR for?