Integration Guideline [JavaScript]

This page contains instructions on how to integrate LIQA into your web service

LIQA library logic

LIQA represents a complex combination of multiple AI algorithms. Some of them require model weights to be fetched from the server. Also, some time is necessary for model compilation on the device. Finally, LIQA requires some DOM elements and settings to be provided to initialize the visualization correctly before the camera starts.

To manipulate all these actions, LIQA has a pipeline of simple methods:

These methods require the LIQA instance class to be already present on the page. See the API Documentation for all described methods for the API.

Next, we will use a general term of a web application with at least 1 page called Image Collecting Page - this is a webpage with LIQA loaded, initialized, and called for live video capturing and processing.

The load method with compileModels: true (see API Documentation) requires some time (see Few words about compiling models). This time can be "hidden" from an end-user experience in the case of a Single Page Application (SPA).

Integration with SPA (single-page application)

If your web application model is a single-page application (SPA) with several "user" pages before the image collecting page (e.g. welcome page, survey before image collection, etc.), the call of the load method with compileModels: true can be separated from the init and play methods. This will decrease the loading time on the image collecting page before the video appears.

Integration scheme for SPA. Used icons [*] and [**]

Integration for common websites

In case if your web application model is a SPA with only one page or a common website with independent pages, the load method can not be separated from init and play. In this case the best practise would be to call the load method with compileModels: false followed by calling the init and play methods on the same page.

Integration scheme for common websites. Used icons [*] and [**]

Connect to NPM registry

Request access and credentials

LIQA is distributed as a private NPM package with simple installation. Please follow the Try LIQA: access request instructions to contact us to get access to the registry. Leave us a message with a short description of your company, planned usage (test of technology embedding, small / medium / large business, etc.), and a contact e-mail so we could get back to you

After approving the request, you will be contacted via e-mail by Haut.AI and receive credentials for accessing the registry. For authorization, you will get your personal token as a long string:

• password - BASE64_ENCODED_PERSONAL_ACCESS_TOKEN

• .npmrc file

• registry_link - URL to set connection to the repository (smth like https://pkgs.dev.azure.com/haut-ai/.../npm/registry/)

• Please, pay attention that the credentials always have an expiration date.

• Please, store the provided credentials securely on your side

Create authorization files

• Go to your system Home directory:

  • for Windows (by default, installed on C drive): C:\Users\%username%

  • for Linux and MacOS: $HOME/

• Put the provided .npmrc file in your Home directory

• If you already have .npmrc file, open it and append the lines from the shared by Haut.AI

• Put your password instead of "<BASE64_ENCODED_PERSONAL_ACCESS_TOKEN>" placeholder

• Put your contact e-mail instead of <your contact email>

Install LIQA in your project:

• Call these commands in your project with the provided to you registry_link

npm config set @liqa:registry=registry_link

npm install --save @liqa/e-commerce@^5.0

Update LIQA in your project

Update LIQA to fetch the latest bug fixes:

npm update @liqa/e-commerce

Update LIQA to download the latest major version ( ⚠️ may influence API). Here x is the latest major version.

npm update @liqa/e-commerce^x.0 

Make sure to update scripts:

cp -r ./node_modules/@liqa/e-commerce/scripts ./path/to/public/assets

LIQA preparation

Library files location

Copy external scripts that will be fetched in LIQA:

• if you use LIQA as an npm package

cp -r ./node_modules/@liqa/e-commerce/scripts ./path/to/public/assets/

• if you use LIQA as an independent local library

cp -r ./liqa/scripts ./path/to/public/assets/

Loading & compilation

Import LIQA inside of your code:

import { Liqa } from '@liqa/e-commerce'

Initialize LIQA instance:

const liqa = new Liqa();

Load LIQA (fetch and optionally compile models):

liqa.load({
    staticPath: '/path/to/public/assets/',
    compileModels: true,
    useLocalModel: true
}).catch(err => console.error(err));

The duration of theloadmethod depends on the Internet connection speed. For example, it requires only 0.3s at 30Mbps if compileModels is set to false.

Few words about compiling models

You may turn off a model compilation during the load method, and in this case, the compilation will happen during the first frame of a video stream.

The model compilation is computationally intensive. It requires different times depending on hardware: from ~1s on modern devices to more than 10s on desktop or old devices.

Cache Header

Cache header allows your webpage to cache static files on user device. For LIQA, this allows to store neural networks model weights after being once downloaded.

Add <meta /> tag to your index.html:

<meta http-equiv="Cache-control" content="public" max-age="151200" />

The max-age=<N> response directive indicates that the response remains fresh until N seconds after the response is generated.

We do not recommend making this value > 2 days to keep your system always up to date (2 days = 151200 seconds)

Initialization

Provide a container and settings (see more at API Documentation):

liqa.init({
    container: container,
    width: window.innerWidth,
    height: window.innerHeight,
    config
});

Default initialization

Provide an empty div container. LIQA will create the necessary elements by itself:

<div ref="container"></div>

Manual initialization

Manually create a canvas and video elements inside of div container and pass it as a parameter. LIQA will use the provided elements instead of creating a new one:

<div ref="container">
  <video id="liqaVideo" style="-webkit-transform: scaleX(-1); transform: scaleX(-1); display: none;" playsinline controls></video>
  <canvas id="liqaCanvas"></canvas>
</div>

Please, note, that the manual initialization is not a default way of LIQA usage and you are responsible for the correctness of the provided elements. At the same time, the manual initialization with a custom div container creation may help in case of integration to third-party services and elements ownership issues.

Live video stream

Start stream:

await liqa.play();

Setup a subscription to RxJS Observable to receive LIQA results per frame.

liqa.qualityStatus$.subscribe( (options) => { 
    /* Here goes some actions to transform LIQA output to user feedback */
    updateStatus(options);
});

What to do with LIQA results?

LIQA can help your end-user to understand the image quality problem and fix it. LIQA returns simple and clear statuses of parameters, influencing image quality. Please, visit the qualityStatus page to learn more about the values that are returned.

Humans mostly perceive the information visually. We suggest your service transform LIQA results to visual user feedback by demonstrating warnings, status bars, and hints with only you to decide how this may look like.

To help you choose the best way to deliver the feedback to your end-user, we prepared several examples of LIQA results parsing with different levels of detail. You are welcome to start building your perfect feedback loop by investigating our examples:

• Level 1: "All or nothing"

• Level 2: "State the reason"

• Level 3: "Comprehensive feedback"

Selfie capturing

The main goal of LIQA usage is to help collect a good quality high-resolution image of an end-user face. LIQA returns results of image collection action as RxJS Observables. To get the results it is necessary to set up subscriptions before image collection actions may happen.

Set up a subscription for image collection

Set up a subscription to selfieImage RxJS Observable to receive a final high-resolution selfie image:

liqa.selfieImage$.subscribe( (selfie) => { 
    /* prepare actions with image */
    showImage(selfie);
    /* call liqa.play() if you want to run videostream again */
});

We highly recommend to show the resulting image to your end-user right after the selfie image collected. It might be also very helpful to allow the user to retake the image in case of failure.

Please, visit our example here: Selfie capturing results parsing

Set up a subscription for landmarks collection

From the 5.0 version, LIQA allows to predict landmarks (facial key points) on the resulting selfie image. Please, refer to Observables: Data Collection to see the map of landmarks.

Set up a subscription to selfieLandmarks RxJS Observable to receive an array of facial key points (landmarks) with (X, Y, depth) coordinates:

liqa.selfieLandmarks$.subscribe( (landmarks) => {
    /* prepare actions with landmarks visualization */
    showLandmarks(landmarks);
  });

Landmarks prediction is an optional behavior of LIQA. To activate it, set predictLandmarks parameter in config.json to true. If it is not activated, the selfieLandmarks observable would contain None

The predicted landmarks provide your system with a possibility for user engagement. The exact usage of the landmarks is limited mostly by your system design.

Please, visit our example here: Selfie capturing results parsing

Capture photo

From the 5.0 version, LIQA allows both manual and automatic image capturing when all the image quality requirements are satisfied. Simply speaking, when the image quality of the live stream is good enough, the system (auto) or the end-user (manual) can trigger a captureSelfie action. To allow a manual call, set up some UI with this call:

liqa.captureSelfie();

Autocapturing is an optional behavior of LIQA. To activate it, set autoSelfieCapture parameter in config.json to true. If it is not activated, the system would expect the manual selfie capturing control (by actions from an end-user)

The call of captureSelfiemethod before the playmethod is completed will return an error.

LIQA work finalization

When you are ready to finish LIQA work on the current page it is better to remove the container (provided in the init method) from the page by calling the unmount method:

liqa.unmount();

Finally, call the exit method to quit all work with LIQA.

liqa.exit();

On this page several free for commercial use materials were utilized for illustration purposes:

• * an icon made by Icongeek26 from https://www.flaticon.com/

• ** an icon made by wanicon from https://www.flaticon.com/