8 Guidance to Advanced Users

This page provides some advice and technics on advanced usage of Mercury Cloud OpenAPI.

8.1 Advanced use of images

Typically, only one face from a single image is used in image-related APIs. But Mercury Cloud offers more powerful functions to fulfill the needs of different user scenarios.

8.1.1 Specific a region in the image

In some scenarios, you might not want to detect faces within the whole image but a specific region, such as a specific area of the ID card. A rectangle area can be specified in some APIs. With the rectangle area specified, the API will only scan face(s) in that region, and only the faces that overlap with that region will be returned. If no rectangle is specified, the whole image will be scanned. Incidentally, specifying a rectangle area is faster than processing the whole image, especially for HD images.

The rectangle field behaves the same in the Face Detection API, the Face Compare API, the Batch Add Faces API, and the Face Searching API.

Request Example

{
  "images": [
    {
      "data": "/9j/4AAQSkZJRgA...Tpi3Q1lZTCn//Z",
      "rectangle": {
        "top": 50,
        "left": 100,
        "width": 500,
        "height": 800
      }
    }
  ]
}

8.1.2 Multiple Faces Detection

The Face Detection function is used in several APIs, namely the Face Detection API, the Face Comparison API, the Quality Check API, the Batch Add Face API, and the Face Searching API. However, the detection behaviors are slightly different. The Face Detection API scans all detectable faces within the image. If an image contains multiple faces, the Face Detection API can detect all faces within the image. The batches.faces in the API response is a list containing all detected faces in the image.

But the Quality Check API, the Face Comparison API, the Batch Add Face API, and the Face Searching API only use the largest face in the image.

Response Example

{
  "results": [ { "code": 0, "message": "Success.", "internal_code": 0 } ],
  "batches": [
    {
      "faces": [
        {
          "rectangle": { "top": 771, "left": 403, "width": 695, "height": 718 },
          ...
        },
        {
          "rectangle": { "top": 92, "left": 274, "width": 132, "height": 130 },
          ...
        }
      ]
    }
  ]
}

8.1.3 Batch Image Detection

In the Face Detection API, the images field in the request and results and batches fields in the response are all list types. These lists can contain multiple images. The order of images in the images field in the request body and the order of items in results and batches fields in the response are strictly matched. You can use the same index value to access images, detect results, and detected faces.

Besides the Face Detection API, the Batch Add Face API also supports batch mode. Due to the performance considerations, the maximum number of images within a single request is restricted to 16, and each image should meet the image standards.

8.2 Advanced use of Key and Extra Info

While adding features to the feature database, it is recommended to add key or extra_info value along with the image, so that the added feature can be managed and accessed using more comprehensive methods other than the feature_id. If you add a face to the feature database several times, you will receive several feature_ids, and there may exist some duplicated features in the database. Without a key or an extra_info value, it is hard to manage if the feature_id is lost.

8.2.1 Key

The key of a feature is a user-defined string composed of alphabets, digits, hyphens ("-"), and up to 48 half-width characters long. If a feature is represented as a person in the database, the key is the index of that person. You can access a feature by a feature_id to retrieve its key and extra_info as well. The value of key is not necessarily to be unique across all feature databases. Therefore you can set an identical key value to a set of features.

In most user systems, the system will distribute a unique user_id for each user, for example, an employee number or a membership number. This unique user_id can be set as the key value in the Batch Add Feature API to map the user_id of the user system and the feature in Mercury Cloud. The user system should be responsible for maintaining the uniqueness of the key across feature databases instead of Mercury Cloud.

In the face identification (1:N) case, with a given user image, the API response contains the top results with the highest similarity compared to that face feature. The key in the result can help you quickly identify which user is the most similar feature belonging to. Then you can perform further business logic and process for that user agilely.

8.2.2 Extra Info

The Mercury Cloud does not offer dedicated columns to store user information. But Mercury Cloud provides a more flexible and advanced solution. The extra_info field is a user-defined string, able to store up to 1024 half-width characters. It can include any string types, including but not limited to a serialized JSON object, a Base64 encoded binary, or an URL of the user avatar. An example of a membership service can be structured as follows.

{
  "customer_info": {
    "id": 1000000,
    "first_name": "foo",
    "last_name": "bar",
    "create_at": "2021-01-01 15:00:00",
    "avatar": "base_url/2021/1000000_foo_bar.jpg"
  },
  "point_info": {
    "normal_points": {
      "card_no": 000011112222,
      "point": 100.00,
      "enabled": true,
      "expiration": "2022-01-01 00:00:00"
    },
    "2021_bonus_points": {
      "card_no": 000033334444,
      "point": 50.00,
      "enabled": false,
      "expiration": "2021-01-31 23:59:59"
    }
  }
}

In this example, since the name and avatar URL is included in extra_info, your system can rapidly extract the registered user avatar or user name without query your own database.

It is highly recommended to store only frequently used data in extra_info, as well as in Mercury Cloud. As a best practice, you should avoid saving sensitive personal data or do it at your own risk, even though Mercury Cloud provides high-level security measures to ensure the isolation of tenants and data security.

Starting from V1.5.0, you can update the extra_info by using the Update Feature Extra Info API.

8.3 Advanced use of Face Searching

The Face Searching API is the most important API in the 1:N identification scenario. Snapshots captured by cameras are compared with registered features in the feature database to find matching results.

8.3.1 About the search result score

A feature is a multi-dimensional vector. In the face identification scenario, the API compares a given feature with existing features in the database and calculates how close each pair is. Thescorein the API response ranging from 0 ~ 1, indicates the similarity between them. Different images, angles, brightness conditions, dates, etc. will affect the verification score. Even if you compare two identical images, there will be some differences in their similarity, though their value is very close to 1.

8.3.2 Min score and Top K

When sending a face searching API request, the min_score field can be used as a threshold to limit the lowest score in the response. Only results withscore equal to or higher than the min_score will be returned. You should decide your own min_score. Some common threshold values are set from 0.94 to 0.96 to avoid FAR as much as possible. But, please adjust and configure it based on your business requirements and test results.

Another field top_k is used to limit the number of results returned by API. In most cases, only the top one result is needed. But getting more top results may be helpful in some other cases. Notice that the min_score setting has a higher priority than top_k, meaning that if any of the top k features does not fulfill the min_score requirement, it will not be included in the API response.

8.3.3 Search across multiple feature databases

In some scenarios, your system may want to store features in multiple feature databases, such as separated feature databases for different regions.

Generally, the face searching API will only run in a single feature database (region). But in special cases, it is necessary to search across all feature databases (regions). The Face Searching API in Mercury Cloud provides this capability by supporting multiple db_id . When specify some db_id values, the API will search features across several feature databases and respond top results separately.

Notice that the Cross-database Face Search will increase back and forth network latency since it performs N times the database searches if the number of db_id in the API request is set to N.