JCV Insight SDK - Android Service SDK Developer Manual

Change History

Version No.
Date
Description
V2.0.0
2019-12-3
First release of GE (Global Edition).
V2.0.1
2019-12-18
Delete InsightFaceAttribute.
V2.0.2
2019-12-23
Fix minor issues.

1 Overview

1.1 Preparations for Access

The received SDK package contains the aar file for SDK integration, and the sample program.

1.2 SDK Description

The SenseInsight Customer Insight System is designed for offline business environment, providing analyses of and insights into the offline behavior of customer groups. It is committed to optimizing customers' offline experience by helping customers obtain accurate data or customize services in more efficient ways.

2 SDK Integration

2.1 Configuring AndroidManifest.xml File

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.CAMERA"/>

2.2 Introducing the SDK

    1.
    Put the aar file of the SDK into the libs folder under the module.
    1.
    Make the following changes in build.gradle under the module which introduces the SDK:
​x
android {
...
repositories {
flatDir {
dirs 'libs' // aar dir
}
}
}
dependencies {
...
//insight sdk
implementation(name: 'insight-sdk-1.0.0-release', ext: 'aar')
implementation 'com.squareup.retrofit2:retrofit:2.4.0'
}

2.3 SDK Initialization

xxxxxxxxxx
SDKParams params = SDKParams.builder()
.context(this.getApplicationContext())
.deviceCode(deviceCode)
.branchName("sensetime001")
.vendee("sensetime")
.licensePath("dev_license.lic")// License file path: assets or sdcard
.businessConfigPath(businessPath)
.build();
InsightSDK.init(params, new SDKInitCallback() {
@Override
public void success() {
Log.i("TAG", "insight sdk init success !");
}
@Override
public void failed(int code, String msg) {
Log.i("TAG", "insight sdk init failed : code " + code + "msg" + msg);
}
});

2.4 SDK Release

xxxxxxxxxx
InsightSDK.release();

2.5 Detailed Parameter Description for SDK Initialization

xxxxxxxxxx
//Optional parameters
HashMap<String, String> modelsConfig = new HashMap<>();
modelsConfig.put(SDKModels.KEY_FACE_ALIGN, "M_Align_occlusion_106_1.16.4.model");
modelsConfig.put(SDKModels.KEY_FACE_ATTRIBUTE, "M_Attribute_Face_Advertisement_3.5.0.model");
modelsConfig.put(SDKModels.KEY_FACE_DETECT, "MM_ObjectFlow_Face_SenseInsight_Common_BGR_1.0.1.model");
modelsConfig.put(SDKModels.KEY_FACE_VERIFY, "M_Verify_VehicleFace_Common_2.47.0.model");
modelsConfig.put(SDKModels.KEY_HEAD_POSE, "M_Align_CalcPose_Ann_106_2.4.0.model");
modelsConfig.put(SDKModels.KEY_BODY_DETECT, "M_Detect_Body_Hunter_1.5.0.model");
modelsConfig.put(SDKModels.KEY_BODY_ALIGN, "M_Detect_Body_Keypoints_5.0.17.model");
modelsConfig.put(SDKModels.KEY_BODY_ATTRIBUTE, "M_Attribute_Body_Fir_1.1.0.model");
SDKModelConfig sdkModelConfig = SDKModelConfig.builder()
.modelDirPath("model")// assets or sdcard path
.modelsConfig(modelsConfig)
.build();
SDKParams params = SDKParams.builder()
.branchAddress("branchAddress")
.deviceType(1)
.country("China")
.previewWidth(1280)
.previewHeight(720)
.tradingArea("")
.areaId(1)
.master(1)
.timeZone("GMT+8")
.isDebugMode(true)
.businessConfigPath(Environment.getExternalStorageDirectory()+ File.separator + "device.cfg")
.sdkModelConfig(sdkModelConfig)
.build();
InsightSDK.init(params, new SDKInitCallback() {
@Override
public void success() {
Log.i("TAG", "success : ");
}
@Override
public void failed(int code, String msg) {
Log.i("TAG", "failed : code " + code + "msg" + msg);
}
});

2.6 SDK configuration

xxxxxxxxxx
#
# SDK功能项开关 0-不启动 1-启动 / SDK function switch 0-close 1-open
#
# Enable face attribute (defaults:1)
function_switch_face_attribute=1
# Enable face watch (defaults: 1)
function_switch_face_watch = 1
# Enable face image crop (defaults:1)
function_switch_face_image_crop=1
# Enable face merge for statistics based on face feature extraction and comparison (defaults:1)
function_switch_face_recognition=1
# Enable body detection and tracking for statistics (defaults:0)
# Only when function_switch_body_track is set to 1, function_switch_body_attribution and function_switch_body_image_crop can be set to 1.
function_switch_body_track=0
# Enable body attribute (defaults:0)
# Only when function_switch_body_track is set to 1, this item can be set to 1.
function_switch_body_attribute=0
# Enable body image crop (defaults:0)
# Only when function_switch_body_track is set to 1, this item can be set to 1.
function_switch_body_image_crop=0
#
# SDK statistics uploading configuration
# items in SDK statistics uploading configuration are effective on the condition that function_switch_statistics_uploading is set to 1.
#
# Set time period to upload statistics data. unit: ms (defaults: 60000)
statistics_uploading_period=60000
# 0: Upload the coordinates of the face image with highest quality score for one face track;
# 1: Upload the coordinates of all face images fr one face track. (defaults: 0)
statistics_face_track_trail=0
#0: Upload the coordinates of the body image with highest quality score for one body track;
#1: Upload the coordinates of all body images for one body track. (defaults: 0)
statistics_body_track_trail=0
#
# SDK snap configuration
# items in SDK snap configuration are effective on the condition that function_switch_snap is set to 1.
#
#Set time period to upload snap data unit: ms (defaults: 500)
snap_uploading_period=500
#Enable face snap. 0: Not enable face snap 1: Enable face snap (defaults: 1)
snap_face=1
#Enable body snap. 0: Not enable body snap 1: Enable body snap (defaults: 0)
snap_body=0
#The maximum times of face snaps for one face track (defaults: 3)
snap_max_face_snap_times=3
#The maximum times of body snaps for one body track (defaults: 3)
snap_max_body_snap_times=3
#Enable face feature uploading 0: Not enable face feature uploading 1: Enable face feature uploading (defaults: 0)
snap_face_feature_uploading=0
#
# SDK face watch configuration
# items in SDK face watch configuration are effective on the condition that function_switch_face_watch is set to 1.
#
# The negative value of horizontal watch range
key_parameter_negative_yaw=-20
# The positive value of horizontal watch range
key_parameter_positive_yaw=20
# The negative value of vertical watch range
key_parameter_negative_pitch=-10
# The positive value of vertical watch range
key_parameter_positive_pitch=30
#
# SDK key parameter configuration
#
#0: Not call face attribute on a timed basis
#1: Call face attribute on a timed basis (default: 0)
#Once this parameter is set to 1, key_parameter_face_attribute_times will be ineffective and key_parameter_face_attribute_period be effective.
key_parameter_timed_face_attribute=0
# The time period to call face attribute. unit:ms (defaults: 1000)
key_parameter_face_attribute_period=1000
#The minimal face quality score. Only when face image quality for one face track is equal to or above this value, face attribution or face verify are to be called or face image is to be cropped. (defaults: 0.3)
key_parameter_min_face_quality=0.3
# The incremental face quality score. Only when the incremental quality score of a new face image related to the previous image quality for one face track, face attribution or face verify are to be called or face image is to be cropped one more time. (defaults: 0.2)
key_parameter_incremental_face_quality=0.2
# The minimal body quality score. Only when body image quality for one body track is equal to or above this value, body attribution is to be called or body image is to be cropped. (defaults: 0.3)
key_parameter_min_body_quality=0.3
# The incremental body quality score. Only when the incremental quality score of a new body image related to the previous image quality for one body track, body attribution is to be called or body image is to be cropped one more time. (defaults: 0.2)
key_parameter_incremental_body_quality=0.2
# The time range in which multiple face tracks of one person are merged for statistics. unit: min
key_parameter_face_merge_time_range=5
# face compare similarity threshold
key_parameter_face_similarity_threshold=0.6

3 Interface Description

3.1 offer

xxxxxxxxxx
InsightSDK.service().offer();

3.2 setConfig

xxxxxxxxxx
//After initialization of the SDK, you can update the parameters without re-initialization during the operation.
//Set the width and height for the Camera preview.
InsightSDK.config().setPreviewSize(cameraPreviewWidth, cameraPreviewHeight);
//Set the Debug mode. More detailed logs are available in the Debug mode.
InsightSDK.config().setDebugMode(true);
// Set the device information.
InsightSDK.config().setDeviceInfo(DeviceInfo.builder()
.deviceCode()
.deviceType()
.areaId()
.tradingArea()
.master()
.timeZone()
.country()
.branchAddress()
.branchName()
.build());
//SDK key parameter configuration
SDKConfig build = SDKConfig.builder()
.timedFaceAttribute(1)//Call face attribution on a timed basis
.faceAttributionPeriod(1000)//The time period to call face attribution.
.faceMergeTimeRange(5) // unit: min //The time range in which multiple face tracks of one person are merged for statistics.
.incrementalFaceQuality(0.2f)//The incremental face quality score. Only when the incremental quality score of a new face image related to the previous image quality for one face track, face attribution or face verify are to be called or face image is to be cropped one more time.
.incrementalBodyQuality(0.2f)//The incremental body quality score. Only when the incremental quality score of a new body image related to the previous image quality for one body track, body attribution is to be called or body image is to be cropped one more time.
.minBodyQuality(0.3f)//The minimal body quality score. Only when body image quality for one body track is equal to or above this value, body attribution is to be called or body image is to be cropped.
.minFaceQuality(0.3f)//The minimal face quality score. Only when face image quality for one face track is equal to or above this value, face attribution or face verify are to be called or face image is to be cropped.
.build();
InsightSDK.config().setSDKKeyParameterConfig(build);
InsightSDK.config().enableFaceAttribute(true);
InsightSDK.config().enableFaceWatch(true);
InsightSDK.config().enableFaceImageCrop(true);
InsightSDK.config().enableFaceRecognition(true);
InsightSDK.config().enableBodyTrack(true);
InsightSDK.config().enableBodyAttribute(true);
InsightSDK.config().enableBodyImageCrop(true);

3.3 currentTrackingFace

xxxxxxxxxx
//Get the current face information.List<TrackingFace> trackingFaces = InsightSDK.service().currentTrackingFace();

3.4 facesOfPastTimeRange

xxxxxxxxxx
// Get the face information (after deduplication) in a past time range.// The following forms are equivalent for calling.long now = System.currentTimeMillis();List<Face> faces = InsightSDK.service().facesOfPastTimeRange(now - 3000, now);List<Face> faces = InsightSDK.service().facesOfPastTimeRange(3000);

3.5 currentTrackingBody

xxxxxxxxxx
//Get the current body information.
List<TrackingBody> trackingBodies = InsightSDK.service().currentTrackingBody();

4 Type Description

4.1 TrackingFace

File Name
Field Type
Field Description
trackingFaceId
int
Face track id (no deduplication)
faceId
int
Face ID (unique ID of the face after deduplication)
enter
long
Time of entry
leave
long
Time of exit
gender
String
Gender (M: Male, F: Female)
age
int
Age
quality
float
Face recognition quality score (0 - 1)
faceRect
Rect
Face rectangle
watchTime
long
Watch time duration (ms)
expression
String
Facial expression
glasses
int
Glasses (0: no glasses, 1: has glasses, 2: sunglasses, -1: unknow)
mask
int
Mask (0: no mask, 1: has mask, -1: unknow)
smileScore
int
Smile score
charmScore
int
Charm score
faceFeature
byte[]
Face feature value
faceImage
StImage
Cropped image of face
faceImageQuality
float
Quality of cropped image of face
beard
int
Beard (0: no beard, 1: has beard, -1: unknow)
associatedBodyTrackID
int
associated body id

4.2 Face Attribute

Field Name
Field Description
Attribute Tag
Attribute Description
expression
Facial expression
others
Others
happy
Happy
calm
Calm
xxxxxxxxxx
</table>

4.3 Face

Field Name
Field Type
Field Description
faceId
int
face id (unique ID of the face after deduplication)
gender
String
Gender (M: Male; F: Female)
age
int
Age
faceImage
StImage
Cropped image of face
watchTime
long
Watch time (ms)
othersEmotionTime
long
Other expression time duration besides happy and calm (ms)
happyEmotionTime
long
Happy expression time duration (ms)
calmEmotionTime
long
Calm expression time duration (ms)
glasses
int
Glasses (0: no glasses, 1: has glasses, 2: sunglasses, -1: unknow)
mask
int
Mask (0: no mask, 1: has mask; -1: unknow)
charmScore
int
Charm score
faceFeature
byte[]
Face feature value
faceImageQuality
float
Quality of cropped image of face
beard
int
Beard (0: no beard, 1: has beard, -1: unknow)

4.5 TrackingBody

Field Name
Field Type
Field Description
trackId
int
Body Track Id
bodyQuality
float
Body quality(0 - 1)
bodyRect
StRect
body frame
associatedFaceTrackID
int
Face ID for the body (-1: no face matched)
attribute
BodyAttribute
Body attribute
age
String
Age (Adult, Elderly, Child)
gender
String
Gender (Male, Female)
bodyImage
StImage
Cropped image of body

4.5 BodyAttribute

Field Name
Return Type
Method Description
upperClothing
String
Type of the upper clothing (ShortSleeve, LongSleeve)
lowerClothing
String
Type of the lower clothing(Trousers, Shorts, Skirt)
upperMode
String
Model of the upper clothing (Pure, Stripe, Design, Joint, Lattic)
lowerMode
String
Model of the lower clothing (Pure, Stripe, Design, Joint, Lattic)
upperType
String
Style of the upper clothing (LongCoat, Jacket, T-Shirt, SportsWear, DownCoat, Shirt, Dress, BusinessSuit)
upperColor
String
Color of the upper clothing (Black, White, Gray, Red, Yellow, Blue, Green, Purple)
lowerColor
String
Color of the lower clothing (Black, White, Gray, Red, Yellow, Blue, Green, Purple)
age
String
Age (Adult, Elderly, Child)
gender
String
Gender (Male, Female)

4.6 Code table

Code
Description
100
CODE_OK
101
reinit sdk
103
running error
104
params illegal
201
algorightm no detect face
202
algorightm sdk release
203
algorightm no feature
204
algorightm unknown img format
205
algorightm img decode failed
206
algorightm img size illegal
207
algorightm params illegal
208
algorightm pls_enable face recognition
210
algorightm st exception

5 Code Obfuscation

Obfuscation settings in a project: Add the following codes to proguard-rules.pro:
xxxxxxxxxx
######insight sdk########
-keep class com.sensetime.** {*;}
########retrofit#######
#retrofit2 Obfuscation
-keepattributes Signature, InnerClasses, EnclosingMethod
-keepattributes RuntimeVisibleAnnotations, RuntimeVisibleParameterAnnotations
-keepclassmembers,allowshrinking,allowobfuscation interface * {
@retrofit2.http.* <methods>;
}
-dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement
-if interface * { @retrofit2.http.* <methods>; }
-keep,allowobfuscation interface <1>
# OkHttp3
-dontwarn javax.annotation.**
-keepnames class okhttp3.internal.publicsuffix.PublicSuffixDatabase
-dontwarn org.codehaus.mojo.animal_sniffer.*
-dontwarn okhttp3.internal.platform.ConscryptPlatform
# Okio
-dontwarn org.codehaus.mojo.animal_sniffer.*
########Insight-Gson##########
-keepattributes Signature
-keepattributes *Annotation*
-keep class sun.misc.Unsafe { *; }
-keep class com.sensetime.insight.gson.** {*;}

5 FAQ

    1.
    The currentTrackingFace interface fails to get face data. Why?
A: When failing to get the data, check the SDK for successful initialization, then check the module for connection to the device, and finally check the camera for correct direction.
    1.
    Can the currentTrackingFace interface be called simultaneously in multiple threads? A: No.
    2.
    Can the SDK be re-initialized after being released?
A: Yes, it can be re-initialized.
    1.
    Sometimes, the face attributes collected by the currentTrackingFace interface are empty (age and gender not available). Why?
A:When extracting an attribute, the module will judge it based on the quality of the recognized face. Therefore, sometimes it may fail to collect certain attributes. In the case of a high quality score, however, it can collect such attributes.
    1.
    What are the frequencies and situations for calling the currentTrackingFace interface?
A: You can call the currentTrackingFace interface in a situation for your application.
ex: 1) Call it every 30ms for child threads. The acquired data can be drawn on the screen.
2) Call it every second to get face data during the carousel.
3) Call it when an event occurs (for example, an event for purchase) to get the face data before the current screen...
xxxxxxxxxx
Last modified 18d ago