JCV Insight SDK - Android Service SDK Developer Manual

Change History

Version No.	Date	Description
V2.0.0	2019-12-3	First release of GE (Global Edition).
V2.0.1	2019-12-18	Delete InsightFaceAttribute.
V2.0.2	2019-12-23	Fix minor issues.

1 Overview

1.1 Preparations for Access

The received SDK package contains the aar file for SDK integration, and the sample program.

1.2 SDK Description

The SenseInsight Customer Insight System is designed for offline business environment, providing analyses of and insights into the offline behavior of customer groups. It is committed to optimizing customers' offline experience by helping customers obtain accurate data or customize services in more efficient ways.

2 SDK Integration

2.1 Configuring AndroidManifest.xml File

<uses-permission android:name="android.permission.INTERNET"/>

<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>

<uses-permission android:name="android.permission.CAMERA"/>

2.2 Introducing the SDK

1. Put the aar file of the SDK into the libs folder under the module.

1. Make the following changes in build.gradle under the module which introduces the SDK:

​x

android {

...

repositories {

flatDir {

dirs 'libs' // aar dir

}

}

}

​

dependencies {

...

//insight sdk

implementation(name: 'insight-sdk-1.0.0-release', ext: 'aar')

implementation 'com.squareup.retrofit2:retrofit:2.4.0'

}

2.3 SDK Initialization

xxxxxxxxxx

SDKParams params = SDKParams.builder()

.context(this.getApplicationContext())

.deviceCode(deviceCode)

.branchName("sensetime001")

.vendee("sensetime")

.licensePath("dev_license.lic")// License file path: assets or sdcard

.businessConfigPath(businessPath)

.build();

InsightSDK.init(params, new SDKInitCallback() {

@Override

public void success() {

Log.i("TAG", "insight sdk init success !");

}

​

@Override

public void failed(int code, String msg) {

Log.i("TAG", "insight sdk init failed : code " + code + "msg" + msg);

}

});

2.4 SDK Release

xxxxxxxxxx

InsightSDK.release();

2.5 Detailed Parameter Description for SDK Initialization

xxxxxxxxxx

//Optional parameters

HashMap<String, String> modelsConfig = new HashMap<>();

modelsConfig.put(SDKModels.KEY_FACE_ALIGN, "M_Align_occlusion_106_1.16.4.model");

modelsConfig.put(SDKModels.KEY_FACE_ATTRIBUTE, "M_Attribute_Face_Advertisement_3.5.0.model");

modelsConfig.put(SDKModels.KEY_FACE_DETECT, "MM_ObjectFlow_Face_SenseInsight_Common_BGR_1.0.1.model");

modelsConfig.put(SDKModels.KEY_FACE_VERIFY, "M_Verify_VehicleFace_Common_2.47.0.model");

modelsConfig.put(SDKModels.KEY_HEAD_POSE, "M_Align_CalcPose_Ann_106_2.4.0.model");

modelsConfig.put(SDKModels.KEY_BODY_DETECT, "M_Detect_Body_Hunter_1.5.0.model");

modelsConfig.put(SDKModels.KEY_BODY_ALIGN, "M_Detect_Body_Keypoints_5.0.17.model");

modelsConfig.put(SDKModels.KEY_BODY_ATTRIBUTE, "M_Attribute_Body_Fir_1.1.0.model");

​

SDKModelConfig sdkModelConfig = SDKModelConfig.builder()

.modelDirPath("model")// assets or sdcard path

.modelsConfig(modelsConfig)

.build();

​

SDKParams params = SDKParams.builder()

.branchAddress("branchAddress")

.deviceType(1)

.country("China")

.previewWidth(1280)

.previewHeight(720)

.tradingArea("")

.areaId(1)

.master(1)

.timeZone("GMT+8")

.isDebugMode(true)

.businessConfigPath(Environment.getExternalStorageDirectory()+ File.separator + "device.cfg")

.sdkModelConfig(sdkModelConfig)

.build();

InsightSDK.init(params, new SDKInitCallback() {

@Override

public void success() {

Log.i("TAG", "success : ");

}

​

@Override

public void failed(int code, String msg) {

Log.i("TAG", "failed : code " + code + "msg" + msg);

}

});

2.6 SDK configuration

xxxxxxxxxx

​

​

​

#

# SDK功能项开关 0-不启动 1-启动 / SDK function switch 0-close 1-open

#

​

# Enable face attribute (defaults:1)

function_switch_face_attribute=1

​

# Enable face watch (defaults: 1)

function_switch_face_watch = 1

​

# Enable face image crop (defaults:1)

function_switch_face_image_crop=1

​

# Enable face merge for statistics based on face feature extraction and comparison (defaults:1)

function_switch_face_recognition=1

​

# Enable body detection and tracking for statistics (defaults:0)

# Only when function_switch_body_track is set to 1, function_switch_body_attribution and function_switch_body_image_crop can be set to 1.

function_switch_body_track=0

​

# Enable body attribute (defaults:0)

# Only when function_switch_body_track is set to 1, this item can be set to 1.

function_switch_body_attribute=0

​

# Enable body image crop (defaults:0)

# Only when function_switch_body_track is set to 1, this item can be set to 1.

function_switch_body_image_crop=0

​

​

#

# SDK statistics uploading configuration

# items in SDK statistics uploading configuration are effective on the condition that function_switch_statistics_uploading is set to 1.

#

​

# Set time period to upload statistics data. unit: ms (defaults: 60000)

statistics_uploading_period=60000

​

# 0: Upload the coordinates of the face image with highest quality score for one face track;

# 1: Upload the coordinates of all face images fr one face track. (defaults: 0)

statistics_face_track_trail=0

​

#0: Upload the coordinates of the body image with highest quality score for one body track;

#1: Upload the coordinates of all body images for one body track. (defaults: 0)

statistics_body_track_trail=0

​

#

# SDK snap configuration

# items in SDK snap configuration are effective on the condition that function_switch_snap is set to 1.

#

​

#Set time period to upload snap data unit: ms (defaults: 500)

snap_uploading_period=500

​

#Enable face snap. 0: Not enable face snap 1: Enable face snap (defaults: 1)

snap_face=1

​

#Enable body snap. 0: Not enable body snap 1: Enable body snap (defaults: 0)

snap_body=0

​

#The maximum times of face snaps for one face track (defaults: 3)

snap_max_face_snap_times=3

​

#The maximum times of body snaps for one body track (defaults: 3)

snap_max_body_snap_times=3

​

#Enable face feature uploading 0: Not enable face feature uploading 1: Enable face feature uploading (defaults: 0)

snap_face_feature_uploading=0

​

​

#

# SDK face watch configuration

# items in SDK face watch configuration are effective on the condition that function_switch_face_watch is set to 1.

#

​

# The negative value of horizontal watch range

key_parameter_negative_yaw=-20

​

# The positive value of horizontal watch range

key_parameter_positive_yaw=20

​

# The negative value of vertical watch range

key_parameter_negative_pitch=-10

​

# The positive value of vertical watch range

key_parameter_positive_pitch=30

​

​

#

# SDK key parameter configuration

#

​

#0: Not call face attribute on a timed basis

#1: Call face attribute on a timed basis (default: 0)

#Once this parameter is set to 1, key_parameter_face_attribute_times will be ineffective and key_parameter_face_attribute_period be effective.

key_parameter_timed_face_attribute=0

​

# The time period to call face attribute. unit:ms (defaults: 1000)

key_parameter_face_attribute_period=1000

​

#The minimal face quality score. Only when face image quality for one face track is equal to or above this value, face attribution or face verify are to be called or face image is to be cropped. (defaults: 0.3)

key_parameter_min_face_quality=0.3

​

# The incremental face quality score. Only when the incremental quality score of a new face image related to the previous image quality for one face track, face attribution or face verify are to be called or face image is to be cropped one more time. (defaults: 0.2)

key_parameter_incremental_face_quality=0.2

​

# The minimal body quality score. Only when body image quality for one body track is equal to or above this value, body attribution is to be called or body image is to be cropped. (defaults: 0.3)

key_parameter_min_body_quality=0.3

​

# The incremental body quality score. Only when the incremental quality score of a new body image related to the previous image quality for one body track, body attribution is to be called or body image is to be cropped one more time. (defaults: 0.2)

key_parameter_incremental_body_quality=0.2

​

# The time range in which multiple face tracks of one person are merged for statistics. unit: min

key_parameter_face_merge_time_range=5

​

# face compare similarity threshold

key_parameter_face_similarity_threshold=0.6

​

​

​

3 Interface Description

3.1 offer

xxxxxxxxxx

InsightSDK.service().offer();

3.2 setConfig

xxxxxxxxxx

//After initialization of the SDK, you can update the parameters without re-initialization during the operation.

​

//Set the width and height for the Camera preview.

InsightSDK.config().setPreviewSize(cameraPreviewWidth, cameraPreviewHeight);

​

//Set the Debug mode. More detailed logs are available in the Debug mode.

InsightSDK.config().setDebugMode(true);

​

// Set the device information.

InsightSDK.config().setDeviceInfo(DeviceInfo.builder()

.deviceCode()

.deviceType()

.areaId()

.tradingArea()

.master()

.timeZone()

.country()

.branchAddress()

.branchName()

.build());

//SDK key parameter configuration

SDKConfig build = SDKConfig.builder()

.timedFaceAttribute(1)//Call face attribution on a timed basis

.faceAttributionPeriod(1000)//The time period to call face attribution.

.faceMergeTimeRange(5) // unit: min //The time range in which multiple face tracks of one person are merged for statistics.

.incrementalFaceQuality(0.2f)//The incremental face quality score. Only when the incremental quality score of a new face image related to the previous image quality for one face track, face attribution or face verify are to be called or face image is to be cropped one more time.

.incrementalBodyQuality(0.2f)//The incremental body quality score. Only when the incremental quality score of a new body image related to the previous image quality for one body track, body attribution is to be called or body image is to be cropped one more time.

.minBodyQuality(0.3f)//The minimal body quality score. Only when body image quality for one body track is equal to or above this value, body attribution is to be called or body image is to be cropped.

.minFaceQuality(0.3f)//The minimal face quality score. Only when face image quality for one face track is equal to or above this value, face attribution or face verify are to be called or face image is to be cropped.

.build();

InsightSDK.config().setSDKKeyParameterConfig(build);

InsightSDK.config().enableFaceAttribute(true);

InsightSDK.config().enableFaceWatch(true);

InsightSDK.config().enableFaceImageCrop(true);

InsightSDK.config().enableFaceRecognition(true);

InsightSDK.config().enableBodyTrack(true);

InsightSDK.config().enableBodyAttribute(true);

InsightSDK.config().enableBodyImageCrop(true);

3.3 currentTrackingFace

xxxxxxxxxx

//Get the current face information.List<TrackingFace> trackingFaces = InsightSDK.service().currentTrackingFace();

3.4 facesOfPastTimeRange

xxxxxxxxxx

// Get the face information (after deduplication) in a past time range.// The following forms are equivalent for calling.long now = System.currentTimeMillis();List<Face> faces = InsightSDK.service().facesOfPastTimeRange(now - 3000, now);List<Face> faces = InsightSDK.service().facesOfPastTimeRange(3000);

3.5 currentTrackingBody

xxxxxxxxxx

//Get the current body information.

List<TrackingBody> trackingBodies = InsightSDK.service().currentTrackingBody();

4 Type Description

4.1 TrackingFace

File Name	Field Type	Field Description
trackingFaceId	int	Face track id (no deduplication)
faceId	int	Face ID (unique ID of the face after deduplication)
enter	long	Time of entry
leave	long	Time of exit
gender	String	Gender (M: Male, F: Female)
age	int	Age
quality	float	Face recognition quality score (0 - 1)
faceRect	Rect	Face rectangle
watchTime	long	Watch time duration (ms)
expression	String	Facial expression
glasses	int	Glasses (0: no glasses, 1: has glasses, 2: sunglasses, -1: unknow)
mask	int	Mask (0: no mask, 1: has mask, -1: unknow)
smileScore	int	Smile score
charmScore	int	Charm score
faceFeature	byte[]	Face feature value
faceImage	StImage	Cropped image of face
faceImageQuality	float	Quality of cropped image of face
beard	int	Beard (0: no beard, 1: has beard, -1: unknow)
associatedBodyTrackID	int	associated body id

4.2 Face Attribute

Field Name	Field Description	Attribute Tag	Attribute Description
expression	Facial expression	others	Others
happy	Happy
calm	Calm

xxxxxxxxxx

</table>

4.3 Face

Field Name	Field Type	Field Description
faceId	int	face id (unique ID of the face after deduplication)
gender	String	Gender (M: Male; F: Female)
age	int	Age
faceImage	StImage	Cropped image of face
watchTime	long	Watch time (ms)
othersEmotionTime	long	Other expression time duration besides happy and calm (ms)
happyEmotionTime	long	Happy expression time duration (ms)
calmEmotionTime	long	Calm expression time duration (ms)
glasses	int	Glasses (0: no glasses, 1: has glasses, 2: sunglasses, -1: unknow)
mask	int	Mask (0: no mask, 1: has mask; -1: unknow)
charmScore	int	Charm score
faceFeature	byte[]	Face feature value
faceImageQuality	float	Quality of cropped image of face
beard	int	Beard (0: no beard, 1: has beard, -1: unknow)

4.5 TrackingBody

Field Name	Field Type	Field Description
trackId	int	Body Track Id
bodyQuality	float	Body quality(0 - 1)
bodyRect	StRect	body frame
associatedFaceTrackID	int	Face ID for the body (-1: no face matched)
attribute	BodyAttribute	Body attribute
age	String	Age (Adult, Elderly, Child)
gender	String	Gender (Male, Female)
bodyImage	StImage	Cropped image of body

4.5 BodyAttribute

Field Name	Return Type	Method Description
upperClothing	String	Type of the upper clothing (ShortSleeve, LongSleeve)
lowerClothing	String	Type of the lower clothing(Trousers, Shorts, Skirt)
upperMode	String	Model of the upper clothing (Pure, Stripe, Design, Joint, Lattic)
lowerMode	String	Model of the lower clothing (Pure, Stripe, Design, Joint, Lattic)
upperType	String	Style of the upper clothing (LongCoat, Jacket, T-Shirt, SportsWear, DownCoat, Shirt, Dress, BusinessSuit)
upperColor	String	Color of the upper clothing (Black, White, Gray, Red, Yellow, Blue, Green, Purple)
lowerColor	String	Color of the lower clothing (Black, White, Gray, Red, Yellow, Blue, Green, Purple)
age	String	Age (Adult, Elderly, Child)
gender	String	Gender (Male, Female)

4.6 Code table

Code	Description
100	CODE_OK
101	reinit sdk
103	running error
104	params illegal
201	algorightm no detect face
202	algorightm sdk release
203	algorightm no feature
204	algorightm unknown img format
205	algorightm img decode failed
206	algorightm img size illegal
207	algorightm params illegal
208	algorightm pls_enable face recognition
210	algorightm st exception

5 Code Obfuscation

Obfuscation settings in a project: Add the following codes to proguard-rules.pro:

xxxxxxxxxx

######insight sdk########

-keep class com.sensetime.** {*;}

​

########retrofit#######

#retrofit2 Obfuscation

-keepattributes Signature, InnerClasses, EnclosingMethod

-keepattributes RuntimeVisibleAnnotations, RuntimeVisibleParameterAnnotations

-keepclassmembers,allowshrinking,allowobfuscation interface * {

@retrofit2.http.* <methods>;

}

-dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement

-if interface * { @retrofit2.http.* <methods>; }

-keep,allowobfuscation interface <1>

​

# OkHttp3

-dontwarn javax.annotation.**

-keepnames class okhttp3.internal.publicsuffix.PublicSuffixDatabase

-dontwarn org.codehaus.mojo.animal_sniffer.*

-dontwarn okhttp3.internal.platform.ConscryptPlatform

​

# Okio

-dontwarn org.codehaus.mojo.animal_sniffer.*

​

########Insight-Gson##########

-keepattributes Signature

-keepattributes *Annotation*

-keep class sun.misc.Unsafe { *; }

-keep class com.sensetime.insight.gson.** {*;}

5 FAQ

1. The currentTrackingFace interface fails to get face data. Why?

A: When failing to get the data, check the SDK for successful initialization, then check the module for connection to the device, and finally check the camera for correct direction.

1. Can the currentTrackingFace interface be called simultaneously in multiple threads? A: No.

2. Can the SDK be re-initialized after being released?

A: Yes, it can be re-initialized.

1. Sometimes, the face attributes collected by the currentTrackingFace interface are empty (age and gender not available). Why?

A：When extracting an attribute, the module will judge it based on the quality of the recognized face. Therefore, sometimes it may fail to collect certain attributes. In the case of a high quality score, however, it can collect such attributes.

1. What are the frequencies and situations for calling the currentTrackingFace interface?

A: You can call the currentTrackingFace interface in a situation for your application.

ex: 1) Call it every 30ms for child threads. The acquired data can be drawn on the screen.

2) Call it every second to get face data during the carousel.

3) Call it when an event occurs (for example, an event for purchase) to get the face data before the current screen...

xxxxxxxxxx

​