Skip to main content

​
Key Interfaces

​
1). Initialization

Initialize the SDK with your appId and appSecret, then obtain a Player instance: 
VirtualFactory.init("your appId", "your appSecret")
player = VirtualFactory.getPlayer(mContext, eglBaseContext)

​
2) Connect to the digital human

Connect using your platform conversationId: 
    player?.addCallback(callback)
    player?.connect("your conversation id")
The player exposes the following callbacks:

​
onShow — Digital human is ready to display

void onShow();

​
onError — Error occurred during connection

void onError(int msgType, int msgSubType, String msg);
Parameters:
Parameter NameTypeDescription
msgTypeintError type
msgSubTypeintSub-error type
msgStringException message
Possible msgType values:
ValueDescription
1000Authorization exception
1001Session creation exception
1002Resource retrieval exception
1010IM connection creation failed
1011Rendering service returned exception
1020RTC status exception
1030Rendering service actively closed
1040IM connection lost
1050RTC connection lost

​
onVideoTrack — RTC media channel created successfully

Bind your digital human view to the track in this callback 
void onVideoTrack(VideoTrack track);
Parameter NameTypeDescription
trackVideoTrackVideo media track

​
onAudioSamples — Local audio sampling data callback

Use this callback for audio visualization or processing. 
default void onAudioSamples(int audioFormat, int channelCount, int sampleRat, byte[] data){}

​
onTtsSpeakStart — TTS playback started

default void onTtsSpeakStart(){}

​
onTtsSpeakText — TTS text content

default void onTtsSpeakText(String text){}

​
onTtsSpeakStop — TTS playback completed

default void onTtsSpeakStop(){}

​
onSpeakStart — Speaking started

default void onSpeakStart(){}

​
onSpeakText — Spoken text content

default void onSpeakText(String text){}

​
onSpeakStop — Speaking completed

default void onSpeakStop(){}

​
onAsrResult — ASR recognition result

default void onAsrResult(String text, boolean sentenceEnd){}

​
3) Drive the digital human to speak with an audio URL

Use a WAV audio URL with 16 kHz sample rate, 16‑bit, mono channel to drive speech 
player?.speakWithWav(wavUrl, true)
Parameter NameTypeDescription
wavUrlStringURL of the WAV audio file
interruptbooleanWhether to interrupt the current speaking state

​
4) Drive the digital human to speak with text

Provide text input. The digital human will speak using the session’s configured voice. 
player?.speakWithTxt(text, true)
Parameter NameTypeDescription
textStringText for the digital human to speak
interruptbooleanWhether to interrupt the current speaking state

​
5) Q&A Interaction

Ask a question and receive a response from the digital human. 
player?.speakWithQuestion(text, true)
Parameter NameTypeDescription
textStringExpress your question to the digital human
interruptbooleanWhether to interrupt the current speaking state

​
6) Interrupt speaking

Stop the current speech immediately. 
player?.stopAudio()