fe9f3512ff
Signed-off-by: lvqiang214 <lvqiang1@huawei.com> |
||
---|---|---|
figures | ||
frameworks | ||
interfaces | ||
llt/hdt | ||
sa_profile | ||
services | ||
tests | ||
utils | ||
bundle.json | ||
LICENSE | ||
OAT.xml | ||
README_zh.md | ||
README.md |
Intelligent Voice Framework
Overview
Introduction
The intelligent voice framework consists of the intelligent voice service framework and intelligent voice driver. It implements voice enrollment and voice wakeup.
Figure 1 Architecture of the intelligent voice framework
The intelligent voice service framework provides the following features:
-
System event monitoring: monitoring system events such as unlocking upon power-on and screen-on/off
-
Concurrency policy: intelligent voice service concurrency management
-
Intelligent voice service: voice enrollment, voice wakeup, and more
-
Sound trigger: Digital Signal Processor (DSP) model loading, DSP algorithm enabling/disabling, and DSP event processing
The intelligent voice driver provides the following features:
-
Engine algorithm: intelligent voice algorithm engine and event reporting
-
Device driver: DSP model loading/unloading, DSP algorithm enabling/disabling, event reporting, and hardware-related channel configuration
Basic Concepts
- Voice enrollment: process of converting a wakeup word spoken by a user into an acoustic model and a voiceprint feature, which will be used for comparison during voice wakeup
- Voice wakeup: process of checking whether the current speaker is a registered user and if yes, waking up the system
- DSP chip: chip that implements digital signal processing
Directory Structure
The structure of the repository directory is as follows:
/foundation/ai/intelligent_voice_framework # Service code of the intelligent audio framework
├── frameworks # Framework code
│ ├── native # Internal API implementation
│ └── js # External API implementation
├── interfaces # API code
│ ├── inner_api # Internal APIs
│ └── kits # External APIs
├── sa_profile # Service configuration profile
├├── services # Service code
├── LICENSE # License file
├── tests # Developer test
└── utils # Public functions
Constraints
Currently, the intelligent voice framework supports the enrollment and wakeup of only one wakeup word.
Available APIs
APIs Used for Voice Enrollment
API | Description |
---|---|
createEnrollIntelligentVoiceEngine(descriptor: EnrollIntelligentVoiceEngineDescriptor): EnrollIntelligentVoiceEngine | Creates an enrollment engine. |
init(config: EnrollEngineConfig): EnrollIntelligentVoiceEngineCallbackInfo | Initializes this enrollment engine. |
start(isLast: boolean): EnrollIntelligentVoiceEngineCallbackInfo | Starts enrollment. |
stop(): void | Stops enrollment. |
commit(): EnrollIntelligentVoiceEngineCallbackInfo | Commits the enrollment data. |
setWakeupHapInfo(info: WakeupHapInfo): void | Sets the wakeup application information. |
setSensibility(sensibility: SensibilityType): void | Sets the sensitivity. |
release(): void | Releases this enrollment engine. |
APIs Used for Voice Wakeup
API | Description |
---|---|
createWakeupIntelligentVoiceEngine(descriptor: WakeupIntelligentVoiceEngineDescriptor): WakeupIntelligentVoiceEngine | Creates a wakeup engine. |
setWakeupHapInfo(info: WakeupHapInfo): void | Sets the wakeup application information. |
setSensibility(sensibility: SensibilityType): void | Sets the sensitivity. |
on(type: 'wakeupIntelligentVoiceEvent', callback: Callback): void | Subscribes to wakeup events. |
release(): void | Releases this wakeup engine. |
How to Develop
Voice Enrollment
The voice enrollment process is an interaction process initiated by a user through the enrollment page of an application. The main process is as follows:
- A user starts enrollment (creating and initializing the enrollment engine), and the enrollment page is displayed.
- The enrollment page asks the user to speak a wakeup word, and the user speaks the wakeup word (starting enrollment). The enrollment page asks the user to speak the wakeup word again several times.
- After the enrollment data is committed, the enrollment process is complete. The code snippet is as follows:
// Import the intelligentVoice module.
import intelligentVoice from '@ohos.ai.intelligentVoice';
// Obtain the intelligent audio management service.
var manager = intellVoice.getIntelligentVoiceManager();
if (manager == null) {
console.error("Get IntelligentVoiceManager failed.");
} else {
console.info("Get IntelligentVoiceManager success.");
return;
}
// Create an enrollment engine.
var engine = null;
let engineDescriptor = {
wakeupPhrase: '', // Set a wakeup word.
}
await intellVoice.createEnrollIntelligentVoiceEngine(engineDescriptor).then((data) => {
engine = data;
console.info('Create EnrollIntelligentVoice Engine finish');
}).catch((err) => {
console.error('Create EnrollIntelligentVoice Engine failed, err: ' + err.message);
});
if (engine == null) {
console.error('Create EnrollIntelligentVoice Engine failed');
return;
}
// Initialize the enrollment engine.
let config = {
language: "zh", // Chinese
area: "CN", // China
}
engine.init(config).then((data) => {
console.info('Init EnrollIntelligentVoice Engine finish');
}).catch((err) => {
console.info('Init EnrollIntelligentVoice Engine failed, err: '+ err.message);
});
// Start enrollment.
let isLast = true; // The value true means that this is the last time to start enrollment, and false means the opposite. The value true is used here.
engine.start(isLast).then((data) => {
console.info('Start enrollment finish');
}).catch((err) => {
console.info('Start enrollment failed, err: '+ err.message);
});
// Commit the enrollment data.
engine.commit().then((data) => {
console.info('Commit enroll result finish');
}).catch((err) => {
console.info('Commit enroll result failed, err: '+ err.message);
});
// Deliver the voice wakeup application information.
let info = {
bundleName: "demo", // Bundle name of the application. demo here is for reference only. Set this parameter based on your application.
abilityName: "demo", // Ability name of the application. demo here is for reference only. Set this parameter based on your application.
}
engine.setWakeupHapInfo(info).then((data) => {
console.info('Set wakeup hap info finish');
}).catch((err) => {
console.info('Set wakeup hap info failed, err: '+ err.message);
});
// Release the enrollment engine.
engine.release().then((data) => {
console.info('Release EnrollIntelligentVoice engine success.');
}).catch((err) => {
console.info('Release EnrollIntelligentVoice engine failed, err: '+ err.message);
});
Voice Wakeup
Voice wakeup is controlled by the intelligent voice framework. Upper-layer applications only need to create a wakeup engine by calling createWakeupIntelligentVoiceEngine and then subscribe to wakeup events.
// Obtain the wakeup engine.
var engine = null;
let engineDescriptor = {
needApAlgEngine: true, // Specify whether the framework needs to provide the AP algorithm engine.
wakeupPhrase: '', // Set a wakeup word.
}
await intellVoice.createWakeupIntelligentVoiceEngine(engineDescriptor).then((data) => {
engine = data;
console.info('Create WakeupIntelligentVoice Engine finish');
}).catch((err) => {
console.error('Create WakeupIntelligentVoice Engine failed, err: ' + err.message);
});
if (engine == null) {
console.error('Create WakeupIntelligentVoice Engine failed');
return;
}
// Subscribe to wakeup events.
engine.on('wakeupIntelligentVoiceEvent',(callback) => {
console.info('wakeupIntelligentVoiceEvent CallBackInfo:')
for (let prop in callback) {
console.info('wakeupIntelligentVoiceEvent prop: ' + prop);
console.info('wakeupIntelligentVoiceEvent value: ' + callback[prop]);
}
});
Repositories Involved
intelligent_voice_framework