Welcome to the Microsoft Speech SDK

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Welcome to the Microsoft Speech SDK, version 5.0

The Microsoft® Speech SDK 5.0 is the developer kit for the Microsoft® Windows environment. Tools, information, and sample engines and applications are provided to help you integrate and optimize your speech recognition and speech synthesis engines with the new Microsoft Speech API 5.0 (SAPI 5.0). The Speech SDK also includes updated releases of the Microsoft advanced speech recognition engine and Microsoft concatenative speech synthesis engine previously code-named "Whistler" and "Whisper," respectively.

End-User License Agreement
Please read and understand the End-User License Agreement before using the Microsoft Speech SDK.
Redistribution Code Rights
Please read and understand the Redistributable Code Rights before building applications or engines. This document outlines which files may be redistributed with our own products. It also describes limitations for modifing the Speech SDK 5.0 files and samples.
System Requirements
This section lists the software required including supported operating systems and the compiler environment. Hardware requirements are also listed and include recommended computer speeds, available RAM and audio equipment.
Developer Support
This section provides information on the developer support choices available to you. For additional support options, see support phone numbers and options.
Getting Started for First-Time Users
Getting Started introduces the Microsoft® Speech SDK to first-time users and explains its contents, system requirements, and features.
Release Notes
Release Notes describes new information at the time of release, documents known issues, and identifies any new redistribution file updates.
Microsoft Speech Technologies Web Site
This site provides product news updates, technical articles, and links to useful resources.
Is the documentation helpful?
We'd like to know. Please send any comments and suggestions you have to: spchsdk@microsoft.com.

© 1995-2000 Microsoft Corporation. All rights reserved.


Welcome to the Microsoft Speech SDK

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Getting Started for First-Time Users

The following topics introduce the Microsoft Speech SDK (SDK) to first-time users and explain its contents and features:
  • What is the SDK?
  • What can I do with the SDK?
  • What is covered in the SDK documentation?
  • What is not covered by the SDK?
  • How can I start using the SDK?
  • What special things do I need to know?

What is the SDK?

Microsoft Speech SDK is a software development kit for building speech engines and applications for Microsoft Windows. Designed primarily for the desktop speech developer, the SDK contains the Microsoft® Win32®-compatible speech application programming interface (SAPI), the Microsoft continuous speech recognition (MCSR) engine and Microsoft concatenative speech synthesis (or text-to-speech (TTS)) engine, a collection of speech-oriented development tools for compiling source code and executing commands, sample application and tutorials that demonstrate the use of Speech with other engine technologies, sample speech recognition and speech synthesis engines for testing with speech-enabled applications, and documentation on the most important SDK features.

What can I do with the SDK?

You can use the SDK components and redistributable SAPI/engine run-time to build applications that incorporate speech recognition and speech synthesis.

Speech Components and Services
Included in the Speech API architecture is a collection of speech components for directly managing the audio, training wizard, events, grammar compiler, resources, speech recognition manager, and TTS manager for low-level control and greater flexibility. The Speech API also enables support and manages shared recognition events for running multiple speech-enabled applications.

SDK Tools
The tools in the Tools directory assist with the verification and testing of SAPI development. This directory contains source code and project for compliance testing and may be modified to fit your needs.

SDK Samples
The Microsoft Speech SDK includes samples that can be used as a reference for creating speech-enabled applications. The compiled samples and demonstration applications are available on the Start->Programs->Microsoft Speech SDK 5.0 menu. The binary and source files, projects, are available in the Samples folder of the Microsoft Speech SDK 5.0 folder. A description of each sample, installation, and set of usage instructions is provided.

Coexistence and Third Party Support
Microsoft Speech API 5.0 has been designed to coexist on the same device with prior versions of the Microsoft Speech API (ver. 3.0, 4.0, and v. 4.0a). Microsoft is also working with many of the top speech recognition engine vendors on providing SAPI 5.0 support. Visit http://www.microsoft.com/speech/thirdparty for more the latest list on SAPI 5.0-compatible engines.

File Name | Approximate File Size | Setup Merge Names
Sapi.dll & Sapisvr.exe | .5Mb | Sp5.msm
Sapi.cpl | 36k | Sp5Intl.msm
SR Engine | 1.7Mb | Sp5Sr.msm
C&C Datafiles | 13.4Mb | Sp5CCInt.msm
Dictation Datafiles | 33Mb | Sp5DCInt.msm
TTS Engine & voices | 7.8Mb | Sp5TTInt.msm

For more information on setup, see the Setup White Paper.

What is covered in the SDK Documentation?

The Microsoft Speech SDK documentation provides information for both the experienced speech developer and the beginner. It is located in the Start->Programs->Microsoft Speech SDK 5.0 menu.

Programmer's Guide

The Programmer's Guide provides information on the following Microsoft Speech API topics:

  • Application level interfaces
  • Engine level interfaces
  • Structures
  • Enumerations
  • Helper functions

Tools Reference
The Tools Reference describes all the SDK tools.

Samples Reference
The Samples Reference describes all the SDK samples.

SDK Tutorial Reference
The SDK tutorial includes a step-by-step tutorial using the most commonly used functions and methods.

Engine Compliance Testing Reference
The Testing Reference describes the compliance testing requirements for engine vendors porting their speech engine to SAPI 5.0.

White Papers
The White Papers include technical background articles on the technology. They also include sample code that addresses more specific programming solutions.

What is not covered by the Speech SDK?

The Microsoft Speech SDK is not a GUI or voice-user interface (VUI) development environment with menus, buttons, toolbars, and so on. Most of the tools in the SDK are run from the MS-DOS® command line (e.g., gc.exe) or with executable applications. The Microsoft Speech SDK assumes knowledge of programming for C, C++ or COM to directly instantiate the Speech APIs.

There is no Automation interface support in this release of the Speech API. Therefore, support is limited for VB or Web script development without writing an Automation wrapper or VXML interpreter. For more information on third parties who provide additional SAPI 5.0-compatible Web controls, please visit http://www.microsoft.com/speech/thirdparty.

There is also no Microsoft Visual Basic, Sun Java, script, or telephony-specific controls included in this release of the Speech SDK. For more information on third parties who provide additional SAPI 5.0-compatible Web or telephony controls, please visit http://www.microsoft.com/speech/thirdparty.

How can I start using the SDK?

The organization of the Speech SDK documentation is similar to other traditional Microsoft SDKs. The Finding Information section of the Microsoft Speech SDK documentation contains important information on how to use the documentation's Help Viewer, including use of the toolbar buttons and full text search, and finding a Help topic, and much more.

Visit the Microsoft® Speech.NET Technologies home page frequently. Here you can find the latest news and updates to the SDK and the Microsoft speech engines.

If for some reason you cannot locate a particular type of documentation in the help system, please e-mail spchsdk@microsoft.com to fill a request.

What special things do I need to know?

If you are developing an application that intends to use the Microsoft speech setup files, your Setup.exe needs to install the Microsoft Windows Installer if it is not already present. Please go to http://msdn.microsoft.com/developer/sdk/wininst.asp to download the Windows Installer SDK.


Welcome to the Microsoft Speech SDK

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

System Requirements

Operating Systems

Supported operating systems are:

  • Microsoft Windows 2000 Professional Workstation, English edition or English edition with Japanese or Simplified Chinese Language support.
  • Microsoft Windows Millennium edition.
  • Microsoft Windows ® NT Workstation 4.0, service pack 6a, English, Japanese, or Simplified Chinese edition.
  • Microsoft Windows 98 (Windows 95 is not supported).
Software Requirements
  • Microsoft Visual C++ 6.0, service pack 3 or later version.
  • Platform SDK (PSDK) April 2000 or later edition. Compiling SDK projects requires components of the PSDK. Within Microsoft Visual C++ 6.0, the PSDK include directories must be listed before the Visual C++. You can change the order in Tools->Options menu under the Directories tab. Move PSDK directories above all Visual C++ directories, if needed.

    To save disk space, you can load a minimal configuration. This includes enabling only the following two options:

  1. Configuration Options
  2. Build Environment
    • These options require 13 MB on the system drive and another 80 MB on any other drive. No other options are needed. You can download the PSDK from http://msdn.microsoft.com/downloads/sdks/platform/platform.asp.

    • Microsoft Internet Explorer 5.0 or later version. Users of NT4 with any version of the service packs require Microsoft Internet Explorer 5.5 or later. You need this to read the online documentation and for executing Microsoft XML. You can download the latest version of Microsoft Internet Explorer from the web at http://www.microsoft.com/ie.
    Hardware Requirements
    • A PentiumII\PentiumII-equivalent or later processor at 233 MHz with 128 MB is recommended.
    • SAPI 5.0 can now take advantage of a computer and operating system that supports multiple processors, including all those mentioned above. Additionally, you can use SAPI 5.0 in a distributed application environment.
    • Not all sound cards or sound devices are supported by SAPI 5.0, even if the operating system supports them otherwise.
    • The follow table outlines the RAM usage:
    Component | Minimum RAM | Recommended RAM
    TTS Engine | 14.5 Mb | 32.0 Mb
    SR C&C | 16 Mb | 32 Mb
    SR Dict | 25.5 Mb | 128 Mb
    SR Both | 26.5 Mb | 128 Mb
    • The follow table outlines the disk usage:
    File Name | Approximate File Size | Setup Merge Names
    Sapi.dll & Sapisvr.exe | .5Mb | Sp5.msm
    Sapi.cpl | 36k | Sp5Intl.msm
    SR Engine | 1.7Mb | Sp5Sr.msm
    C&C Datafiles | 13.4Mb | Sp5CCInt.msm
    Dictation Datafiles | 33Mb | Sp5DCInt.msm
    TTS Engine & voices | 7.8Mb | Sp5TTInt.msm

    For more information on setup, see the Setup White Paper.


About This SDK

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

About the SDK

The Microsoft Speech SDK is designed to work with the industry-leading Speech Application Programming Interface (SAPI) and Microsoft continuous speech recognition engine (MCSR) and Microsoft concatenative speech synthesis engine (or text-to-speech (TTS)). Microsoft SDK includes tools, samples, and documentation for building speech applications.

Microsoft Visual Basic Support

A set of COM-supported speech Automation interfaces are not included in this version of the Microsoft Speech SDK. Therefore, you must develop or obtain a COM-supported Automation wrapper to program using Microsoft Visual Basic® or Web scripting language.

This section includes the following topics:

  • Who Should Use This SDK
  • End User License Agreement
  • Redistributable Code Rights
  • Documentation Conventions
  • Legal Information

      • © 1995-2000 Microsoft Corporation. All rights reserved.


Who Should Use This SDK

To get the most out of this SDK, you should be familiar with the following:

  • C/C++ programming concepts.
  • Component Object Model (COM). Developers should understand COM programming concepts, obtaining pointers to interfaces, and calling methods.
  • All developers should understand the Win32 application programming interface (API).

For more information about Windows programming, see the documentation included in the Microsoft Win32 Software Development Kit.

For more information about C/C++ programming, see the documentation for the Microsoft® Visual C++™ development system for Windows.


END-USER LICENSE AGREEMENT FOR MICROSOFT SOFTWARE

IMPORTANT-READ CAREFULLY: This Microsoft End-User License Agreement ("EULA") is a legal agreement between you (either an individual or a single entity) and Microsoft Corporation for the Microsoft software product identified above, which includes DEVICE software (including SAPI 5.0, Microsoft continuous speech recognition engine and Microsoft concatenative speech synthesis engine), and may include associated media, printed materials, and "online" or electronic documentation ("SOFTWARE PRODUCT"). The SOFTWARE PRODUCT also includes any updates and supplements to the original SOFTWARE PRODUCT provided to you by Microsoft. Any software that may be provided along with the SOFTWARE PRODUCT that is associated with a separate end-user license agreement is licensed to you under the terms of that license agreement. By installing, copying, downloading, accessing or otherwise using the SOFTWARE PRODUCT, you agree to be bound by the terms of this EULA. If you do not agree to the terms of this EULA, do not install or use the SOFTWARE PRODUCT.

SOFTWARE PRODUCT LICENSE

The SOFTWARE PRODUCT is protected by copyright laws and international copyright treaties, as well as other intellectual property laws and treaties. The SOFTWARE PRODUCT is licensed, not sold.

1. GRANT OF LICENSE. This EULA grants you the following rights:

- SOFTWARE PRODUCT. You may install copies of the SOFTWARE PRODUCT on up to ten (10) digital electronic devices, including computers, workstations, terminals, handheld PCs, pagers, "smart phones," or other digital electronic devices (each a "DEVICE") to design, develop, and test software programs that use the Microsoft Speech Application Programming Interface ("SAPI 5.0") and run on one or more Microsoft Windows operating system products that support SAPI 5.0 ("Windows Platforms"), provided that you are the only individual using the SOFTWARE PRODUCT on each such DEVICE . If you are a single entity, you may designate one individual within your organization to have the right to use the SOFTWARE PRODUCT in the manner described herein.

- Sample Code. Solely with respect to those portions of the SOFTWARE PRODUCT identified as sample code ("Sample Code"), Microsoft also grants you the right to modify the source code version of the Sample Code for the sole purposes of designing, developing, and testing software programs that use SAPI 5.0 (each, "a SAPI Application") and to reproduce and distribute the Sample Code along with any modifications thereof, in object code form only, provided that you comply with the Distribution Requirements described below. For purposes of this section, "modifications" shall mean enhancements to the functionality of the Sample Code.

- Redistributable Code. Portions of the SOFTWARE PRODUCT are designated as "Redistributable Code” file located in REDISTRIB.CHM Your distribution rights associated with each file of the Redistributable Code are subject to the distribution requirements described below.

- Distribution Requirements. You may copy and redistribute the Sample Code and/or Redistributable Code (collectively "REDISTRIBUTABLE COMPONENTS") as described above, provided that (a) you distribute the REDISTRIBUTABLE COMPONENTS only in conjunction with, and as a part of, your SAPI Application; (b) your SAPI Application adds significant and primary functionality to the REDISTRIBUTABLE COMPONENTS; (c) the REDISTRIBUTABLE COMPONENTS only operate in conjunction with the Windows Platforms; (d) you do not permit further redistribution of the REDISTRIBUTABLE COMPONENTS by your end-user customers; (e) you do not use Microsoft's name, logo, or trademarks to market your SAPI Application; (f) you include a valid copyright notice on your SAPI Application; (g) you include the entire text located in REDISTRIB.CHM in your SAPI Application End User License Agreement; and (h) you agree to indemnify, hold harmless, and defend Microsoft from and against any claims or lawsuits, including attorneys' fees, that arise or result from the use or distribution of your SAPI Application. Contact Microsoft for the applicable royalties due and other licensing terms for all other uses and/or distribution of the REDISTRIBUTABLE COMPONENTS.

- Reservation of Rights. All rights not expressly granted are reserved by Microsoft.

2. DESCRIPTION OF OTHER RIGHTS AND LIMITATIONS.

- Limitations on Reverse Engineering, Decompilation, and Disassembly. You may not reverse engineer, decompile, or disassemble the SOFTWARE PRODUCT, except and only to the extent that such activity is expressly permitted by applicable law notwithstanding this limitation.

- Separation of Components. The SOFTWARE PRODUCT is licensed as a single product. Its component parts may not be separated for use on more than one DEVICE.

- Trademarks. This EULA does not grant you any rights in connection with any trademarks or service marks of Microsoft.

- Rental. You may not rent, lease, or lend the SOFTWARE PRODUCT.

- Support Services. No technical support will be provided for the SOFTWARE PRODUCT.

- Termination. Without prejudice to any other rights, Microsoft may terminate this EULA if you fail to comply with the terms and conditions of this EULA. In such event, you must destroy all copies of the SOFTWARE PRODUCT and all of its component parts.

3. COPYRIGHT.

All title and copyrights in and to the SOFTWARE PRODUCT (including but not limited to any images, photographs, animations, video, audio, music, text, and "applets" incorporated into the SOFTWARE PRODUCT), the accompanying printed materials, and any copies of the SOFTWARE PRODUCT are owned by Microsoft or its suppliers. All title and intellectual property rights in and to the content which may be accessed through use of the SOFTWARE PRODUCT is the property of the respective content owner and may be protected by applicable copyright or other intellectual property laws and treaties. This EULA grants you no rights to use such content. If this SOFTWARE PRODUCT contains documentation which is provided only in electronic form, you may print one copy of such electronic documentation. You may not copy the printed materials accompanying the SOFTWARE PRODUCT.

4. DUAL-MEDIA SOFTWARE.

You may receive the SOFTWARE PRODUCT in more than one medium. Regardless of the type or size of medium you receive, you may use only one medium that is appropriate for your single DEVICE. You may not install, copy or use the other medium on another DEVICE. You may not loan, rent, lease, or otherwise transfer the other medium to another user, except as part of the permanent transfer (as provided above) of the SOFTWARE PRODUCT.

5. U.S. GOVERNMENT RESTRICTED RIGHTS.

All SOFTWARE PRODUCT provided to the U.S. Government pursuant to solicitations issued on or after December 1, 1995 is provided with the commercial license rights and restrictions described elsewhere herein. All SOFTWARE PRODUCT provided to the U.S. Government pursuant to solicitations issued prior to December 1, 1995 is provided with "Restricted Rights" as provided for in FAR, 48 CFR 52.227-14 (JUNE 1987) or DFAR, 48 CFR 252.227-7013 (OCT 1988), as applicable. The reseller is responsible for ensuring SOFTWARE PRODUCT is marked with the "Restricted Rights Notice" or "Restricted Rights Legend," as required. All rights not expressly granted are reserved.

6. EXPORT RESTRICTIONS.

You acknowledge that the SOFTWARE PRODUCT is of U.S. origin. You agree to comply with all applicable international and national laws that apply to the SOFTWARE PRODUCT, including the U.S. Export Administration Regulations, as well as end-user, end-use and country destination restrictions issued by U.S. and other governments. For additional information on exporting Microsoft products, see http://www.microsoft.com/exporting/.

MISCELLANEOUS.

If you acquired this SOFTWARE PRODUCT in the United States, this EULA is governed by the laws of the State of Washington. If you acquired this SOFTWARE PRODUCT in Canada, unless expressly prohibited by local law, this EULA is governed by the laws in force in the Province of Ontario, Canada; and, in respect of any dispute which may arise hereunder, you consent to the jurisdiction of the federal and provincial courts sitting in Toronto, Ontario. If this SOFTWARE PRODUCT was acquired outside the United States, then local law may apply.

Should you have any questions concerning this EULA, or if you desire to contact Microsoft for any reason, please contact the Microsoft subsidiary serving your country, or write:

Microsoft

Speech Dot Net

One Microsoft Way

Redmond, WA 98052-6399.

NO WARRANTIES. MICROSOFT EXPRESSLY DISCLAIMS ANY WARRANTY FOR THE SOFTWARE PRODUCT. THE SOFTWARE PRODUCT AND ANY RELATED DOCUMENTATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OR MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. THE ENTIRE RISK ARISING OUT OF USE OR PERFORMANCE OF THE SOFTWARE PRODUCT REMAINS WITH YOU.

LIMITATION OF LIABILITY. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL MICROSOFT OR ITS SUPPLIERS BE LIABLE FOR ANY SPECIAL, INCIDENTAL, INDIRECT, OR CONSEQUENTIAL DAMAGES WHATSOEVER (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION, LOSS OF BUSINESS INFORMATION, OR ANY OTHER PECUNIARY LOSS) ARISING OUT OF THE USE OF OR INABILITY TO USE THE SOFTWARE PRODUCT OR THE PROVISION OF OR FAILURE TO PROVIDE SUPPORT SERVICES, EVEN IF MICROSOFT HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. BECAUSE SOME STATES AND JURISDICTIONS DO NOT ALLOW THE EXCLUSION OR LIMITATION OF LIABILITY, THE ABOVE LIMITATION MAY NOT APPLY TO YOU.

EXCLUSION DE GARANTIE. MICROSOFT EXCLUT EXPRESSÉMENT TOUTE GARANTIE RELATIVE AU PRODUIT LOGICIEL. LE PRODUIT LOGICIEL ET LA DOCUMENTATION Y AFFÉRENTE SONT FOURNIS "EN L'ÉTAT", SANS GARANTIE D'AUCUNE SORTE, EXPRESSE OU IMPLICITE, NOTAMMENT SANS AUCUNE GARANTIE IMPLICITE DE QUALITÉ, D'ADÉQUATION À UN USAGE PARTICULIER OU D’ABSENCE DE CONTREFAÇON. VOUS ASSUMEZ L'ENSEMBLE DES RISQUES DÉCOULANT DE L'UTILISATION OU DES PERFORMANCES DU PRODUIT LOGICIEL. LE PRÉSENT ARTICLE EST SANS PRÉJUDICE DE LA GARANTIE LÉGALE CONTRE LES VICES CACHÉS DONT VOUS POURRIEZ BÉNÉFICIER, LE CAS ÉCHÉANT.

LIMITATION DE RESPONSABILITÉ. DANS TOUTE LA MESURE PERMISE PAR LA RÉGLEMENTATION EN VIGUEUR, MICROSOFT OU SES FOURNISSEURS NE POURRONT EN AUCUN CAS ÊTRE TENUS POUR RESPONSABLES DE TOUT DOMMAGE, DE QUELQUE NATURE QUE CE SOIT, (NOTAMMENT ET DE MANIÈRE NON LIMITATIVE, TOUTE PERTE DE BÉNÉFICES, INTERRUPTION D’ACTIVITÉ, PERTE D’INFORMATIONS COMMERCIALES OU TOUTE AUTRE PERTE PÉCUNIAIRE) RÉSULTANT DE L'UTILISATION OU DE L'IMPOSSIBILITÉ D'UTILISER LE PRODUIT LOGICIEL OU DE LA FOURNITURE OU DU DÉFAUT DE FOURNITURE DES SERVICES D'ASSISTANCE, MÊME SI MICROSOFT A ÉTÉ PRÉVENU DE L’ÉVENTUALITÉ DE TELS DOMMAGES. CERTAINS PAYS ET CERTAINES JURIDICTIONS N'AUTORISENT PAS LES EXCLUSIONS OU LIMITATIONS DE RESPONSABILITÉ, DE SORTE QUE LA LIMITATION CI-DESSUS PEUT NE PAS VOUS ÊTRE APPLICABLE.


Redistributable Code

The Redistributable Code is the property of Microsoft Corporation and its suppliers and is protected by copyright law and international treaty provisions. You are authorized to make and use copies of the Redistributable Code either as part of the application in which you received the Redistributable Code, or in conjunction with the application for which its use is intended.Except as expressly provided in the foregoing sentence, you are not authorized to reproduce and distribute the Redistributable Code. Microsoft reserves all rights not expressly granted.You may not reverse engineer, decompile, or disassemble the Redistributable Code, except and only to the extent that such activity is expressly permitted by applicable law notwithstanding this limitation.

THE REDISTRIBUTABLE CODE IS PROVIDED TO YOU "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUTNOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE.YOU ASSUME THE ENTIRE RISK AS TO THE ACCURACY AND THE USE OF THE REDISTRIBUTABLE CODE.MICROSOFT SHALL NOT BE LIABLE FOR ANY DAMAGES WHATSOEVER ARISING OUT OF THE USE OF OR INABILITY TO USE THE REDISTRIBUTABLE CODE, EVEN IF MICROSOFT HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Redistributable Code

Redistributable Code is identified as the following files and all of the files can be found at the following location:

  • http://www.microsoft.com/speech/legal


Document Conventions

 The Microsoft.com Speech website

Documentation Conventions

This SDK uses the following typographic conventions:

Convention | Meaning
Bold text | Indicates a word that is a function name, data type, or other fixed element of the Windows, COM, or Speech API. For example, Pause().
Italic text | Indicates a word that is a placeholder or variable. For example, pdwPitch. Function parameters in API reference material are italicized to indicate that any variable name can be used.
UPPERCASE TEXT | Indicates a constant or structure. For example, the structure SPAUDIOSTATUS or the constant SPAS_CLOSED. Uppercase also indicates Microsoft® MS-DOS® filenames and paths.
Monospace text | Indicates source code examples.

Also, the interface syntax in this SDK follows the variable-naming convention known as Hungarian notation, in which variables are prefixed with lower-case letters to indicate their data type. For example, pszString is a pointer to a zero-terminated string.

For more information about Hungarian notation, see Programming Windows by Charles Petzold.


Welcome to the Microsoft Speech SDK

 Intelligent Interface Technologies Home Page

How to Read Newsgroups

You can use any newsreader software to access the Microsoft newsgroups. Microsoft Outlook Express is installed as part of Microsoft Internet Explorer for your convenience. Visit the Microsoft Internet Explorer Home page for information on how to download and install this product.

With Outlook Express installed, after you click on a newsgroup link, you will be prompted for configuration information. When prompted for News Server, specify msnews.microsoft.com. You do not need to enter an account name or password.

If you are using an NNTP newsreader (sometimes called a news client) other than Outlook Express, be sure to configure it to read the Microsoft newsgroups. You can access the Microsoft news server at msnews.microsoft.com.

Before posting to the newsgroups, please review the Microsoft Newsgroup Rules of Conduct.

Microsoft Newsgroups

This lists available newsgroups from Microsoft.


Welcome to the Microsoft Speech SDK

 Intelligent Interface Technologies Home Page

Developer Support

Microsoft Support Center

The Microsoft Support Center site maintains a wealth of resources to help you get the most from your product. Here you can search the entire Knowledge Base, view all the Troubleshooting Wizards, and access all the downloadable files. You can also view the extensive online glossary of computer terms, and find phone numbers and support options for all Microsoft products.

Speech newsgroups

For assistance with specific problems, there are a variety of newsgroups available. These newsgroups are actively monitored and address issues raised in the groups. For advice on configuring a newsreader, please see How to Read Newsgroups.

  • microsoft.speech.beta newsgroup
  • microsoft.speech.technical
  • microsoft.speech newsgroup
Microsoft Speech.NET Technologies Feedback
If you have feedback on the Microsoft Speech API (SAPI) or other Microsoft Speech-related questions, please use the appropriate email aliases:
spchsdk@microsoft.com
Speech SDK tools, samples, documentation questions.
sapidrg@microsoft.com
Developer relations, business development, and developer marketing questions.
sapi5@microsoft.com
SAPI 5.0 support issues.
voiceinfo@microsoft.com
SAPI 3.0, 4.0, 4.0a support issues only.
sapibeta@microsoft.com
Beta issues, when beta is in progress.
These email aliases are monitored regularly, but will receive limited response. Please use the newsgroup forum above for most questions and . inquiries
Known Issues
Information on known issues in SAPI 5.0 can be found in the knowledge base articles in the Microsoft Speech.NET Technologies support home page.


Legal Information

 Intelligent Interface Technologies Home Page

Legal Information for Microsoft Speech SDK

Information in this document is subject to change without notice and does not represent a commitment on the part of Microsoft Corporation. Companies, names, and data used in examples herein are fictitious unless otherwise noted. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, unless expressly permitted by Microsoft Corporation.

Microsoft may have patents or pending patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. The furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property rights.


SAPI 5.0 Introduction

This section provides a SAPI 5.0 introduction. The following topics are available:

  • SAPI5 Overview
  • Engine Characteristics
  • American English Phoneme Representation
  • International Phoneme Representation


SAPI 5.0 Overview

SAPI 5.0 consists of two interfaces: the application programming interface (API) and the device driver interface (DDI). The SAPI 5.0 API dramatically reduces the code overhead required for an application to use speech recognition and synthesis, making speech more usable for more applications by more developers.

This section covers the following topics:

  • API and DDI Overview
  • API Events
  • API for Speech Synthesis
  • DDI for Speech Synthesis
  • API for Speech Recognition
  • DDI for Speech Recognition

API and DDI Overview

The SAPI 5.0 DDI and API remove the implementation details such as multi-threading and audio device management making speech synthesis and recognition engines and applications convenient. The SAPI 5.0 interfaces are new and do not use the legacy of SAPI 4.0 interfaces.

SAPI layer abstraction graphic

return to the top of this page Back to top

API Events

In both speech synthesis and speech recognition, applications receive events. For example, when words of text have been spoken or when phrases of speech have been recognized. SAPI components that generate events implement an ISpNotifySource interface. Specifically, SAPI uses SetNotifySink. An application passes to ISpNotifySource::SetNotifySink an ISpNotifySink interface pointer which receives notification calls to ISpNotifySink::Notify when there are one or more events about which the application can retrieve data. Usually, applications will not implement ISpNotifySink because it must be free-threaded. Instead, applications can conveniently use CoCreateInstance (with COM) to make a proxy ISpNotifySink object implemented by the component CLSID_SpNotify. This proxy object provides an ISpNotifyControl interface. It can set up one of three notification mechanisms.

Method | Function
ISpNotifyTranslator ::InitWindowMessage | Enables a window callback function to receive notifications as window messages.
ISpNotifyTranslator ::InitCallback | Enables a callback function to receive notifications.
ISpNotifyTranslator ::InitWin32Event | Enables notifications through Win32 events.

The ISpNotifySource and ISpNotifySink interfaces only provide a mechanism for a notification, but no information on the events that caused the notification. Using an ISpEventSource object, an application can retrieve information about the event that caused the notification. An ISpEventSource also provides the mechanism to filter and queue events. By default, an application (really an ISpNotifySink) does not receive any notification from ISpEventSource until ISpEventSource::SetInterest has been called to specify about which events to provide notification and for which events repetitions should be queued. Events are identified by the flags SPEVENTENUM.

When an application receives a notification, there may be information on multiple events. By calling ISpEventSource::GetInfo, the ulCount member in the returned SPEVENTSOURCEINFO structure indicates the number of events for which information is available. Using ISpEventSource::GetEvents, an application extracts a specified number of SPEVENT structures, each containing information about a specific event.

return to the top of this page Back to top

API for Speech Synthesis

For basic speech synthesis, an application needs to use the interface ISpVoice. An application calls COM's CoCreateInstance for the component CLSID_SpVoice to get a pointer to the ISpVoice interface of a voice object. An application can start calling the methods of ISpVoice such as ISpVoice::Speak, which will synthesize speech for the text passed in as a parameter. By default, a voice object is set to the default voice unless the voice has been changed by a call to ISpVoice::SetVoice. Similarly, a voice object will also be set to a default audio data destination unless ISpVoice::SetOutput is called with an IUnknown and an appropriate SPSTREAMFORMAT format.

Resources

General management functions are provided through an ISpResourceManager object that is created by calling CoCreateInstance with CLSID_SpResourceManager.

Beyond Basics

In addition to generating speech from plain text, an application can place synthesis markup language within the text passed into ISpVoice::Speak. This markup language is based on the XML format. An application can also play raw audio data using the ISpVoice::SpeakStream. Text files can be spoken using the ISpVoice::SpeakStream as well. The format of the text (either Unicode or ANSI) will automatically be detected. Both functions have flags to control whether to purge any sounds not already played and whether calls are asynchronous.

Because an ISpVoice object is also an ISpEventSource object, an application can start receiving notifications of events by calling ISpNotifySource::SetNotifySink, by passing in an ISpNotifySink object. In association with an ISpVoice::Speak or ISpVoice::SpeakStream call, ISpVoice provides an SPEI_START_INPUT_STREAM event before the engine begins synthesizing and an SPEI_END_INPUT_STREAM event when the engine has finished synthesizing. For each call to ISpVoice::Speak or ISpVoice::SpeakStream, a stream number is assigned (pulStreamNum). For each SPEVENT structure describing an event, the ulStreamNum member identifies the stream number associated with an ISpVoice::Speak or ISpVoice::SpeakStream call. ISpVoice generates the following events of type SPEVENTENUM.

Event | Function
SPEI_VOICE_CHANGE | Voice ID or voice type has changed during stream synthesis.
SPEI_TTS_BOOKMARK | A bookmark was encountered during synthesis.
SPEI_WORD_BOUNDARY | Start of word being spoken.
SPEI_SENTENCE_BOUNDARY | Start of sentence being spoken.
SPEI_PHONEME | Phoneme being spoken.
SPEI_VISEME | Viseme being spoken.

Applications can call ISpVoice::GetStatus to receive the current status of speech synthesis.

return to the top of this page Back to top

DDI for Speech Synthesis

A SAPI speech synthesis engine implements the ISpTTSEngine interface. The primary method called by SAPI to perform speech rendering is ISpTTSEngine::Speak. SAPI, rather than the engine, performs XML parsing of the input text stream. The Speak method is handed a linked list of text fragments with their associated XML attribute state. The Speak method also receives a pointer to the SpVoice object's ISpTTSEngineSite interface. The TTS engine uses this interface to queue events and write output data.

Even though SAPI is a free-threaded architecture, the TTS engine instances will always be called by SAPI on a single thread. TTS engines are never directly accessed by applications. SAPI ensures that all parameter validation and thread synchronization has been properly performed before the TTS engine is called. For more Speech Synthesis DDI information, please see Text-to-Speech Engine Manager.

return to the top of this page Back to top

API for Speech Recognition

Just as ISpVoice is the main interface for speech synthesis, ISpRecoContext is the main interface for speech recognition. Like the ISpVoice, it is an ISpEventSource, which means that it is the speech application's vehicle for receiving notifications for the requested speech recognition events.

An application has the choice of two different types of speech recognition engines (ISpRecognizer). A shared recognizer that could possibly be shared with other speech recognition applications is recommended for most speech applications. To create an ISpRecoContext for a shared ISpRecognizer, an application need only call COM's CoCreateInstance on the component CLSID_SpSharedRecoContext. In this case, SAPI will set up the audio input stream, setting it to SAPI's default audio input stream. For large server applications that would run alone on a system, and for which performance is key, an InProc speech recognition engine is more appropriate. In order to create an ISpRecoContext for an InProc ISpRecognizer, the application must first call CoCreateInstance on the component CLSID_SpInprocRecoInstance to create its own InProc ISpRecognizer. Then the application must make a call to ISpRecognizer::SetInput (see also ISpObjectToken) in order to set up the audio input. Finally, the application can call ISpRecognizer::CreateRecoContext to obtain an ISpRecoContext.

The next step is to set up notifications for events the application is interested in. As the ISpRecognizer is also an ISpEventSource, which in turn is an ISpNotifySource, the application can call one of the ISpNotifySource methods from its ISpRecoContext to indicate where the events for that ISpRecoContext should be reported. Then it should call ISpEventSource::SetInterest to indicate which events it needs to be notified of. The most important event is the SPEI_RECOGNITION, which indicates that the ISpRecognizer has recognized some speech for this ISpRecoContext. See SPEVENTENUM for details on the other available speech recognition events.

Finally, a speech application must create, load, and activate an ISpRecoGrammar, which essentially indicates what type of utterances to recognize, i.e., dictation or a command and control grammar. First, the application creates an ISpRecoGrammar using ISpRecoContext::CreateGrammar. Then, the application loads the appropriate grammar, either by calling ISpRecoGrammar::LoadDictation for dictation or one of the ISpRecoGrammar::LoadCmdxxx methods for command and control. Finally, in order to activate these grammars so that recognition can start, the application calls ISpRecoGrammar::SetDictationState for dictation or ISpRecoGrammar::SetRuleState or ISpRecoGrammar::SetRuleIdState for command and control.

When recognitions come back to the application by means of the requested notification mechanism, the lParam member of the SPEVENT structure will be an ISpRecoResult by which the application can determine what was recognized and for which ISpRecoGrammar of the ISpRecoContext.

An ISpRecognizer, whether shared or InProc, can have multiple ISpRecoContexts associated with it, and each one can be notified in its own way of events pertaining to it. An ISpRecoContext can have multiple ISpRecoGrammars created from it, each one for recognizing different types of utterances.

return to the top of this page Back to top

DDI for Speech Recognition

At the most basic level, the SAPI 5.0 Recognition DDI provides the functionality for an engine to receive audio data from SAPI and return phrase recognitions. The two interfaces used for this are ISpSREngine, which is implemented by the engine, and ISpSREngineSite, which is implemented by SAPI. The SAPI interface ISpSREngineSite also provides speech engine methods for communicating more detailed information on what the engine recognizes. Grammars and speakers provide information to engines that better help them recognize speech and are a critical part of the communication between SAPI and speech engines. There are two final aspects of the communication between engine and SAPI: The order in which calls can be made, and threading issues. One of the key benefits from SAPI 5.0 is the simplification of threading issues.

Basics

An engine provides its services to SAPI through the ISpSREngine interface. The function through which all recognitions are made is the ISpSREngine::RecognizeStream. When SAPI calls ISpSREngine::SetSite, it passes in a pointer to the interface ISpSREngineSite through which the engine will communicate to SAPI during the execution of ISpSREngine::RecognizeStream. SAPI dedicates a thread to an ISpSREngine object and the engine should not return from ISpSREngine::RecognizeStream until there is a failure or SAPI has indicated using ISpSREngineSite::Read, that there will be no more data to process and the engine has done whatever cleanup is appropriate.

SAPI isolates engine developers from the details of managing an audio device. SAPI maintains a logical stream of raw audio data that it indexes with a stream position index. With a stream position index, an engine can call ISpSREngineSite::Read to receive a buffer of raw audio data during the execution of ISpSREngine::RecognizeStream. This call will block until all requested data is available. If ISpSREngineSite::Read returns with less data than requested, this means that there is no more data and the engine should return from ISpSREngine::RecognizeStream.

Beyond Basics

Once an engine has recognized a phrase with enough certainty that it warrants a notification to the application, it should call ISpSREngineSite::AddEvent. The stream position passed into AddEvent indicates the point in the audio stream after which the engine is seeking recognitions. To pass phrase hypotheses to the application, the engine calls ISpSREngineSite::Recognition and passes an SPRECORESULTINFO structure with the hypothesis flag set. Once an engine has a final recognition candidate, it must call ISpSREngineSite::Recognition with an SPRECORESULTINFO structure with the hypothesis flag not set. If an engine rejects all possible recognitions, it calls ISpSREngineSite::Recognition and passes in a NULL even though it called ISpSREngineSite::AddEvent. Although every call to ISpSREngineSite::AddEvent should be followed by one or more calls to ISpSREngineSite::Recognition, this restriction is not enforced by SAPI and must not be depended upon.

Calling order of functions and threading

The SAPI DDI layers make it possible for an engine to only have one thread which executes between SAPI and an engine. The only method in ISpSREngine which does not enter and exit relatively quickly is ISpSREngine::RecognizeStream. The engine can remain executing and blocking inside of ::RecognizeStream until it fails or receives no more data to process from ISpSREngineSite::Read. When the engine has the opportunity to give SAPI a chance to call back into ISpSREngine, it should call ISpSREngine::Synchronize and pass the stream position index of where the engine has finished recognizing audio data. From whatever thread Synchronize is called on, SAPI can call back into any of the other ISpSREngine methods except Recognize. For example, the speaker can change, grammars can unload, activate or deactivate dynamically. At any point in time, ISpSREngine will be called only on the original thread which created the ISpSREngine or the thread on which the engine called ISpSREngineSite::Sychronize. If the engine only calls ISpSREngineSite::Synchronize on the same thread which entered ISpSREngine::RecognizeStream, there is only one thread with which SAPI ever calls into an instance of ISpSREngine.

return to the top of this page Back to top


Welcome COM Developers!

 The Microsoft.com Speech website

Engine Characteristics

Engines use the three characteristics of Volume, Pitch, and Rate to partially define speech traits. At the application level, setting these values is simple, you need only set them to a given number. However, implementation of these traits is more complex for the engine.

Volume

At the application level, volume is a number from 0 to 100 where 100 is the maximum value for a voice. It is a linear progression and a value of 50 represents half of the loudest permitted. The increments should be the range divided by 100.

Pitch Adjustment

The value can range from -10 to +10. A value of zero sets a voice to speak at its default pitch. A value of -10 sets a voice to speak at three-fourths of its default pitch. A value of +10 sets a voice to speak at four-thirds of its default pitch. Each increment between -10 and +10 is logarithmically distributed such that incrementing or decrementing by 1 is multiplying or dividing the pitch by the 24th root of 2 (about 1.03). Values outside of the -10 and +10 range will be passed to an engine. However, SAPI 5.0-compliant engines may not support such extremes and may clip the pitch to the maximum or minimum the engine supports. Values of -24 and +24 must lower and raise pitch by 1 octave respectively. All incrementing or decrementing by 1 must multiply or divide the pitch by the 24th root of 2.

Rate Adjustment

The value can range from -10 to +10. A value of zero sets a voice to speak at its default rate. A value of -10 sets a voice to speak at one-third of its default rate. A value of +10 sets a voice to speak at three times its default rate. Each increment between -10 and +10 is logarithmically distributed such that incrementing or decrementing by 1 is multiplying or dividing the rate by the 10th root of 3 (about 1.1). Values more extreme than -10 and +10 will be passed to an engine. However, SAPI 5.0-compliant engines may not support such extremes and may clip the rate to the maximum or minimum rate the engine supports.


American English Phoneme Representation

This is a brief introduction to the use and implementation of the SAPI phoneme representations.

Symbolic and Numerical Representation

Application developers can create pronunciations for words that are not currently in the lexicon by using the English phonemes represented in the following table. The phoneme set is composed of a symbolic phonetic representation (SYM).

The application developer will be able to enter the SYM representation to create the pronunciation using the XML PRON tag, or by creating a new lexicon entry. Each phoneme entry should be space delimited.

Tag | Description
PRON SYM | Tag used to insert a pronunciation using symbolic representation.

Example: pronunciation for "hello":

<PRON SYM = "h eh l ow"/>

For improved accuracy, the primary (1), secondary (2) stress markers, and the syllabic markers (-) can be added to the pronunciation.

Example: pronunciation for "hello" using the primary stress (1) and syllabic (-) markers:

<PRON SYM = "h eh - l ow 1"/>

American English Phoneme Table

SYM | Example | PhoneID
- | syllable boundary (hyphen) | 1
! | Sentence terminator (exclamation mark) | 2
& | word boundary | 3
, | Sentence terminator (comma) | 4
. | Sentence terminator (period) | 5
? | Sentence terminator (question mark) | 6
_ | Silence (underscore) | 7
1 | Primary stress | 8
2 | Secondary stress | 9
aa | father | 10
ae | cat | 11
ah | cut | 12
ao | dog | 13
aw | foul | 14
ax | ag0 | 15
ay | bite | 16
b | big | 17
ch | chin | 18
d | dig | 19
dh | then | 20
eh | pet | 21
er | fur | 22
ey | ate | 23
f | fork | 24
g | gut | 25
h | help | 26
ih | fill | 27
iy | feel | 28
jh | joy | 29
k | cut | 30
l | lid | 31
m | mat | 32
n | no | 33
ng | sing | 34
ow | go | 35
oy | toy | 36
p | put | 37
r | red | 38
s | sit | 39
sh | she | 40
t | talk | 41
th | thin | 42
uh | book | 43
uw | too | 44
v | vat | 45
w | with | 46
y | yard | 47
z | zap | 48
zh | pleasure | 49

Please see International Phonemes for information on other phoneme sets.


International Phoneme Representation

You can create pronunciations for words that are not currently in the lexicon using the phonemes represented in the attached appendices. The proposed phoneme set is composed of a symbolic phonetic representation (SYM).

You can enter the SYM representation to create the pronunciation by using the XML PRON tag, or by creating a new lexicon entry. Each phoneme should be space delimited.

The engine is passed a USHORT structure called SPPHONEID (a number between 1 and n where n is the total number of phonemes for that language). The conversion from the SYM to SPPHONEID occurs in the SAPI PhoneConverter.

Mark Up Tag | Description
PRON SYM | Tag used to insert a pronunciation using symbolic representation

Example: pronunciation for "hello"

<PRON SYM = "h eh l ow"/>

For improved accuracy, the primary (1), secondary (2) stress markers, and the syllabic markers (-) can be added to the pronunciation.

Example: pronunciation for "hello" using the primary stress (1) and syllabic (-) markers:

<PRON SYM = "h eh - l ow 1"/>

SAPI-compliant engines are required to accept the PHONEID representation, and produce an articulation. The specific allophonic articulation is defined by the engine.

Non-SAPI phonemes are passed to the engine and the high bit on the PHONEID will be set. This will indicate to the engine that SAPI did not understand the phoneme. Your engine should be capable of bounds checking and either using the phoneme, or recovering gracefully. In the case where an application asks for a pronunciation (using the ISpLexicon::GetPronunication method) from the engine's lexicon, SAPI may pass an unknown phoneme back to the application. The application should verify the high bit of the SPHONEID to determine if the phoneme is a SAPI-defined phoneme or an engine-specific phoneme.

Main goals for defining the language dependent phoneme set:
  • Provide an engine-independent architecture for application developers to create user and application lexicons.
  • Make the English phonetic table simple enough to be used and understood by non-linguists who use the American English phoneme set.
  • Pass phonemes to the engine that are not recognized by SAPI.

International phoneme use

Using the international phoneme schema, you can create a phoneme set which can be used for each language independently. Using the numeric representation as opposed to the International Phonetic Alphabet (IPA) code will eliminate some of the problems regarding the possibile differences in the IPA values for the same phonemes. Hence, an 'r' in English will correspond to a certain number (38) and an 'r' in French may correspond to a different number. It is up to the individual engine to provide the exact IPA value for the two 'r's.

Each language will be associated with a set of phonemes numbered from 1 to X. You can use either the symbolic representation or the number representation to enter the pronunciation. Since you are probably not a linguist, the IPA code will probably have little meaning.

Please note that consistent pronunciation is NOT a goal, while predictable pronunciation is. Using the phoneme set, an application developer can guarantee a minimal pronunciation, but not the exact allophonic expression. So, the word "first" will always be pronounced as "first", never as "fist" or "feast", etc, but the accent of the engine may be slightly different due to the fact that the internal allophone values may differ

For more information and definitions for international phoneme sets, please see:

  • Chinese Phonemes
  • Japanese Phonemes


Chinese Phonemes

The following table defines the Chinese language phoneme set.

Symbol | PhoneID | Example
- | 1 | Syllable boundary (hyphen)
! | 2 | Sentence terminator (exclamation mark)
& | 3 | word boundary
, | 4 | Sentence terminator (comma)
. | 5 | Sentence terminator (period)
? | 6 | Sentence terminator (question mark)
_ | 7 | Silence (underscore)
+ | 8 | primary stress
* | 9 | secondary stress
1 | 10 | Tone 1
2 | 11 | Tone 2
3 | 12 | Tone 3
4 | 13 | Tone 4
5 | 14 | Tone 5
a | 15 | a 1 fei 1 (rogue)
ai | 16 | ai 4 ren 2 (lover)
an | 17 | an 1 quan 2 (safe)
ang | 18 | ang 1 zang 1 (dirty)
ao | 19 | jiao 1 ao 4 (proud)
ba | 20 | ba 4 ba 5 (dad)
bai | 21 | bai 2 se 4 (white)
ban | 22 | mu 4 ban 3 (board)
bang | 23 | bang 3 jia 4 (kidnap)
bao | 24 | yong 1 bao 4 (embrace)
bei | 25 | bei 3 fang 1 (north)
ben | 26 | ben 4 dan 4 (fool)
beng | 27 | beng 4 tiao 4 (jump)
bi | 28 | bi 2 zi 5 (nose)
bian | 29 | bian 4 hua 5 (change)
biao | 30 | biao 3 ge 2 (table)
bie | 31 | li 2 bie 2 (part with)
bin | 32 | bin 1 ke 4 (guest)
bing | 33 | shi 4 bing 1 (soldier)
bo | 34 | bo 2 dou 4 (wrestle)
bu | 35 | bu 4 xing 2 (walk)
ca | 36 | ca 1 gan 1 (wipe)
cai | 37 | cai 1 ce 4 (guess)
can | 38 | can 1 jia 1 (join)
cang | 39 | cang 1 ying 5 (fly)
cao | 40 | cao 1 chang 3 (playground)
ce | 41 | ce 4 liang 2 (measure)
cen | 42 | cen 1 ci 1 (uneven)
ceng | 43 | ceng 2 jing 1 (once)
cha | 44 | jian 3 cha 2 (check)
chai | 45 | chai 1 hui 3 (demolish)
chan | 46 | chan 3 ye 4 (industry)
chang | 47 | jing 1 chang 2 (often)
chao | 48 | chao 1 yue 4 (exceed)
che | 49 | qi 4 che 1 (automobile)
chen | 50 | chen 2 jiu 4 (old)
cheng | 51 | cheng 2 che 1 (ride)
chi | 52 | chi 2 dao 4 (late for)
chong | 53 | chong 1 man 3 (full of)
chou | 54 | chou 2 hen 4 (hatred)
chu | 55 | chu 2 fa 3 (division)
chuai | 56 | chuai 3 ce 4 (guess)
chuan | 57 | lun 2 chuan 2 (ship)
chuang | 58 | chuang 4 ye 4 (carve out)
chui | 59 | tie 3 chui 2 (hammer)
chun | 60 | chun 1 tian 1 (spring)
chuo | 61 | wo 4 chuo 4 (dirty)
ci | 62 | ci 2 qi 4 (porcelain)
cong | 63 | cong 2 lin 2 (thicket)
cou | 64 | jin 3 cou 4 (compact)
cu | 65 | cu 4 xiao 1 (sales promotion)
cuan | 66 | tao 2 cuan 4 (run away)
cui | 67 | cui 1 cu 4 (press)
cun | 68 | xiang 1 cun 1 (country)
cuo | 69 | cuo 4 wu 4 (error)
da | 70 | da 4 (big)
dai | 71 | dai 4 biao 3 (delegate)
dan | 72 | dan 4 shi 4 (but)
dang | 73 | dang 1 ran 2 (sure)
dao | 74 | dao 1 (knife)
de | 75 | de 5 (function word)
dei | 76 | dei 3 (must)
den | 77 | den 4 (yank)
deng | 78 | ban 3 deng 4 (wooden stool)
di | 79 | di 2 que 4 (certainly)
dia | 80 | dia 3 (coquetry voice)
dian | 81 | dian 3 (dot)
diao | 82 | diao 4 (hang)
die | 83 | die 1 dao 3 (tumble)
ding | 84 | jue 2 ding 4 (decide)
diu | 85 | diu 1 qi 4 (discard)
dong | 86 | dong 1 fang 1 (east)
dou | 87 | zhan 4 dou 4 (struggle)
du | 88 | du 4 jue 2 (stop)
duan | 89 | duan 3 (short)
dui | 90 | dui 4 (right)
dun | 91 | ting 2 dun 4 (halt)
duo | 92 | duo 1 yu 2 (unnecessary)
e | 93 | e 4 (hungry)
ei | 94 | ei 4 (yes)
en | 95 | en 1 hui 4 (favor)
er | 96 | er 3 (ear)
fa | 97 | fa 1 zhan 3 (development)
fan | 98 | fan 4 (rice)
fang | 99 | fang 2 jian 1 (room)
fei | 100 | fei 1 (fly)
fen | 101 | fen 1 bie 2 (separate)
feng | 102 | feng 1 (wind)
fo | 103 | fo 2 (buddha)
fou | 104 | fou 3 ding 4 (denial)
fu | 105 | fu 4 qin 1 (father)
ga | 106 | gan 1 ga 4 (awkward)
gai | 107 | ying 1 gai 1 (should)
gan | 108 | gan 1 jing 4 (neatness)
gang | 109 | gang 1 cai 2 (just)
gao | 110 | gao 1 (tall)
ge | 111 | pin 3 ge 2 (character)
gei | 112 | gei 3 yu 3 (give)
gen | 113 | gen 1 (root)
geng | 114 | geng 4 jia 1 (much more)
gong | 115 | gong 1 ren 2 (worker)
gou | 116 | zu 2 gou 4 (enough)
gu | 117 | gu 4 xiang 1 (hometown)
gua | 118 | xi 1 gua 1 (watermelon)
guai | 119 | qi 2 guai 4 (oddness)
guan | 120 | guan 1 bi 4 (close)
guang | 121 | guang 1 (light)
gui | 122 | gui 3 (ghost)
gun | 123 | gun 4 (stick)
guo | 124 | guo 4 qu 4 (past)
ha | 125 | ha 1 (sound of laugh)
hai | 126 | hai 2 zi 5 (child)
han | 127 | chu 1 han 4 (sweat)
hang | 128 | hang 2 xing 2 (sail)
hao | 129 | hao 3 (good)
he | 130 | he 2 (river)
hei | 131 | hei 1 (black)
hen | 132 | hen 3 (very)
heng | 133 | heng 2 xiang 4 (landscape orientation)
hong | 134 | hong 2 (red)
hou | 135 | hou 2 zi 5 (monkey)
hu | 136 | hu 2 xu 1 (beard)
hua | 137 | hua 4 (picture)
huai | 138 | huai 4 (bad)
huan | 139 | huan 2 jing 4 (environment)
huang | 140 | huang 2 se 4 (yellow)
hui | 141 | hui 1 huang 2 (refulgence)
hun | 142 | hun 2 zhuo 2 (muddy)
huo | 143 | huo 3 (fire)
ji | 144 | ji 2 shi 2 (in time)
jia | 145 | jia 4 qi 1 (holiday)
jian | 146 | jian 3 dan 1 (simple)
jiang | 147 | jiang 1 (river)
jiao | 148 | jiao 1 tong 1 (traffic)
jie | 149 | jie 2 ri 4 (feast)
jin | 150 | jin 3 zhang 1 (strain)
jing | 151 | gan 1 jing 4 (neatness)
jiong | 152 | jiong 3 po 4 (embarrassed)
jiu | 153 | jiu 3 (nine)
ju | 154 | ju 4 zi 5 (sentence)
juan | 155 | juan 3 qu 1 (curl)
jue | 156 | jue 2 ding 4 (decide)
jun | 157 | jun 1 dui 4 (army)
ka | 158 | ka 3 che 1 (truck)
kai | 159 | kai 3 xuan 2 (triumph)
kan | 160 | kan 4 (see)
kang | 161 | di 3 kang 4 (resist)
kao | 162 | kao 3 shi 4 (test)
ke | 163 | ke 3 (thirsty)
kei | 164 | kei 1 (scold)
ken | 165 | ken 3 ding 4 (affirm)
keng | 166 | keng 1 hai 4 (entrap)
kong | 167 | kong 1 qi 4 (air)
kou | 168 | kou 3 (mouth)
ku | 169 | jian 1 ku 3 (trial)
kua | 170 | kua 1 jiang 3 (praise)
kuai | 171 | kuai 4 (fast)
kuan | 172 | kuan 1 kuo 4 (openness)
kuang | 173 | kong 1 kuang 4 (void)
kui | 174 | kui 1 qian 4 (owe)
kun | 175 | kun 4 nan 2 (hard)
kuo | 176 | kuo 4 da 4 (enlarge)
la | 177 | la 4 jiao 1 (hot pepper)
lai | 178 | lai 2 (come)
lan | 179 | lan 2 se 4 (blue)
lang | 180 | lang 2 (wolf)
lao | 181 | lao 3 (old)
le | 182 | kuai 4 le 4 (happy)
lei | 183 | lei 2 (thunder)
leng | 184 | leng 3 ku 4 (steeliness)
li | 185 | li 2 (pear)
lia | 186 | lia 2 (two)
lian | 187 | lian 3 (face)
liang | 188 | li 4 liang 4 (power)
liao | 189 | zuo 2 liao 4 (seasoning)
lie | 190 | lin 3 lie 4 (severe)
lin | 191 | lin 3 lie 4 (severe)
ling | 192 | ling 2 mu 4 (mausoleum)
liu | 193 | liu 2 dong 4 (flow)
lo | 194 | lo 5 (function word)
long | 195 | long 2 (dragon)
lou | 196 | lou 4 (leak)
lu | 197 | lu 4 di 4 (land)
luan | 198 | hun 4 luan 4 (chaos)
lue | 199 | ce 4 lue 4 (tactic)
lun | 200 | yi 4 lun 4 (discuss)
luo | 201 | xia 4 luo 4 (whereabouts)
lv | 202 | lv 4 se 4 (green)
ma | 203 | ma 3 (horse)
mai | 204 | mai 2 zang 4 (bury)
man | 205 | man 3 zu 2 (satisfy)
mang | 206 | cong 1 mang 2 (hurry)
mao | 207 | mao 1 (cat)
me | 208 | shen 2 me 5 (what)
mei | 209 | mei 2 you 3 (no)
men | 210 | wo 3 men 2 (we)
meng | 211 | meng 4 (dream)
mi | 212 | hun 1 mi 2 (coma)
mian | 213 | mian 4 ji 5 (area)
miao | 214 | miao 4 (temple)
mie | 215 | mie 4 jue 2 (annihilation)
min | 216 | ren 2 min 2 (people)
ming | 217 | ming 2 bai 2 (clearness)
miu | 218 | huang 1 miu 4 (absurd)
mo | 219 | mo 4 shui 3 (ink)
mou | 220 | mou 2 lue 4 (trick)
mu | 221 | mu 4 biao 1 (target)
na | 222 | na 4 li 3 (there)
nai | 223 | nai 3 nai 5 (grandmother)
nan | 224 | nan 2 fang 1 (south)
nang | 225 | nang 2 kuo 4 (include)
nao | 226 | re 4 nao 4 (liveliness)
ne | 227 | mu 4 ne 4 (numb)
nei | 228 | nei 4 bu 4 (inside)
nen | 229 | nen 4 lv 4 (light green)
neng | 230 | neng 2 gou 4 (be capable of)
ni | 231 | ni 3 (you)
nian | 232 | nian 2 (year)
niang | 233 | gu 1 niang 5 (girl)
niao | 234 | niao 3 (bird)
nie | 235 | nie 4 (bite)
nin | 236 | nin 2 (you)
ning | 237 | an 1 ning 2 (peace)
niu | 238 | niu 2 (bull)
nong | 239 | nong 2 min 2 (farmer)
nou | 240 | nou 4 (weeding)
nu | 241 | nu 3 li 4 (try hard)
nuan | 242 | wen 1 nuan 3 (warm)
nue | 243 | nue 4 dai 4 (abuse)
nuo | 244 | nuo 4 yan 2 (promise)
nv | 245 | nv 3 zi 3 (woman)
o | 246 | o 5 (function word)
ou | 247 | ou 1 yang 2 (Chinese first name)
pa | 248 | hai 4 pa 4 (scare)
pai | 249 | pai 4 qian 3 (send)
pan | 250 | pan 4 tu 2 (betrayer)
pang | 251 | pang 2 da 4 (huge)
pao | 252 | pao 3 (run)
pei | 253 | pei 2 ban 4 (accompany)
pen | 254 | pen 2 di 4 (pan)
peng | 255 | peng 2 pai 4 (surge)
pi | 256 | pi 2 fu 1 (skin)
pian | 257 | qi 1 pian 4 (cheat)
piao | 258 | piao 4 liang 4 (pretty)
pie | 259 | pie 1 kai 1 (put aside)
pin | 260 | pin 2 fan 2 (frequency)
ping | 261 | ping 2 zi 5 (bottle)
po | 262 | po 4 huai 4 (damage)
pou | 263 | pou 2 (hold someting with cupped hand)
pu | 264 | pu 2 tao 5 (grape)
qi | 265 | qi 3 qiu 2 (beg)
qia | 266 | qia 4 dang 4 (proper)
qian | 267 | qian 1 xu 1 (humility)
qiang | 268 | qiang 2 da 4 (powerful)
qiao | 269 | qiao 3 miao 4 (artifice)
qie | 270 | xiu 1 qie 4 (shyness)
qin | 271 | qin 1 zi 4 (oneself)
qing | 272 | qing 1 song 1 (easy)
qiong | 273 | qiong 2 (poverty)
qiu | 274 | qiu 2 fan 4 (prisoner)
qu | 275 | qu 1 dong 4 (drive)
quan | 276 | quan 2 bu 4 (all)
que | 277 | que 4 ding 4 (ensure)
qun | 278 | qun 2 zhong 4 (crowd)
ran | 279 | ran 2 hou 4 (then)
rang | 280 | tu 3 rang 3 (soil)
rao | 281 | wei 2 rao 4 (surround)
re | 282 | re 4 (hot)
ren | 283 | ren 2 lei 4 (human being)
reng | 284 | reng 2 ran 2 (all the same)
ri | 285 | ri 4 chu 1 (sunrise)
rong | 286 | rong 2 yao 4 (glory)
rou | 287 | niu 2 rou 4 (beef)
ru | 288 | ru 2 guo 3 (if)
ruan | 289 | rou 2 ruan 3 (soft)
rui | 290 | rui 4 zhi 4 (smart)
run | 291 | shi 1 run 4 (wetness)
ruo | 292 | ruo 4 xiao 3 (puniness)
sa | 293 | sa 1 (three)
sai | 294 | bi 3 sai 4 (compete)
san | 295 | fen 1 san 4 (disperse)
sang | 296 | sang 4 (die)
sao | 297 | da 3 sao 3 (sweep)
se | 298 | yan 2 se 4 (color)
sen | 299 | sen 1 lin 2 (forest)
seng | 300 | seng 1 lv 3 (monk)
sha | 301 | sha 1 chang 3 (battlefield)
shai | 302 | ri 4 shai 4 (be exposed to the sun)
shan | 303 | shan 4 liang 2 (goodness)
shang | 304 | shang 4 mian 4 (top)
shao | 305 | shao 4 nian 2 (youth)
she | 306 | she 2 (snake)
shei | 307 | shei 2 (who)
shen | 308 | shen 2 me 5 (what)
sheng | 309 | sheng 1 yin 1 (voice)
shi | 310 | shi 4 fei 1 (dispute)
shou | 311 | shou 3 (hand)
shu | 312 | shu 4 mu 4 (tree)
shua | 313 | shua 1 zi 5 (brush)
shuai | 314 | shuai 1 da 3 (beat)
shuan | 315 | men 2 shuan 1 (latch)
shuang | 316 | shuang 1 (pair)
shui | 317 | shui 3 (water)
shun | 318 | shun 4 li 4 (all right)
shuo | 319 | shuo 1 (say)
si | 320 | si 4 (four)
song | 321 | song 4 bie 2 (send-off)
sou | 322 | sou 1 cha 2 (search)
su | 323 | su 4 du 4 (speed)
suan | 324 | suan 4 fa 3 (arithmetic)
sui | 325 | sui 1 ran 2 (though)
sun | 326 | sun 3 shi 1 (loss)
suo | 327 | suo 3 yi 3 (so)
ta | 328 | ta 1 men 2 (they)
tai | 329 | tai 4 (too)
tan | 330 | tan 2 hua 4 (talk)
tang | 331 | tang 2 (sugar)
tao | 332 | tao 2 pao 3 (flee)
te | 333 | te 4 bie 2 (special)
tei | 334 | tei 1 (very)
teng | 335 | ben 1 teng 2 (riot)
ti | 336 | ti 1 zi 5 (ladder)
tian | 337 | tian 1 (sky)
tiao | 338 | tiao 4 yue 4 (jump)
tie | 339 | tie 3 (iron)
ting | 340 | ting 1 (listen)
tong | 341 | tong 3 yi 1 (unify)
tou | 342 | tou 2 (head)
tu | 343 | tu 3 di 4 (earth)
tuan | 344 | tuan 2 jie 2 (solidify)
tui | 345 | tui 1 dong 4 (push)
tun | 346 | tun 2 ji 1 (store up)
tuo | 347 | tuo 1 yi 1 (undress)
wa | 348 | qing 1 wa 1 (frog)
wai | 349 | wai 4 mian 4 (outside)
wan | 350 | wan 4 (ten thousands)
wang | 351 | si 3 wang 2 (death)
wei | 352 | wei 3 da 4 (great)
wen | 353 | wen 2 hua 4 (culture)
weng | 354 | lao 3 weng 1 (oldman)
wo | 355 | wo 3 (I)
wu | 356 | wu 2 lun 4 (no matter what)
xi | 357 | xi 1 fang 1 (west)
xia | 358 | xia 4 mian 4 (bottom)
xian | 359 | xian 4 zai 4 (now)
xiang | 360 | xiang 4 (like)
xiao | 361 | xiao 3 (small)
xie | 362 | xie 3 (write)
xin | 363 | xin 1 (new)
xing | 364 | xing 2 dong 4 (act)
xiong | 365 | xiong 2 (bear)
xiu | 366 | xiu 1 xi 5 (break)
xu | 367 | xu 1 yao 4 (need)
xuan | 368 | xuan 1 bu 4 (declare)
xue | 369 | xue 3 (snow)
xun | 370 | xun 2 wen 4 (ask for)
ya | 371 | ya 4 zhou 1 (Asia)
yan | 372 | yan 4 zi 5 (swallow)
yang | 373 | tai 4 yang 2 (sun)
yao | 374 | yao 1 qiu 2 (require)
ye | 375 | ye 2 ye 5 (grandfather)
yi | 376 | yi 1 (one)
yin | 377 | yin 1 yue 4 (music)
ying | 378 | ying 1 gai 1 (should)
yo | 379 | yo 5 (function word)
yong | 380 | yong 3 yuan 3 (forever)
you | 381 | you 2 yu 2 (due to)
yu | 382 | yu 2 (fish)
yuan | 383 | yuan 2 lai 2 (formerly)
yue | 384 | yue 4 (month)
yun | 385 | yun 2 (cloud)
za | 386 | za 2 luan 4 (disorder)
zai | 387 | zai 4 jian 4 (farewell)
zan | 388 | zan 2 men 5 (our)
zang | 389 | ang 1 zang 1 (dirty)
zao | 390 | zao 3 chen 2 (morning)
ze | 391 | ze 2 ren 4 (response)
zei | 392 | zei 2 (thief)
zen | 393 | zan 4 yang 2 (praise)
zeng | 394 | zeng 1 jia 1 (add)
zha | 395 | zha 4 dan 4 (bomb)
zhai | 396 | qian 4 zhai 4 (owe)
zhan | 397 | zhan 4 li 4 (stand)
zhang | 398 | zhang 1 kai 1 (open)
zhao | 399 | zhao 1 huan 4 (summon)
zhe | 400 | zhe 4 li 3 (here)
zhei | 401 | zhei 5 (this)
zhen | 402 | zhen 1 zheng 4 (real)
zheng | 403 | zheng 4 yi 4 (justice)
zhi | 404 | yi 4 zhi 4 (restrain)
zhong | 405 | zhong 1 guo 2 (China)
zhou | 406 | si 4 zhou 1 (around)
zhu | 407 | zhu 1 (pig)
zhua | 408 | zhua 1 (grab)
zhuai | 409 | zhuai 4 (pull)
zhuan | 410 | zhuan 1 (brick)
zhuang | 411 | zhuang 1 (pretend)
zhui | 412 | zhui 1 (chase)
zhun | 413 | zhun 3 (precise)
zhuo | 414 | zhuo 1 (table)
zi | 415 | zi 4 (character)
zong | 416 | zong 3 (total)
zou | 417 | zou 3 (walk)
zu | 418 | zu 1 (rent)
zuan | 419 | zuan 4 (diamond)
zui | 420 | zui 4 (most)
zun | 421 | zun 1 (respect)
zuo | 422 | zuo 4 (sit)


Japanese Phonemes

SAPI Supported Japanese Phonemes (Katakana)

Symbol | Phonetic | Unicode Value | Remarks
 |  | 309C | Semi-voiced sound
 | a | 30A1 | Modifier
 | a | 30A2 |
 | i | 30A3 | Modifier
 | i | 30A4 |
 | u | 30A5 |
 | u | 30A6 |
 | e | 30A7 | Modifier
 | e | 30A8 |
 | o | 30A9 | Modifier
 | o | 30AA |
 | ka | 30AB |
 | ga | 30AC |
 | ki | 30AD |
 | gi | 30AE |
 | ku | 30AF |
 | gu | 30B0 |
 | ke | 30B1 |
 | ge | 30B2 |
 | ko | 30B3 |
 | go | 30B4 |
 | sa | 30B5 |
 | za | 30B6 |
 | shi | 30B7 |
 | ji | 30B8 |
 | su | 30B9 |
 | zu | 30BA |
 | se | 30BB |
 | ze | 30BC |
 | so | 30BD |
 | zo | 30BE |
 | ta | 30BF |
 | da | 30C0 |
 | chi | 30C1 |
 | di | 30C2 | Deprecated - use 30B8
 | q | 30C3 |
 | tsu | 30C4 |
 | du | 30C5 | Deprecated - use 30BA
 | te | 30C6 |
 | de | 30C7 |
 | to | 30C8 |
 | do | 30C9 |
 | na | 30CA |
 | ni | 30CB |
 | nu | 30CC |
 | ne | 30CD |
 | no | 30CE |
 | ha | 30CF |
 | ba | 30D0 |
 | pa | 30D1 |
 | hi | 30D2 |
 | bi | 30D3 |
 | pi | 30D4 |
 | hu | 30D5 |
 | bu | 30D6 |
 | pu | 30D7 |
 | he | 30D8 |
 | be | 30D9 |
 | pe | 30DA |
 | ho | 30DB |
 | bo | 30DC |
 | po | 30DD |
 | ma | 30DE |
 | mi | 30DF |
 | mu | 30E0 |
 | me | 30E1 |
 | mo | 30E2 |
 | ya | 30E3 | Modifier
 | ya | 30E4 |
 | yu | 30E5 | Modifier
 | yu | 30E6 |
 | yo | 30E7 | Modifier
 | yo | 30E8 |
 | ra | 30E9 |
 | ri | 30EA |
 | ru | 30EB |
 | re | 30EC |
 | ro | 30ED |
 | wa | 30EE | Modifier
 | wa | 30EF |
 | wi | 30F0 | Deprecated - use 30A4
 | we | 30F1 | Deprecated - use 30A8
 | wo | 30F2 |
 | nn | 30F3 |
 | vu | 30F4 |
 | ka | 30F5 |
 | ke | 30F6 |
 | va | 30F7 |
 | vi | 30F8 |
 | ve | 30F9 |
 | vo | 30FA |
 |  | 30FB | Middle dot
 |  | 30FC | Prolonged sound
 |  | 30FD | Iteration
 |  | 30FE | Voiced iteration


Application-Level Interfaces

This section describes the interfaces and methods for incorporating speech into applications. They are intended for use at the API or application level. Some managers or interfaces may have entries also in Engine-Level Interface section. However, entries listed here apply only to the application level.

  • Audio interfaces
  • Event interfaces
  • Grammar Compiler interfaces
  • Lexicon interfaces
  • Resource interfaces
  • Speech Recognition interfaces
  • Text-to-Speech interfaces


Audio interfaces

This section provides SAPI 5.0 audio interfaces.

Audio inherits from the standard COM IStream interface. See the MSDN documentation for a complete discussion of IStream and associated methods. However, since the audio devices represent hardware, ::Clone may be not be used and will return E_NOTIMPL.

  • ISpAudio
  • ISpMMSysAudio
  • ISpStream
  • ISpStreamFormat
  • ISpStreamFormatConverter

The following interface does not inherit from IStream:

  • ISpTranscript

Development Helpers

Helper Enumerations, Functions and Classes | Description
SPSTREAMFORMAT | SAPI supported stream formats
CSpEvent | Class for decoding event structures.
CSpDynamicString | Class for managing dynamically sized WCHAR strings
SpBindToFile | Function converts the specified stream format into a wave format structure.
CSpStreamFormat | Class for managing SAPI supported stream formats and WAVEFORMATEX structures


ISpAudio

Objects implementing this interface are real-time audio streams, such as those connected to a live microphone or telephone line. ISpAudio methods allow control over the real-time behavior of the stream. IStream Read and Write methods transfer data to or from an object.

When to Implement

This interface should be implemented when the audio input or output source is not a standard windows Multimedia device. It is expected to supply an infinite amount of data and hence its state should not change externally to SAPI. For the majority of use, it will not be necessary to implement an object providing this interface. An example of where this might be needed would be to provide a telephony audio device or to perform echo cancellation of audio output on the input.

Telephone application programming interface (TAPI) provides a mechanism to treat a telephony device as a Windows multimedia device allowing the use of the SAPI provided multimedia audio objects.

If this implements a real time audio input or output object and runs in a desktop or graphical environment, support may be needed for volume (see SPDUI_AudioVolume) and audio properties (see SPDUI_AudioProperties) UI. The SAPI best-coding-practice for implementing the UI is have the object inherit from ISpTokenUI. This will enable applications (including the Speech Control Panel) to display the UI in a simple and consistent manner.

Implemented By
  • SpMMAudioIn
  • SpMMAudioOut

Methods in Vtable Order

Value | Description
ISpStreamFormat interface | Inherits from ISpStreamFormat and all those methods are accessible from an ISpAudio object.
SetState | Sets the state of the audio device.
SetFormat | Sets the format of the audio device.
GetStatus | Passes back the status of the audio device.
SetBufferInfo | Sets the audio stream buffer information.
GetBufferInfo | Passes back the audio stream buffer information.
GetDefaultFormat | Passes back the default audio format.
EventHandle | Returns a Win32 event handle that applications can use to wait for status changes in the I/O stream.
GetVolumeLevel | Passes back the current volume level.
SetVolumeLevel | Sets the current volume level.
GetBufferNotifySize | Retrieves the audio stream buffer size information.
SetBufferNotifySize | Sets the audio stream buffer size information.

Development Helpers

Helper Enumerations, Functions and Classes | Description
SPSTREAMFORMAT | SAPI supported stream formats
CSpStreamFormat | Class for managing SAPI supported stream formats and WAVEFORMATEX structures


ISpAudio::SetState

ISpAudio::SetState sets the state of the audio device.

When transitioning from the SPAS_CLOSED state to any other state, the caller should be ready to handle various error conditions, specifically, SPERR_UNSUPPORTED_FORMAT and SPERR_DEVICE_BUSY. Many multi-media devices do not correctly report their capabilities for handling different audio formats and fail only when an attempt is made to open the device.

Also, in many older systems, audio output devices can only be opened by a single process. In all current versions of Windows, only a single process can open an audio input device. Therefore, SPERR_DEVICE_BUSY will return if an attempt is made to open a device that is being used by a different process or thread.

On some older sound cards, recording and playback are not possible simultaneously or only possible at the same frequency. An application making use of the input and output audio should be aware of this and in particular attempt to gracefully degrade from higher quality frequencies to the same frequency for both if the sound card makes this necessary.

In general, applications need not change the state of the audio device directly. With the shared recognizer in particular, this will often cause unexpected results. SAPI will automatically manage the state of the audio device based on the state of all the grammars, recognition contexts and the recognizer instance.

HRESULT SetState(
   SPAUDIOSTATE   NewState,
   ULONGLONG      ullReserved
);

Parameters

NewState
[in] The flag of type SPAUDIOSTATE for the new state of the audio device.
ullReserved
[in] Reserved, do not use. This value must be zero.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | ullReserved is not zero or NewState is not one of the allowed values.
SPERR_DEVICE_BUSY | Hardware device is in use by another thread or process.
SPERR_UNSUPPORTED_FORMAT | Current format set by ISpAudio::SetFormat is not supported by the hardware device.


ISpAudio::SetFormat

ISpAudio::SetFormat sets the format of the audio device.

This method can be called only when the audio device is in the SPAS_CLOSED state. Note that successfully setting the format on an audio device does not necessarily mean that the format is supported. An attempt must be made to place the device into a non-closed state (SPAS_STOP, SPAS_PAUSE or SPAS_RUN) to be sure that the device can handle the format.

The format can be retrieved by calling the ISpStreamFormat::GetFormat method.

HRESULT SetFormat(
   REFGUID                rguidFmtId,
   const  WAVEFORMATEX   *pWaveFormatEx
);

Parameters

rguidFmtId
[in] The REFGUID for the format to set. Typically this will be SPDFID_WaveFormatEx. This is required for the SAPI multimedia objects.
pWaveFormatEx
[in] Address of the WAVEFORMATEX structure containing the wave file format information.

Return values

Value | Description
S_OK | Function completed successfully. See note about supported formats.
E_INVALIDARG | pWaveFormatEx is invalid or bad.
SPERR_DEVICE_BUSY | Device is not in the SPAS_CLOSED state.
SPERR_UNINITIALIZED | Audio stream not initialized.
SPERR_UNSUPPORTED_FORMAT | Specified format is not supported.
FAILED(hr) | Appropriate error message.

Remarks

The helper class CSpStreamFormat and the SPSTREAMFORMAT enumeration can be used to avoid the possibility of typos or mistakes when filling in the WAVEFORMATEX structure.


ISpAudio::GetStatus

ISpAudio::GetStatus gets the status of the audio device.

Use this method to determine whether the device is running, stopped, closed, or paused. It also determines the size of any buffered data.

HRESULT GetStatus(
  SPAUDIOSTATUS  *pStatus
);

Parameters

pStatus
[out] Pointer to an SPAUDIOSTATUS buffer to be filled with the status details.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pStatus is invalid.


ISpAudio::SetBufferInfo

ISpAudio::SetBufferInfo sets the audio stream buffer information.

This method can be called only when the audio device is in the SPAS_CLOSED state. The SPAUDIOBUFFERINFO members must conform to the following restrictions:

SPAudioBufferInfo.ulMsMinNotification cannot be larger than one quarter the size of SPAudioBufferInfo.ulMsBufferSize and must not be zero.
SPAudioBufferInfo.ulMsEventBias cannot be larger than SPAudioBufferInfo.ulMsBufferSize.
SPAudioBufferInfo.ulMsBufferSize must be greater than or equal to 200 milliseconds.
HRESULT SetBufferInfo(
   const SPAUDIOBUFFERINFO  *pBuffInfo
);

Parameters

pBuffInfo
[in] Pointer to the SPAUDIOBUFFERINFO buffer providing the requested settings.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Audio stream not initialized.
E_INVALIDARG | pBuffInfo is invalid or the parameters do not meet the criteria described above.
SPERR_DEVICE_BUSY | Audio device is not in the SPAS_CLOSED state.
FAILED(hr) | Appropriate error message.


ISpAudio::GetBufferInfo

ISpAudio::GetBufferInfo gets the audio stream buffer information.

HRESULT GetBufferInfo(
  SPAUDIOBUFFERINFO   *pBuffInfo
);

Parameters

pBuffInfo
[out] Pointer to the SPAUDIOBUFFERINFO buffer.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pBuffInfo is invalid.


ISpAudio::GetDefaultFormat

ISpAudio::GetDefaultFormat gets the default audio format.

Other formats may be supported by the audio device; however, this format is guaranteed to work. Older sound cards can potentially fail when attempting to use this format if they are not fully duplex or do not support full duplex operation at different frequencies. An application should attempt to degrade gracefully when this occurs.

HRESULT GetDefaultFormat(
   GUID            *pFormatId,
   WAVEFORMATEX   **ppCoMemWaveFormatEx
);

Parameters

pFormatId
[out] Pointer to the GUID of the default format.
ppCoMemWaveFormatEx
[out] Address of a pointer to the WAVEFORMATEX structure that receives the wave file format information. SAPI allocates the memory for the WAVEFORMATEX data structure using CoTaskMemAlloc, but it is the caller's responsibility to call CoTaskMemFree on the returned WAVEFORMATEX pointer.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Stream is uninitialized.
E_POINTER | pFormatId is invalid.


ISpAudio::EventHandle

ISpAudio::EventHandle returns a Win32 event handle that applications can use to wait for status changes in the I/O stream.

The handle may use one of the various Win32 wait functions, such as WaitForSingleObject or WaitForMultipleObjects.

For read streams, set the event when there is data available to read and reset it whenever there is no available data. For write streams, set the event when all of the data has been written to the device, and reset it at any time when there is still data available to be played.

The caller should not close the returned handle, nor should the caller ever use the event handle after calling Release() on the audio object. The audio device will close the handle on the final release of the object.

HANDLE  EventHandle( void );

Parameters

None

Return values

Value | Description
HANDLE | Returns valid event handle.


ISpAudio::GetVolumeLevel

ISpAudio::GetVolumeLevel passes back the current volume level.

The volume level is on a linear scale from 0 to 10000.

For input devices with a boost control for the microphone, SAPI will split the volume range into two to allow automated use of the boost. The boost will be off from 0 to 4999 and on from 5000 to 10,000. In each range, the full volume range of the device will be used independently. This can lead to discontinuity in the input energy level for a constant volume sound source. On some sound cards, the boost is applied to the input volume, but on others, the boost is applied to the playback volume resulting in the two ranges behaving identically because the input level is unaffected.

Microphone wizards attempting to choose the best input volume level should be aware of this fact and take the potential discontinuity into consideration to ensure the algorithm used to adjust the input volume level can handle the various possible forms of the discontinuity at 5,000.

For devices without a boost, there is no discontinuity at 5000.

HRESULT GetVolumeLevel(
   ULONG   *pLevel
);

Parameters

pLevel
[out] Pointer to the returned volume level.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Audio interface is not initialized.
SPERR_DEVICE_NOT_SUPPORTED | The device is not valid or does not support volumes.
E_POINTER | pLevel is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpAudio::SetVolumeLevel

ISpAudio::SetVolumeLevel sets the current volume level.

It is on a linear scale from 0 to 10000.

For input devices with a boost control for the microphone, SAPI will split the volume range into two to allow automated use of the boost. The boost will be off from 0 to 4,999 and on from 5,000 to 10,000. In each range, the full volume range of the device will be used independently. This can lead to discontinuity in the input energy level for a constant volume sound source. On some sound cards, the boost is applied to the input volume, but on others, the boost is applied to the playback volume resulting in the two ranges performing identically because the input level is unaffected.

Microphone wizards attempting to choose the best input volume level should be aware of this fact and take the potential discontinuity into consideration to ensure the algorithm used to adjust the input volume level can handle the various possible forms of the discontinuity at 5,000.

For devices without a boost, there is no discontinuity at 5,000.

HRESULT SetVolumeLevel(
   ULONG   Level
);

Parameters

Level
[in] The new volume level.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Level is greater than 10,000.
SPERR_DEVICE_NOT_SUPPORTED | The device is not valid or does not support volumes.
FAILED(hr) | Appropriate error message.


ISpAudio::GetBufferNotifySize

ISpAudio::GetBufferNotifySize retrieves the audio stream buffer size information. This information is used to determine when the event returned by ISpAudio::EventHandle is set or reset.

For read streams, the event is set if the audio buffered is greater than or equal to the value set in pcbSize, otherwise the event information is reset.

For write streams, the event is set if the audio buffered is less than the value set in pcbSize, otherwise the event information is reset.

HRESULT GetBufferNotifySize(
   ULONG   *pcbSize
);

Parameters

pcbSize
[out] Address of the size information, specified in bytes, that is associated with the audio stream buffer.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.


ISpAudio::SetBufferNotifySize

ISpAudio::SetBufferNotifySize sets the audio stream buffer size information. This information is used to determine when the event returned by ISpAudio::EventHandle is set or reset.

For read streams, the event is set if the audio buffered is greater than or equal to the value set in cbSize, otherwise the event information is reset.

For write streams, the event is set if the audio buffered is less than the value set in cbSize, otherwise the event information is reset.

HRESULT SetBufferNotifySize(
   ULONG   cbSize
);

Parameters

cbSize
[in] The size, specified in bytes, of the information associated with the audio stream buffer.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpMMSysAudio

This is the interface to the audio implementation for the standard Windows multimedia layer (wave in and wave out). Audio objects created through an object token do not allow the ISpMMSysAudio::SetDeviceId method to work because the token specifies which audio device ID to use. If an application wants to associate an audio object with a specific multimedia wave in or wave out device ID, it should use CoCreateInstance with CLSID_SpMMAudioOut or CLSID_SpMMAudioIn and then use the ISpMMSysAudio::SetDeviceId method to select the device. In normal application development, this will not be necessary for two reasons:

  • Desktop applications will generally use the shared recognizer instance which automatically sets itself to use the default audio device. This cannot be changed.
  • For the InProc recognizer instance, tokens exist to allow quick creation of the correct SpMMAudioIn or SpMMAudioOut for all of the multimedia devices on the system.

For input devices, SAPI will attempt to automatically identify the microphone line on the input device. On non-English versions of Windows and on a small number of English systems, it may not be possible for SAPI to automatically detect the correct microphone line. In this case, no error will be detected or returned. In such cases, if speech input is not correctly detected, the user must set the microphone input line directly using Speech properties in Control Panel-->Speech Recognition tab-->Audio Input Settings-->Properties. In particular, it may be necessary to adjust the microphone input line used on devices with multiple microphone inputs such as the SoundBlaster Live Platinum because the default input line may not be the input preferred by the user.

Implemented By
  • SpMMAudioIn
  • SpMMAudioOut

Methods in Vtable Order

ISpMMSysAudio Methods | Description
ISpStreamFormat interface | Inherits from ISpStreamFormat and all methods are accessible from an ISpMMSysAudio object.
ISpAudio interface | Inherits from ISpAudio and all methods are accessible from an ISpMMSysAudio object.
GetDeviceId | Passes back the multimedia device ID being used by the audio object.
SetDeviceId | Sets the multimedia device ID.
GetMMHandle | Passes back a multimedia audio stream handle.
GetLineId | Retrieves the line identifier associated with the multimedia device.
SetLineId | Sets the line identifier associated with the multimedia device.

IStream behavior as Inherited from ISpAudio

Please see the Microsoft® Platform Software Development Kit (PSDK) for a complete description of the IStream interface.

IStream Methods | Description
Read | Reads data from the multimedia audio device.
Write | Writes data to the multimedia audio device.
Seek | Retrieves only the current device position because multimedia devices represent hardware.
SetSize | Not used because multimedia devices represent hardware.
CopyTo | Copies a specified number of bytes from the current seek pointer in the stream to the current seek pointer in another stream.
Commit | Updates device state and commit buffered data. SAPI will automatically manage device state and buffered data, so that the developer is not expected to call this method.
Revert | Returns E_NOTIMPL because multimedia devices represent hardware.
LockRegion | Returns E_NOTIMPL because multimedia devices represent hardware.
UnlockRegion | Returns E_NOTIMPL because multimedia devices represent hardware.
Stat | Retrieves the current device position.
Clone | Returns E_NOTIMPL because multimedia devices represent hardware.


ISpMMSysAudio::Read

ISpMMSysAudio::Read reads the data from the multimedia audio device.

The audio device should not be directly manipulated when performing speech recognition (see ISpRecognizer).

HRESULT Read(
   void    *pv,
   ULONG    cb,
   ULONG   *pcbRead
);

Parameters

pv
[in] Pointer to the buffer into which the stream data is read. If an error occurs, this value is NULL.
cb
[in] Specifies the number of bytes of data to attempt to read from the audio device.
pcbRead
[out] Pointer to a ULONG variable that receives the actual number of bytes read from the stream object. If set to NULL, no byte value is passed back.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_AUDIO_BUFFER_OVERFLOW | SAPI's internal audio buffer has filled, and the device has been closed. See Remarks section.
SPERR_AUDIO_BUFFER_UNDERFLOW | The multimedia object has not received audio data from the device quickly enough, and the device has been closed. See Remarks section.
SPERR_AUDIO_STOPPED | Multimedia device state has been set to stopped.
E_OUTOFMEMORY | Exceeded available memory
E_POINTER | At least one of pcbRead or pv are invalid or bad.
STG_E_ACCESSDENIED | Multimedia device is read-only and no bytes will be read. Error will occur when reading from an output device.
FAILED (hr) | Appropriate error message.

Remarks about Audio Buffer Overflows and Underflows

SAPI automatically stores data in a buffer before it is read from the device. Buffering the audio data ensures that applications and SAPI-compliant speech recognition engines will not lose real-time audio data.

An errant application or speech recognition engine that does not call Read often enough could frequently fill the audio buffer. To ensure that large amounts of system memory are not filled, SAPI limits the buffer size to 30 times the average bytes per second (WAVEFORMATEX->nAvgBytesPerSec)- approximately 30 seconds. If the audio buffer is filled, SAPI will automatically set the device state to SPAS_CLOSED (see SPAUDIOSTATE) and return a buffer overflow error (i.e., SPERR_AUDIO_BUFFER_OVERFLOW) when ISpMMSysAudio::Read is called.

An errant multimedia device (and/or driver) that does not return audio data quickly enough could greatly reduce the speed of a speech application or SR engine. To prevent the degradation of application or SR engine performance, SAPI requires that the multimedia device return data at least once every five seconds. If the audio is not returned before five seconds, SAPI will automatically set the device state to SPAS_CLOSED (see SPAUDIOSTATE) and return a buffer underflow error (i.e., SPERR_AUDIO_BUFFER_UNDERFLOW) when ISpMMSysAudio::Read is called.

Applications should manually reopen the audio device (see ISpAudio::SetState) to prevent losing input data that could impact the user.

For Speech Recognition engines, SAPI will automatically attempt to restart the multimedia device after the SR Engine exits the ISpSREngine::RecognizeStream call.


ISpMMSysAudio::Write

ISpMMSysAudio::Write writes data to the multimedia audio device.

The audio device should not be directly manipulated when performing speech recognition (see ISpRecognizer).

HRESULT Write(
   const void  *pv,
   ULONG        cb,
   ULONG       *pcbWritten
);

Parameters

pv
[in] Pointer to the buffer containing the data that is to be written to the audio device. A valid pointer must be provided for this parameter even when cb is zero.
cb
[in] The number of bytes of data to attempt to write to the audio device. This value may be zero.
pcbWritten
[out] Pointer to a ULONG variable where this method writes the actual number of bytes written to the audio device. If set to NULL, no byte value is passed back.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_AUDIO_STOPPED | Multimedia device has been stopped.
E_OUTOFMEMORY | Exceeded available memory.
E_POINTER | At least one of pcbWritten or pv are invalid or bad.
STG_E_ACCESSDENIED | Multimedia device is write-only and no bytes will be written. Error will occur when writing to an input device.
FAILED (hr) | Appropriate error message.


ISpMMSysAudio::GetDeviceId

ISpMMSysAudio::GetDeviceId passes back the multimedia device ID being used by the audio object.

The default device ID for SpMMSysAudio objects that are created using CoCreateInstance is the WAVE_MAPPER. For audio objects created using an object token, the ID will always be a specific wave in or wave out device ID.

HRESULT GetDeviceId(
   UINT  *puDeviceId
);

Parameters

puDeviceId
[out] Pointer receiving the device ID.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | puDeviceId is a bad pointer.

Example

The following code snippet illustrates the use of ISpMMSysAudio::GetDeviceId using CoCreateInstance.

    HRESULT hr = S_OK;

    // create the multimedia input object
    hr = cpMMSysAudio.CoCreateInstance(CLSID_SpMMAudioIn);
    // Check hr

    // get the default device id
    UINT uiDeviceId;
    hr = cpMMSysAudio->GetDeviceId(&uiDeviceId);
    // Check hr

    // uiDeviceId == WAVE_MAPPER

The following code snippet illustrates the use of ISpMMSysAudio::GetDeviceId using an ISpObjectToken

    HRESULT hr = S_OK;

    // get the current multimedia object's object token
    hr = cpMMSysAudio.QueryInterface(&cpObjectWithToken);
    // Check hr

    // Find the preferred multimedia object token
    hr = SpFindBestToken(SPCAT_AUDIOIN, L"Technology=MMSys", NULL, &cpObjectToken);
    // Check hr

    // set the current multimedia object to the preferred multimedia object token
    hr = cpObjectWithToken->SetObjectToken(cpObjectToken);
    // Check hr

    // get the device id for the object
    UINT uiDeviceId;
    hr = cpMMSysAudio->GetDeviceId(&uiDeviceId);
    // Check hr

    // uiDeviceId != WAVE_MAPPER


ISpMMSysAudio::SetDeviceId

ISpMMSysAudio::SetDeviceId sets the multimedia device ID.

This method works only on audio objects that were not created using an object token, and only when the object is in the SPAS_CLOSED state. This method should not be used in normal application development. SAPI provides tokens for all the available sound devices in a computer and these can be used to create an initialized SpMMSysAudio object. This method is available for non-standard multimedia audio devices. See the Simple Telephony sample for an example of when this method is useful.

HRESULT SetDeviceId(
   UINT   uDeviceId
);

Parameters

uDeviceId
[in] The device ID of the device to set.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_DEVICE_BUSY | Object is not in the SPAS_CLOSED state.
SPERR_ALREADY_INITIALIZED | Object was created using an object token.
E_INVALIDARG | uDeviceId is invalid. It is not set to WAVE_MAPPER or device does not exist.

Example

The following code snippet illustrates the use of ISpMMSysAudio::SetDeviceId.

    HRESULT hr = S_OK;

    // create the multimedia output object
    hr = cpMMSysAudio.CoCreateInstance(CLSID_SpMMAudioOut);
    // Check hr

    // set the output device to an alternate multimedia device (e.g. modem)
    hr = cpMMSysAudio->SetDeviceId(ALTERNATE_MM_DEVICE);
    // Check hr


ISpMMSysAudio::GetMMHandle

ISpMMSysAudio::GetMMHandle passes back a multimedia audio devicestream handle.

The audio object must not be in the SPAS_CLOSED state or this call will fail because the multimedia device will not have been opened yet. The caller must not close the passed back handle. The caller must not use the handle either after changing the state of the audio object to SPAS_CLOSED or after releasing the object.

HRESULT GetMMHandle(
   void   **pHandle
);

Parameters

pHandle
The wave in or wave out device handle.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pHandle is invalid.
SPERR_UNINITIALIZED | Audio object is in the SPAS_CLOSED state.


ISpMMSysAudio::GetLineId

ISpMMSysAudio::GetLineId retrieves the current line identifier that is associated with the multimedia device. Mixer lines are not supported for output devices.

For more information on the uses of device lines, please see the Win32 multimedia mixer API (e.g., mixerOpen, mixerGetId, mixerGetLineInfo, etc.)

For input devices, SAPI will attempt to automatically identify the microphone line on the input device. On non-English versions of Windows and on a small number of English systems, it may not be possible for SAPI to automatically detect the correct microphone line. In this case, no error will be detected or returned. In such cases, if speech input is not correctly detected, the user must set the microphone input line directly using Control Panel-->Speech properties-->Speech Recognition tab-->Audio Settings-->Properties. In particular, this may be necessary to adjust the microphone input line used on devices with multiple microphone inputs such as the SoundBlaster Live Platinum as the automatically chosen line may not be the input preferred by the user.

HRESULT GetLineId(
   UINT   *puLineId
);

Parameters

puLineId
[out] Address of the structure that receives the line identifier information.

Return values

Value | Description
S_OK | Function completed successfully.
E_NOTIMPL | Not implemented for output devices.
E_POINTER | puLineId is invalid.
SPERR_NOT_FOUND | The audio device must have been created from a token.
FAILED(hr) | Appropriate error message.


ISpMMSysAudio::SetLineId

ISpMMSysAudio::SetLineId sets the line identifier associated with the multimedia device. Mixer lines are not supported for output devices.

For more information on the uses of device lines, please see the Win32 multimedia mixer API (e.g., mixerOpen, mixerGetId, mixerGetLineInfo, etc.)

For input devices, SAPI will attempt to automatically identify the microphone line on the input device. On non-English versions of Windows and on a few English systems, it may not be possible for SAPI to automatically detect the correct microphone line. In this case, no error will be detected or returned. If speech input is not correctly detected, the user must set the microphone input line directly using Control Panel-->Speech properties-->Speech Recognition tab-->Audio Settings-->Properties. It may be necessary to adjust the microphone input line used on devices with multiple microphone inputs such as the SoundBlaster Live Platinum. In these cases, the automatically chosen line may not be the input preferred by the user.

HRESULT SetLineId(
   UINT   uLineId
);

Parameters

uLineId
[in] Value specifying the line identifier information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | The specified uLineId is not supported on the current device.
SPERR_NOT_FOUND | The audio device must have been created from a token.
E_NOTIMPL | Not implemented for output devices.
FAILED(hr) | Appropriate error message.


ISpStream

This interface provides two distinct functions:

  • The application developer can wrap an existing stream up by providing both an IStream and its format so that the underlying ISpStreamFormat can provide this data to SAPI when required. The new ISpStream object can be used as an input for SAPI wherever SAPI requires an ISpStreamFormat.
  • It allows the creation of an object suitable for SAPI usage from a file using the BindToFile method. The helper function SPBindToFile may also be used to simplify this process even further.
Implemented By
  • SpStream

Methods in Vtable Order

ISpStream Methods | Description
SetBaseStream | Sets the base address of the audio stream.
GetBaseStream | Retrieves the base address of the audio stream.
BindToFile | Binds the audio stream to the file that it identifies.
Close | Closes the audio stream.

Development Helpers

Helper Enumerations, Functions and Classes | Description
SPSTREAMFORMAT | SAPI supported stream formats
CSpStreamFormat | Class for managing SAPI supported stream formats and WAVEFORMATEX structures


ISpStream::SetBaseStream

ISpStream::SetBaseStream initializes the ISpStream object withthe format of the IStreamIStream and an object to encapsulate.

HRESULT SetBaseStream(
    IStream                *pStream,
    REFGUID                 rguidFormat,
    const WAVEFORMATEX     *pWaveFormatEx
);

Parameters

pStream
Address of the IStream containing the base stream data.
rguidFormat
The data format identifier associated with the stream.
pWaveFormatEx
Address of the WAVEFORMATEX structure that contains the wave file format information. If guidFormatId is SPDFID_WaveFormatEx, this must point to a valid WAVEFORMATEX structure. For other formats, it should be NULL.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
SPERR_ALREADY_INITIALIZED | The object has already been initialized.
FAILED (hr) | Appropriate error message.

Remarks

The helper class CSpStreamFormat and the SPSTREAMFORMAT enumeration can be used to avoid the possibility of typos or mistakes when filling in the WAVEFORMATEX structure.


ISpStream::GetBaseStream

ISpStream::GetBaseStream retrieves the encapsulated IStream object for an instance of the ISpStream object.

HRESULT GetBaseStream(
   IStream   **ppStream
);

Parameters

ppStream
Address of a pointer to the encapsulated IStream that contains an audio stream or text stream.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | The ppStream pointer is invalid or bad.
SPERR_STREAM_CLOSED | The stream is closed or unavailable.
S_FALSE | The ISpStream instance has not been initialized with an IStream.


ISpStream::BindToFile

ISpStream::BindToFile binds the input stream to the file that it identifies.

For Speech Recognition, this is typically a standard wave file. For Text to Speech, it is typically a text or XML file. ISpVoice::SpeakStream has a flag which allows the text to be rendered as XML or pure text (default is XML).

HRESULT BindToFile(
   const WCHAR          *pszFileName,
   SPFILEMODE            eMode,
   const GUID           *pguidFormatId,
   const WAVEFORMATEX	*pWaveFormatEx,
   ULONGLONG             ullEventInterest
);

Parameters

pszFileName
Address of a null-terminated string containing the file name of the file to bind the stream to.
eMode
Flag of the type SPFILEMODE to define the file opening mode. When opening an audio wave file, this must be SPFM_OPEN_READONLY or SPFM_CREATE_ALWAYS, otherwise the call will fail.
pguidFormatId
The data format identifier associated with the stream. This can be NULL and the format will be determined from the supplied wave file, if the file has the wav extension. If it does not, the file is assumed to be a text file.
pWaveFormatEx
Address of the WAVEFORMATEX structure that contains the wave file format information. If guidFormatId is SPDFID_WaveFormatEx, this must point to a valid WAVEFORMATEX structure. For other formats, it should be NULL.
ullEventInterest
Flags of type SPEVENTENUM for the format converter to watch.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | At least one of the following was encountered. pszFileName or pguidFormatId is invalid or bad; eMode exceeds SPFM_CREATE_ALWAYS; an operation could not be completed.
E_OUTOFMEMORY | Exceeded available memory.
STG_E_FILENOTFOUND | File pszFileName does not exist.
SPERR_ALREADY_INITIALIZED | The object has already been initialized.
FAILED (hr) | Appropriate error message.

Remarks

The helper class CSpStreamFormat and the SPSTREAMFORMAT enumeration can be used to avoid the possibility of typos or mistakes when filling in the WAVEFORMATEX structure.

Example

The following code snippet illustrates the use of ISpStream::BindToFile for creating a writeable wave file

    HRESULT hr = S_OK;

    // create the stream object
    hr = cpSpStream.CoCreateInstance(CLSID_SpStream);
    // Check hr

    // create a stream format helper for 22khzm 16-bit, mono wave stream
    CSpStreamFormat Fmt(SPSF_22kHz16BitMono, &hr);
    // Check hr

    // create the new stream and its corresponding file on the hard disk
    // NOTE: Specify the file format when creating the file
    hr = cpSpStream->BindToFile(WAVE_FILENAME, SPFM_CREATE_ALWAYS, &Fmt.FormatId(), Fmt.WaveFormatExPtr(), NULL);
    // Check hr

    // write some data to the stream
    hr = cpSpStream->Write(WAVE_DATA_CHUNK, SIZEOF_WAVE_DATA_CHUNK, &cbWritten);
    // Check hr

The following code snippet illustrates the use of ISpStream::BindToFile for creating a read-only wave file

    HRESULT hr = S_OK;

    // create the stream object
    hr = cpSpStream.CoCreateInstance(CLSID_SpStream);
    // Check hr

    // create a new stream, by opening a wave file from the hard disk
    // NOTE: Since an existing file is being read, SAPI will read the wave stream format automatically
    hr = cpSpStream->BindToFile(WAVE_FILENAME, SPFM_OPEN_READONLY, NULL, NULL, NULL);
    // Check hr

    // read some data from the stream
    hr = cpSpStream->Read(&bData, SIZEOF_WAVE_DATA_CHUNK, &cbWritten);
    // Check hr


ISpStream::Close

ISpStream::Close closes the audio stream. Use this to validate the close operation. Releasing the ISpStream object will automatically call this method. However, any errors encountered will not be sent as notifications to the application. Hence the application can explicitly call this method first to check that no errors occurred during the close operation.

HRESULT  Close  ( void );

Parameters

None.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_STREAM_CLOSED | The stream is closed or unavailable.
FAILED (hr) | Appropriate error message.


ISpStreamFormat

ISpStreamFormat inherits from IStream.

This is the minimum extra interface required by SAPI in addition to the IStream interface. Using this interface, SAPI can query the stream to determine the format of the stream data. Almost all SAPI functions requiring or returning a stream will require or return an ISpStreamFormat.

When to Implement

This interface should be implemented when implementing a stream from scratch. If an IStream already exists, SAPI can use it with the provided ISpStream object. In normal usage, this interface does not need to be implemented.

Implemented By
  • SpMMAudioIn
  • SpMMAudioOut
  • SpStreamFormatConverter
  • SpStream

Methods in Vtable Order

ISpStreamFormat Methods | Description
GetFormat | Passes back the cached format of the stream.

Development Helpers

Helper Enumerations, Functions and Classes | Description
SPSTREAMFORMAT | SAPI supported stream formats
CSpStreamFormat | Class for managing SAPI supported stream formats and WAVEFORMATEX structures


ISpStreamFormat::GetFormat

ISpStreamFormat::GetFormat passes back the cached format of the stream.

SAPI uses this data to determine how to handle the stream data present in the underlying IStream.

HRESULT GetFormat(
   GUID           *pguidFormatId,
   WAVEFORMATEX  **ppCoMemWaveFormatEx
);

Parameters

pguidFormatId
Address of a pointer to GUID data object that receives the format of the stream being used. This is typically either SPDFID_Text or SPDFID_WaveFormatEx.
ppCoMemWaveFormatEx
Address of a pointer to a WAVEFORMATEX data structure that receives the wave file format information. This is only applicable when the return GUID is SPDFID_WaveFormatEx. SAPI allocates the memory for the WAVEFORMATEX data structure using CoTaskMemAlloc, but it is the caller's responsibility to call CoTaskMemFree on the returned WAVEFORMATEX pointer.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Pointer is locating a memory block that is NULL or either too small or is not writeable.
SPERR_UNINITIALIZED | The object has not been properly initialized.
SPERR_STREAM_CLOSED | The stream is closed or unavailable.


ISpStreamFormatConverter

ISpStreamFormatConverter is the primary interface implemented by the SAPI audio data format converter. SAPI uses the format converter to compensate for differences between supported SR and TTS engine formats, and the I/O formats requested by the application. Typically applications and engines do not use this object directly. The format converter is a wrapper object that encapsulates the specified base stream. It performs conversion on the fly during read/write operations. The Windows ACM (Audio Compression Manager) layer performs the conversion.

Several methods are included in addition to the ISpStreamFormat interface to allow data conversion.

Implemented By
  • SpStreamFormatConverter

Remarks

SAPI utilizes the host system's installed audio codecs to perform the conversion. SAPI currently supports 1-stage and 2-stage stream conversions, but does not support 3-or-more-stage conversions.

An example of a 1-stage stream format conversion is the conversion of a PCM format to another PCM format (e.g., 8kHz 16-bit Stereo PCM [SPSF_8kHz16BitStereo] -> 44kHz 8-bit Mono [SPSF_44kHz8BitMono]). This requires only one codec (e.g., "Microsoft PCM Converter").

An example of a 2-stage stream conversion is the conversion of a compressed format to a PCM format (e.g., TrueSpeech 8kHz 1-Bit Mono [SPSF_TrueSpeech_8kHz1BitMono] -> 8kHz 8-bit Mono PCM [SPSF_8kHz8BitMono] -> 44kHz 16-bit Stereo [SPSF_44kHz16BitStereo]). This requires two codecs (e.g., "DSP Group TrueSpeech(TM) Audio" and "Microsoft PCM Converter"). Note that one of the formats must be a PCM format.

An example of an unsupported 3-stage stream conversion is the conversion of a compressed format to another compressed format (e.g., TrueSpeech 8kHz 1-Bit Mono [SPSF_TrueSpeech_8kHz1BitMono] -> 8kHz 8-bit Mono PCM [SPSF_8kHz8BitMono] -> 8kHz 8-bit Stereo PCM [SPSF_8kHz8BitStereo] -> ALaw 8kHz Stereo [SPSF_CCITT_ALaw_8kHzStereo]). This would require three codecs (e.g., "DSP Group TrueSpeech(TM) Audio", "Microsoft PCM Converter", and "Microsoft CCITT G.771 Audio"). Note that SAPI is capable of converting between two compressed non-PCM formats if a single codec can do the entire conversion.

Methods in Vtable Order

ISpStreamFormatConverter Methods | Description
ISpStreamFormat interface | Inherits from ISpStreamFormat and all those methods are accessible from an ISpStreamFormatConverter object.
SetBaseStream | Sets audio stream to be wrapped by the format converter.
GetBaseStream | Gets the base audio stream that is being wrapped.
SetFormat | Sets the conversion (output) format.
ResetSeekPosition | Resets the format converter's stream seek position to the start of the stream.
ScaleConvertedToBaseOffset | Maps an offset in the converted stream into an offset in the base stream.
ScaleBaseToConvertedOffset | Maps an offset in the base stream into an offset in the converted stream.

Development Helpers

Helper Enumerations, Functions and Classes | Description
SPSTREAMFORMAT | SAPI supported stream formats
CSpStreamFormat | Class for managing SAPI supported stream formats and WAVEFORMATEX structures


ISpStreamFormatConverter::SetBaseStream

ISpStreamFormatConverter::SetBaseStream sets an audio stream to be wrapped by the format converter. The format converter is a stream object that encapsulates the base stream and performs format conversion on the fly during read/write operations.

HRESULT SetBaseStream(
   ISpStreamFormat   *pStream,
   BOOL               fSetFormatToBaseStreamFormat,
   BOOL               fWriteToBaseStream
);

Parameters

pStream
[in] The stream to be wrapped. If NULL, the current base stream is released and any associated resources are released.
fSetFormatToBaseStreamFormat
[in] Flag specifies that the converter's stream format will be set to the same format as the base stream (set up as a pass through).

If pStream == NULL and this is set to TRUE, the format converter's stream format is reset to be undefined.

fWriteToBaseStream
[in] If TRUE, data will be written to the base stream. If FALSE, data will be read from the base stream. The format converter can only be in one I/O mode or the other at a time.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pStream was invalid.


ISpStreamFormatConverter::GetBaseStream

ISpStreamFormatConverter::GetBaseStream gets the base audio stream that is being wrapped.

HRESULT GetBaseStream(
   ISpStreamFormat  **ppStream
);

Parameters

ppStream
[out] This parameter can be NULL to simply test if there is an associated base stream.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | No base stream is present.
E_POINTER | Pointer is bad or invalid.


ISpStreamFormatConverter::SetFormat

ISpStreamFormatConverter::SetFormat sets the output format of the converter.

The ISpStreamFormat::GetFormat method returns the format of the output (converted) stream.

HRESULT SetFormat(
   REFGUID               rguidFormatIdOfConvertedStream,
   const WAVEFORMATEX   *pWaveFormatExOfConvertedStream
);

Parameters

rguidFormatIdOfConvertedStream
[in] Address of the data format identifier associated with the requested output stream. Can be GUID_NULL or SPDFID_WaveFormatEx.
pWaveFormatExOfConvertedStream
[in] Address of the WAVEFORMATEX structure containing the wave file format information of the converted stream. Must be NULL with GUID_NULL. Must be a valid WAVEFORMATEX with SPDFID_WaveFormatEx.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One of the following was encountered: rguidFormatIdOfConvertedStream is neither GUID_NULL or SPDFID_WaveFormatEx; pWaveFormatExOfConvertedStream is not valid for the supplied REFGUID.

Remarks

The helper class CSpStreamFormat and the SPSTREAMFORMAT enumeration can be used to avoid the possibility of typos or mistakes when filling in the WAVEFORMATEX structure.


ISpStreamFormatConverter::ResetSeekPosition

ISpStreamFormatConverter::ResetSeekPosition resets the format converter's stream seek position to the start of the stream. This method changes the seek position of the base stream to zero.

HRESULT ResetSeekPosition( void );

Parameters

None.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Current stream base is uninitialized.


ISpStreamFormatConverter::ScaleConvertedToBaseOffset

ISpStreamFormatConverter::ScaleConvertedToBaseOffset maps a stream offset from the converted stream to the equivalent offset in the base stream.

HRESULT ScaleConvertedToBaseOffset(
   ULONGLONG    ullOffsetConvertedStream,
   ULONGLONG   *pullOffsetBaseStream
);

Parameters

ullOffsetConvertedStream
The offset of the output (converted) stream.
pullOffsetBaseStream
The equivalent offset in the base (unconverted) stream.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pullOffsetBaseStream is invalid.
SPERR_UNINITIALIZED | The base stream has not been initialized.

Remarks

When performing a mapping with a compressed format, it is possible to introduce small rounding errors, since the content of the audio is not used to perform the conversion.


ISpStreamFormatConverter::ScaleBaseToConvertedOffset

ISpStreamFormatConverter::ScaleBaseToConvertedOffset converts an offset in the base stream into the equivalent offset in the converted stream. This method is primarily used internally to map event offsets.

HRESULT ScaleBaseToConvertedOffset(
   ULONGLONG   ullOffsetBaseStream,
   ULONGLONG  *pullOffsetConvertedStream
);

Parameters

ullOffsetBaseStream
The current offset in the base (unconverted) stream.
pullOffsetConvertedStream
The new offset in the output (converted) stream.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pullOffsetConvertedStream is bad or invalid.
SPERR_UNINITIALIZED | The base stream is not initialized.
E_INVALIDARG | ullOffsetBaseStream is less than the initial seek position of the current steam. *pullOffsetConvertedStream is set to 0xFFFFFFFFFFFFFFFF.

Remarks

When performing a mapping with a compressed format, it is possible to introduce small rounding errors, since the content of the audio is not used to perform the conversion.


ISpTranscript

A transcript is a text string associated with a piece of audio data. The SAPI SpStream object supports the ISpTranscript interface for wav audio files.

Associated Class IDs

The following class IDs (CLSID) may be used with this interface. A complete CLSID listing for all interfaces is in the Class IDs section.

    CLSID_SpStream

Methods in Vtable Order

ISpTranscript Methods | Description
GetTranscript | Gets the current transcript.
AppendTranscript | Adds the current text to the transcript.

Development Helpers

Helper Enumerations, Functions and Classes | Description
CSpDynamicString | Class for managing dynamically sized unicode strings


ISpTranscript::GetTranscript

ISpTranscript::GetTranscript gets the current transcript. The string returned will be allocated by CoTaskMemAlloc. Applications implementing this method must call CoTaskMemFree() to free memory associated with this string.

HRESULT GetTranscript(
   WCHAR   **ppszTranscript
);

Parameters

ppszTranscript
[out, string] A pointer to the null-terminated transcription string.

Return values

Value | Description
S_OK | Function completed successfully. ppszTranscript contains a CoTaskMemAllocated string.
E_OUTOFMEMORY | Exceeded available memory.
SPERR_UNINITIALIZED | Object has not been initialized.
E_POINTER | ppszTranscript is bad or invalid.
S_FALSE | No transcript is present and ppszTranscript will be NULL.
FAILED (hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpTranscript::GetTranscript.

    HRESULT hr = S_OK;

    // Bind a stream to an existing wavefile
    hr = SPBindToFile( FILENAME, SPFM_READ_ONLY, &cpStream);
    // Check hr

    hr = cpStream.QueryInterface(&cpTranscript);
    // Check hr

    PWCHAR pwszTranscript;
    hr = cpTranscript->GetTranscript(&pwszTranscript);
    // Check hr


ISpTranscript::AppendTranscript

ISpTranscript::AppendTranscript adds the current text to the transcript.

HRESULT AppendTranscript(
   const WCHAR   *pszTranscript
);

Parameters

pszTranscript
[in, string] The text of the transcript. If pszTranscript is NULL, the current transcript is deleted. Otherwise, the text is appended to the current transcript.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszTranscript is bad or invalid.
E_OUTOFMEMORY | Exceeded available memory.
FAILED (hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpTranscript::AppendTranscript.

    HRESULT hr = S_OK;

    // Bind a stream to an existing wavefile
    hr = SPBindToFile( FILENAME, SPFM_CREATE_ALWAYS, &cpStream);
    // Check hr

    hr = cpStream.QueryInterface(&cpTranscript);
    // Check hr

    hr = cpTranscript->AppendTranscript(L"this is a test");
    // Check hr


Eventing interfaces

This section provides SAPI 5.0 event information.

  • ISpNotifySource
  • ISpNotifySink
  • ISpNotifyTranslator
  • ISpEventSink
  • ISpEventSource
  • ISpNotifyCallback


ISpNotifySource

In both speech synthesis and speech recognition, applications receive notifications when words have been spoken or when phrases have been recognized. SAPI components that generate notifications implement an ISpNotifySource.

The ISpNotifySource and ISpNotifySink interfaces alone only provide a mechanism for a notification but no information on the events that caused the notification. With an ISpEventSource object, an application can retrieve information about the events that caused the notification.

Applications will not typically use the free-threaded ISpNotifySink mechanism for receiving SAPI event notifications. They will use one of the simplified methods of either a window message, callback or Win32 event.

Note that both variations of callbacks as well as the window message notification require a window message pump to run on the thread that initialized the notification source. Callback will only be called as the result of window message processing, and will always be called on the same thread that initialized the notify source.

Implemented By
  • SpRecoContext
  • SpSharedRecoContext
  • SpVoice
  • SpMMAudioIn
  • SpMMAudioOut
  • SpRecPlayAudio
  • SpStreamFormatConverter

Methods in Vtable Order

ISpNotifySource Methods | Description
SetNotifySink | Sets up the instance to make free-threaded calls through ISpNotifySink::Notify. This method can also be used to unregister an existing notification.
SetNotifyWindowMessage | Sets a window handle to receive notifications as window messages.
SetNotifyCallbackFunction | Sets a callback function to receive notifications.
SetNotifyCallbackInterface | Sets an object derived from ISpTask to receive notifications.
SetNotifyWin32Event | Sets up a Win32 event object to be used by this instance for notifications.
WaitForNotifyEvent | A blocking call which waits for a notification.
GetNotifyEventHandle | Retrieves Win32 event handle associated with this notify source.


ISpNotifySource::SetNotifySink

ISpNotifySource::SetNotifySink sets up the instance to make free-threaded notification calls through ISpNotifySink::Notify.

If pNotifySink is NULL, any notification mechanism currently associated with this notify source is removed.

Because free-threaded notifications can occur on any thread, at any point during execution, they are extremely prone to deadlocks and re-entrance problems. See the documentation for ISpNotifySink for more details. Most applications will find using one of the other notification mechanisms much easier to use.

HRESULT SetNotifySink(
   ISpNotifySink  *pNotifySink
);

Parameters

pNotifySink
[in] Pointer to the notification interface. If pNotifySink is NULL, any current notification mechanism (notify sink, window message, callback, or Win32 event) is removed.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Interface pointer is invalid.
FAILED (hr) | Appropriate error message.


ISpNotifySource::SetNotifyWindowMessage

ISpNotifySource::SetNotifyWindowMessage sets up the instance to send window messages to a specified window.

HRESULT SetNotifyWindowMessage(
   HWND     hWnd,
   UINT     Msg,
   WPARAM   wParam,
   LPARAM   lParam
);

Parameters

hWnd
[in] Handle to the window whose message handler function will receive SAPI notifications.
Msg
[in] Message number which will be passed into the message handler function of the window hWnd.
wParam
[in] wParam that will be passed into the message handler function of the window hWnd.
lParam
[in] lParam that will be passed into the message handler function of the window hWnd.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | hWnd is an invalid window handle.
FAILED (hr) | Appropriate error message.


ISpNotifySource::SetNotifyCallbackFunction

ISpNotifySource::SetNotifyCallbackFunction sets up this instance to send notifications using a standard C-style callback function. It is the responsibility of the client code to control the lifetime of a notification. To remove an installed notify callback, call ISpEventSource::SetNotifySink (NULL). The final release of an object that supports ISpEventSource will automatically remove an installed notify callback.

The SAPI implementation uses a hidden window to call back the client on the same thread that was used to initialize the event source. Notification callbacks are the result of processing a window message. When this notification mechanism is used:

  1. The SPNOTIFYCALLBACK method will always be called on the thread that initialized the event source or notify translator object.
  2. The thread must have a window message pump.

The SPNOTIFYCALLBACK function is declared as follows:

typedef void __stdcall SPNOTIFYCALLBACK(WPARAM wParam, LPARAM lParam);

HRESULT SetNotifyCallbackFunction(
   SPNOTIFYCALLBACK   *pfnCallback,
   WPARAM              wParam,
   LPARAM              lParam
);

Parameters

pfnCallback
[in] The notification callback function to be used.
wParam
[in] Constant WPARAM value that will be passed to the pfnCallback function when it is called.
lParam
[in] Constant LPARAM value that will be passed to the pfnCallback function when it is called.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Callback function is invalid.
FAILED (hr) | Appropriate error message.


ISpNotifySource::SetNotifyCallbackInterface

ISpNotifySource::SetNotifyCallbackInterface sets up this instance to call the virtual method ISpNotifyCallback::NotifyCallback for notifications.

The application will be called back on the same thread that calls this method. The callback will be called as a result of window message processing, so the thread must have a message pump. For more details, see the documentation for ISpNotifyCallback.

HRESULT SetNotifyCallbackInterface(
   ISpNotifyCallback   *pSpCallback,
   WPARAM               wParam,
   LPARAM               lParam
);

Parameters

pSpCallback
[in] A pointer to an application-defined implementation of the ISpNotifyCallback interface.
wParam
[in] Constant WPARAM value that will be passed to the ISpNotifyCallback::NotifyCallback method when it is called.
lParam
[in] Constant LPARAM value that will be passed to the ISpNotifyCallback::NotifyCallback method when it is called.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pSpCallback is invalid.
FAILED (hr) | Appropriate error message.


ISpNotifySource::SetNotifyWin32Event

ISpNotifySource::SetNotifyWin32Event sets up a Win32 event object to be used by this instance for notifications. The event handle can be retrieved through the GetNotifyEventHandle method.

For an explanation of Win32 event objects, see the Win32 Platform SDK documentation. Once an event object has been initialized for this instance, use either the WaitForNotifyEvent method or use the GetNotfyEventHandle method and wait using one of the various Win32 synchronization functions. Note that Win32 event objects and SAPI events are different objects.

HRESULT   SetNotifyWin32Event  ( void );

Parameters

None

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


ISpNotifySource::WaitForNotifyEvent

ISpNotifySource::WaitForNotifyEvent is a blocking call that waits on a Win32 event handle for a SAPI notification.

A blocking call returns when a SAPI notification has fired, a timeout has passed or the initialized Win32 event object has signaled. Calling this method will automatically initialize the event source to use an event handle if no other notification mechanism has been initialized.

HRESULT WaitForNotifyEvent(
   DWORD   dwMilliseconds
);

Parameters

dwMilliseconds
[in] Number of milliseconds for the timeout on a blocking call. If set to INFINITE, there is no timeout.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The operation timed-out.
SPERR_ALREADY_INITIALIZED | This event source has been initialized to use a notification mechanism other than a Win32 event. It was unable to re-initialize the notification.
FAILED (hr) | Appropriate error message.


ISpNotifySource::GetNotifyEventHandle

ISpNotifySource::GetNotifyEventHandle retrieves the Win32 event object handle. This event can be used in any of the Win32 WaitForxxx methods. Do not close the returned handle, as it is owned by the event source object. Calling this method will automatically initialize the event source to use an event handle if no other notification mechanism has been initialized.

HANDLE   GetNotifyEventHandle  ( void );

Parameters

None

Return values

Value | Description
Win32 event handle | Initialized by SetNotifyWin32Event on this instance.
INVALID_HANDLE_VALUE | Interface not initialized.


ISpNotifySink

In both speech synthesis and speech recognition, applications receive notifications when words have been spoken or when phrases have been recognized. SAPI components that generate notifications implement an ISpNotifySource.

The ISpNotifySource and ISpNotifySink interfaces alone only provide a mechanism for a notification but no information on the events that caused the notification. With an ISpEventSource object, an application can retrieve information about the events that caused the notification. An ISpEventSource also provides the mechanism to filter and queue events. By default, an application (really an ISpNotifySink) receives no notifications from ISpEventSource until SetInterests has been called to specify on which events to notify or queue.

When an application is notified of an event that is not queued, an application will take measures based on which event sink is receiving the notification. From context an application might know exactly what it needs to do, or it may need to interact with the components which sent the notifications. If an application is notified of an event that is queued, the application will call ISpEventSource::GetEvents to retrieve the actual events that caused a notification.

When to Implement

Implement this interface only if the application can take advantage of the slightly reduced latency of a free-threaded notification. Most applications should use one of the simplified notification mechanisms in ISpEventSource instead of implementing this interface directly. Free-threaded notifications are difficult to implement without deadlocking the system. Furthermore, other notification mechanisms require less code on the part of the developer. The application needs to handle free-threaded notifications and implement the ISpNotifySink interface when an ISpNotifySink object is to be notified.

Because free-threaded notifications can occur on any thread, at any point during execution, they are extremely prone to deadlocks and re-entrance problems. The only interface that can be called on an event source object is ISpEventSource::GetEvents. If the ISpNotifySink interface is implemented directly, the code should only use some mechanism to signal another thread to process the notification (for example, a Win32 event or an I/O completion port), and, if needed, call ISpEventSource::GetEvents. Do not call any other methods.

Methods in Vtable Order

ISpNotifySink Methods | Description
Notify | Called by an ISpNotifySource object to notify the sink when the state of the notify source has changed.


ISpNotifySink::Notify

ISpNotifySink::Notify is called by ISpNotifySource whenever the state of the notification source has changed.

The only interface that can be called on an event source object is ISpEventSource::GetEvents. The Notify() method should only use some mechanism to signal another thread to process the notification (for example, a Win32 event or an I/O completion port), and if needed, call ISpEventSource::GetEvents. Do not call any other methods.

HRESULT   Notify  ( void );

Parameters

None

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Object failed notification.


ISpNotifyTranslator

Once the SpNotifyTranslator object has been initialized, the ISpNotifySource::SetNotifySink method can be called, passing the translator object interface as the parameter. The translator will then convert the call to the ISpNotifySink::Notify method into the appropriate notification.

Because ISpNotifySource supports most of the functionality of this interface, application writers will not normally use either this interface or the SpNotifyTranslator object. In fact, SAPI uses this object to implement the various methods of ISpNotifySource. The method InitWin32Event supports the ability to initialize the translator with a specific event object, and so could be used for that purpose.

Implemented By
  • SpNotifyTranslator

Methods in Vtable Order

ISpNotifyTranslator Methods | Description
ISpNotifySink interface | Inherits from ISpNotifySink and those methods are accessible from an ISpNotifyTranslator object.
InitWindowMessage | Sets up the instance to send window messages to a specified window.
InitCallback | Sets up this instance to send notifications using a standard C-style callback function.
InitSpNotifyCallback | Enables an object derived from ISpNotifyCallback to receive notifications.
InitWin32Event | Sets up a Win32 event object to be used by this instance.
Wait | A blocking call which returns when a SAPI notification has fired and the associated Win32 event object has been signaled or a timeout has passed.
GetEventHandle | Retrieves the Win32 event handle associated with the translator.


ISpNotifyTranslator::InitWindowMessage

ISpNotifyTranslator::InitWindowMessage sets up the instance to send window messages to a specified window.

HRESULT InitWindowMessage(
   HWND      hWnd,
   UINT      Msg,
   WPARAM    wParam,
   LPARAM    lParam
);

Parameters

hWnd
[in] Handle to the window whose message handler function will receive SAPI notifications.
Msg
[in] Message number which will be passed into the message handler function of the window hWnd.
wParam
[in] wParam that will be passed into the message handler function of the window hWnd.
lParam
[in] lParam that will be passed into the message handler function of the window hWnd

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_ALREADY_INITIALIZED | SpTranslator object already initialized.
E_INVALIDARG | hWnd is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpNotifyTranslator::InitCallback

ISpNotifyTranslator::InitCallback sets up this instance to send notifications using a standard C-style callback function.

The translator implementation uses a hidden window to call back the client on the same thread that was used to initialize the event source. Notify callbacks are the result of processing a window message. When this notification mechanism is used:

  1. The SPNOTIFYCALLBACK method will always be called on the thread that initialized the event source or notify translator object.
  2. The thread must have a window message pump.

The SPNOTIFYCALLBACK function is declared as follows:

typedef void __stdcall SPNOTIFYCALLBACK(WPARAM wParam, LPARAM lParam);

HRESULT InitCallback(
   SPNOTIFYCALLBACK   *pfnCallback,
   WPARAM              wParam,
   LPARAM              lParam
);

Parameters

pfnCallback
[in] The notification callback function to be used.
wParam
[in] Constant WPARAM value that will be passed to the pfnCallback function when it is called.
lParam
[in] Constant LPARAM value that will be passed to the pfnCallback function when it is called.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_ALREADY_INITIALIZED | SpTranslator object is already initialized.
E_INVALIDARG | pfnCallback is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpNotifyTranslator::InitSpNotifyCallback

ISpNotifyTranslator::InitSpNotifyCallback sets up this instance to call the virtual method ISpNotifyCallback::NotifyCallback for notifications.

The application will be called back on the same thread that calls this method. The callback will be called as a result of window message processing, so the thread must have a message pump. For more details, see the documentation for ISpNotifyCallback.

HRESULT InitSpNotifyCallback(
   ISpNotifyCallback   *pSpCallback,
   WPARAM               wParam,
   LPARAM               lParam
);

Parameters

pSpCallback
[in] A pointer to an application-defined implementation of the ISpNotifyCallback interface.
wParam
[in] Constant WPARAM value that will be passed to the ISpNotifyCallback::NotifyCallback method when it is called.
lParam
[in] Constant LPARAM value that will be passed to the ISpNotifyCallback::NotifyCallback method when it is called.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_ALREADY_INITIALIZED | Translator object is already initialized.
E_INVALIDARG | pSpNotifyCallback is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpNotifyTranslator::InitWin32Event

ISpNotifyTranslator::InitWin32Event sets up a Win32 event object to be used by this instance. For an explanation of Win32 event objects see the Win32 Platform SDK documentation. The translator will call the Win32 method ::SetEvent() whenever the translator's Notify() method is called.

Initialize an event object for this instance, and then use either the WaitForNotifyEvent or GetNotfyEventHandle method. Win32 event objects and SAPI events are different. This method is similar to ISpNotifySource::SetNotifyWin32Event.

HRESULT InitWin32Event(
   [in] HANDLE   hEvent,
   [in] BOOL     fCloseHandleOnRelease
);

Parameters

hEvent
Handle of an existing Win32 event object for the application to use with ISpNotifyTranslator. If this parameter is NULL, a new event will be created.
fCloseHandleOnRelease
Specifies whether the hEvent handle should be closed when the object is released. If hEvent is NULL, this parameter is ignored and the handle will always be closed upon release of the object.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_ALREADY_INITIALIZED | Interface is already initialized.
FAILED(hr) | Appropriate error message.


ISpNotifyTranslator::Wait

ISpNotifyTranslator::Wait is a blocking call which returns when a SAPI notification has fired and the associated Win32 event object has been signaled or a timeout has passed. This method is applicable only with objects using Win32 events.

HRESULT Wait(
   DWORD   dwMilliseconds
);

Parameters

dwMilliseconds
[in] Number of milliseconds for the timeout on a blocking call. If set to INFINITE, there is no timeout.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The event was not set and the call was timed out.
SPERR_UNINITIALIZED | InitWin32Event did not return successfully or has not been called.


ISpNotifyTranslator::GetEventHandle

ISpNotifyTranslator::GetEventHandle returns the Win32 event object handle initialized by InitWin32Event on this ISpNotifyTranslator instance. This method is applicable only with objects using Win32 events.

The handle is not a duplicated handle and should not be closed by the caller.

HANDLE   GetEventHandle   ( void );

Parameters

None

Return values

Value | Description
handle | The handle to the event
INVALID_HANDLE_VALUE | Translator has not been properly initialized by calling InitWin32Event().


ISpEventSink

This interface allows event sources to send events directly to an event sink through a free-threaded call.

When to Implement

This interface is never used by most applications.

Associated Class IDs

The following class IDs (CLSID) may be used with this interface. A complete CLSID listing for all interfaces is in the Class IDs section.

    CLSID_SpStreamFormatConverter
    CLSID_SpStream

Methods in Vtable Order

ISpEventSink Methods | Description
AddEvents | Adds events directly to an event sink.
GetEventInterest | Passes back the event interest for the voice.


ISpEventSink::AddEvents

ISpEventSink::AddEvents adds events directly to an event sink.

HRESULT AddEvents(
   const SPEVENT    *pEventArray,
   ULONG             ulCount
);

Parameters

pEventArray
Pointer to an array of SPEVENT event structures.
ulCount
Number of event structures being passed in.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pEventArray is bad or invalid
FAILED(hr) | Appropriate error message.


ISpEventSink::GetEventInterest

ISpEventSink::GetEventInterest passes back the event interest for the voice.

HRESULT GetEventInterest(
   ULONGLONG   *pullEventInterest
);

Parameters

pullEventInterest
[out] Set of flags of type SPEVENTENUM defining the event interest.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Pointer bad or invalid.
FAILED(hr) | Appropriate error message.


ISpEventSource

The ISpEventSource inherits from the ISpNotifySource interface. The methods on ISpNotifySource allow an application to specify the mechanism by which they want to receive notifications. This interface allows the application to configure which events should trigger notifications and to retrieve queued events.

An ISpEventSource provides the mechanism to filter and queue events. By default, an application (really an ISpNotifySink) receives no notifications from the SpVoice object, until SetInterest has been called to specify on which events to notify or queue. For the SpRecoContext object, the default event interest is set to queue only recognition events.

When an application is notified of an event that is not queued, an application will take measures based on which event sink receives the notification. From context, an application might know exactly what it needs to do, or it may need to interact with the components which sent the notifications. If an application is notified of a queued event, the application will call ISpEventSource::GetEvents to retrieve the actual events that caused a notification.

Implemented By
  • SpRecoContext
  • SpSharedRecoContext
  • SpVoice
  • SpMMAudioIn
  • SpMMAudioOut
  • SpRecPlayAudio
  • SpStreamFormatConverter

Methods in Vtable Order

ISpEventSource Methods | Description
ISpNotifySource inherited methods | All methods of ISpNotifySource are accessible from this interface
SetInterest | Sets the type of events the client is interested in.
GetEvents | Retrieves and removes the queued events.
GetInfo | Retrieves information about the event queue.


ISpEventSource::SetInterest

ISpEventSource::SetInterest sets the type of events which will invoke a notification and become queued.

If SetInterest is never called, the SR engine defaults to SPEI_RECOGNITION as the only event and queued interest. A TTS engine defaults to zero for both event and queued interest. With either engine, no events will be passed through if both parameters are set to zero.

HRESULT SetInterest(
   ULONGLONG   ullEventInterest,
   ULONGLONG   ullQueuedInterest
);

Parameters

ullEventInterest
[in] Event ID flags indicating which events should invoke a notification to the event sink that this event source uses. Must be of type SPEVENTENUM.
ullQueuedInterest
[in] Event ID flags indicating which events should be queued. The event flags set here must also be set in dwEventInterest. Must be of type SPEVENTENUM.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Invalid flags passed in one or more fields.
FAILED(hr) | Appropriate error message.

Remarks

Note that the SPFEI() macro will be used to convert an event enum into the appropriate flags to pass to this method. For example, to receive the SPEI_RECOGNITION and SPEI_HYPOTHESIS events, call this function as follows:

ULONGLONG ullMyEvents = SPFEI(SPEI_RECOGNITION) | SPFEI(SPEI_HYPOTHESIS);
    hr = pEventSource->SetInterest(ullMyEvents, ullMyEvents);

Events specified in ullEventInterest must be a superset of those specified in the ullQueuedInterest. Therefore it is possible to have notifications of events but not actually queue them. This can be useful for polling the GetStatus method, especially for text to speech.


ISpEventSource::GetEvents

ISpEventSource::GetEvents retrieves and removes the queued events. Clients will want to use the helper class CSpEvent to retrieve and manipulate these events.

HRESULT GetEvents(
   ULONG     ulCount,
   SPEVENT  *pEventArray,
   ULONG    *pulFetched
);

Parameters

ulCount
[in] Maximum number of events that SPEVENT structures can return.
pEventArray
[out] Pointer to array of SPEVENT structures. Each returned event is written to one of these SPEVENT structures.
pulFetched
[out] Pointer to the number of events returned. If ulCount is one, this parameter is not required.

The events are then removed from the queue. The events not returned are left for a future call to GetEvents. It is possible that by the time an application calls GetEvents, another thread has processed the events and there are no events to be returned. This may be the result of subsequent Notify calls.

Return values

Value | Description
S_OK | Function completed successfully and all requested events were returned.
S_FALSE | Success, but less than the requested amount of events were returned.
E_POINTER | pEventArray is invalid.
FAILED(hr) | Appropriate error message.


ISpEventSource::GetInfo

ISpEventSource::GetInfo retrieves information about the event queue.

HRESULT GetInfo(
  SPEVENTSOURCEINFO  *pInfo
);

Parameters

pInfo
[out] Pointer to a SPEVENTSOURCEINFO structure about the event.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pInfo is invalid.
FAILED(hr) | Appropriate error message.


ISpNotifyCallback

This is not a COM interface. This is a C++ virtual interface that can be implemented by a SAPI client application to receive notifications. Since it is not a COM interface the application does not need to implement QueryInterface, AddRef, or Release. It is the responsibility of the client code to control the lifetime of an ISpNotifyCallback-style notification. To remove an installed notify callback, call ISpEventSource::SetNotifySink( NULL ). The final release of an object that supports ISpEventSource will automatically remove an installed notify callback.

The SAPI implementation uses a hidden window to call the client back on the same thread that was used to initialize the event source. Notification callbacks are the result of processing a window message. This means that when the notification mechanism is used:

  1. The NoitifyCallback method will always be called on the thread that initialized the event source or notify translator object.
  2. The thread must have a window message pump.

Methods in Vtable Order

ISpNotifySource Methods | Description
NotifyCallback | Client implemented method is called by an object that supports ISpEventSource when an event occurs.


ISpNotifyCallback::NotifyCallback

ISpNotifyCallback::NotifyCallback is implemented by the client of the SpEventSource object. When this method is called, the wParam and lParam parameters are set to the values specified by the client when it called the ISpNotifySource::SetNotifyCallbackInterface method. The client should examine the appropriate event source object for events.

HRESULT NotifyCallback(
   WPARAM   wParam,
   LPARAM   lParam
);

Parameters

wParam
[in] wParam parameter is specified by the client when it called the ISpEventSource::SetNotifyCallback interface.
lParam
[in] lParam parameter is specified by the client when it called the ISpEventSource::SetNotifyCallback interface.

Return values

Clients should return S_OK. Other values may be used for debugging purposes, but will be ignored by SpEventSource objects.


Grammar Compiler interfaces

The following section covers:

  • Text grammar format
  • ISpGrammarBuilder


Text grammar format

The context-free grammar (CFG) format in SAPI 5.0 defines the structure of grammars and grammar rules. Extensible Markup Language (XML) using the tagging language. The CFG compiler transforms the XML tags defining the grammar elements into a binary format used by speech engines. This compiling process can be performed either before or during application runtime. Speech recognition engines use CFGs to constrain the user's words to words it will recognize.

The following section covers:

  • Text grammar format overview
  • Grammar rules
  • Designing grammar rules
  • Using grammar rules
  • Grammar Format Tags


Text grammar format overview

The Extensible Markup Language (XML) format inside a GRAMMAR XML element (block), is an "expert–only–readable" declaration of a grammar that a speech application uses to accomplish the following:

  • Improve recognition accuracy by restricting and indicating to an engine what words it should expect.
  • Improve translation of recognized speech into application actions. This is made easier by providing "semantic tags," (property name, and value associations) to words/phrases declared inside the grammar.

A GRAMMAR XML element (block) appears in a XML source code file. The XML source is compiled into a binary grammar format and is the format used by SAPI during application runtime.

The following section covers:

  • Extensible Markup Language (XML)
  • Attributes
  • Contents
  • Comments
  • How SAPI utilizes XML information
  • Frequently used definitions
  • Non–empty concatenated recognition contents

Extensible Markup Language

The textual grammar format is an application of the XML. Every XML element consists of a start tag (<SOME_TAG>) and an end tag (</SOME_TAG>) with a case-insensitive tag name and contents between these tags. The start tag and the end tag are the same if the element is empty. For example, the tag (<SOME_TAG/>). For more information on the use of XML grammars, please see the Grammar XML Schema section. Additionally, more information about XML and the XML specification is available at: http://www.w3.org/TR/REC-xml.

return to the top of this page Back to top

Attributes

Attributes of an XML element appear inside the start tag. Each attribute is in the form of a name followed by an equal sign followed by a string which must be surrounded by either single or double quotation marks. An attribute of a given name may only appear once in a start tag.

In summary, the literal string cannot contain either < or ', if the string is surrounded by single quotation marks. It may not contain ", if the string is surrounded by double quotation marks. Furthermore, use all ampersand (&) characters only in an entity reference such as &amp; and &gt;. When a literal string is parsed, the resulting replacement text will resolve all entity references such as &gt; into its corresponding text, such as >. In this specification, only the resulting replacement text needs to be defined for attribute value strings. More information about XML and the XML specification is available at: http://www.w3.org/TR/REC-xml.

return to the top of this page Back to top

Contents

The contents of an element consists of text or subelements. Formal definitions of valid contents in this specification are provided as regular and "multi-set" expressions. The pseudo-element name "Text" indicates untagged text. With these definitions, the XML specification defines the exact file syntax details.

return to the top of this page Back to top

Comments

The SAPI 5.0 XML parser treats HTML comment tags as unknown XML tag elements. The engine should provide support for comments and other unknown XML elements.

return to the top of this page Back to top

How SAPI utilizes XML information

SAPI uses XML content in the following two methods:

  1. The SAPI context-free grammar compiler, compiles the XML grammar into a binary grammar format. The compiled binary grammar is loaded into the SAPI runtime environment from a file, memory, or object (.DLL) resource.
  2. The speech recognition (SR) engine queries the runtime environment for available grammar information.

return to the top of this page Back to top

Frequently used definitions

Untagged text declaring a sequence of words that the recognition engine will recognize. Tentatively this text is only the not-necessarily-phonetic representation of words used for reading words whose pronunciation is unknown to the user (for example, for Japanese, kana, not kanji); this form will be called the spelling form. In further definitions in this section, Text will be referenced as though it were a pseudo-element.

return to the top of this page Back to top

Non–empty concatenated recognition contents

The contents of a number of XML elements in this specification such as, the P element, contain a sequence of grammar constructs which are concatenated together (one grammar construct after another). These grammar elements must be recognized in order for the contents defined to be recognized.

The contents must be one of the following (and not both):

Text and any number of L, P, O, or RULEREF elements in any order with at least one L, P, or RULEREF.

For more information on the use of XML grammars, please see the Grammar XML Schema section.

return to the top of this page Back to top


Grammar rules

Grammar rules are elements that SAPI 5.0 compliant recognition engines use to restrict the possible word or sentence choices during the speech recognition process. Recognition engines use grammar rules to control the elements of sentence construction by utilizing the predetermined list of recognized word or phrase choices. This list of recognized words or phrase choices contained in the grammar rules forms the basis of the recognition engine vocabulary.

The phrase or sentence uses each grammar rule element to determine the recognition path.

For example, examine the phrase describing travel plans, "I would like to travel from Seattle to New York," and note that there are elements that determine the resulting information. In this example, a person is planning to fly to New York from Seattle. This is a very simple illustration of what could be a very complex problem. Determining the same travel plans without limiting the method, direction, and travel destination would result in an infinite number of travel options.

The resulting information can be determined by restricting the available choices for a given sentence. Through this method, the resulting information can be composed only from certain available choices, thus eliminating the possibility of an infinite number of travel plan combinations.

 I would like to travel from Seattle to New York.
                  |      |     |     |     |
               [Method]  |     |     |     |
               /    \    |     |     |     |
            Fly   Drive  |     |     |     |
                         |     |     |     |
                  [Direction]  |     |     |
                    /     \    |     |     |
                  From    To   |     |     |
                               |     |     |
                            [City]   |     |
                           Seattle   |     |
                          New York   |     |
                       Los Angeles   |     |
                       Albuquerque   |     |
                                     |     |
                              [Direction]  |
                                 /    \    |
                                To   From  |
                                           |
                                        [City]
                                        Seattle
                                        New York
                                        Los Angeles
                                        Albuquerque

The elements of interest in the example phrase are as follows:

  • Method of travel (fly or drive)
  • Travel direction (from or to)
  • The city of origin for the travel plan (from)
  • The city of destination for the travel plan (to)

I would like to travel from Seattle to New York.

Grammar rules become concatenated phrase elements. These phrase elements are limited to the defined set of grammars. Control can be significantly improved over the resulting information by restricting the input choice to a limited set of possibilities. Otherwise, obtaining the travel plan information from the same sample phrase, "I would like to travel from Seattle to New York," would be considerably more ambiguous.

The complexity of parsing the same sentence increases exponentially without using a defined set of choices. Imagine the possible number of combination in a sentence that is not restricted to a finite list of combinations. For example, examine the possible choice combinations by moving the mouse over the following sentence.

To display the available choice selections in the example phrase, move the mouse over the underlined text below:

"I want to—(unknown travel method)—(unknown travel direction)—(unknown city)—(unknown travel direction) (unknown city)." The amount of predictable information is significantly reduced without the ability to constrain the available choices within a sentence.

Grammar rules apply to the following:

TOPLEVEL
A grammar tagged as TOPLEVEL can be in an active or inactive state. The rules that import a grammar can override the activation state of a rule. This conditional state can be configured dynamically at runtime. If an inactive grammar is included in another grammar or grammar rule, ignore the inactive state. When a rule is activated, a speech recognition engine will accept only speech satisfying at least one of the active rules contained in the loaded grammar.
Non-terminal
A grammar node is considered to be non-terminal if it is the beginning of a choice selection or a group of choice selections. For example, the grammar node Dog is non-terminal when the subsequent choice selections are types of dogs. This type of grammar is defined as non-terminal because of its choice selections.
Terminal
A grammar node is considered to be terminal if it's the only word in the recognized vocabulary which can be spoken. Using the Dog example above, terminal grammar nodes are the type of dogs.
                      --------------------+
               Animal                     +--- Non-terminal node
                 |    --------------------+
                 |
              /--+--\          -----------+
  Cat--------/       \------Dog           +--- Non-terminal node
   |                       |   -----------+
   |                       |
   |                       |   -----------+
   +-- Burmese             +-- Airedale   |
   +-- Himalayan           +-- Poodle     +--- Terminal nodes
   +-- Persian             +-- Schnauzer  |
   +-- Siamese             +-- Whippet    |
                               -----------+

The text format grammar XML tags follow block scope methods that are similar to HTML tags. That is, each tag has an opening tag and a corresponding closing tag. There is more information about XML syntax in the Grammar XML Schema section.

XML tag syntax | Contents
<sometag NAME="some_name" VAL="some_value"> | Start of "sometag" tag scope which includes the name and value information.
</sometag> | End of the "sometag" scope.

return to the top of this page Back to top


Designing grammar rules

Speech applications often use context-free grammars (CFG) to parse the recognizer output and in some instances, to act as the recognizer's language model. A speech recognition engine uses the CFG to constrain the words it will recognize that are contained in the user's utterance. If the CFG is augmented with semantic information (property names and property values as explained below), a SAPI component converts the recognized word string into a name/value-meaning representation. The application then uses the meaning representation to control its part of the conversation with the user.

The following section covers:

  • Semantic Properties or Tags
  • Separation of Dynamic and Static Content
  • Use Dynamic Rules for Language Flexibility
  • Retrieving Semantic Tags or Properties from Recognition Results

Semantic Properties or Tags

For example, the phrase "Please schedule a meeting with Amy Anderson," could be annotated as follows:

Phrase element             Grammar element            Contents
-------------------------------------------------------------------------
"schedule a meeting"      "request: meeting"       // attribute and value
"with"                    "participants:"          // only attribute
"Amy Anderson"            "<email alias>"          // value type

Defining the different grammar element components could result in the following:

Please schedule a meeting with Amy Anderson.
         |         |       |       |
         |         |       |       |
         |         |       |       |
     request: meeting      |       |
                           |       |
                   participants: AmyAnd

The example sentence "Please schedule a meeting with Amy Anderson," generates the following SAPI 5.0 grammar:

<RULE TOPLEVEL=ACTIVE>
        <P PROPNAME="request" VAL="meeting">schedule a meeting</P>
        <P>with</P>
        <L PROPNAME="participants">
            <P VAL="AmyAnd">Amy Anderson</P>
            <P VAL="tbremer">Ted Bremer</P>
            <P VAL="fralee">Frank Lee</P>
            <P VAL="crandall">Cynthia Randall</P>
            <P VAL="swhite">Suki White</P>
            <P VAL="kyoshida">Kim Yoshida</P>
        </L>
   </RULE>

The result of saying the example sentence "Please schedule a meeting with Amy Anderson," would be as follows:

request:meeting

participants:AmyAnd

return to the top of this page Back to top

Separation of Dynamic and Static Content

Applications should separate dynamic rule content from static rule content to make clear grammar design and to improve initial SAPI grammar compiler performance. For example, using the above grammar that uses a list of names, the application could create a separate rule (isolated in its own grammar) that contained only the names. The list of names, based on an address book or past user data, can be updated at runtime. The static grammar would then contain a rule reference (e.g. RULEREF) to the dynamic content. When the application starts up, it can quickly load the static content, without loading the SAPI grammar compiler, to prevent latency in the startup sequence. Then, the application could load the dynamic content, which requires SAPI to initialize the backend grammar compiler.

return to the top of this page Back to top

Use Dynamic Rules for Language Flexibility

Suppose an application needs to support a phrase such as "send new e-mail to NAME." The phrase "send new e-mail to" is static, and known by the application at design time, - well before runtime. The application could use the following static XML grammar to support these phrases.

<GRAMMAR LANGID="409"><!-- american english grammar -->
   <RULE NAME="EMAIL" TOPLEVEL="INACTIVE"><!-- inactive by default, to prevent premature recognitions -->
      <PHRASE>send new email to</P>
      <RULEREF NAME="ADDRESS_BOOK" PROPNAME="NAME"/><!-- add TRACK_PROP semantic property tag for easy info retrieval -->
   </RULE>
   <RULE NAME="ADDRESS_BOOK" DYNAMIC="TRUE">
      <PHRASE>placeholder</PHRASE><!-- we'll stick placeholder text here that we'll replace immediately at runtime -->
   </RULE>
</GRAMMAR>

The source code to manipulate the dynamic rule, "ADDRESS_BOOK" follows:

     HRESULT hr = S_OK;

    // create a new grammar object
    hr = cpRecoContext->CreateGrammar(GRAM_ID, &cpRecoGrammar);
    // Check hr

    // deactivate the grammar to prevent premature recognitions to an "under-construction" grammar
    hr = cpRecoGrammar->SetGrammarState(SPGS_DISABLED);
    // Check hr

    // load the email grammar dynamically, so changes can be made at runtime
    hr = cpRecoGrammar->LoadCmdFromFile(L"email.xml", SPLO_DYNAMIC);
    // Check hr

    SPSTATEHANDLE hRule;

    // first retrieve the dynamic rule ADDRESS_BOOK
    hr = cpRecoGrammar->GetRule(L"ADDRESS_BOOK", NULL, SPRAF_Dynamic, FALSE, &hRule);
    // Check hr

    // clear the placeholder text, and everything else in the dynamic ADDRESS_BOOK rule
    hr = cpRecoGrammar->ClearRule(hRule);
    // Check hr

    // add the real address book (e.g. "Frank Lee", "self", "SAPI beta", etc.).
    // Note that ISpRecoGrammar inherits from ISpGrammarBuilder,
    // so application gets the grammar compiler and ::AddWordTransition for free!

    hr = cpRecoGrammar->AddWordTransition(hRule, NULL, L"Frank Lee", NULL, SPWT_LEXICAL, 1, NULL);
    // Check hr
    hr = cpRecoGrammar->AddWordTransition(hRule, NULL, L"self", NULL, SPWT_LEXICAL, 1, NULL);
    // Check hr
    hr = cpRecoGrammar->AddWordTransition(hRule, NULL, L"SAPI beta", NULL, SPWT_LEXICAL, 1, NULL);
    // Check hr
    // ... add rest of address book

    // commit the grammar changes, which updates the grammar inside SAPI,
    //    and notifies the SR Engine about the rule change (i.e. "ADDRESS_BOOK"
    hr = cpRecoGrammar->Commit(NULL);
    // Check hr

    // activate the grammar since "construction" is finished,
    //    and ready for receiving recognitions
    hr = cpRecoGrammar->SetGrammarState(SPGS_ENABLED);
    // Check hr

return to the top of this page Back to top

Retrieving Semantic Tags or Properties from Recognition Results

Note that the XML grammar used a semantic property tag (i.e., NAME) in the static grammar. The property will enable the application to retrieve the dynamic phrase very easily at runtime. Whenever recognition is received with rule name, "E-MAIL," search the property tree (see SPPHRASE.pProperties) for the property named "NAME." Then call ISpRecoResult::GetPhrase with (SPPHRASEPROPERTY)pNameProp.ulFirstElement and (SPPHRASEPROPERTY)pNameProp.ulFirstElement, and the application can retrieve the exact text that the user spoke into the dynamic rule (e.g., user says "send new e-mail to Frank Lee," and you retrieve "Frank Lee," user says "send new e-mail to self," and you retrieve "self," etc.).

    // activate the email rule to begin receiving recognitions
    hr = cpRecoGrammar->SetRuleState(L"EMAIL", NULL, SPRS_ACTIVE);
    // Check hr

    PWCHAR pwszEmailName = NULL;

    // default event interest is recognition, so wait for recognition event
    // NOTE: this could be placed in a loop to process multiple recognitions
    hr = cpRecoContext->WaitForNotifyEvent(MY_REASONABLE_TIMEOUT);
    // Check hr

    // event notification fired
    if (S_OK == hr) {

     CSpEvent spEvent;
         // if event retrieved and it is a recognition
    if (S_OK == spEvent.GetFrom(cpRecoContext) && SPEI_RECOGNITION == spEvent.eEventId) {

             // get the recognition result
        CComPtr<ISpRecoResult> cpRecoResult = spEvent.RecoResult();

             if (cpRecoResult) {
                 SPPHRASE* pPhrase = NULL;

                 // get the phrase object from the recognition result
                 hr = cpRecoResult->GetPhrase(&pPhrase);
                 if (SUCCEEDED(hr) && pPhrase) {

                     // if "EMAIL" rule was recognized ...
                     if (0 == wcscmp(L"EMAIL", pPhrase->Rule.pszName) {

                         // ... ensure that first property is "NAME"
                         if (0 == wcscmp(L"NAME", pPhrase->pProperties->pszName)) {

                             // store the user's spoken "send-to" name
                             //    in a variable for later processing
                             hr = pPhrase->GetText(pPhrase->pProperties->ulFirstElement,
                                                   pPhrase->pProperties->ulCountOfElements,
                                                   FALSE,
                                                   &pwszEmailName,
                                                   NULL);
                             // Check hr

                          }
                     }
                     // free the phrase object
                     if (pPhrase) ::CoTaskMemFree(pPhrase);

                 }
             }
    }
   }

return to the top of this page Back to top


Using grammar rules

Grammar rules define sentence contents and phrase elements. Each grammar and grammar element determines the recognition engine's ability to effectively construct phrase elements. Phrases and sub–expressions are commonly represented by a separate rule and combined into larger phrases and sentences with higher level rules. For more information, see the Grammar rules section.

The following example illustrates how to implement a grammar for a game of solitaire.

<GRAMMAR LANGID="409">
    <DEFINE>
        <ID NAME="FROM" VAL="1"/>
        <ID NAME="TO" VAL="2"/>
        <ID NAME="SUIT" VAL="3"/>
        <ID NAME="COLOR" VAL="4"/>
        <ID NAME="RANK" VAL="5"/>
        <ID NAME="ColorRed" VAL="11101"/>
        <ID NAME="ColorBlack" VAL="10011"/>
    </DEFINE>
   <RULE NAME="newgame" TOPLEVEL="ACTIVE">
      <P>new +game</P><O>-please</O>
   </RULE>
   <RULE NAME="playcard" TOPLEVEL="ACTIVE" EXPORT="1">
      <O>please</O>
      <P>play the</P>
      <O>...</O>
      <RULEREF REF="card"/>
      <O>please</O>
   </RULE>
   <RULE NAME="movecard" TOPLEVEL="ACTIVE">
      <O>please</O>
      <P>
         <L>
            <P>move</P>
            <P>put</P>
         </L>
         <P>the</P>
      </P>
      <RULEREF PROPNAME="from" PROPID="FROM" NAME="card"/>
      <O>
         <L>
            <P>on</P>
            <P>to</P>
         </L>
         <P>the</P>
         <RULEREF PROPNAME="to" PROPID="TO" NAME="card"/>
      </O>
      <O>please</O>
   </RULE>
   <RULE NAME="card">
      <L>
         <P>
            <L PROPNAME="color" PROPID="COLOR">
               <P VAL="ColorRed">red</P>
               <P VAL="ColorBlack">black</P>
            </L>
            <RULEREF NAME="rank"/>
         </P>
         <P>
            <RULEREF NAME="rank"/>
            <O>
               <P>of</P>
               <L PROPNAME="suit" PROPID="SUIT">
                  <P VAL="0">clubs</P>
                  <P VAL="1">hearts</P>
                  <P VAL="2">diamonds</P>
                  <P VAL="3">spades</P>
               </L>
            </O>
         </P>
         <L PROPNAME="suit" PROPID="SUIT">
            <P VAL="0">club</P>
            <P VAL="1">heart</P>
            <P VAL="2">diamond</P>
            <P VAL="3">spade</P>
         </L>
      </L>
   </RULE>
   <RULE NAME="rank">
      <L PROPNAME="rank" PROPID="RANK">
         <P VAL="1">ace</P>
         <P VAL="2">two</P>
         <P VAL="3">three</P>
         <P VAL="4">four</P>
         <P VAL="5">five</P>
         <P VAL="6">six</P>
         <P VAL="7">seven</P>
         <P VAL="8">eight</P>
         <P VAL="9">nine</P>
         <P VAL="10">ten</P>
         <P VAL="11">jack</P>
         <P VAL="12">queen</P>
         <P VAL="13">king</P>
         <P VAL="12">lady</P>
         <P VAL="13">emperor</P>
      </L>
   </RULE>
</GRAMMAR>

return to the top of this page Back to top


Using grammar format tags

 The Microsoft.com Speech website

Grammar Format Tags

The following Grammar Format tags are used to modify words within a rule or used to modify the rule itself. These tags allow greater flexibility of a rule by accepting a wide range of words for a given position.

The following tags are available:
    Optional word: ?
    Wildcard: ...
    Dictation: *
    Confidence increase: +
    Confidence decrease: -

Tags

Optional word: ?

The question before a word marks that word as optional. The word may or may not be used in the position and have no effect on possible rule activation. Using optional words allows for a more natural speaking manner. For example,

<P>?Please play the card<P>
This rule permits the user not to say "please" and still have a valid rule fire. If it were not optional, the user would have to say "please" in order for the speech engine to recognize the rule.

return to the top of this page Back to top

Wildcard: ...

An ellipsis is used to accept words not critical to the rule's intent, but allows any word or words to be spoken for that position. Unlike other words in a rule that are explicitly listed, the speaker may use any word or words in this position. The speech engine will attempt to recognize the words, and if successful, will accept them as valid elements for activating the rule. However, the words are not returned in the subsequent phrase list. Regardless of the number of words spoken, the phrase element will contain only one representation for all the words. Using wildcards allows for a more natural speaking manner and is meant for words that are unimportant to the intent of the rule. Wildcards extend the optional (question mark tag) by not requiring an explicit list all the words. For example,

<P>I would like a hamburger<P>
<P>... hamburger<P>

Both phrases attempt the same action, that of ordering a hamburger. In the first instance, the user would have to use the exact syntax stated. Even optional words would have to be listed and the rule could be cumbersome. The second instance allows the user to say virtually anything preceding the word "hamburger."

However, listing other words is not redundant to using just the wildcard. Explicitly listing alternative phrases increases the confidence of the statement. It also increases the speech engine's recognition process because more is known about the possible spoken content. "Please get me a hamburger," is a much more common phrase, and hence is easier to recognize than a non sequitur word allowed by a wildcard. As an example, "aardvark hamburger," is a valid statement according to the rules above, but would be harder for the speech engine to recognize.

return to the top of this page Back to top

Dictation: *

The dictation asterisk allows any word or words to be spoken for that position and each word is returned by the speech engine in the phrase element list. Unlike the wildcard, the word or words are considered important to the rule. Using the dictation asterisk, a user can say any word without the engine expecting or anticipating its context. For example, the following rule allows users to speak their first name:

<P>My first name is *<P>

If additional words are needed, use the plus sign after dictation to indicate multiple words:

<P>My full name and address is *+<P>

return to the top of this page Back to top

Confidence increase: +
Confidence decrease: -

One of these two signs placed in front of words respectively increases or decreases the required confidence for a successful recognition. Increasing the required confidence means that the speech recognition engine will have to be much more certain that the word it recognizes really is the expected word. For example, if the user is responding to an important question such as "Reform hard disk?," increasing the required confidence is additional confirmation a "yes" really is "yes." To make sure, the rule would be noted as "+yes".

Likewise, the minus sign decreases the required confidence for the word. This de-emphasizes words. Although the word is required for the rule, it is not important to verify that the user actually said it.

For example, in the following rule the user makes a request. However, while the command is required, it is more important that "name" is properly recognized rather than "enter."

<P>-enter +name<P>

return to the top of this page Back to top


ISpGrammarBuilder

This interface exposes the backend of the SAPI context-free-grammar (CFG) compiler. These methods can be used to programmatically construct grammars (e.g. implement a grammar compiler).

When To Use

Applications should use the ISpGrammarBuilder interface to change dynamically loaded grammars (see ISpRecoGrammar) and to change previously compiled binary grammars (see ISpGramCompBackend).

Methods in Vtable Order

ISpGrammarBuilder Methods | Description
ResetGrammar | Clears all grammar rules (un-defines them) and resets the grammar's language to NewLanguage.
GetRule | Retrieves a grammar rule's initial state information (and defines the rule if requested).
ClearRule | Removes all of the grammar rule information except for the rule's initial state handle.
CreateNewState | Creates a new state in the same grammar rule as hState.
AddWordTransition | Adds a word or a sequence of words to the grammar.
AddRuleTransition | Adds a rule (reference) transition from one grammar rule to another.
AddResource | Adds a resource (name and string value) to the grammar rule specified in hRuleState.
Commit | Performs consistency checks of the grammar structure, creates the serialized format and either saves it to the stream provided by SetSaveObjects, or reloads it into the SR engine.
Additionally, an sample code is provided to demonstrate ISpGrammarBuilder.
  • Example application of ISpGrammarBuilder


Example application of ISpGrammarBuilder

The code example below illustrates an implementation of the ISpGrammarBuilder interface.

    // HRESULT checking code omitted

    SPSTATEHANDLE hStateTravel;
    hr = pGrammarBuilder->GetRule(L"Travel", 0, SPRAF_TopLevel | SPRAF_Active, TRUE, &hStateTravel);

    // Approach 1: list all possible phrases
    hr = pGrammarBuilder->AddWordTransition(hStateTravel, NULL, L"fly to Seattle", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateTravel, NULL, L"fly to New York", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateTravel, NULL, L"fly to Washington DC", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateTravel, NULL, L"drive to Washington DC", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateTravel, NULL, L"drive to New York", L" ", SPWT_LEXICAL, 1, NULL);
    // This is the most intuitive approach, and it does not sacrefice efficiency
    // because the grammar builder will merge shared sub-phrases when possible.

    // Approach 2: construct the directed-graph using intermediate states
    SPSTATEHANDLE hStateTravel1;
    hr = pGrammarBuilder->CreateNewState(hStateTravel, &hStateTravel1);

    hr = pGrammarBuilder->AddWordTransition(hStateTravel, hStateTravel1, L"fly to", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateTravel, hStateTravel1, L"drive to", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateTravel, hStateTravel1, L"take train to", L" ", SPWT_LEXICAL, 1, NULL);

    hr = pGrammarBuilder->AddWordTransition(hStateTravel1, NULL, L"Seattle", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateTravel1, NULL, L"New York", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateTravel1, NULL, L"Washington DC", L" ", SPWT_LEXICAL, 1, NULL);
    // This approach gives you more control of the grammar layout, and may be
    // easier to implement when you have some combinations.

    // Approach 3: using sub rules
    SPSTATEHANDLE hStateMethod;
    SPSTATEHANDLE hStateDest;
    hr = pGrammarBuilder->GetRule(L"Method", 0, 0, TRUE, &hStateMethod);
    hr = pGrammarBuilder->GetRule(L"Dest", 0, SPRAF_Dynamic, TRUE, &hStateDest); // dynamic rules can be modified after commit and reload

    SPSTATEHANDLE hStateTravel2;
    hr = pGrammarBuilder->CreateNewState(hStateTravel, &hStateTravel2);
    hr = pGrammarBuilder->AddRuleTransition(hStateTravel, hStateTravel2, hStateMethod, 1, NULL);
    hr = pGrammarBuilder->AddRuleTransition(hStateTravel2, NULL, hStateDest, 1, NULL);

    hr = pGrammarBuilder->AddWordTransition(hStateMethod, NULL, L"fly to", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateMethod, NULL, L"drive to", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateMethod, NULL, L"take train to", L" ", SPWT_LEXICAL, 1, NULL);

    hr = pGrammarBuilder->AddWordTransition(hStateDest, NULL, L"Seattle", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateDest, NULL, L"New York", L" ", SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->AddWordTransition(hStateDest, NULL, L"Washington DC", L" ", SPWT_LEXICAL, 1, NULL);
    // This approach let you structure the grammars and is useful when building large grammars

    // Must Commit before using the grammar
    hr = pGrammarBuilder->Commit(0);


ISpGrammarBuilder::ResetGrammar

ISpGrammarBuilder::ResetGrammar clears all grammar rules (un-defines them) and resets the grammar's language to NewLanguage. The state handles for this grammar are no longer valid after this point.

HRESULT ResetGrammar(
   LANGID   NewLanguage
);

Parameters

NewLanguage
[in] Language identifier associated with the grammar rule.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ResetGrammar.

    HRESULT hr = S_OK;

    // ResetGrammar when no rules
    hr = pGrammarBuilder->ResetGrammar(409);
    // Check hr

    // Set language to default user language
    hr = pGrammarBuilder->ResetGrammar(SpGetUserDefaultUILanguage());
    // Check hr

    // Set language to non-english
    hr = pGrammarBuilder->ResetGrammar(MAKELANGID(LANG_CHINESE, SUBLANG_CHINESE_SIMPLIFIED));
    // Check hr

    hr = pGrammarBuilder->ResetGrammar(MAKELANGID(LANG_JAPANESE, SUBLANG_DEFAULT));
    // Check hr


ISpGrammarBuilder::GetRule

ISpGrammarBuilder::GetRule retrieves grammar rule's initial state.

Either the rule's name or ID have to be provided (the other unused parameter can either be NULL or zero). If both a grammar rule name and identifier are provided they both have to match for this call to succeed. If the grammar rule doesn't already exist and fCreateIfNotExists is true, the grammar rule is defined, otherwise this call will return an error.

HRESULT GetRule(
   const WCHAR    *pszRuleName,
   DWORD           dwRuleId,
   DWORD           dwAttributes,
   BOOL            fCreateIfNotExist,
   SPSTATEHANDLE   *phInitialState
);

Parameters

pszRuleName
[in] Address of the null-terminated string containing the grammar rule name. If NULL, no search is made for the name.
dwRuleId
[in] Grammar rule identifier. If zero, no search is made for the rule ID.
dwAttributes
[in] Grammar rule attributes for the new rule created. Ignored if the rule already exists. Must be of type SPCFGRULEATTRIBUTES. Values may be combined to allow for multiple attributes.
fCreateIfNotExist
[in] Boolean indicating that the grammar rule is to be created if one does not currently exist. TRUE allows the creation; FALSE does not.
phInitialState
[out] The initial state of the rule. May be NULL.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_RULE_NOT_FOUND | No rule matching the specified criteria can be found and a new rule is not created.
SPERR_RULE_NAME_ID_CONFLICT | One of the name and ID matches an existing rule but the other does not match the same rule.
E_INVALIDARG | At least one parameter is invalid. Also returned when both pszRuleName and dwRuleId are NULL.
E_OUTOFMEMORY | Not enough memory to complete operation.

Example

The following code snippet illustrates the use of GetRule.

    HRESULT hr = S_OK;
    SPSTATEHANDLE hState;

//======================================================================
//  Create a rule with name and ID
    hr = pGrammarBuilder->GetRule(L"rule1", 1, SPRAF_Dynamic, TRUE, &hState);
    //Check return value

//======================================================================
//  Create a rule with name only
    hr = pGrammarBuilder->GetRule(L"rule", 0, SPRAF_Dynamic, TRUE, &hState);
    //Check return value

//======================================================================
//  Create a rule with ID only
    hr = pGrammarBuilder->GetRule(NULL, 2, SPRAF_Dynamic, TRUE, &hState);
    //Check return value

//======================================================================
//  Get an existing rule by ID
    hr = pGrammarBuilder->GetRule(L"rule1", 1, SPRAF_Dynamic, FALSE, &hState);
    //Check return value
    hr = pGrammarBuilder->GetRule(NULL, 1, SPRAF_Dynamic, FALSE, &hState);
    //Check return value

//======================================================================
//  Get an existing rule by name
    hr = pGrammarBuilder->GetRule(L"rule1", 0, SPRAF_Dynamic, FALSE, &hState);
    //Check return value

//======================================================================
//  Get rule references to other grammars
//  Compose the name of the rule as follows
//  Please note the double back-slash before the rule name.
//  OBJECT --> pszRuleName = L"SAPI5OBJECT:MyApp.ClassId\\\\RuleName"
//  URL --> pszRuleName = L"URL:http://myserver.com\\\\RuleName"

    hr = pBackend->GetRule(pszRuleName, 0 , SPRAF_Import, TRUE, phTarget)
    //Check return value

// phTarget contains a valid rule handle that can be used to reference the rule.


ISpGrammarBuilder::ClearRule

ISpGrammarBuilder::ClearRule removes all of the grammar rule structural information (states and transitions) except for the handle to the rule's initial state.

HRESULT ClearRule(
   SPSTATEHANDLE   hState
);

Parameters

hState
[in] Handle to the any of the states in the grammar rule to be cleared. Note that all but the rule's initial state handle are no longer valid.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Value specified in hState is not valid.

Example

The following code snippet illustrates the use of ClearRule.

    HRESULT hr = S_OK;

    SPSTATEHANDLE hInit;
    SPSTATEHANDLE hState;
    hr = pGrammarBuilder->GetRule(L"rule1", 1, 0, TRUE, &hInit);

    // ClearRule using hInitState
    hr = pGrammarBuilder->CreateNewState(hInit, &hState);
    hr = pGrammarBuilder->AddWordTransition(hInit, hState, L"word", NULL, SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->ClearRule(hInit);
    // Check hr

    hr = pGrammarBuilder->AddWordTransition(hInit, hState, L"word", NULL, SPWT_LEXICAL, 1, NULL);
    // E_INVALIDARG because hState in no longer valid

    // ClearRule using hState != hInit
    hr = pGrammarBuilder->CreateNewState(hInit, &hState);
    hr = pGrammarBuilder->AddWordTransition(hInit, hState, L"word", NULL, SPWT_LEXICAL, 1, NULL);
    hr = pGrammarBuilder->ClearRule(hState);
    // Check hr

    hr = pGrammarBuilder->AddWordTransition(hInit, hState, L"word", NULL, SPWT_LEXICAL, 1, NULL);
    // E_INVALIDARG because hState in no longer valid


ISpGrammarBuilder::CreateNewState

ISpGrammarBuilder::CreateNewState creates a new state in the same grammar rule as hState.

HRESULT CreateNewState(
   SPSTATEHANDLE   hState,
   SPSTATEHANDLE  *phState
);

Parameters

hState
[in] Handle to any existing state in the grammar rule.
phState
[out] Address of the state handle for a new state in the same grammar rule.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | The hState is not a valid state handle.
E_POINTER | The phState pointer is invalid.
E_OUTOFMEMORY | Exceeded available memory.

Example

The following code snippet illustrates the use of CreateNewState.

    HRESULT hr = S_OK;

    SPSTATEHANDLE hInit;
    hr = pGrammarBuilder->GetRule(L"rule1", 1, 0, TRUE, &hInit);

    // CreateNewState using the hInitState
    SPSTATEHANDLE hState;
    hr = pGrammarBuilder->CreateNewState(hInit, &hState);
    // Check hr

    // CreateNewState using hState != hInit
    SPSTATEHANDLE hState2;
    hr = pGrammarBuilder->CreateNewState(hState, &hState2);
    // Check hr


ISpGrammarBuilder::AddWordTransition

ISpGrammarBuilder::AddWordTransition adds a word or a sequence of words to the grammar.

HRESULT AddWordTransition(
   SPSTATEHANDLE          hFromState,
   SPSTATEHANDLE          hToState,
   const WCHAR           *psz,
   const WCHAR           *pszSeperators,
   SPGRAMMARWORDTYPE      eWordType,
   float                  Weight,
   const SPPROPERTYINFO  *pPropInfo
);

Parameters

hFromState
[in] Handle of the state from which the arc (or sequence of arcs in the case of multiple words) should originate.
hToState
[in] Handle of the state where the arc (or sequence of arcs) should terminate. If NULL the final arc will be to the (implicit) terminal node of this grammar rule.
psz
[in] Address of a null-terminated string containing the word or words to be added. If psz is NULL, an epsilon arc will be added.
pszSeperators
[in] Address of a null-terminated string containing the transition word separation characters.
psz points to a single word if pszSeperators is NULL, or else pszSeperators specifies the valid separator characters. This parameter may not contain a forward slash ("/") as that is used for the complex word format.
eWordType
[in] The SPGRAMMARWORDTYPE enumeration that specifies the word type. Currently, only SPWT_LEXICAL is supported.
Weight
[in] Value specifying the arc's relative weight in case there are multiple arcs originating from hFromState.
pPropInfo
[in] The SPPROPERTYINFO structure containing property name and value information that is associated with this arc or sequence of arcs.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | At least one of psz, pszSeparators, or pPropInfo is invalid or bad; eWordType is a value other than SPWT_LEXICAL; flWeight is less than 0.0; a slash ("/") is used as a separators.
SPERR_WORDFORMAT_ERROR | Invalid word format.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.


ISpGrammarBuilder::AddRuleTransition

ISpGrammarBuilder::AddRuleTransition adds a rule (reference) transition from one grammar rule to another.

HRESULT AddRuleTransition(
   SPSTATEHANDLE           hFromState,
   SPSTATEHANDLE           hToState,
   SPSTATEHANDLE           hRule,
   float                   Weight,
   const SPPROPERTYINFO   *pPropInfo
);

Parameters

hFromState
[in] Handle of the state from which the arc should originate.
hToState
[in] Handle of the state where the arc should terminate. If NULL the final arc will be to the (implicit) terminal node of this grammar rule.
hRule
[in] Handle of any state of the rule to be called with this transition. Get the hRule using the ISpGrammarBuilder::GetRule() call. To refer to a rule in another grammar, "import" that rule by calling ISpGrammarBuilder::GetRule( ... , SPRF_Import, TRUE /*fCreatIfNotExist*/, ...).

hRule can also be one of the following special transition handles:

Transition handle | Description
SPRULETRANS_WILDCARD | <WILDCARD> transition
SPRULETRANS_DICTATION | <DICTATION> single word from dictation
SPRULETRANS_TEXTBUFFER | <TEXTBUFFER> transition
Weight
[in] Value specifying the arc's relative weight in case there are multiple arcs originating from hFromState.
pPropInfo
[in] The SPPROPERTYINFO structure containing property name and value information that is associated with this arc.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | At least one parameter is invalid.
E_OUTOFMEMORY | Not enough memory to complete operation.
FAILED(hr) | Appropriate error message.


ISpGrammarBuilder::AddResource

ISpGrammarBuilder::AddResource adds a resource (name and string value) to the grammar rule specified in hRuleState. The resource can be queried by a rule interpreter using ISpCFGInterpreterSite::GetResourceValue().

HRESULT AddResource(
   SPSTATEHANDLE   hRuleState,
   const WCHAR    *pszResourceName,
   const WCHAR    *pszResourceValue
);

Parameters

hRuleState
[in] Handle of a state in the rule to which the resource is to be added.
pszResourceName
[in] Address of a null-terminated string specifying the resource name.
pszResourceValue
[in] Address of a null-terminated string specifying the resource value.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | At least one of the parameters is invalid.
SPERR_DUPLICATE_RESOURCE_NAME | The resource already exists.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of AddResource.

    HRESULT hr = S_OK;

    SPSTATEHANDLE hInit;
    hr = pGrammarBuilder->GetRule(L"rule1", 1, 0, TRUE, &hInit);

    SPSTATEHANDLE hState;
    hr = pGrammarBuilder->CreateNewState(hInit, &hState);
    // Check hr

    // AddResource using the hInitState
    hr = pGrammarBuilder->AddResource(hInit, L"ResName1", L"ResValue1");
    // Check hr

    // AddResource using hState != hInit
    hr = pGrammarBuilder->AddResource(hState, L"ResName2", L"ResValue2");
    // Check hr


ISpGrammarBuilder::Commit

ISpGrammarBuilder::Commit performs consistency checks of the grammar structure, creates the serialized format and either saves it to the stream provided by SetSaveObjects, or reloads it into the SR engine. Commit must be called before any changes to the grammar could take effect.

HRESULT Commit(
   DWORD   dwReserved
);

Parameters

dwReserved
Reserved. Must be zero.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | dwReserved is not zero.
SPERR_UNINITIALIZED | Stream not initialized. Call SetSaveObjects before Commit.
SPERR_NO_RULES | A grammar must have at least one rule and one word.
SPERR_NO_TERMINATING_RULE_PATH | At least one rule is not empty but has no terminating path (path of transitions from the initial state to a NULL state).
SPERR_CIRCULAR_RULE_REF | At least one rule has left recursion (a direct or indirect rule reference to itself originated from the initial state).
SPERR_STATE_WITH_NO_ARCS | At least one rule has a node with no outgoing transitions.
SPERR_EXPORT_DYNAMIC_RULE | Dynamic rules or rules referencing dynamic rules (directly or indirectly) cannot be exported.


Lexicon

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Lexicon interfaces

The following section covers:

  • ISpContainerLexicon
  • ISpLexicon
  • ISpPhoneConverter


ISpContainerLexicon

The container lexicon object automatically loads the user lexicon and all available application lexicons when created. This allows an application and the engine to quickly access all the additional lexicon information present on the system.

ISpContainerLexicon inherits from ISpLexicon.

Implemented By
  • SpLexicon

Methods in Vtable Order

ISpContainerLexicon Methods | Description
AddLexicon | Adds a lexicon and its type to the lexicon stack.


ISpContainerLexicon::AddLexicon

ISpContainerLexicon::AddLexicon adds a lexicon and its type to the lexicon stack. Mainly used by engines to add private lexicons to their instance of the container lexicon for consistency of lexicon access.

Note that in the case of an application creating a new application lexicon, calling AddLexicon for the new lexicon on the application's instance of the container lexicon will not update the engine's instance of the container lexicon. The correct way to update an instance of the container lexicon is to release it and recreate the object at which point it will re-enumerate all available application lexicons. To guarantee an update of the engine's instance of the container lexicon, the engine must be released and recreated at which point it will recreate its instance of the container lexicon.

HRESULT AddLexicon(
   ISpLexicon   *pAddLexicon,
   DWORD         dwFlags
);

Parameters

pAddLexicon
[in] Pointer to the lexicon to be added.
dwFlags
[in] flags of type SPLEXICONTYPE indicating the lexicon type. Should use exactly one of the types from eLEXTYPE_PRIVATE1 through eLEXTYPE_PRIVATE20.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Either dwFlag is invalid or bad, or the lexicon could not be added.
SPERR_ALREADY_INITIALIZED | The user or application lexicon was attempted to be added.
E_POINTER | pAddLexicon is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.


ISpLexicon

The ISpLexicon interface is used to access the lexicons, which contain information about words that can be recognized or spoken. For more information, please see the ISpLexicon Overview.

Implemented By
  • SpLexicon
  • SpCompressedLexicon
  • SpUncompressedLexicon

Methods in Vtable Order

ISpLexicon Methods | Description
GetPronunciations | Gets pronunciations and parts of speech for a word.
AddPronunciation | Adds pronunciation and parts of speech of a word to the user lexicon.
RemovePronunciation | Removes a word from the user lexicon.
GetGeneration | Passes back the generation ID for a word.
GetGenerationChange | Passes back a list of words which have changed between the current and a specified generation.
GetWords | Gets a list of all words in the lexicon.


ISpLexicon Overview

The ISpLexicon interface provides a uniform way for applications and engines to access the user lexicon, application lexicon, and engine private lexicons.

The following topics are covered in this section:

  • ISpLexicon information for application developers
  • ISpLexicon information for engine developers

ISpLexicon information for application developers

SpLexicon is the SAPI standard lexicon object which implements the ISpLexicon interface. It contains the user lexicon and all application lexicons registered in the system when the SpLexicon object was CoCreated. Engines can add their private lexicons through ISpContainerLexicon interface. However, if an application uses ISpContainerLexicon to add a lexicon to an instance of SpLexicon, the engine will not use the lexicon.

You can get pronunciations from both the user and application lexicons:

  hr = cpLexicon->GetPronunciation(... eLEXTYPE_USER ...);
  hr = cpLexicon->GetPronunciation(... eLEXTYPE_APP ...);
  hr = cpLexicon->GetPronunciation(... eLEXTYPE_USER | eLEXTYPE_APP ...);

You can also add or remove an application lexicon. To do so, let your COM object implement the ISpLexicon and ISpObjectWithToken interfaces, and register it as illustrated in the following example:

  hr = SpCreateNewTokenEx(SPCAT_APPLEXICONS, pszLangIndependentName, &CLSID_YourLexicon, pszLangIndependentName, langid, pszLangDependentName, &cpToken, &cpDataKeyAttribs);
  // hr = cpDataKeyAttribs->SetStringValue(name1, value1); // optional.  Can be used to find your lexicon later
  // hr = cpDataKeyAttribs->SetStringValue(name2, value2); // optional
  // ...

You can also use the SAPI-provided CLSID_SpUnCompressedLexicon to implement your application lexicon as follows (the CLSID_SpCompressedLexicon is intended for engine vendors):

  hr = SpCreateNewTokenEx(SPCAT_APPLEXICONS, pszLangIndependentName, &CLSID_SpUnCompressedLexicon, pszLangIndependentName, langid, pszLangDependentName, &cpToken, &cpDataKeyAttribs);
  // hr = cpDataKeyAttribs->SetStringValue(name1, value1); // optional.  Can be used to find your lexicon later
  // hr = cpDataKeyAttribs->SetStringValue(name2, value2); // optional
  // ...

  hr = SpCreateObjectFromToken(cpToken, &cpAppLexicon);

  cpAppLexicon->AddPronunciation(...);
  cpAppLexicon->AddPronunciation(...);
  cpAppLexicon.Release();  // the CLSID_SpUnCompressedLexicon object will be read-only after this point

To remove an application lexicon from the system:

  1. Locate your lexicon with the SpFindBestToken(..., &cpToken) function.
  2. Call cpToken->Remove (NULL) to remove the lexicon from the system.

When an application lexicon is added to the system, it is shared by all applications. Any SpLexicon object created afterward will automatically load the application lexicon. Application lexicons will override engine private lexicons.

return to the top of this page Back to top

ISpLexicon information for engine developers

ISpLexicon provides a uniform format to access user, application, and engine private lexicons. CLSID_SpLexicon implements ISpContainerLexicon, which is derived from ISpLexicon and has one more method, AddLexicon.

CComPtr<ISpContainerLexicon> cpLexicon;
  // load user lexicon and all application lexicons registered in the system automatically
     hr = cpLexicon.CoCreateInstance(CLSID_SpLexicon);

  // create your private lexicons implementing ISpLexicon, e.g. pMyLex1, pMyLex2
     hr = cpLexicon->AddLexicon(pMyLex1, eLEXTYPE_PRIVATE1);
     hr = cpLexicon->AddLexicon(pMyLex2, eLEXTYPE_PRIVATE2);
     ...

Engines can can access lexicons in the following manner:

hr = cpLexicon->GetPronunciations(... eLEXTYPE_USER ...);
hr = cpLexicon->GetPronunciations(... eLEXTYPE_APP ...);
hr = cpLexicon->GetPronunciations(... eLEXTYPE_USER | eLEXTYPE_APP | eLEXTYPE_PRIVATE1 | eLEXTYPE_PRIVATE2 ...);

GetPronunciations will return a SPWORDPRONUNCIATIONLIST structure consisting of pronunciations found in all the specified lexicons.

The lexicon pronunciation information is returned from GetPronunciations in the following order:

  1. Pronunciation from user lexicon (could have multiple pronunciations)
  2. Pronunciation from application lexicon(s) (could have multiple pronunciations)
  3. Pronunciation from the added lexicons in the same order the lexicons were added.

The expected order of priority is the same as what is returned from GetPronunciations in the above list.

The ISpLexicon interface can be used to add or remove words from the user lexicon. However, engine developers will not typically use ISpLexicon::AddPronunciation and ISpLexicon::RemovePronunciation to add or remove words from user lexicons.

When private lexicons are implemented through the ISpLexicon interface, SAPI will only call GetPronunciations and GetWords on the private lexicon. Implementing the AddPronunciation or RemovePronunciation methods to populate lexicons can modify private lexicons.

If you cache the pronunciations from user or application lexicons, you need to call GetGeneration periodically to determine if the pronunciation has been modified. For optimum efficiency, an engine should maintain synchronization with the user and application lexicons. The SAPI 5.0 compliance test can verify an engine's ability to detect changes in the user or application lexicons. For more information, please see the Compliance Tests White Paper.

When the call to GetGeneration returns a larger generation number than the previous call, the engine should call GetGenerationChange or GetWords to update the cache.

Private lexicons can be added to the ISpContainerLexicon interface, or engine developers can elect to create their own method of implementing a private lexicon. However, to ensure consistent performance among all applications, engines should always use the pronunciations from the user and application lexicons.

return to the top of this page Back to top


ISpLexicon::GetPronunciations

ISpLexicon::GetPronunciations gets pronunciations and parts of speech for a word.

HRESULT GetPronunciations(
   const WCHAR              *pszWord,
   LANGID                    LangID,
   DWORD                     dwFlags,
   SPWORDPRONUNCIATIONLIST  *pWordPronunciationList
);

Parameters

pszWord
[in] Pointer to a null-terminated text string as a search keyword. Length must be equal to less than SP_MAX_WORD_LENGTH.
LangID
[in] The language ID of the word. May be zero to indicate that the word can be of any LANGID.
dwFlags
[in] Bitwise flags of type SPLEXICONTYPE indicating that the lexicons searched for this word.
pWordPronunciationList
[in, out] Pointer to SPWORDPRONUNCIATIONLIST structure in which the pronunciations and parts of speech are returned.

Return values

Value | Description
S_OK | Function completed successfully.
SP_WORD_EXISTS_WITHOUT_PRONUNCIATION | The word exists but does not have a pronunciation.
E_POINTER | pWordPronunciationList is not a valid write pointer.
E_INVALIDARG | At least one of the parameters is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.
SPERR_UNINITIALIZED | The interface has not been initialized.
SPERR_NOT_IN_LEX | Word is not found in the lexicon.
FAILED(hr) | Appropriate error message.

Example

The following example is a code fragment demonstrating the use of GetPronunciations.

        SPWORDPRONUNCIATIONLIST spwordpronlist;
        memset(&spwordpronlist, 0, sizeof(spwordpronlist));

        hr = pISpLexicon->GetPronunciations(L"resume", 409, eLEXTYPE_USER | eLEXTYPE_APP, &spwordpronlist);
        //test for results
        if( !SUCCEEDED(hr)) return;

        for (
             SPWORDPRONUNCIATION pwordpron = pwordpronlist->pFirstWordPron;
             wordpron != NULL;
             wordpron = pwordpron->pNextWordPron
               )
        {
            DoSomethingWith(pwordpron->ePartOfSpeech, pwordpron->szPronunciation);
        }

        //free all the buffers
        CoTaskMemFree(spwordpronlist.pvBuffer);


ISpLexicon::AddPronunciation

ISpLexicon::AddPronunciation adds word pronunciations and parts of speech (POS) to the user lexicon. SAPI will not modify the word if spelling, pronunciation, and POS are the same as an existing entry in the user lexicon. A word can be added without pronunciation by passing in NULL as the pszPronunciation.

HRESULT AddPronunciation(
   const WCHAR      *pszWord,
   LANGID            LangID,
   SPPARTOFSPEECH    ePartOfSpeech,
   const SPPHONEID  *pszPronunciation
);

Parameters

pszWord
[in] The word to add.
LangID
[in] The language ID of the word. The speech user default will be used if LANGID is omitted. Length must be equal to or less than SP_MAX_WORD_LENGTH.
ePartOfSpeech
[in] The part of speech of type SPPARTOFSPEECH.
pszPronunciation
[in] Null-terminated pronunciation of the word in the NUM phone set. Multiple pronunciations may be added for a single word. The length must be equal to or less than SP_MAX_PRON_LENGTH. pszPronunciation may be NULL.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | At least one of the parameters is not valid.
SP_ALREADY_IN_LEX | The same pronunciation of the word already exists in the user lexicon.
SPERR_APPLEX_READ_ONLY | Cannot add a word to application lexicon.
SPERR_UNINITIALIZED | The interface has not been initialized.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.

Remarks

See the documentation on ISpPhoneConverter for more information on phone sets.

Example

The following is an example of AddPronunciation.

    HRESULT hr;
    CComPtr cpLexicon;
    hr = cpLexicon.CoCreateInstance(CLSID_SpLexicon);

    // 0x409 for English
    LANGID langidUS = MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US);
    CComPtr cpPhoneConv;
    SPPHONEID wszId[SP_MAX_PRON_LENGTH];
    if(SUCCEEDED(hr))
    {
        hr = SpCreatePhoneConverter(langidUS, NULL, NULL, &cpPhoneConv);
    }
    if(SUCCEEDED(hr))
    {
        hr = cpPhoneConv->PhoneToId(L"r eh d", wszId);
    }
    if(SUCCEEDED(hr))
    {
        hr = cpLexicon->AddPronunciation(L"red", langidUS, SPPS_Noun, wszId);
    }


ISpLexicon::RemovePronunciation

ISpLexicon::RemovePronunciation removes a word and all its pronunciations from a user lexicon.

HRESULT RemovePronunciation(
   const WCHAR     *pszWord,
   LANGID           LangID,
   SPPARTOFSPEECH   ePartOfSpeech,
   void            *pvReserved
);

Parameters

pszWord
[in] The word to remove.
LangID
[in] The language ID of the word. The speech user default will be used if LangID is omitted.
ePartOfSpeech
[in] The part of speech of type SPPARTOFSPEECH.
pvReserved
[in] Reserved variable. This is required to be NULL.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One of the parameters is not valid.
E_OUTOFMEMORY | Exceeded available memory.
SPERR_NOT_IN_LEX | Word is not found in the lexicon.
SPERR_APPLEX_READ_ONLY | Cannot remove a word from application lexicon.
SPERR_UNINITIALIZED | Interface not initialized.
FAILED(hr) | Appropriate error message.


ISpLexicon::GetGeneration

ISpLexicon::GetGeneration passes back the current generation ID of the user lexicon. It is used to detect the changes in the user lexicon because each change in the user lexicon (add/remove a word or install/uninstall an application lexicon) will increment the generation ID.

HRESULT GetGeneration(
   DWORD   *pdwGeneration
);

Parameters

pdwGeneration
The generation ID. This is a relative count of how many times the custom lexicons have changed.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pdwGeneration is not a valid write pointer.
SPERR_UNINITIALIZED | Interface is not initialized.
FAILED(hr) | Appropriate error message.

Remarks

ISpLexicon::GetGenerationChange and ISpLexicon::GetGeneration can be used when an application wants to determine what it has done to the lexicon over a given period of time (e.g., to back out of changes it has made due to a user cancel). To do this, the application calls ISpLexicon::GetGeneration and stores that generation ID before it starts modifying the lexicon. Later, when the application wants to see what words in the lexicon it has modified, it would call ISpLexicon::GetGenerationChanges with the stored ID. This can only be done for small changes, as SPERR_LEX_VERY_OUT_OF_SYNC will be returned once sufficient changes have been made.


ISpLexicon::GetGenerationChange

ISpLexicon::GetGenerationChange passes back a list of words which have changed between the current and a specified generation.

HRESULT GetGenerationChange(
   DWORD         dwFlags,
   DWORD        *pdwGeneration,
   SPWORDLIST   *pWordList
);

Parameters

dwFlags
[in] The lexicon category of type SPLEXICONTYPE. Currently it must be zero for the SpLexicon (container lexicon) object, and must be the correct flag for the type of SpUnCompressedLexicon object (either eLEXTYPE_USER or eLEXTYPE_APP).
pdwGeneration
[in, out] The generation ID of client when passed in. The current generation ID is passed back on successful completion of the call.
pWordList
[in, out] The buffer containing the word list and its related information.

Return values

Value | Description
S_OK | Function completed successfully.
SP_LEX_NOTHING_TO_SYNC | Nothing changed since the passed in generation ID.
SPERR_LEX_VERY_OUT_OF_SYNC | There are too many changes since the passed in generation ID, so that a change history is not available. It could also be returned after installation/uninstallation of an application lexicon. Use ISpLexicon::GetWords if GetGenerationChange returns SPERR_LEX_VERY_OUT_OF_SYNC to regenerate an entire list of words based on the current generation.
E_POINTER | pdwGeneration or pWordList is not a valid write pointer.
E_INVALIDARG | dwFlags is invalid.
SPERR_UNINITIALIZED | Interface has not been initialized.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.

Remarks

An application can determine what has been done to a lexicon over a given period of time using ISpLexicon::GetGenerationChange and ISpLexicon::GetGeneration. That is, it can back out of changes it has made due to a user cancel. To do this, before it starts modifying the lexicon, the application would call ISpLexicon::GetGeneration and store the generation ID. Later, when the application wants to see what words in the lexicon it has modified, it would call ISpLexicon::GetGenerationChanges with the stored ID. This can only be done for small changes because, past a certain point, SPERR_LEX_VERY_OUT_OF_SYNC will be returned and the change history will not be available from the original generation.

Example

The following is an example of GetGenerationChange.

    for (;;)
    {
        hr = pISpLexicon->GetGenerationChange(eLEXTYPE_USER, &m_dwGeneration, &spwordlist);

        // If, for example, a new application lexicon was added, we'll have
        // to rebuild from scratch.
        if (hr == SPERR_LEX_VERY_OUT_OF_SYNC)
        {
            Rebuild();	// Call GetWords
        }
        else if (FAILED(hr))
        {
            DealWithOtherErrors();
        }
        else
        {
            // Loop thru the changed words, and their new pronunciations
            for (SPWORD *pword = spwordlist.pFirstWord;
                pword != NULL;
                pword = pword->pNextWord)
            {
                for (SPWORDPRON pwordpron = pword->pFirstWordPron;
                    pwordpron != NULL;
                    pwordpron = pwordpron->pNextWordPron)
                {
                    if(pword->eWordType == eWORDTYPE_ADDED)
                    {
                        AddPronunciationToEngineDataStructures(
                            pword->pszWord,
                            pwordpron->ePartOfSpeech,
                            pwordpron->pszPronIPA);
                    }
                    else // pword->eWordType == eWORDTYPE_DELETED
                    {
                        RemovePronunciationFromEngineDataStructures(
                            pword->pszWord,
                            pwordpron->ePartOfSpeech,
                            pwordpron->pszPronIPA);
                    }
                }
            }
        }
    }


ISpLexicon::GetWords

ISpLexicon::GetWords gets a list of all words in the lexicon.

This method is called repeatedly with the cookie (set to zero before the first time) until S_OK is returned. S_FALSE is returned indicating additional information is left. Optionally, the passed in cookie pointer may be NULL, which specifies the application wants all of the words at once. However, the lexicon is not required to support this and may return the error SP_LEX_REQUIRES_COOKIE. The SpLexicon object (container lexicon) requires a cookie currently.

HRESULT GetWords(
   DWORD         dwFlags,
   DWORD        *pdwGeneration,
   DWORD        *pdwCookie,
   SPWORDLIST   *pWordList
);

Parameters

dwFlags
[in] Bitwise flags of type SPLEXICONTYPE from which words are to be retrieved.
pdwGeneration
[out] The current generation ID of the custom lexicon.
pdwCookie
[in, out] Cookie passed back by this call. It should subsequently be passed back in to get more data. If the call returns S_FALSE, data is remaining and GetWords should be called again. The initial value of the cookie passed in must be zero or pdwCookie be a NULL pointer. NULL pdwCookie indicates the method should return all words contained in the lexicon at once. If it cannot, SP_LEX_REQUIRES_COOKIE is returned instead.
pWordList
[in, out] The buffer containing the word list and its related information.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | Additional words are left in the lexicon(s) to process.
SPERR_LEX_REQUIRES_COOKIE | A complete list of words cannot be returned at once from the container lexicon. pdwCookie must not be NULL.
E_POINTER | At least one of pdwGeneration, pdwCookie, pWordList is not valid. Alternatively, the block of memory is too small or not writeable.
E_INVALIDARG | At least one of the parameters is not valid.
E_OUTOFMEMORY | Exceeded available memory.
SPERR_UNINITIALIZED | Interface not initialized.
FAILED(hr) | Appropriate error message.

Example

The following is an example of using GetWords.

    SPWORDLIST spwordlist;
    memset(&spwordlist, 0, sizeof(spwordlist));
    dwCookie = 0;

    while (SUCCEEDED(hr = pISpLexicon->GetWords(eLEXTYPE_USER | eLEXTYPE_APP, &dwGeneration, &dwCookie, &spwordlist)))
    {
        for (SPWORD *pword = spwordlist.pFirstWord;
            pword != NULL;
            pword = pword->pNextWord)
        {
            for (SPWORDPRONUNCIATION *pwordpron = pword->pFirstWordPronunciation;
                pwordpron != NULL;
                pwordpron = pwordpron->pNextWordPronunciation)
            {
                DoSomethingWith(pwordpron->ePartOfSpeech, pwordpron->pszPronIPA);
            }
        }

        if (hr == S_OK)
               break;  // nothing more to retrieve
    }

    //free all the buffers
    CoTaskMemFree(spwordlist.pvBuffer);


ISpPhoneConverter

The ISpPhoneConverter interface enables the client to convert from the SAPI character phoneset to the Id phoneset.

When to Use
Call methods of the ISpPhoneConverter interface to convert between character and NUM phonesets.
Implemented By
  • SpPhoneConverter

Methods in Vtable Order

ISpPhoneConverter Methods | Description
ISpObjectWithToken interface | Inherits from ISpObjectWithToken and those methods are accessible from an ISpPhoneConverter object.
PhoneToId | Converts a character phoneme string to an ID code string.
IdToPhone | Converts a zero terminated ID code array to the SAPI character format.


ISpPhoneConverter::PhoneToId

ISpPhoneConverter::PhoneToId converts a character phoneme string to an ID code string.

The English and Chinese phoneme sets require the phonemes to be space separated. The Japanese phoneme set requires the phoneme character form to be continuous. See the individual entries for more details about the character sets.

HRESULT PhoneToId(
   const WCHAR  *pszPhone,
   SPPHONEID    *pId
);

Parameters

pszPhone
[in] Address of a null-terminated string that contains the phoneme string information.
pId
[out] Address of the SPPHONEID array that receives the phoneme identifiers. On return the array will be a zero terminated list of SPPHONEIDs.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid, or phoneme not found.
SPERR_UNINITIALIZED | Interface not initialized.
E_FAIL | pId is invalid or bad.
E_FAILED(hr) | Appropriate error message.


ISpPhoneConverter::IdToPhone

ISpPhoneConverter::IdToPhone converts a zero terminated ID code array to the SAPI character format.

The English and Chinese phoneme character sets require the phonemes to be space separated. The Japanese phoneme set requires the phoneme character form to be continuous. See the individual entries for more details about the character sets.

HRESULT IdToPhone(
   const SPPHONEID  *pId,
   WCHAR            *pszPhone
);

Parameters

pId
[in] Address of the zero terminated array of SPPHONEIDs that contains the phoneme identifiers.
pszPhone
[out] Address of a null-terminated string that receives the phoneme string information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid. Alternatively, pId exceeds SP_MAX_PRON_LENGTH.
E_POINTER | pszPhone or pId is invalid or bad.
SPERR_UNINITIALIZED | Interface not initialized.
E_FAIL | Member exceeds available size.
FAILED(hr) | Appropriate error message.


Resource management interfaces

The following section covers:

  • ISpDataKey
  • ISpRegDataKey
  • ISpObjectTokenInit
  • ISpObjectTokenCategory
  • ISpObjectToken
  • IEnumSpObjectTokens
  • ISpObjectWithToken
  • ISpResourceManager
  • ISpTask


ISpDataKey

The ISpDataKey interface provides a mechanism for storing and retrieving string and other data. The main use for this interface is in conjunction with object tokens, which implement the ISpObjectToken interface, which inherits from this interface. For example, data can be stored in an object token representing a recognizer or TTS engine by using this interface.

Implemented By
  • SpObjectToken object. This is the standard class used for all existing SAPI object tokens. The data for each object token is stored in the registry.
  • SpDataKey object. This class stores the data associated with the data key in the registry.
  • Applications or engines can implement this interface directly if they wish to provide a custom data key implementation.

Methods in Vtable Order

ISpDataKey Methods | Description
SetData | Sets the binary data for a token.
GetData | Retrieves the binary data for a token.
SetStringValue | Sets the string value information for a specified token.
GetStringValue | Retrieves the string value information from a specified token.
SetDWORD | Sets the value information for a specified token.
GetDWORD | Retrieves the value information from a specified token.
OpenKey | Opens a specified token subkey.
CreateKey | Creates a new token subkey.
DeleteKey | Deletes a specified token key and all its descendants.
DeleteValue | Deletes a named value from the specified token.
EnumKeys | Enumerates the subkeys of the specified token.
EnumValues | Enumerates the values of the specified token.


ISpDataKey::SetData

ISpDataKey::SetData sets the binary data for a token.

HRESULT SetData(
   const WCHAR   *pszValueName,
   ULONG          cbData,
   const BYTE    *pData
);

Parameters

pszValueName
[in] Address of a null-terminated string that contains the registry key value name.
cbData
[in] Size of the pData parameter.
pData
[out] Pointer to the buffer containing the information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Either pszValueName or pData is an invalid or bad pointer.
FAILED(hr) | Appropriate error message.


ISpDataKey::GetData

ISpDataKey::GetData retrieves the binary data for a token.

HRESULT GetData(
   const WCHAR   *pszValueName,
   ULONG         *pcbData,
   BYTE          *pData
);

Parameters

pszValueName
Address of a null-terminated string containing the name of the registry key from which to retrieve the registry key value.
pcbData
[in] Size of the pData parameter.
pData
[out] Pointer to the buffer receiving the information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszValueName is invalid or bad.
E_POINTER | Either pcbData or pData is an invalid or bad pointer.
SPERR_NOT_FOUND | Token key not found.
FAILED(hr) | Appropriate error message.


ISpDataKey::SetStringValue

ISpDataKey::SetStringValue sets the string value information for a specified token.

HRESULT SetStringValue(
   const WCHAR   *pszValueName,
   const WCHAR   *pszValue
);

Parameters

pszValueName
Address of the null-terminated string specifing the name of the string value. If NULL, the default value of the token is used.
pszValue
Address of a null-terminated string that contains the string value to be set for the specified key.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Either pszValueName or pszValue is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpDataKey::GetStringValue

ISpDataKey::GetStringValue retrieves the string value information from a specified token.

HRESULT GetStringValue(
   const WCHAR   *pszValueName,
   WCHAR        **ppszValue
);

Parameters

pszValueName
Address of a null-terminated string that specifies the name of the registry key. If NULL, the default value of the token is read.
ppszValue
Address of a pointer to a null-terminated string that receives the string value for the specified key.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszValueName is invalid or bad.
E_POINTER | ppszValue is invalid or bad.
SPERR_NOT_FOUND | Registry file not found.
FAILED(hr) | Appropriate error message.


ISpDataKey::SetDWORD

ISpDataKey::SetDWORD sets the value information for a specified attribute.

HRESULT SetDWORD(
   const WCHAR   *pszKeyName,
   DWORD          dwValue
);

Parameters

pszKeyName
Address of a null-terminated string that contains the attribute name.
dwValue
The data buffer containing the attribute key value.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszKeyName is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpDataKey::GetDWORD

ISpDataKey::GetDWORD retrieves the value information from a specified token.

HRESULT GetDWORD(
   const WCHAR   *pszKeyName,
   DWORD         *pdwValue
);

Parameters

pszKeyName
[in] Address of a null-terminated string containing the token name.
pdwValue
[out] Address of the destination data buffer receiving the token key value.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszKeyName is invalid or bad.
E_POINTER | pdwValue is invalid or bad.
SPERR_NOT_FOUND | Registry key not found.
FAILED(hr) | Appropriate error message.


ISpDataKey::OpenKey

ISpDataKey::OpenKey opens a subkey and passes back a new object that supports ISpDataKey for the specified subkey.

HRESULT OpenKey(
   const WCHAR   *pszSubKeyName,
   ISpDataKey   **ppSubKey
);

Parameters

pszSubKeyName
Address of a null-terminated string specifying the name of the key to open.
ppSubKey
Address of a pointer to an ISpDataKey interface.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszSubKeyName is invalid or bad.
E_POINTER | ppSubKey is invalid or bad.
SPERR_NOT_FOUND | Registry key not found.
FAILED(hr) | Appropriate error message.


ISpDataKey::CreateKey

ISpDataKey::CreateKey creates a subkey and returns a new object which supports ISpDataKey for the specified sub-key. If the key already exists, the function will open the existing key instead of overwriting it.

HRESULT CreateKey(
   const WCHAR   *pszSubKeyName,
   ISpDataKey   **ppSubKey
);

Parameters

pszSubKeyName
Address of a null-terminated string specifying the name of the key to create.
ppSubKey
Address of a pointer to an ISpDataKey interface.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Either pszSubKeyName or ppKey is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpDataKey::DeleteKey

ISpDataKey::DeleteKey deletes a specified token key and all its descendants.

HRESULT DeleteKey(
  const WCHAR   *pszSubKeyName
);

Parameters

pszSubKeyName
Address of a null-terminated string specifying the name of the key or subkey to delete.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszSubKeyName is invalid or bad.
SPERR_NOT_FOUND | Token key not found.
FAILED(hr) | Appropriate error message.


ISpDataKey::DeleteValue

ISpDataKey::DeleteValue deletes a named value from the specified token.

HRESULT DeleteValue(
   const WCHAR   *pszValueName
);

Parameters

pszValueName
Address of a null-terminated string specifying the value name to be deleted.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszValueName is invalid or bad.
SPERR_NOT_FOUND | Registry key not found.
FAILED(hr) | Appropriate error message.


ISpDataKey::EnumKeys

ISpDataKey::EnumKeys enumerates the subkeys of the specified token.

HRESULT EnumKeys(
   ULONG     Index,
   WCHAR   **ppszSubKeyName
);

Parameters

Index
[in] Value indicating which token in the enumeration sequence to locate.
ppszSubKeyName
[out] Address of a pointer to a null-terminated string that receives the enumerated key name. This must be freed with CoMemTaskFree() when no longer required.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | ppszSubKeyName is invalid or bad.
SPERR_NOT_FOUND | Registry key not found.
E_OUTOFMEMORY | Not enough memory to allocate string.
SPERR_NO_MORE_ITEMS | No items could be accessed.
FAILED(hr) | Appropriate error message.


ISpDataKey::EnumValues

ISpDataKey::EnumValues enumerates the values of the specified token.

HRESULT EnumValues(
   ULONG     Index,
   WCHAR   **ppszValueName
);

Parameters

Index
[in] Value indicating which token in the enumeration sequence to locate.
ppszValueName
Address of a pointer to a null-terminated string that receives the enumerated registry key values. This must be freed with CoMemTaskFree() when no longer required.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | ppszValueName is invalid or bad.
SPERR_NOT_FOUND | Registry key not found.
E_OUTOFMEMORY | Not enough memory to allocate string.
SPERR_NO_MORE_ITEMS | No items could be accessed.
FAILED(hr) | Appropriate error message.


ISpRegDataKey

This interface is used to create a new data key using a specific key in the registry for storage. The ISpRegDataKey inherits from ISpDataKey.

The ISpRegDataKey inherits from ISpDataKey.

Applications will not normally need to use or implement this interface.

Implemented By
  • SpDataKey. This class stores all the data for a data key in the registry.

    • Methods in Vtable Order

    ISpRegDataKey Methods | Description
SetKey | Sets the hive registry key (HKEY) to use for subsequent token operations.


ISpRegDataKey::SetKey

ISpRegDataKey::SetKey sets the hive registry key (HKEY) to use for subsequent token operations.

HRESULT SetKey(
   HKEY   hkey,
   BOOL   fReadOnly
);

Parameters

hkey
[in] The registry key to use.
fReadOnly
[in] Boolean flag setting the keys to read/write status. If TRUE, the registry is read only; FALSE sets it to read and write.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_ALREADY_INITIALIZED | Interface is already initialized.

Example

The following code snippet adds, tests and deletes a superfluous key from the speech registry.

    HRESULT hr;

    CComPtr cpSpRegDataKey;
    CComPtr cpSpCreatedDataKey;
    CComPtr cpSpCategory;
    CComPtr cpSpDataKey;
    HKEY hkey;

    //create a bogus key under Voices
    hr = g_Unicode.RegCreateKeyEx(HKEY_LOCAL_MACHINE,
        L"SOFTWARE\\Microsoft\\Speech\\Voices\\bogus",
        0, NULL, 0, KEY_READ | KEY_WRITE, NULL, &hkey, NULL);
    //Check error

    hr = cpSpRegDataKey.CoCreateInstance(CLSID_SpDataKey);
    //Check error

    hr = cpSpRegDataKey->SetKey(hkey, false);
    //Check error

    hkey = NULL;
    //Do not need to do RegCloseKey on this hkey, the handle gets released inside SetKey().

    hr = cpSpRegDataKey->QueryInterface(&cpSpCreatedDataKey);
    //Check error

    //delete this bogus key
    hr = SpGetCategoryFromId(SPCAT_VOICES, &cpSpCategory);
    //Check error

    hr = cpSpCategory->GetDataKey(SPDKL_LocalMachine, &cpSpDataKey);
    //Check error

    hr = cpSpDataKey->DeleteKey(L"bogus");
    //Check error


ISpObjectTokenInit

This interface inherits from ISpObjectToken.
Associated Class IDs

The following class IDs (CLSID) may be used with this interface. A complete CLSID listing for all interfaces is in the Class IDs section.

    CLSID_SpObjectToken

Methods in Vtable Order

ISpObjectTokenInit Methods | Description
InitFromDataKey | Initializes a token to use a specified datakey.


ISpObjectTokenInit::InitFromDataKey

ISpObjectTokenInit::SetObjectToken initializes a token to use a specified datakey.

Dynamic token enumerators can use this to create tokens under their token enumerator's token. Once created, this enables ISpDataKey::CreateKey to make a new data key, create a new object token, and then use InitFromDataKey.

HRESULT InitFromDataKey(
   const WCHAR   *pszCategoryId,
   const WCHAR   *pszTokenId,
   ISpDataKey    *pDataKey
);

Parameters

pszCategoryId
[in] The null-terminated string name of the categoryId from which to create the token.
pszTokenId
[in] The null-terminated string name of the TokenId.
pDataKey
[in] Address of an ISpDataKey interface that specifies the system registry key from which to create the token.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | At least one of the parameters is invalid or bad.
SPERR_ALREADY_INITIALIZED | Token is already initialized.
SPERR_TOKEN_DELETED | Key has been deleted.
E_OUTOFMEMORY | Exceeded available memory.


ISpObjectTokenCategory

Each object token category represents a collection of similar tokens, such as voices, recognizers, and audio input devices. Categories may be created or manipulated with helper functions or methods from this interface.

Each category has a Category ID that is unique and identifies only one type of object token. The set of available categories are listed in Token Category IDs. Category IDs are always null-terminated strings.

An application can create an SpObjectTokenCategory object, which implements this interface. Then it calls SetId in order to set the Category ID that this object is using. The application can then enumerate the object tokens associated with this category by using EnumTokens. Applications can also locate and change the default object token for a category with the methods SetDefaultTokenId and GetDefaultTokenId.

ISpObjectTokenCategory inherits from ISpDataKey.

Implemented By
  • SpObjectTokenCategory object. This is the standard class used for categories in SAPI. The category and the list of associated tokens are stored in the registry.
How Created
  • Applications will normally create object token categories by directly creating the SpObjectTokenCategory class.
  • If the application is using the category to find an associated token that matches certain attributes, it is often easier to use the helper functions SpEnumTokens or SpFindBestToken.

Methods in Vtable Order

ISpObjectToken Methods | Description
SetId | Sets the category ID.
GetId | Retrieves the token ID.
GetDataKey | Gets the data key associated with a specific location.
EnumTokens | Enumerates the tokens for the category.
SetDefaultTokenId | Sets a specific TokenId as the default for the category.
GetDefaultTokenId | Retrieves the default TokenId for the category.


ISpObjectTokenCategory::SetId

ISpObjectTokenCategory::SetId sets the category ID.

This method may be called only once. If called more than once, SPERR_ALREADY_INITIALIZED will return.

HRESULT SetId(
   const WCHAR   *pszCategoryId,
   BOOL           fCreateIfNotExist
);

Parameters

pszCategoryId
[in] The null-terminated string name of category to set. SAPI-defined categories are listed in Token Category IDs.
fCreateIfNotExist
[in] Indicates creating the category if one is not already present. TRUE creates the entry. FALSE does not.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_ALREADY_INITIALIZED | Category interface is already initialized.
E_INVALIDARG | pszCategoryId is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpObjectTokenCategory::GetId

ISpObjectTokenCategory::GetId retrieves the category ID.

HRESULT GetId(
   WCHAR   **ppszCoMemCategoryId
);

Parameters

ppszCoMemCategoryId
[in] The null-terminated string name of the current category. ppszCoMemCategoryId must be freed with CoMemTaskFree when no longer required.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Category interface is not initialized.
E_POINTER | ppszCoMemCategoryId is invalid or bad.
FAILED(hr) | Appropriate error message.

Example

The following code snippet retrieves CategoryId for SPCAT_VOICES.

    HRESULT hr;

    CComPtr cpSpCategory;
    CSpCoTaskMemPtr cpwszOldID;

    hr = SpGetCategoryFromId(SPCAT_VOICES, &cpSpCategory);
    //Check return code

    hr = cpSpCategory->GetId(&cpwszOldID);
    //Check return code

    CoMemTaskFree(cpwszOldID);


ISpObjectTokenCategory::GetDataKey

ISpObjectTokenCategory::GetDataKey gets the data key associated with a specific location.

HRESULT GetDataKey(
   SPDATAKEYLOCATION   spdkl,
   ISpDataKey        **ppDataKey
);

Parameters

spdkl
[in] The registry's top-level node to be searched.
ppDataKey
[out] The data key interface associated with the location spdkl.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Data key interface is not initialized.
E_POINTER | ppDataKey is invalid or bad.
FAILED(hr) | Appropriate error message.

Example

The following code snippet retrieves the data key associated with the local computer registry for SPCAT_VOICES.

    HRESULT hr;

    CComPtr cpSpCategory;
    CComPtr cpSpDataKey;

    hr = SpGetCategoryFromId(SPCAT_VOICES, &cpSpCategory);
    //Check return code

    hr = cpSpCategory->GetDataKey(SPDKL_LocalMachine, &cpSpDataKey);
    //Check return code


ISpObjectTokenCategory::EnumTokens

ISpObjectTokenCategory::EnumTokens enumerates the tokens for the category by attempting to match specified requirements. Attributes, enumerations, and searches are discussed in Object Tokens and Registry Settings White Paper.

HRESULT EnumTokens(
   const WCHAR           *pszReqAttribs,
   const WCHAR           *pszOptAttribs,
   IEnumSpObjectTokens  **ppEnum
);

Parameters

pszReqAttribs
[in] The null terminated string of required attributes for the token.
pszOptAttribs
[in] The null terminated string of optional attributes for the token. The order in which the tokens are listed in ppEnum is based on the order they match pszOptAttribs.
ppEnum
[out] The enumerated list of tokens found.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Data key interface is not initialized.
E_POINTER | At least one of the parameters is invalid or bad.
FAILED(hr) | Appropriate error message.

Example

The following code snippet demonstrates getting a complete enumerated token list. Since no specific requirement is given (pszReqAttribs and pszOptAttribs are NULL), all values are returned for SPCAT_VOICES.

    CComPtr<ISpObjectTokenCategory> cpSpCategory;
    CComPtr<IEnumSpObjectTokens> cpSpEnumTokens;
    HRESULT hr;

    hr = SpGetCategoryFromId(SPCAT_VOICES, &cpSpCategory);
    //Check hr

    hr = cpSpCategory->EnumTokens(NULL, NULL, &cpSpEnumTokens);
    //Check hr


ISpObjectTokenCategory::SetDefaultTokenId

ISpObjectTokenCategory::SetDefaultTokenId sets a specific token ID as the default for the category.

The defaults are stored either directly in the category by setting the DefaultTokenID value in the category data key, or indirectly by the DefaultTokenIDLocation. Default tokens are discussed in Object Tokens and Registry Settings White Paper.

HRESULT SetDefaultTokenId(
   const WCHAR   *pszTokenId
);

Parameters

pszTokenId
[in] The null-terminated string name of the token ID to be used as the default.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Data key interface is not initialized.
E_INVALIDARG | pszTokenId is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpObjectTokenCategory::GetDefaultTokenId

ISpObjectTokenCategory::GetDefaultTokenId retrieves the default token ID for the category.

There is a hierachy for returning a default token ID value. A default token has an attribute marked as DefaultTokenID. When a token is not explicitly marked as such, SAPI attempts to return the default from the user profile. If none exists there, SAPI returns a specially named token called DefaultdefaultTokenID for the category ID. Default tokens are discussed in Object Tokens and Registry Settings White Paper.

HRESULT GetDefaultTokenId(
   const WCHAR   **ppszCoMemTokenId
);

Parameters

ppszCoMemTokenId
[in] The null-terminated string name of the token ID used as the default. Must be released with CoMemTaskFree () when no longer needed.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Data key interface is not initialized.
E_POINTER | ppszCoMemTokenId is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpObjectToken

The ISpObjectToken interface handles object token entries.

An object token is an object representing a resource that is available on a computer, such as a voice, recognizer, or an audio input device. A token provides an application a simple way to inspect the various attributes of a resource without having to instantiate it. The Vendor of a Recognizer, and Gender of a Voice are examples of attributes of resources. An application can enumerate the various tokens that exist on the computer by using the SpEnumTokens helper function, or by using the ISpObjectTokenCategory::EnumTokens method to enumerate the tokens of a particular category. Applications can find the best token that matches certain attributes by using the SpFindBestToken function.

Conceptually, a token contains the following information:

  • An identifier that uniquely identifies the object token.
  • The language-independent name is the name that should be displayed wherever the name of the token is displayed. The implementer of the token may also choose to provide a set of language-dependent names in several languages.
  • The CLSID used to instantiate the object from the token.
  • A set of Attributes, which are the set of queriable values in a token. SAPI provides a mechanism to query for tokens whose attributes match certain values.

A token may also contain the following:

  • If a token has user interfaces (UIs), such as the properties of a Recognizer or a wizard to customize a Voice to display, the token will also contain the CLSID for the COM object used to instantiate each type of UI.
  • The set of Files from which SAPI returns the paths to all the associated files for the token.

Attributes are null-terminated strings forming a series of key-pair entries. This is usually in the form of definition relationships. For example, a token may be defined as:

"vendor=microsoft;language=409;someflag"

In this instance:

  • "vendor=microsoft" means a string exists under TokenID\attributes with name vendor and value "microsoft";
  • "language=409" means a string exists under TokenID\attributes with name language and value "409" (representing US English);
  • "someflag" means a string exists under TokenID\attributes with name someflag but has no additional information. Sometimes the presence or absence of the attribute name itself is indicative.

Implemented By
  • SpObjectToken object. This is the standard class used for all existing SAPI object tokens. The data for each object token is stored in the registry.
  • Applications or engines can implement this interface directly to provide a custom object token implementation. For instance, this could be used to avoid storing object token data in the registry, or to provide the ability for object tokens to be downloaded from a server. In this case IEnumSpObjectTokens would also need to be implemented so that the new object tokens to be enumerated.
How Created
  • Applications will normally create object tokens from an object token enumerator or helper function, rather than by directly creating them.
  • Various API methods also return an object token referring to a specific type of resource, e.g., ISpRecognizer::GetRecognizer returns the object token associated with the current recognition engine; ISpRecognizer::GetRecoProfile returns the object token referring to the current recognition profile; and ISpVoice::GetVoice returns the object token referring to the current TTS engine.

ISpObjectToken inherits from ISpDataKey.

Object tokens are discussed in more detail in the Object Tokens and Registry Settings White Paper.

Methods in Vtable Order

ISpObjectToken Methods | Description
SetId | Sets the category identifier for object token.
GetId | Retrieves the object identifier for an object token.
GetCategory | Retrieves the category for a specified token if one is available.
CreateInstance | Creates an instance of an object.
GetStorageFileName | Retrieves the object token file name.
RemoveStorageFileName | Removes the object token file name.
Remove | Removes an object token.
IsUISupported | Determines if the UI associated with the object is supported.
DisplayUI | Displays the UI associated with the object.
MatchesAttributes | Determines if an object token supports a specified attribute.


ISpObjectToken::SetId

ISpObjectToken::SetId sets the CategoryId for object token.

This may be called only once.

HRESULT SetId(
   const WCHAR   *pszCategoryId,
   const WCHAR   *pszTokenId,
   BOOL           fCreateIfNotExist
);

Parameters

pszCategoryId
[in] The null-terminated string name of category to set.
pszTokenId
[in] The null-terminated string name of token to set.
fCreateIfNotExist
[in] A Boolean indicating the object is to be created if not currently existing. TRUE allows the creation; FALSE does not.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_ALREADY_INITIALIZED | Category interface is already initialized.
SPERR_TOKEN_DELETED | Key has been deleted.
E_INVALIDARG | Either pszCategoryId and/or pszTokenId is invalid or bad.
FAILED(hr) | Appropriate error message.

Notes

CategoryIds appear in the fully qualified form as:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Recognizers

The only acceptable HKEYs are:

HKEY_CLASSES_ROOT,

HKEY_CURRENT_USER,

HKEY_LOCAL_MACHINE,

HKEY_CURRENT_CONFIG


ISpObjectToken::GetId

ISpObjectToken::GetId retrieves the object identifier for an object token. This identifier can be used later to recreate a token instance.

HRESULT GetId(
   WCHAR   **ppszCoMemTokenId
);

Parameters

ppszCoMemTokenId
Address of a pointer to a null-terminated string that receives the identifier for the token object. The caller must call CoTaskMemFree() to free the string pointer.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppszCoMemTokenId is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.
SPERR_UNINITIALIZED | TokenId interface is not initialized.
FAILED(hr) | Appropriate error message.


ISpObjectToken::GetCategory

ISpObjectToken::GetCategory retrieves the category for a specified token if one is available.

HRESULT GetCategory(
   ISpObjectTokenCategory   **ppTokenCategory
);

Parameters

ppTokenCategory
[out] The category interface for the token. ppTokenCategory must be freed when no longer required.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppTokenCategory is invalid or bad.
SPERR_UNINITIALIZED | Token does not have a category.
FAILED(hr) | Appropriate error message.


ISpObjectToken::CreateInstance

ISpObjectToken::CreateInstance creates an instance of an object.

This method is used to create the underlying object that the object token represents. This method looks at the CLSID value stored in the object token and creates a COM object from this CLSID.

For example, when this method is called on an object token from the audio input category, an audio object that implements ISpStreamFormat will be created and returned.

This method is not used to create speech recognition or TTS engines. Instead, an SpRecognizer or SpVoice object is created and the engine is then created by passing an object token to the ISpRecognizer::SetRecognizer or ISpVoice::SetVoice methods.

HRESULT CreateInstance(
   IUnknown   *pUnkOuter,
   DWORD       dwClsContext,
   REFIID      riid,
   void      **ppvObject
);

Parameters

pUnkOuter
[in] If the object is being created as part of an aggregate, this is a pointer to the controlling IUnknown interface of the aggregate. Otherwise, pUnkOuter must be NULL.
dwClsContext
[in] Context in which the code that manages the newly created object will run. It should be one of the following values:
  • CLSCTX_INPROC_SERVER
  • CLSCTX_INPROC_HANDLER
  • CLSCTX_LOCAL_SERVER
  • CLSCTX_REMOTE_SERVER
riid
[in] Reference to the identifier of the interface used to communicate with the newly created object. If pUnkOuter is NULL, this parameter is frequently the IID of the initializing interface; if pUnkOuter is non-NULL, riid must be IID_IUnknown.
ppvObject
[out, iid_is(riid)] Address of pointer variable that receives the interface pointer requested in riid. Upon successful return, ppvObject contains the requested interface pointer. If the object does not support the interface specified in riid, the implementation must set ppvObject to NULL.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppvObject is invalid or bad.
E_INVALIDARG | pUnkOuter is invalid or bad.
SPERR_UNINITIALIZED | Either the data key or the token delegator interface is not initialized.
SPERR_TOKEN_DELETED | Key has been deleted.
FAILED(hr) | Appropriate error message.

Return values

The following code snippet creates an InProc server instance.

    HRESULT hr;

    CComPtr cpSpObjectToken;
    CComPtr cpSpObjectWithToken;

    hr = SpGetDefaultTokenFromCategoryId(SPCAT_VOICES, &cpSpObjectToken);
    //Check return value

    hr = cpSpObjectToken->CreateInstance(
            NULL, CLSCTX_INPROC_SERVER, IID_ISpObjectWithToken,
            (void **)&cpSpObjectWithToken
        );
    //Check return value


ISpObjectToken::GetStorageFileName

ISpObjectToken::GetStorageFileName retrieves the object token file name from the registry.

HRESULT GetStorageFileName(
    REFCLSID      clsidCaller,
    const WCHAR  *pszValueName,
    const WCHAR  *pszFileNameSpecifier,
    ULONG         nFolder,
    WCHAR       **ppszFilePath
);

Parameters

clsidCaller
[in] Globally unique identifier (GUID) of the calling object. The registry is searched for an entry key name of clsidCaller, and then a corresponding "Files" subkey. If the registry entry is not present, one is created.
pszValueName
[in] The name of the attribute file for the registry entry of clsidCaller. This attribute stores the location of the resource file.
pszFileNameSpecifier
[in] The specifier that is either NULL or a path/filename for storage file.
    If this starts with "X:\" or "\\" it is assumed to be a full path.
    Otherwise it is assumed to be relative to special folders given in the nFolder parameter.
    If it ends with a '\', or is NULL a unique filename will be created. The filename will be something like:
    "SP_7454901D23334AAF87707147726EC235.dat". "SP_" and ".dat" are the default prefix name and file extension name. The numbers in between are generated guid number to make sure the filename is unique.
    If the name contains a %d the %d is replaced by a number to give a unique filename. The default file extension is .dat, the user can specify anything else.
    Intermediate directories are created.
    If a relative file is being used the value stored in the registry includes the nFolder value as %nFolder% before the rest of the path.
nFolder
[in] A CSIDL value that identifies the folder whose path is to be retrieved. The user can force the creation of a folder by combining the folder's CSIDL with CSIDL_FLAG_CREATE. If pszFileNameSpecifier is NULL or "\", nFolder must have a specified CSIDL folder combined with CSIDL_FLAG_CREATE if the user wants to force to create the file.
ppszFilePath
[out] Address of a pointer to the null-terminated string that receives the file path information. Must be freed when no longer required.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppszFilePath is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.
S_FALSE | A new file was created.
E_INVALIDARG | pszValueName is invalid or bad.
SPERR_UNINITIALIZED | Either the data key or the token delegate interface is uninitialized.
SPERR_TOKEN_DELETED | Key has been deleted.
FAILED(hr) | Appropriate error message.

Example

The following code snippet creates and removes a token object for a test file.

    HRESULT hr;
    GUID guid0;
    GUID guid1;

    CComPtr cpSpObjectToken;
    CSpCoTaskMemPtr cpFileName;
    CSpCoTaskMemPtr cpFileName2;

    hr = SpGetDefaultTokenFromCategoryId(SPCAT_VOICES, &cpSpObjectToken);
    //Check return value

    ZeroStruct(guid0);
    hr = CoCreateGuid(&guid1);
    //Check return value

    hr = cpSpObjectToken->GetStorageFileName(
            guid0, L"TestFile", NULL, CSIDL_FLAG_CREATE|CSIDL_APPDATA, &cpFileName
        );
    //The created file will have default format, and will be stored under file system directory that
    //serves as a common repository for application-specific data.
    //Check return value

    hr = cpSpObjectToken->Remove(&guid0);
    //Check return value

    hr = cpSpObjectToken->GetStorageFileName(
            guid1, L"TestFile2", L"c:\\Program Files\\MyData%d.dump", CSIDL_FLAG_CREATE, &cpFileName2
        );
    //The created file will be stored under C:\Program Files, and will have a name like MyData7412341D23334A7321707145534EC235.dump
    //Check return value
    hr = cpSpObjectToken->Remove(&guid1);
    //Check return value


ISpObjectToken::RemoveStorageFileName

ISpObjectToken::RemoveStorageFileName removes the object token file name.

HRESULT RemoveStorageFileName(
   REFCLSID       clsidCaller,
   const WCHAR   *pszKeyName,
   BOOL           fDeleteFile
);

Parameters

clsidCaller
[in] Globally unique identifier (GUID) of the calling object.
pszKeyName
[in] Address of a null-terminated string containing the registry key name.
fDeleteFile
[in] Value specifying if the file should be deleted. TRUE deletes the file afterwards; FALSE does not.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszKeyName is invalid or bad.
SPERR_UNINITIALIZED | Either the data key or token delegate interface is not initialized.
SPERR_TOKEN_DELETED | Key has been deleted.
FAILED(hr) | Appropriate error message.

Example

The following code snippet creates a test file, removes it and manually deletes it. It may also have been deleted automatically by setting fDeleteFile to TRUE.

    HRESULT hr;
    GUID guid0;

    CComPtr cpSpObjectToken;
    CComPtr cpSpObjectWithToken;
    CSpCoTaskMemPtr cpFileName;

    hr = SpGetDefaultTokenFromCategoryId(SPCAT_VOICES, &cpSpObjectToken);
    //Check return value

    ZeroStruct(guid0);
    // Create subkeys and value item to be deleted
    hr = cpSpObjectToken->GetStorageFileName(
            guid0, L"test file", NULL, CSIDL_FLAG_CREATE|CSIDL_APPDATA, &cpFileName
            );

    if (SUCCEEDED(hr))
    {
        hr = cpSpObjectToken->RemoveStorageFileName(guid0, L"test file", false);
        //Check return value

        cpFileName.Clear();
    }


ISpObjectToken::Remove

ISpObjectToken::Remove removes an object token.

HRESULT Remove(
   const CLSID  *pclsidCaller
);

Parameters

pclsidCaller
[in] Address of the identifier associated with the object token to remove. If pclsidCaller is NULL, the entire token is removed; otherwise, only the specified section is removed.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pclsidCaller is invalid or bad.
SPERR_UNINITIALIZED | The token ID interface is uninitialized.
SPERR_TOKEN_DELETED | Key has been deleted.
FAILED(hr) | Appropriate error message.

Example

The following code snippet creates and removes a token object for a test file.

    HRESULT hr;
    GUID guid0;

    CComPtr cpSpObjectToken;
    CSpCoTaskMemPtr cpFileName;

    hr = SpGetDefaultTokenFromCategoryId(SPCAT_VOICES, &cpSpObjectToken);
    //Check return value

    ZeroStruct(guid0);
    hr = cpSpObjectToken->GetStorageFileName(
            guid0, L"TestFile", NULL, CSIDL_FLAG_CREATE|CSIDL_APPDATA, &cpFileName
        );
    //Check return value

    hr = cpSpObjectToken->Remove(&guid0);
    //Check return value


ISpObjectToken::IsUISupported

ISpObjectToken::IsUISupported determines if the user interface (UI) associated with the object is supported.

Ultimately, ISpObjectToken::IsUISupported is similar to creating an ISpTokenUI object and calling ISpTokenUI::ISpIsUISupported.

[local] HRESULT IsUISupported(
    const WCHAR  *pszTypeOfUI,
    void         *pvExtraData,
    ULONG         cbExtraData,
    IUnknown     *punkObject,
    BOOL         *pfSupported
);

Parameters

pszTypeOfUI
[in] Address of the null-terminated string containing the UI type that is being queried. Must be a SPDUI_xxx type.
pvExtraData
[in] Pointer to additional information needed for the object. The ISp TokenUI object implementer dictates the format and usage of the data provided. See Remarks section.
cbExtraData
[in] Size, in bytes, of the ExtraData. The ISpTokenUI object implementer dictates the format and usage of the data provided.
punkObject
[in] Address of the IUnknown interface pointer. See Remarks section.
pfSupported
[out] Address of a variable that receives the value indicating support for the interface. This value is set to TRUE when this interface is supported, and FALSE when it is not. If this value is TRUE, but the return code is S_FALSE, the UI type (guidTypeOfUI) is supported, but not with the current parameters or runtime environment. Check with the implementer of the UI object to verify runtime requirements.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The UI is supported but not with the current runtime environment or parameters.
E_INVALIDARG | One of the parameters is invalid or bad.
SPERR_UNINITIALIZED | Either the data key or token delegate interface is not initialized.
SPERR_TOKEN_DELETED | Key has been deleted.
FAILED(hr) | Appropriate error message.

Remarks

pvExtraData and punkObject Parameters: When asking an ISpObjectToken to display a particular piece of UI, the UI object may require extra functionality that only it understands. Common implementation practice for accessing this functionality is to QueryInterface off of a known IUnknown interface. The caller of ISpTokenUI::IsUISupported can set the punkObject parameter with the necessary IUnknown interface. For example, to display a Speech Recognition Training UI (see SPDUI_UserTraining) requires a specific SR engine.

Example

The following code snippet illustrates the use of ISpObjectToken::IsUISupported using SPGUID_EngineProperties.

    HRESULT hr = S_OK;

    // get the default text-to-speech engine object token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_VOICES, &cpObjectToken);
    // Check hr

    // create the engine object based on the object token
    hr = SpCreateObjectFromToken(cpObjectToken, &cpVoice);
    // Check hr

    // create a data key for the voice's UI objects
    hr = cpObjectToken->OpenKey(L"UI", &cpUIDataKey);
    // Check hr

    // create a data key for the specific Engine Properties UI
    hr = cpUIDataKey->OpenKey(SPDUI_EngineProperties, &cpEngPropsDataKey);
    // Check hr

    // get the GUID for the voice's engine properties UI
    hr = cpEngPropsDataKey->GetStringValue(L"CLSID", &pwszEngPropsCLSID);
    // Check hr

    // convert GUID string to pure GUID
    hr = CLSIDFromString(pwszEngPropsCLSID, &clsidEngProps);
    // Check hr

    // check if the default voice object has UI for Properties
    hr = cpObjectToken->IsUISupported(&clsidEngProps, NULL, NULL, cpVoice, &fSupported);
    // Check hr

    // if fSupported == TRUE, then default voice object has UI for Engine Properties


ISpObjectToken::DisplayUI

ISpObjectToken::DisplayUI displays the user interface (UI) associated with the object.

The best practice for using ISpObjectToken::DisplayUI is to call ISpObjectToken::IsUISupported with a specific UI type before calling DisplayUI. Ultimately, ISpObjectToken::DisplayUI is similar to creating an ISpTokenUI object and calling ISpTokenUI::DisplayUI

The call to DisplayUI is synchronous and the call will not return until the UI has been closed.

[local] HRESULT DisplayUI(
   HWND           hwndParent,
   const WCHAR   *pszTitle,
   const WCHAR    pszTypeOfUI,
   void          *pvExtraData,
   ULONG          cbExtraData,
   IUnknown      *punkObject
);

Parameters

hwndParent
[in] Specifies the handle of the parent window.
pszTitle
[in] Address of a null-terminated string containing the window title. Set this value to NULL to indicate that the ISpTokenUI object should use its default window title.
pszTypeOfUI
[in] Address of the null-terminated string containing the UI type that is being queried. Must be a SPDUI_xxx type.
pvExtraData
[in] Pointer to additional information needed for the object. The ISp TokenUI object implementer dictates the format and usage of the data provided.
cbExtraData
[in] Size, in bytes, of the ExtraData. See Remarks section.
punkObject
[in] Address of the IUnknown interface pointer. See Remarks section.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The UI is supported but not with the current runtime environment or parameters.
E_INVALIDARG | One of the parameters is invalid or bad.
SPERR_UNINITIALIZED | Either the data key or token delegate interface is not initialized.
SPERR_TOKEN_DELETED | Key has been deleted.
FAILED(hr) | Appropriate error message.

Remarks

pvExtraData and punkObject Parameters: When requesting an ISpObjectToken to display a particular piece of UI, the UI object may require extra functionality. Common implementation practice for accessing this functionality is to QueryInterface from a known IUnknown interface. The caller of ISpTokenUI::DisplayUI can set the punkObject parameter with the necessary IUnknown interface. For example, asking to display Speech Recognition Training UI (see SPDUI_UserTraining) requires the use of a specific SR Engine.

Example

The following code snippet illustrates the use of ISpObjectToken::DisplayUI using SPGUID_EngineProperties.

    HRESULT hr = S_OK;

    // get the default text-to-speech engine object token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_VOICES, &cpObjectToken);
    // Check hr

    // create the engine object based on the object token
    hr = SpCreateObjectFromToken(cpObjectToken, &cpVoice);
    // Check hr

    // create a data key for the voice's UI objects
    hr = cpObjectToken->OpenKey(L"UI", &cpUIDataKey);
    // Check hr

    // create a data key for the specific Engine Properties UI
    hr = cpUIDataKey->OpenKey(SPDUI_EngineProperties, &cpEngPropsDataKey);
    // Check hr

    // get the GUID for the voice's engine properties UI
    hr = cpEngPropsDataKey->GetStringValue(L"CLSID", &pwszEngPropsCLSID);
    // Check hr

    // convert GUID string to pure GUID
    hr = CLSIDFromString(pwszEngPropsCLSID, &clsidEngProps);
    // Check hr

    // check if the default voice object has UI for Properties
    hr = cpObjectToken->DisplayUI(MY_HWND, MY_APP_VOICE_PROPERTIES, &clsidEngProps, NULL, NULL, cpVoice);
    // Check hr


ISpObjectToken::MatchesAttributes

ISpObjectToken::MatchesAttributes determines if an object token supports a specified attribute.

HRESULT MatchesAttributes(
   const WCHAR   *pszAttributes,
   BOOL          *pfMatches
);

Parameters

pszAttributes
[in] Address of the null-terminated string specifying the object token attribute to match.
pfMatches
[out] Address of a variable that receives the value indicating a match of the object token attribute specified in pszAttributes. This value is set to TRUE when the object token attribute matches, and FALSE when it does not match.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
SPERR_UNINITIALIZED | The object has not been properly initialized.
FAILED(hr) | Appropriate error message.


IEnumSpObjectTokens

The IEnumSpObjectTokens interface is used to enumerate speech object tokens.

When to Use

Call methods of the IEnumSpObjectTokens interface to enumerate the speech object tokens.

Associated Class IDs

The following class IDs (CLSID) may be used with this interface. A complete CLSID listing for all interfaces is in the Class IDs section.

    CLSID_SpMMAudioEnum

Methods in Vtable Order

IEnumSpObjectTokens Methods | Description
Next | Retrieves the next object token in the enumeration sequence.
Skip | Skips a specified number of object tokens in the enumeration sequence.
Reset | Resets the enumeration sequence to the beginning.
Clone | Creates a new enumerator object with the same items.
Item | Locates a specific token in the enumeration.
GetCount | Retrieves the number of object tokens contained in the enumeration sequence.


IEnumSpObjectTokens::Next

IEnumSpObjectTokens::Next retrieves the next object token in the enumeration sequence.

If there are fewer than the requested number of elements left in the sequence, the remaining elements are retrieved.

HRESULT Next(
   ULONG              celt,
   ISpObjectToken   **pelt,
   ULONG             *pceltFetched
);

Parameters

celt
[in] The number of object tokens to retrieve.
pelt
[out] Address of an array that receives ISpObjectToken pointers. If an error value is returned, no entries in the array are valid.
pceltFetched
[out] Address of a variable that receives the number of ISpObjectToken pointers actually copied to the array. This parameter cannot be NULL if celt is greater than one. If this parameter is NULL, celt must be one.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER One of the following errors was encountered:
pelt is bad or invalid,
pceltFetched is bad or invalid,
pceltFetched is bad and celt is greater than one.
E_INVALIDARG | celt is zero.
SPERR_UNINITIALIZED | Attribute parser interface is not initialized.
S_FALSE | celt is greater than the number of objects available.
FAILED (hr) | Appropriate error message.


IEnumSpObjectTokens::Skip

IEnumSpObjectTokens::Skip skips a specified number of object tokens in the enumeration sequence.

HRESULT Skip(
   ULONG   celt
);

Parameters

celt
[in] Number of object tokens to skip in the enumeration sequence.

Return values

Value | Description
S_OK | Number of elements skipped was celt.
S_FALSE | Number of elements skipped was less than celt.
SPERR_UNINITIALIZED | Attribute parser interface is not initialized.
FAILED (hr) | Appropriate error message.


IEnumSpObjectTokens::Reset

IEnumSpObjectTokens::Reset resets the enumeration sequence to the beginning.

HRESULT Reset ( void );

Parameters

None

Return values

Value | Description
S_OK | Method completed successfully.
SPERR_UNINITIALIZED | Attribute parser interface is not initialized.


IEnumSpObjectTokens::Clone

IEnumSpObjectTokens::Clone creates a new enumerator object with the same items.

Returns a new enumerator object with the same items but an independent index. The items in the clone are not guaranteed to be in the same order as the original enumerator.

HRESULT Clone(
   IEnumSpObjectTokens   **ppEnum
);

Parameters

ppEnum
[out] Address of the IEnumSpObjectTokens pointer variable that receives the interface pointer to the cloned enumerator. Using Clone, it is possible to record a particular point in the enumeration sequence and then return to that point at a later time. The enumerator returned is of the same interface type as the one being cloned.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_UNINITIALIZED | Attribute parser interface is not initialized.
FAILED (hr) | Appropriate error message.


IEnumSpObjectTokens::Item

IEnumSpObjectTokens::Item locates a specific token in the enumeration.

HRESULT Item(
   ULONG             Index,
   ISpObjectToken  **ppToken
);

Parameters

Index
[in] Value indicating which token in the enumeration sequence to locate.
ppToken
[out] Address of an ISpObjectToken interface pointer.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_NO_MORE_ITEMS | Index is greater than the amount of items available.
E_POINTER | ppToken is bad or invalid.
SPERR_UNINITIALIZED | Attribute parser interface is not initialized.
FAILED (hr) | Appropriate error message.


IEnumSpObjectTokens::GetCount

IEnumSpObjectTokens::GetCount retrieves the number of object tokens contained in the enumeration sequence.

HRESULT GetCount(
   ULONG   *pulCount
);

Parameters

pulCount
[out] The number of object token items contained in the enumeration sequence.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pulCount is bad or invalid.
SPERR_UNINITIALIZED | Attribute parser interface is not initialized.
FAILED (hr) | Appropriate error message.


ISpObjectWithToken

Any object associated with an object token implements the ISpObjectWithToken interface. When the method ISpObjectToken::CreateInstance is called on the object token, SAPI creates the associated object. If this object implements ISpObjectWithToken, the SetObjectToken method will be called, passing in a reference to the object token.

After SAPI calls ISpObjectWithToken::SetObjectToken, a token object is created and its data may be accessed.

Applications do not need to implement this interface, but engine, lexicon, or custom audio developers may implement it in order to access their object token data.

Implemented By
  • SpMMAudioEnum
  • SpRecPlayAudio
  • SpUnCompressedLexicon
  • SpCompressedLexicon

Methods in Vtable Order

ISpObjectWithToken Methods | Description
SetObjectToken | Binds the instance object to the specified token.
GetObjectToken | Retrieves an object token.


ISpObjectWithToken::SetObjectToken

ISpObjectWithToken::SetObjectToken binds the instance object to the specified token.

HRESULT SetObjectToken(
   ISpObjectToken  *pToken
);

Parameters

pToken
[in] The token interface pointer this instance object is to combine with.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pToken is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.


ISpObjectWithToken::GetObjectToken

ISpObjectWithToken::GetObjectToken retrieves an object token.

HRESULT GetObjectToken(
   ISpObjectToken   **ppToken
);

Parameters

ppToken
[out] Address of an ISpObjectToken interface that receives the object token.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppToken is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.


ISpResourceManager

The ISpResourceManager interface provides access to the shared resources between different speech components in the same process. The object CLSID_SpResourecManager is a COM singleton. That is, the same resource manager will be shared by all clients in a single process that uses CoCreateInstance to create the object with CLSID_SpResourceManager. This allows for a common point of control for sharing other COM objects.

Applications and engines do not need to use or implement this interface.

This interface inherits from IServiceProvider. The IServiceProvider interface supports a single method, QueryService, which provides access to objects in the resource manager.

Associated Class IDs

The following class IDs (CLSID) may be used with this interface. A complete CLSID listing for all interfaces is in the Class IDs section.

    CLSID_SpResourceManager

Methods in Vtable Order

ISpResourceManager Methods | Description
SetObject | Adds a service object to the current service list.
GetObject | Retrieves a service object from the current service list, or creates one if it does not exist.


ISpResourceManager::SetObject

ISpResourceManager::SetObject adds a COM object to the current service list. If an object is already set for the specified service, the ISpResourceManager::SetObject method will replace the current object with the new one. If pUnkObject is NULL, the current service object is removed.

HRESULT SetObject(
   REFGUID     guidServiceId,
   IUnknown   *pUnkObject
);

Parameters

guidServiceId
[in] The unique identifier of the service.
pUnkObject
[in] Address of the IUnknown interface of the object that is setting the service. Any existing service object is removed if this parameter is NULL.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pUnkObject is bad or invalid.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.


ISpResourceManager::GetObject

ISpResourceManager::GetObject retrieves a service object from the current service list, or creates one if it does not exist.

If the object does not exist, the ObjectCLSID parameter will be used to CoCreateInstance the object. This functionality allows multiple threads to ensure that a single shared object is created atomically by the resource manager. If the fReleaseWhenNoRefs flag is set to TRUE, the final release of the object will remove it from the service list. If fReleaseWhenNoRefs is FALSE, the service will remain in the service list until the resource manager is released or the service is explicitly removed through a SetObject call.

HRESULT GetObject(
   REFGUID    guidServiceId,
   REFCLSID   ObjectCLSID,
   REFIID     ObjectIID,
   BOOL       fReleaseWhenNoRefs,
   void     **ppObject
);

Parameters

guidServiceId
[in] The unique identifier of the service.
ObjectCLSID
[in] Class identifier of the object.
ObjectIID
[in] Identifier of interface to return in ppObject
fReleaseWhenNoRefs
[in] Boolean indicating whether the object should be freed when the last client outside of the resource manager releases the object. If this flag is set, the object specified by ObjectCLSID must support aggregation.
ppObject
[out] Address of a pointer that receives the interface pointer of the service.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | ppObject is bad or invalid.
REGDB_E_CLASSNOTREG | Class is not registered.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.


ISpTask

The ISpTask interface is a C++ pure virtual interface, and not a COM interface. It is used by objects to perform atomic operations, which have been optimized for a multiprocessor computer. The ISpTaskManager can be used to create task objects that support the ISpNotifySink interface. When the Notify is called on these objects, a thread will be allocated from a thread pool, and ISpTask::Execute will be called on that thread. The client code then performs the necessary operation on that thread and returns from Execute when finished. Applications should avoid blocking on I/O operations because they are consuming a thread from the shared thread pool.

When to Implement

ISpTask is most useful with multiprocessor computers. IspTask allocates tasks efficiently based on the current availability of processor time. Implement ISpTask on objects that perform tasks that can be broken into smaller tasks.

This is not a COM interface.

Methods in Vtable Order

ISpTask Methods | Description
Execute | Calls on a worker thread to allow the client to perform necessary task operations.


ISpTask::Execute

ISpTask::Execute implements the work unit for an object. This will be application specific.

virtual HRESULT STDMETHODCALLTYPE Execute(
   void                  *pvTaskData,
   volatile const BOOL   *pfContinueProcessing
);

Parameters

pvTaskData
[in] The pointer passed to ISpTaskManager::CreateReoccurringTask pvTaskData parameter.
pfContinueProcessing
[in] Boolean indicating if the process should continue. TRUE continues the process; otherwise FALSE. Clients should examine this variable during processing and exit if this flag set to FALSE, as it indicates that another thread has released the reoccurring task object.

Return values

The return value is ignored by the SAPI task manager.


Speech Recognition interfaces

The following section covers:

  • ISpRecoContext
  • ISpRecoGrammar
  • ISpRecoResult
  • ISpRecognizer
  • ISpPhrase
  • ISpPhraseAlt
  • ISpProperties


ISpRecoContext

The ISpRecoContext interface enables applications to create different functional views or contexts of the SR engine. Each ISpRecoContext object can take interest in different SR events (see also ISpEventSource and SPEVENTENUM) and use different recognition grammars (see also ISpRecoGrammar). Applications must have at least one ISpRecoContext instance to receive recognitions. Applications can also create multiple ISpRecoContext instances to separate different types of recognition with their application. For example, a multiple-document-interface (MDI) application could associate a different ISpRecoContext instance with each document pane to localize the grammar and support and event processing.

A new ISpRecoContext object can be created by calling ISpRecognizer::CreateRecoContext.

To use a shared recognizer (see description of ISpRecognizer), an application can easily create a shared ISpRecoContext by calling ::CoCreateInstance with CLSID_SpSharedRecoContext.

Associated Class IDs

The following class IDs (CLSID) may be used with this interface. A complete CLSID listing for all interfaces is in the Class IDs section.

    CLSID_SpSharedRecoContext

Methods in Vtable Order

ISpRecoContext Methods | Description
ISpEventSource | Inherits from ISpEventSource and those methods are accessible from ISpRecoContext.
GetRecognizer | Returns a reference to the current recognizer object associated with this context.
CreateGrammar | Creates an SpGrammar object.
GetStatus | Retrieves current state information associated with a context.
GetMaxAlternates | Retrieves the maximum number of alternates that will be generated for command and control grammars.
SetMaxAlternates | Sets the maximum number of alternates returned for command and control grammars.
SetAudioOptions | Sets the audio options for results from this recognition context.
GetAudioOptions | Retrieves the audio options for the context.
DeserializeResult | Creates a new result object from a serialized result.
Bookmark | Sets a bookmark within the current recognition stream.
SetAdaptationData | Passes a block of text to the SR engine which can be used to adapt the active language models.
Pause | Pauses the engine object to synchronize with the SR engine.
Resume | Resumes the SR engine from the paused state and restarts the recognition process.
SetVoice | Sets the associated ISpVoice to this context.
GetVoice | Retrieves a reference to the associated ISpVoice object.
SetVoicePurgeEvent | Sets the SR engine events that stop audio output, and purges the current speaking queue.
GetVoicePurgeEvent | Retrieves the set of SR engine events that stop audio output, and purges the current speaking queue.
SetContextState | Sets the state of the recognition context.
GetContextState | Retrieves the state of the recognition context.


ISpRecoContext::GetRecognizer

ISpRecoContext::GetRecognizer returns a reference to the current recognizer object associated with this context.

HRESULT GetRecognizer(
   ISpRecognizer   **ppRecognizer
);

Parameters

ppRecognizer
[out] Address of a pointer that receives the ISpRecognizer interface. The application must call IUnknown::Release when finished with the interface.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.
FAILED (hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecoContext::GetRecognizer with a shared context

    HRESULT hr = S_OK;

    // create a shared recognition context
    hr = cpRecoContext.CoCreateInstance(CLSID_SpSharedRecoContext);
    // Check hr

    // get a reference to the associated recognizer
    hr = cpRecoContext->GetRecognizer(&cpRecognizer);
    // Check hr

    // assert that our shared context has a shared recognizer
    hr = cpRecognizer->IsSharedInstance();
    // Check that hr == S_OK


ISpRecoContext::CreateGrammar

ISpRecoContext::CreateGrammar creates an SpRecoGrammar object.

HRESULT CreateGrammar(
   ULONGLONG          ullGrammarId,
   ISpRecoGrammar   **ppGrammar
);

Parameters

ullGrammarId
[in] Specifies the grammar identifier. The identifier is used by the application and is not required. This identifier is associated with all result objects from the grammar (see SPPHRASE.ullGrammarID).
ppGrammar
[out] Address of a pointer which receives the ISpRecoGrammar object. The application must call IUnknown::Release on the object when finished using it.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppGrammar is invalid.
E_OUTOFMEMORY | Not enough system memory to create a grammar object.
SPERR_SR_ENGINE_EXCEPTION | An exception was thrown by the SR Engine during OnCreateGrammar.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecoContext::CreateGrammar.

    HRESULT hr = S_OK;

    hr = cpRecoContext->CreateGrammar(GRAM_ID, &cpRecoGrammar);
    // Check hr

    // load a cfg from a file
    hr = cpRecoGrammar->LoadCmdFromFile(MY_CFG_FILENAME, SPLO_STATIC);
    // Check hr

    // activate the top-level rules
    hr = cpRecoGrammar->SetRuleState(NULL, NULL, SPRS_ACTIVE);
    // Check hr

    // get a recognition
    // ...

    // get the recognized phrase from the recognition result object
    hr = cpRecoResult->GetPhrase(&pPhrase);
    // Check hr

    // check the grammar id of the recognition result
    SPDBG_ASSERT(GRAM_ID == pPhrase->ullGrammarID);


ISpRecoContext::GetStatus

ISpRecoContext::GetStatus retrieves current state information associated with a context (e.g., last SR Engine requested UI, audio signal status, etc.).

HRESULT GetStatus(
   SPRECOCONTEXTSTATUS   *pStatus
);

Parameters

pStatus
[out] Address of the SPRECOCONTEXTSTATUS structure that receives the context state information.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pStatus is invalid or bad.
FAILED (hr) | Appropriate error message.

Remarks

A graphical application that is interested in SPEI_REQUEST_UI events from the SR Engine can call ISpRecoContext::GetStatus, and check the szRequestTypeOfUI field to check the last requested UI-type. After the application has called ISpRecognizer::DisplayUI, the SR Engine can clear the szRequestTypeOfUI field by calling ISpSREngineSite::AddEvent with a NULL UI-type.

An application can also periodically query the recognition context status to check the audio signal quality (see also SPINTERFERENCE) and respond appropriately. An application can prompt the user to access the SR Engine's microphone training UI to improve the audio signal quality (see SPDUI_MicTraining), or prompt the user to modify the audio settings using Speech properties in Control Panel (see SPDUI_AudioProperties and SPDUI_AudioVolume).

Example

The following code snippet illustrates the use of ISpRecoContext::GetStatus for responding to SR Engine UI requests.

    HRESULT hr = S_OK;

    // assume UI request [SPEI_REQUEST_UI] has been received

    // check what kind of UI the SR Engine wants
    hr = cpRecoContext->GetStatus(&contextStatus);
    // Check hr

    // get a reference to the SR Engine
    hr = cpRecoContext->GetRecognizer(&cpRecognizer);
    // Check hr

    // sanity check that the UI type is supported
    hr = cpRecognizer->IsUISupported(contextStatus.szRequestTypeOfUI, NULL, NULL, &fSupported);
    // Check hr

    // ask the SR engine to display the UI, and use the default window title
    hr = cpRecognizer->DisplayUI(MY_HWND, NULL, contextStatus.szRequestTypeOfUI, NULL, NULL);
    // Check hr


ISpRecoContext::GetMaxAlternates

ISpRecoContext::GetMaxAlternates retrieves the maximum number of alternates that the SR engine will return for command and control or proprietary grammars associated with this context. See Remarks section.

This method has no effect on dictation alternates. See ISpRecoResult::GetAlternates for information regarding dictation alternates.

The current version of the Microsoft SR Engine does not support command and control alternates.

HRESULT GetMaxAlternates(
   ULONG   *pcMaxAlternates
);

Parameters

pcMaxAlternates
[out] The maximum number of alternates. The default value is zero, unless the application specifies it by ISpRecoContext::SetMaxAlternates.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pcMaxAlternates is invalid or bad.
FAILED(hr) | Appropriate error message.

Remarks

The value is the maximum number of alternates that will be returned to the application. For SR engine's that do not support command and control (or proprietary grammar) alternates, this method will succeed, but the alternates returned will always be zero.

For applications and SR Engines that are using proprietary grammars and proprietary alternates, ISpRecoContext::GetMaxAlternates and ISpRecoContext::SetMaxAlternates is the recommended method of coordinating maximum alternate values between the application and SR Engine.

The SR Engine can query each context's maximum requested alternates value by calling ISpSREngineSite::GetContextMaxAlternates with the context handle. When using SAPI command and control grammars, the SR engine can call ISpSREngineSite::GetMaxAlternates with the rule handle.


ISpRecoContext::SetMaxAlternates

ISpRecoContext::SetMaxAlternates sets the maximum number of alternates that the SR engine will return for command and control or proprietary grammars associated with this context. See Remarks section.

This method has no effect on dictation alternates. See ISpRecoResult::GetAlternates for information regarding dictation alternates.

The current version of the Microsoft speech recognition engine does not support command and control alternates.

HRESULT SetMaxAlternates(
   ULONG   cAlternates
);

Parameters

cAlternates
[in] Specifies the maximum number of alternates the engine will return.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.

Remarks

By default, the maximum alternates value is zero, so an application must call ::SetMaxAlternates to retrieve alternates.

The value is the maximum number of alternates that will be returned to the application. For SR engine's that do not support command and control (or proprietary grammar) alternates, this method will succeed, but the alternates returned will always be zero.

For applications and SR Engines using proprietary grammars and proprietary alternates, ISpRecoContext::GetMaxAlternates and ISpRecoContext::SetMaxAlternates are the recommended methods of coordinating maximum alternate values between the application and the SR Engine.

The SR Engine can query each context's maximum requested alternates value by calling ISpSREngineSite::GetContextMaxAlternates with the context handle. When using SAPI command and control grammars, the SR engine can call ISpSREngineSite::GetMaxAlternates with the rule handle.


ISpRecoContext::SetAudioOptions

ISpRecoContext::SetAudioOptions sets the audio options for result objects from this recognition context. This method also enables or disables the retention of audio with result objects and can change the retained audio format.

By default, SAPI does not retain recognition audio.

By default, when an audio format is not specified, the audio will be retained in the same format that the SR engine used to perform the recognition.

HRESULT SetAudioOptions(
   SPAUDIOOPTIONS        Options,
   const GUID           *pAudioFormatId,
   const WAVEFORMATEX   *pWaveFormatEx
);

Parameters

Options
[in] Flag of type SPAUDIOOPTIONS indicating the option. It must be one of the following:
Value
SPAO_NONE | Do not retain audio for results.
SPAO_RETAIN_AUDIO | Retain audio for all future results.
pAudioFormatId
[in] The audio stream format id [of type GUID]. Usually this value is SPDFID_WaveFormatEx. If this value is NULL, the retained audio format will not be changed. Reset the retained audio format to the SR engine's recognition format by setting this value to GUID_NULL and pWaveFormatEx to NULL.
pWaveFormatEx
[in] The audio stream wave format [of type WAVEFORMATEX]. This is only valid if *pAudioFormatId == SPFID_WaveFormatEx.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Options is not one of the correct types, or the specified audio format is not valid.
FAILED(hr) | Appropriate error message.

Remarks

If a WAVEFORMATEX-based retained audio format is specified, but the SR engine and the audio input stream agree on a non-WAVEFORMATEX-based audio input format (e.g., custom audio object/format), ISpRecoContext::SetAudioOptions will return successfully, but will not retain the audio.

Example

The following code snippet illustrates the use of ISpRecoContext::SetAudioOptions.

    HRESULT hr = S_OK;

    // activate retained audio settings with default format (i.e. SR engine recognition format)
    hr = cpRecoContext->SetAudioOptions(SPAO_RETAIN_AUDIO, NULL, NULL);
    // Check hr
    // deactivate retained audio settings
    hr = cpRecoContext->SetAudioOptions(SPAO_NONE, NULL, NULL);
    // Check hr
    // change the retained audio format to 11 kHz, 16-bit Stereo

    // use the stream format helper to fill in the WAVEFORMATEX structure
    CSpStreamFormat sfRetained(SPSF_24kHz16BitStereo, &hr);
    // Check hr

    // change the settings to the selected stream format
    hr = cpRecoContext->SetAudioOptions(SPAO_RETAIN_AUDIO, &sfRetained.FormatId(), sfRetained.WaveFormatExPtr());
    // Check hr

Development Helpers

Helper Enumerations and Functions | Description
SPSTREAMFORMAT | SAPI supported stream formats
CSpStreamFormat | Class for managing SAPI supported stream formats and WAVEFORMATEX structures


ISpRecoContext::GetAudioOptions

ISpRecoContext::GetAudioOptions retrieves the audio options for a given stream.

The default audio options are none (i.e., SPAO_NONE). The default retained audio format is the speech recognition engine's recognition format (see ISpRecognizer::GetFormat with SPWF_SRENGINE).

See also ISpRecoContext::SetAudioOptions.

HRESULT GetAudioOptions(
   SPAUDIOOPTIONS  *Options,
   GUID            *pAudioFormatId,
   WAVEFORMATEX   **ppCoMemWFE
);

Parameters

Options
[out] Address that will receive pointer to SPAUDIOOPTIONS flag, indicating the options set for this context. If this value is not to be retrieved, specify NULL. The flag can be one of the following values:
Value | Meaning
SPAO_NONE | Do not retain audio for results.
SPAO_RETAIN_AUDIO | Retain audio for all future results.
pAudioFormatId
[in] Address that will receive the audio stream format type (i.e., GUID). If the application is not interested in the retained audio format, NULL is specified (i.e., ignore both pAudioFormatId and pWaveFormatEx parameters).
ppCoMemWFE
[in] Address that will receive a pointer to the audio stream wave format structure (i.e., WAVEFORMATEX). This can be NULL if the application is not interested in the retained audio format. If WAVEFORMATEX data is retrieved, it must be freed using ::CoTaskMemFree().

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | One of the pointers is invalid or bad.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecoContext::GetAudioOptions and querying the different retained audio settings.

    HRESULT hr = S_OK;

    // check if audio is being retained (default is NO)
    hr = cpRecoContext->GetAudioOptions(&pAudioOptions, NULL, NULL);
    // Check hr

    // check what audio format would be retained
    hr = cpRecoContext->GetAudioOptions(NULL, &guidFormat, &pWaveFormatEx);
    // Check hr

    // ... do stuff

    // free the wave format memory
    ::CoTaskMemFree(pWaveFormatEx);

    // check if audio is being retained, and if so what the format is
    hr = cpRecoContext->GetAudioOptions(&guid, &guidFormat, &pWaveFormatEx);
    // Check hr

    // ... do stuff

    // free the wave format memory
    ::CoTaskMemFree(pWaveFormatEx);

Development Helpers

Helper Enumerations and Functions | Description
SPSTREAMFORMAT | SAPI supported stream formats
CSpStreamFormat | Class for managing SAPI supported stream formats and WAVEFORMATEX structures


ISpRecoContext::DeserializeResult

ISpRecoContext::DeserializeResult creates a new result object from a serialized result.

HRESULT DeserializeResult(
   const SPSERIALIZEDRESULT  *pSerializedResult,
   ISpRecoResult            **ppResult
);

Parameters

pSerializedResult
[in] Pointer to a serialized result. See also SPSERIALIZEDRESULT
ppResult
[out] The unserialized result object. The application must call IUnknown::Release when finished using the ISpRecoResult reference.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pSerializedResult is invalid or bad.
E_POINTER | ppResult is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.

Remarks

After deserializing the ISpRecoResult object, the application can retrieve alternates for the RecoResult, retrieve the retained audio, or examine the recognition.

The same SR engine that was originally used to generate the ISpRecoResult object (or recognition) must be the same SR engine that is associated with the ISpRecoContext that called ::DeserializeResult (see SPPHRASE.SREngineID). The SR engine requirement ensures that the SR engine is capable of recognizing the original phrase, and that it understands any SR engine private result data (see SPPHRASE.ulSREnginePrivateDataSize).

Example

The following code snippet illustrates the use ISpRecoContext::Deserialize a previously serialized result object.

    HRESULT hr = S_OK;

    // ... obtain a recognition result object from the recognizer...

    SPSERIALIZEDRESULT* pSerializedResult = NULL;
    ULONG cbWritten = 0;
    ULONG ulSerializedSize = 0;
    LARGE_INTEGER liseek;
    LARGE_INTEGER li;
    CComPtr<IStream> cpStreamWithResult;

    hr = CreateStreamOnHGlobal(NULL, true, &cpStreamWithResult);
    // Check hr

    // Serialize result to memory
    hr = cpRecoResult->Serialize(&pSerializedResult);
    // Check hr

    //serialized to a stream pointer
    hr = cpStreamWithResult->Write(pSerializedResult, pSerializedResult->ulSerializedSize, &cbWritten);
    // Check hr

    // free the serialized result
    if (pSerializedResult) ::CoTaskMemFree(pSerializedResult);

    // commit the stream changes
    hr = cpStreamWithResult->Commit(STGC_DEFAULT);
    // Check hr

    // ... persist stream to disk, network share, etc...
    // ... shutdown application ....

    // ... restart application and get the persisted stream

    // reset the stream seek pointer to the start before deserialization
    li.QuadPart = 0;
    hr = cpStreamWithResult->Seek(li, STREAM_SEEK_SET, NULL);
    // Check hr

    // find the size of the stream
    hr = cpStreamWithResult->Read(&ulSerializedSize, sizeof(ULONG), NULL);
    // Check hr

    // reset the seek pointer
    liseek.QuadPart = 0 - sizeof(ULONG);
    hr = cpStreamWithResult->Seek(liseek, STREAM_SEEK_CUR, NULL);
    // Check hr

    // allocate the memory for the result
    pSerializedResult = (SPSERIALIZEDRESULT*)::CoTaskMemAlloc(ulSerializedSize);
    // Check pSerializedResult in case out "out-of-memory"

    // copy the stream into a serialized result object
    hr = cpStreamWithResult->Read(pSerializedResult, ulSerializedSize, NULL);
    // Check hr

    // Deserialize result from memory
    hr = cpRecoContext->DeserializeResult(pSerializedResult, &cpRecoResultNew);
    // Check hr

    // free the pSerializedResult memory
    if (pSerializedResult) {
        CoTaskMemFree(pSerializedResult);
    }

    // As long as the same engine was used to generate
    // the original result object, as is now being used,
    // applications can now get alternates for the cpRecoResultNew's phrase


ISpRecoContext::Bookmark

ISpRecoContext::Bookmark sets a bookmark within the current recognition stream. When the engine reaches the specified stream position, a bookmark event is added to the event queue.

HRESULT Bookmark(
   SPBOOKMARKOPTIONS   Options,
   ULONGLONG           ullStreamPosition,
   LPARAM              lParamEvent
);

Parameters

Options
[in] Flags of type SPBOOKMARKOPTIONS indicating the options associated with the bookmark. Must be one of the following values:
SPBO_NONE | Context will not be paused when a bookmark event occurs.
SPBO_PAUSE | Context is paused when a bookmark event occurs. See Remarks section.
ullStreamPosition
[in] The position of the bookmark within the stream. See Remarks section.
SP_STREAMPOS_ASAP | Bookmark event will occur when the SR engine reaches a synchronization point.
SP_STREAMPOS_REALTIME | Bookmark event will occur when the SR engine reaches the current audio device position.
Specific stream position value | Bookmark event will occur when the SR engine reaches the specified stream position value.
lParamEvent
[in] The lparam for the SAPI bookmark event, and can be any value the application wants returned with the bookmark event (e.g., unique identifier, data pointer, NULL, etc.).

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Options has a bad value.
FAILED(hr) | Appropriate error message.

Remarks

If Options is set to SPBO_PAUSE, the SPEVENT wParam variable will be set to SPREF_AutoPause.

An application that wants to implement display a recognition progress/latency meter could use ISpRecoContext::Bookmark with SP_STREAMPOS_REALTIME, and update the UI when each bookmark is received. See also ISpRecognizer::GetStatus

An application that wants to pause the SR engine to perform specific processing could use ISpRecoContext::Bookmark with SPBO_PAUSE SP_STREAMPOS_ASAP, which will pause the SR engine automatically, and asynchronously for the application. The application should call ISpRecoContext::Resume to resume the recognition process. See also ISpRecoGrammar::SetRuleState regarding "auto-pause" rules.

Example

The following code snippet illustrates the use of ISpRecoContext::Bookmark with an "auto-pause" bookmark.

    HRESULT hr = S_OK;

    // setup the recognition context and grammar
    // ...

    // start listening for recognitions
    hr = cpRecoGrammar->SetRuleState( MY_RULE, NULL, SPRS_ACTIVE );
    // Check hr

    hr = cpRecoContext->Bookmark( SPBO_PAUSE, SP_STREAMPOS_ASAP, NULL );

    // get the bookmark event in a CSpEvent object
    // ...

    // assert that the recognition context paused after the "auto-pause" bookmark was sent
    SPDBG_ASSERT(spEvent.IsPaused());

    // Since the context was paused from the "auto-pause" rule, it must now be reactivated to recognize the second rule
    hr = cpRecoContext->Resume( NULL );
    // Check hr


ISpRecoContext::SetAdaptationData

ISpRecoContext::SetAdaptationData sends string data to the SR Engine to update the current language model.

HRESULT SetAdaptationData(
   const WCHAR   *pAdaptationData,
   const ULONG    cch
);

Parameters

pAdaptationData
[in] The string to adapt.
cch
[in] The number of characters in pAdaptationData.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pAdaptationData is invalid or cch equals zero.
E_OUTOFMEMORY | Exceeded available memory.
SPERR_SR_ENGINE_EXCEPTION | An exception was thrown by the SR Engine during ISpSREngine::SetAdaptationData.
FAILED(hr) | Appropriate error message.

Remarks

An application can improve recognition accuracy for dictating uncommon words, or uncommon word groupings, by training the SR engine for the new words, or word groupings, by creating or obtaining typical text and send the results to the engine using ::SetAdaptationData.

For example, a word processing application can give users the ability to train its users on previously created text documents. Similarly, an e-mail or collaboration application could train the SR engine on previously sent e-mail or collaborated documents. After training, the SR engine could perform better within specific domains of information, and improve the application's and SAPI's personalization experience.

The SAPI middleware component (and Microsoft) do not store this information - instead it is passed directly to the SR Engine using ISpSREngine::SetAdaptationData. The SR engine (vendor) determines what to do with the string data, including how the data is persisted and for how long. Some SR engines may require a significant amount of time to process the string data, so the application may want to break up the data in smaller chunks to send individually.

Applications that use SetAdaptationData should break the data into small (1K or less) blocks, call SetAdaptationData, and then wait for an SPEI_ADAPTATION event before sending the next small block of data. This method should not be used with large buffers, because calling SetAdapataionData with buffers larger than 32K can cause unpredictable results.

When the SR engine is ready to receive more string data to adapt its language model, it can fire the SPEI_ADAPTATION event to the application.

Example

The following code snippet illustrates the use of ISpRecoContext::SetAdaptationData

    HRESULT hr = S_OK;

    // get the "training" data, and break it into manageable chunks [e.g. an array of strings]
    // ...

    // set interest in the adaptation event
    hr = cpRecoContext->SetInterest(SPFEI(SPEI_ADAPTATION), SPFEI(SPEI_ADAPTATION));
    // Check hr

    // adapt to each chunk of data
    for (int i = 0; i < iCountOfDataChunk; i ++)
    {
       // send each chunk of data the engine
       hr = cpRecoContext->SetAdaptationData(ppwszAdaptationData[i], wcslen(ppwszAdaptationData[i]));
       // Check hr

       // wait for the engine to ask for more data
       hr = cpRecoContext->WaitForNotifyEvent(PROCESSING_WAIT_TIME);
       // Check hr
    }

    // SR Engine has adaptated its language model to data


ISpRecoContext::Pause

ISpRecoContext::Pause pauses the engine object to synchronize with the SR engine.

The SR engine pauses at its synchronization point to allow grammars and rule states to be changed freely. The engine remains paused until the Resume method is called.

The caller must call Resume once for every call that is made to Pause.

HRESULT Pause(
   DWORD   dwFlags
);

Parameters

dwFlags
[in] Reserved, must be zero.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | dwFlags is not set to zero.

Remarks (when using ISpMMSysAudio as audio input)

Pausing the SR engine will stop recognition, but input audio will continue to be collected and stored by SAPI in an audio buffer. After the application is done with the state change, it should call ISpRecoContext::Resume. SAPI will automatically feed the buffered audio data into the SR engine, ensuring that no real-time audio data is lost and that the user experience is not interrupted.

However, the SAPI audio buffer has a static limit (see ISpMMSysAudio::Read) to prevent large amounts of system memory from being consumed by SAPI applications or SR Engines. Therefore, Pause should be used by the SR Engine for very short periods of time for state changes (e.g., updating grammar, or rule states). Pausing the SR engine will affect all recognition contexts connected to that SR Engine including other Speech applications currently running.

If the SR engine is paused too long, and the audio buffer is filled, a buffer overflow will occur. The application can detect this error by setting an event interest in SPEI_END_SR_STREAM (see SPEVENTENUM), and checking the LPARAM of the SPEVENT structure (see CSpEvent::EndStreamResult).

SAPI will automatically attempt to restart the SR Engine's recognition thread once the final Resume has been called. Consequently, the audio data collected between the point when the buffer overflow occurred, and when the stream was reactivated, will be completely lost. This would result in a less than optimal user experience, and have a negative effect on all running speech applications, the SR Engine, and SAPI.

The following code snippet illustrates the use of ISpRecoContext::Pause

    HRESULT hr = S_OK;

    // setup the recognition context
    // ...

    // pause the context so that event notifications are not received
    hr = cpRecoContext->Pause( NULL );
    // Check hr

    // [quickly] perform the processing - see ISpRecoContext::Pause Remarks section
    // ...

    hr = cpRecoContext->Resume( NULL );
    // Check hr

    // applications will start receiving event notifications again


ISpRecoContext::Resume

ISpRecoContext::Resume releases the SR engine from the paused state and restarts the recognition process.

This method must be called after a call to ISpRecoContext::Pause, a bookmark event occurs that pauses the recognition engine, or an auto-pause rule is recognized (see ISpRecoGrammar::SetRuleState).

The caller must call Resume once for every call that is made to ISpRecoContext::Pause.

HRESULT  Resume (
  DWORD dwReserved
);

Parameters

dwReserved
[in] Reserved, must be 0.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | dwFlags is not set to zero.

Example

The following code snippet illustrates the use of ISpRecoContext::Resume after a call to ISpRecoContext::Pause

    HRESULT hr = S_OK;

    // setup the recognition context
    // ...

    // pause the context so that event notifications are not received
    hr = cpRecoContext->Pause( NULL );
    // Check hr

    // [quickly] perform the processing - see ISpRecoContext::Pause Remarks section
    // ...

    hr = cpRecoContext->Resume( NULL );
    // Check hr

    // applications will start receiving event notifications again

The following code snippet illustrates the use of ISpRecoContext::Resume with an "auto-pause" rule.

    HRESULT hr = S_OK;

    // setup the recognition context and grammar
    // ...

    // activate a top-level rule as an "auto-pause" rule
    hr = cpRecoGrammar->SetRuleState( MY_AUTOPAUSE_RULE, NULL, SPRS_ACTIVE_WITH_AUTO_PAUSE);
    // Check hr

    // get the recognition event for MY_AUTOPAUSE_RULE in a CSpEvent object
    // ...

    // assert that the recognition context paused after the "auto-pause" rule was recognized
    SPDBG_ASSERT(spEvent.IsPaused());

    // deactivate the "auto-pause" rule
    hr = cpRecoGrammar->SetRuleState( MY_AUTOPAUSE_RULE, NULL, SPRS_INACTIVE );
    // Check hr

    // activate the second rule
    hr = cpRecoGrammar->SetRuleState( MY_SECOND_RULE, NULL, SPRS_ACTIVE );
    // Check hr

    // Since the context was paused from the "auto-pause" rule, it must now be reactivated to recognize the second rule
    hr = cpRecoContext->Resume( NULL );
    // Check hr

    // get the second recognition...


ISpRecoContext::SetVoice

ISpRecoContext::SetVoice sets the associated ISpVoice to an object.

HRESULT SetVoice(
   ISpVoice   *pVoice,
   BOOL        fAllowFormatChanges
);

Parameters

pVoice
[in] The voice interface to be associated. If NULL, the currently associated Voice is Released.
fAllowFormatChanges
[in] Boolean allowing the voice format alteration by the engine. See Remarks section.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pVoice is invalid or bad.
FAILED(hr) | Appropriate error message.

Remarks

If fAllowFormatChanges is TRUE, the Voice's output format will be the same format as the associated SR Engine's audio input format (see ISpRecognizer and ISpSREngine::GetInputAudioFormat). Using the same audio format for input and output source is useful for sound cards that do not support full-duplex audio (i.e., input format must match output format). If the input format quality is lower than the output format quality, the output format quality will be reduced to equal the input quality.

After calling ISpRecoContext::SetVoice, an application that calls ISpRecoContext::GetVoice will retrieve the originally "set" ISpVoice interface pointer.

Example

The following code snippet illustrates the use of ISpRecoContext::SetVoice and "barge-in" setup

    HRESULT hr = S_OK;

    // create a shared recognition context
    hr = cpRecoContext.CoCreateInstance(CLSID_SpSharedRecoContext);
    // Check hr

    // create a voice
    hr = cpVoice.CoCreateInstance(CLSID_SpVoice);
    // Check hr

    // associate the voice with the context (with same audio format as context)
    hr = cpRecoContext->SetVoice(cpVoice, TRUE);
    // Check hr

    // tell the associated Voice to stop speaking when the SR Engine hears a recognizable sound
    hr = cpRecoContext->SetVoicePurgeEvent(SPFEI(SPEI_SOUND_START));
    // Check hr


ISpRecoContext::GetVoice

ISpRecoContext::GetVoice retrieves a reference to an ISpVoice object that is associated with the ISpRecoContext object.

HRESULT GetVoice(
   ISpVoice   **ppVoice
);

Parameters

ppVoice
[in] Address of the ISpVoice interface. IUnknown::Release must be called on the ISpVoice interface when finished.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.
FAILED(hr) | Appropriate error message.

Remarks

If an application previously called ISpRecoContext::SetVoice on the same ISpRecoContext object, the Voice interface retrieved from GetVoice will match that of the SetVoice call. Release must still call the ISpVoice reference for each GetVoice call, even though the interface pointer is the same.

The ISpVoice's output format will be the same format as the associated SR Engine's audio input format (see ISpRecognizer and ISpSREngine::GetInputAudioFormat). Using the same audio format for input and output source is useful for sound cards that do not support full-duplex audio (i.e., input format must match output format). If the input format quality is lower than the output format quality, the output format quality will be down-sampled to the lower quality.

Applications implementing a "barge-in" type functionality will need to tie the Voice object to the Speech Recognition object. Applications can also use ISpRecoContext::GetVoice (see ISpRecoContext::SetVoicePurgeEvent).

Related Samples

The CoffeeS3 Sample application uses the ISpRecoContext::GetVoice method and "barge-in" functionality.

Example

The following code snippet illustrates the use of ISpRecoContext::GetVoice and "barge-in" setup

    HRESULT hr = S_OK;

    // create a shared recognition context
    hr = cpRecoContext.CoCreateInstance(CLSID_SpSharedRecoContext);
    // Check hr

    // create a voice from the context (with same audio format as context)
    hr = cpRecoContext->GetVoice(&cpVoice);
    // Check hr

    // tell the associated Voice to stop speaking when the SR Engine hears a recognizable sound
    hr = cpRecoContext->SetVoicePurgeEvent(SPFEI(SPEI_SOUND_START));
    // Check hr


ISpRecoContext::SetVoicePurgeEvent

ISpRecoContext::SetVoicePurgeEvent sets the SR engine events that stop audio output, and purges the current speaking queue.

Applications can call SetVoicePurgeEvent when implementing "barge-in" type functionality. For example, when a user calls a telephony server, and the server uses TTS Voice prompts, the Voice should stop speaking when the user is speaking. The application would want the ISpRecoContext's associated Voice object (see ISpRecoContext::GetVoice) to stop and purge when the SR Engine hears a sound (see SPEI_SOUND_START).

HRESULT SetVoicePurgeEvent(
   ULONGLONG   ullEventInterest
);

Parameters

ullEventInterest
[in] The set of flags indicating the event interest(s). The event interest(s) must be in the set of Speech Recognition events (i.e., between SPEI_MIN_SR and SPEI_MAX_SR) (see SPEVENTENUM and SPFEI_ALL_SR_EVENTS)

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more of the event interests set is not allowed.
FAILED(hr) | Appropriate error message.

Remarks

The ISpRecoContext event interest will be updated to include the Voice Purge Events (see ISpEventSource::SetInterest).

To find the current Voice Purge Event(s), use ISpRecoContext::GetVoicePurgeEvent

Example

The following code snippet illustrates the use of ISpRecoContext::SetVoicePurgeEvent and "barge-in" setup

    HRESULT hr = S_OK;

    // create a shared recognition context
    hr = cpRecoContext.CoCreateInstance(CLSID_SpSharedRecoContext);
    // Check hr

    // create a voice from the context (with same audio format as context)
    hr = cpRecoContext->GetVoice(&cpVoice);
    // Check hr

    // tell the associated Voice to stop speaking when the SR Engine hears a recognizable sound
    hr = cpRecoContext->SetVoicePurgeEvent(SPFEI(SPEI_SOUND_START));
    // Check hr


ISpRecoContext::GetVoicePurgeEvent

ISpRecoContext::GetVoicePurgeEvent retrieves the set of SR engine events that stop audio output, and purges the current speaking queue. The events are set by ISpRecoContext::SetVoicePurgeEvent.

HRESULT GetVoicePurgeEvent(
   ULONGLONG   *pullEventInterest
);

Parameters

pullEventInterest
[out] The set of flags indicating the event interests. The event interests will be a member of the SR event set (see SPFEI_ALL_SR_EVENTS).

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pullEventInterest is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpRecoContext::SetContextState

ISpRecoContext::SetContextState sets the state of the recognition context.

HRESULT SetContextState(
   SPCONTEXTSTATE   eContextState
);

Parameters

eContextState
[in] The SPCONTEXTSTATE enumeration sequence specifying the recognition context state.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | eContextState is not one of the correct types.
FAILED(hr) | Appropriate error message.

Remarks

The default recognition context state for an ISpRecoContext object is SPCS_ENABLED.

Applications can use ISpRecoContext::SetContextState to toggle sets of grammars. For example, a multi-document interface application that uses a different ISpRecoContext object for each document could toggle the context states as each document gained and lost the focus.

Applications can use ISpRecoContext::GetContextState to query the context state.


ISpRecoContext::GetContextState

ISpRecoContext::GetContextState retrieves the state of the recognition context.

HRESULT GetContextState(
   SPCONTEXTSTATE   *peContextState
);

Parameters

peContextState
[out] Address of the SPCONTEXTSTATE enumeration that receives the context state information.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | One of the pointers is invalid or bad.
FAILED(hr) | Appropriate error message.

Remarks

The default recognition context state for an ISpRecoContext object is SPCS_ENABLED.

Applications can use ISpRecoContext::SetContextState to set the context state.


ISpRecoGrammar

The ISpRecoGrammar interface enables applications to manage the words and phrases that the SR engine will recognize.

A single SpRecognizer object can have multiple SpRecoContext objects associated with it. And similarly, a single SpRecoContext object can have multiple SpRecoGrammar objects associated with it. Using a one-to-many relationship with SpRecoContext objects and SpRecoGrammar objects allows applications to separate types of recognizable phrases and content into separate objects for clearer application logic. Each SpRecoGrammar object can also have a context-free grammar (CFG) and a dictation grammar loaded simultaneously (e.g., use the CFG if possible, but back off to dictation if CFG fails to parse).

See "Designing Grammar Rules" for examples of how to create context-free-grammars.

Methods in Vtable Order

ISpRecoGrammar Methods | Description
ISpGrammarBuilder | Inherits from ISpGrammarBuilder and all those methods are accessible from an ISpRecoGrammar object.
GetGrammarId | Retrieves the grammar identifier associated with the application.
GetRecoContext | Retrieves the context object that created this grammar.
LoadCmdFromFile | Loads a command and control grammar from a file.
LoadCmdFromObject | Loads a command and control grammar from a COM object.
LoadCmdFromResource | Loads a command and control grammar from a Win32 resource.
LoadCmdFromMemory | Loads a command and control grammar from memory.
LoadCmdFromProprietaryGrammar | Loads an engine proprietary format command and control grammar.
SetRuleState | Activates or deactivates a rule by its rule name.
SetRuleIdState | Activates or deactivates a rule by its rule ID.
LoadDictation | Loads and initializes a dictation topic.
UnloadDictation | Unloads the active dictation topic from the grammar.
SetDictationState | Sets a dictation state to active or inactive.
SetWordSequenceData | Sets a word sequence buffer in the SR engine.
SetTextSelection | Sets the insertion point (using word sequence data buffer).
IsPronounceable | Determines if the word has a pronunciation.
SetGrammarState | Sets the grammar state.
SaveCmd | Allows applications using dynamic grammars to save the current grammar state to a stream.
GetGrammarState | Retrieves recognition grammar state information.


ISpRecoGrammar::GetGrammarId

ISpRecoGrammar::GetGrammarId retrieves the identifier associated with the grammar when the grammar was created.

The grammar ID is set by the application by calling ISpRecoContext::CreateGrammar.

HRESULT GetGrammarId(
   ULONGLONG   *pullGrammarId
);

Parameters

pullGrammarId
[out] Address of a ULONGLONG variable to receive the grammar ID.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pullGrammarId is invalid or bad.

Remarks

The grammar ID will be set in the SPPHRASE object when a recognition is sent to the application (see ISpPhrase::GetPhrase).


ISpRecoGrammar::GetRecoContext

ISpRecoGrammar::GetRecoContext retrieves the ISpRecoContext object that created this grammar. If this method succeeds, the application using this method must call Release() on the SpRecoContext object returned.

HRESULT GetRecoContext(
   ISpRecoContext   **ppRecoCtxt
);

Parameters

ppRecoCtxt
[out] Address of a pointer to an ISpRecoContext interface that receives the recognition context object pointer.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppRecoCtxt is invalid or bad.


ISpRecoGrammar::LoadCmdFromFile

ISpRecoGrammar::LoadCmdFromFile loads a SAPI 5 command and control grammar from a file. The file can either be a compiled or uncompiled grammar file. To modify the rules of the grammar after it has been loaded, specify SPLO_DYNAMIC for the Options parameter, otherwise specify the SPLO_STATIC flag.

HRESULT LoadCmdFromFile(
    const WCHAR    *pszFileName,
    SPLOADOPTIONS   Options
);

Parameters

pszFileName
[in, string] The name of the file containing the command and control grammar. SAPI 5 support loading of compiled and static grammars using URL.
Options
[in] Flag of type SPLOADOPTIONS indicating whether the grammar will be modified dynamically.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszFileName is invalid or bad. Alternatively, Options is neither SPLO_STATIC nor SPLO_DYNAMIC.
FAILED(hr) | Appropriate error message.


ISpRecoGrammar::LoadCmdFromObject

ISpRecoGrammar::LoadCmdFromObject loads a CFG from a COM object.

See also ISpCFGInterpreter::InitGrammar.

The COM object must be located inside of a Windows DLL.

HRESULT LoadCmdFromObject(
   REFCLSID        rcid,
   const WCHAR    *pszGrammarName,
   SPLOADOPTIONS   Options
);

Parameters

rcid
[in] The reference class ID of the object containing the command.
pszGrammarName
[in, string] The grammar name of the object containing the command.
Options
[in] Flag of type SPLOADOPTIONS indicating whether the file should be loaded statically or dynamically.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszGrammarName is invalid or bad. Alternatively, Options is neither SPLO_STATIC nor SPLO_DYNAMIC.
FAILED(hr) | Appropriate error message.

Remarks

When an application calls ::LoadCmdFromObject, the currently loaded CFG or proprietary grammar will be unloaded.


ISpRecoGrammar::LoadCmdFromResource

ISpRecoGrammar::LoadCmdFromResource loads a command and control grammar from a Win32 resource.

HRESULT LoadCmdFromResource(
   HMODULE         hModule,
   const WCHAR    *pszResourceName,
   const WCHAR    *pszResourceType,
   WORD            wLanguage,
   SPLOADOPTIONS   Options
);

Parameters

hModule
[in] Handle to the module whose file name is being requested. If this parameter is NULL, it passes back the path for the file containing the current process.
pszResourceName
[in, string] The name of the resource.
pszResourceType
[in, string] The type of the resource.
wLanguage
[in] The language ID.
Options
[in] Flag of type SPLOADOPTIONS indicating whether the file should be loaded statically or dynamically.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Either pszResourceName or pszResourceType is invalid or bad. It may also indicate hModule could not be found. Alternatively, Options is neither SPLO_STATIC nor SPLO_DYNAMIC.
FAILED(hr) | Appropriate error message.

Remarks

When an application calls ::LoadCmdFromResource, the currently loaded context free grammar or proprietary grammar will be unloaded.

The CFG resource must be a compiled SAPI 5 binary version of a context free grammar (see ISpGrammarCompiler::CompileStream).

Example

    HRESULT hr = S_OK;

    // create a new grammar object
    hr = cpRecoContext->CreateGrammar(GRAM_ID, &cpRecoGrammar);
    // Check hr

    // load a CFG resource from the current module, named SRGRAMMAR
    hr = cpRecoGrammar->LoadCmdFromResource(hModule,
                                            (const WCHAR*)MAKEINTRESOURCE(IDR_GRAMMAR),
                                            L"SRGRAMMAR",
                                            ::SpGetUserDefaultUILanguage()
                                            , SPLO_STATIC);
    // Check hr


ISpRecoGrammar::LoadCmdFromMemory

ISpRecoGrammar::LoadCmdFromMemory loads a compiled CFG binary from memory.

HRESULT LoadCmdFromMemory(
   const SPBINARYGRAMMAR   *pBinaryData,
   SPLOADOPTIONS            Options
);

Parameters

pBinaryData
[in] The serialized header buffer of type SPBINARYGRAMMAR.
Options
[in] Flag of type SPLOADOPTIONS indicating whether the file should be loaded statically or dynamically.

Remarks

When an application calls ::LoadCmdFromMemory, the currently loaded CFG or proprietary grammar will be unloaded.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Either pBinaryData or one of its members is invalid or bad. It may also indicate pBinaryData->FormatId is not SPGDF_ContextFree. Alternatively, Options is neither SPLO_STATIC nor SPLO_DYNAMIC.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates how to use ISpRecoGrammar::LoadCmdFromMemory to serialize the CFG from one SpRecoGrammar object and deserialize it into another SpRecoGrammar object.

    HRESULT hr = S_OK;

    // ... build and use a SpRecoGrammar object

    // create a Win32 global stream
    hr = ::CreateStreamOnHGlobal(NULL, true, &cpHStream);
    // Check hr

    // save the current grammar to the global stream
    hr = cpRecoGrammar->SaveCmd(cpHStream, NULL);
    // Check hr

    // create the second grammar to deserialize into
    hr = cpRecoContext->CreateGrammar(0, &cpReloadedGrammar);
    // Check hr

    // get a handle to the stream with the serialized grammar
    ::GetHGlobalFromStream(cpHStream, &hGrammar);
    // Check hr

    // deserialize the CFG into a new grammar object
    hr = cpReloadedGrammar->LoadCmdFromMemory((SPBINARYGRAMMAR *)::GlobalLock(hGrammar), SPLO_DYNAMIC);
    // Check hr


ISpRecoGrammar::LoadCmdFromProprietaryGrammar

ISpRecoGrammar::LoadCmdFromProprietaryGrammar loads a proprietary grammar.

Applications should use ::LoadCmdFromProprietaryGrammar when using a proprietary grammar format that the SR engine supports (see ISpSREngine::LoadProprietaryGrammar). If the current SR engine does not support the proprietary grammar format (specified using rguidParam and pszStringParam), the SR engine may return E_NOTIMPL.

HRESULT LoadCmdFromProprietaryGrammar(
   REFGUID         rguidParam,
   const WCHAR    *pszStringParam,
   const void     *pvDataParam,
   ULONG           cbDataSize,
   SPLOADOPTIONS   Options
);

Parameters

rguidParam
[in] Unique identifier of the grammar. The GUID will be used by the application and the SR engine to uniquely identify the SR engine for verifying support.
pszStringParam
[in, string] The null-terminated string command. The string can be used by the application and the SR engine to specify which part of a grammar to utitlize.
pvDataParam
[in] Additional information for the process. SAPI will handle the marshaling of the data to the SR engine.
cbDataSize
[in] The size, in bytes, of pvDataParam. SAPI will handle the marshaling of the data to the SR engine.
Options
[in] Flag of type SPLOADOPTIONS indicating whether the file should be loaded statically or dynamically. This value must be SPLO_STATIC.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszStringParam or pvDataParam is invalid or bad. Alternatively, Options is not SPLO_STATIC.
FAILED(hr) | Appropriate error message.

Remarks

When an application calls ::LoadCmdFromProprietaryGrammar, the currently loaded CFG or proprietary will be unloaded.

Example

    HRESULT hr = S_OK;

    // create a new grammar object
    hr = cpRecoContext->CreateGrammar(GRAM_ID, &cpRecoGrammar);
    // Check hr

    // load our proprietary grammar
    hr = cpRecoGrammar->LoadCmdFromProprietaryGrammar(GUID_MyProprietaryGrammarType, NULL, bGrammarData, ulSizeOfGrammarData, SPLO_STATIC);
    // Check hr


ISpRecoGrammar::SetRuleState

ISpRecoGrammar::SetRuleState activates or deactivates a rule by its rule name.

The rule name is specified in the XML grammar (using the rule NAME tag), or when ISpGrammarBuilder::GetRule is called.

See also ISpSREngine::RuleNotify for information on the how SAPI notifies the SR engine.

HRESULT SetRuleState(
   const WCHAR   *pszName,
   void          *pReserved,
   SPRULESTATE    NewState
);

Parameters

pszName
[in, string] Address of a null-terminated string containing the rule name. If NULL, all rules with attribute SPRAF_TopLevel and SPRAF_Active and set (at rule creation time) are affected.
pReserved
Reserved. Do not use; must be NULL.
NewState
[in] Flag of type SPRULESTATE indicating the new rule state. See Remarks section.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszName is invalid or bad. Alternatively, pReserved is non-NULL.
SP_STREAM_UNINITIALIZED | ISpRecognizer::SetInput has not been called with the InProc recognizer
SPERR_UNINITIALIZED | The object has not been properly initialized.
SPERR_UNSUPPORTED_FORMAT | Audio format is bad or is not recognized. Alternatively, the device driver may be busy by another application and cannot be accessed.
SPERR_NOT_TOPLEVEL_RULE | The rule pszName exists, but is not a top-level rule.
FAILED(hr) | Appropriate error message.

Remarks

An application can use the SPRS_ACTIVE_WITH_AUTO_PAUSE state to pause the engine after each CFG recognition is sent. The application must reactivate the SR engine (see ISpRecoContext::Resume) to prevent the loss of input audio data (see ISpSREngineSite::Read and SPERR_AUDIO_BUFFER_OVERFLOW).

By default, the recognizer state (SPRECOSTATE) is SPRST_ACTIVE, and the recognition will begin as soon as one or more rule are activated. Consequently, an application should not activate the rule state until it is prepared to receive recognitions. An application can also disable the SpRecoContext object (see ISpRecoContext::SetContextState) or SpRecoGrammar objects (see ISpRecoGrammar::SetGrammarState) to prevent recognitions from being fired for active rules.

If the recognizer state is SPRST_ACTIVE, SAPI will first attempt to open the audio input stream when dictation (or a rule) is activated. Consequently, if the audio device is already in use by another application, or the stream fails to open, the failure code will be returned using ::SetRuleState. The application should handle this failure gracefully.

If an application uses an InProc recognizer, it must call ISpRecognizer::SetInput with a non-NULL setting before the recognizer will return recognitions, regardless of how many rules are active.

Example

The following snippet loads a grammar, activate a single rule ("playcard") and then immediately deactivates it.
    HRESULT hr;

         // create a grammar object
         hr = cpRecoContext->CreateGrammar(GRAM_ID, &cpRecoGrammar);
    //Check return value

         // activate the rule
    hr = cpRecoGrammar->SetRuleState(L"playcard", NULL, SPRS_ACTIVE);
    //Check return value

    //Deactivate the rule
    hr = cpRecoGrammar->SetRuleState(L"playcard", NULL, SPRS_INACTIVE);
    //Check return value


ISpRecoGrammar::SetRuleIdState

ISpRecoGrammar::SetRuleIdState activates or deactivates a rule by its rule ID.

The rule ID is specified in the XML grammar (using the rule ID tag), or when ISpGrammarBuilder::GetRule is called.

See also ISpSREngine::RuleNotify for information on the how SAPI notifies the SR engine.

HRESULT SetRuleIdState(
   ULONG         ulRuleId,
   SPRULESTATE   NewState
);

Parameters

ulRuleId
[in] Value specifying the grammar rule identifier. If zero, all rules with attribute SPRAF_TopLevel and SPRAF_Active and set (at rule creation time) are affected.
NewState
[in] Flag of type SPRULESTATE indicating the new rule state.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | dwRuleId is invalid.
SP_STREAM_UNINITIALIZED | ISpRecognizer::SetInput has not been called with the InProc recognizer.
SPERR_UNINITIALIZED | The object has not been properly initialized.
SPERR_UNSUPPORTED_FORMAT | Audio format is bad or is not recognized. Alternatively, the device driver may be busy by another application and cannot be accessed.
SPERR_DEVICE_BUSY | The audio
SPERR_NOT_TOPLEVEL_RULE | The rule ID ulRuleId exists, but is not a top-level rule.
FAILED(hr) | Appropriate error message.

Remarks

An application can use the SPRS_ACTIVE_WITH_AUTO_PAUSE state to pause the engine after each dictation recognition is sent. The application must reactivate the SR engine (see ISpRecoContext::Resume) to prevent the loss of input audio data (see ISpSREngineSite::Read and SPERR_AUDIO_BUFFER_OVERFLOW).

By default, the recognizer state (SPRECOSTATE) is SPRST_ACTIVE, which means that recognition will begin as soon as dictation is activated. Consequently, an application should not activate the dictation state until it is prepared to receive recognitions. An application can also disable the SpRecoContext object (see ISpRecoContext::SetContextState) or SpRecoGrammar objects (see ISpRecoGrammar::SetGrammarState) to prevent recognitions from being fired for active dictation topics.

If the recognizer state is SPRST_ACTIVE, SAPI will first attempt to open the audio input stream when a rule (or dictation) is activated. Consequently, if the audio device is already in use by another application, or the stream fails to open, the failure code will be returned using ::SetRuleIdState. The application should handle this failure gracefully.

If an application uses an InProc recognizer, it must call ISpRecognizer::SetInput with a non-NULL setting before the recognizer will return recognitions, regardless of the dictation topic state.

Example

The following code snippet illustrates the use of ISpRecoGrammar::SetRuleIdState by programmatically creating a rule and activating it

    HRESULT hr = S_OK;

    // create a new rule
    SPSTATEHANDLE hStateTravel;
    hr = cpRecoGrammar->GetRule(MYRULENAME, MY_RULE_ID, SPRAF_TopLevel | SPRAF_Active, TRUE, &hState);
    // Check hr

    // .. add word transitions...

    // activate the rule by its id
    hr = cpRecoGrammar->SetRuleIdState(MY_RULE_ID, SPRS_ACTIVE);
    // Check hr


ISpRecoGrammar::LoadDictation

ISpRecoGrammar::LoadDictation loads a dictation topic into the SpRecoGrammar object and the SR engine.

See also ISpSREngine::OnCreateGrammar.

HRESULT LoadDictation(
   const WCHAR   *pszTopicName,
   SPLOADOPTIONS  Options
);

Parameters

pszTopicName
[in, optional, string] The null-terminated string containing the topic name. If NULL, the general dictation is loaded. See Remarks section.
Options
[in] Flag of type SPLOADOPTIONS indicating whether the file should be loaded statically or dynamically. This value must be SPLO_STATIC.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pszTopicName is invalid or bad. Alternatively, Options is not SPLO_STATIC.
FAILED(hr) | Appropriate error message.

Remarks

SAPI currently defines one specialized dictation topic: SPTOPIC_SPELLING. SR engines are not required to support specialized dictation topic (including spelling).

See the SR engine vendor for information on what specialized dictation topics if any are supported.

Example

The following code snippet illustrates the use of ISpRecoGrammar::LoadDictation to load a spelling topic and activate it.

    HRESULT hr = S_OK;

    // create a grammar object
    hr = cpRecoContext->CreateGrammar(GRAM_ID, &cpRecoGrammar);
    // Check hr

    // load the general dictation topic
    hr = cpRecoGrammar->LoadDictation(NULL, SPLO_STATIC);
    // Check hr

    // activate the dictation topic to receive recognitions
    hr = cpRecoGrammar->SetDictationState(SPRS_ACTIVE);
    // check hr


ISpRecoGrammar::UnloadDictation

ISpRecoGrammar::UnloadDictation unloads the dictation grammar from the current SpRecoGrammar object.

HRESULT  UnloadDictation  ( void );

Parameters

None.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.

Remarks

If an application uses a CFG with dictation tags, and then unloads the dictation grammar component, the dictation tags will default to the generic dictation topic (see ISpRecoGrammar::LoadDictation).


ISpRecoGrammar::SetDictationState

ISpRecoGrammar::SetDictationState sets the dictation topic state.

The dictation topic is specified by calling ISpRecoGrammar::LoadDictation.

See also ISpSREngine::SetSLMState for information on how SAPI notifies the SR engine.

HRESULT SetDictationState(
   SPRULESTATE   NewState
);

Parameters

NewState
[in] Flag of type SPRULESTATE indicating the new state of dictation. See Remarks section

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | NewState is not an acceptable value.
SP_STREAM_UNINITIALIZED | ISpRecognizer::SetInput has not been called with the InProc SR engine
SPERR_UNINITIALIZED | A dictation is not currently loaded.
SPERR_UNSUPPORTED_FORMAT | Audio format is bad or is not recognized. Alternatively, the device driver may be busy by another application and cannot be accessed.
FAILED(hr) | Appropriate error message.

Remarks

An application can use the SPRS_ACTIVE_WITH_AUTO_PAUSE state to pause the engine after each dictation recognition is sent. The application must reactivate the SR engine (see ISpRecoContext::Resume) to prevent the loss of input audio data (see ISpSREngineSite::Read and SPERR_AUDIO_BUFFER_OVERFLOW).

By default, the recognizer state (SPRECOSTATE) is SPRST_ACTIVE, which means that recognition will begin as soon as dictation is activated. Consequently, an application should not activate the dictation state until it is prepared to receive recognitions. An application can also disable the SpRecoContext object (see ISpRecoContext::SetContextState) or SpRecoGrammar objects (see ISpRecoGrammar::SetGrammarState) to prevent recognitions from being fired for active dictation topics.

If the recognizer state is SPRST_ACTIVE, SAPI will first attempt to open the audio input stream when dictation (or a rule) is activated. Consequently, if the audio device is already in use by another application, or the stream fails to open, the failure code will be returned using ::SetDictationState. The application should handle this failure gracefully.

If an application uses an InProc recognizer, it must call ISpRecognizer::SetInput with a non-NULL setting before the recognizer will return recognitions, regardless of the dictation topic state.

Example

The following code snippet illustrates the use of ISpRecoGrammar::SetDictationState to load a spelling topic and activate it.

    HRESULT hr = S_OK;

    // create a grammar object
    hr = cpRecoContext->CreateGrammar(GRAM_ID, &cpRecoGrammar);
    // Check hr

    // load the general dictation topic
    hr = cpRecoGrammar->LoadDictation(NULL, SPLO_STATIC);
    // Check hr

    // activate the dictation topic to receive recognitions
    hr = cpRecoGrammar->SetDictationState(SPRS_ACTIVE);
    // check hr


ISpRecoGrammar::SetWordSequenceData

ISpRecoGrammar::SetWordSequenceData sets a word sequence buffer in the SR engine. The command and control grammar can refer to any subsequence of words in this buffer using the <TEXTBUFFER> tag, or the SPRULETRANS_TEXTBUFFER special transition type in ISpGrammarBuilder::AddRuleTransition().

See also ISpRecoGrammar::SetTextSelection for information on how to update the text selection information independent of the word sequence data.

See also ISpSREngine::SetWordSequenceData for information on how SAPI passes the word sequence data to the SR engine.

HRESULT SetWordSequenceData(
   const WCHAR   *pText,
   ULONG          cchText,
   const  SPTEXTSELECTIONINFO  *pInfo
);

Parameters

pText
[in] Buffer containing the text to search for possible word sequences. The buffer is double-NULL terminated. The whole buffer could be separated into different groups by '\0'. Any sub-sequence of words in the same group is recognizable, any sub-sequence of words across different groups is not recognizable. Word could be in simple format or complex format: /disp/lex/pron. It is up to the SR engines to perform word breaking and text normalization for better performance. For example: If the buffer dispalys: "please play this\0new game\0\0", "please play" is recognizable, while "this new game" is not recognizable.
cchText
[in] The number of characters (WCHAR) in pText.
pInfo
[optional, in] Address of the SPTEXTSELECTIONINFO structure that contains the selection information. If NULL, the SR engine will use the entire contents of pText.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
FAILED(hr) | Appropriate error message.

Remarks

An application that has a text box could allow the user to speak commands into the text box to edit the text. One way to design this functionality would be to create a CFG which supports such commands as "cut the text *", "bold the text *", or "italicize the words *". The grammar would then use a TEXTBUFFER tag in place of the * which would allow the SR engine to recognize the text buffer information. At runtime, the application would update the SR engine's view of the text buffer using ::SetWordSequenceData. So if a user had the text "hello world" in the text box, the SR engine could recognize "bold the text world".

Example

The following code snippet illustrates how an application could send a text buffer to the SR engine using ISpRecoGrammar::SetWordSequenceData.

        HRESULT hr = S_OK;

        // place the contents of text buffer into pwszCoMem and the length of the text in cch

        SPTEXTSELECTIONINFO tsi;
        tsi.ulStartActiveOffset = 0;
        tsi.cchActiveChars = cch;
        tsi.ulStartSelection = 0;
        tsi.cchSelection = cch;

        pwszCoMem2 = (WCHAR *)CoTaskMemAlloc(sizeof(WCHAR) * (cch + 2));

        if (SUCCEEDED(hr) && pwszCoMem2)
        {
            // SetWordSequenceData requires double NULL terminator.
            memcpy(pwszCoMem2, pwszCoMem, sizeof(WCHAR) * cch);
            pwszCoMem2[cch] = L'\0';
            pwszCoMem2[cch+1] = L'\0';

            // set the text buffer data
            hr = cpRecoGrammar->SetWordSequenceData(pwszCoMem2, cch + 2, &tsi);
            // Check hr

            CoTaskMemFree(pwszCoMem2);
        }
        CoTaskMemFree(pwszCoMem);

        // the SR engine is now capable of recognizing the contents of the text buffer


ISpRecoGrammar::SetTextSelection

ISpRecoGrammar::SetTextSelection sets the current text selection and insertion point information.

See also ISpRecoGrammar::SetWordSequenceData for information on how to set the text data.

See also ISpSREngine::SetTextSelection for information on how SAPI passes the text selection info to the SR engine.

HRESULT SetTextSelection(
   const   SPTEXTSELECTIONINFO  *pInfo
);

Parameters

pInfo
[in] Address of the SPTEXTSELECTIONINFO structure that contains the text selection and insertion point information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
FAILED(hr) | Appropriate error message.

Remarks

An application that has a text box could allow the user to speak commands into the text box to edit the text. One way to design this functionality would be to create a CFG which supports such commands as "cut the text *", "bold the text *", or "italicize the words *". The grammar would then use a TEXTBUFFER tag in place of the * which would allow the SR engine to recognize the text buffer information. At runtime, the application would update the SR engine's view of the text buffer using ISpRecoGrammar::SetWordSequenceData. When the user highlights a selection of text and the text selection using ::SetTextSelection.

If a user had the text "hello world" in the text box and no text highlighted, the SR engine could recognize "bold the text world". If the user highlighted "hello", and the application changed the active text selection only contain "hello", "bold the text world" would fail to recognize.

The application should change the active text selection when the text highlight changes, rather than the entire word sequence data, to ensure the SR engine has a textual context to help the recognition language model.

Example

The following code snippet illustrates how to use ISpRecoGrammar::SetTextSelection after sending a text buffer to the SR engine using ISpRecoGrammar::SetWordSequenceData.

        HRESULT hr = S_OK;

        // place the contents of text buffer into pwszCoMem and the length of the text in cch

        SPTEXTSELECTIONINFO tsi;
        tsi.ulStartActiveOffset = 0;
        tsi.cchActiveChars = cch;
        tsi.ulStartSelection = 0;
        tsi.cchSelection = cch;

        pwszCoMem2 = (WCHAR *)CoTaskMemAlloc(sizeof(WCHAR) * (cch + 2));

        if (SUCCEEDED(hr) && pwszCoMem2)
        {
            // SetWordSequenceData requires double NULL terminator.
            memcpy(pwszCoMem2, pwszCoMem, sizeof(WCHAR) * cch);
            pwszCoMem2[cch] = L'\0';
            pwszCoMem2[cch+1] = L'\0';

            // set the text buffer data
            hr = cpRecoGrammar->SetWordSequenceData(pwszCoMem2, cch + 2, NULL);
            // Check hr

            // set the text selection information
            hr = cpRecoGrammar->SetTextSelection(&tsi);
            // Check hr

            CoTaskMemFree(pwszCoMem2);
        }
        CoTaskMemFree(pwszCoMem);

        // the SR engine is now capable of recognizing the contents of the text buffer


ISpRecoGrammar::IsPronounceable

ISpRecoGrammar::IsPronounceable calls the SR engine object to determine if the word has a pronunciation.

See also ISpSREngine::IsPronounceable for more information on the SR engine's role.

When to Use

Typically, there are two scenarios when an application might use the method ::IsPronounceable.

If an application is using a number of specialized or uncommon words (e.g., legal, medical, or scientific terms), the application may want to verify that the words are contained in either the lexicon (see also ISpLexicon or the SR engine's dictionary. If the words are not contained in the lexicon or the dictionary (even if they are pronounceable), the application can add them to the lexicon to improve the chances of a successful recognition.

An application may also want to verify that the SR engine will actually recognize the words in a CFG (even though loading the CFG succeeded). If the SR engine returns SPWP_UNKNOWN_WORD_UNPRONOUNCEABLE, the application can update the lexicon pronunciation entry (see ISpLexicon).

HRESULT IsPronounceable(
   const WCHAR         *pszWord,
   SPWORDPRONOUNCEABLE *pfPronounceable
);

Parameters

pszWord
[in, string] The word to test. Length must be equal to or less than SP_MAX_WORD_LENGTH.
pfPronounceable
[out] Flag, from among the following list, indicating the if the word is pronounceable by the SR Engine. See Remarks section.
Value
SPWP_UNKNOWN_WORD_UNPRONOUNCEABLE | The word is not pronounceable by the SR engine, and is not located in the lexicon and/or the engine's dictionary.
SPWP_UNKNOWN_WORD_PRONOUNCEABLE | The word is pronounceable by the SR engine, but is not located in the lexicon and/or the engine's dictionary.
SPWP_KNOWN_WORD_UNPRONOUNCEABLE | The word is pronounceable by the SR engine, and is located in the lexicon and/or the engine's dictionary.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Either pszWord or pfPronounceable is invalid or bad.
FAILED (hr) | Appropriate error message.

Remarks

The exact implementation and usage for the SR engine's dictionary and pronuncable words may vary between engines. For example, an SR engine may attempt to pronounce all words passed using ::IsPronounceable, even if it is not located in the lexicon or the dictionary, it would rarely or never, return SPWP_UNKNOWN_WORD_UNPRONOUNCEABLE.

Example

The following code snippet illustrates the use of ISpRecoGrammar:IsPronounceable. The words used are examples only, as the pronounceability by different SR engines may vary. See Remarks section.

    HRESULT hr = S_OK;

    // check if a common word is pronounceable
    hr = cpRecoGrammar->IsPronounceable(L"hello", &wordPronounceable);
    // Check hr

    // wordPronounceable is probably equal to SPWP_KNOWN_WORD_PRONOUNCEABLE

    // check if an uncommon, or imaginary, word is pronounceable
    hr = cpRecoGrammar->IsPronounceable(L"snork", &wordPronounceable);
    // Check hr

    // wordPronounceable is probably equal to SPWP_UNKNOWN_WORD_PRONOUNCEABLE

    // check if a non-word, or imaginary, word is unpronounceable
    hr = cpRecoGrammar->IsPronounceable(L"lpdzsd", &wordPronounceable);
    // Check hr

    // wordPronounceable is probably equal to SPWP_UNKNOWN_WORD_UNPRONOUNCEABLE


ISpRecoGrammar::SetGrammarState

ISpRecoGrammar::SetGrammarState sets the grammar state.

When To Use

Applications can use the grammar state to control whether it will receive recognitions for rules in that SpRecoGrammar object. For example, an application create a new SpRecoGrammar object, set the grammar state to SPGS_DISABLED, dynamically generate the rules, and finally set the grammar state to SPFS_ENABLED when grammar construction is completed.

See also ISpRecoGrammar::GetGrammarState.

HRESULT SetGrammarState(
   SPGRAMMARSTATE   eGrammarState
);

Parameters

eGrammarState
[in] Flag of type SPGRAMMARSTATE indicating the new state of the grammar.

If eGrammarState is SPGM_DISABLED, SAPI will retain the current rule activation state, so that when the grammar state is set to SPGM_ENABLED, it restores the grammar rules back to each of the original activation states. While the grammar is set to SPGM_DISABLED, the application can still activate and deactivate rule. The effect is not communicated to the SR engine (but retained by SAPI) until the grammar is enabled again.

If eGrammarState is SPGM_EXCLUSIVE, SAPI will disable all other grammars in the system, unless another grammar is already exclusive. Activation and deactivation commands are buffered for all other grammars until the exclusive grammar is set to SPGM_ENABLED again.

The default grammar state is SPGS_ENABLED, meaning the grammar can receive recognitions.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | eGrammarState is not a valid state.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecoGrammar::SetGrammarState to set the state of an grammar

    HRESULT hr = S_OK;

    // create a new grammar object
    hr = cpRecoContext->CreateGrammar(GRAM_ID, &cpRecoGrammar);
    // Check hr

    // disable the grammar, so that recognitions are not received
    hr = cpRecoGrammar->SetGrammarState(SPGS_DISABLED);
    // Check hr

    // ... build the grammar ...

    // activate the grammar, so applications start receiving recognitions
    hr = cpRecoGrammar->SetGrammarState(SPGS_ENABLED);
    // Check hr


ISpRecoGrammar::SaveCmd

ISpRecoGrammar::SaveCmd allows applications using dynamic grammars to save the current grammar state to a stream.

When to Use

Applications can use the ::SaveCmd method to serialize grammar changes that were made at runtime for use at a later time. See also ISpRecoGrammar::LoadCmdFromMemory.

HRESULT SaveCmd(
   IStream   *pSaveStream,
   WCHAR    **ppCoMemErrorText
);

Parameters

pSaveStream
[in] The stream to save the compiler binary grammar into.
ppCoMemErrorText
[out] Optional parameter of a null-terminated string containing error messages that occurred during the save operation.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pSaveStream is invalid or bad.
SPERR_NOT_DYNAMIC_GRAMMAR | Command was loaded but compiler is not available.
SPERR_UNINITIALIZED | Compiler is not available.
E_POINTER | ppCoMemErrorText is invalid or bad.
FAILED (hr) | Appropriate error message.

Example

The following code snippet illustrates how to use ISpRecoGrammar::SaveCmd to serialize the CFG from one SpRecoGrammar object and deserialize it into another SpRecoGrammar object.

    HRESULT hr = S_OK;

    // ... build and use a SpRecoGrammar object

    // create a Win32 global stream
    hr = ::CreateStreamOnHGlobal(NULL, true, &cpHStream);
    // Check hr

    // save the current grammar to the global stream
    hr = cpRecoGrammar->SaveCmd(cpHStream, NULL);
    // Check hr

    // create the second grammar to deserialize into
    hr = cpRecoContext->CreateGrammar(0, &cpReloadedGrammar);
    // Check hr

    // get a handle to the stream with the serialized grammar
    ::GetHGlobalFromStream(cpHStream, &hGrammar);
    // Check hr

    // deserialize the CFG into a new grammar object
    hr = cpReloadedGrammar->LoadCmdFromMemory((SPBINARYGRAMMAR *)::GlobalLock(hGrammar), SPLO_DYNAMIC);
    // Check hr


ISpRecoGrammar::GetGrammarState

ISpRecoGrammar::GetGrammarState retrieves the current state of the recognition grammar.

The default grammar state is SPGS_ENABLED.

See also ISpRecoGrammar::SetGrammarState

HRESULT GetGrammarState(
  SPGRAMMARSTATE   *peGrammarState
);

Parameters

peGrammarState
[out] Address of the SPGRAMMARSTATE enumeration that receives the grammar state information.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | One of the pointers is invalid or bad.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecoGrammar::GetGrammarState to query the default state of an SpRecoGrammar object.

    HRESULT hr = S_OK;

    // create a new grammar object
    hr = cpRecoContext->CreateGrammar(GRAM_ID, &cpRecoGrammar);
    // Check hr

    // query the default grammar state
    hr = cpRecoGrammar->GetGrammarState(&grammarState);
    // Check hr

    // ASSERT that grammarState == SPGS_ENABLED


ISpRecoResult

The ISpRecoResult interface is used by an application to retrieve information about the SR engine's hypotheses, recognitions, and false recognitions.

The most common use of the ISpRecoResult interface is retrieval of text recognized by the SR engine (see ISpPhrase::GetText). The ISpRecoResult interface also supports the retrieval of the original audio that the SR engine recognized. SAPI can automatically retain the audio for an application using ISpRecoContext::SetAudioOptions.

An application can set interest in SR engine hypotheses by calling ISpEventSource::SetInterest with SPEI_HYPOTHESIS. As each hypothesis event is received, the application can examine the proposed text and update its UI to display dynamic runtime recognition status.

An application can set interest in the SR engine's failed recognitions by calling ISpEventSource::SetInterest with SPEI_FALSE_RECOGNITION. If a false recognition occurs, the application can examine the audio (or even a partial recognition result) to reprocess the recognition or attempt to process the partially recognized text. SAPI does not require that an SR engine send a phrase with the false recognition event.

ISpRecoResult Methods | Description
ISpPhrase | Inherits from ISpPhrase and those methods are accessible from an ISpRecoResult object.
GetResultTimes | Retrieves the time information associated with the result.
GetAlternates | Retrieves an array containing alternate phrases.
GetAudio | Creates an audio stream for a given number of elements.
SpeakAudio | Retrieves and speaks the specified audio.
Serialize | Creates a serialized copy of the recognition result object.
ScaleAudio | Converts the format of the retained audio to a different audio format.
GetRecoContext | Returns the recognition context object that is associated with this result.


ISpRecoResult::GetResultTimes

ISpRecoResult::GetResultTimes retrieves the time information associated with the result.

When To Use

An application can use ::GetResultTimes to determine the system time that a recognition occurred, the length in seconds of the recognized phrase, and the length of time between when the SR engine began listening and when the recognition occurred.

HRESULT GetResultTimes(
   SPRECORESULTTIMES   *pTimes
);

Parameters

pTimes
[out] Address of the SPRECORESULTTIMES data structure containing the time information associated with the result.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pTimes is invalid or bad.
SPERR_NOT_FOUND | Interface not found.

Examples Using This Method

SDK: CoffeeS2; CoffeeS3; CoffeeS4.


ISpRecoResult::GetAlternates

ISpRecoResult::GetAlternates retrieves an array of pointers to ISpPhraseAlt objects containing alternate phrases.

HRESULT GetAlternates(
   ULONG           ulStartElement,
   ULONG           cElements,
   ULONG           ulRequestCount,
   ISpPhraseAlt  **ppPhrases,
   ULONG          *pcPhrasesReturned
);

Parameters

ulStartElement
[in] The starting element to consider for the alternates. This is a zero-based value.
cElements
[in] The number of elements to consider. All elements can be requested using the enumeration value SPPR_ALL_ELEMENTS of type SPPHRASERNG.
ulRequestCount
[in] The number of requested alternate phrase elements.
ppPhrases
[out] Address of an array of ISpPhraseAlt interface pointers that contain the alternate phrases. The elements between the start of the ulStartElement element and the end of the ulStartElement and cElements element combined is the portion that will change. The rest of the elements will be included in each alternate phrase.
pcPhrasesReturned
[out] Pointer to a ULONG that receives the actual number of alternate phrases retrieved.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pcPhrasesReturned is an invalid pointer. However, ppPhrases does not contain ulRequestCount allocations.
E_OUTOFMEMORY | Exceeded available memory.
E_INVALIDARG | ulStartElement is not less than the number of elements in the owning interface. However, the number of expected elements exceeds the number of available elements in the owning interface.
S_FALSE | No analyzer is present or there is no driver data.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use ISpRecoResult::GetAlternates to retrieve and commit an alternate phrase.

    HRESULT hr = S_OK;

    // ... obtain a recognition result object from the recognizer...

    // get the recognized phrase object
    hr = cpRecoResult->GetPhrase(&pPhrase);
    // Check hr

    // get the phrase's text
    hr = pPhrase->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, TRUE, &pwszText, NULL);
    // Check hr

    // ... check the phrase's text... assume the phrase isn't a correct recognition

    // setup MY_MAX_ALTERNATES phrase alternate objects
    CComPtr<ISpPhraseAlt>   pcpPhrase[MY_MAX_ALTERNATES];
    ULONG ulCount;

    // get the top MY_MAX_ALTERNATES alternates to the entire recognized phrase
    hr = cpRecoResult->GetAlternates(pPhrase->Rule.ulFirstElement,
                                     pPhrase->Rule.ulCountOfElements,
                                     MY_MAX_ALTERNATES,
                                     pcpPhraseAlt,
                                     &ulCount);
    // Check hr

    // check each alternate in order of highest likelihood
    for (int i = 0; i < ulCount; i++) {
         hr = pcpPhraseAlt[i]->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, TRUE, &pwszAlternate, NULL);
         // Check hr

         // ... check if this alternate is more appropriate ...

         // if it is more appropriate, then commit the alternate
         if (fMoreAppropriate) {
             hr = pcpPhraseAlt[i]->Commit();
             // Check hr
         }

         // free the alternate text
         if (pwszAlternate) ::CoTaskMemFree(pwszAlternate);

    }

    // free the initial phrase object
    if (pPhrase) ::CoTaskMemFree(pPhrase);


ISpRecoResult::GetAudio

ISpRecoResult::GetAudio creates an audio stream of the requested words from the audio data in the result object.

Even if there are no elements, that is, ulStartElement = 0 and cElements = 0, the audio will still be played. There are "unrecognized" results that have no elements but have audio.

An application can find the time offsets for each element by examining the SPPHRASE object retrieved using ISpRecoResult::GetPhrase.

HRESULT GetAudio(
   ULONG              ulStartElement,
   ULONG              cElements,
   ISpStreamFormat  **ppStream
);

Parameters

ulStartElement
[in] Value specifying from which element in the result data to start the audio stream.
cElements
[in] Value specifying the total number of words.
ppStream
[out] Address that will receive a pointer to an ISpStreamFormat object containing the audio data requested.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | cElements is zero or the expected number of elements to count exceeds the number available.
E_POINTER | ppStream is an invalid pointer.
SPERR_NO_AUDIO_DATA | This result object does not have any audio data.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use ISpRecoResult::GetAudio to retrieve the retained audio.

    HRESULT hr = S_OK;

    // ... obtain a recognition result object from the recognizer...

    hr = cpRecoResult->GetAudio( 0, 0, &cpStreamFormat );
    // Check hr

    // check the format of the stream for fun...
    hr = cpStreamFormat->GetFormat(&formatId, &pWaveFormatEx);
    // Check hr


ISpRecoResult::SpeakAudio

ISpRecoResult::SpeakAudio retrieves and speaks the specified audio. This combines two other methods; first calling ISpRecoResult::GetAudio and then calling ISpVoice::SpeakStream on the parent recognition context.

If the application did not activate retained audio (see ISpRecoContext::SetAudioOptions), or make a previous call to ISpPhrase::Discard and eliminate the retained audio, ::SpeakAudio will fail with SPERR_NO_AUDIO_DATA.

HRESULT SpeakAudio(
   ULONG    ulStartElement,
   ULONG    cElements,
   DWORD    dwFlags,
   ULONG   *pulStreamNumber
);

Parameters

ulStartElement
[in] Value specifying with which element to start.
cElements
[in] Value specifying the number of elements contained in the stream. A value of zero speaks all elements.
dwFlags
[in] Value indicating the attributes of the text stream. These values are contained in the SPEAKFLAGS enumeration.
pulStreamNumber
[optional, out] Address of a variable containing the stream number information. If NULL, the stream number will not be retrieved.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_NO_AUDIO_DATA | Result does not contain audio data.
E_POINTER | pulStreamNumber is a non-NULL, bad pointer.
FAILED(hr) | Appropriate error message.
Return values may also be those from ISpVoice::SpeakStream.

Example

The following code snippet illustrates the use of ISpObjectToken::IsUISupported using SPGUID_EngineProperties.

    HRESULT hr = S_OK;

    // ... get a recognition result object from the SR engine

    // replay the user's spoken audio to the user
    hr = cpRecoResult->SpeakAudio( 0, 0, 0, &ulStreamNum );
    // Check hr


ISpRecoResult::Serialize

ISpRecoResult::Serialize creates a serialized copy of the recognition result object. The serialized copy can be saved and later restored using the ISpRecoContext::DeserializeResult method.

HRESULT Serialize(
   SPSERIALIZEDRESULT   **ppCoMemSerializedResult
);

Parameters

ppCoMemSerializedResult
[out] Address of a pointer to the SPSERIALIZEDRESULT structure that receives the serialized result information. Call CoTaskMemFree() to free the memory associated with the serialized result object.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppCoMemSerializedResult is an invalid pointer.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use ISpRecoResult::Serialize to serialize a result and deserialize it back into an ISpRecoContext object.

    HRESULT hr = S_OK;

    // ... obtain a recognition result object from the recognizer...

    SPSERIALIZEDRESULT* pSerializedResult = NULL;
    ULONG cbWritten = 0;
    ULONG ulSerializedSize = 0;
    LARGE_INTEGER liseek;
    LARGE_INTEGER li;
    CComPtr<IStream> cpStreamWithResult;

    hr = CreateStreamOnHGlobal(NULL, true, &cpStreamWithResult);
    // Check hr

    // Serialize result to memory
    hr = cpRecoResult->Serialize(&pSerializedResult);
    // Check hr

    //serialized to a stream pointer
    hr = cpStreamWithResult->Write(pSerializedResult, pSerializedResult->ulSerializedSize, &cbWritten);
    // Check hr

    // free the serialized result
    if (pSerializedResult) ::CoTaskMemFree(pSerializedResult);

    // commit the stream changes
    hr = cpStreamWithResult->Commit(STGC_DEFAULT);
    // Check hr

    // ... persist stream to disk, network share, etc...
    // ... shutdown application ....

    // ... restart application and get the persisted stream

    // reset the stream seek pointer to the start before deserialization
    li.QuadPart = 0;
    hr = cpStreamWithResult->Seek(li, STREAM_SEEK_SET, NULL);
    // Check hr

    // find the size of the stream
    hr = cpStreamWithResult->Read(&ulSerializedSize, sizeof(ULONG), NULL);
    // Check hr

    // reset the seek pointer
    liseek.QuadPart = 0 - sizeof(ULONG);
    hr = cpStreamWithResult->Seek(liseek, STREAM_SEEK_CUR, NULL);
    // Check hr

    // allocate the memory for the result
    pSerializedResult = (SPSERIALIZEDRESULT*)::CoTaskMemAlloc(ulSerializedSize);
    // Check pSerializedResult in case out "out-of-memory"

    // copy the stream into a serialized result object
    hr = cpStreamWithResult->Read(pSerializedResult, ulSerializedSize, NULL);
    // Check hr

    // Deserialize result from memory
    hr = cpRecoContext->DeserializeResult(pSerializedResult, &cpRecoResultNew);
    // Check hr

    // free the pSerializedResult memory
    if (pSerializedResult) {
        CoTaskMemFree(pSerializedResult);
    }

    // As long as the same engine was used to generate
    // the original result object, as is now being used,
    // applications can now get alternates for the cpRecoResultNew's phrase


ISpRecoResult::ScaleAudio

ISpRecoResult::ScaleAudio converts an existing audio stream into a different audio format. Use the ISpPhrase::Discard method to completely discard audio data associated with a result object.

The application can also set the default retained audio format for the ISpRecoResult object by calling ISpRecoContext::SetAudioOptions. Calling ::SetAudioOptions will only apply to all subsequent recognitions, not the current ISpRecoResult object.

HRESULT ScaleAudio(
  const GUID           *pAudioFormatId,
  const WAVEFORMATEX   *pWaveFormatEx
);

Parameters

pAudioFormatId
[in] Address of the data format identifier. Typically, this value is SPDFID_WaveFormatEx.
pWaveFormatEx
[in] Address of the WAVEFORMATEX structure that contains the audio format to convert to. This value must be NULL if pAudioFormatId is not specified as SPDFID_WaveForamtEx.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Either pAudioFormatId or pWaveFormatEx is invalid or bad.
SPERR_NO_AUDIO_DATA | Audio stream is unavailable.
SPERR_UNSUPPORTED_FORMAT | The engine format is non-waveformatex and the retained format the same format.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.

Remarks

When performing a scaling with a compressed format, it is possible to introduce small rounding errors, since the content of the audio is not used to perform the conversion.

Scaling between certain compressed formats is not supported by the SAPI format converter (See the Remarks section for ISpStreamFormatConverter).

Example

The following code snippet illustrates the use ISpRecoResult::ScaleAudio to scale the audio to a low quality format before serialization to the disk (to save space).

    HRESULT hr = S_OK;

    // ... obtain a recognition result object from the recognizer...

    // create a format helper with a very low quality format
    CSpStreamFormat ScaleFormat(SPSF_8kHz8BitMono, &hr);
    // Check hr

    hr = cpRecoResult->ScaleAudio(&(ScaleFormat.FormatId()), ScaleFormat.WaveFormatExPtr());
    // Check hr

    // get a result serialization pointer
    SPSERIALIZEDRESULT* pSerializedResult;

    // serialize the result
    hr = cpRecoResult->Serialize(&pSerializedResult);
    // Check hr

    // ... write pSerializedResult to the disk

Development Helpers

Helper Enumerations, Functions and Classes | Description
SPSTREAMFORMAT | SAPI supported stream formats
CSpStreamFormat | Class for managing SAPI supported stream formats and WAVEFORMATEX structures


ISpRecoResult::GetRecoContext

ISpRecoResult::GetRecoContext returns the recognition context object that is associated with this result.

HRESULT GetRecoContext(
   ISpRecoContext   **ppRecoContext
);

Parameters

ppRecoContext
[out] A pointer that receives the recognition context interface pointer. The caller must call ::Release on the ISpRecoContext references when it is finished.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppRecoContext is invalid or bad.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecoResult::GetRecoContext to retrieve a reference to the ISpRecoContext instance that is associated with a recognized phrase and determine the maximum number of CFG alternates that can be generated for it.

    HRESULT hr = S_OK;

    DWORD dwMaxAlternates;

   // ... obtain a recognition result object from the recognizer...

   // get the associated ISpRecoContext
   hr = cpRecoResult->GetRecoContext(&cpRecoContext);
   // Check hr

   hr = cpRecoContext->GetMaxAlternates(&dwMaxAlternates);
   // Check hr


ISpRecognizer

The ISpRecognizer interface enables applications to control aspects of the speech recognition (SR) engine. Each ISpRecognizer interface represents a single SR engine. The application can connect to each recognizer object one or more recognition contexts, from which the application can control the recognition grammars to be used, start and stop recognition, and receive events and recognition results. The ISpRecognizer interface allows some additional control of the SR engine and its audio input. A standard application may not need to call many of the methods on this interface as SAPI tries to set the engine up sensibly by default.

There are two implementations of the ISpRecognizer and ISpRecoContext in SAPI. One is for recognition "in-process" (InProc), where the SR engine is created in the same process as the application. Only this application can connect to this recognizer. The other implementation is the "shared-recognizer," where the SR engine is created in a separate process. There will only be one shared engine running on a system, and all applications using the shared engine connect to the same recognizer. This allows several speech applications to work simultaneously, and allows the user to speak to any application, as recognition is done from the grammars of all applications. For desktop-based speech applications it is recommended to use the shared recognizer because of the way it allows multiple SAPI applications to work at once. For other types of application, such as recognizing from wave files or a telephony server application where multiple SR engines will be required, the InProc recognizer should be used.

When to Use

Call methods of the ISpRecognizer interface to configure or retrieve the attributes of the SR engine.

Implemented By
  • This interface is implemented by SAPI. Application developers use this interface but do not implement it.
How Created
There are two objects that implement this interface. These are created by applications by creating a COM object with either of the following CLSIDs:
    SpInprocRecognizer (CLSID_SpInprocRecognizer)
    SpSharedRecognizer (CLSID_SpSharedRecognizer)

Alternatively, the shared recognizer can be created by creating a SpSharedRecoContext (CLSID_SpSharedRecoContext), and then calling ISpRecoContext::GetRecognizer on this object to get a reference to the SpSharedRecognizer object.

Methods in Vtable Order

ISpRecognizer Methods | Description
ISpProperties | Inherits from ISpProperties and all those methods are accessible from ISpRecognizer.
SetRecognizer | Specifies the SR engine to be used.
GetRecognizer | Retrieves which SR engine is currently being used.
SetInput | Specifies which input stream the SR engine should use.
GetInputObjectToken | Retrieves the input token object for the stream.
GetInputStream | Retrieves the input stream.
CreateRecoContext | Creates a recognition context for this instance of an SR engine.
GetRecoProfile | Retrieves the current recognition profile token.
SetRecoProfile | Sets the recognition profile to be used by the recognizer.
IsSharedInstance | Determines if the recognizer is the shared or InProc implementation.
GetRecoState | Retrieves the state of the recognition engine.
SetRecoState | Sets the state of the recognition engine.
GetStatus | Retrieves current status information for the engine.
GetFormat | Retrieves the format of the current audio input.
IsUISupported | Checks if the SR engine supports a particular user interface component.
DisplayUI | Displays a user interface component.
EmulateRecognition | Emulates a recognition from a text phrase rather than from spoken audio.


ISpRecognizer::SetRecognizer

ISpRecognizer::SetRecognizer specifies the particular speech recognition engine to be used. This method allows the application to select a particular engine object token to be used (For example, the method SpFindBestToken could be used to find an engine supporting certain attributes, and the resulting token could be passed to this method).

If this method is not called, SAPI will use the current default SR engine.

If this method is passed NULL, SAPI will switch to the current default SR engine.

This method cannot be called when the current SR engine is already running and processing audio. In addition, when using the shared recognizer, it cannot be called if another application is also using the shared recognizer.

HRESULT SetRecognizer(
   ISpObjectToken   *pEngineToken
);

Parameters

pEngineToken
[in] The object token referring to the speech recognition engine to be used.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pEngineToken is invalid or bad.
SPERR_ENGINE_BUSY | Recognition is currently running or other applications are connected to the shared recognizer.
FAILED(hr) | Appropriate error message.


ISpRecognizer::GetRecognizer

ISpRecognizer::GetRecognizer retrieves the current speech recognition engine being used with this ISpRecognizer.

HRESULT GetRecognizer(
   ISpObjectToken   **ppEngineToken
);

Parameters

ppEngineToken
[out] The object token representing the current speech recognition engine.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppEngineToken is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpRecognizer::SetInput

ISpRecognizer::SetInput specifies which input stream the SR engine should use.

This method can be used to switch the input for the recognizer to a wave input stream, a different soundcard device, or to a custom audio object. The pUnkInput parameter can be a pointer to an object token representing an audio input device or a pointer to an actual object implementing ISpStreamFormat.

HRESULT SetInput(
   IUnknown  *pUnkInput,
   BOOL       fAllowFormatChanges
);

Parameters

pUnkInput
[in] The stream object token. See Remarks section.
fAllowFormatChanges
[in] Boolean indicating whether SAPI should try to change the input stream format to the engine's preferred format. This method can normally be set to TRUE; however, when performing both speech recognition and speech output at the simultaneously, some soundcards may require that both input and output are in the same audio format. Setting this to FALSE prevents the audio format on the input device from being changed. Instead, SAPI will try to convert the audio format itself to something the SR engine can use.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pUnkInput is invalid or not a stream.
SPERR_ENGINE_BUSY | The current method cannot be performed while the engine is currently processing audio.
FAILED(hr) | Appropriate error message.

Remarks

The input stream object will implement IStream, ISpStreamFormat, and ISpAudio for real-time streams. Applications should not use methods on these interfaces that actually change the state of the audio device or read data from it at the same time that the stream is being used by SAPI. For example, reading data from the application with IStream::Read will prevent the correct data from being passed to the SR engine. Altering the state of the audio using ISpAudio::SetState will put the audio device into an unexpected state and may cause errors. All control of the audio is done by SAPI.

When using the InProc recognizer, SAPI does not automatically setup the audio input. ISpRecognizer::SetInput must be called with a non-NULL pUnkInput to setup and start the audio input stream. Until ISpRecognizer::SetInput is called, methods such as ISpRecoGrammar::SetRuleState will return success code SP_STREAM_UNITIALIZED, but actual recognition will not start.

When using the shared recognizer, SAPI automatically sets up the audio input. However, ISpRecognizer::SetInput may be called with NULL as the pUnkInput parameter to force the recognizer to re-check the default audio input and re-set up the audio input (e.g., the default audio input object changes while recognizing, and the new audio input is to be used).

If the engine is currently processing audio, this call will fail with SPERR_ENGINE_BUSY.

Example

The following code snippet illustrates the use of ISpRecognizer::SetInput.

    // setup the inproc recognizer audio input with an audio input object token

    // get the default audio input token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_AUDIOIN, &cpObjectToken);
    // Check hr

    // set the audio input to our token
    hr = cpRecognizer->SetInput(cpObjectToken, TRUE);
    // Check hr

    // setup the inproc recognizer audio input with an audio input object

    // create the default audio input object
    hr = SpCreateDefaultObjectFromCategoryId(SPCAT_AUDIOIN, &cpAudio);
    // Check hr

    // set the audio input to our object
    hr = cpRecognizer->SetInput(cpAudio, TRUE);
    // Check hr
    // ask the shared recognizer to re-check the default audio input token
    hr = cpRecognizer->SetInput(NULL, TRUE);
    // Check hr - if SPERR_ENGINE_BUSY, then retry later


ISpRecognizer::GetInputObjectToken

ISpRecognizer::GetInputObjectToken retrieves the input token object for the stream currently being used.

GetInputObjectToken will always return the default audio input object token when using a shared recognizer.

When To Use

Applications will not normally need to use this method, but it can be used to find out specific details of the object token that was used to create the audio input stream.

If an application receives feedback from the SR engine that recognition quality is low, (e.g., poor audio signal quality (see SPEI_INTERFERENCE, or that the microphone needs adjustment (see SPEI_REQUEST_UI for SPDUI_MicTraining), etc.), it may be helpful to reconfigure the audio input settings. SAPI defines two specific types of audio UI that an audio object token can provide: volume (SPDUI_AudioVolume) and properties (SPDUI_AudioProperties). An application can use ::GetInputObjectToken, ISpObjectToken::IsUISupported, and ISpObjectToken::DisplayUI to display audio UI in an effort to improve speech recognition.

HRESULT GetInputObjectToken(
   ISpObjectToken   **ppToken
);

Parameters

ppToken
[out] Gets filled in with the current input object token pointer.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | Function completed successfully, but the input stream object has no object token associated with it.
E_POINTER | ppToken is invalid or bad.
SPERR_UNINITIALIZED | No audio input has yet been set with SetInput (InProc engine only).
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecognizer::GetInputObjectToken when displaying audio UI.

    HRESULT hr = S_OK;

    // check if the current recognizer has an object token
    hr = cpRecognizer->GetInputObjectToken(&cpObjectToken);
    // Check hr == S_OK

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // check if the default audio input object has UI for Volume
    hr = cpTokenUI->IsUISupported(SPDUI_AudioVolume, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then audio input object has UI for Volume
    // Display the default audio input object's Volume UI
    hr = cpTokenUI->DisplayUI(MY_HWND, MY_AUDIO_DIALOG_TITLE, SPDUI_AudioVolume, NULL, NULL, cpObjectToken, NULL);
    // Check hr


ISpRecognizer::GetInputStream

ISpRecognizer::GetInputStream retrieves the input stream that is currently being used.

Applications will not normally need to use this method, but it can be used to find the specific audio input stream that is being used. This method can be used only on InProc recognizers, not on the shared recognizer.

The returned object will implement IStream, ISpStreamFormat, and ISpAudio for real-time streams. Applications should not use methods on these interfaces that actually change the state of the audio device or read data from it. For example, reading data from the application with IStream::Read will prevent the correct data from being passed to the SR engine. Altering the state of the audio using ISpAudio::SetState will put the audio device into an unexpected state and may cause errors. All control of the audio is done by SAPI.

HRESULT GetInputStream(
   ISpStreamFormat   **ppStream
);

Parameters

ppStream
[out] Address of a pointer to the ISpStreamFormat object that receives the input stream information.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppStream is invalid or bad.
SPERR_NOT_SUPPORTED_FOR_SHARED_RECOGNIZER | Method is not available when using the shared recognizer.
FAILED(hr) | Appropriate error message.


ISpRecognizer::CreateRecoContext

ISpRecognizer::CreateRecoContext creates a recognition context for this instance of an SR engine. The recognition context is used to load recognition grammars, start and stop recognition, and receive events and recognition results.

Each application can have one or more recognition contexts, although normally each application will have only one.

HRESULT CreateRecoContext(
   ISpRecoContext   **ppNewContext
);

Parameters

ppNewContext
[out] Address of a pointer to an ISpRecoContext interface receiving the recognition context.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppNewContext is invalid or bad.
FAILED(hr) | Appropriate error message.

Examples Using This Method

SDK: CoffeeS0; CoffeeS1; CoffeeS2.


ISpRecognizer::GetRecoProfile

ISpRecognizer::GetRecoProfile retrieves the current recognition profile token.

A recognition profile represents a single user and training sessions on the system. The user can create, delete, and set the current profile using Speech properties in Control Panel. SAPI will always create the engine using the current default profile. This method can be used to find which profile is currently being used.

If an application needs to store information in a specific recognition profile, it can use the ISpObjectToken::GetStorageFilename method.

HRESULT GetRecoProfile(
   ISpObjectToken   **ppToken
);

Parameters

ppToken
[out] Address of a pointer of an ISpObjectToken that receives the profile information.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | The ppToken is a bad or invalid pointer.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecognizer::GetRecoProfile to determine the profile name

    HRESULT hr = S_OK;

    // get the current recognizer's recognition profile token
    hr = cpRecognizer->GetRecoProfile(&cpObjectToken);
    // Check hr

    // get the reco profile name (i.e. the default value of the token)
    hr = cpObjectToken->GetStringValue(NULL, &pwszRecoProfileName);
    // Check hr


ISpRecognizer::SetRecoProfile

ISpRecognizer::SetRecoProfile sets the recognition profile to be used by the recognizer.

A recognition profile represents a single user and training sessions on the system. The user can create, delete, and set the current profile using Speech properties in Control Panel. SAPI will always create the engine using the current default profile. This method can be used to set the SR engine to use a profile other than the default.

This method should not be called when the engine is currently processing audio. Calling ::SetRecoProfile with an active recognition engine can cause unexpected results, depending on how and when the SR engine reads the profile information.

HRESULT SetRecoProfile(
   ISpObjectToken   *pToken
);

Parameters

pToken
[in] Address of an ISpObjectToken object that contains the profile information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
FAILED(hr) | Appropriate error message.


ISpRecognizer::IsSharedInstance

ISpRecognizer::IsSharedInstance determines the recognizer is the shared or InProc implementation.

HRESULT  IsSharedInstance  ( void );

Parameters

None.

Return values

Value | Description
S_OK | Indicates that this instance of the recognition engine is being shared.
S_FALSE | Indicates that this instance of the recognition engine is not being shared.


ISpRecognizer::GetRecoState

ISpRecognizer::GetRecoState retrieves the current state of the recognition engine.

This method determines whether audio is currently being read from the audio input stream and passed to the SR engine.

See also ISpRecognizer::SetRecoState

HRESULT GetRecoState(
   SPRECOSTATE   *pState
);

Parameters

pState
[out] One of the input state flags contained in the SPRECOSTATE enumeration.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Invalid pointer.
FAILED(hr) | Appropriate error message.

Remarks

The default recognizer state is SPRST_ACTIVE, which means SAPI will activate the audio input stream only when at least one top-level rule is active.

To be notified when the recognizer state changes (e.g. another application changes the shared SR engine's recognizer state), rather than polling the state with ::GetRecoState, call ISpEventSource::SetInterest with SPEI_RECO_STATE.


ISpRecognizer::SetRecoState

ISpRecognizer::SetRecoState sets the state of the recognition engine.

The default state of an engine is SPRST_ACTIVE, which indicates that SAPI will start recognition whenever there are active rules and stop recognition when there are none. This is the normal state to be used by most applications. In the shared recognizer case, if an application has no active rules it will receive no recognition events, and if other applications have active rules they will receive relevant events.

HRESULT SetRecoState(
   SPRECOSTATE   NewState
);

Parameters

NewState
[in] One of the flags contained in the SPRECOSTATE enumeration.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.
E_INVALIDARG | One or more arguments are invalid.

Remarks

When using the shared recognizer, the recognizer state is a global setting. If one application changes the recognizer state, it will affect all other applications connected to the shared recognizer. For this reason, applications using a shared recognizer should take great caution before calling SetRecoState.

Changing the recognition state leads to a SPEI_RECO_STATE_CHANGE event for all interested recognition contexts.


ISpRecognizer::GetStatus

ISpRecognizer::GetStatus retrieves current status information for the engine. This method provides static information about the SR engine such as the languages it supports. It also provides dynamic information such as current stream position the engine has recognized up to, and if the stream is actively being sent to the engine.

See SPRECOGNIZERSTATUS for further explanation of the status information that can be retrieved.

HRESULT GetStatus(
   SPRECOGNIZERSTATUS   *pStatus
);

Parameters

pStatus
[out] The current status of the engine.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pStatus is invalid or bad.


ISpRecognizer::GetFormat

ISpRecognizer::GetFormat retrieves the current input audio format.

This method can return either the input format or the engine format. Normally these two values will be the same, but if SAPI is using a format converter to convert the input data from the audio input to the engine format these will be different.

HRESULT GetFormat(
   SPSTREAMFORMATTYPE   WaveFormatType,
   GUID                *pFormatId,
   WAVEFORMATEX       **ppCoMemWFEX
);

Parameters

WaveFormatType
[in] One of the wave file format types specified in SPSTREAMFORMATTYPE.
pFormatId
[out] The address of the unique identifier associated with the format type.
ppCoMemWFEX
[out] Address of a pointer to a WAVEFORMATEX structure that receives the format information. This is set only if the input is of a wave format type. The application must free this data with CoTaskMemFree after use.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.
SPERR_UNINITIALIZED | Audio input not yet set.
FAILED(hr) | Appropriate error message.


ISpRecognizer::IsUISupported

ISpRecognizer::IsUISupported checks if the underlying speech engine implements a certain type of user-interface component.

See the "SR Engine Guide" for further information on how an SR Engine implements UI.

[local] HRESULT IsUISupported(
   const WCHAR   *pszTypeOfUI,
   void          *pvExtraData,
   ULONG          cbExtraData,
   BOOL          *pfSupported
);

Parameters

pszTypeOfUI
[in] Address of a pointer to a null-terminated string containing the UI type information.
pvExtraData
[in] Additional information for the call. The SR Engine implementer dictates the format and usage of the data provided.
cbExtraData
[in] Size, in bytes, of pvExtraData. The SR Engine implementer dictates the format and usage of the data provided.
pfSupported
[out] Address of a variable that receives the value indicating support for the interface. This value is set to TRUE when this interface is supported; otherwise set to FALSE. If this value is TRUE, but the return code is S_FALSE, the UI type (pszTypeOfUI) is supported, but not with the current parameters or runtime environment. Check with the engine implementer to verify runtime requirements.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The UI is supported but not with the current runtime environment or parameters.
E_INVALIDARG | pfSupported is invalid or bad.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecognizer::IsUISupported using SPDUI_UserTraining.

    HRESULT hr = S_OK;

    // ask current recognizer if it supports user training
    hr = cpRecognizer->IsUISupported(SPDUI_UserTraining, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then current speech recognizer supports user training


ISpRecognizer::DisplayUI

ISpRecognizer::DisplayUI displays the requested UI component from the underlying SR engine.

To best apply ISpRecognizer::DisplayUI, call ISpRecognizer::IsUISupported with a specific UI type before calling DisplayUI. (see the SR Engine Guide for further information on how an SR Engine should implement UI.

The call to DisplayUI is synchronous, so the call will not return until the UI has been closed.

[local] HRESULT DisplayUI(
   HWND           hwndParent,
   const WCHAR   *pszTitle,
   const WCHAR   *pszTypeOfUI,
   void          *pvExtraData,
   ULONG          cbExtraData
);

Parameters

hwndParent
[in] Specifies the handle of the parent window.
pszTitle
[in] Address of a null-terminated string containing the window title. Set this value to NULL to indicate that the SR engine should use its default window title for this UI type.
pszTypeOfUI
[in] Address of a null-terminated string containing the UI type information.
pvExtraData
[in] Additional information for the call. The SR Engine implementer dictates the format and use of the data provided.
cbExtraData
[in] Size, in bytes, of the contents of pvExtraData. The SR Engine implementer dictates the format and usage of the data provided.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The UI is supported but not with the current runtime environment or parameters.
FAILED(hr) | Appropriate error message.

Remarks

SAPI 5 Speech Recognition engines are capable of sending UI requests back to the application using SPEI_REQUEST_UI. For example, if the SR engine is receiving a poor audio input signal, it may request the user to perform Microphone Training (see SPDUI_MicTraining). The application can receive these requests by calling ISpRecognizer::SetInterest with SPEI_REQUEST_UI. When the UI request is received, it can call ISpRecognizer::DisplayUI at an appropriate point. The typical SR engine UI requests could be User Training (see SPDUI_UserTraining), Microphone Training (see SPDUI_MicTraining), and Lexicon Updates (see SPDUI_AddRemoveWord). An application can call DisplayUI at any time, and does not necessarily have to wait for a UI request from the SR engine.

Example

The following code snippet illustrates the use of ISpRecognizer::DisplayUI using SPDUI_UserTraining.

    HRESULT hr = S_OK;

    // display user training UI for the current recognizer
    hr = cpRecognizer->DisplayUI(MY_HWND, MY_APP_USER_TRAINING, SPDUI_UserTraining, NULL, NULL);
    // Check hr


ISpRecognizer::EmulateRecognition

ISpRecognizer::EmulateRecognition emulates a recognition from a specified phrase rather than from spoken content.

This method can be used for testing applications that use speech recognition by simulating user speech. It can also be used by applications where users have the option to type or speak a command. The phrase can be generated by creating a phrase builder object and then adding elements representing the text to it. See the SDK Sample Simple Recognition (i.e., Reco.exe) for the function CreatePhraseFromText as an example of using ISpPhraseBuilder.

All the events will be fired back to the application exactly as if a normal recognition had taken place. The result phrase will have the semantic properties set in the same way a real result would. A recognition event will be produced only if the text actually parses through the active rules (if dictation is active, any text will parse). Another application or ISpRecoContext containing an active rule that can parse the text can receive the emulated recognition.

HRESULT EmulateRecognition(
   ISpPhrase   *pPhrase
);

Parameters

pPhrase
[in] The phrase to emulate.

Return values

Value | Description
S_OK | Function completed successfully.
SP_NO_PARSE_FOUND | Function completed successfully but the phrase does not parse through any active rule.
SP_NO_RULES_ACTIVE | Function completed successfully but there are no active rules to parse.
E_POINTER | ppCoMemPhrase is invalid or bad.
SPERR_UNINITIALIZED | Phrase is uninitialized.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.

Remarks

In the case of ambiguous rules or CFG paths, the ::EmulateRecognition method will return an arbitrary rule or path. For example, if a grammar has two ambiguous rules, the first containing the phrase "a b c", and the second containing only a dictation tag (i.e., <DICTATION/>), the rule recognized at runtime may not be consistent.


ISpPhrase

This is the main interface used to access information contained in a phrase. Using this interface, applications can retrieve recognition information such as the recognized (or hypothesized) text, the recognized rule, and semantic tag or property information. An application can also serialize the phrase data to a stream to enable persisting of recognitions to the disk, the network, or memory.

Methods in Vtable Order

ISpPhrase Methods | Description
GetPhrase | Retrieves a block of memory created by ::CoTaskMemAlloc that contains SPPHRASE, the data associated with this phrase.
GetSerializedPhrase | Retrieves a block of memory created by ::CoTaskMemAlloc that contains SPSERIALIZEDPHRASE, the serialized data associated with this phrase.
GetText | Retrieves the display text of the elements from this phrase and constructs a text string created by ::CoTaskMemAlloc by applying the pbDisplayAttributes of each SPPHRASEELEMENT.
Discard | Discards the requested data from the phrase object.


ISpPhrase::GetPhrase

ISpPhrase::GetPhrase retrieves data elements associated with a phrase.

HRESULT GetPhrase(
   SPPHRASE   **ppCoMemPhrase
);

Parameters

ppCoMemPhrase
[out] Address of a pointer to an SPPHRASE data structure receiving the phrase information. May be NULL if no phrase is recognized. If NULL, no memory is allocated for the structure. It is the caller's responsibilty to call ::CoTaskMemFree to free the object; however, the caller does not need to call ::CoTaskMemFree on each of the elements in SPPHRASE.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.
E_OUTOFMEMORY | Exceeded available memory.

Returned data includes all elements associated with this phrase.

Example

The following code snippet illustrates the use of ISpRecoResult::GetPhrase as inherited from ISpPhrase to retrieve the recognized text, and display the rule recognized and the phrase.

    HRESULT hr = S_OK;

    // ... obtain a recognition result object from the recognizer...

    // get the recognized phrase object
    hr = cpRecoResult->GetPhrase(&pPhrase);
    // Check hr

    // get the phrase's text
    hr = pPhrase->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, TRUE, &pwszText, NULL);
    // Check hr

    // display the recognized text and the rule name in a message box
    MessageBoxW(MY_HWND, pwszText, pPhrase->Rule.pszName, MB_OK);


ISpPhrase::GetPhrase

 The Microsoft.com Speech website

ISpPhrase::GetSerializedPhrase

ISpPhrase::GetSerializedPhrase returns the phrase information in a serialized form. The phrase structure can be saved to a file, and later restored by calling ISpPhraseBuilder::InitFromSerializedPhrase.

The caller passes in the address of a pointer which is initialized to point to a block of memory which is allocated using ::CoTaskMemAlloc. It is the caller's responsibility to call ::CoTaskMemFree to free this object. The structure returned is defined to be a SPSERIALIZEDPHRASE. However, the actual size of the block is contained in (*ppCoMemPhrase)->ulSerializedSize. This size includes the size of the SPSERIALIZEDPHRASE structure.

When To Use

An application that will not need recognition alternates or retained audio and needs to save space, can serialize only the phrase information (e.g., phrase text, rule name, SR Engine ID, etc.).

HRESULT GetSerializedPhrase(
   SPSERIALIZEDPHRASE   **ppCoMemPhrase
;)

Parameters

ppCoMemPhrase
[out] Address of a pointer which will be initialized to point to the serialized phrase data.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppCoMemPhrase is invalid or bad.
SPERR_UNINITIALIZED | Phrase is uninitialized.
E_OUTOFMEMORY | Exceeded available memory.

Example

The following code snippet illustrates the use ISpRecoResult::GetSerializedPhrase as inherited from ISpPhrase to serialize only the phrase portion of a result object.

    HRESULT hr = S_OK;

    // ... obtain a recognition result object from the recognizer...

    SPSERIALIZEDPHRASE* pSerializedPhrase = NULL;

    // get the recognized phrase object
    hr = cpRecoResult->GetSerializdPhrase(&pSerializedPhrase);
    // Check hr


ISpPhrase::GetText

ISpPhrase::GetText retrieves elements from a text phrase.

HRESULT GetText(
   ULONG     ulStart,
   ULONG     ulCount,
   BOOL      fUseTextReplacements,
   WCHAR   **ppszCoMemText,
   BYTE     *pbDisplayAttributes
);

Parameters

ulStart
[in] Specifies the first element in the text phrase to retrieve.
ulCount
[in] Specifies the number of elements to retrieve from the text phrase.
fUseTextReplacements
[in] Boolean value that indicates if replacement text should be used. An example of a text replacement is saying "write new check for twenty dollars" and retrieving the replaced text as "write new check for $20". For more information on replacements, see the "SR Engine White Paper".
ppszCoMemText
[out] Address of a pointer to the data structure that contains the display text information. It is the caller's responsibility to call ::CoTaskMemFree to free the memory.
pbDisplayAttributes
[out] Address of the SPDISPLAYATTRIBUTES enumeration that contains the text display attribute information. Text display attribute information can be used by the application to display the text to the user in a reasonable manner. For example, speaking "hello comma world period" includes a trailing period, so the recognition might include SPAF_TWO_TRAILING_SPACES to inform the application without requiring extra text processing logic for the application.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | A phrase that does not contain text or ppszCoMemText is NULL.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Invalid pointer.
E_OUTOFMEMORY | Exceeded available memory.

Example

The following code snippet illustrates the use ISpPhrase::GetText to retrieve parts of the recognized phrase.

    HRESULT hr = S_OK;

    // ... obtain a recognition result object from the recognizer...

    // get the recognized phrase object
    hr = cpRecoResult->GetPhrase(&pPhrase);
    // Check hr

    // get the phrase's entire text string, including replacements
    hr = pPhrase->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, TRUE, &pwszText, NULL);
    // Check hr

    // get the phrase's first 2 words, excluding replacements
    hr = pPhrase->GetText(pPhrase->Rule.ulFirstElement, 2, FALSE, &pwszText, NULL);


ISpPhrase::Discard

ISpPhrase::Discard discards the requested data from a phrase object.

Applications that have no use for certain types of retained data, and will be persisting or serializing the phrase or result objects, may wish to discard the unnecessary data. For example, an application performing offline transcription may need to retain only the audio and the final result. It can call ::Discard with SPDF_ALTERNATES to eliminate the alternate data (possibly including a large amount of private engine data).

Note that once retained audio is discarded, a call to ISpRecoResult::GetAudio will fail.

HRESULT Discard(
   DWORD   dwValueTypes
;)

Parameters

dwValueTypes
[in] Flags of type SPVALUETYPE indicating elements to discard. Multiple values may be combined.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | dwValueTypes is not a valid value type flag.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpRecoResult::Discard as inherited from ISpPhrase to discard the retained audio.

    HRESULT hr = S_OK;

    // .. get a recognition result object from the SR engine

    // discard audio
    hr = cpRecoResult->Discard(SPDF_AUDIO);
    // Check hr

    // .. serialize the "shrunken" result to the disk ...


ISpPhraseAlt

The ISpPhraseAlt interface is implemented on a phrase alternate object that can be obtained by calling ISpRecoResult::GetAlternates. The ISpPhraseAlt object is the interface that enables applications to retrieve alternate phrase information from an SR engine, and to update the SR engine's language model to reflect committed alternate changes.

Methods in Vtable Order

ISpPhraseAlt Methods | Description
ISpPhrase | Inherits from ISpPhrase and those methods are accessible from an ISpPhraseAlt object.
GetAltInfo | Retrieves data elements associated with an alternate phrase.
Commit | Replaces a section of the parent phrase to which this alternate corresponds.<


ISpPhraseAlt::GetAltInfo

ISpPhraseAlt::GetAltInfo retrieves data elements associated with an alternate phrase.

HRESULT GetAltInfo(
   ISpPhrase   **ppParent,
   ULONG        *pulStartElementInParent,
   ULONG        *pcElementsInParent,
   ULONG        *pcElementsInAlt
);

Parameters

ppParent
[out] Address to store the pointer to the parent SpPhrase object.
pulStartElementInParent
[out] Address to store the starting element position within the parent phrase that this alternate applies to.
pcElementsInParent
[out] Address to store the number of elements within the parent that this alternate replaces.
pcElementsInAlt
[out] Address to store the number of elements that the alternate contains.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | At least one of the parameters is invalid or bad.
SPERR_NOT_FOUND | The alternate is not associated with a valid parent phrase object.


ISpPhraseAlt::Commit

ISpPhraseAlt::Commit replaces a section of the parent phrase to which the alternate corresponds. After an alternate has been committed, the parent phrase will be modified to reflect the substitution.

Upon committing the alternate phrase, the SR engine also has the ability to update its language model to improve future recognitions of the same or similar phrases (see ISpSRAlternates::Commit).

HRESULT  Commit  ( void );

Parameters

None

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_NOT_FOUND | The alternate object is not related to a valid parent phrase object.

Example

The following code snippet illustrates the use ISpPhraseAlt::Commit to commit an alternate phrase.

    HRESULT hr = S_OK;

    // ... obtain a recognition result object from the recognizer...

    // get the recognized phrase object
    hr = cpRecoResult->GetPhrase(&pPhrase);
    // Check hr

    // get the phrase's text
    hr = pPhrase->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, TRUE, &pwszText, NULL);
    // Check hr

    // ... check the phrase's text... assume the phrase isn't a correct recognition

    // setup MY_MAX_ALTERNATES phrase alternate objects
    CComPtr<ISpPhraseAlt>   pcpPhrase[MY_MAX_ALTERNATES];
    ULONG ulCount;

    // get the top MY_MAX_ALTERNATES alternates to the entire recognized phrase
    hr = cpRecoResult->GetAlternates(pPhrase->Rule.ulFirstElement,
                                     pPhrase->Rule.ulCountOfElements,
                                     MY_MAX_ALTERNATES,
                                     pcpPhraseAlt,
                                     &ulCount);
    // Check hr

    // check each alternate in order of highest likelihood
    for (int i = 0; i < ulCount; i++) {
         hr = pcpPhraseAlt[i]->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, TRUE, &pwszAlternate, NULL);
         // Check hr

         // ... check if this alternate is more appropriate ...

         // if it is more appropriate, then commit the alternate
         if (fMoreAppropriate) {
             hr = pcpPhraseAlt[i]->Commit();
             // Check hr
         }

         // free the alternate text
         if (pwszAlternate) ::CoTaskMemFree(pwszAlternate);

    }

    // free the initial phrase object
    if (pPhrase) ::CoTaskMemFree(pPhrase);


ISpProperties

ISpProperties sets and retrieves property attribute information.

This interface is currently implemented only by the SR Engine (ISpRecognizer), and can be used for setting SR Engine properties (see SAPI 5.0 SR Properties White Paper).

Methods in Vtable Order

ISpProperties Methods | Description
SetPropertyNum | Sets a numeric property corresponding to the specified name.
GetPropertyNum | Retrieves a numeric value specified by the named key.
SetPropertyString | Sets a text property corresponding to the specified name.
GetPropertyString | Retrieves the string value corresponding to the specified key name.


ISpProperties::SetPropertyNum

ISpProperties::SetPropertyNum sets a numeric property corresponding to the specified name.

HRESULT SetPropertyNum(
   const WCHAR   *pName,
   LONG           lValue
 );

Parameters

pName
[in] Null-terminated string containing the property name. Valid values are listed in the SR Properties section of the SAPI 5.0 SR Properties White Paper.
lValue
[in] The property value to set.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | SR Engine does not support specified property name.
E_INVALIDARG | One or more arguments are invalid.
FAILED(hr) | SR Engine returned specific error.

Remarks

If the SR Engine supports the property, SAPI will fire a property-changed event (see SPEI_PROPERTY_NUM_CHANGE) to all interested recognizer contexts (ISpRecoContext). Broadcasting the corresponding event notifies any recognizer contexts that had interests in the property (see CSpEvent::PropertyName and CSpEvent::PropertyNumValue).


ISpProperties::GetPropertyNum

ISpProperties::GetPropertyNum retrieves a numeric value specified by the named key.

HRESULT GetPropertyNum(
   const WCHAR   *pName,
   LONG          *plValue
);

Parameters

pName
[in] String containing the property name. Valid values are listed in the SR Properties section of the SAPI 5.0 SR Properties White Paper.
plValue
[out] Address to store the property value.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | SR Engine does not support specified property name.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Value pointer is invalid.
FAILED(hr) | SR Engine returned specific error.


ISpProperties::SetPropertyString

ISpProperties::SetPropertyString sets a text property corresponding to the specified name.

HRESULT SetPropertyString(
   const WCHAR   *pName,
   const WCHAR   *pValue
);

Parameters

pName
[in, string] Null-terminated string containing the property name.
pValue
[in, string] Null-terminated string containing the property value.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | SR Engine does not support specified property name.
E_INVALIDARG | One or more arguments are invalid.
FAILED(hr) | SR Engine returned specific error.

Remarks

If the SR Engine supports the property, SAPI will fire a property-changed event (see SPEI_PROPERTY_STRING_CHANGE) to all interested recognizer contexts (ISpRecoContext). Broadcasting the corresponding event notifies any recognizer contexts that had interests in the property (see CSpEvent::PropertyName and CSpEvent::PropertyStringValue).


ISpProperties::GetPropertyString

ISpProperties::GetPropertyString retrieves the string value corresponding to the specified key name.

HRESULT GetPropertyString(
   const WCHAR   *pName,
   WCHAR        **ppCoMemValue
);

Parameters

pName
[in] Null-terminated string containing the property name. Valid values are listed in the SR Properties section of the SAPI 5.0 SR Properties White Paper.
ppCoMemValue
[out] Address to store the pointer to the string value.
The caller must call CoTaskMemFree() to free the string pointer.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | SR Engine does not support specified property name.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Value pointer is invalid.
FAILED(hr) | SR Engine returned specific error.


SAPI text to speech

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

SAPI Text-to-Speech (TTS)

The following section covers:

  • TTS Overview
  • Text Synthesis
  • ISpVoice


ISpVoice

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Text-To-Speech Overview

ISpVoice Introduction

The central SAPI API for text-to-speech (TTS) is called ISpVoice. Using this interface, applications can add TTS support such as speaking text, modifying speech characteristics, changing voices and responding to real-time events while speaking. In fact, most applications should need only this single interface to accomplish everything that is needed for basic TTS support.

Applications obtain access to ISpVoice interface methods by creating a COM object. As the name implies, an ISpVoice object is simply a single instance of a specific TTS voice. Every ISpVoice object is an individual voice. Even if two different ISpVoice objects select the same base voice (for example "Mike"), each of the two voices can be changed and modified independent of the other.

Speaking

When an application first creates an ISpVoice object, the object initializes to the default voice (set in the Speech Control Panel). This means that the new object is immediately ready to speak text, no special initialization is needed. At this point, applications can use Speak or SpeakStream methods to speak any Unicode text data.

Synchronous vs. Asynchronous Speaking

The two speaking functions can generate speech either synchronously (function does not return until text has completely spoken) or asynchronously (function returns immediately but keep speaking as a background process). Asynchronous operation is chosen if the application needs to do something else (highlight text, paint animation, monitor controls, etc.) while speaking. Otherwise, the simplest case is to speak synchronously.

Getting Status Information

During asynchronous speech, applications can get current status information (text position, speech done state, bookmarks, etc.) in one of two ways. The simplest way is to periodically poll the ISpVoice object using the GetStatus method. The other way is to initialize the ISpVoice object so that it sends real-time events to the application as they happen.

Flow Control

As a convenience, most TTS applications allow users to temporarily suspend speech output. The Pause and Resume methods are typically called in response to a user initiated action.

Modifying Voice Attributes

Often with TTS, voice output needs to be modified from its default setting. There are two ways to do this is; either by calling certain ISpVoice API methods, or by embedding special Extended Markup Language (XML) tags within the spoken text. Typically, the API functions are used as global settings that affect the speech independent of current selected voice or document that is spoken. While the XML tags are usually used in much narrower scope, affecting only the spoken style in a single document.

Audio Output

Although usually the default for desktop applications, audio output for TTS is not restricted to hardware sound card destinations. SAPI TTS supports, either directly or indirectly, just about any audio configuration an application may require. Whether the destination is a PC sound card, buffer in memory, or a special telephony hardware, ISpVoice has several audio control methods to change the audio path from its default configuration.

ISpVoice Methods

Speaking Text
Speak | Speaks a text string or file.
SpeakStream | Speaks a text stream or plays an audio (WAV) stream.
Real-time Status
GetStatus | Returns current speech and event status information.
WaitUntilDone | Delays until either the voice has completed speaking or the specified time interval has elapsed.
SpeakCompleteEvent | Returns an event handle that will be signaled when speech is done.
Flow Control
Pause | Pauses the output speech at the nearest alert boundary.
Resume | Resumes speaking.
Skip | Skips ahead or backward to a new input text position while speaking.
Changing Voice Attributes
SetRate | Sets the speaking rate in real time.
GetRate | Returns the current speaking rate.
SetVolume | Sets the speech volume level in real time.
GetVolume | Returns the current speech volume level.
SetVoice | Sets the identity of the voice used for synthesis.
GetVoice | Retrieves the object token that identifies the current voice.
Real-time Event Management (inherited from ISpEventSource)
SetInterest | Sets the type of events to queue.
GetEvents | Returns the queued events.
GetInfo | Returns information about the event queue.
SetNotifySink | Sets up the instance to make free-threaded calls through ISpNotifySink::Notify.
SetNotifyWindowMessage | Sets a window handle to receive notifications as window messages.
SetNotifyCallbackFunction | Sets a callback function to receive notifications.
SetNotifyCallbackInterface | Sets an object derived from ISpTask to receive notifications.
SetNotifyWin32Event | Sets up a Win32 event object to be used by this instance for notifications.
WaitForNotifyEvent | A blocking call which waits for a notification.
GetNotifyEventHandle | Retrieves Win32 event handle associated with this notify source.

Audio Output Control
SetOutput | Sets the current output object. A value of NULL may be used to select the default audio device.
GetOutputStream | Retrieves a pointer to the current output stream.
GetOutputObjectToken | Retrieves the object token for the current output object.
Miscellaneous
SetPriority | Sets the priority for the voice.
GetPriority | Retrieves the current voice priority level.
SetAlertBoundary | Specifies which event should be used as the insertion point for alerts.
GetAlertBoundary | Retrieves the event that is currently being used as the insertion point for alerts.
IsUISupported | Determines if the specified type of UI is supported.
DisplayUI | Displays the requested UI.
SetSyncSpeakTimeout | Sets the timeout interval in milliseconds after which, synchronous Speak and SpeakStream calls to this instance of the voice will timeout.
GetSyncSpeakTimeout | Retrieves the timeout interval for synchronous speech operations for this ISpVoice instance.


Text synthesis

SAPI 5.0 utilizes the Extensible Markup Language (XML) to define text synthesis characteristics and application configuration settings.

A text to speech (TTS) engine that uses synthesis generates sounds similar to those created by the human voice and applies various filters to simulate throat length, mouth cavity, lip shape, and tongue position. Although the voice produced through text synthesis often sounds less human than a voice produced by diphone concatenation, it is possible to obtain different qualities of voice through modifying TTS configuration settings. SAPI 5.0 compliant TTS engines can achieve improved synthesized text to speech voice qualities using XML to control the configuration settings for text synthesis.

The following section covers:

  • Synthesis markup
  • English Context tag definitions
  • Chinese Context tag definitions
  • Japanese Context tag definitions


Synthesis markup

SAPI 5.0 synthesis markup is the collection of XML tags inserted into text to modify the speech synthesis of that text. These XML tags, which provide functionality such as volume control and word emphasis, are inserted into text passed into ISpVoice::Speak and text streams of format SPDFID_XML which are then passed into ISpVoice::SpeakStream. By default, the SAPI XML parser auto-detects XML. In the case of an invalid XML structure, a speak error may be returned to the application. SAPI is not intended to be used to validate the XML structure, as it is the responisbility of the developer to validate the XML with an XML validation tool. Please see ISpVoice for more information.

SAPI 5.0 synthesis markup is an application of XML. Every XML element consists of a start tag <Some_tag> and an end tag </Some_tag> with a case-insensitive tag name and contents between these tags. If the element is empty, it has no contents <Some_tag></Some_tag> and the start tag and the end tag might be the same <Some_tag/>. More information about XML and the XML specification is available at: http://www.w3.org/TR/1998/REC-xml-19980210.html.

The following section covers:

  • SAPI 5.0 XML tags
  • Attributes
  • Contents
  • Relationship to HTML web pages and SABLE

SAPI 5.0 XML tags

XML tags in SAPI 5.0 follow a defined structure program scope and implementation. SAPI 5.0 XML tags have a specific purpose and affect the input text in a predetermined manner.

The SAPI 5.0 XML tags are divided into four different scope categories.

  1. Non-scoped
  2. Scoped
  3. Global
  4. Scoped/Global

The behavior and application-specific properties can be controlled through the use of XML tags. Additional information on SAPI 5.0 XML elements is available at: SAPI XML Schema.

Attributes
Attributes of an XML element appear inside the start tag. Each attribute is in the form of a name, followed by an equal character, followed by a quoted string value. An attribute of a given name may only appear once in a start tag. Exact details on what characters may appear between quotes can be found at http://www.w3.org/TR/REC-xml#NT-AttValue.

Briefly, the literal string cannot contain a less than character "<" if the string is surrounded by single quotation marks, it cannot contain a single quotation mark. If the string is surrounded by double quotation marks it cannot contain a double quotation mark. Furthermore, all ampersands (&) can be used only in an entity reference such as &amp; and "&gt;". When a literal string is parsed, the resulting replacement text will resolve all entity references such as "&gt;" into its corresponding text, such as ">".

In this specification, only the resulting replacement text needs to be defined for attribute value strings. The XML specification defines the exact file syntax details. Character references allow entity references in ASCII characters to specify replacement text which has unprintable characters such as extended UNICODE characters. The entity reference "&#x0259;" specifies the single UNICODE character for the International Phonetic Alphabet symbol for a mid-central unrounded vowel. See http://www.w3.org/TR/1998/REC-xml-19980210#sec-references for details.

The <LANG> and <VOICE> XML tags are specific to the Microsoft engines and provide support for language and dialect attributes for a given voice.

The following is an example of what 409;9 refers to and how to correctly use it in XML tags:

<LANG LANGID="409">This is the US English language</LANG>
<LANG LANGID="9">This is the English Language</LANG>
<VOICE REQUIRED="language=409">This is the required voice for the US English language</VOICE>
<VOICE REQUIRED="language=9">This is the required voice that speaks in any dialect of the English language</VOICE>

A speak error will occur when entering voice attribute information as it appears in the Windows Registry:

For example:

<LANG LANGID="409;9">Speak this text with the US English language</LANG>
<VOICE REQUIRED="language=409;9">Require a voice be used that speaks the US English language</VOICE>

In the Windows Registry, the language attribute for the Microsoft SAPI 5.0 English voices is labeled as '409;9' The '409' attribute information indicates the voice is specifically US English, and '9' refers to the English language. This language labeling convention for voices may not be followed by all engine manufacturers. For example, the LH voices may use '409' to indicate an English voice, while Microsoft uses '409;9' to specify the voice is specifically US English.

For example:

409;9 = US English
809;9 = British English

Start RegEdit and expand the tree view pane to the following registry key location:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Voices\Tokens

Select one of the available voices and view the corresponding attribute information.

The following is an example of the MSMary voice attributes:

Contents
The contents of an element consist of text or sub-elements. With these definitions, the XML specification defines the exact file syntax details.

Relationship to HTML web pages and SABLE

The XML format that SAPI 5.0 uses is NOT placed inside web pages. Web page authors who want to mark up sections of HTML text so that it is synthesized correctly, should use the W3C Aural Cascading Style Sheets (ACSS). More information is available at: http://www.w3.org/TR/WD-acss

SAPI applications that are synthesizing text from a web page will "render" HTML+ACSS into SAPI's synthesis markup format. Programs apply a default ACSS file when synthesizing web pages that do not have an associated ACSS file.

SAPI 5.0 synthesis markup format is similar to the format published by the SABLE Consortium. However, this format and SABLE version 1.0 are not interoperable. At this time, it's not determined if they will become partially interoperable in the future. More information about the SABLE specification is available at: http://www.bell-labs.com/project/tts/sable.html.

return to the top of this page Back to top


English Context tag definitions

The CONTEXT tag specifies the normalization of a block of text. This specification defines the SAPI predefined attributes (ID) for the CONTEXT tag. These IDs are strings. SAPI does not perform any parameter validation on the string passed to the engine, hence, the application can specify engine-specific normalization IDs to the engine. Engine-specific strings begin with the engine vendor's name to avoid confusion between engines.

For example:

<CONTEXT ID = "MS_My_Context"> text </CONTEXT>

The exact implementation of some of these values is dependent on the engine used in SAPI 5. In order to force a certain normalization, application developers can choose to normalize the text, or use another SAPI tag or engine-specific ID. Each context tag can contain more than one string.

For example:

<CONTEXT ID = "date_mdy"> 12/21/99 11/21/99 10/21/99 </CONTEXT>
would be normalized to "December twenty first nineteen ninety nine November twenty first nineteen ninety nine October twenty first nineteen ninety nine."

The following predefined context types are covered in this section:

  • Date
  • Time
  • Number
  • Phone_Number
  • Currency
  • Web
  • E-mail
  • Address

Date

This context specifies that the number passed to the engine is a date. Dates will generally have the format of number [delimiter] number [delimiter] number or number [delimiter] number where the delimiter can be a '.', '/' or '-', and numbers are typically between 01 and 12 for months, 01 and 31 for days. A year is generally a two or four-digit number.
The following are valid string types:

date_mdy
This will normalize the date so that the first group of numbers is the month, the second group is the day, and the third group is the year. In the case where the year is a two-digit number, the engine reads it as a two-digit number or a four-digit number.

For example:

<context ID = "date_mdy">12/21/99</context>
will be normalized to "December twenty first ninety nine"
or "December twenty first nineteen ninety nine"

<context ID = "date_mdy">12/21/1999</context>
will be normalized to "December twenty first nineteen ninety nine"

return to the top of this page Back to top

date_dmy
This will normalize the date so that the first group of numbers is the day, the second group is the month, and the third group is the year. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is represented as a four-digit year.

For example:

<context ID = "date_dmy">21.12.99</context>
will be normalized to "December twenty first ninety nine"
or "December twenty first nineteen ninety nine"

<context ID = "date_ dmy">21-12-1999</context>
will be normalized to "December twenty first nineteen ninety nine"

return to the top of this page Back to top

date_ymd
This will normalize the date so that the first group of numbers is the year, the second group is the month, and the third group is the day. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is represented as a four-digit year.

For example:

<context ID = "date_ymd">99-12-21</context>
will be normalized to "December twenty first ninety nine"
or "December twenty first nineteen ninety nine"

<context ID = "date_ ymd">1999.12.21</context>
will be normalized to "December twenty first nineteen ninety nine"

return to the top of this page Back to top

date_ym
This will normalize the date so that the first group of numbers is the year, and the second group is the month. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is represented as a four-digit year.

For example:

<context ID = "date_ym">99-12</context>
will be normalized to "December ninety nine"
or "December nineteen ninety nine"
<context ID = "date_ ym">1999.12</context>
will be normalized to "December nineteen ninety nine"

return to the top of this page Back to top

date_my
This will normalize the date so that the first group of numbers is the month, and the second group is the year. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is represented as a four-digit year.

For example:

<context ID = "date_my">12/99</context>
will be normalized to "December ninety nine"
or "December nineteen ninety nine"

<context ID = "date_my">12/1999</context>
will be normalized to "December nineteen ninety nine"

return to the top of this page Back to top

date_dm
This will normalize the date so that the first group of numbers is the day and the second group is the month.

For example:

<context ID = "date_dm">21.12</context>
will be normalized to "December twenty first"

return to the top of this page Back to top

date_md
This will normalize the date so that the first group of numbers is the month and the second group is the day.

For example:

<context ID = "date_md">12/21</context>
will be normalized to "December twenty first"

return to the top of this page Back to top

date_year
This will normalize the date so that the number is read as a year.

For example:

<context ID = "date_year">1999</context>
will be normalized to "nineteen ninety nine"

<context ID = "date_year">2001</context>
will be normalized to "Two thousand one"

return to the top of this page Back to top

Time

This context specifies that the number passed to the engine is a time. Times will generally have the format of number [delimiter] number [delimiter] number or number [delimiter] number where the delimiter is ':' or ' ' ' or ' " ' and numbers are typically between 01 and 24 for hours, 01 and 59 for minutes and seconds.

When a zero is present in numbers between 01 and 09, the engine can ignore this, or normalize it as "oh". The engine can place an "and" in the normalized time. The valid string types are:

For example:

<context ID = "time">12:30</context>
will be normalized to "twelve thirty"

<context ID = "time">01:21</context>
is normalized as "one twenty one"
or "oh one twenty one"

<context ID = "time">1'21"</context>
is normalized as "one minute twenty one seconds"
or "one minute and twenty one seconds"

return to the top of this page Back to top

Number

number_cardinal
The text is normalized as a number using the regular format of ones, tens, etc. The engine can place "and" in the normalized text.

For example:

<context ID = "number_cardinal">3432</context>
will be normalized to "three thousand four hundred thirty two"

<context ID = "number_cardinal">3432</context>
will be normalized to "three thousand four hundred and thirty two"

return to the top of this page Back to top

number_digit
The text is normalized digit by digit.

For example:

<context ID = "number_digit">3432</context>
will be normalized to "three four three two"

return to the top of this page Back to top

number_fraction
The text is normalized as a fraction.

For example:

<context ID = "number_fraction">3/15</context>
will be normalized to "three fifteenths" or "three over fifteen"

return to the top of this page Back to top

number_decimal
The text is normalized as a decimal value.

For example:

<context ID = "number_decimal">423.1243</context>
will be normalized to "four hundred and twenty three point one two four three"

return to the top of this page Back to top

Phone_Number
The text is normalized as a phone number. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI.

return to the top of this page Back to top

Currency
The text is normalized as a currency. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI.

For example:

<context ID = "currency">$34.90</context>
will be normalized to "thirty four dollars and ninety cents"

return to the top of this page Back to top

Web

The text is normalized as a URL. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI.

web_url

For example:

<context ID = "web_url">www.Microsoft.com</context>
will be normalized to "is normalized to "w w w dot Microsoft dot com"

return to the top of this page Back to top

E-mail

The text is normalized as e-mail. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI.

E-mail_address
The text is normalized as an e-mail address. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI.

For example:

<context ID = "E-mail_Address">someone@microsoft.com</context>
is normalized to "Someone at Microsoft dot com"

return to the top of this page Back to top

Address

The text is normalized as an address. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI.

For example:

<context ID = "address">One Microsoft Way, Redmond, WA, 98052</context>
will be normalized to "One Microsoft Way Redmond Washington nine eight zero five two"

address_postal
The text is normalized as a postal address. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI.

For example:

<context ID = "address_postal">A2C 4X5</context>
will be normalized to "A 2 C 4 X 5"

return to the top of this page Back to top


Chinese Context tag definitions

The CONTEXT tag specifies the normalization of a block of text. The context tag ID attribute contains the text string specifying the type of normalization to apply to the text block. The ID attribute of the Chinese engine specific context tags begin with "CHS_". CONTEXT tag ID attribute strings that do not begin with quot;CHS_" use English context text normalization.

For example:
<CONTEXT ID = "CHS_My_Context"> 12/21/99 </CONTEXT>

Each context tag can contain more than one string.

For example:
<CONTEXT ID = "date_mdy"> 12/21/99 11/21/99 10/21/99 </CONTEXT>
is normalized to "Nine nine Nian twelve Yue twenty-one Ri nine nine Nian eleven Yue twenty-one Ri nine nine Nian ten Yue twenty-one Ri"

The following topics are covered in this section:

  • Date
  • Time
  • Number
  • Phone Number and Postal Address

Date

This context specifies that the number passed to the engine is a date. Dates generally have the format of number [delimiter] number [delimiter] number or number [delimiter] number where the delimiter may be a '.', '/' or '-', and numbers are typically between 01 and 12 for months, 01 and 31 for days. A year is a two- or four-digit number.

If a date format does not fall within the range shown below, the application cannot expect a consistent result and the engine may interpret it freely. The valid string types are:

  • Date_mdy
  • Date_dmy
  • Date_ymd
  • CHS_Date_ymdhm
  • Date_ym
  • Date_md
  • Date_dm
  • Date_year

return to the top of this page Back to top

Date_mdy

The text specifying the date is normalized so that the first group of numbers is the month, the second group is the day and the third group is the year. In the case where the year is a two-digit number, the engine reads it as a two-digit number. In the case where the year is a four-digit number, the engine reads it as a or a four-digit number.

This a common tag with the English engine.

return to the top of this page Back to top

Date_dmy

The text specifying the date is normalized so that the first group of numbers is the day, the second group is the month and the third group is the year. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is represented as a four-digit year.

This a common tag with the English engine.

return to the top of this page Back to top

Date_ymd

The text specifying the date is normalized so that the first group of numbers is the year, the second group is the month and the third group is the day. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is represented as a four-digit year.

This a common tag with the English engine.

return to the top of this page Back to top

CHS_Date_ymdhm

The text specifying the date is normalized so that the first group of numbers is the year, the second group is the month, the third group is the day, the fourth group is the hour and the fifth group is the minute. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is represented as a four-digit year. The text format for time needs to specify the hour as a number between 00 and 23 (in case of with PM/AM before, between 01 and 12), and the minute as a number between 01 and 59.

This is a Chinese specific context tag.

return to the top of this page Back to top

Date_ym

The text specifying the date is normalized so that the first group of numbers is the year and the second group is the month. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is represented as a four-digit year.

This a common tag with the English engine

return to the top of this page Back to top

Date_my

The text specifying the date is normalized so that the first group of numbers is the month and the second group is the year. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is represented as a four-digit year.

This a common tag with the English engine.

return to the top of this page Back to top

Date_md

The text specifying the date is normalized so that the first group of numbers is the month and the second group is the day.

This a common tag with the English engine.

return to the top of this page Back to top

Date_dm

The text specifying the date is normalized so that the first group of numbers is the day and the second group is the month.

This a common tag with the English engine.

return to the top of this page Back to top

Date_year

The text specifying the date is normalized so that the number is read as a year. This year should be a four-digit number.

This a common tag with the English engine.

return to the top of this page Back to top

Time

This context specifies that the number passed to the engine is a time. Times generally have the format of number [delimiter] number [delimiter] number or number [delimiter] number where the delimiter is ':' and numbers are typically between 00 and 23 for hours (in case of with PM/AM before, it needs to be between 01 and 12), 00 and 59 for minutes and seconds.

  • CHS_time_hms
  • CHS_time_hm

When a zero is present in numbers between 01 and 09, the engine ignores it.

This is a Chinese engine specific context tag.

return to the top of this page Back to top

CHS_time_hms

The text specifying the time is normalized so that the first group of numbers is the hour, the second group is the minute and the third group is the second.

This is a Chinese engine specific context tag.

return to the top of this page Back to top

CHS_time_hm

The text specifying the time is normalized so that the first group of numbers is the hour and the second group is the minute.

This is a Chinese engine specific context tag.

return to the top of this page Back to top

Number

This context specifies how to read the number passed to the engine.

  • number_cardinal
  • number_digit
  • number_fraction
  • number_decimal
  • CHS_number_percentage
  • CHS_number_scientific

number_cardinal

The text is normalized as a number using the regular format of ones, tens, etc.

This a common tag with the English engine.

return to the top of this page Back to top

number_digit

The text is normalized digit by digit.

This a common tag with the English engine.

return to the top of this page Back to top

number_fraction

The text is normalized as a fraction

This a common tag with the English engine.

return to the top of this page Back to top

number_decimal

The text is normalized as a decimal value.

This a common tag with the English engine.

return to the top of this page Back to top

CHS_number_percentage

The text is normalized as a percentage value.

This is a Chinese engine specific context tag.

return to the top of this page Back to top

CHS_number_scientific

The text is normalized as a scientific value.

This is a Chinese engine specific context tag.

return to the top of this page Back to top

Phone_Number/address_postal

The text is normalized as a phone number or address postal.

  • CHS_phone_postal

CHS_phone_postal

The phone number and address postal is almost the same as that of number_cardinal--the only difference is that "1" is read as "yao1"

This is a Chinese engine specific context tag.

return to the top of this page Back to top


Japanese Context tag definitions

The CONTEXT tag specifies the normalization of a block of text. This specification defines the SAPI predefined attributes (ID) for the CONTEXT tag. These IDs are strings. SAPI does not validate any parameters on the string passed to the engine, and hence, the application can specify engine-specific normalization IDs to the engine. Engine-specific strings begin with the engine vendor's name to avoid confusion between engines.

For example:
<CONTEXT ID = "MS_My_Context"> text </CONTEXT>

The exact implementation of some of these values is dependent on the engine in SAPI 5. In order to force a certain normalization, application developers can choose to normalize the text, or use another SAPI tag or engine-specific ID. Each context tag can contain more than one string.

For example:
<CONTEXT ID = "MS_My_Context"> text1 text2 text3 </CONTEXT>

A clearer example is shown in the following:

example:

The following topics are covered in this section:

  • Date
  • Time
  • Number
  • Phone_Number
  • Currency
  • Web
  • Address

Date

This context specifies that the number passed to the engine is a date. Dates generally have the format of number [delimiter] number [delimiter] number or number [delimiter] number where the delimiter may be a '.', '/' or '-', and numbers are typically between 01 and 12 for months, 01 and 31 for days. A year is generally a two- or four-digit number. If a date format does not fall within the range shown below, the application cannot expect a consistent result and the engine may interpret it freely. The valid string types are:

  • date_mdy
  • date_dmy
  • date_ymd
  • date_ym
  • date_my
  • date_dm
  • date_md
  • date_year

return to the top of this page Back to top

date_mdy

The text specifying the date is normalized so that the first group of numbers is the month, the second group is the day and the third group is the year. In the case where the year is a two-digit number, the engine reads it as a two-digit number or a four-digit number.

return to the top of this page Back to top

date_dmy

The text specifying the date is normalized so that the first group of numbers is the day, the second group is the month and the third group is the year. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is be represented as a four-digit year.

return to the top of this page Back to top

date_ymd

The text specifying the date is normalized so that the first group of numbers is the year, the second group is the month and the third group is the day. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is be represented as a four-digit year.

return to the top of this page Back to top

date_ym

The text specifying the date is normalized so that the first group of numbers is the year and the second group is the month. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is be represented as a four-digit year.

return to the top of this page Back to top

date_my

The text specifying the date is normalized so that the first group of numbers is the month and the second group is the year. In the case where the year is a two-digit number, the engine reads it as a two-digit number. If the year is represented as a four-digit number, it is be represented as a four-digit year.

return to the top of this page Back to top

date_dm

The text specifying the date is normalized so that the first group of numbers is the day and the second group is the month.

return to the top of this page Back to top

date_md

The text specifying the date is normalized so that the first group of numbers is the month and the second group is the day.

return to the top of this page Back to top

date_year

The text specifying the date is normalized so that the number is read as a year.

return to the top of this page Back to top

Time

This context specifies that the number passed to the engine is a time. Times generally have the format of number [delimiter] number [delimiter] number or number [delimiter] number where the delimiter is ':' or ''' or ' " 'and numbers are typically between 01 and 24 for hours, 01 and 59 for minutes and seconds.

When a zero is present in numbers between 01 and 09, the engine can ignore this, or normalize it as "oh". The engine can also place an "and" in the normalized time. The valid string types are:

return to the top of this page Back to top

Number

  • number_cardinal
  • number_digit
  • number_fraction
  • number_decimal

return to the top of this page Back to top

number_cardinal

The text is normalized as a number using the regular format of ones, tens, etc. The engine can place "and" in the normalized text.

return to the top of this page Back to top

number_digit

The text is normalized digit by digit.

return to the top of this page Back to top

number_fraction

The text is normalized as a fraction.

return to the top of this page Back to top

number_decimal

The text is normalized as a decimal value.

return to the top of this page Back to top

Phone_Number

The text is normalized as a phone number. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI. An example is provided below:

return to the top of this page Back to top

Currency

The text is normalized as a currency. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI. An example is provided below:

return to the top of this page Back to top

Web

web_url

The text is normalized as a URL. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI. An example is provided below:

return to the top of this page Back to top

Email

Email_address

The text is normalized as e-mail. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI.

return to the top of this page Back to top

Address

The text is normalized as an address. The exact implementation of this is left to the engine developer and may be defined in a future release of SAPI. An example is provided below:

return to the top of this page Back to top


ISpVoice

The ISpVoice interface enables an application to perform text synthesis operations. Applications can speak text strings and text files, or play audio files through this interface. All of these can be done synchronously or asynchronously.

Applications can choose a specific TTS voice using the ISpVoice::SetVoice method. The state of the voice (for example, rate, pitch, and volume), can be modified using SAPI XML tags that are embedded into the spoken text. Some attributes, like rate and volume, can be changed in real time using the ISpVoice::SetRate and ISpVoice::SetVolume methods. Voices can be set to different priorities using the ISpVoice::SetPriority.

ISpVoice inherits from the ISpEventSource interface. An ISpVoice object forwards events back to the application when the corresponding audio data has been rendered to the output device.

Associated Class IDs

The following class IDs (CLSID) may be used with this interface. A complete CLSID listing for all interfaces is in the Class IDs section. CLSID_SpVoice

Methods in Vtable Order

ISpVoice Methods | Description
ISpEventSource inherited methods | All methods of ISpEventSource are accessible from this interface
SetOutput | Sets the current output object. A value of NULL may be used to select the default audio device.
GetOutputObjectToken | Retrieves the object token for the current output object.
GetOutputStream | Retrieves a pointer to the current output stream.
Pause | Pauses the voice at the nearest alert boundary and closes the output device.
Resume | Sets the output device to the RUN state and resumes rendering.
SetVoice | Sets the identity of the voice used for text synthesis. By default, the ISpVoice will use the voice information set in the control panel.
GetVoice | Retrieves the object token that identifies the voice used in text synthesis.
Speak | Speaks the contents of a text string or file.
SpeakStream | Speaks the contents of a stream.
GetStatus | Retrieves the current rendering and event status associated with this ISpVoice instance.
Skip | Causes the voice to skip ahead the specified number of items within the current speak request.
SetPriority | Sets the priority for the voice. Normal, Alert, Over.
GetPriority | Retrieves the current voice priority level.
SetAlertBoundary | Specifies which event should be used as the insertion point for alerts.
GetAlertBoundary | Retrieves the event that is currently being used as the insertion point for alerts.
SetRate | Sets the text rendering rate adjustment in real time.
GetRate | Retrieves the current text rendering rate adjustment.
SetVolume | Sets the synthesizer output volume level in real time.
GetVolume | Retrieves the current output volume level of the synthesizer.
WaitUntilDone | Blocks the caller until either the voice has completed speaking or the specified time interval has elapsed.
SetSyncSpeakTimeout | Sets the timeout interval in milliseconds after which, synchronous Speak and SpeakStream calls to this instance of the voice will timeout.
GetSyncSpeakTimeout | Retrieves the timeout interval for synchronous speech operations for this ISpVoice instance.
SpeakCompleteEvent | Returns an event handle that will be signaled when the voice has completed speaking all pending requests.
IsUISupported | Determines if the specified type of UI is supported.
DisplayUI | Displays the requested UI.


ISpVoice::SetOutput

ISpVoice::SetOutput sets the current output object. The object may either be a stream, audio device, or an object token for an output audio device. If pUnkOutput is NULL the default audio device will be used.

HRESULT SetOutput(
   IUnknown   *pUnkOutput,
   BOOL        fAllowFormatChanges
);

Parameters

pUnkOutput
[in] IUnknown pointer to output object. This argument must be an object that implements ISpStreamFormat (a stream or audio device), or an object that implements ISpObjectToken. If a token is provided, this method will create the object described by the token and use it. If pUnkOutput is NULL, the default audio out device will be used.
fAllowFormatChanges
[in] Flag specifying whether the voice is allowed to change the format of the audio output object to match that of the engine, or a wav stream being spoken. If FALSE, the voice will use the SAPI format converter to translate between the data being rendered and the format of the output object. This should be set to TRUE if using the default audio device and the output format is of no consequence. If pUnkOutput is an ISpStreamFormat object, fAllowFormatChanges is ignored. In this case, the voice instance will render the output audio data in the format of the specified stream to the application.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
SPERR_UNINITIALIZED | pUnkOutput is an uninitialized ISpStream object.
E_OUTOFMEMORY | Exceeded available memory.
Example:

The following is an example of how to enumerate all the available audio output devices registered under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\AudioOutput.

HRESULT                  hr = S_OK;
CComPtr        cpAudioOutToken;
CComPtr        cpEnum;
CComPtr        cpVoice;
ULONG          ulCount = 0;

// Create the SAPI voice
if(SUCCEEDED(hr))
    hr = cpVoice.CoCreateInstance( CLSID_SpVoice );

//Enumerate the available audio output devices
if(SUCCEEDED(hr))
    hr = SpEnumTokens( SPCAT_AUDIOOUT, NULL, NULL, &cpEnum );

//Get the number of audio output devices
if(SUCCEEDED(hr))
    hr = cpEnum->GetCount( &ulCount );

// Obtain a list of available audio output tokens, set the output to the token, and call Speak
while ( SUCCEEDED(hr) && ulCount-- )
{
    if(SUCCEEDED(hr))
        hr = cpEnum->Next( 1, &cpAudioOutToken, NULL );

    if(SUCCEEDED(hr))
        hr = cpVoice->SetOutput( cpAudioOutToken, TRUE );

    if(SUCCEEDED(hr))
        hr = cpVoice->Speak( L"How are you?", SPF_DEFAULT, NULL );
}


ISpVoice::GetOutputObjectToken

ISpVoice::GetOutputObjectToken retrieves the object token for the current audio output object.

HRESULT GetOutputObjectToken(
   ISpObjectToken   **ppObjectToken
);

Parameters

ppObjectToken
[out] Address of an ISpObjectToken pointer that receives the audio output object token. If the current output is set to an ISpStream object, GetOutputObjectToken will return S_FALSE and ppObjectToken will be NULL.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppObjectToken is invalid.
S_FALSE | The current output stream does not have an object token


ISpVoice::GetOutputStream

ISpVoice::GetOutputStream retrieves a pointer to the current output stream.

HRESULT GetOutputStream(
   ISpStreamFormat   **ppStream
);

Parameters

ppStream
[out] Address of a pointer to an ISpStreamFormat object that receives the output stream.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppstream is invalid.


ISpVoice::Pause

ISpVoice::Pause pauses the voice at the nearest alert boundary and closes the output device, allowing access to pending speak requests from other voices. The voice maintains a pause count, so each call to pause must be balanced with a corresponding call to ISpVoice::Resume.

The default alert boundary is at the beginning of each word. See ISpVoice::SetAlertBoundary for details.

HRESULT  Pause  ( void );

Parameters

None.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message
Remarks:

Calling ISpVoice::Pause while the voice is not speaking increments the pause count and will put the voice into a paused state until ISpVoice::Resume is called the same number of times.


ISpVoice::Resume

ISpVoice::Resume decrements the pause count (which is incremented by ISpVoice::Pause) if the voice is currently paused. If the pause count hits zero, attempts to reclaim the output device and resumes rendering. This method has no effect if the voice was not in a paused state.

HRESULT  Resume  ( void );

Parameters

None.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpVoice::SetVoice

ISpVoice::SetVoice sets the identity of the voice used for text synthesis. ISpVoice normally uses the default voice, which is set through the Speech Control Panel.

HRESULT SetVoice(
   ISpObjectToken   *pToken
);

Parameters

pToken
[in] Pointer to token that describes the requested voice. If pToken is NULL, the system default voice is used.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.

Remarks

  • Changing the voice selection will preserve the same volume and rate levels for a ISpVoice object.

Example

The following is an example to enumerate all the available voices registered under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Voices.

HRESULT                             hr = S_OK;
CComPtr<ISpObjectToken>             cpVoiceToken;
CComPtr<IEnumSpObjectTokens>        cpEnum;
CComPtr<ISpVoice>                   cpVoice;
ULONG                               ulCount = 0;

// Create the SAPI voice
if(SUCCEEDED(hr))
    hr = cpVoice.CoCreateInstance( CLSID_SpVoice );

//Enumerate the available voices
if(SUCCEEDED(hr))
    hr = SpEnumTokens(SPCAT_VOICES, NULL, NULL, &cpEnum);

//Get the number of voices
if(SUCCEEDED(hr))
    hr = cpEnum->GetCount(&ulCount);

// Obtain a list of available voice tokens, set the voice to the token, and call Speak
while (SUCCEEDED(hr) && ulCount -- )
{
    cpVoiceToken.Release();
    if(SUCCEEDED(hr))
        hr = cpEnum->Next( 1, &cpVoiceToken, NULL );

    if(SUCCEEDED(hr))
        hr = cpVoice->SetVoice(cpVoiceToken);

    if(SUCCEEDED(hr))
        hr = cpVoice->Speak( L"How are you?", SPF_DEFAULT, NULL);
}


ISpVoice::GetVoice

ISpVoice::GetVoice retrieves the object token that identifies the voice currently in use. If there is not a voice currently currently in use, this method will return the token for the default voice.

HRESULT GetVoice(
   ISpObjectToken   **ppToken
);

Parameters

ppToken
[out] Pointer which will be set to point to the current voice's object token.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | ppToken is invalid.


ISpVoice::Speak

ISpVoice::Speak speaks the contents of a text string or file.

HRESULT Speak(
   const WCHAR   *pwcs,
   DWORD          dwFlags,
   ULONG         *pulStreamNumber
);

Parameters

pwcs
[in, string] Pointer to the null-terminated text string (possibly containing XML markup) to be synthesized. This value can be NULL when dwFlags is set to SPF_PURGEBEFORESPEAK indicating that any remaining data to be synthesized should be discarded. If dwFlags is set to SPF_IS_FILENAME, this value should point to a null-terminated, fully qualified path to a file.
dwFlags
[in] Flags used to control the rendering process for this call. The flag values are contained in the SPEAKFLAGS enumeration.
pulStreamNumber
[out] Pointer to a ULONG which receives the current input stream number associated with this Speak request. Each time a string is spoken, an associated stream number is returned. Events queued back to the application related to this string will contain this number. If NULL, no value is passed back.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Invalid pointer.
E_OUTOFMEMORY | Exceeded available memory.
SPERR_INVALID_FLAGS | Invalid flags specified for this operation.
SPERR_DEVICE_BUSY | Timeout occurred on synchronous call.

Remarks

  • Normally, pulStreamNumber will just be 1. If, however, several asynchronous Speak (or SpeakStream) calls are received and have to be queued, the stream number will be incremented for each call.


ISpVoice::SpeakStream

ISpVoice::SpeakStream is used to speak the contents of a stream.

HRESULT SpeakStream(
   IStream   *pStream,
   DWORD      dwFlags,
   ULONG     *pulStreamNumber
);

Parameters

pStream
[in] Address of an IStream interface containing the input stream. If the ISpStreamFormat interface is not implemented by the input stream the format type is assumed to be SPDFID_Text.
dwFlags
[in] Flags used to control the rendering process for this call. The flag values are contained in the SPEAKFLAGS enumeration, however the SPF_IS_FILENAME flag is not used for SpeakStream.
pulStreamNumber
[out] Pointer to a ULONG which receives the current input stream number associated with this SpeakStream request. Each time a string is spoken, an associated stream number is returned. Events queued back to the application related to this string will contain this number.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Invalid pointer.
E_OUTOFMEMORY | Exceeded available memory.
SPERR_INVALID_FLAGS | Invalid flags specified for this operation.
SPERR_DEVICE_BUSY | Timeout on synchronous call.

Remarks

  • If the input stream is wav data, it is sent directly to the output object.
  • If the input stream is text data, it is processed by the text-to-speech (TTS) engine.
  • Normally, pulStreamNumber will just be 1. If, however, several asynchronous SpeakStream (or Speak) calls are received and have to be queued, the stream number will be incremented for each call.


ISpVoice::GetStatus

ISpVoice::GetStatus retrieves the current rendering and event status associated with this voice.

HRESULT GetStatus(
   SPVOICESTATUS  *pStatus,
   WCHAR         **ppszLastBookmark
);

Parameters

pStatus
[out] Pointer to an SPVOICESTATUS structure which receives the status information. This pointer can be NULL if the caller does not want this information.
ppszLastBookmark
[out, string] Pointer to a pointer which receives a CoTaskMemAlloc allocated null-terminated string containing the text of the last bookmark reached. If there is no last bookmark, NULL will be returned. Applications calling this method must call CoTaskMemFree() to free the returned string. This pointer can be NULL if the caller does not want this information.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.
E_OUTOFMEMORY | Exceeded available memory.


ISpVoice::Skip

ISpVoice::Skip causes the voice to skip forward or backward the specified number of items within the text of the current speak call.

HRESULT Skip(
   WCHAR   *pItemType,
   long     lNumItems,
   ULONG   *pulNumSkipped
);

Parameters

pItemType
[in,string] Specifies the type of item to skip. Currently "SENTENCE" is the only type supported.
lNumItems
[in] Specifies the number of items to skip in the current speak request. If lNumItems is a positive number, the voice will skip forward, and if it is negative, the voice will skip backward. If lNumItems is 0, the voice will skip back to the beginning of the current item.
pulNumSkipped
[out] Pointer to a ULONG which will be set to the actual number of items skipped.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pItemType is invalid or bad.
E_POINTER | pulNumSkipped is invalid or bad.
SPERR_VOICE_PAUSED | Voice is in a paused state and may not be skipped.


ISpVoice::SetPriority

ISpVoice::SetPriority sets the priority for the voice. The default priority is SPVPRI_NORMAL.

HRESULT SetPriority(
   SPVPRIORITY   ePriority
);

Parameters

ePriority
[in] Priority of type SPVPRIORITY associated with the current voice.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | ePriority is not an acceptable priority value.

Remarks

  • Assuming an output object which implements ISpAudio, speak requests of similar priority voices are queued, and are spoken one at a time in the order they are issued. That is, speak requests from normal priority voices are put in one queue, while speak requests from alert priority voices (with priority SPVPRI_ALERT) are put in another queue.

  • Alert priority voices take priority over normal voices. If there are one or more speak requests from alert priority voices pending, a normal voice that is speaking will be interrupted on the next alert boundary (see ISpVoice::SetAlertBoundary), all the queued alert priority voice speak requests will be processed, and then the normal voice will continue.

  • Voices with the SPVPRI_OVER priority speak over (mix with) all other audio in the system with no synchronization. SPVPRI_OVER priority voices only mix on Windows 2000.

  • If the output object does not implement ISpAudio, no serialization will occur, and all voices will be treated as if their priority is SPVPRI_OVER.


ISpVoice::GetPriority

ISpVoice::GetPriority retrieves the current voice priority level. See ISpVoice::SetPriority for more information on voice priorities.

HRESULT GetPriority(
   SPVPRIORITY   *pePriority
);

Parameters

pePriority
[out] Pointer to priority information of type SPVPRIORITY.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.


ISpVoice::SetAlertBoundary

ISpVoice::SetAlertBoundary specifies which event should be used as the insertion point for alert priority voice interruptions (see ISpVoice::SetPriority for more details on voice priorities) and ISpVoice::Pause calls.

HRESULT SetAlertBoundary(
   SPEVENTENUM   eBoundary
);

Parameters

eBoundary
[in] SPEVENTENUM enumeration value that specifies which event to use for the alert insertion point. Appropriate events to use for this purpose include SPEI_WORD_BOUNDARY, SPEI_SENTENCE_BOUNDARY, SPEI_PHONEME, SPEI_VISEME, and possibly SPEI_VOICE_CHANGE or SPEI_TTS_BOOKMARK (though using these last two might be a bit strange).

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | eBoundary is invalid.

Remarks

  • Events, such as word and sentence boundaries, are queued by the TTS engine during text rendering. Alert priority voices will only be able to interrupt normal priority voices, and pauses will only be able to occur, on the specified event boundaries.

  • The default alert boundary event is SPEI_WORD_BOUNDARY.


ISpVoice::GetAlertBoundary

ISpVoice::GetAlertBoundary retrieves the event that is currently being used as the insertion point for alerts. For more information on alert boundaries, see ISpVoice::SetAlertBoundary.

HRESULT GetAlertBoundary(
   SPEVENTENUM   *peBoundary
);

Parameters

peBoundary
[out] Address of an SPEVENTENUM that receives the event type of the alert boundary.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.


ISpVoice::SetRate

ISpVoice::SetRate sets the speaking rate of the voice in real time. The default rate for a voice is set through the Speech Control Panel.

HRESULT SetRate(
   long   RateAdjust
);

Parameters

RateAdjust
[in] Value specifying the speaking rate of the voice. Supported values range from -10 to 10 - values outside this range may be truncated.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.

Remarks

  • Voices do not all have the same default rate.
  • The granularity of the rate is engine dependent.
  • Applications can adjust the rate of a voice either through this function call, or through XML passed to the voice with the input text of a speak call (see the XML Schema : SAPI white paper). The voice should combine rate adjustments made in these two ways to arrive at a final rate.


ISpVoice::GetRate

ISpVoice::GetRate retrieves the current base rate (either the default rate or the last value set by ISpVoice::SetRate).

HRESULT GetRate(
   long   *pRateAdjust
);

Parameters

pRateAdjust
[out] Pointer to a long which receives the base rate.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pRateAdjust is invalid.


ISpVoice::SetVolume

ISpVoice::SetVolume sets the base volume level of the voice in real time. The default base volume for all voices is 100.

HRESULT SetVolume(
   USHORT 	usVolume
);

Parameters

usVolume
[in] Value containing the requested volume level. Volume levels are specified in percentage values ranging from 0 to 100 - values outside this range may be truncated.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.

Remarks

  • Volume is specified as a percentage of the maximum volume of the current voice. Different voices may have different maximum volume levels.
  • Applications can adjust the volume of a voice either through this function call, or through XML passed to the voice with the input text of a speak call (see the XML Schema : SAPI white paper). The voice should combine volume adjustments made in these two ways to arrive at a final volume.


ISpVoice::GetVolume

ISpVoice::GetVolume retrieves the current base volume level (either the default volume level or the last level set by ISpVoice::SetVolume).

HRESULT GetVolume(
   USHORT   *pusVolume
);

Parameters

pusVolume
[out] Address to receive the current base volume level.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pusVolume is invalid.


ISpVoice::WaitUntilDone

ISpVoice::WaitUntilDone blocks the caller until either the voice has completed speaking or the specified time interval has elapsed.

HRESULT WaitUntilDone(
   ULONG   msTimeout
);

Parameters

msTimeout
[in] Timeout period in milliseconds. INFINITE may be used to prevent this method from timing out.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | Wait time interval was exceeded.

Notes:

  • This call may be used after a single asynchronous Speak (or SpeakStream) call, or after several calls have been queued. In either case it will return only after all pending calls have been completed (or after the specified time interval has elapsed).


ISpVoice::SetSyncSpeakTimeout

ISpVoice::SetSyncSpeakTimeout sets the timeout interval in milliseconds after which synchronous Speak and SpeakStream calls to this voice will timeout.

HRESULT SetSyncSpeakTimeout(
   ULONG   msTimeout
);

Parameters

msTimeout
[in] Value specifying the timeout interval in milliseconds. The default is 10 seconds. INFINITE may also be used to prevent timeouts.

Return values

Value | Description
S_OK | Function completed successfully.

Remarks:

  • Timeouts occur when waiting for access to the output object. This means that for a normal priority voice (see ISpVoice::SetPriority for more information on priorities) and an output device which implements ISpAudio, a timeout may occur while waiting to reacquire the output object after an interruption by an alert priority voice. For voices of both normal and alert priorities, a timeout may also occur while waiting to reacquire the output object after the voice has been paused and resumed (see ISpVoice::Pause and ISpVoice::Resume).

  • Wait times are not accumulated - that is, if a voice waits for n milliseconds to initially acquire the output object, and is then paused and resumed, it will again wait for up to msTimeout milliseconds to reacquire the output object, not msTimeout - n milliseconds.


ISpVoice::GetSyncSpeakTimeout

ISpVoice::GetSyncSpeakTimeout retrieves the timeout interval for synchronous speech operations for this voice. For more information on timeouts, see ISpVoice::SetSyncSpeakTimeout.

HRESULT GetSyncSpeakTimeout(
   ULONG   *pmsTimeout
);

Parameters

pmsTimeout
[out] Pointer to a ULONG which receives the timeout interval in milliseconds for synchronous speech operations.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pmsTimeout is invalid.


ISpVoice::SpeakCompleteEvent

ISpVoice::SpeakCompleteEvent returns an event handle that will be signaled when the voice has completed speaking all pending requests. This is similar to the functionality provided by ISpVoice::WaitUntilDone, but allows the caller to wait on the event handle.

[local] HANDLE  SpeakCompleteEvent  ( void );

Parameters

None.

Return values

Value | Description
Event Handle | For WAIT operation.

Remarks:

  • The caller should not call CloseHandle(), nor should the caller ever use the handle after releasing the COM reference to this voice.


ISpVoice::IsUISupported

ISpVoice::IsUISupported checks if the underlying text-to-speech engine's object token supports the requested UI.

See the TTS Engine White Paper for further information on how a TTS Engine should implement UI.

[local] HRESULT IsUISupported(
    const WCHAR   *pszTypeOfUI,
    void          *pvExtraData,
    ULONG          cbExtraData,
    BOOL          *pfSupported
);

Parameters

pszTypeOfUI
[in] Address of the null-terminated string containing the UI type that is being queried.
pvExtraData
[in] Pointer to additional information needed for the object. The TTS Engine implementer dictates the format and usage of the data provided.
cbExtraData
[in] Size, in bytes, of the ExtraData. The TTS Engine implementer dictates the format and usage of the data provided.
pfSupported
[out] Flag specifying whether the specified UI is supported. TRUE indicates the UI is supported, and FALSE indicates the UI is not supported. If this value is TRUE, but the return code is S_FALSE, the UI type (pszTypeOfUI) is supported, but not with the current parameters or runtime environment. Check with the engine implementer to verify runtime requirements.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The UI is supported but not with the current runtime environment or parameters.
E_INVALIDARG | One or more arguments are invalid.
FAILED(hr) | Appropriate error message.

Example

The following code snippet illustrates the use of ISpVoice::IsUISupported using SPDUI_EngineProperties.

    HRESULT hr = S_OK;

    // display properties UI for the current TTS engine
    hr = cpVoice->IsUISupported(SPDUI_EngineProperties, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then current TTS engine supports properties UI


ISpVoice::DisplayUI

ISpVoice::DisplayUI displays the UI from the underlying text-to-speech engine's object token.

The best-practice for using ISpVoice::DisplayUI is to call ISpVoice::IsUISupported with a specific UI type before calling DisplayUI.

See the TTS Engine White Paper for further information on how a TTS Engine should implement UI.

The call to DisplayUI is synchronous, so the call will not return until the UI has been closed.

[local] HRESULT DisplayUI(
    HWND           hwndParent,
    const WCHAR   *pszTitle,
    const WCHAR   *pszTypeOfUI,
    void          *pvExtraData,
    ULONG          cbExtraData
);

Parameters

hwndParent
[in] Specifies the parent window handle information.
pszTitle
[in] Address of a null-terminated string containing the window title information. You can set this value to NULL to indicate that the TTS engine should use its default window title for this UI type.
pszTypeOfUI
[in] Address of the null-terminated string containing the requested UI type to display.
pvExtraData
[in] Pointer to additional information needed for the object. The TTS Engine implementer dictates the format and usage of the data provided.
cbExtraData
[in] Size, in bytes, of the ExtraData. The TTS Engine implementer dictates the format and usage of the data provided.

Example

The following code snippet illustrates the use of ISpVoice::DisplayUI using SPDUI_EngineProperties.

    HRESULT hr = S_OK;

    // display properties UI for the current TTS engine
    hr = cpVoice->DisplayUI(MY_HWND, MY_APP_VOICE_PROPERTIES, SPDUI_EngineProperties, NULL, NULL);
    // Check hr

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The UI is supported but not with the current runtime environment or parameters.
E_INVALIDARG | One or more arguments are invalid.
FAILED(hr) | Appropriate error message.


Engine-Level Interfaces

This section describes the interfaces and methods for incorporating speech engines into applications. They are intended for use at the DDI or device driver interface level. Some managers or interfaces may have entries also in the Application-Level Interfaces section. However, entries listed here apply only to the device driver or engine level.

  • Grammar Compiler interfaces
  • Resource interfaces
  • Speech Recognition interfaces
  • Speech Recognition Engine interfaces
  • Text-to-Speech Engine interfaces


Grammar Compiler Manager (DDI-level)

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Grammar Compiler interfaces (DDI-level)

The following section covers:

  • ISpErrorLog
  • ISpGramCompBackend
  • ISpGrammarCompiler
  • ISpITNProcessor
  • ISpCFGInterpreter
  • ISpCFGInterpreterSite


ISpErrorLog

An object implementing this interface can be supplied to ISpGrammarCompiler::CompileStream() to receive compilation error messages.

Methods in Vtable Order

ISpErrorLog Methods | Description
AddError | Writes an error to the log file.


ISpErrorLog::AddError

ISpErrorLog::AddError will be called by ISpGrammarCompiler::CompileStream() when a compilation error occurs. Application can implement this method to process the compilation error messages.

HRESULT AddError(
   const long     lLineNumber,
   HRESULT        hr,
   const WCHAR   *pszDescription,
   const WCHAR   *pszHelpFile,
   DWORD          dwHelpContext
);

Parameters

lLineNumber
The line number of the error in the XML grammar file.
hr
The error code being logged.
pszDescription
A textual description of the error.
pszHelpFile
The file being written to.
dwHelpContext
Flags providing additional information for the log.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message. Currently any hr returned from AddError will be ignored by SAPI.

Because this method is application defined, the return value may change. See specific vendor documentation for details.


ISpGramCompBackend

ISpGramCompBackend inherits from the ISpGrammarBuilder interface.

Methods in Vtable Order

ISpGramCompBackend Methods | Description
SetSaveObjects | Sets the storage location of the binary grammar.
InitFromBinaryGrammar | Initializes a grammar from binary data.


ISpGramCompBackend::SetSaveObjects

ISpGramCompBackend::SetSaveObjects sets the storage location of the binary grammar.

When the ISpGrammarBuilder::Commit method is called, the grammar compiler back end writes the binary grammar to the location of pStream. When calling the SetSaveObjects method multiple times, the last call made before calling the Commit method, receives the binary grammar.

HRESULT SetSaveObjects(
   IStream      *pStream,
   ISpErrorLog  *pErrorLog
);

Parameters

pStream
Address of the IStream that receives the binary grammar.
pErrorLog
Address of the ISpErrorLog interface that receives the error information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Either pStream or pErrorLog is bad or invalid.
FAILED(hr) | Appropriate error message.


ISpGramCompBackend::InitFromBinaryGrammar

ISpGramCompBackend::InitFromBinaryGrammar initializes a grammar from binary data.

HRESULT InitFromBinaryGrammar(
   const   SPBINARYGRAMMAR  *pBinaryData
);

Parameters

pBinaryData
Pointer to the grammar list.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pBinaryData is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.
FAILED (hr) | Appropriate error message.


ISpGrammarCompiler

SAPI 5 Text Grammar compiler.

Methods in Vtable Order

ISpGrammarCompiler Methods | Description
CompileStream | Loads the XML and produces the binary grammar format.


ISpGrammarCompiler::CompileStream

ISpGrammarCompiler::CompileStream compiles the SAPI 5 Speech Text Grammar pointed to by pSource stream and writes the output to the pDest stream. It can optionally generate C/C++ header information from the <DEFINE> <ID> tags.

HRESULT CompileStream(
   IStream       *pSource,
   IStream       *pDest,
   IStream       *pHeader,
   IUnknown      *pReserved,
   ISpErrorLog   *pErrorLog,
   DWORD          dwFlags
);

Parameters

pSource
Pointer to the source of the XML grammar text.
pDest
Pointer to the destination stream for the binary grammar.
pHeader
Pointer to the stream to write the C/C++ header information (from the <DEFINE> tags) to (e.g., #define myterm 3).
pReserved
Reserved. Do not use.
pErrorLog
Pointer to the error log receiving the messages.
dwFlags
[in] Not currently used. Must be zero.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One of the parameters is bad or invalid.
FAILED (hr) | Appropriate error message.


ISpITNProcessor

ISpITNProcessor interface is implemented by SAPI to do the Inverse Text Normalization (ITN).

Methods in Vtable Order

ISpITNProcessor Methods | Description
LoadITNGrammar | Loads an inverse text normalization grammar.
ITNPhrase | Parses an inverse text normalization phrase.


ISpITNProcessor::LoadITNGrammar

ISpITNProcessor::LoadITNGrammar loads an inverse text normalization (ITN) grammar. The loaded grammar can be used by either SAPI or the speech recognition (SR) engine.

HRESULT LoadITNGrammar(
   WCHAR   *pszCLSID
);

Parameters

pszCLSID
Address of the null-terminated string containing the CLSID of the ITN grammar object implementing ISpCFGInterpreter.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pszCLSID is invalid or bad.
FAILED(hr) | Appropriate error message.


ISpITNProcessor::ITNPhrase

ISpITNProcessor::ITNPhrase parses an inverse text normalization (ITN) phrase.

The ITNPhrase will attempt to parse the pPhrase passed in using the ITN grammar loaded by ISpITNProcessor::LoadITNGrammar. If a parse is found, the ITN grammar will add the display text replacement. For example, AddReplacement "$100" for "one hundred dollars".

HRESULT ITNPhrase(
   ISpPhraseBuilder   *pPhrase
);

Parameters

pPhrase
Address of the phrase to parse.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | No words are available.
SP_NO_RULE_ACTIVE | No rules are available.
E_OUTOFMEMORY | Not enough memory to complete operation.
FAILED(hr) | Appropriate error message.


ISpCFGInterpreter

Methods in Vtable Order

ISpCFGInterpreter Methods | Description
InitGrammar | Initializes a grammar that is loaded from an object or DLL.
Interpret | Examines a grammar and generates new properties and text replacements.


ISpCFGInterpreter::InitGrammar

ISpCFGInterpreter::InitGrammar initializes a grammar that is loaded from an object or DLL.

HRESULT InitGrammar(
   const WCHAR   *pszGrammarName,
   const void   **pvGrammarData
);

Parameters

pszGrammarName
[in] Address of a null-terminated string of the grammar to be loaded.
pvGrammarData
[in] Address of a pointer to the serialized binary grammar.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpCFGInterpreter::Interpret

ISpCFGInterpreter::Interpret examines a grammar and generates new properties and text replacements.

HRESULT Interpret(
   ISpPhraseBuilder        *pPhrase,
   const ULONG              ulFirstElement,
   const ULONG              ulCountOfElements,
   ISpCFGInterpreterSite   *pSite
);

Parameters

pPhrase
[in] Address of the ISpPhraseBuilder interface containing the phrase information.
ulFirstElement
[in] Value specifying the location on the first element in pPhrase.
ulCountOfElements
[in] Value specifying the number of phrase elements in pPhrase.
pSite
[in] Address of the ISpCFGInterpreterSite interface containing which can be used to attach semantic properties or text replacements to the parent phrase.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpCFGInterpreterSite

This interface is used by rule interpreters to set properties and text replacements in the parent phrase.

Methods in Vtable Order

ISpCFGInterpreterSite Methods | Description
AddTextReplacement | Adds one text replacement to the phrase.
AddProperty | Adds a property entry to the phrase object.
GetResourceValue | Retrieves the resource information for a grammar.


ISpCFGInterpreterSite::AddTextReplacement

ISpCFGInterpreterSite::AddTextReplacement adds one text replacement to the phrase. The object must have been initialized by calling SetPhrase prior to calling this method.

HRESULT AddTextReplacement(
   SPPHRASEREPLACEMENT   *pReplace
);

Parameters

pReplace
[in] Address of the SPPHRASEREPLACEMENT that contains the replacement text.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pReplace is invalid or bad.
SPERR_UNINITIALIZED | The object is uninitialized.
FAILED(hr) | Appropriate error message.


ISpCFGInterpreterSite::AddProperty

ISpCFGInterpreterSite::AddProperty adds a property entry to the phrase object.

HRESULT AddProperty(
  const  SPPHRASEPROPERTY   *pProperty
);

Parameters

pProperty
[in] Address of the SPPHRASEPROPERTY structure that contains the property information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pProperty is bad or invalid.
SPERR_UNINITIALIZED | The object is uninitialized.
FAILED(hr) | Appropriate error message.


ISpCFGInterpreterSite::GetResourceValue

ISpCFGInterpreterSite::GetResourceValue retrieves the resource information for a grammar.

HRESULT GetResourceValue(
   const WCHAR     *pszResourceName,
   WCHAR          **ppCoMemResource
);

Parameters

pszResourceName
[in] The name of the resource from which to retrieve the grammar information.
ppCoMemResource
[out] Pointer containing the passed back resource value.
Applications implementing this method must call CoTaskMemFree() to free memory associated with this resource.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One of the parameters is bad or invalid.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.


Resource Manager (DDI-level)

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Resource Manager interfaces (DDI-level)

The following section covers:

  • ISpObjectTokenEnumBuilder
  • ISpTokenUI
  • ISpTaskManager
  • ISpThreadControl
  • ISpThreadTask


ISpObjectTokenEnumBuilder

This interface inherits from IEnumSpObjectTokens.

Methods in Vtable Order

ISpObjectTokenEnumBuilder Methods | Description
SetAttribs | Sets the required and optional token enumerator attribute information.
AddTokens | Adds tokens to the object token enumerator.
AddTokensFromDataKey | Adds a new token using specified subkey and CategoryId information.
AddTokensFromTokenEnum | Adds a new token from an enumerated list of object tokens.
Sort | Sorts the list of enumerated object tokens.


ISpObjectTokenEnumBuilder::SetAttribs

ISpObjectTokenEnumBuilder::SetAttribs sets the required and optional token enumerator attribute information. This function can be called only once for the same object.

HRESULT SetAttribs(
   const WCHAR   *pszReqAttribs,
   const WCHAR   *pszOptAttribs
);

Parameters

pszReqAttribs
Address of a null-terminated string containing the required attribute information.
pszOptAttribs
Address of a null-terminated string containing the optional attribute information.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_ALREADY_INITIALIZED | The object has already been initialized.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.


ISpObjectTokenEnumBuilder::AddTokens

ISpObjectTokenEnumBuilder::AddTokens adds tokens to the object token enumerator.

HRESULT AddTokens(
   ULONG              cTokens,
   ISpObjectToken   **pToken
);

Parameters

cTokens
The number of object tokens being added to the sequence.
pToken
Address of a pointer to an ISpObjectToken object containing the information associated with the tokens being added.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Invalid pointer.
E_OUTOFMEMORY | Exceeded available memory.
SPERR_UNINITIALIZED | The object has not been properly initialized.
FAILED(hr) | Appropriate error message.


ISpObjectTokenEnumBuilder::AddTokensFromDataKey

ISpObjectTokenEnumBuilder::AddTokensFromDataKey adds a new token using specified subkey and CategoryId information.

HRESULT AddTokensFromDataKey(
   ISpDataKey    *pDataKey,
   const WCHAR   *pszSubKey,
   const WCHAR   *pszCategoryId
);

Parameters

pDataKey
Address of an ISpDataKey interface that specifies the system registry key from which to create the token.
pszSubKey
Address of a null-terminated string containing the system registry subkey name.
pszCategoryId
Address of a null-terminated string containing the category identifier information for the system registry subkey.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
SPERR_UNINITIALIZED | The object has not been properly initialized.
FAILED(hr) | Appropriate error message.


ISpObjectTokenEnumBuilder::AddTokensFromTokenEnum

ISpObjectTokenEnumBuilder::AddTokensFromTokenEnum adds new tokens from an enumerated list of object tokens.

HRESULT AddTokensFromTokenEnum(
   IEnumSpObjectTokens   *pTokenEnum
);

Parameters

pTokenEnum
Address of an IEnumSpObjectTokens interface containing the list of enumerated object tokens to add.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
SPERR_UNINITIALIZED | The object has not been properly initialized.
FAILED(hr) | Appropriate error message.


ISpObjectTokenEnumBuilder::Sort

ISpObjectTokenEnumBuilder::Sort sorts the list of enumerated object tokens.If the optional attributes of the EnumBuilder have been set, the first token in the EnumBuilder after the Sort call will be the token that matches the optional attributes best, not the token that pszTokenIdToListFirst refers to.

HRESULT Sort(
   const WCHAR   *pszTokenIdToListFirst
);

Parameters

pszTokenIdToListFirst
Address of a null-terminated string of tokenId for the first token in the sorted list.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.
SPERR_UNINITIALIZED | The object has not been properly initialized.
FAILED(hr) | Appropriate error message.


ISpTokenUI

Provides developers with a means to programmatically manage user-interface associated with an ISpObjectToken.

When To Implement

The ISpTokenUI interface should be implemented if the object wants to allow other applications to display the UI. For example, an SR Engine (see ISpRecognizer) has a UI for Training (see SPDUI_UserTraining), and it would be helpful for an application to be able to display the Training UI as appropriate.

Example

The following code snippet illustrates the use of ISpTokenUI using QueryInterface.

    HRESULT hr = S_OK;

    // find the preferred multimedia input object token
    hr = SpFindBestToken(SPCAT_AUDIOIN, L"Technology=MMSys", NULL, &cpObjectToken);
    // Check hr

    // get the multimedia object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

The following code snippet illustrates the use of ISpTokenUI using CoCreateInstance. The user must know the exact CLSID of the intended UI object.

    HRESULT hr = S_OK;

    // create the Token UI for the UI object CLSID_MY_TOKEN_UI
    hr = cpTokenUI.CoCreateInstance(CLSID_MY_TOKEN_UI);
    // Check hr

Methods in Vtable Order

ISpTokenUI Methods | Description
IsUISupported | Determines if the specified UI type is supported by the token.
DisplayUI | Displays the UI associated with the object token.

Development Helpers

Helper Functions | Description
SpCreateBestObject | Creates the most appropriate object based on specific criteria.
SpFindBestToken | Finds the most appropriate IspObjectToken based on specific criteria.
SpCreateObjectFromToken | Creates an object based on a specified ISpObjectToken.
SpCreateDefaultObjectFromCategoryId | Creates the default object from a specific category.
SpGetTokenFromId | Creates an ISpObjectToken based on a token id.
SpGetDefaultTokenFromCategoryId | Creates the default ISpObjectToken from a specific category.


ISpTokenUI::IsUISupported

ISpTokenUI::IsUISupported determines if the specified UI type is supported by the token.

[local] HRESULT IsUISupported(
   const WCHAR   *pszTypeOfUI,
   void          *pvExtraData,
   ULONG          cbExtraData,
   IUnknown      *punkObject,
   BOOL          *pfSupported
);

Parameters

pszTypeOfUI
[in] Address of a null-terminated string containing the object's UI type.
pvExtraData
[in] Pointer to additional information needed for the object. The ISpTokenUI object implementer dictates the format and usage of the data provided.
cbExtraData
[in] Size, in bytes, of the ExtraData. The ISpTokenUI object implementer dictates the format and usage of the data provided.
punkObject
[in] Address of the object's IUnknown interface. See Remarks section.
pfSupported
[out] Address of a variable that receives the value indicating support for the interface. This value is set to TRUE when this interface is supported and FALSE otherwise. If this value is TRUE, but the return code is S_FALSE, the UI type (pszTypeOfUI) is supported, but not with the current parameters or runtime environment. Check with the implementer of the UI object to verify runtime requirements.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The UI is supported but not with the current runtime environment or parameters.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Invalid or bad pointer.
FAILED(hr) | Error returned by UI object.

Remarks

When asking a token to display a particular piece of UI, the token may require extra functionality that only it understands. Common implementation practice for accessing this functionality is to QueryInterface off of a known IUnknown interface. The caller of ISpTokenUI::IsUISupported can set the punkObject parameter with the necessary IUnknown interface. For example, asking to display Speech Recognition Training UI requires that a specific SR Engine be used.

Example

The following code snippet illustrates the use of ISpTokenUI::IsUISupported using SPDUI_AudioProperties.

    HRESULT hr = S_OK;

    // get the default input audio object token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_AUDIOIN, &cpObjectToken);
    // Check hr

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // check if the default audio input object has UI for Properties
    hr = cpTokenUI->IsUISupported(SPDUI_AudioProperties, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then default audio input object has UI for Properties


ISpTokenUI::DisplayUI

ISpTokenUI::DisplayUI displays the UI associated with the object token.

The best-practice for using ISpTokenUI is to call ISpTokenUI::IsUISupported with a specific UI type before calling DisplayUI.

The call to DisplayUI is synchronous, so the call will not return until the UI has been closed.

[local] HRESULT DisplayUI(
    HWND              hwndParent,
    const WCHAR      *pszTitle,
    const WCHAR      *pszTypeOfUI,
    void             *pvExtraData,
    ULONG             cbExtraData,
    ISpObjectToken   *pToken,
    IUnknown         *punkObject
);

Parameters

hwndParent
[in] Specifies the handle of the parent window.
pszTitle
[in] Address of a null-terminated string containing the window title to display on the UI. This value can be set to NULL to indicate that the TokenUI object should use its default window title.
pszTypeOfUI
[in] Address of a null-terminated string containing the UI type to display.
pvExtraData
[in] Pointer to additional information needed for the object. The ISp TokenUI object implementer dictates the format and usage of the data provided.
cbExtraData
[in] Size, in bytes, of the ExtraData. The ISp TokenUI object implementer dictates the format and usage of the data provided.
pToken
[in] Address of the ISpObjectToken containing the object token identifier. See Remarks section.
punkObject
[in] Address of the IUnknown interface pointer. See Remarks section.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | The UI is supported but not with the current runtime environment or parameters.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Invalid or bad pointer.
FAILED(hr) | Error returned by UI object.

Remarks

When asking a token to display a particular piece of UI, the token may require extra functionality that only it understands. Common implementation practice for accessing this functionality is to QueryInterface off of a known IUnknown interface or create the object associated a known ISpObjectToken instance. The caller of ISpTokenUI::DisplayUI can set punkObject with the necessary IUnknown interface or set pToken with the necessary ISpObjectToken interface. For example, asking to display Speech Recognition Training UI requires that a specific SR Engine be used.

Example

The following code snippet illustrates the use of ISpTokenUI::DisplayUI using SPDUI_AudioVolume.

    HRESULT hr = S_OK;

    // get the default input audio object token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_AUDIOIN, &cpObjectToken);
    // Check hr

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // Check if default audio input object has UI for volume
    hr = cpTokenUI->IsUISupported(SPDUI_AudioVolume, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then default audio input object has UI for Volume

    // Display the default audio input object's Volume UI
    hr = cpTokenUI->DisplayUI(MY_HWND, MY_AUDIO_DIALOG_TITLE, SPDUI_AudioVolume, NULL, NULL, cpObjectToken, NULL);
    // Check hr


ISpTaskManager

When to Implement

This interface is used to implement a task management service provider to optimize thread usage.

Associated Class IDs

The following class IDs (CLSID) may be used with this interface. A complete CLSID listing for all interfaces is in the Class IDs section.

    CLSID_SpResourceManager

Methods in Vtable Order

ISpTaskManager Methods | Description
SetThreadPoolInfo | Sets the attributes for thread pool management.
GetThreadPoolInfo | Retrieves the current thread pool management attributes.
QueueTask | Adds a task to the queue for asynchronous task processing.
CreateReoccurringTask | Creates a task entry that will be processed on a high priority thread.
CreateThreadControl | Creates a thread control object.
TerminateTask | Interrupts a specified task.
TerminateTaskGroup | Terminates a group of tasks that match a specific group identifier.


ISpTaskManager::SetThreadPoolInfo

ISpTaskManager::SetThreadPoolInfo defines the thread pool attributes.

HRESULT SetThreadPoolInfo(
   const   SPTMTHREADINFO   *pPoolInfo
);

Parameters

pPoolInfo
[in] Address of an SPTMTHREADINFO structure that receives the thread management information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pPoolInfo is invalid or pPoolInfo->lPoolSize size is less than -1.
FAILED (hr) | Appropriate error message.


ISpTaskManager::GetThreadPoolInfo

ISpTaskManager::GetThreadPoolInfo retrieves the current thread pool management attributes.

HRESULT GetThreadPoolInfo(
   SPTMTHREADINFO   *pPoolInfo
);

Parameters

pPoolInfo
[out] Address of an SPTMTHREADINFO structure that contains the current thread management information.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pPoolInfo is invalid or bad.
FAILED (hr) | Appropriate error message.


ISpTaskManager::QueueTask

ISpTaskManager::QueueTask adds a task to the queue for asynchronous task processing.

HRESULT QueueTask(
   ISpTask   pTask,
   void     *pvTaskData,
   HANDLE    hCompEvent,
   DWORD*   *pdwGroupId,
   DWORD*   *pTaskID
);

Parameters

pTask
[in] Address of an ISpTask interface containing the task.
pvTaskData
[in] Address of the task data that will be passed to the ISpTask::Execute method.
hCompEvent
[in] Handle of the task completion event. This event will be set when the Execute method returns. This paramerter can be NULL.
pdwGroupId
[in, out] Value specifying the identifier for the task group. This value may be NULL. This can be used to cancel a group of pending tasks.
pTaskID
[out] Value specifying the task identifier. This parameter can be NULL if this information is not needed.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pTask is invalid or bad.
E_POINTER | pTaskId or pdwGroupId is invalid or bad.
FAILED (hr) | Appropriate error message.


ISpTaskManager::CreateReoccurringTask

ISpTaskManager::CreateReoccurringTask creates a task entry that will be processed on a thread when the ISpNotifySink::Notify method is called on the task control object.

HRESULT CreateReoccurringTask(
   ISpTask          *pTask,
   void             *pvTaskData,
   HANDLE            hCompEvent,
   ISpNotifySink   **ppTaskCtrl
);

Parameters

pTask
[in] Address of an ISpTask interface containing the task.
pvTaskData
[in] Pointer that will be passed to the ISpTask::Execute method.
hCompEvent
[in] Handle of the task completion event. This is optional and can be NULL. If non-NULL, this event handle will be signaled when the Execute method returns
ppTaskCtrl
[out] Address of a pointer to an ISpNotifySink interface. Call the Notify() method on this object to cause the task to be scheduled.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pTask is invalid or bad.
E_POINTER | ppTaskCtrl is invalid or bad.
FAILED (hr) | Appropriate error message.


ISpTaskManager::CreateThreadControl

ISpTaskManager::CreateThreadControl allocates a thread control object and does not allocate a thread.

HRESULT CreateThreadControl(
   ISpThreadTask       *pTask,
   void                *pvTaskData,
   long                 nPriority,
   ISpThreadControl   **ppThreadCtrl
);

Parameters

pTask
[in] Address of the ISpThreadTask interface that is used to initialize and execute the task thread.
pvTaskData
[in] Data passed to all ISpThreadTask member functions. This value can be NULL.
nPriority
[in] The Win32 priority for the allocated thread.
ppThreadCtrl
[out] Address of a pointer to an ISpThreadControl interface that receives the thread control.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pTask is invalid or bad.
E_POINTER | ppThreadCtrl is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.
FAILED (hr) | Appropriate error message.


ISpTaskManager::TerminateTask

ISpTaskManager::TerminateTask interrupts the specified task.

HRESULT TerminateTask(
   DWORD   dwTaskId,
   ULONG   ulWaitPeriod
);

Parameters

dwTaskId
[in] Value specifying the identifier of the task to interrupt.
ulWaitPeriod
[in] Number of milliseconds to wait before interrupting the task.

Return values

Value | Description
S_OK | Function completed successfully.
S_FALSE | Method timed out.
FAILED (hr) | Appropriate error message.


ISpTaskManager::TerminateTaskGroup

ISpTaskManager::TerminateTaskGroup terminates a group of tasks matching the specified group identifier.

HRESULT TerminateTaskGroup(
   DWORD   dwGroupId,
   ULONG   ulWaitPeriod
);

Parameters

dwGroupId
[in] Value specifying the identifier for the task group to interrupt.
ulWaitPeriod
[in] Number of milliseconds to wait before interrupting the task group.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


ISpThreadControl

The ISpThreadControl interface inherits from the ISpNotifySink interface.

Methods in Vtable Order

ISpThreadControl Methods | Description
StartThread | Initializes a thread and returns a window handle.
WaitForThreadDone | Specifies the time interval to wait before ending thread processing.
TerminateThread | Forces immediate termination of the thread.
ThreadHandle | Returns the thread handle of the allocated thread.
ThreadId | Returns the thread ID of the allocated thread.
NotifyEvent | Returns the Win32 event handle that will be set when the Notify() method is called.
WindowHandle | Returns the window handle associated with the thread.
ThreadCompleteEvent | Returns an event that the client can use to wait until the thread processing has completed.
ExitThreadEvent | Returns the event passed to the ISpThreadTask::ThreadProc method.


ISpThreadControl::StartThread

ISpThreadControl::StartThread initializes a thread and optionally returns a window handle.

HRESULT StartThread(
   DWORD   dwFlags,
   HWND   *phwnd
);

Parameters

dwFlags
Reserved. Must be 0.
phwnd
Optional address of an handle to a window. The handle of the new window will be returned to phwnd if this parameter is non-NULL. A window will not be created if this parameter is NULL.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Invalid pointer.
E_OUTOFMEMORY | Exceeded available memory.


ISpThreadControl::WaitForThreadDone

ISpThreadControl::WaitForThreadDone requests that a thread stop running, wait until the thread stops running, or both. Note that specifying fForceStop=TRUE with a timeout of zero will simply request that the thread exit and this method will return immediately. The caller could then wait for the ThreadCompleteEvent using a Win32 wait function.

HRESULT WaitForThreadDone(
   BOOL       fForceStop,
   HRESULT   *phrThreadResult,
   ULONG      msTimeOut
);

Parameters

fForceStop
Flag specifies to stop thread processing. If this is TRUE, the hExitThreadEvent will be set and the pfContinueProcessing flag will be set to false for the thread proc of the worker thread.
phrThreadResult
If this function returns S_OK, this address will contain the value returned from the thread proc.
msTimeOut
Time-out interval in milliseconds to wait before timing out the wait operation.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.


ISpThreadControl::TerminateThread

ISpThreadControl::TerminateThread forces immediate termination of the thread.

This function should be used with caution, as unintended results can occur. This method should only be used to force a thread to stop when all other attempts to stop this thread have been unsuccessful.

HRESULT  TerminateThread ( void );

Parameters

None.

Return values

Value | Description
S_OK | Function completed successfully.


ISpThreadControl::ThreadHandle

Returns the thread handle of the allocated thread. This handle should not be used to wait for the completion of the thread procedure. Use the ISpThreadControl::ThreadCompleteEvent, or the ISpThreadControl::WaitForThreadDone method.

HANDLE ThreadHandle ( void );

Parameters

None.

Return values

Returns the thread handle, or NULL if ISpThreadControl::StartThread has not been called for this thread control object.


ISpThreadControl::ThreadId

ISpThreadControl::ThreadId returns the Win32 thread ID of the thread associated with this thread control object. If StartThread has not been called, this returns 0.

DWORD   ThreadId  ( void );

Parameters

None.

Return values

The ID of the thread.


ISpThreadControl::NotifyEvent

ISpThreadControl::NotifyEvent returns the event handle passed to the ISpThreadTask::ThreadProc method. This event will be set when the ISpThreadControl::Notify method is called.

HANDLE  NotifyEvent ( void );

Parameters

None.

Return values

Returns the event handle.


ISpThreadControl::WindowHandle

ISpThreadControl::WindowHandle returns the window handle associated with this ISpThreadControl object. This will be NULL unless the caller of ISpThreadControl::StartThread specified a non-NULL HWND pointer.

HWND  WindowHandle ( void );

Parameters

None.

Return values

Returns the window handle. NULL if no handle is associated with this object.


ISpThreadControl::ThreadCompleteEvent

ISpThreadControl::ThreadCompleteEvent returns a Win32 event handle that can be used to wait for the completion of the thread.

HANDLE  ThreadCompleteEvent ( void );

Parameters

None.

Return values

Returns the Win32 event handle.


ISpThreadControl::ExitThreadEvent

ISpThreadControl::ExitThreadEvent returns the event handle passed to the ISpThreadTask::ThreadProc. This should never be set manually. Use WaitForThreadDone to force a thread to exit.

HANDLE   ExitThreadEvent(void);

Parameters

none.

Return values

Win32 event handle.


ISpThreadTask

The ISpThreadTask interface simplifies thread-based operations. Clients can implement this virtual C++ interface to simplify the management of threads. SAPI provides coordination methods for starting, initializing, and stopping the thread. See the ISpThreadControl documentation for more information. Clients simply implement three methods which will be called on the allocated thread.

When to Implement

Use SpThreadControl objects to allocate threads for an object.

This is not a COM interface.

Methods in Vtable Order

ISpThreadTask Methods | Description
InitThread | Method called on the allocated thread. If the method returns an error, the ISpThreadControl::StartThread method will fail.
ThreadProc | Implements the processing of the thread.
WindowMessage | Implements the processing of window messages.


ISpThreadTask::InitThread

ISpThreadTask::InitThread is called on the newly allocated thread when the ISpThreadControl::StartThread method is called. The caller of StartThread method will be blocked until the InitThread method completes, and the HRESULT returned from this method will be returned from StartThread. If the return code from this method indicates failure, the thread will be terminated, and the ThreadProc and WindowMessage methods will never be called.

virtual HRESULT STDMETHODCALLTYPE InitThread(
   void   *pvTaskData,
   HWND    hwnd
) = 0;

Parameters

pvTaskData
[in] The same pointer is passed to ISpTaskManager::CreateThreadControl.
hwnd
[in] A window handle if, and only if, the caller to ISpTaskManager::CreateThreadControl specified that a window handle be created by passing a non-NULL HWND pointer to CreateThreadControl. Otherwise this parameter is NULL.

Return values

S_OK | Function completed successfully.
Other success | Success code will be returned to caller of the StartThread method and the thread will continue.
Failure code | Function failed. Failure code will be returned to caller of the StartThread method and the thread will not continue.


ISpThreadTask::ThreadProc

ISpThreadTask::ThreadProc implements the main processing loop of the thread. This method will be application specific.

virtual HRESULT STDMETHODCALLTYPE ThreadProc(
   void                 *pvTaskData,
   HANDLE                hExitThreadEvent,
   HANDLE                hNotifyEvent,
   HWND                  hwndWorker,
   volatile const BOOL  *pfContinueProcessing
) = 0;

Parameters

pvTaskData
[in] Pointer passed to ISpTaskManager::CreateThreadControl.
hExitThreadEvent
[in, out] An event handle which when signaled indicates that the thread process should exit.
hNotifyEvent
[in] A handle to an auto-reset event object that will be set if the ISpThreadControl::Notify method is called. This functionality is provided for any notification event the client determines, or it can optionally be ignored if it is not needed.
hwndWorker
[in] A window handle. This parameter will be NULL if the caller of ISpThreadControl::StartThread passed a NULL HWND pointer to StartThread.
pfContinueProcessing
[in] Boolean flag indicating whether to continue processing. TRUE indicates the process should continue; FALSE otherwise. This mirrors the functionality of the hExitThreadEvent, but provides a lightweight mechanism for checking for a request to exit without calling a Win32 WaitForxxx object function.

Return values

S_OK | Function completed successfully.
S_FAILED | Function failed.


ISpThreadTask::WindowMessage

ISpThreadTask::WindowMessage implements the processing of window messages. Not all applications will need a window and this method may be left unimplemented. If the caller of ISpThreadControl::StartThread passes a non-NULL HWND pointer, the client must implement this function and must also use a MessageWaitForMultipleObjects() loop in the ThreadProc.

virtual LRESULT STDMETHODCALLTYPE WindowMessage(
   void    *pvTaskData,
   HWND     hWnd,
   UINT     Msg,
   WPARAM   wParam,
   LPARAM   lParam
) = 0;

Parameters

pvTaskData
[in] Pointer passed to ISpTaskManager::CreateThreadControl.
hWnd
[in] A window handle.
Msg
[in] The type of window message.
wParam
Message-specific information. This will change based on the Msg value.
lParam
Message-specific information. This will change based on the Msg value.

Return values

The return value is message specific.


Speech Recognition Manager (DDI-level)

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Speech Recognition interfaces (DDI-level)

The following section covers:

  • ISpPhraseBuilder


ISpPhraseBuilder

The ISpPhraseBuilder interface inherits from ISpPhrase.

Methods in Vtable Order

ISpPhraseBuilder Methods | Description
InitFromPhrase | Initializes from a phrase.
InitFromSerializedPhrase | Initializes a phrase from a serialized phrase.
AddElements | Adds a copy of the given element to the end of this object's element list.
AddRules | Adds phrase rules to the phrase object.
AddProperties | Adds property entries to the phrase object.
AddReplacements | Adds one or more text replacements to the phrase.


 The Microsoft.com Speech website

ISpPhraseBuilder::InitFromPhrase

ISpPhraseBuilder::InitFromPhrase initializes from a phrase.

HRESULT InitFromPhrase(
   const   SPPHRASE   *pSrcPhrase
);

Parameters

pSrcPhrase
Address of a SPPHRASE data structure containing the phrase information. If pSrcPhrase is NULL, the object is reset to its initial state.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pSrcPhrase or pSrcPhrase->Rule.pNextSibling is invalid or bad. Alternatively, pSrcPhrase->LangID may be zero or pSrcPhrase->cbSize does not indicate the same size as pSrcPhrase.
FAILED(hr) | Appropriate error message.

Example

The following code snippet demonstrates creating and initializing from a phrase.

HRESULT hr;

CComPtr<ISpPhraseBuilder> cpPhraseBuilder;
CComPtr<ISpPhrase>        cpPhrase;
SPPHRASE    Phrase;

hr = cpPhraseBuilder.CoCreateInstance( CLSID_SpPhraseBuilder );

if (SUCCEEDED(hr))
{
    //We initialize the Phrase data structure
}

if (SUCCEEDED(hr))
{
    hr = cpPhraseBuilder->InitFromPhrase( &Phrase );
}


ISpPhraseBuilder::InitFromSerializedPhrase

ISpPhraseBuilder::InitFromSerializedPhrase initializes a phrase from a serialized phrase.

HRESULT InitFromSerializedPhrase(
   const   SPSERIALIZEDPHRASE   *pPhrase
);

Parameters

pPhrase
Address of the SPSERIALIZEDPHRASE structure that contains the phrase information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pSrcPhrase or pSrcPhrase->cbSerializedSize is invalid or bad.
FAILED(hr) | Appropriate error message.

Example

The following code fragment demonstrates InitFromSerializedPhrase.

HRESULT hr;
CComPtr<ISpRecoResult>       RecoResult;
CComPtr<ISpPhraseBuilder>    pPhraseBuilder;
SPSERIALIZEDPHRASE           *pSerializedPhrase=NULL;
CComPtr<ISpStream>           cpStream;

LARGE_INTEGER   liZero = {0,0};
hr = SPBindToFile(L"SerializedPhrase.sp", SPFM_OPEN_READONLY, &cpStream, NULL, NULL, SPFEI_ALL_EVENTS);
if (hr == S_OK)
{
   hr = cpStream->Seek(liZero, STREAM_SEEK_SET, NULL);
   ULONG ulSerializedSize = 0;
   hr = cpStream->Read(&ulSerializedSize, sizeof(ULONG), NULL);
   if (SUCCEEDED(hr))
   {
    //We need to seek back and read the whole chunk of data in.
    LARGE_INTEGER liseek;
    liseek.QuadPart  -= sizeof(ULONG);
    hr = cpStream->Seek(liseek, STREAM_SEEK_CUR, NULL);

    pSerializedPhrase = (SPSERIALIZEDPHRASE*)::CoTaskMemAlloc(ulSerializedSize);
    if (SUCCEEDED(hr) && pSerializedPhrase)
    {
        hr = cpStream->Read(pSerializedPhrase, ulSerializedSize, NULL);
    }
    if (SUCCEEDED(hr))
    {
        CComPtr cpPhraseBuilder;
                hr = cpPhraseBuilder.CoCreateInstance(CLSID_SpPhraseBuilder);
                if (SUCCEEDED(hr))
                {
                      hr = cpPhraseBuilder->InitFromSerializedPhrase(pSerializedPhrase);
                }
    }
    ::CoTaskMemFree( pSerializedPhrase );
    }
}


ISpPhraseBuilder::AddElements

ISpPhraseBuilder::AddElements adds a copy of the given element to the end of this object's element list.

HRESULT AddElements(
   ULONG                    cElements,
   const SPPHRASEELEMENT   *pElement
);

Parameters

cElements
Specifies the number of phrase elements to add.
pElement
Address of the SPPHRASEELEMENT data structure containing the phrase element to add.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
SPERR_UNINITIALIZED | The object has not been properly initialized.
FAILED(hr) | Appropriate error message.


ISpPhraseBuilder::AddRules

ISpPhraseBuilder::AddRules adds phrase rules to the phrase object.

HRESULT AddRules(
  const SPPHRASERULEHANDLE   hParent,
  const SPPHRASERULE        *pRule,
  SPPHRASERULEHANDLE        *phNewRule
);

Parameters

hParent
[in] Handle to the parent phrase rule.
pRule
[in] Address of the SPPHRASERULE structure that contains the phrase rule information.
phNewRule
[out] Address of the handle of SPPHRASERULEHANDLE that contains the new phrase rule information.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Invalid pointer.
SPERR_UNINITIALIZED | The object has not been properly initialized.
FAILED(hr) | Appropriate error message.


ISpPhraseBuilder::AddProperties

ISpPhraseBuilder::AddProperties adds property entries to the phrase object.

HRESULT AddProperties(
   const SPPHRASEPROPERTYHANDLE hParent,
   const SPPHRASEPROPERTY      *pProperty,
   SPPHRASEPROPERTYHANDLE      *phNewProperty
);

Parameters

hParent
[in] Handle to the parent phrase element.
pProperty
[in] Address of the SPPHRASEPROPERTY structure that contains the property information.
phNewProperty
[out] Address of the handle of SPPHRASEPROPERTY that contains the new property information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Invalid pointer.
SPERR_UNINITIALIZED | The object has not been properly initialized.
SPERR_ALREADY_INITIALIZED | The object has already been initialized.
FAILED(hr) | Appropriate error message.


ISpPhraseBuilder::AddReplacements

ISpPhraseBuilder::AddReplacements adds one or more text replacements to the phrase.

HRESULT AddReplacements(
   ULONG                        cReplacements,
   const SPPHRASEREPLACEMENT   *pReplacements
);

Parameters

cReplacements
The number of replacement phrase elements.
pReplacements
Address of the SPPHRASEREPLACEMENT structure that contains the phrase element replacement information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
SPERR_UNINITIALIZED | The object has not been properly initialized.
FAILED(hr) | Appropriate error message.


TTS Recognition Engine Manager (DDI-level)

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Speech Recognition Engine interfaces (DDI-level)

The following section covers:

  • ISpPrivateEngineCall
  • ISpSREngine
  • ISpSREngineSite
  • ISpSRAlternates


ISpPrivateEngineCall

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

_ISpPrivateEngineCall

This interface is obtained by calling the QueryInterface method of the ISpRecoContext interface. However, applications should not directly use _ISpPrivateEngineCall interface.

Applications should call the QueryInterface method of the recognition context for a particular engine extension interface that is implemented by their engine. SAPI will create the engine extension class identified in ExtensionCLSID attribute of the engine object token. This interface should call the QueryInterface method for _ISpPrivateEngineCall, where it can then call CallEngine to make a private call to the engine. The data passed into the CallEngine method is passed to the engine's PrivateCall method.

When to Implement

Implemented by SAPI and inherits from ISpRecoContext.

Associated Class IDs

The following class IDs (CLSID) may be used with this interface. A complete CLSID listing for all interfaces is in the Class IDs section.

    CLSID_SpSharedRecoContext

Methods in Vtable Order

_ISpPrivateEngineCall Methods | Description
CallEngine | Allows an engine-specific call.
CallEngineEx | Returns the non-fixed size data block response information associated with the SR engine.


ISpPrivateEngineCall::CallEngine

ISpPrivateEngineCall::CallEngine allows an engine-specific call.

It is called from the engine extension object to the engine object. Data passed into this call is given to the main SR engine through the ISpSREngine::PrivateCall method.

HRESULT CallEngine(
   VOID   *pCallFrame,
   ULONG   ulCallFrameSize
);

Parameters

pCallFrame
[in, out] The engine-specific structured block of memory parameters. This block will be marshaled in the shared engine case and must not contain pointers to other memory allocations. It must be fully self–contained and relative only to itself.
ulCallFrameSize
[in] Size, in bytes, of the pCallFrame structure.

Return values

Value | Description
S_OK | Function completed successfully.
E_FAILED | No engine could be found.
FAILED (hr) | Appropriate error message.


ISpPrivateEngineCall::CallEngineEx

ISpPrivateEngineCall::CallEngineEx returns the non-fixed size data block response information associated with the SR engine.

Applications implementing this method must call CoTaskMemFree() to free memory associated with the returned response.

HRESULT CallEngineEx(
   const void  *pInFrame,
   ULONG        ulInFrameSize,
   void       **ppCoMemOutFrame,
   ULONG       *pulOutFrameSize
 );

Parameters

pInFrame
[in] Address of the recognition engine data.
ulInFrameSize
[in] Size, in bytes, of the pInCallFrame structure.
ppCoMemOutFrame
[out] Address of a pointer to the data block information associated with the SR engine.
pulOutFrameSize
[out] Size, in bytes, of the ppCoMemOutFrame structure.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
E_POINTER | Invalid pointer.
FAILED(hr) | Appropriate error message.


ISpSREngine

This is the main interface that engine developers must implement to have a working SAPI SR engine. Engine vendors need to implement this interface in an InProc COM object with the threading model “Both”. This object can also implement the ISpObjectWithToken interface. SR engines may also have additional objects implementing ISpTokenUI, ISpSRAlternates, and a custom engine-extension object.

The SR Engine Guide describes in detail how an SR engine interacts with SAPI and how the methods of this interface are implemented.

Applications do not directly call the methods on this interface, all calls are made by SAPI.

Implemented By
  • SR Engine developers as the main interface between their SR engine and SAPI.
How Created
  • Only SAPI will directly create this object. When an application creates a RecoContext object, either shared or InProc, then SAPI will find the appropriate recognizer object token and create the object implementing this interface from the CLSID in the object token.

Methods in Vtable Order

ISpSREngine Methods | Description
SetSite | Sets the ISpSREngineSite interface for the engine to use.
GetInputAudioFormat | Gets the audio format that the SR engine supports.
RecognizeStream | Begins recognition processing on a stream.
SetRecoProfile | Passes the current active user profile to the engine.
OnCreateGrammar | Creates a new recognition grammar.
OnDeleteGrammar | Informs the engine of the deletion of a grammar.
LoadProprietaryGrammar | Instructs the engine to load a grammar in an engine-specific format.
UnloadProprietaryGrammar | Unloads an engine-specific grammar.
SetProprietaryRuleState | Sets the proprietary grammar rule state.
SetProprietaryRuleIdState | Sets the proprietary grammar rule ID state.
LoadSLM | Instructs the engine to load a dictation statistical language model (SLM).
UnloadSLM | Instructs the engine to unload an SLM.
SetSLMState | Sets the recognition state of the SLM to active or inactive.
SetWordSequenceData | Sets the text buffer information.
SetTextSelection | Informs the engine of the displayed and selected areas of the text buffer.
IsPronounceable | Determines if a word can be recognized by the engine.
OnCreateRecoContext | Informs the engine of the creation of a recognition context.
OnDeleteRecoContext | Notifies the engine that a recognition context is being destroyed.
PrivateCall | Engine-specific call to the engine.
SetAdaptationData | Provides text data for language model adaptation to the engine.
SetPropertyNum | Sets a numerical property attribute on the SR engine.
GetPropertyNum | Retrieves a numerical property attribute from the SR engine.
SetPropertyString | Sets a text property attribute on the SR engine.
GetPropertyString | Retrieves a text property attribute from the SR engine.
SetGrammarState | Informs the engine if a grammar has been activated or deactivated.
WordNotify | Notifies the SR engine of the words in CFG grammars.
RuleNotify | Notifies the SR engine of the rule information in CFG grammars.
PrivateCallEx | Calls the engine which allows a variable sized data block can be returned from the engine to the engine extension object..
SetContextState | Indicates that a recognition context has been deactivated or activated.


ISpSREngine::SetSite

ISpSREngine::SetSite sets the ISpSREngineSite interface for the engine to use. The SR engine can call back to SAPI using the methods in the ISpSREngineSite.

HRESULT SetSite(
   ISpSREngineSite   *pSite
);

Parameters

pSite
Pointer to the ISpEngineSite interface for the engine to use to call back to SAPI.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::GetInputAudioFormat

ISpSREngine::GetInputAudioFormat gets the audio format that the SR engine supports.

Audio formats in SAPI are described by two parameters: a GUID, and for wave formats, a WAVEFORMATEX structure.

The first two parameters in this method define the audio format that SAPI is asking the engine about. If these parameters are set to a specific format, the engine determines if it can support this format, and returns that format or the nearest format that the engine can support using the second pair of parameters to this function. Alternatively, if the first two parameters are NULL, the engine should return its preferred audio format in the second pair of parameters.

HRESULT GetInputAudioFormat(
   const GUID           *pguidSourceFormatId,
   const WAVEFORMATEX   *pSourceWaveFormatEx,
   GUID                 *pguidDesiredFormatId,
   WAVEFORMATEX        **ppCoMemDesiredWaveFormatEx,
);

Parameters

pguidSourceFormatId
[in] The GUID of the audio format SAPI determines if the engine can support. This will either be NULL, indicating that the engine should select its preferred audio format, or it will be set to a given format, which the engine determines if it can support.
pSourceWFEX
[in] Address of the WAVEFORMATEX structure containing information about the audio format SAPI is querying the engine about. This will only be set if pSourceFormatId is equal to SPDFID_WaveFormatEx.
pguidDesiredFormatId
[out] The GUID of the format that the engine can support.
ppCoMemDesiredWFEX
[out] The engine should call a WAVEFORMATEX structure with CoTaskMemAlloc to determine the format the engine supports. The address of the structure should be placed in this parameter. For non-wave formats, this parameter should be NULL.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


ISpSREngine::RecognizeStream

ISpSREngine::RecognizeStream begins recognition processing on a stream. From this point on, the engine can read data, perform recognition, and send results and events back to SAPI. When all the data to be recognized and read, or the application has deactivated recognition, the engine finishes processing and returns from this method.

The engine can read audio data using the ISpSREngineSite::Read method. The engine determines how much data is available for reading with the ISpSREngineSite::DataAvailable method, or by using the hDataAvailable event handle. The engine does not have direct access to the input audio device and will perform in a consistent way regardless of whether input is from desktop audio, wave files, or a custom audio device. The audio format is given by the rguidFmtId and pWaveFormatEx parameters, and additional details of the audio device can be found from the fNewAudioStream, fRealTimeAudio, and pAudioObjectToken parameters. When a Read call indicates that there is no more data available, the engine should complete processing on the data it has and return from the RecognizeStream method.

The engine recognizes from all rules and/or dictation grammars that have been activated. If there are multiple active rules and/or dictations, the engine is expected to recognize from all things "in parallel." That is, the user is able to say something from any rule that is active. It is possible for this method to be called with nothing active. In this case, the engine can just read data and then discard it, or use it to gather environmental noise information.

Because the engine remains in the RecognizeStream method all the time that it is recognizing, SAPI has effectively given the engine one thread on which to perform recognition. It is possible to write an engine which processes everything on this one thread and thus does not require any additional threads, critical sections, or other thread-locking.

It is also possible to have alternative arrangements with additional threads. For example, one thread could read data, while another thread could do the actual recognition processing. SAPI makes no restrictions about which threads call which methods or whether they are called simultaneously.

The engine uses the ISpSREngineSite::Synchronize method to be notified of any grammar or other changes that are pending, and uses the ISpSREngineSite::UpdateRecoPos to keep SAPI informed of how much of the stream has been recognized. The engine passes details of events and recognition results back to SAPI with the ISpSREngineSite::AddEvent and ISpSREngineSite::Recognition methods.

HRESULT RecognizeStream(
   REFGUID               rguidFmtId,
   const WAVEFORMATEX   *pWaveFormatEx,
   HANDLE                hRequestSync,
   HANDLE                hDataAvailable,
   HANDLE                hExit,
   BOOL                  fNewAudioStream,
   BOOL                  fRealTimeAudio,
   ISpObjectToken       *pAudioObjectToken
);

Parameters

rguidFmtId
[in] The REFGUID of the input audio format to recognize. This will be SPDFID_WaveFormatEx for wave format files.
pWaveFormatEx
[in] The WAVEFORMATEX structure describing the input format (if it is a wave format). Only a format that the engine has already indicated it can process (by returning the format from ISpSREngine::GetInputAudioFormat) will be used.
hRequestSync
[in] This is a Win32 event handle that is set whenever there are tasks (such as grammar changes etc.) waiting for the engine to respond to. The tasks get processed whenever the engine calls Synchronize. The engine can call Synchronize regularly or do so only when this event is set.
hDataAvailable
[in] This is a Win32 event handle that is set when data is available for reading. The amount of data to be available before this event is set can be controlled by calling the ISpSREngineSite::SetBufferNotifySize method. By default, this event will be set whenever any amount of data is available. This event can be used as an alternative to the ISpSREngineSite::DataAvailable method.
hExit
[in] This is a Win32 event handle indicating when the engine should exit. The engine on one of two conditions:
  • When there is no more data in the stream and it has finished processing, or
  • If this event is set. Recognition or Synchronize calls returning S_FALSE indicate that this event has been set.
fNewAudioStream
[in] Indicates whether the input is a new stream. TRUE indicates it is a newly created stream; FALSE otherwise. For example, if an application deactivates the rules, RecognizeStream returns, and later the application activates some rules, the RecognizeStream call will have this parameter set as FALSE because the stream had exited previously. Only if the application calls ISpRecognizer::SetInput to create a new stream, will this return TRUE. Some engines will find this information useful if resetting channel adaptation, for example, a new telephone call.
fRealTimeAudio
[in] Indicates whether the input is real time audio. TRUE means it is real time audio; FALSE otherwise. Real-time inputs in SAPI are those that implement the ISpAudio interface – for example the standard multi-media microphone input. Non-real time streams are those that only implement ISpStreamFormat - for example input from wave files using the ISpStream object. With non real-time streams all the data is available for reading immediately. The hDataAvailable event is always set and the DataAvailable method will always return INFINITE.
pAudioObjectToken
[in] The object token interface for the audio object that the stream was created from. Engines do not need to do anything with this parameter, but it may be useful in some circumstances.

Return values

Value | Description
S_OK | Function completed successfully. This should be returned if the engine is exiting because the stream has ended, or because it was signaled to exit by SAPI.
FAILED (hr) | Appropriate error message if the engine is terminating for an unexpected reason.


ISpSREngine::SetRecoProfile

ISpSREngine::SetRecoProfile passes the current active user profile to the engine.

RecoProfiles are added or removed using Speech properties in Control Panel.

Engines can query the token and include any information needed about this profile, for example, enrollment or training information. Engine specific information should be stored in a sub-key of the recognition profile token named with the engine class ID. This is to avoid conflicts with other engines using the same profile.

In order to provide user enrollment the engine can implement a UI component with the name "UserTraining." To instantiate, click Control Panel->Speech properties->SR tab->Train Profile.

SAPI does not restrict applications from calling ISpRecognizer::SetRecoProfile during the middle of recognition (see ISpSREngine::RecognizeStream). If an SR engine does not allow recognition profile changes during recognition, it should fail the ::SetRecoProfile call gracefully (e.g., return SPERR_ENGINE_BUSY).

HRESULT SetRecoProfile(
   ISpObjectToken   *pProfile
);

Parameters

pProfile
Address of an ISpObjectToken object that contains the recognition profile token information.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | The pProfile parameter is not a valid ISpObjectToken interface pointer.
FAILED(hr) | Appropriate error message.


ISpSREngine::OnCreateGrammar

ISpSREngine::OnCreateGrammar creates a new recognition grammar. Each grammar belongs to a recognition context and can contain dictation, CFG, or proprietary grammar information. The engine can associate a pointer with each grammar that is then passed back to the engine in other methods using this grammar.

HRESULT OnCreateGrammar(
   void             *pvEngineRecoContext,
   SPGRAMMARHANDLE   hSAPIGrammar,
   void            **ppvEngineGrammar
);

Parameters

pvEngineRecoContext
[in] The engine's recognition context pointer indicating the context this grammar belongs to. This is the value that the engine passes back to SAPI in the OnCreateRecoContext method.
hSAPIGrammar
[in] Unique handle to the grammar.
ppvEngineGrammar
[out] The engine should set the contents of this to an arbitrary pointer containing any information the engine has associated with this grammar.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::OnDeleteGrammar

ISpSREngine::OnDeleteGrammar notifies the engine that a recognition grammar is deleted. By the time this method is called, any rules or a loaded SLM in this grammar will already have been deleted.

HRESULT OnDeleteGrammar(
   void   *pvEngineGrammar
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer for this grammar, as returned from a previous call to the OnCreateGrammar method.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::LoadProprietaryGrammar

ISpSREngine::LoadProprietaryGrammar instructs the engine to load a grammar in an engine-specific format. This is used to load grammars that are not in the standard SAPI CFG format.

The application can supply the engine with either a GUID, string data, or binary data or some combination of these, in order to describe the grammar. SAPI does nothing with this data apart from correctly marshaling it to the SR engine.

HRESULT LoadProprietaryGrammar(
   void          *pvEngineGrammar,
   REFGUID        rguidParam,
   const WCHAR   *pszStringParam,
   const void    *pvDataParam,
   ULONG          ulDataSize.
   SPLOADOPTIONS  Options
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer, as returned from the OnCreateGrammar method.
rguidParam
[in] Unique identifier for the grammar.
pszStringParam
[in, string] Null-terminated string containing proprietary grammar string data.
pvDataParam
[in] Pointer to grammar image data.
ulDataSize
[in] Size, in bytes, of the grammar image data.
Options
[in] One of the grammar loading options specified in the SPLOADOPTIONS enumeration sequence.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


ISpSREngine::UnloadProprietaryGrammar

ISpSREngine::UnloadProprietaryGrammar unloads an engine-specific grammar.

HRESULT UnloadProprietaryGrammar(
   void   *pvEngineGrammar
);

Parameters

pvEngineGrammar
[in] Address of the engine's grammar pointer.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


ISpSREngine::SetProprietaryRuleState

ISpSREngine::SetProprietaryRuleState sets the proprietary grammar rule state. This is used to activate or deactivate rules in non-standard proprietary grammars, where each rule is identified by a string name. If ::SetProprietaryRuleState is called with the rule name set to NULL, the engine should activate or deactivate all top-level rules in this grammar. The pcRulesChanged parameter must be set to the number of rules whose state has changed.

HRESULT SetProprietaryRuleState(
   void          *pvEngineGrammar,
   const WCHAR   *pszName,
   const WCHAR   *pszValue,
   SPRULESTATE    NewState,
   ULONG         *pcRulesChanged
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer for this grammar, as returned from a previous call to the OnCreateGrammar method.
pszName
[in, string] Null-terminated string that contains the grammar rule name, or NULL to indicate all top-level rules should be activated in this grammar.
pszValue
[in, string] Null-terminated string that contains rule value information (Currently always NULL).
NewState
[in] One of the grammar rule states specified in the SPRULESTATE enumeration sequence.
pcRulesChanged
[out] The number of rules whose state has been changed. This should be set to 1 if a specific rule name was supplied.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::SetProprietaryRuleIdState

ISpSREngine::SetProprietaryRuleIdState sets the proprietary grammar rule ID state. This is used to activate or deactivate rules in non-standard proprietary grammars where each rule is identified by an ID.

HRESULT SetProprietaryRuleIdState(
   void         *pvEngineGrammar,
   DWORD         dwRuleId,
   SPRULESTATE   NewState
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer for this grammar, as returned from a previous call to the OnCreateGrammar method.
dwRuleId
[in] The engine proprietary grammar rule identifier.
NewState
[in] One of the grammar rule states specified in the SPRULESTATE enumeration sequence.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::LoadSLM

ISpSREngine::LoadSLM instructs the engine to load a statistical language model (SLM) for dictation.

HRESULT LoadSLM(
   void          *pvEngineGrammar,
   const WCHAR   *pszTopicName
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer, as returned from the OnCreateGrammar method.
pszTopicName
[in, string] Null-terminated string that specifies a topic name. The default SLM should be loaded if the value of pszTopicName is NULL.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::UnloadSLM

ISpSREngine::UnloadSLM instructs the engine to unload an SLM.

HRESULT UnloadSLM(
   void   *pvEngineGrammar
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer, as returned from the OnCreateGrammar method.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::SetSLMState

ISpSREngine::SetSLMState activates or deactivates dictation on this grammar.

HRESULT SetSLMState(
   void         *pvEngineGrammar,
   SPRULESTATE   NewState
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer for this grammar, as returned from a previous call to the OnCreateGrammar method.
NewState
[in] One of the grammar rule states specified in the SPRULESTATE enumeration sequence. This can be SPRS_ACTIVE to indicate that dictation is being activated; SPRS_INACTIVE to indicate dictation is being deactivated, and SPRS_ACTIVE_WITH_AUTO_PAUSE to indicate dictation is being activated with auto-pause This means that the engine will be put into the paused state each time it returns a final recognition result on this grammar. This is just for information and can be handled in the same way as SPRS_ACTIVE.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::SetWordSequenceData

ISpSREngine::SetWordSequenceData sets a text buffer that the engine can use for recognition.

HRESULT SetWordSequenceData(
   void                         *pvEngineGrammar,
   const WCHAR                  *pText,
   ULONG                         cchText,
   const  SPTEXTSELECTIONINFO   *pInfo
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer for this grammar, as returned from a previous call to the OnCreateGrammar method.
pText
[in] The text buffer.
cchText
[in] The length, in characters, of the text buffer.
pInfo
[in] Address of the SPTEXTSELECTIONINFO structure that contains information of which parts of the buffer are active and currently selected.

Return values

Value | Description
S_OK | Function completed successfully.
E_NOTIMPL | Engine does not support text-buffer functionality
FAILED(hr) | Other appropriate error message.

Remarks

Whenever a text-buffer transition is reached in a CFG, the engine should attempt to recognize a sub-string of words from the current text buffer. This provides a very simple way for applications to select from a set of text.

This method is called when an application calls ISpRecoGrammar::SetWordSequenceData. The format of the buffer is a sequence of one or more null-terminated strings, with a double null-termination at the end. The engine recognizes any sub-string of words from any of the strings in the buffer. It is up to the SR engines to perform word breaking and text normalization for better performance.

It is also possible for the application to alter the areas of the buffer that are used for recognition. The initial range can be set with the structure SPTEXTSELECTIONINFO, and later calls to ISpSREngine::SetTextSelection can alter this without changing the actual buffer. The ulStartActiveOffset and cchActiveChars indicate which area of the buffer should be active for recognition.

The other two fields of the SPTEXTSELECTIONINFO, ulStartSelection and cchSelection can be used with dictation. These could indicate, on screen for example, which area of the buffer is currently selected. If cchSelection is zero, this could display the current location of the insertion point. The engine can use SPTEXTSELECTIONINFO to get extra language model context from the preceding words in the dictated text.

This text buffer feature is optional for engines, and support for it is determined using the TextBuffer attribute in the engine object token. If this method is called on an engine that does not support this feature, the engine should return E_NOTIMPL.


ISpSREngine::SetTextSelection

ISpSREngine::SetTextSelection informs the engine of the displayed and selected areas of the text buffer.(See ISpSREngine::SetWordSequenceData). Once a text buffer has been supplied to the engine, this method can be used to control which parts of the buffer are active for recognition using a text-buffer transition in a CFG. This method can communicate to the engine the location of the current text insertion point for dictation.

HRESULT SetTextSelection(
   void                         *pvEngineGrammar,
   const  SPTEXTSELECTIONINFO   *pInfo
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer for this grammar, as returned from a previous call to the OnCreateGrammar method.
pInfo
[in] Pointer to the text selection information structure.

Return values

Value | Description
S_OK | Function completed successfully.
E_NOTIMPL | Engine does not support text-buffer functionality
FAILED(hr) | Other appropriate error message.

Remarks

The first two fields of the SPTEXTSELECTIONINFO structure, ulStartActiveOffset and cchActiveChars indicate the area of the buffer that should be active for recognition when using a text-buffer transition in a CFG.

The other two fields of the SPTEXTSELECTIONINFO, ulStartSelection and cchSelection can be used with dictation. These could indicate, for example, which area of the buffer is currently selected on the screen. If cchSelection is zero, this could display the current location of the insertion point. This can be used by the engine to get extra language model context from preceding words in the dictated text.

This text buffer feature is optional for engines, and support for it is indicated using the TextBuffer attribute. If this method is called on an engine that does not support this feature, the engine should return E_NOTIMPL.


ISpSREngine::IsPronounceable

ISpSREngine::IsPronounceable determines if a word can be recognized by the engine.

This method is called by SAPI after an application calls ISpRecoGrammar::IsPronounceable.

HRESULT IsPronounceable(
   void                    *pvDrvGrammar,
   const WCHAR             *pszWord,
   SPWORDPRONOUNCEABLE     *pfPronounceable
);

Parameters

pvDrvGrammar
[in] The engine's grammar pointer, as returned from the OnCreateGrammar method.
pszWord
[in] The word to test.
pfPronounceable
[out] Address of the SPWORDPRONOUNCEABLE enumeration indicating the results of the test. If the SR engine can generate a reasonable pronunciation for the given word, it should return TRUE in the pfPronounceable pointer; otherwise FALSE.
Value
SPWP_UNKNOWN_WORD_UNPRONOUNCEABLE | The word is not pronounceable by the SR engine, and is not located in the lexicon and/or the engine's dictionary.
SPWP_UNKNOWN_WORD_PRONOUNCEABLE | The word is pronounceable by the SR engine, but is not located in the lexicon and/or the engine's dictionary.
SPWP_KNOWN_WORD_UNPRONOUNCEABLE | The word is pronounceable by the SR engine, and is located in the lexicon and/or the engine's dictionary.

Return values

Value | Description
S_OK | Method completed successfully.
FAILED(hr) | Appropriate error message.

See Also

ISpRecoGrammar::IsPronounceable


ISpSREngine::OnCreateRecoContext

ISpSREngine::OnCreateRecoContext notifies the engine that a recognition context is being created. Each application connected with the SR engine can have one or more recognition contexts. The engine can associate a pointer with each recognition context that is then passed back to the engine in other methods using this context.

HRESULT OnCreateRecoContext(
   SPRECOCONTEXTHANDLE   hSAPIRecoContext,
   void                **ppvEngineContext
);

Parameters

hSAPIRecoContext
[in] Unique handle to the recognition context.
ppvEngineContext
[out] The engine should set the contents of this to a pointer to any engine-specific information it wishes to associate with this context.

Return values

Value | Description
NOERROR | No error is possible with this function.


ISpSREngine::OnDeleteRecoContext

ISpSREngine::OnDeleteRecoContext notifies the engine that a recognition context is being destroyed. By the time this call is made, all grammars associated with this context will have been deleted.

HRESULT OnDeleteRecoContext(
   void   *pvEngineContext
);

Parameters

pvEngineContext
[in] Pointer to the engine's data for this context, as returned from a previous call to OnCreateRecoContext.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::PrivateCall

ISpSREngine::PrivateCall makes an engine-specific extension call to the SR engine. This method is called when the engine extension COM object calls _ISpPrivateEngineCall::CallEngine. SAPI marshals the data to the main engine object and calls this method.

The engine must implement an engine-extension COM object and implement whatever interfaces the engine wants. Then an application can call QueryInterface for these interfaces on the recognition context object. The engine extension-object can query for the _ISpPrivateEngineCall interface from the recognition context. If the engine must directly communicate with the main engine object, it can call the CallEngine method, which passes the data to this method (See the SR Engine Guide for more details on this process).

HRESULT PrivateCall(
   void    *pvEngineContext,
   PVOID   *pCallFrame,
   ULONG    ulCallFrameSize,
);

Parameters

pvEngineContext
[in] Pointer to the engine's pointer for this context, as returned from a previous call to OnCreateRecoContext.
pCallFrame
[in] Pointer to the engine-specific data. This can be used both to pass in and return data from the engine. The returned data must be the same size as the passed in data.
ulCallFrameSize
[in] Size, in bytes, of the engine-specific data.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


ISpSREngine::SetAdaptationData

ISpSREngine::SetAdaptationData provides the SR engine with text data from the application for language-model adaptation purposes. This method is called when an application calls ISpRecoContext::SetAdaptationData. If the engine does not support this type of adaptation, it should do nothing in this function and return S_OK.

Some engines may take considerable processing time to do language model adaptation. Thus, applications should submit adaptation data in chunks to the engine. The engine can then fire an event SPEI_ADAPTATION to indicate that it can receive more adaptation data.

Engines can either persist the adaptation in the current RecoProfile or reset it every session.

HRESULT SetAdaptationData(
   void          *pvEngineContext,
   const WCHAR   *pAdaptationData,
   const ULONG    cch
);

Parameters

pvEngineContext
[in] Engine's recognition context pointer for the context that is sending the data.
pAdaptationData
[in] Buffer containing the adaptation data. Applications should copy this data before returning from this function if they are going to use this data.
cch
[in] The size, in WCHARs, of the adaptation data in pAdaptationData.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::SetPropertyNum

ISpSREngine::SetPropertyNum sets a numerical property value on the SR engine.

Applications can use properties to control run-time results of the SR engine. The application can set and get values for specific attributes on the engine. Some values are predefined by SAPI and others may be added by an engine. See the "SAPI 5.0 SR Properties White Paper" in the White Papers section of the Help files for more details.

This method is called on the engine by SAPI when the application calls ISpProperties::SetPropertyNum on its recognition context object. If the engine returns S_OK from this method, indicating that it supports this property and has changed the value for it, SAPI will send an SPEI_PROPERTY_NUM_CHANGE event to all contexts to inform them of this change.

HRESULT SetPropertyNum(
   SPPROPSRC         eSrc,
   void             *pvSrcObj,
   const WCHAR      *pName,
   LONG              lValue
);

Parameters

eSrc
[in] One of the types specified in the SPPROPSRC enumeration sequence. (This will currently always be SPPROPSRC_RECO_INST).
pvSrcObj
[in] Pointer to additional information. (Currently always NULL).
pName
[in] String containing the property name.
lValue
[in] Value that the property should be set to.

Return values

Value | Description
S_OK | Function completed successfully. Engine supports this property attribute and has set it to the requested value.
S_FALSE | Function completed successfully but engine does not support this property.
FAILED(hr) | Appropriate error message.


ISpSREngine::GetPropertyNum

ISpSREngine::GetPropertyNum retrieves a numerical property value from the SR engine.

Applications can use properties to control run-time results of the SR engine. The application can set and retrieve values for specific attributes on the engine. Some values are predefined by SAPI and others may be added by an engine. See the "SAPI 5.0 SR Properties White Paper" in the White papers section of the help for more details.

This method is called on the engine by SAPI when the application calls ISpProperties::GetPropertyNum on its recognition context object.

HRESULT GetPropertyNum(
   SPPROPSRC      eSrc,
   void          *pvSrcObj,
   const WCHAR   *pName,
   LONG          *lValue
);

Parameters

eSrc
[in] One of the types specified in the SPPROPSRC enumeration sequence. (This will currently always be SPPROPSRC_RECO_INST).
pvSrcObj
[in] Pointer to additional information. (Currently always NULL).
pName
[in] String containing the property name.
lValue
[out] Pointer that the SR engine supplies with the property value information. If the engine does not support this property attribute, it should set the contents of this pointer to zero.

Return values

Value | Description
S_OK | Function completed successfully. Engine supports this property attribute and has returned a value for it.
S_FALSE | Function completed successfully but engine does not support this property.
FAILED(hr) | Appropriate error message.


ISpSREngine::SetPropertyString

ISpSREngine::SetPropertyString sets a string property value on the SR engine.

Applications can use properties to control run-time results of the SR engine. The application can set and get values for specific attributes on the engine. Some values are predefined by SAPI and others may be added by an engine. See the "SAPI 5.0 SR Properties White Paper" in the White Papers section of the help for more details.

This method is called on the engine by SAPI when the application calls ISpProperties::SetPropertyString on its recognition context object. If the engine returns S_OK from this method, indicating that it supports this property and has changed the value for it, SAPI will send an SPEI_PROPERTY_STRING_CHANGE event to all contexts to inform them of this change.

HRESULT SetPropertyString(
   SPPROPSRC      eSrc,
   void          *pvSrcObj,
   const WCHAR   *pName,
   const WCHAR   *pValue
);

Parameters

eSrc
[in] One of the types specified in the SPPROPSRC enumeration sequence. (This will currently always be SPPROPSRC_RECO_INST).
pvSrcObj
[in] Pointer to additional information. (Currently always NULL).
pName
[in] String containing the property name.
pValue
[in] String value that the property should be set to.

Return values

Value | Description
S_OK | Function completed successfully. Engine supports this property attribute and has set it to the requested value.
S_FALSE | Function completed successfully but engine does not support this property.
FAILED(hr) | Appropriate error message.


ISpSREngine::GetPropertyString

ISpSREngine::GetPropertyString retrieves a string property value from the SR engine.

Properties can be used by the application to control run-time results of the SR engine. The application can set and retrieve values for specific attributes on the engine. Some values are predefined by SAPI and others may be added by an engine. See the "SAPI 5.0 SR Properties White Paper" in the White papers section of the help for more details.

SAPI calls this method on an engine when the application calls ISpProperties::GetPropertyString using its recognition context object.

HRESULT GetPropertyString(
   SPPROPSRC        eSrc,
   void            *pvSrcObj,
   const   WCHAR   *pName,
   WCHAR          **ppCoMemValue
);

Parameters

eSrc
[in] One of the types specified in the SPPROPSRC enumeration sequence. (This will currently always be SPPROPSRC_RECO_INST).
pvSrcObj
[in] Pointer to additional information. (Currently always NULL).
pName
[in] String containing the property name.
ppCoMemValue
[out] Pointer to a string that the SR engine should supply with the property value string. The string should be allocated with CoTaskMemAlloc; SAPI will delete the allocated memory after return from this function. If the engine does not support this property attribute, the parameter should be NULL.

Return values

Value | Description
S_OK | Function completed successfully. Engine supports this property attribute and has returned a value for it.
S_FALSE | Function completed successfully but engine does not support this property.
FAILED(hr) | Appropriate error message.


ISpSREngine::SetGrammarState

ISpSREngine::SetGrammarState indicates that a grammar has been activated or deactivated by the application calling ISpRecoGrammar::SetGrammarState. When using standard CFG and dictation grammars, the engine may not need to be informed of the grammar state, as SAPI will automatically activate and deactivate the grammars as necessary. However, when using proprietary grammars, it may be necessary to look at this information.

HRESULT SetGrammarState(
   void             *pvEngineGrammar,
   SPGRAMMARSTATE   *eGrammarState
);

Parameters

pvEngineGrammar
[in] The engine's grammar pointer for this grammar, as returned from a previous call to the OnCreateGrammar method.
eGrammarState
[in] Flag of type SPGRAMMARSTATE indicating the new state of the grammar. This will either be SPGS_DISABLED or SPGS_ENABLED to indicate the grammar is being disabled or enabled; or SPGS_EXCLUSIVE to indicate this grammar has been enabled exclusively and other non-exclusive grammars will be disabled.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::WordNotify

ISpSREngine::WordNotify notifies the SR engine when words in command and control grammars are being added or removed.

This method is called by SAPI when words are added or removed when the application loads, unloads, or modifies grammars. SAPI internally keeps a reference count and each word will be added only if it is not already present in any existing grammar.

HRESULT WordNotify(
  SPCFGNOTIFY          Action,
  ULONG                cWords,
  const SPWORDENTRY  * pWords
);

Parameters

Action
The SPCFGNOTIFY enumeration value specifying which action, add or delete, is occurring. This will either be SPCFGN_ADD when words are added, or SPCFGN_REMOVE when they are deleted.
cWords
The number of words contained in pWords.
pWords
Array of SPWORDENTRY structures containing information on each word.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_NO_WORD_PRONUNCIATION | The engine could not generate a pronunciation for some of the words added.
FAILED(hr) | Other appropriate error message.


ISpSREngine::RuleNotify

ISpSREngine::RuleNotify notifies the SR engine when CFG rules are added, changed, or removed.

HRESULT RuleNotify(
  SPCFGNOTIFY          Action,
  ULONG                cRules,
  const SPRULEENTRY   *pRules
);

Parameters

Action
The SPCFGNOTIFY enumeration value specifying the action that is occurring.
cRules
The number of rules contained in pRules.
pRules
Array of SPRULEENTRY structures containing information on each rule.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.

Remarks

The engine recognizes each CFG grammar as containing one or more rules. Rules can be top-level, indicating that they can be activated for recognition. Each rule has an initial state and additional states, connected by transitions. Each transition can be one of several types: a word transition indicating a word to be recognized, a rule transition indicating a reference to a sub-rule, an epsilon (null) transition, and some special transitions for such features as embedding dictation within a CFG.

References to sub-rules can be recursive – i.e., rules can reference themselves, either directly or indirectly. Left recursion is not supported and SAPI will reject such grammars upon loading. Inside a grammar, transitions can have semantic properties, although the engine does not normally need to identify these.

SAPI takes full control of loading of a grammar when an application asks it to. The loading can be from a file, a URL, a resource, or from memory, and can involve loading either binary or XML forms of the grammar, and resolving imports. SAPI then notifies the SR engine about the contents of the grammar through various DDI methods.

There are five actions that are performed on rules:

- New rules can be added (SPCFGN_ADD) or existing rules removed (SPCFGN_REMOVE).
- Rules can be activated (SPCFGN_ACTIVATE) or deactivated (SPCFGN_DEATIVATE) for recognition.
- Rule can be invalidated (SPCFGN_INVALIDATE), which means the rule has been edited by the application and thus the engine needs to reread the contents of the rule.

Each rule is represented by an SPRULEENTRY structure. This contains an hRule, which gives a unique handle identifying the rule. The pvClientRuleContext is a pointer that the engine can set by using ISpSREngineSite::SetRuleClientContext. Then, subsequent calls to the ISpSREngineSite::GetRuleInfo method will return the same structure but with the pvClientRuleContext field filled in. The pvClientGrammarContext is the pointer that the engine set in the ISpSREngine::OnCreateGrammar method. This indicates which grammar the rule belongs to. The Attributes field, of type SPCFGRULEATTRIBUTES, contains some flags giving extra information about the rule:

SPRAF_TopLevel if the rule is top level and thus can be activated for recognition.

SPRAF_Active if the rule is currently activated.

SPRAF_Interpreter if the rule is associated with an Interpreter object for semantic processing.

SPRAF_AutoPause if the rule is autopause.

The hInitialState gives the initial state of the rule.

In order for the SR engine to find out the full contents of the rule (either immediately, or later during recognition), it can use the ISpSREngineSite::GetStateInfo method. This gives information about all the subsequent transitions and states following on from any given state (starting with the initial state of the rule).


ISpSREngine::PrivateCallEx

ISpSREngine::PrivateCallEx performs the same task as ISpSREngine::PrivateCall, except that a variable sized data block can be returned from the engine to the engine extension object.

HRESULT PrivateCallEx(
   void        *pvEngineContext,
   const void  *pInCallFrame,
   ULONG        ulInCallFrameSize,
   void       **ppvCoMemResponse,
   ULONG       *pulResponseSize
 );

Parameters

pvEngineContext
[in] The engine's pointer for this context, as returned from a previous call to OnCreateRecoContext.
pInCallFrame
[in] Address of the engine-specific input data.
ulInCallFrameSize
[in] Size, in bytes, of the engine-specific data contained in pInCallFrame.
ppvCoMemResponse
[out] Address of a pointer to the response block information from the SR engine. This must be allocated with CoTaskMemAlloc.
pulResponseSize
[out] Size, in bytes, of the ppvCoMemResponse data.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngine::SetContextState

ISpSREngine::SetContextState indicates that a recognition context has been activated or deactivated by the application calling ISpRecoContext::SetContextState. When using standard CFG and dictation grammars, the engine may not need to be informed of the context state, as SAPI will automatically activate and deactivate the grammars as necessary. However, when using proprietary grammars, it may be necessary to look at this information.

HRESULT SetContextState(
  void          * pvEngineContext,
  SPCONTEXTSTATE  eContextState
);

Parameters

pvEngineContext
[in] Pointer to the engine's data for this context, as returned from a previous call to OnCreateRecoContext.
eContextState
[in] The SPCONTEXTSTATE enumeration value specifying the new recognition context state.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED(hr) | Appropriate error message.


ISpSREngineSite

The interface ISpEngineSite is implemented by SAPI and is called by the SR engine. It is used by the SR engine to get audio data, retrieve grammar information, send event events, and return recognition information to SAPI.

Full details on how an SR engine interacts with SAPI and how it should call the methods on this interface are given in the "SR Engine Guide" in the white papers section of the Help files.

Implemented By
  • SAPI. Applications or engines do not implement this method.
Created By
  • SAPI. The SR engine obtains a pointer to this interface when the ISpSREngine::SetSite is called, after SAPI creates the SR engine.

Methods in Vtable Order

ISpSREngineSite Methods | Description
Read | Reads audio data from the input stream.
DataAvailable | Retrieves the amount of data that can be read using ISpSREngineSite::Read without blocking.
SetBufferNotifySize | Sets the amount of data to be available before data available event is set.
ParseFromTransitions | Produces an ISpPhraseBuilder result from a list of transitions.
Recognition | Returns a recognition result (final, partial, or false) to SAPI.
AddEvent | Sends an event back from the engine to applications.
Synchronize | Informs SAPI that the engine is ready to process changes in its grammars.
GetWordInfo | Retrieves information about a word in a CFG grammar.
SetWordClientContext | Sets an engine-defined pointer on a CFG word.
GetRuleInfo | Retrieves information about a CFG rule.
SetRuleClientContext | Sets an engine-defined pointer on a CFG rule.
GetStateInfo | Retrieves information on the transitions from a CFG state.
GetResource | Retrieves a named resource from a grammar.
GetTransitionProperty | Retrieves semantic property information for a transition in a grammar.
IsAlternate | Determines whether one rule is a valid alternate of another.
GetMaxAlternates | Returns the maximum number of alternates that should be generated for the specified rule.
GetContextMaxAlternates | Returns the maximum number of alternates that should be generated for the specified recognition context.
UpdateRecoPos | Informs SAPI of the current position of the recognizer in the stream to SAPI.


ISpSREngineSite::Read

ISpSREngineSite::Read returns a recognition result (final, partial, or false) to SAPI.

The engine requests a certain amount of data to read and supplies a buffer of this size. SAPI will read this amount of data from the audio input. If the amount of data that is requested is not available immediately, SAPI will block this call until the requested amount is available.

If this call returns with a failure code or if the amount that was read is less than the amount requested, this indicates that the stream has ended. The engine should return from the RecognizeStream method after it has finished processing all data.

This method can only be called while the SR engine is inside a ISpSREngine::RecognizeStream call, although it can be called on any thread.

HRESULT Read(
   void    *pv,
   ULONG    cb,
   ULONG   *pcbRead
);

Parameters

pv
[in] Pointer to the buffer into which the audio input stream data is read.
cb
[in] Specifies the number of bytes of data to attempt to read from the audio input stream.
pcbRead
[out] Pointer to a ULONG variable that receives the actual number of bytes read from the audio input stream.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_AUDIO_BUFFER_OVERFLOW | SAPI's internal audio buffer has filled, and the device has been closed. See Remarks section.
SPERR_AUDIO_BUFFER_UNDERFLOW | The audio object has not received audio data from the device quickly enough, and the device has been closed. See Remarks section.
SPERR_AUDIO_STOPPED | Audio device state has been set to stop.
SPERR_STREAM_NOT_ACTIVE | Method called when engine is not inside RecognizeStream call.
E_OUTOFMEMORY | Exceeded available memory
E_POINTER | At least one of pcbRead or pv are invalid or bad.
FAILED (hr) | Other appropriate error message.

Remarks

When the SR engine calls ISpSREngineSite::Read, SAPI will ultimately call ISpAudio::Read and pass the return code back to the SR engine using ISpSREngineSite::Read. If an error code is returned and the error is recoverable, SAPI will automatically detect that an audio error occurred and attempt to reactivate the audio device. The SR Engine will then receive a new call, ISpSREngine::RecognizeStream, that will enable it to continue recognizing with minimal audio data loss.

When using a real-time audio device as input, it is important for an SR Engine to call Read as often as it is able to avoid an audio buffer overflow (for example see ISpMMSysAudio::Read).

In all cases, if the Read call returns a failure code, the engine should not return this as the return value of the RecognizeStream method. This failure code can occur for normal conditions indicating the audio has finished.


ISpSREngineSite::DataAvailable

ISpSREngineSite::DataAvailable retrieves the amount of data that can be read using ISpSREngineSite::Read without blocking.

This method can only be called while the SR engine is inside a ISpSREngine::RecognizeStream call, although it can be called on any thread.

HRESULT DataAvailable(
   ULONG   *pcb
);

Parameters

pcb
[out] The amount, in bytes, of data available. For real-time audio streams this is the actual amount of data currently available. For non real-time streams this method will always return the value INFINITE. Using the fRealTimeAudio parameter on the RecognizeStream method, engines can determine whether this is a real-time stream.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pcb is a bad write pointer.
SPERR_STREAM_NOT_ACTIVE | Method called when engine is not inside RecognizeStream call.
FAILED(hr) | Other appropriate error message.


ISpSREngineSite::SetBufferNotifySize

ISpSREngineSite::SetBufferNotifySize controls how much data will be available to read before this event gets set. This method is used in conjunction with the hDataAvailable Win32 event that is passed as a parameter in the ISpSREngine::RecognizeStream method.

This can be used if an engine calls Read only when at least a certain amount of data is available.

On non-real time streams, for example when reading from a wave file, this event will always be set, as all the data in the file is always available for reading. This method will return SP_UNSUPPORTED_ON_STREAM_INPUT in this case.

This method can only be called while the SR engine is inside a ISpSREngine::RecognizeStream call, although it can be called on any thread.

HRESULT SetBufferNotifySize(
   ULONG   cbSize
);

Parameters

cbSize
[in] The minimum amount of data that should be available before the data available event is set.

Return values

Value | Description
S_OK | Function completed successfully.
SP_UNSUPPORTED_ON_STREAM_INPUT | Function call has no effect as this is a non-real time audio stream.
FAILED(hr) | Other appropriate error message.


ISpSREngineSite::ParseFromTransitions

ISpSREngineSite::ParseFromTransitions parses an ISpPhraseBuilder result from a list of CFG transitions.

This method is called to produce a phrase which can be used to send CFG results back to SAPI.

For full more details on this method, see the SR Engine Guide.

HRESULT ParseFromTransitions(
   const   SPPARSEINFO   *pParseInfo,
   ISpPhraseBuilder     **ppPhrase
);

Parameters

pParseInfo
[in] Address of the SPPARSEINFO structure containing phrase information.
ppPhrase
[out] Address of a pointer to an ISpPhraseBuilder interface that receives the phrase information.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


ISpSREngineSite::Recognition

ISpSREngineSite::Recognition returns a recognition result to SAPI.

The phrase can be either a hypothesis or a final result. If it is a hypothesis, a hypothesis event is issued to all interested recognition contexts. A final result event is issued to the target grammar that the result refers to. An engine can also send a false recognition with this method, indicating it has low confidence in the result.

If the return value from this call is S_FALSE, there are no more active rules and the engine is free to exit the RecognizeStream call without reading or processing any more data. Otherwise, the engine should continue reading data and continue recognition.

An ISpSREngineSite::AddEvent call with an SPEI_PHRASE_START parameter as the event type must precede the call to Recognition.

For full more details on this method, see the "SR Engine Guide".

HRESULT Recognition(
   SPRECORESULTINFO   *pResultInfo
);

Parameters

pResultInfo
[in] Pointer to type SPRECORESULTINFO indicating the results.

Return values

Value | Description
S_OK | Function completed successfully and to continue recognition.
S_FALSE | Function completed successfully and the engine can terminate recognition without reading the rest of the stream.
FAILED (hr) | Appropriate error message.


ISpSREngineSite::AddEvent

ISpSREngineSite::AddEvent sends an event back from the engine to applications.

For more details on this method and the different events that can be fired, see the SR Engine Guide.

HRESULT AddEvent(
   const   SPEVENT       *pEvent,
   SPRECOCONTEXTHANDLE    hContext
);

Parameters

pEvent
[in] Address of the SPEVENT structure containing the event information.
hContext
[in] The RecoContext is the event handle passed to SR Engine from SAPI through ISpSREngine::OnCreateRecoContext. This value should normally be set to NULL indicating the event is a global one.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | At least one of pEvent or hContext is invalid or bad. Alternatively, it indicates that an event is being added to an inappropriate mode.
E_POINTER | Invalid pointer.
SPERR_STREAM_POS_INVALID | The current audio stream offset is greater than either the current seek position or the last sync position. Alternatively, if the event stream is not initialized, the stream position is not zero.
FAILED(hr) | Appropriate error message.


ISpSREngineSite::Synchronize

ISpSREngineSite::Synchronize informs SAPI that the engine is ready to process changes in its grammars.

If there are any changes pending, SAPI will call back to the engine to inform it of any changes using WordNotify or RuleNotify. When the engine returns from these methods, SAPI will return back from the Synchronize call. If the return value from this call is S_FALSE, and there are no more active rules, the engine is free to exit the RecognizeStream call without reading or processing more data.

The engine can choose when to call Synchronize. Often an engine will respond to state changes when no speech is detected, but it will not respond when the user is speaking. It is important, however, to periodically, if not routinely, call Synchronize. Specifically, if an application attempts to release its final reference to SAPI, and no other applications are connected, SAPI will attempt to shutdown the SR engine. However, the shutdown process will wait indefinitely for the SR Engine to reach its next synchronization point (e.g., the speech recognition engine calls ISpSREngineSite::Synchronize).

This method can only be called while the SR engine is inside a ISpSREngine::RecognizeStream call, although it can be called on any thread.

For full more details on this method, see the SR Engine Guide.

HRESULT Synchronize(
   ULONGLONG   ullStreamPos
);

Parameters

ullStreamPos
[in] The position within the audio stream that the engine has completed recognizing. SAPI discards its stored audio up to this point. The engine cannot fire more events prior to this position. However, the engine will still be informed of current grammar changes regardless of the value of the parameter.

Return values

Value | Description
S_OK | Function completed successfully; recognition should continue.
S_FALSE | Function completed successfully and the engine can terminate recognition without reading the rest of the stream.
SPERR_STREAM_NOT_ACTIVE | Stream is not initialized.
SPERR_STREAM_POS_INVALID | Stream position is either greater than the current seek position or less than the last synchronized position.
FAILED (hr) | Appropriate error message.

Notes

An example of the synchronization for the user would be starting dictation mode, and activating a non-silence noise source in the background that will generate a single continuous recognition. When the user attempts to exit the application, the exit will be blocked until the SR engine finishes recognizing the audio stream. For this reason, the SR engine should call ISpSREngineSite::Synchronize periodically to prevent extended delays in state changes (e.g., application shutdown, grammar changes, etc.), even when performing a long recognition. This ensures that the SR Engine is able to properly clean up and exit its ISpSREngine::RecognizeStream method.


ISpSREngineSite::GetWordInfo

ISpSREngineSite::GetWordInfo retrieves information about a word in a CFG grammar.

HRESULT GetWordInfo(
   SPWORDENTRY    *pWordEntry,
   SPWORDINFOOPT   Options
);

Parameters

pWordEntry
[in, out] Address of the SPWORDENTRY structure that contains the grammar word entry information. This can be called with only the hWord word handle set. The following members may be allocated with CoTaskMemAlloc() and if so, must each be freed by the engine with CoTaskMemTaskFree() when no longer required.
pWordEntry->pszDisplayText
pWordEntry->pszLexicalForm
pWordEntry->aPhoneId
Options

[in] One of the grammar word options specified in the SPWORDINFOOPT enumeration. If SPWIO_NONE, the LangID and pvClientContext are filled in. If SPWIO_WANT_TEXT, the display and lexical text and pronunciation are also filled in.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pWordEntry points to invalid memory.
E_INVALIDARG | Either invalid hWord word handle or Options contains invalid flags.
E_OUTOFMEMORY | Not enough memory to complete the operation.
FAILED (hr) | Appropriate error message.


ISpSREngineSite::SetWordClientContext

ISpSREngineSite::SetWordClientContext sets an engine-defined context pointer on a CFG word.

This allows an engine to associate a pointer to its own data with each word. This can be quickly recovered with the ISpSREngineSite::GetWordInfo method.

HRESULT SetWordClientContext(
   SPWORDHANDLE   hWord,
   void          *pvClientContext
);

Parameters

hWord
[in] The handle for a word.
pvClientContext
[in] Pointer to the engine's data it wishes to associate with this word.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_INVALID_HANDLE | Invalid word handle argument.
FAILED (hr) | Other appropriate error message.


ISpSREngineSite::GetRuleInfo

ISpSREngineSite::GetRuleInfo retrieves information about a CFG rule.

This method can be called after the engine has been informed of a rule with the ISpSREngine::RuleNotify method. This method can be used to recover rule information from the rule handle. Only the hRule rule handle field in the SPRULEENTRY needs to be filled in by the engine when calling this method. The engine will fill in:

  • The pvClientRuleContext, which is a pointer that the engine can set by using ISpSREngineSite::SetRuleClientContext.
  • The pvClientGrammarContext, which is the pointer that the engine set in the ISpSREngine::OnCreateGrammar method. This indicates which grammar the rule belongs to.
  • The Attributes field, of type SPCFGRULEATTRIBUTES, which contains some flags giving extra information about the rule:
SPAudioBufferInfo.ulMsMinNotification cannot be larger than one quarter the size of SPAudioBufferInfo.ulMsBufferSize and must not be zero.
SPRAF_TopLevel if the rule is top-level and thus can be activated for recognition.
SPRAF_Interpreter if the rule is associated with an Interpreter object for semantic processing.
SPRAF_AutoPause if the rule is auto-pause.
  • The hInitialState field, which gives the initial state of the rule.Information on this and subsequent states can be obtained by calling the ISpSREngineSite::GetStateInfo method.

HRESULT GetRuleInfo(
   SPRULEENTRY    *pRuleEntry,
   SPRULEINFOOPT   Options
);

Parameters

pRuleEntry
[in, out] Address of the SPRULEENTRY structure that contains the grammar rule entry information.
Options
[in] One of the grammar rule options specified in the SPRULEINFOOPT enumeration sequence. (This should always be set to SPRIO_NONE).

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pRuleEntry points to invalid memory.
E_INVALIDARG | Either invalid hRule rule handle or Options contains invalid flags.
FAILED (hr) | Appropriate error message.


ISpSREngineSite::SetRuleClientContext

ISpSREngineSite::SetRuleClientContext sets an engine-defined pointer on a CFG rule.

This allows an engine to associate a pointer to its own data with each rule. This can be quickly recovered with the ISpSREngineSite::GetRuleInfo method.

HRESULT SetRuleClientContext(
   SPRULEHANDLE   hRule,
   void          *pvClientContext
);

Parameters

hRule
[in] Handle of a rule.
pvClientContext
[in] Pointer to the engine's data it wishes to associate with this rule.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_INVALID_HANDLE | Invalid rule handle argument.
FAILED (hr) | Other appropriate error message.


ISpSREngineSite::GetStateInfo

ISpSREngineSite::GetStateInfo retrieves information on the transitions from a CFG state.

This method is called so that the SR engine can discern the full contents of the rule. GetStateInfo can be called immediately upon receiving a ISpSREngine::RuleNotify call about a rule, or later during recognition. This method supplies information about all the subsequent states from any given state.

HRESULT GetStateInfo(
   SPSTATEHANDLE   hState,
   SPSTATEINFO    *pStateInfo
);

Parameters

hState
[in] Handle to the current state.
pStateInfo
[out] The state information.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Either pStateInfo or pTransitions in pStateInfo points to invalid memory.
E_OUTOFMEMORY | Not enough memory to complete the operation.
SPERR_INVALID_HANDLE | Invalid state handle argument.
FAILED (hr) | Appropriate error message.

Remarks

The engine passes this method a state handle (starting with the hInitialState of the rule), and a pointer to an SPSTATEINFO structure with all its fields initially zeroed. This structure is filled out with information on all of the transitions out of that state in the pTransitions array. SAPI uses CoTaskMemAlloc to allocate this array. The engine can call this method again on each of the states following the current state, and so on, in order to get information about all of the states in the rule. Loop-back transitions are possible in a rule and the engine may need to confirm that it has not visited the state before.

When the engine calls GetStateInfo subsequent times, it can call it with the cAllocatedEntries and pTransitions fields unchanged from the last call. If possible, SAPI will then re-use the memory from the transition array rather that re-allocating it. Alternatively, the engine can use CoTaskMemFree to free the pTransitions memory, set these fields to NULL and SAPI will re-allocate the memory every time.

Each transition represents a link from one state to another state. Each transition is represented by an SPTRANSITIONENTRY structure. This structure contains an ID field that uniquely identifies the transition, an hNextState handle that indicates the state the transition is connected to, and a Type field that indicates the type of transition.

There are three common types of transition, which all engines should support:

  • Word transitions (SPTRANSWORD). These represent single words that the recognizer will recognize before advancing to the next state. The handle to the word and the engine’s word pointer are supplied inside the SPTRANSITIONENTRY structure. To produce recognition results, the engine needs to keep track of the transition IDs of word transitions as they are used in the ParseFromTransitions method.
  • Rule transitions (SPTRANSRULE). These represent transitions into sub-rules. This transition is only passed when a path through the sub-rule has been recognized. The rule handle, engine’s rule pointer and initial state of the sub-rule are supplied. Rules can be recursive (not left recursive).
  • Epsilon transitions (SPTRANSEPSILON). These are NULL transitions that can be traversed without recognizing anything.

A state that has a transition to a null state handle indicates the end of a rule. There can also be ‘void’ states, which block and indicate that there is no recognition path from this state. A void state has zero transitions out of it.

There are also a number of special transitions, which may not be supported by all engines.

See the SR Engine Guide document for more information on CFG grammars.


ISpSREngineSite::GetResource

ISpSREngineSite::GetResource retrieves a named resource from a grammar.

Within a CFG, each rule can contain one or more named strings containing arbitrary string data. The engine can recover this data using ::GetResource and passing in the rule handle and resource name.

HRESULT GetResource(
   SPRULEHANDLE   hRule,
   WCHAR         *pszResourceName,
   WCHAR        **ppCoMemResource
);

Parameters

hRule
[in] The rule handle.
pszResourceName
[in] Null-terminated string containing the name of the resource to recover.
ppCoMemResource
[out] The resource associated with the rule. Applications calling this method must call CoTaskMemFree() to free memory associated with this resource.

Return values

Value | Description
S_OK | Function completed successfully and the rule contained a resource of the correct name.
S_FALSE | Function completed successfully but no resource was found.
E_INVALIDARG | pszResourceName points to invalid string
E_POINTER | ppCoMemResource invalid or bad.
E_NOTIMPL | Method is not implemented.
SPERR_INVALID_HANDLE | Invalid hRule handle.
FAILED (hr) | Appropriate error message.


ISpSREngineSite::GetTransitionProperty

ISpSREngineSite::GetTransitionProperty retrieves semantic property information for a transition in a grammar.

CFG grammars can contain properties (also known as ‘semantic tags’) within a grammar. This provides a means for semantic information to be embedded inside a grammar.

HRESULT GetTransitionProperty(
   SPTRANSITIONID           ID,
   SPTRANSITIONPROPERTY   **ppCoMemProperty
);

Parameters

ID
[in] The transition identifier.
ppCoMemProperty
[out] Address of a pointer to a SPTRANSITIONPROPERTY that receives the property information. Applications calling this method must call CoTaskMemFree() to free memory returned. If the transition does not have a semantic property, this will be set to point to NULL, and S_FALSE will be returned.

Return values

Value | Description
S_OK | Function completed successfully and transition has a property.
S_FALSE | Function completed successfully but transition does not have a property.
E_INVALIDARG | One or more arguments are invalid.
E_OUTOFMEMORY | Exceeded available memory.
FAILED(hr) | Appropriate error message.

Remarks

By default, the engine does not recognize these properties. Typically, an engine will recognize only the speech from the words in the grammar, and SAPI will parse and add the property information in the ISpSREngineSite::ParseFromTransitions call. However, it is possible for an engine to recognize this information by calling this method on any transition. If there is a property on this transition, the property name or id and value are returned in the SPTRANSITIONPROPERTY structure, which needs to be freed after each use using CoTaskMemFree.

An engine can find out if a transition has a semantic property attached before calling this method by looking at the fHasProperty flag in the SPTRANSITIONENTRY structure associated with this transition.


ISpSREngineSite::IsAlternate

ISpSREngineSite::IsAlternate determines whether one rule is an alternate of another.

This method is used because it is not possible to return alternates for CFG rules from different recognition contexts. This method provides an easy way for engines to determine whether two rules belong to the same context.

HRESULT IsAlternate(
   SPRULEHANDLE   hPriRule,
   SPRULEHANDLE   hAltRule
);

Parameters

hPriRule
[in] The primary rule.
hAltRule
[in] The alternate rule to be checked.

Return values

Value | Description
S_OK | hAltRule is an alternate of hPriRule.
S_FALSE | hAltRule is not an alternate of hPriRule.
FAILED (hr) | Appropriate error message.


ISpSREngineSite::GetMaxAlternates

ISpSREngineSite::GetMaxAlternates passes back the maximum number of alternates that should be generated for the specified CFG rule.

HRESULT GetMaxAlternates(
   SPRULEHANDLE   hRule,
   ULONG         *pulNumAlts
);

Parameters

hRule
[in] The rule to check.
pulNumAlts
[out] The maximum number of alternates for the rule.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pulNumAlts is invalid or bad.
FAILED (hr) | Appropriate error message.


ISpSREngineSite::GetContextMaxAlternates

ISpSREngineSite::GetContextMaxAlternates passes back the maximum number of alternates that should be generated for the specified recognition context.

For SAPI command and control grammars, it is usually easier to use the ISpSREngineSite::GetMaxAlternates method.

This method currently applies to command and control and proprietary grammars, but not to dicatation grammars.

HRESULT GetContextMaxAlternates(
   SPRECOCONTEXTHANDLE   hContext,
   ULONG                *pulNumAlts
);

Parameters

hContext
[in] Handle to the context.
pulNumAlts
[out] The maximum number of alternates to generate.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | pulNumAlts is invalid or bad.
FAILED (hr) | Appropriate error message.


ISpSREngineSite::UpdateRecoPos

ISpSREngineSite::UpdateRecoPos returns the current position of the recognizer in the stream to SAPI. An engine should call this regularly, up to several times a second, regardless of whether it is recognizing speech or silence.

This method can only be called while the SR engine is inside a ISpSREngine::RecognizeStream call, although it can be called on any thread.

HRESULT UpdateRecoPos(
   ULONGLONG   ullStreamPos
);

Parameters

ullStreamPos
[out] The current stream position of the recognizer.

Return values

Value | Description
S_OK | Function completed successfully.


ISpSRAlternates

ISpSRAlternates allows alternate word selection and implementation for speech recognition.

Methods in Vtable Order

ISpSRAlternates Methods | Description
GetAlternates | Retrieves a list of alternate words.
Commit | Chooses the lexicon manager's word probability.


ISpSRAlternates::GetAlternates

ISpSRAlternates::GetAlternates retrieves a list of alternate words.

HRESULT GetAlternates(
   SPPHRASEALTREQUEST   *pAltRequest,
   SPPHRASEALT         **ppAlts,
   ULONG                *pcAlts
);

Parameters

pAltRequest
[in] A structure to the requested alternate words.
ppAlts
[out] A list of SPPHRASEALT for alternate words.
pcAlts
[out] The number of alternates in ppAltslist.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


ISpSRAlternates::Commit

ISpSRAlternates::Commit chooses the lexicon manager's word probability. This allows fine adjustments for the decoder to pick the new, alternate words over the current ones.

HRESULT Commit(
   SPPHRASEALTREQUEST  *pAltRequest,
   SPPHRASEALT         *pAlt,
   void               **ppvResultExtra,
   ULONG               *pcbResultExtra
);

Parameters

pAltRequest
[in] A pointer to the structure to the requested alternate words.
pAlt
[in] A pointer to the structure to alternate words.
ppvResultExtra
[out] Additional information for the new results.
pcbResultExtra
[out] Size, in bytes, of ppvResultExtra.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Text To Speech Recognition Engine Manager (DDI-level)

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Text To Speech Engine interfaces (DDI-level)

The following section covers:

  • ISpTTSEngine
  • ISpTTSEngineSite


ISpTTSEngine

The SAPI speech synthesis (text-to-speech, or TTS) engine (driver) implements the ISpTTSEngine interface.

ISpTTSEngine::Speak is the primary method called by SAPI to perform speech rendering. SAPI, rather than the engine, performs XML parsing of the input text stream. The engine's Speak method is handed a linked list of text fragments with their associated XML attribute states. The Speak method also receives a pointer to the ISpVoice's ISpTTSEngineSite interface. The TTS engine uses this interface to queue events and to write the output audio data.

Even though SAPI 5.0 is a free–threaded architecture, TTS engine instances will always be called by SAPI on a single thread. TTS engines are never directly accessed by applications. SAPI ensures that all parameter validation and thread synchronization has been performed properly before calling the TTS engine. All calls to the TTS engine in the release build of SAPI are within a try or except block to protect applications from faulting.

Methods in Vtable Order

ISpTTSEngine Methods | Description
Speak | Renders the specified text fragment list in the specified output format.
GetOutputFormat | Determines the output format the engine wants to use.


ISpTTSEngine::Speak

ISpTTSEngine::Speak passes the engine the text to be rendered, an output format to render it in, and an output site to which the engine should write audio data and events.

HRESULT Speak(
   DWORD                 dwSpeakFlags,
   REFGUID               rguidFormatId,
   const WaveFormatEx   *pWaveFormatEx,
   const SPVTEXTFRAG    *pTextFragList,
   ISpTTSEngineSite     *pOutputSite
);

Parameters

dwSpeakFlags
[in] Possible values are contained in the SPEAKFLAGS enumeration, but all values other than SPF_NLP_SPEAK_PUNC will be masked off. If SPF_NLP_SPEAK_PUNC is set, the engine should speak all punctuation (e.g., "This is a sentence." should be expanded to "This is a sentence period").
rguidFormatId
[in] The stream format identifier describing the required output format. This format is guaranteed to be one that the engine specified as supported in a previous GetOutputFormat call.
SPDFID_Text | Engines are not required to support this format, nor are they required to do anything specific with this format if they do support it. It is provided merely for debugging purposes.
SPDFID_WaveFormatEx | pWaveFormatEx will be a WAVEFORMATEX structure describing an output format specified by the engine in a previous GetOutputFormat call.
pWaveFormatEx
[in] Pointer to a WAVEFORMATEX structure describing the output format. Will be NULL if rguidFormatID is SPDFID_Text.
pTextFragList
[in] A linked list of SPVTEXTFRAGs to synthesize. See the TTS Engine Vendor Porting Guide for more information.
pOutputSite
[in] Pointer to an ISpTTSEngineSite where audio data and events should be written.


ISpTTSEngine::GetOutputFormat

ISpTTSEngine::GetOutputFormat queries the engine about a specific output format. The engine should examine the requested output format, and return the closest format which it supports.

HRESULT GetOutputFormat(
   const GUID          *pTargetFmtId,
   const WAVEFORMATEX  *pTargetWaveFormatEx,
   GUID                *pOutputFormatId,
   WAVEFORMATEX       **ppCoMemOutputWaveFormatEx
);

Parameters

pTargetFmtId
[in] Address of the GUID describing the requested output format.
SPDFID_Text | Engines are not required to support this format, nor are they required to do anything specific with this format if they do support it. It is provided merely for debugging purposes.
SPDFID_WaveFormatEx | pWaveFormatEx will be a WAVEFORMATEX structure.
pTargetWaveFormatEx
[in] Pointer to the WAVEFORMATEX structure describing the requested output format.
Will be NULL if pTargetFmtId is SPDFID_Text.
pOutputFormatId
[out] Address of a GUID to receive the engine's supported output format identifier.
SPDFID_Text | If the engine does support SPDFID_Text, SPDFID_Text should be used, and ppCoMemOutputWaveFormatEx should be set to NULL.
SPDFID_WaveFormatEx | Output format will be described by a WAVEFORMATEX structure.
ppCoMemOutputWaveFormatEx
[out] The engine should allocate space for a WAVEFORMATEX structure using CoTaskMemAlloc. This structure should describe the supported output format.


ISpTTSEngineSite

The ISpTTSEngineSite interface is implemented by the SAPI SpVoice object. It is used to write audio data and events. See the TTS Engine Vendor Porting Guide for more information.

Note: The ISpTTSEngineSite interface inherits from ISpEventSink. This interface is where AddEvents and GetEventInterest can be found.

Methods in Vtable Order

ISpTTSEngineSite Methods | Description
ISpEventSink inherited methods. | All the methods of ISpEventSink are accessible from this interface.
GetActions | Queries the ISpVoice to determine what action(s) to perform.
Write | Sends output data (normally audio) to SAPI.
GetRate | Retrieves the base rate adjustment that should be used by the engine.
GetVolume | Retrieves the base volume level that should be used by the engine.
GetSkipInfo | Retrieves the number and type of items to be skipped in the text stream.
CompleteSkip | Notifies the SpVoice object that the last skip request has been completed and to pass it the results.


ISpTTSEngineSite::GetActions

ISpTTSEngineSite::GetActions queries the SpVoice object to determine what real-time action(s) to perform. An engine should call this method frequently during the rendering process to be as responsive as possible. SAPI returns a DWORD indicating which action(s) contained in the SPVESACTIONS enumeration should be performed. See GetRate, GetVolume, and GetSkipInfo for more information.

DWORD GetActions ( void );

Parameters

None.

Return values

DWORD containing one or more values from SPVESACTIONS specifying the action(s) to perform.


ISpTTSEngineSite::Write

ISpTTSEngineSite::Write sends output data (normally audio) to SAPI.

SAPI handles sending the audio data to the correct output destination. It is important that any events associated with the audio data are queued by calling the ISpEventSink::AddEvents method prior to calling this method. This ensures proper synchronization of event firing and audio rendering.

HRESULT Write(
   const void   *pBuff,
   ULONG         cb,
   ULONG        *pcbWritten
);

Parameters

pBuff
Pointer to synthesized speech audio data. The output format is specified by SAPI as a parameter to the ISpTTSEngine::Speak call.
cb
The buffer size, in bytes (not samples), of pBuff.
pcbWritten
Pointer to a ULONG which receives the number of bytes actually copied.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pBuff is bad or invalid.
E_POINTER | pcbWritten is bad or invalid.
SPERR_UNINITIALIZED | Output stream cannot be initialized.


ISpTTSEngineSite::GetRate

ISpTTSEngineSite::GetRate retrieves the current TTS rendering rate adjustment that should be used by the engine. This function should be called when a call to GetActions returns SPVES_RATE. The retrieved value establishes a baseline rate. Additional rate adjustments in the XML state should be combined with this value to determine the actual absolute rate adjustment.

HRESULT GetRate(
   long   *pRateAdjust
);

Parameters

pRateAdjust
[out] Pointer to a long which specifies the baseline rate.

Return values

Value | Description
E_POINTER | pRateAdjust is invalid
S_OK | Function completed successfully.


ISpTTSEngineSite::GetVolume

ISpTTSEngineSite::GetVolume retrieves the base output volume level the engine should use during synthesis. This function should be called when a call to GetActions returns SPVES_VOLUME. The retrieved value establishes a baseline volume. Additional volume adjustments in the XML state should be combined with this value to determine the actual absolute volume level.

HRESULT GetVolume(
  USHORT* pusVolume
);

Parameters

pusVolume
[out] Pointer to a USHORT which specifies the baseline volume level.

Return values

Value | Description
S_OK | Function completed successfully.
E POINTER | pusVolume is invalid.


ISpTTSEngineSite::GetSkipInfo

ISpTTSEngineSite::GetSkipInfo retrieves the number and type of items to be skipped in the text stream. This function should be called when a call to GetActions returns SPVES_SKIP. plNumItems can be positive, signifying a forward skip, or negative, signifying a backward skip, or zero, signifying a skip to the beginning of the current item.

HRESULT GetSkipInfo(
   SPVSKIPTYPE   *peType,
   long          *plNumItems
);

Parameters

peType
[out] Pointer to an SPVSKIPTYPE which specifies the type of item to skip. Currently only sentences.
plNumItems
[out] Pointer to a long which specifies the number of items to skip.

Return values

Value | Description
S_OK | Function completed successfully.
E POINTER | One of the return addresses is invalid.

Remarks

  • After the engine has skipped as many items as possible, it must call ISpTTSEngineSite::CompleteSkip to let SAPI know how many items were successfully skipped. If the engine was unable to skip the requested number of items, it should end its current Speak call immediately.


ISpTTSEngineSite::CompleteSkip

ISpTTSEngineSite::CompleteSkip notifies the SpVoice object that the last skip request has been completed and to pass it the results.

HRESULT CompleteSkip(
   long   ulNumSkipped
);

Parameters

ulNumSkipped
[in] Specifies the number of items that actually were skipped. It is invalid to skip more items than were requested.

Return values

Value | Description
E_INVALIDARG | ulNumSkipped is greater than number of items requested
S_OK | Function completed successfully.


Structures

The following structures are used with SAPI 5.

  • SPAUDIOBUFFERINFO
  • SPAUDIOSTATUS
  • SPBINARYGRAMMAR
  • SPEVENT
  • SPEVENTSOURCEINFO
  • SPPARSEINFO
  • SPPATHENTRY
  • SPPHRASE
  • SPPHRASEALT
  • SPPHRASEALTREQUEST
  • SPPHRASEELEMENT
  • SPPHRASEPROPERTY
  • SPPHRASEREPLACEMENT
  • SPPHRASERULE
  • SPPROPERTYINFO
  • SPRECOCONTEXTSTATUS
  • SPRECOGNIZERSTATUS
  • SPRECORESULTINFO
  • SPRECORESULTTIMES
  • SPRULEENTRY
  • SPSERIALIZEDEVENT
  • SPSERIALIZEDEVENT64
  • SPSERIALIZEDPHRASE
  • SPSERIALIZEDRESULT
  • SPSTATEINFO
  • SPTEXTSELECTIONINFO
  • SPTMTHREADINFO
  • SPTRANSITIONENTRY
  • SPTRANSITIONPROPERTY
  • SPVCONTEXT
  • SPVOICESTATUS
  • SPVPITCH
  • SPVSTATE
  • SPVTEXTFRAG
  • SPWORD
  • SPWORDENTRY
  • SPWORDLIST
  • SPWORDPRONUNCIATION
  • SPWORDPRONUNCIATIONLIST
  • WAVEFORMATEX


SPAUDIOBUFFERINFO

SPAUDIOBUFFERINFO contains the audio stream buffer information.

typedef struct SPAUDIOBUFFERINFO
{
    ULONG        ulMsMinNotification;
    ULONG        ulMsBufferSize;
    ULONG        ulMsEventBias;
} SPAUDIOBUFFERINFO;

Members

ulMsMinNotification
The minimum preferred time, in milliseconds, allowed between the actual time an event notification occurs and the ideal time. The smaller this number is, the more CPU overhead is required, but the event notifications will be more timely. This value must be at most one quarter the size of the ulMsBuffersize.
ulMsBufferSize
The size of the audio object’s buffer, in milliseconds. For readable audio objects, this is simply a preferred size – readable objects will automatically expand their buffers to accommodate data. For writeable audio objects, this is the amount of audio data that will be buffered before a call to Write will block.
ulMsEventBias
The amount of time, in milliseconds, that events will be completed before they actually occur. For example, setting a value of 100 for the event bias would cause all events to be notified 100 milliseconds prior to the audio data being played. This can be useful for applications needing time to animate mouths for synthetic speech.


SPAUDIOSTATUS

typedef [restricted] struct SPAUDIOSTATUS
{
    long            cbFreeBuffSpace;
    ULONG           cbNonBlockingIO;
    SPAUDIOSTATE    State;
    ULONGLONG       CurSeekPos;
    ULONGLONG       CurDevicePos;
    DWORD           dwReserved1;
    DWORD           dwReserved2;
} SPAUDIOSTATUS;

Members

cbFreeBuffSpace
Size, in bytes, of free space for reading and/or writing in the audio object.
cbNonBlockingIO
This is amount of data which can be read from or written to a device without blocking occurring.
State
The state (of type SPAUDIOSTATE) of the audio device.
CurSeekPos
The current seek position, in bytes, within the audio stream. This is the position in the stream at which the next read or write will be performed.
CurDevicePos
The current read position, in bytes, of the device. This is the position in the stream where the device is currently reading or writing. For readable streams, this value will always be greater than or equal to CurSeekPos. For writeable streams, this value will always be less than or equal to CurSeekPos.
dwReserved1
Reserved for future expansion.
dwReserved2
Reserved for future expansion.


SPBINARYGRAMMAR

SPBINARYGRAMMAR contains the grammar size information. A binary grammar is the resulting data after either calling ISpGrammarBuilder::Commit() or ISpGrammarCompiler::CompileStream().

typedef struct SPBINARYGRAMMAR
{
    ULONG     ulTotalSerializedSize;
} SPBINARYGRAMMAR;

Members

ulTotalSerializedSize
Total size, in bytes, of the serialized grammar.


SPEVENT

SPEVENT contains information about an event. Events are passed from the TTS or SR engines or audio devices back to applications.

typedef [restricted] struct SPEVENT
{
    WORD         eEventId;
    WORD         elParamType;
    ULONG        ulStreamNum;
    ULONGLONG    ullAudioStreamOffset;
    WPARAM       wParam;
    LPARAM       lParam;
} SPEVENT;

Members

elParamType
The parameter type of type SPEVENTLPARAMTYPE.
eEventId
The event ID of type SPEVENTENUM.
ulStreamNum
The input stream number of associated with the event.
ullAudioStreamOffset
The byte offset into the audio stream associated with the event that the event was fired at. For synthesis, the output stream is the synthesized data. For recognition, this indicates the position in the input audio stream.
wParam
The generic word field. For event IDs with the SPFEI_LPARAM_IS_POINTER set, this is the size, in bytes, for the data pointed to by lParam. In some cases, the type of event will change the function of this parameter. See SPEVENTENUM for information about specific events.
lParam
The generic event field. For event IDs with the SPFEI_LPARAM_IS_POINTER set, this points to the data allocated by CoTaskMemAlloc. The caller is responsible for freeing this memory using CoTaskMemFree(). In some cases, the type of event will change the function of this parameter. See SPEVENTENUM for information about specific events.


SPEVENTSOURCEINFO

A structure used by ISpEventSource::GetInfo to pass back event information.

typedef struct SPEVENTSOURCEINFO
{
    ULONGLONG   ullEventInterest;
    ULONGLONG   ullQueuedInterest;
    ULONG       ulCount;
} SPEVENTSOURCEINFO;

Members

ullEventInterest
Event ID flags of type SPEVENTENUM marking events which invoke a notification.
ullQueuedInterest
Queue of event IDs. These remain until ISpEventSource::GetEvents removes them.
ulCount
Number of events currently queued.


SPPARSEINFO

typedef struct SPPARSEINFO
{
   ULONG          cbSize;
   SPRULEHANDLE   hRule;
   ULONGLONG      ullAudioStreamPosition;
   ULONG          ulAudioSize;
   ULONG          cTransitions;
   SPPATHENTRY   *pPath;
   BOOL           fHypothesis;
   GUID           SREngineID;
   ULONG          ulSREnginePrivateDataSize;
   const BYTE    *pSREnginePrivateData;
} SPPARSEINFO;

Members

cbSize
The size of the SPPARSEINFO structure, as set by the SR engine.
hRule
The handle of the top-level rule for the recognition or hypothesis.
ullAudioStreamPosition
The position in the stream where the recognition started.
ulAudioSize
The size in bytes of the portion of the stream that is recognized.
cTransitions
The number of word transitions in the pPath array.
pPath
The transition path through the CFG that is recognized.
fHypothesis
TRUE if a hypothesis, false if a final recognition.
SREngineID
The unique identifier of the SR engine. Can be NULL.
ulSREnginePrivateDataSize
The size of private SR engine data that will be sent to the SR engine alternates analyzer. Can be zero.
pSREnginePrivateData
A pointer the private SR engine data (memory) that will be sent to the SR engine alternates analyzer. Can be NULL. SAPI will handle the serialization and marshaling of the data for the SR engine.


SPPATHENTRY

SPPATHENTRY array is passed by the SR engine to ISpSREngineSite::ParseFromTransitions as part of SPPARSEINFO

typedef [restricted] struct SPPATHENTRY
{
    union
    {
        SPTRANSITIONID      hTransition;
        SPPHRASEELEMENT     elem;
    };
} SPPATHENTRY;

Members

hTransition
Handle of the transition that was recognized at this point.
elem
Element information (can be left blank in which case SAPI will fill in the information from the grammar).


SPPHRASE

typedef [restricted] struct SPPHRASE
{
    ULONG                       cbSize;
    LANGID                      LangID;
    WORD                        wReserved;
    ULONGLONG                   ullGrammarID;
    ULONGLONG                   ftStartTime;
    ULONGLONG                   ullAudioStreamPosition;
    ULONG                       ulAudioSizeBytes;
    ULONG                       ulRetainedSizeBytes;
    ULONG                       ulAudioSizeTime;
    SPPHRASERULE                Rule;
    const SPPHRASEPROPERTY     *pProperties;
    const SPPHRASEELEMENT      *pElements;
    ULONG                       cReplacements;
    const SPPHRASEREPLACEMENT  *pReplacements;
    GUID                        SREngineID;
    ULONG                       ulSREnginePrivateDataSize;
    const BYTE                 *pSREnginePrivateData;
} SPPHRASE;

Members

cbSize
The size of this structure in bytes.
LangID
The language ID of the phrase elements.
wReserved
Reserved for future use.
ullGrammarID

ID of the grammar that contains the top-level rule used to recognize this phrase.

ftStartTime

Absolute time for start of phrase audio as a 64-bit value representing the number of 100-nanosecond intervals since January 1, 1601.

ullAudioStreamPosition

Start time in the audio stream for this phrase.

ulAudioSizeBytes

Size of audio data in bytes for this phrase.

ulRetainedSizeBytes

Size in bytes of the retained audio data (in the user specified format).

ulAudioSizeTime

Length of phrase audio in 100-nanosecond units.

Rule

Information about the top-level rule that was used to recognize this phrase.

pProperties

Pointer to the root of the semantic property tree.

pElements

Pointer to the array of phrase elements (the number of elements is contained in Rule).

cReplacements

Number of text replacements.

pReplacements

Pointer to the array of text replacements.

SREngineID

GUID that identifies the particular SR engine that recognized this phrase.

ulSREnginePrivateDataSize

Size of the engine's private data (in bytes).

pSREnginePrivateData

Pointer to the engine's private data.


SPPHRASEALT

typedef struct tagSPPHRASEALT
{
   ISpPhraseBuilder   *pPhrase;
   ULONG               ulStartElementInParent;
   ULONG               cElementsInParent;
   ULONG               cElementsInAlternate;
   void               *pvAltExtra;
   ULONG               cbAltExtra;
} SPPHRASEALT;

Members

pPhrase
The alternate phrase.
ulStartElementInParent
The starting element for the change in the parent for the alternate phrase. For example, if the parent phrase is "a b c", and the alternate is "a b d", the starting
cElementsInParent
The number of elements in the original recognition.
cElementsInAlternate
The number of elements that are changed in the alternate.
pvAltExtra
A pointer to the private SR engine alternate analyzer data. Can be NULL if no data is provided.
cbAltExtra
The size of the private SR engine alternate analyzer data. Can be 0 if no data is provided.


SPPHRASEALTREQUEST

typedef struct tagSPPHRASEALTREQUEST
{
    ULONG            ulStartElement;
    ULONG            cElements;
    ULONG            ulRequestAltCount;
    void            *pvResultExtra;
    ULONG            cbResultExtra;
    ISpPhrase       *pPhrase;
    ISpRecoContext  *pRecoContext;
} SPPHRASEALTREQUEST;

Members

ulStartElement
Based on the original phrase, the starting element of the span for which to retrieve alternates.
cElements
Based on the original phrase, the number of elements in the span for which to retrieve alternates.
ulRequestAltCount
The maximum number of alternates that an application wants to retrieve.
pvResultExtra
Pointer to the private SR engine data as sent from the SR engine. Can be NULL of no data was supplied.
cbResultExtra
The size of the private SR engine data as sent from the SR engine. Can be 0 of no data was supplied.
pPhrase
The original recognition as sent from the SR engine.
pRecoContext
Pointer to an ISpRecoContext interface that allows the alternate analyzer to communicate with itself and the SR engine.


SPPHRASEELEMENT

SPPHRASEELEMENT contains the information for a spoken word.

typedef [restricted] struct SPPHRASEELEMENT
{
    ULONG            ulAudioTimeOffset;
    ULONG            ulAudioSizeTime;
    ULONG            ulAudioStreamOffset;
    ULONG            ulAudioSizeBytes;
    ULONG            ulRetainedStreamOffset;
    ULONG            ulRetainedSizeBytes;
    const WCHAR     *pszDisplayText;
    const WCHAR     *pszLexicalForm;
    const SPPHONEID *pszPronunciation;
    BYTE             bDisplayAttributes;
    char             RequiredConfidence;
    char             ActualConfidence;
    BYTE             Reserved;
    float            SREngineConfidence;
} SPPHRASEELEMENT;

Members

ulAudioTimeOffset
This is the starting offset of the element in 100-nanosecond units of time relative to the start of the phrase.
ulAudioSizeTime
This is the length of the element in 100-nanosecond units of time.
ulAudioStreamOffset
This is the starting offset of the element in bytes relative to the start of the phrase in the original input stream.
ulAudioSizeBytes
This is the size of the element in bytes in the original input stream.
ulRetainedStreamOffset
This is the starting offset of the element in bytes relative to the start of the phrase in the retained audio stream
ulRetainedSizeBytes
This is the size of the element in bytes in the retained audio stream.
pszDisplayText
The display text for this elements (e.g., "2" for the spoken word "two").
pszLexicalForm
The lexical form of this element (e.g., "two").
pszPronunciation
The pronunciation for this element as a zero terminated array of SPPHONEID.
bDisplayAttributes
A bit field of SPDISPLAYATTRIBUTES defining extra display information which the application should honor when displaying this word.
RequiredConfidence
The required confidence for this element (either SP_LOW_CONFIDENCE, SP_NORMAL_CONFIDENCE, or SP_HIGH_CONFIDENCE). If a word is prefixed with a '-' (minus), the RequiredConfidence is SP_LOW_CONFIDENCE, and '+' (plus) will set this field to SP_HIGH_CONFIDENCE (e.g., "This -is -a +test").
ActualConfidence
The actual confidence for this element (either SP_LOW_CONFIDENCE, SP_NORMAL_CONFIDENCE, or SP_HIGH_CONFIDENCE). This is always at least the RequiredConfidence.
Reserved
Reserved for future use.
SREngineConfidence
The confidence score computed by the SR engine. The value range is engine-dependent. It can be used to optimize an application's performance with a specific engine. Using this value will improve the application with a particular speech engine but more than likely will make it worse with other engines and should be used with care. This value is of most use with speaker-independent engines as it allows a large corpus of recorded usage to be used to correctly optimize the overall accuracy of the application.


SPPHRASEPROPERTY

SPPHRASEPROPERTY stores the information for one semantic property. It can be used to construct a semantic property tree.

struct SPPHRASEPROPERTY
{
    const   WCHAR              *pszName;
    ULONG                       ulId;
    const   WCHAR              *pszValue;
    VARIANT                     vValue;
    ULONG                       ulFirstElement;
    ULONG                       ulCountOfElements;
    const   SPPHRASEPROPERTY   *pNextSibling;
    const   SPPHRASEPROPERTY   *pFirstChild;
    char                        SREngineConfidence;
    char                        Confidence;
};

Members

pszName
Name of the null-terminated string of the semantic property (in the Speech Text Grammar Format set using the PROPNAME attribute).
ulId
ID of the semantic property (in the Speech Text Grammar Format set using the PROPID attribute).
pszValue
Null-terminated string value of the semantic property (in the Speech Text Grammar Format set using the VALSTR attribute).
vValue
VARIANT value of a semantic property. The type has to be on of the following: VT_BOOL, VT_I4, VT_R4, VT_R8, or VT_BYREF (only for dynamic grammars). This is set using the VAL attribute in the Speech Text Grammar Format.
ulFirstElement
The first spoken element "spanned" by this property.
ulCountOfElements
The number of spoken elements "spanned" by this property.
pNextSibling
Pointer to next sibling in property tree.
pFirstChild
Pointer to the first child of this semantic property.
SREngineConfidence
Confidence value for this semantic property computed by the SR engine. The value range is specific to each SR engine.
Confidence
Confidence value for this semantic property computed by SAPI. The value is either SP_LOW_CONFIDENCE, SP_NORMAL_CONFIDENCE, or SP_HIGH_CONFIDENCE.


SPPHRASEREPLACEMENT

SPPHRASEREPLACEMENT replaces the display text of one or more of the spoken words. This is used by speech recognition engines to perform Inverse Text Normalization (ITN). For example the spoken words "twenty" and "three" are replaced by the replacement text "23".

typedef struct tagSPPHRASEREPLACEMENT
{
    BYTE             bDisplayAttributes;
    const   WCHAR   *pszReplacementText;
    ULONG            ulFirstElement;
    ULONG            ulCountOfElements;
} SPPHRASEREPLACEMENT;

Members

bDisplayAttributes
One or more SPDISPLAYATTRIBUTES for the replacement text.
pszReplacementText
Text for the replacement.
ulFirstElement
Offset of the first spoken element to be replaced.
ulCountOfElements
Number of spoken elements to replace.


SPPHRASERULE

SPPHRASERULE contains the information for a rule in a grammar result. SAPI is using the pFirstChild and pNextSibling pointers to represent the parse tree. SPPHRASE.Rule is the root node of the parse tree.

struct tagSPPHRASERULE
{
    const   WCHAR          *pszName;
    ULONG                   ulId;
    ULONG                   ulFirstElement;
    ULONG                   ulCountOfElements;
    const   SPPHRASERULE   *pNextSibling;
    const   SPPHRASERULE   *pFirstChild;
    float                   SREngineConfidence;
    signed char             Confidence;
 };

Members

pszName
Name of this rule (in Speech Text Grammar Format set using <RULE NAME="MyName">).
ulId
ID of this rule (set using <RULE ID="123">).
ulFirstElement
The index of the first spoken element (word) of this rule.
ulCountOfElements
Number of spoken elements (words) spanned by this rule.
pNextSibling
Pointer to the next sibling in the parse tree.
pFirstChild
Pointer to the first child node in the parse tree.
SREngineConfidence
Confidence for this rule computed by the SR engine. The value is engine-dependent and not standardized across multiple SR engines.
Confidence
Confidence for this rule computed by SAPI. The value is either SP_LOW_CONFIDENCE, SP_NORMAL_CONFIDENCE, or SP_HIGH_CONFIDENCE.


SPPROPERTYINFO

SPPROPERTYINFO contains the information for a semantic property.

typedef struct tagSPPROPERTYINFO
{
    const   WCHAR    *pszName;
    ULONG             ulId;
    const   WCHAR    *pszValue;
    VARIANT           vValue;
} SPPROPERTYINFO;

Members

pszName
Pointer to the null-terminated string that contains the name information of the property. This can be set using the PROPNAME attribute in the Speech Text Grammar Format.
ulId
Identifier associated with the property. This can be set using the PROPID attribute in the Speech Text Grammar Format.
pszValue
Pointer to the null-terminated string that contains the value information of the property. This can be set using the VALSTR attribute in the Speech Text Grammar Format.
vValue
Must be one of the following: VT_BOOL, VT_I4, VT_R4, VT_R8, or VT_BYREF (for dynamic grammars only.) This can be set using the VAL attribute in the Speech Text Grammar Format.


SPRECOCONTEXTSTATUS

typedef [restricted] struct SPRECOCONTEXTSTATUS
{
    SPINTERFERENCE  eInterference;
    WCHAR           szRequestTypeOfUI[255];
    DWORD           dwReserved1;
    DWORD           dwReserved2;
} SPRECOCONTEXTSTATUS;

Members

eInterference
One of the interference types contained in the SPINTERFERENCE enumeration. An application can check this value to get a "general" idea of the input audio signal quality.
szRequestTypeOfUI[255]
Specifies the type of UI requested. If the first byte is NULL, no UI is requested. See also SPDUI_* for a list of the SAPI-defined UI types, and ISpRecognizer::DisplayUI.
dwReserved1
Reserved for future expansion.
dwReserved2
Reserved for future expansion.


SPRECOGNIZERSTATUS

typedef [restricted] struct SPRECOGNIZERSTATUS
{
    SPAUDIOSTATUS   AudioStatus;
    ULONGLONG       ullRecognitionStreamPos;
    ULONG           ulStreamNumber;
    ULONG           ulNumActive;
    CLSID           clsidEngine;
    ULONG           cLangIDs;
    LANGID          aLangID[ SP_MAX_LANGIDS ];
    DWORD           dwReserved1;
    DWORD           dwReserved2;
} SPRECOGNIZERSTATUS;

Members

AudioStatus
The SPAUDIOSTATUS structure containing the current audio device information.
ullRecognitionStreamPos
The stream position the engine has currently recognized up to. Stream positions are measured in bytes. This value can be used to check the engine's progress using the audio data.
ulStreamNumber
This value is incremented every time SAPI starts or stops recognition on an engine (see SPEI_START_SR_STREAM and SPEI_END_SR_STREAM). Each time this happens the ullRecognitionStreamPos gets reset to zero. Events fired from the engine have equivalent stream number and position information also.
ulNumActive
The current engine's number of active rules.
clsidEngine
The unique identifier associated with the current engine.
cLangIDs
The number of languages the current engine supports.
aLangID
Array containing the languages that the current engine supports.
dwReserved1
Reserved for future expansion.
dwReserved2
Reserved for future expansion.


SPRECORESULTINFO

SPRECORESULTINFO is the result structure passed from the engine to SAPI.

typedef struct SPRECORESULTINFO
{
    ULONG              cbSize;
    SPRESULTTYPE       eResultType;
    BOOL               fHypothesis;
    BOOL               fProprietaryAutoPause;
    ULONGLONG          ullStreamPosStart;
    ULONGLONG          ullStreamPosEnd;
    SPGRAMMARHANDLE    hGrammar;
    ULONG              ulSizeEngineData;
    void              *pvEngineData;
    ISpPhraseBuilder  *pPhrase;
} SPRECORESULTINFO;

Members

cbSize
Total size, in bytes, of this structure.
eResultType
Type of result object (CFG, SLM, or Proprietary).
fHypothesis
If TRUE then this recognition is a hypothesis.
fProprietaryAutoPause
This field is only used for SPERT_PROPRITARY grammars. If TRUE, the recognition will pause.
ullStreamPosStart
Starting position within the input stream.
ullStreamPosEnd
Ending position within the input stream.
hGrammar
Required for SPERT_SLM and SPERT_PROPRIETARY, otherwise this value is NULL
ulSizeEngineData
Specifies the size of pvEngineData.
pvEngineData
Pointer to the engine data.
pPhrase
Pointer to phrase object


SPRECORESULTTIMES

SPRECORESULTTIMES contains the time information for speech recognition. This data structure is used by the ISpRecoResult::GetResultTimes method.

typedef struct SPRECORESULTTIMES
{
    FILETIME    ftStreamTime
    ULONGLONG   ullLength
    DWORD       dwTickCount;
    ULONGLONG   ullStart;
} SPRECORESULTTIMES;

Members

ftStreamTime
Number of 100 nanosecond units in UTC time from January 1, 1601 to the start of the current result. This is the same as calling the Win32 GetSystemTimeAsFileTime() function for the result.
ullLength
Value containing the length of the phrase specified in 100 nanosecond units.
dwTickCount
Number of 100 nanosecond units elasped from the start of the system to the start of the current result.
ulStart
Value containing the total 100 nanosecond units from the start of the stream to the start of the phrase.


SPRULEENTRY

SPRULEENTRY contains information about a rule.

typedef [restricted] struct SPRULEENTRY
{
    SPRULEHANDLE    hRule;
    SPSTATEHANDLE   hInitialState;
    DWORD           Attributes;
    void           *pvClientContext;
    void           *pvClientGrammarContext;
} SPRULEENTRY;

Members

hRule
Handle to the rule.
hInitialState
Handle to the rule's intial state.
Attributes
The rule's attribute.
pvClientContext
The client context pointer passed using ISpSREngineSite::SetRuleClientContext
pvClientGrammarContext
The client's grammar context pointer.


SPSERIALIZEDEVENT

The SPSERIALIZEDEVENT structure is used when an application wants to serialize an SAPI event.

See SPSERIALIZEDEVENT64 for a 64-bit version of the serialized event structure.

See also SPEVENT for further information on the related event variable fields.

typedef [restricted] struct SPSERIALIZEDEVENT
{
    WORD        eEventId;
    WORD        elParamType;
    ULONG       ulStreamNum;
    ULONGLONG   ullAudioStreamOffset;
    ULONG       SerializedwParam;
    LONG        SerializedlParam;
} SPSERIALIZEDEVENT;

Members

eEventId
One of the event identifiers from the SPEVENTENUM enumeration.
elParamType
One of the event parameter types from the SPEVENTLPARAMTYPE enumeration.
ulStreamNum
The input stream number associated with this event.
ullAudioStreamOffset
The position in the audio stream where the event occurred.
SerializedwParam
The wParam value that was included in the event.
SerializedlParam
The lParam value that was included in the event.


SPSERIALIZEDEVENT64

The SPSERIALIZEDEVENT64 structure is used when an application wants to serialize a SAPI event.

The 64 label signifies the use of 64-bit values for the wParam and lParam variables. Normally applications will not need to use 64-bit values, unless large pointers or sizes were stored in the original event structure. See also SPSERIALIZEDEVENT for the non-64-bit version.

See also SPEVENT for further information on the related event variable fields.

typedef [restricted] struct SPSERIALIZEDEVENT64
{
    WORD            eEventId;
    WORD            elParamType;
    ULONG           ulStreamNum;
    ULONGLONG       ullAudioStreamOffset;
    ULONGLONG       SerializedwParam;
    LONGLONG        SerializedlParam;
} SPSERIALIZEDEVENT64;

Members

eEventId
One of the event identifiers from the SPEVENTENUM enumeration.
elParamType
One of the event parameter types from the SPEVENTLPARAMTYPE enumeration.
ulStreamNum
The input stream number associated with this event.
ullAudioStreamOffset
The position in the audio stream where the event occurred.
SerializedwParam
The wParam value that was included in the event. Note that the variable uses a 64-bit data type for large addresses and values. See SPEVENTENUM for further information on possible wParam values.
SerializedlParam
The lParam value that was included in the event. Note that the variable uses a 64-bit data type for large addresses and values. See SPEVENTENUM for further information on possible lParam values.


SPSERIALIZEDPHRASE

typedef struct tagSPSERIALIZEDPHRASE
{
    ULONG      ulSerializedSize;
} SPSERIALIZEDPHRASE;

Members

ulSerializedSize
Value specifying the size of the serialized phrase in bytes.


SPSERIALIZEDRESULT

SPSERIALIZEDRESULT contains the phrase size information.

typedef struct SPSERIALIZEDRESULT
{
    ULONG    ulSerializedSize;
} SPSERIALIZEDRESULT;

Members

ulSerializedSize
The size of the entire phrase in bytes, including this ULONG.


SPSTATEINFO

typedef [restricted] struct SPSTATEINFO
{
    ULONG               cAllocatedEntries;
    SPTRANSITIONENTRY  *pTransitions;
    ULONG               cEpsilons;
    ULONG               cRules;
    ULONG               cWords;
    ULONG               cSpecialTransitions;
} SPSTATEINFO;

Members

cAllocatedEntries
Total number of entries in pTransitions
pTransitions
Pointer to a SPTRANSITIONENTRY structure.
cEpsilons
Number of SPTRANSEPSILON transitions in pTransitions.
cRules
Number of SPTRANSRULE transitions in pTransitions.
cWords
Number of SPTRANSWORD transitions in pTransitions.
cSpecialTransitions
Number of special transitions (SPTRANSTEXTBUF, SPTRANSWILDCARD, or SPTRANSDICTATION) in pTransitions.


SPTEXTSELECTIONINFO

This structure combines test selection information that the SR engine can use to improve the recognition accuracy (using ISpRecoGrammar::SetWordSequenceData() and ISpRecoGrammar::SetTextSelection()).

typedef struct tagSPTEXTSELECTIONINFO
{
    ULONG       ulStartActiveOffset;
    ULONG       cchActiveChars;
    ULONG       ulStartSelection;
    ULONG       cchSelection;
} SPTEXTSELECTIONINFO;

Members

ulStartActiveOffset
Count of characters from the start of the WordSequenceData buffer. The word containing the character pointed to is the first word of the active text selection buffer.
cchActiveChars
Count of characters for the active range of the text selection buffer.
ulStartSelection
Start of the selected text (e.g., the user is selecting part of the previously dictated text that he/she is going to edit or correct).
cchSelection
Count of characters of the user selection.


SPTMTHREADINFO

SPTMTHREADINFO contains thread management information implemented by the ISpTaskManager interface.

typedef struct SPTMTHREADINFO
{
    long    lPoolSize;
    long    lPriority;
    ULONG   ulConcurrencyLimit;
    ULONG   ulMaxQuickAllocThreads;
} SPTMTHREADINFO;

Members

lPoolSize
Number of threads in pool (-1 default)
lPriority
Priority of threads in pool
ulConcurrencyLimit
Number of threads allowed to concurrently execute (0 default)
ulMaxQuickAllocThreads
Maximum number of dedicated threads retained


SPTRANSITIONENTRY

typedef [restricted] struct SPTRANSITIONENTRY
{
    SPTRANSITIONID      ID;
    SPSTATEHANDLE       hNextState;
    BYTE                Type;       // SPTRANSITIONTYPE
    char                RequiredConfidence;
    struct
    {
    DWORD               fHasProperty;
    };
    float               Weight;
    union
    {
    struct
        {
    SPSTATEHANDLE   hRuleInitialState;  // Only if Type == SPTRANSRULE
    SPRULEHANDLE    hRule;
    void           *pvClientRuleContext;
        };
    struct
        {
    SPWORDHANDLE    hWord;              // Only if Type == SPTRANSWORD
    void           *pvClientWordContext;
        };
    struct
        {
    void           *pvGrammarCookie;    // Only if Type == SPTRANSTEXTBUF or SPTRANSWILDCARD
        };
    };
} SPTRANSITIONENTRY;

Members

ID
ID of this transition.
hNextState
Handle to the end state of this transition
Type
Type of this transition.
RequiredConfidence
Required confidence for this transition.
fHasProperty
Flag to indicate if this transition has a semantic property associated with it.
Weight
The relative weight of this transition (relative to other transitions out of the same state).
hRuleInitialState
If this is a SPRULETRANS, it points to the rule's initial state.
hRule
If this is a SPRULETRANS, contains the rule's handle.
pvClientRuleContext
Client context set using ISpSREngienSite::SetRuleClientContext.htm
hWord
If this is a SPWORDTRANS, it contains the word handle.
pvClientWordContext
Client context set using ISpSREngienSite::SetWordClientContext.htm
pvGrammarCookie
Grammar cookie needed to associate a text buffer with this transition.


SPTRANSITIONPROPERTY

SPTRANSITIONPROPERTY contains transition property information.

typedef [restricted] struct  SPTRANSITIONPROPERTY
{
    const WCHAR   *pszName;
    ULONG          ulId;
    const WCHAR   *pszValue;
    VARIANT        vValue;
} SPTRANSITIONPROPERTY;

Members

pszName
Address of a null-terminated string containing the name information.
ulId
Identifier associated with the transition property.
pszValue
Address of a null-terminated string containing the value information.
vValue
For dynamic grammars this value will be VT_BOOL, VT_I4, VT_R4, VT_R8, or VT_BYREF.


SPVCONTEXT

SPVCONTEXT contains strings passed into an ISpVoice::Speak call via a <Context> XML tag. See the XML Schema : SAPI white paper for more details.

typedef [restricted] struct SPVCONTEXT
{
    LPCWSTR   pCategory;
    LPCWSTR   pBefore;
    LPCWSTR   pAfter;
} SPVCONTEXT;

Members

pCategory
String passed in with the ID attribute.
pBefore
String passed in with the Before attribute.
pAfter
String passed in with the After attribute.


SPVOICESTATUS

SPVOICESTATUS contains voice status information. This structure is returned by ISpVoice::GetStatus.

typedef struct SPVOICESTATUS
{
    ULONG       ulCurrentStream;
    ULONG       ulLastStreamQueued;
    HRESULT     hrLastResult;
    DWORD       dwRunningState;
    ULONG       ulInputWordPos;
    ULONG       ulInputWordLen;
    ULONG       ulInputSentPos;
    ULONG       ulInputSentLen;
    LONG        lBookmarkId;
    SPPHONEID   PhonemeId;
    SPVISEMES   VisemeId;
    DWORD       dwReserved1;
    DWORD       dwReserved2;
} SPVOICESTATUS;

Members

ulCurrentStream
The number of the current stream being synthesized or receiving output (see ISpVoice::Speak for more information on stream numbers).
ulLastStreamQueued
The number of the last stream queued.
hrLastResult
Result of the last Speak or Speakstream call.
dwRunningState
This value indicates the status of the voice - whether it is currently speaking, is done with all pending speak requests, or is currently waiting to speak. The possible flag values are contained in the SPRUNSTATE enumeration. A value of 0 indicates that the voice is currently waiting to speak.
ulInputWordPos
Character position within the input text of the word currently being processed.
ulInputWordLen
Length of the word currently being processed.
ulInputSentPos
Character position within the input text of the sentence currently being processed.
ulInputSentLen
Length of the sentence currently being processed.
lBookmarkId
Current bookmark string (in base 10) converted to a long integer. If the string of the current bookmark does not begin with an integer, lBookmarkId will be zero. For example, if the bookmark name is "123Bookmark", the lBookmarkId is "123"; and if the bookmark name is "hello", the lBookmarkId is 0.
PhonemeId
Current phoneme ID - see SAPI Phoneme set
VisemeId
Current viseme ID - see SAPI Viseme set
dwReserved1
Reserved for future expansion.
dwReserved2
Reserved for future expansion.


SPVPITCH

SPVPITCH contains a long value passed into an ISpVoice::Speak call via a <Pitch> XML tag. See the XML Schema : SAPI white paper for more details.

typedef struct SPVPITCH
{
    long   MiddleAdj;
    long   RangeAdj;
} SPVPITCH;
MiddleAdj
Value passed in with the Middle or AbsMiddle attributes. Supported values are -10 to 10 with a value of 0 being the default pitch - values outside of this range may be truncated.
RangeAdj
Reserved for future use.


SPVSTATE

SPVSTATE is a member of SPVTEXTFRAG, and contains information about the XML state associated with a text string. A linked list of SPVTEXTFRAGs is passed into every ISpTTSEngine::Speak call after the SpVoice object has parsed the SAPI 5 XML passed into ISpVoice::Speak.

typedef [restricted] struct SPVSTATE
{
    //--- Action
    SPVACTIONS      eAction;
    //--- Running state values
    LANGID          LangID;
    WORD            wReserved;
    long            EmphAdj;
    long            RateAdj;
    ULONG           Volume;
    SPVPITCH        PitchAdj;
    ULONG           SilenceMSecs;
    SPPHONEID*      pPhoneIds;
    SPPARTOFSPEECH  ePartOfSpeech;
    SPVCONTEXT      Context;
} SPVSTATE;

Members

eAction
Describes the action to be performed with the associated text fragment. The normal action is to Speak (SPVA_Speak) the fragment.
LangID
The language ID associated with this text. Set via the <Lang> XML tag.
wReserved
Reserved for future use.
EmphAdj
Specifies whether the text should be emphasized - zero indicates no emphasis, one indicates emphasis. Set via the <Emph> XML tag.
RateAdj
The rate associated with this text. Set via the <Rate> XML tag. This value should be combined with the baseline rate (either the default, or a value set by ISpVoice::SetRate) to yield the final rate value.
Volume
The volume associated with this text. Set via the <Volume> XML tag. This value should be combined with the baseline volume (either the default, or a value set by ISpVoice::SetVolume) to yield the final volume value.
PitchAdj
The pitch associated with this text. Set via the <Pitch> XML tag.
SilenceMSecs
The length of a silence, in milliseconds, to be inserted into the audio output. SilenceMSecs is always 0 unless eAction is SPVA_Silence. Set via the <Silence> XML tag.
pPhoneIds
A pronunciation (possibly associated with text) to be inserted into the audio output. This value is a pointer to a null-terminated array of SPPHONEIDs. Set via the <Pron> XML tag.
ePartOfSpeech
An SPPARTOFSPEECH value to be associated with this text. Set via the <PartOfSp> XML tag.
Context
An SPVCONTEXT to be associated with this text. Set via the <Context> XML tag.


SPVTEXTFRAG

SPVTEXTFRAG is the structure through which input text is passed to the TTS engine. The text passed to an ISpVoice::Speak call will normally be parsed for SAPI 5 XML, and the resulting list of SPVTEXTFRAGs will be passed to an ISpTTSEngine::Speak call.

typedef struct SPVTEXTFRAG
{
    struct SPVTEXTFRAG   *pNext;
    SPVSTATE              State;
    LPCWSTR               pTextStart;
    ULONG                 ulTextLen;
    ULONG                 ulTextSrcOffset;
} SPVTEXTFRAG;

Members

pNext
Pointer to the next text fragment in list. A NULL value indicates the end of the list.
State
The current XML attribute state.
pTextStart
Pointer to the beginning of the text string associated with this fragment.
ulTextLen
The length, in characters, of the text string associated with this fragment.
ulTextSrcOffset
Original character offset of pTextStart within the text string passed to ISpVoice::Speak.


SPWORD

SPWORD is used with ISpLexicon to define changes to the words in a lexicon. It is used in connection with SPWORDLIST which in turn is used by ISpLexicon::GetWords and ISpLexicon::GetGenerationChange.

typedef [restricted] struct SPWORD
{
    struct SPWORD          *pNextWord;
    LANGID                  LangID;
    WORD                    wReserved;
    SPWORDTYPE              eWordType;
    WCHAR                  *pszWord;
    SPWORDPRONUNCIATION    *pFirstWordPronunciation;
} SPWORD;

Members

pNextWord
Pointer to the next word in the list.
LangID
The language ID of the word.
wReserved
Reserved for future use.
eWordType
Flag of type SPWORDTYPE indicating whether the word has been added or deleted.
pszWord
The offset of the word entry.
pFirstWordPronunciation
Pointer to the first possible pronunciation of the word.


SPWORDENTRY

typedef [restricted] struct SPWORDENTRY
{
    SPWORDHANDLE    hWord;
    LANGID          LangID;
    const WCHAR    *pszDisplayText;
    const WCHAR    *pszLexicalForm;
    SPPHONEID      *aPhoneId;
    void           *pvClientContext;
} SPWORDENTRY;

Members

hWord
Handle to the current word.
LangID
Language identifier.
pszDisplayText
Pointer to a null-terminated string containing the display text information.
pszLexicalForm
Pointer to a null-terminated string containing the lexical text information.
aPhoneId
Pointer to a NULL-terminated array containing the phoneme identifier.
pvClientContext
Pointer to a string representing the client context data.


SPWORDLIST

SPWORDLIST is used with ISpLexicon to receive words currently in the lexicon or that have changed since the last time an engine checked. This structure is the beginning of a linked list of SPWORD structures and contains the size and actual buffer of all subsequent word operations. It is used with ISpLexicon::GetWords and ISpLexicon::GetGenerationChange.

typedef struct SPWORDLIST
{
    ULONG      ulSize;
    BYTE      *pvBuffer;
    SPWORD    *pFirstWord;
} SPWORDLIST;

Call ZeroMemory before using SPWORDLIST to initialize it, and call CoTaskMemFree(spWordList.pvBuffer) to free the buffer allocated during the calls. The pvBuffer need not (and should not) be freed between the calls. ISpLexicon::GetWords and ISpLexicon::GetGenerationChange will reuse the buffer for efficiency and reallocate when necessary.

Members

ulSize
The size of the buffer for all of the words, in bytes.
pvBuffer
Pointer to the buffer for all word information or changes.
pFirstWord
Pointer to the first word in the list.

Examples

The following example is a code fragment demonstrating the use and creation of SPWORDLIST. The code initializes the structure prior to use. Note that the returned SPWORDLIST has a CoTaskMemAllocate buffer attached to it. This should be freed after all operations using the list. If not all words are returned from ISpLexicon::GetWords, the structure should not be wiped and can be passed into subsequent calls to allow efficient reuse of the allocated memory block.

SPWORDLIST SPWordList;
hr = ZeroMemory(&SPWordList, sizeof(SPWordList));
// Check return value here. Handle error.
hr = S_FALSE;
while (hr == S_FALSE)
{
   hr = pLex->GetWords(eLEXTYPE_USER, &dwGen, &dwCookie, &SPWordList);
   // Do something with the received words.
   // S_FALSE is returned if there are still words remaining to be got and the cookie is updated.
}
// Have finished with the list. Free the enclosed buffer.
::CoTaskMemFree(SPWordList.pvBuffer);

The following helper class will ensure the correct usage of SPWORDLIST.

class CSpWordList : public SPWORDLIST
{
public:
    CSpWordList()
    {
        ZeroMemory(static_cast<SPWORDLIST*>(this), sizeof(SPWORDLIST));
    }
    ~CSpWordList()
    {
        CoTaskMemFree(pvBuffer);
    }
};

Using the helper class, the above sample becomes:

CSpWordList SPWordList;
hr = S_FALSE;
while (hr == S_FALSE)
{
   hr = pLex->GetWords(eLEXTYPE_USER, &dwGen, &dwCookie, &SPWordList);
   // Do something with the received words.
   // S_FALSE is returned if there are still words remaining to be got and the cookie is updated.
}


SPWORDPRONUNCIATION

SPWORDPRONUNCIATION is used by ISpLexicon for words with possible variations in pronunciation. SPWORDPRONUNCIATION contains a single pronunciation for a word.

typedef [restricted] struct SPWORDPRONUNCIATION
{
    struct SPWORDPRONUNCIATION   *pNextWordPronunciation;
    SPLEXICONTYPE                 eLexiconType;
    LANGID                        LangID;
    WORD                          wReserved;
    SPPARTOFSPEECH                ePartOfSpeech;
    SPPHONEID                     szPronunciation[1];
} SPWORDPRONUNCIATION;

Members

pNextWordPronunciation
Pointer to the next possible pronouncation. May be NULL.
eLexiconType
Flag of type SPLEXICONTYPE indicating what lexicon this pronunciation and part of speech was obtained from.
LangID
The language identifier.
wReserved
Reserved for future use.
ePartOfSpeech
The part of speech used by this particular variation.
szPronunciation[1]
This is a zero terminated array of SPPHONID elements defining the pronunciation. It runs off the end of the SPWORDPRONUNCIATION structure and is part of data buffer in the containing SPWORDPRONUNCIATIONLIST structure.


SPWORDPRONUNCIATIONLIST

SPWORDPRONUNCIATIONLIST is used with ISpLexicon::GetPronunciations to list possible variations in pronunciation for a given word. This structure is the start of a linked list of SPWORDPRONUNCIATION structures and contains the size and actual buffer of all subsequent pronunciations.

typedef struct SPWORDPRONUNCIATIONLIST
{
    ULONG                   ulSize;
    BYTE                   *pvBuffer;
    SPWORDPRONUNCIATION    *pFirstWordPronunciation;
} SPWORDPRONUNCIATIONLIST;

Call ZeroMemory before using SPWORDPRONUNCIATIONLIST to initialize it, and call CoTaskMemFree(spwordpronlist.pvBuffer) to free the buffer allocated during the calls. The pvBuffer need not (and should not) be freed between the calls. ISpLexicon::GetPronunciations will reuse the buffer for efficiency and reallocate when necessary.

Members

ulSize
Size of the pronunciation buffer, in bytes.
pvBuffer
Pointer to the buffer for all pronunciations.
pFirstWordPronunciation
Pointer to the first in a linked list of SPWORDPRONUNCIATION structures within pvBuffer.

Example

The following example is a code fragment demonstrating the use and creation of SPWORDPRONUNCIATIONLIST.

    SPWORDPRONUNCIATIONLIST spwordpronlist;
    memset(spwordpronlist, 0, sizeof(spwordpronlist));

    pISpLexicon->GetPronunciations(L"resume", 0, 0, &spwordpronlist);
    for (SPWORDPRONUNCIATION *pwordpron = spwordpronlist.pFirstWordPronunciation;
         wordpron != NULL;
         wordpron = pwordpron->pNextWordPronunciation)
    {
        DoSomethingWith(pwordpron->ePartOfSpeech, pwordpron->szPronunciation);
    }

    pISpLexicon->GetPronunciations(L"record", 0, 0, &spwordpronlist);
    // repeat the for loop above to process the pronunciations

    CoTaskMemFree(spwordpronlist.pvBuffer);

The following helper class will ensure the correct usage of SPWORDPRONUNCIATIONLIST.

class CSpPronList : public SPWORDPRONUNCIATIONLIST
{
public:
    CSpPronList()
    {
        ZeroMemory(static_cast<SPWORDPRONUNCIATIONLIST*>(this), sizeof(SPWORDPRONUNCIATIONLIST));
    }
    ~CSpPronList()
    {
        CoTaskMemFree(pvBuffer);
    }
};

Using the helper class, the above sample becomes:

    CSpPronList spwordpronlist;

    pISpLexicon->GetPronunciations(L"resume", 0, 0, &spwordpronlist);
    for (SPWORDPRONUNCIATION *pwordpron = spwordpronlist.pFirstWordPronunciation;
         wordpron != NULL;
         wordpron = pwordpron->pNextWordPronunciation)
    {
        DoSomethingWith(pwordpron->ePartOfSpeech, pwordpron->szPronunciation);
    }

    pISpLexicon->GetPronunciations(L"record", 0, 0, &spwordpronlist);
    // repeat the for loop above to process the pronunciations


WAVEFORMATEX

WAVEFORMATEX defines the format of waveform-audio data. Only format information common to all waveform-audio data formats is included in this structure. For formats requiring additional information, this structure is included as the first member in another structure, along with the additional information.

typedef [restricted] struct WAVEFORMATEX
{
    WORD    wFormatTag;
    WORD    nChannels;
    DWORD   nSamplesPerSec;
    DWORD   nAvgBytesPerSec;
    WORD    nBlockAlign;
    WORD    wBitsPerSample;
    WORD    cbSize;
} WAVEFORMATEX;

Members

wFormatTag
Waveform-audio format type. Format tags are registered with Microsoft Corporation for many compression algorithms. A complete list of format tags is located in the Mmsystem.h header file.
nChannels
Number of channels in the waveform-audio data. Monaural data uses one channel and stereo data uses two channels.
nSamplesPerSec
Sample rate, in samples per second (hertz), that each channel should be played or recorded. If wFormatTag is WAVE_FORMAT_PCM, common values for nSamplesPerSec are 8.0 kHz, 11.025 kHz, 22.05 kHz, and 44.1 kHz. For non-PCM formats, this member must be computed according to the manufacturer's specification of the format tag.
nAvgBytesPerSec
Required average data-transfer rate, in bytes per second, for the format tag. If wFormatTag is WAVE_FORMAT_PCM, nAvgBytesPerSec should be equal to the product of nSamplesPerSec and nBlockAlign. For non-PCM formats, this member must be computed according to the manufacturer's specification of the format tag.

Playback and record software can estimate buffer sizes by using the nAvgBytesPerSec member.

nBlockAlign
Block alignment, in bytes. The block alignment is the minimum atomic unit of data for the wFormatTag format type. If wFormatTag is WAVE_FORMAT_PCM, nBlockAlign should be equal to the product of nChannels and wBitsPerSample divided by 8 (bits per byte). For non-PCM formats, this member must be computed according to the manufacturer's specification of the format tag.

Playback and record software must process a multiple of nBlockAlign bytes of data at a time. Data written and read from a device must always start at the beginning of a block. For example, it is illegal to start playback of PCM data in the middle of a sample (that is, on a non-block-aligned boundary).

wBitsPerSample
Bits per sample for the wFormatTag format type. If wFormatTag is WAVE_FORMAT_PCM, wBitsPerSample should be equal to 8 or 16. For non-PCM formats, this member must be set according to the manufacturer's specification of the format tag. Note that some compression schemes cannot define a value for wBitsPerSample, so this member can be zero.
cbSize
Size, in bytes, of extra format information appended to the end of the WAVEFORMATEX structure. This information can be used by non-PCM formats to store extra attributes for the wFormatTag. If no extra information is required by the wFormatTag, this member must be set to zero. For WAVE_FORMAT_PCM formats only, this member is ignored.


Enumerations

The following enumerations are used with SAPI 5.

  • SPAUDIOOPTIONS
  • SPAUDIOSTATE
  • SPBOOKMARKOPTIONS
  • SPCFGNOTIFY
  • SPCFGRULEATTRIBUTES
  • SPCONTEXTSTATE
  • SPDATAKEYLOCATION
  • SPDISPLAYATTRIBUTES
  • SPEAKFLAGS
  • SPENDSRSTREAMFLAGS
  • SPEVENTENUM
  • SPEVENTLPARAMTYPE
  • SPFILEMODE
  • SPGRAMMARSTATE
  • SPGRAMMARWORDTYPE
  • SPINTERFERENCE
  • SPLEXICONTYPE
  • SPLOADOPTIONS
  • SPPARTOFSPEECH
  • SPPHRASERNG
  • SPPROPSRC
  • SPRECOEVENTFLAGS
  • SPRECOSTATE
  • SPRESULTTYPE
  • SPRULEINFOOPT
  • SPRULESTATE
  • SPRUNSTATE
  • SPSTREAMFORMAT
  • SPTRANSITIONTYPE
  • SPVACTIONS
  • SPVALUETYPE
  • SPVESACTIONS
  • SPVFEATURE
  • SPVISEMES
  • SPVLIMITS
  • SPVPRIORITY
  • SPVSKIPTYPE
  • SPWAVEFORMATTYPE
  • SPWORDINFOOPT
  • SPWORDPRONOUNCEABLE
  • SPWORDTYPE


SPAUDIOOPTIONS

SPAUDIOOPTIONS indicates the options for an audio stream.

See also the ISpRecoContext::SetAudioOptions and ISpRecoContext::GetAudioOptions methods.

typedef enum SPAUDIOOPTIONS
{
    SPAO_NONE
    SPAO_RETAIN_AUDIO
} SPAUDIOOPTIONS;

Elements

SPAO_NONE
Flag indicating no options for the audio stream should be used.
SPAO_RETAIN_AUDIO
Flag indicating the audio stream should be retained (e.g. serialization of recognition object, playback of recognized audio, etc.). See also ISpRecoResult::SpeakAudio and ISpRecoResult::GetAudio.


SPAUDIOSTATE

SPAUDIOSTATE sets the audio input or output state to one of four possible states.

Used directly by the ISpAudio::SetState method and as a member of the SPAUDIOSTATUS structure.

typedef enum _SPAUDIOSTATE
{
    SPAS_CLOSED,
    SPAS_STOP,
    SPAS_PAUSE,
    SPAS_RUN
} SPAUDIOSTATE;

Elements

SPAS_CLOSED
Audio is stopped and closed. For multimedia audio input devices (sound cards etc.), the device will be released allowing it to be opened by other processes and potentially made unavailable to SAPI.
SPAS_STOP
Audio is stopped. For multimedia audio input devices (sound cards etc.), the audio device will not be closed guaranteeing it can be restarted by SAPI without an intervening process opening it.
SPAS_PAUSE
Audio is paused. Staying in this state for too long a period will cause audio loss.
SPAS_RUN
Audio is enabled.


SPBOOKMARKOPTIONS

typedef enum SPBOOKMARKOPTIONS
{
    SPBO_NONE
    SPBO_PAUSE
} SPBOOKMARKOPTIONS;

Members

SPBO_NONE
The recognition context will not be paused when the associated bookmark event occurs.
SPBO_PAUSE
The recognition context will be paused when the associated bookmark event occurs. See also ISpRecoContext::Pause.


SPCFGNOTIFY

SPCFGNOTIFY contains event notification information related to the addition, deletion, invalidation, activation, or deactivation of words and rules in the loaded grammars.

typedef enum SPCFGNOTIFY
{
    SPCFGN_ADD,
    SPCFGN_REMOVE,
    SPCFGN_INVALIDATE,
    SPCFGN_ACTIVATE,
    SPCFGN_DEACTIVATE
} SPCFGNOTIFY;

Elements

SPCFGN_ADD
Flag indicating that the grammar rule should be added.
SPCFGN_REMOVE
Flag indicating that the grammar rule should be removed.
SPCFGN_INVALIDATE
Flag indicating that the grammar rule should be invalidated.
SPCFGN_ACTIVATE
Flag indicating that the grammar rule should be activated.
SPCFGN_DEACTIVATE
Flag indicating that the grammar rule should be deactivated.


SPCFGRULEATTRIBUTES

SPCFGRULEATTRIBUTES contains the attribute information of grammar rules.

typedef enum SPCFGRULEATTRIBUTES
{
    SPRAF_TopLevel
    SPRAF_Active
    SPRAF_Export
    SPRAF_Import
    SPRAF_Interpreter
    SPRAF_Dynamic
    SPRAF_AutoPause
} SPCFGRULEATTRIBUTES;

Elements

SPRAF_TopLevel
Flag specifying that the rule is defined as a TopLevel rule. Toplevel rules are the entry points into the grammar and can be de-/activated programmatically. This can be set using the TOPELVEL attribute in the Speech Text Grammar Format.
SPRAF_Active
Flag specifying that the rule is defined as a toplevel rule that is activated by default. These rules can be de-/activated by calling De-/ActivateRule(NULL, 0, ...). This can be set using the TOPLEVEL="ACTIVE" attribute-value pair in the Speech Text Grammar Format.
SPRAF_Export
Flag specifying that the rule is exported and hence can be referred to by a rule in another grammar. This can be set using the EXPORT="YES" attribute-value pair in the Speech Text Grammar Format.
SPRAF_Import
Flag specifying that the rule is imported from another grammar and is therefore not defined in this grammar.
SPRAF_Interpreter
Flag specifying that the rule has an interpreter (custom C/C++ code implementing the ISpCFGInterpreter interface) associated with it.
SPRAF_Dynamic
Flag specifying that the rule is dynamic (can be changed programmatically through the ISpGrammarBuilder interface). Note that the CFG must be loaded with the SPLO_DYNAMIC flag to enable changes at runtime.
SPRAF_AutoPause
Flag specifying the grammar attributes as AutoPause. This flag is only valid at runtime as part of a rule state and is not valid to pass as part of a rule definition.


SPCONTEXTSTATE

SPCONTEXTSTATE provides controls for setting and restoring recognition states on a per-context basis.

typedef enum SPCONTEXTSTATE
{
    SPCS_DISABLED
    SPCS_ENABLED
}   SPCONTEXTSTATE;

Elements

SPCS_DISABLED
Specifies that grammars associated with this recognition context are disabled.
SPCS_ENABLED
Specifies that grammars associated with this recognition context are enabled.


SPDATAKEYLOCATION

SPDATAKEYLOCATION specifies top-level registry keys. It is used for data key locations with ISpObjectTokenCategory::GetDataKey

typedef enum SPDATAKEYLOCATION
{
    SPDKL_DefaultLocation
    SPDKL_CurrentUser
    SPDKL_LocalMachine
    SPDKL_CurrentConfig
} SPDATAKEYLOCATION;

Elements

SPDKL_DefaultLocation
The default location set by ISpObjectTokenCategory.
SPDKL_CurrentUser
The registry key HKEY_CURRENT_USER.
SPDKL_LocalMachine
The registry key HKEY_LOCAL_MACHINE.
SPDKL_CurrentConfig
The registry key HKEY_CURRENT_CONFIG.


SPDISPLAYATTRIBUTES

SPDISPLAYATTRIBUTES properly formats the display text of phrase elements.

typedef enum tagSPDISPLYATTRIBUTES
{
    SPAF_ONE_TRAILING_SPACE
    SPAF_TWO_TRAILING_SPACES
    SPAF_CONSUME_LEADING_SPACES
    SPAF_ALL
} SPDISPLAYATTRIBUTES;

Elements

SPAF_ONE_TRAILING_SPACE
Inserts one trailing space, used for most words.
SPAF_TWO_TRAILING_SPACES
Insert two trailing spaces, often used after a sentence final '.' .
SPAF_CONSUME_LEADING_SPACES
Consume leading space, often used for '.' . If this is absent, the word should have a leading space by default.
SPAF_ALL
A combination of all of the above flags.


SPEAKFLAGS

SPEAKFLAGS controls the behavior of an ISpVoice::Speak call (and thus also the behavior of an ISpTTSEngine::Speak call).

typedef enum SPEAKFLAGS
{
    //--- SpVoice flags
    SPF_DEFAULT
    SPF_ASYNC
    SPF_PURGEBEFORESPEAK
    SPF_IS_FILENAME
    SPF_IS_XML
    SPF_IS_NOT_XML
    SPF_PERSIST_XML

    //--- Normalizer flags
    SPF_NLP_SPEAK_PUNC

    //--- Masks
    SPF_NLP_MASK
    SPF_VOICE_MASK
    SPF_UNUSED_FLAGS
} SPEAKFLAGS;

Elements

SPF_DEFAULT
Specifies that the default behavior should be used. The default behavior is to speak the given text string synchronously, to not purge pending speak requests, to parse the text as XML only if the first character is a left-angle-bracket (<), to not persist global XML state changes across speak calls, and to not expand punctuation characters into words. To override this default behavior, use the other flag values given below.
SPF_ASYNC
Specifies that the Speak call should be asynchronous - that is, it will return immediately after the speak request is queued.
SPF_PURGEBEFORESPEAK
Purges all pending speak requests prior to this speak call.
SPF_IS_FILENAME
The string passed to ISpVoice::Speak is a file name, and the file text should be spoken.
SPF_IS_XML
The input text will be parsed for XML markup.
SPF_IS_NOT_XML
The input text will not be parsed for XML markup.
SPF_PERSIST_XML
Global state changes in the XML markup will persist across speak calls.
SPF_NLP_SPEAK_PUNC
Punctuation characters should be expanded into words (e.g. "This is a sentence." would become "This is a sentence period").
SPF_NLP_MASK
This mask is used to remove the SAPI handled flags before ISpTTSEngine::Speak is called. The only flag which the TTS engine must handle is SPF_NLP_SPEAK_PUNC.
SPF_VOICE_MASK
This mask has every flag bit set.
SPF_UNUSED_FLAGS
This mask has every unused bit set.


SPENDSRSTREAMFLAGS

SPENDSRSTREAMFLAGS enables an application to query for state changes when the end of an speech recognition (SR) stream is encountered.

typedef enum SPENDSRSTREAMFLAGS
{
    SPESF_NONE
    SPESF_STREAM_RELEASED
}   SPENDSRSTREAMFLAGS;

Elements

SPESF_NONE
No flags are associated with the end of stream event.
SPESF_STREAM_RELEASED
The input stream object was released upon reaching the end of the current stream. For example, a wave file is a finite stream of data, and once the end of the stream, and file, is reached, the stream object is released. See also CSpEvent::InputStreamReleased.


SPEVENTENUM

SPEVENTENUM lists the events possible from SAPI.

It is recommended that developers utilize the helper class CSpEvent to easily and clearly decode events.

typedef enum SPEVENTENUM
{

    SPEI_UNDEFINED

    //--- TTS engine
    SPEI_START_INPUT_STREAM
    SPEI_END_INPUT_STREAM
    SPEI_VOICE_CHANGE
    SPEI_TTS_BOOKMARK
    SPEI_WORD_BOUNDARY
    SPEI_PHONEME
    SPEI_SENTENCE_BOUNDARY
    SPEI_VISEME
    SPEI_TTS_AUDIO_LEVEL

    //--- Engine vendors use these reserved bits
    SPEI_TTS_PRIVATE

    SPEI_MIN_TTS
    SPEI_MAX_TTS

    //--- Speech Recognition
    SPEI_END_SR_STREAM
    SPEI_SOUND_START
    SPEI_SOUND_END
    SPEI_PHRASE_START
    SPEI_RECOGNITION
    SPEI_HYPOTHESIS
    SPEI_SR_BOOKMARK
    SPEI_PROPERTY_NUM_CHANGE
    SPEI_PROPERTY_STRING_CHANGE
    SPEI_FALSE_RECOGNITION
    SPEI_INTERFERENCE
    SPEI_REQUEST_UI
    SPEI_RECO_STATE_CHANGE
    SPEI_ADAPTATION
    SPEI_START_SR_STREAM
    SPEI_RECO_OTHER_CONTEXT
    SPEI_SR_AUDIO_LEVEL

    //--- Engine vendors use these reserved bits
    SPEI_SR_PRIVATE

    SPEI_MIN_SR
    SPEI_MAX_SR

    SPEI_RESERVED1
    SPEI_RESERVED2
    SPEI_RESERVED3
} SPEVENTENUM;

Elements

SPEI_START_INPUT_STREAM
The input stream (text or audio) from a Speak or SpeakStream call has begun synthesizing to the output. The event is fired by SAPI.
SPEI_END_INPUT_STREAM
The input stream (text or audio) from a Speak or SpeakStream call has finished synthesizing to the output. The event is fired by SAPI.
SPEI_VOICE_CHANGE
This event will fire by SAPI for voice changes within a single input stream of a Speak call. wParam is either zero or the SPF_PERSIST_XML. If the current speak call takes SPF_PERSIST_XML, wparam is SPF_PERSIST_XML. Otherwise, 0. lParam is the current voice object token. elParamType has to be SPET_LPARAM_IS_TOKEN.
SPEI_TTS_BOOKMARK
The bookmark element is used to insert a bookmark into the input stream. If an application specifies interest in bookmark events, it will receive the bookmark events during synthesis. wParam is the current bookmark name (in base 10) converted to a long integer. If name of current bookmark is not an integer, wParam will be zero. lParam is the bookmark string. elParamType has to be SPET_LPARAM_IS_STRING.
SPEI_WORD_BOUNDARY
A word is beginning to synthesize. Markup language (XML) markers are counted in the boundaries and offsets. wParam is the character length of the word in the current input stream being synthesized. lParam is the character position within the current text input stream of the word being synthesized.
SPEI_PHONEME
Phoneme was returned by the synthesis engine. The high word of wParam is the duration, in milliseconds, of the current phoneme. The low word is the phone id for the next phoneme. The high word of lparam is the phoneme feature defined in SPVFEATURE. This value will be zero if the current phoneme is not primary stress or emphasis. The low word of lParam is the phone id for the current phoneme being synthesized.
SPEI_SENTENCE_BOUNDARY
A sentence is beginning to synthesize. wParam is the character length of the sentence including punctuation in the current input stream being synthesized. lParam is the character position within the current text input stream of the sentence being synthesized.
SPEI_VISEME
Viseme was determined by synthesis engine. The high word of wParam is the duration, in milliseconds, of the current viseme. The low word is for the next viseme of type SPVISEMES. The high word of lParam is the viseme feature defined in SPVFEATURE. This value will be zero if the current viseme is not primary stress or emphasis. The low word of lParam is the current viseme being synthesized.
SPEI_TTS_AUDIO_LEVEL
This event is fired by SAPI. lParam is 0, and wParam is the current audio level from 0 to 100.
SPEI_TTS_PRIVATE
Reserved for private/internal use by the TTS Engine
SPEI_MIN_TTS
Minimum event enumeration value for TTS events
SPEI_MAX_TTS
Maximum event enumeration value for TTS events
SPEI_END_SR_STREAM
The SR engine has finished receiving an audio input stream. LPARAM points to the SR Engine's final HRESULT code (see CSpEvent::EndStreamResult). WPARAM points to a Boolean value signifying whether the audio input stream object was released (see CSpEvent::InputStreamReleased).
SPEI_SOUND_START
The SR engine determined that audible sound is available through the input stream.
SPEI_SOUND_END
The SR engine has determined that audible sound is no longer available through the input stream, or that the sound stream has been inactive for a period.
SPEI_PHRASE_START
The SR engine is starting to recognize a phrase. Note that this MUST be followed by either an SPEI_FALSE_RECOGNITION or SPEI_RECOGNITION event.
SPEI_RECOGNITION
The SR engine is returning a full recognition - its best guess at a text representation of the audio data. LParam is a pointer to an ISpRecoResult object (see CSpEvent::RecoResult).
SPEI_HYPOTHESIS
The SR engine is returning a partial phrase recognition - effectively its best guess up to that point in the stream. LParam is a pointer to an ISpRecoResult object (see CSpEvent::RecoResult).
SPEI_SR_BOOKMARK
Bookmark was reached during synthesis. lParam is an application specified value set using ISpRecoContext::Bookmark. wParam is SPREF_AutoPause if ISpRecoContext::Bookmark was called with SPBO_PAUSE, and NULL otherwise.
SPEI_PROPERTY_NUM_CHANGE
An SR Engine supported property was changed. LPARAM is a string pointer to the property name that changed (see CSpEvent::PropertyName]. WPARAM contains the new value [see CSpEvent::PropertyNumValue).
SPEI_PROPERTY_STRING_CHANGE
LPARAM is a string pointer to the property name that changed (see CSpEvent::PropertyName). Immediately following the NULL-termination of the property name is the new property value. (see CSpEvent::PropertyStringValue)
SPEI_FALSE_RECOGNITION
Apparent speech with no valid recognition. An SR engine can decide to optionally return a result object, which will be referenced by the LPARAM member (see CSpEvent::RecoResult).
SPEI_INTERFERENCE
The SR engine determined that the sound stream has a hindrance and is preventing a successful recognition. lParam is any combination of SPINTERFERENCE flags (See CSpEvent::Interference).
SPEI_REQUEST_UI
The SR engine's request to display specific user interface. LPARAM is a null-terminated string (see CSpEvent::RequestTypeOfUI).
SPEI_RECO_STATE_CHANGE
The recognizer state has changed. WPARAM is the new recognizer state (see SPRECOSTATE and CSpEvent::RecoState).
SPEI_ADAPTATION
The SR engine is ready to process the adaptation buffer.
SPEI_START_SR_STREAM
The SR engine has reached the start of a new audio stream.
SPEI_SR_AUDIO_LEVEL
This event is fired by the audio input stream object. wParam is the current audio level from 0 to 100.
SPEI_RECO_OTHER_CONTEXT
A recognition was sent to another context
SPEI_SR_PRIVATE
Reserved for private/internal use by the SR Engine
SPEI_MIN_SR
Minimum event enumeration value for Speech Recognition events
SPEI_MAX_SR
Maximum event enumeration value for Speech Recognition events
SPEI_RESERVED1
Reserved for SAPI internal use. See SPFEI Remarks section.
SPEI_RESERVED2
Reserved for SAPI internal use. See SPFEI Remarks section.
SPEI_RESERVED3
Reserved for future use, do not use.


SPEVENTLPARAMTYPE

typedef enum SPEVENTLPARAMTYPE
{
    SPET_LPARAM_IS_UNDEFINED
    SPET_LPARAM_IS_TOKEN,
    SPET_LPARAM_IS_OBJECT,
    SPET_LPARAM_IS_POINTER,
    SPET_LPARAM_IS_STRING,
} SPEVENTLPARAMTYPE;

Elements

SPET_LPARAM_IS_UNDEFINED
The SPEVENT.lParam value represents an undefined value.
For all TTS events, except SPEI_VOICE_CHANGE and SPEI_TTS_BOOKMARK, the event type is SPET_LPARAM_IS_UNDEFINED.
SPET_LPARAM_IS_TOKEN
The SPEVENT.lParam value represents a pointer to an ISpObjectToken object.
For example, the TTS voice change event (i.e., SPEI_VOICE_CHANGE) includes a pointer to the new voice's object token, so the lParam type is SPET_LPARAM_IS_TOKEN.
SPET_LPARAM_IS_OBJECT
The SPEVENT.lParam value represents a pointer to an object.
For example, the speech recognition even (i.e., SPEI_RECOGNITION) includes a pointer to the recognition result (e.g. ISpRecoResult), so the lParam type is SPET_LPARAM_IS_OBJECT.
SPET_LPARAM_IS_POINTER
The SPEVENT.lParam value represents a memory pointer.
SPET_LPARAM_IS_STRING
The SPEVENT.lParam value represents a pointer to a string.
For example, the TTS bookmark event (i.e., SPEI_TTS_BOOKMARK) includes a pointer the bookmark name, so the lParam type is SPET_LPARAM_IS_OBJECT.


SPFILEMODE

SPFILEMODE specifies the file opening state used by ISpStream::BindToFile and the helper function SPBindToFile.

typedef enum SPFILEMODE
{
    SPFM_OPEN_READONLY,
    SPFM_OPEN_READWRITE,
    SPFM_CREATE,
    SPFM_CREATE_ALWAYS,
    SPFM_NUM_MODES
} SPFILEMODE;

Elements

SPFM_OPEN_READONLY
Opens the existing file in read-only mode. This will fail if the file does not exist.
SPFM_OPEN_READWRITE
Opens the existing file in read-write mode. This will fail if the file does not exist.
SPFM_CREATE
Opens the file if one exists, or creates the file if one does not exist. This flag indicates that the file will be opened in read-write mode.
SPFM_CREATE_ALWAYS
Creates the file, even if the file already exists and deletes the previous file. This flag indicates that the file will be opened in read-write mode.
SPFM_NUM_MODES
This flag is used for limit checking.


SPGRAMMARSTATE

SPGRAMMARSTATE provides controls for setting and restoring grammar states.

typedef enum SPGRAMMARSTATE
{
    SPGS_ENABLED = 0,
    SPGS_DISABLED = 1,
    SPGS_EXCLUSIVE = 3,
} SPGRAMMARSTATE;

Elements

SPGS_ENABLED
Activates all the rules in the grammar for the SR engine.
SPGS_DISABLED
Deactivates all the rules in the grammar for the SR engine.
SPGS_EXCLUSIVE
Turns off all rules that are not part of this grammar. Currently not implemented.


SPGRAMMARWORDTYPE

SPGRAMMARWORDTYPE specifies the type of the word(s) to be added to a grammar. This type is either specified in the Speech Text Grammar Format as <GRAMMAR WORDTYPE="LEXICAL"> or as a parameter to ISpGrammarBuilder::AddWordTransition(). SAPI currently only allows SPWT_LEXICAL.

typedef enum SPGRAMMARWORDTYPE
{
    SPWT_DISPLAY,
    SPWT_LEXICAL,
    SPWT_PRONUNCIATION
} SPGRAMMARWORDTYPE;

Elements

SPWT_DISPLAY
Each word to be added is in display form, meaning that it possibly will have to be converted into lexical form(s). For example, the word "23" (display form) would have to converted into "twenty three" (lexical form). This is currently not implemented in SAPI.
SPWT_LEXICAL
Each word to be added is in lexical form and can be used to access the lexicon.
SPWT_PRONUNCIATION
Each word is specified solely by its pronunciation. This is currently not implemented.


SPINTERFERENCE

SPINTERFERENCE lists possible causes of interference or poor recognition with the input stream.

typedef enum SPINTERFERENCE
{
    SPINTERFERENCE_NONE
    SPINTERFERENCE_NOISE,
    SPINTERFERENCE_NOSIGNAL,
    SPINTERFERENCE_TOOLOUD,
    SPINTERFERENCE_TOOQUIET,
    SPINTERFERENCE_TOOFAST,
    SPINTERFERENCE_TOOSLOW
} SPINTERFERENCE;

Elements

SPINTERFERENCE_NONE
Private event. Do not use.
SPINTERFERENCE_NOISE
The sound being received is interpreted by the speech recognition engine as noise. This event is generated when there is a SOUND_START followed by a SOUND_END without an intervening PHRASE_START. The event will be also generated during dictation if, after a series of hypotheses, it is determined that the signal is noise.
SPINTERFERENCE_NOSIGNAL
A sound is received but it is of a constant intensity. This also includes the microphone being unplugged or muted.
SPINTERFERENCE_TOOLOUD
A sound is being received but the stream intensity is too high for discrete recognitions.
SPINTERFERENCE_TOOQUIET
A sound is being received but the stream intensity is too low for discrete recognition
SPINTERFERENCE_TOOFAST
The words are spoken too quickly for discrete recognitions.
SPINTERFERENCE_TOOSLOW
The words are spoken too slowly and indicates excessive time between words.


SPLEXICONTYPE

SPLEXICONTYPE specifies the allowed lexicon types. Currently there are only two types in use--user and application lexicons. There are ample reserved types for future expansion and ample private types for potential private use by applications and/or engines.

typedef enum SPLEXICONTYPE
{
   eLEXTYPE_USER        = (
   eLEXTYPE_APP         =
   eLEXTYPE_RESERVED1   =
   eLEXTYPE_RESERVED2   =
   eLEXTYPE_RESERVED3   =
   eLEXTYPE_RESERVED4   =
   eLEXTYPE_RESERVED5   =
   eLEXTYPE_RESERVED6   =
   eLEXTYPE_RESERVED7   =
   eLEXTYPE_RESERVED8   =
   eLEXTYPE_RESERVED9   =
   eLEXTYPE_RESERVED10  =
   eLEXTYPE_PRIVATE1    =
   eLEXTYPE_PRIVATE2    =
   eLEXTYPE_PRIVATE3    =
   eLEXTYPE_PRIVATE4    =
   eLEXTYPE_PRIVATE5    =
   eLEXTYPE_PRIVATE6    =
   eLEXTYPE_PRIVATE7    =
   eLEXTYPE_PRIVATE8    =
   eLEXTYPE_PRIVATE9    =
   eLEXTYPE_PRIVATE10   =
   eLEXTYPE_PRIVATE11   =
   eLEXTYPE_PRIVATE12   =
   eLEXTYPE_PRIVATE13   =
   eLEXTYPE_PRIVATE14   =
   eLEXTYPE_PRIVATE15   =
   eLEXTYPE_PRIVATE16   =
   eLEXTYPE_PRIVATE17   =
   eLEXTYPE_PRIVATE18   =
   eLEXTYPE_PRIVATE19   =
   eLEXTYPE_PRIVATE20   =
} SPLEXICONTYPE;

Elements

eLEXTYPE_USER
Indicates the user lexicon. Each Windows user has a unique user lexicon.
eLEXTYPE_APP
Indicates the application lexicons. Application lexicon is shared by all users.
eLEXTYPE_RESERVED1 through eLEXTYPE_RESERVED10
Reserved for future use.
eLEXTYPE_PRIVATE1 through eLEXTYPE_PRIVATE20
Indicates the private lexicons. Engine can call ISpContainerLexicon::AddLexicon with one of these flags to add its private lexicons to the container lexicon for consistent access. If these lexicon types are used the lexicons must fully comply with the ISpLexicon interface.


SPLOADOPTIONS

SPLOADOPTIONS indicates how a grammar is loaded. This is used by the ISpRecoGrammar interface.

typedef enum SPLOADOPTIONS
{
    SPLO_STATIC
    SPLO_DYNAMIC
} SPLOADOPTIONS;

Elements

SPLO_STATIC
Flag specifying that the grammar is loaded statically.
SPLO_DYNAMIC
Flag specifying that the grammar is loaded dynamically, meaning that rules can be modified and committed at run-time.


SPPARTOFSPEECH

SPPARTOFSPEECH defines the parts-of-speech categories used in SAPI 5. This list of known parts-of-speech types is intentionally small and broad and will be expanded and refined in future releases. SPPARTOFSPEECH in its minimal form is required to support look ups from the standard SAPI lexicon. This information is useful to TTS engines to determine the correct pronunciation for ambiguous words based on their context.

typedef enum SPPARTOFSPEECH
{
    //--- SAPI5 public POS category values (bits 28-31)
    SPPS_NotOverriden
    SPPS_Unknown
    SPPS_Noun
    SPPS_Verb
    SPPS_Modifier
    SPPS_Function
    SPPS_Interjection
} SPPARTOFSPEECH;

Elements

SPPS_NotOverriden
Flag indicating that the part of speech already present in the lexicon should not be overridden.
SPPS_Unknown
Flag indicating that the part of speech is unknown and is probably from the user lexicon.
SPPS_Noun
Flag indicating that the part of speech is a noun.
SPPS_Verb
Flag indicating that the part of speech is a verb.
SPPS_Modifier
Flag indicating that the part of speech is a modifier.
SPPS_Function
Flag indicating that the part of speech is a function.
SPPS_Interjection
Flag indicating that the part of speech is an interjection.


SPPHRASERNG

typedef enum SPPHRASERNG
{
   SPPR_ALL_ELEMENTS = -1
} SPPHRASERNG;

Elements

SPPR_ALL_ELEMENTS
Indicates all elements of an alternate phrase may be used.


SPPROPSRC

SPPROPSRC specifies recognition context information.

typedef enum SPPROPSRC
{
    SPPROPSRC_RECO_INST,
    SPPROPSRC_RECO_CTX,
    SPPROPSRC_RECO_GRAMMAR
} SPPROPSRC;

Elements

SPPROPSRC_RECO_INST
Flag specifying the SPPROPSRC_RECO_INST recognition context.
SPPROPSRC_RECO_CTX
Flag specifying the SPPROPSRC_RECO_CTX recognition context.
SPPROPSRC_RECO_GRAMMAR
Flag specifying the SPPROPSRC_RECO_GRAMMAR recognition context.


SPRECOEVENTFLAGS

SPRECOEVENTFLAGS specifies the state of the SR engine.

typedef enum SPRECOEVENTFLAGS
{
    SPREF_AutoPause    = (1 << 0)
    SPREF_Emulated     = (1 << 1)
} SPRECOECOEVENTFLAGS;

Elements

SPREF_AutoPause
Indicates that the engine is in the auto-paused state.
SPREF_Emulated
Indicates that the engine is in emulation.


SPRECOSTATE

SPRECOSTATE specifies the current state of the recognition engine.

typedef enum SPRECOSTATE
{
    SPRST_INACTIVE,
    SPRST_ACTIVE,
    SPRST_ACTIVE_ALWAYS,
    SPRST_INACTIVE_WITH_PURGE,
    SPRST_NUM_STATES
} SPRECOSTATE;

Elements

SPRST_INACTIVE
The engine and audio input are inactive and no audio is being read, even if there rules active.The audio device will be closed in this state. Normally an application should not set the state to SPRST_INACTIVE because when using the shared engine recognition will be stopped for all applications, not just this one. An application can easily disable recognition on its contexts by calling ISpRecoContext::SetContextState.
SPRST_ACTIVE
This state is the default and indicates that recognition will take place if there are any active rules. If there is an active rule audio will be read and passed to the SR engine and recognition will happen.
SPRST_ACTIVE_ALWAYS
Indicates the audio is running regardless of the rule state. Even if there are no active rules audio will still be read and passed to the engine. This state can be useful for applications if they want to receive volume level events (SPEI_SR_AUDIO_LEVEL), in order to display a VU-meter or similar.
SPRST_INACTIVE_WITH_PURGE
Indicates the engine state will be set to inactive, but all active audio data is purged. This state is used when an application wishes to shut an engine down as quickly as possible, without waiting for it to finish processing any audio data that is currently buffered. This state should be used with care because it will affect all applications in the shared case.
SPRST_NUM_STATES
To be provided in a future release.


SPRESULTTYPE

SPRESULTTYPE contains the result object type information.

typedef enum SPRESULTTYPE {
    SPRT_CFG =
    SPRT_SLM
    SPRT_PROPRIETARY
    SPRT_FALSE_RECOGNITION
} SPRESULTTYPE;

Elements

SPRT_CFG
Flag specifying that the result object is a CFG type.
SPRT_SLM
Flag specifying that the result object is an SLM type.
SPRT_PROPRIETARY
Flag specifying that the result object is a proprietary type.
SPRT_FALSE_RECOGNITION
Flag specifying that the result object is a false recognition type.


SPRULEINFOOPT

SPRULEINFOOPT specifies grammar rule options.

typedef enum SPRULEINFOOPT
{
    SPRIO_NONE
} SPRULEINFOOPT;

Elements

SPRIO_NONE
Flag specifying the SPRIO_NONE option.


SPRULESTATE

SPRULESTATE indicates the state of a grammar rule.

typedef enum SPRULESTATE
{
    SPRS_INACTIVE
    SPRS_ACTIVE
    SPRS_ACTIVE_WITH_AUTO_PAUSE
} SPRULESTATE;

Elements

SPRS_INACTIVE
Grammar rule is inactive.
SPRS_ACTIVE
Grammar rule is active.
SPRS_ACTIVE_WITH_AUTO_PAUSE
SR engine will be placed in a paused state when the grammar rule is recognized. Also known as an "auto-pause" rule.


SPRUNSTATE

SPRUNSTATE defines the voice running state.

typedef enum SPRUNSTATE
{
    SPRS_DONE
    SPRS_IS_SPEAKING
} SPRUNSTATE;

Elements

SPRS_DONE
The voice has completed processing all queued streams.
SPRS_IS_SPEAKING
The voice instance currently has the audio claimed.


SPSTREAMFORMAT

SPSTREAMFORMAT aids the developer in avoiding subtle bugs introduced by typos. The enumeration can be used with the helper function SpConvertStreamFormatEnum and the helper class CSpStreamFormat. These helpers allow the developer, in most cases, to avoid creating a WAVEFORMATEX structure by hand, which can introduce subtle errors.

typedef enum SPSTREAMFORMAT
{
    SPSF_Default = -1,
    SPSF_NoAssignedFormat
    SPSF_Text,
    SPSF_NonStandardFormat,
    SPSF_ExtendedAudioFormat,
    // Standard PCM wave formats
    SPSF_8kHz8BitMono,
    SPSF_8kHz8BitStereo,
    SPSF_8kHz16BitMono,
    SPSF_8kHz16BitStereo,
    SPSF_11kHz8BitMono,
    SPSF_11kHz8BitStereo,
    SPSF_11kHz16BitMono,
    SPSF_11kHz16BitStereo,
    SPSF_12kHz8BitMono,
    SPSF_12kHz8BitStereo,
    SPSF_12kHz16BitMono,
    SPSF_12kHz16BitStereo,
    SPSF_16kHz8BitMono,
    SPSF_16kHz8BitStereo,
    SPSF_16kHz16BitMono,
    SPSF_16kHz16BitStereo,
    SPSF_22kHz8BitMono,
    SPSF_22kHz8BitStereo,
    SPSF_22kHz16BitMono,
    SPSF_22kHz16BitStereo,
    SPSF_24kHz8BitMono,
    SPSF_24kHz8BitStereo,
    SPSF_24kHz16BitMono,
    SPSF_24kHz16BitStereo,
    SPSF_32kHz8BitMono,
    SPSF_32kHz8BitStereo,
    SPSF_32kHz16BitMono,
    SPSF_32kHz16BitStereo,
    SPSF_44kHz8BitMono,
    SPSF_44kHz8BitStereo,
    SPSF_44kHz16BitMono,
    SPSF_44kHz16BitStereo,
    SPSF_48kHz8BitMono,
    SPSF_48kHz8BitStereo,
    SPSF_48kHz16BitMono,
    SPSF_48kHz16BitStereo,
    // TrueSpeech format
    SPSF_TrueSpeech_8kHz1BitMono,
    // A-Law formats
    SPSF_CCITT_ALaw_8kHzMono,
    SPSF_CCITT_ALaw_8kHzStereo,
    SPSF_CCITT_ALaw_11kHzMono,
    SPSF_CCITT_ALaw_11kHzStereo,
    SPSF_CCITT_ALaw_22kHzMono,
    SPSF_CCITT_ALaw_22kHzStereo,
    SPSF_CCITT_ALaw_44kHzMono,
    SPSF_CCITT_ALaw_44kHzStereo,
    // u-Law formats
    SPSF_CCITT_uLaw_8kHzMono,
    SPSF_CCITT_uLaw_8kHzStereo,
    SPSF_CCITT_uLaw_11kHzMono,
    SPSF_CCITT_uLaw_11kHzStereo,
    SPSF_CCITT_uLaw_22kHzMono,
    SPSF_CCITT_uLaw_22kHzStereo,
    SPSF_CCITT_uLaw_44kHzMono,
    SPSF_CCITT_uLaw_44kHzStereo,
    // ADPCM formats
    SPSF_ADPCM_8kHzMono,
    SPSF_ADPCM_8kHzStereo,
    SPSF_ADPCM_11kHzMono,
    SPSF_ADPCM_11kHzStereo,
    SPSF_ADPCM_22kHzMono,
    SPSF_ADPCM_22kHzStereo,
    SPSF_ADPCM_44kHzMono,
    SPSF_ADPCM_44kHzStereo,
    // GSM 6.10 formats
    SPSF_GSM610_8kHzMono,
    SPSF_GSM610_11kHzMono,
    SPSF_GSM610_22kHzMono,
    SPSF_GSM610_44kHzMono,
    SPSF_NUM_FORMATS
} SPSTREAMFORMAT;

Elements

The enumeration elements contain a list of all common audio formats from the uncompressed PCM formats to the more highly compressed formats available as standard on the Windows operating systems that SAPI 5 is designed to run on.


SPTRANSITIONTYPE

SPTRANSITIONTYPE contains grammar rule transition information.

typedef enum SPTRANSITIONTYPE
{
    SPTRANSEPSILON,
    SPTRANSWORD,
    SPTRANSRULE,
    SPTRANSTEXTBUF,
    SPTRANSWILDCARD,
    SPTRANSDICTATION
} SPTRANSITIONTYPE;

Elements

SPTRANSEPSILON
Flag specifying that the grammar rule is an SPTRANSEPSILON type.
SPTRANSWORD
Flag specifying that the grammar rule is an SPTRANSWORD type.
SPTRANSRULE
Flag specifying that the grammar rule is an SPTRANSRULE type.
SPTRANSTEXTBUF
Flag specifying that the grammar rule is an SPTRANSTEXTBUF type.
SPTRANSWILDCARD
Flag specifying that the grammar rule is an SPTRANSWILDCARD type.
SPTRANSDICTATION
Flag specifying that the grammar rule is an SPTRANSDICTATION type.


SPVACTIONS

SPVACTIONS is a member of SPVSTATE, which is a member of SPVTEXTFRAG. This enumeration specifies the action which should be taken for the text fragment with which it is associated.

typedef enum SPVACTIONS
{
    SPVA_Speak
    SPVA_Silence,
    SPVA_Pronounce,
    SPVA_Bookmark,
    SPVA_SpellOut,
    SPVA_Section,
    SPVA_ParseUnknownTag
} SPVACTIONS;

Elements

SPVA_Speak
The default value - the associated text fragment should be processed and spoken.
SPVA_Silence
There is no associated text string - the associated fragment was the result of a <Silence> XML tag, and a silence (of length specified in the associated SPVSTATE) should be inserted into the output stream.
SPVA_Pronounce
The associated text (possibly empty) is associated with a <Pron> XML tag, and should be pronounced as specified in the associated SPVSTATE.
SPVA_Bookmark
The associated text fragment is the contents of a bookmark.
SPVA_SpellOut
Each character, other than white space, of the associated text fragment should be expanded as a word (e.g. "word!" would become "w o r d exlamation point").
SPVA_Section
Reserved for future use.
SPVA_ParseUnknownTag
The associated text fragment is an unknown XML tag that may be interpreted (or ignored) by the engine.


SPVALUETYPE

typedef enum tagSPDISCARDTYPES
{
    SPDF_PROPERTY
    SPDF_REPLACEMENT
    SPDF_RULE
    SPDF_DISPLAYTEXT
    SPDF_LEXICALFORM
    SPDF_PRONUNCIATION
    SPDF_AUDIO
    SPDF_ALTERNATES
    SPDF_ALL
} SPVALUETYPE;

Elements

SPDF_PROPERTY
Removes the property tree.
SPDF_REPLACEMENT
Removes the phrase replacement text for inverse text normalization.
SPDF_RULE
Removes the non-top level rule tree info for a phrase.
SPDF_DISPLAYTEXT
Removes the display text.
SPDF_LEXICALFORM
Removes the lexicon from text.
SPDF_PRONUNCIATION
Removes the pronunciation text.
SPDF_AUDIO
Removes the audio data that is attached to a phrase. However, the audio had to be both set and retained.
SPDF_ALTERNATES
Removes the alternate data that is attached to a phrase. Discarding alternates loses the words permanently and may not be retrieved even with with ISpSRAlternates::GetAlternates or ISpRecoResult::GetAlternates.
SPDF_ALL
Removes all the elements above.


SPVESACTIONS

SPVESACTIONS specifies values returned by the ISpTTSEngineSite::GetActions call. From these values, the TTS engine receives which real-time action requests have been made by an application.

typedef enum SPVESACTIONS
{
    SPVES_CONTINUE
    SPVES_ABORT
    SPVES_SKIP
    SPVES_RATE
    SPVES_VOLUME
} SPVESACTIONS;

Elements

SPVES_CONTINUE
Default value - indicates SAPI has not received any new information for the engine, and it should continue the synthesis process.
SPVES_ABORT
Flag indicating the engine should stop the synthesis process, and return from the current speak call immediately.
SPVES_SKIP
Flag indicating the application has requested a real-time skip. The engine should call ISpTTSEngineSite::GetSkipInfo.
SPVES_RATE
Flag indicating the application has requested a real-time rate change. The engine should call ISpTTSEngineSite::GetRate.
SPVES_VOLUME
Flag indicating the application has requested a real-time volume change. The engine should call ISpTTSEngineSite::GetVolume.


SPVFEATURE

SPVFEATURE gives information about the features of the phonemes and visemes.

typedef enum SPVFEATURE
{
    SPVFEATURE_STRESSED
    SPVFEATURE_EMPHASIS
} SPVFEATURE;

Elements

SPVFEATURE_STRESSED
This flag indicates that the phoneme is stressed relative to the other phonemes within a word.
SPVFEATURE_EMPHASIS
This flag indicates that the word (of which the phoneme is a part) is emphasized relative to the other words within a sentence.


SPVISEMES

SPVISEMES defines the SAPI 5 Viseme set. This set is based on the Disney 13 Visemes. Examples given are for the SAPI 5 English Phoneme set.

typedef enum SPVISEMES
{
                        // English examples
                        //------------------
    SP_VISEME_0 = 0,    // silence
    SP_VISEME_1,        // ae, ax, ah
    SP_VISEME_2,        // aa
    SP_VISEME_3,        // ao
    SP_VISEME_4,        // ey, eh, uh
    SP_VISEME_5,        // er
    SP_VISEME_6,        // y, iy, ih, ix
    SP_VISEME_7,        // w, uw
    SP_VISEME_8,        // ow
    SP_VISEME_9,        // aw
    SP_VISEME_10,       // oy
    SP_VISEME_11,       // ay
    SP_VISEME_12,       // h
    SP_VISEME_13,       // r
    SP_VISEME_14,       // l
    SP_VISEME_15,       // s, z
    SP_VISEME_16,       // sh, ch, jh, zh
    SP_VISEME_17,       // th, dh
    SP_VISEME_18,       // f, v
    SP_VISEME_19,       // d, t, n
    SP_VISEME_20,       // k, g, ng
    SP_VISEME_21,       // p, b, m
} SPVISEMES;

Elements

SP_VISEME_0
Silence
SP_VISEME_1
ae, ax, ah
SP_VISEME_2
aa
SP_VISEME_3
ao
SP_VISEME_4
ey, eh, uh
SP_VISEME_5
er
SP_VISEME_6
y, iy, ih, ix
SP_VISEME_7
w, uw
SP_VISEME_8
ow
SP_VISEME_9
aw
SP_VISEME_10
oy
SP_VISEME_11
ay
SP_VISEME_12
h
SP_VISEME_13
r
SP_VISEME_14
l
SP_VISEME_15
s, z
SP_VISEME_16
sh, ch, jh, zh
SP_VISEME_17
th, dh
SP_VISEME_18
f, v
SP_VISEME_19
d, t, n
SP_VISEME_20
k, g, ng
SP_VISEME_21
p, b, m


SPVLIMITS

SPVLIMITS specifies the minimum and maximum values which SAPI 5 TTS engines are required to support for rate and volume adjustments.

typedef enum SPVLIMITS
{
    SPMIN_VOLUME
    SPMAX_VOLUME
    SPMIN_RATE
    SPMAX_RATE
} SPVLIMITS;

Elements

SPMIN_VOLUME
Value specifying the minimum volume level.
SPMAX_VOLUME
Value specifying the maximum volume level.
SPMIN_RATE
Value specifying the minimum rate level.
SPMAX_RATE
Value specifying the maximum rate level.


SPVPRIORITY

SPVPRIORITY defines the priorities which voices can have. See ISpVoice::SetPriority and ISpVoice::GetPriority for more information.

typedef enum SPVPRIORITY
{
    SPVPRI_NORMAL
    SPVPRI_ALERT
    SPVPRI_OVER
} SPVPRIORITY;

Elements

SPVPRI_NORMAL
Normal priority.
SPVPRI_ALERT
Alert priority.
SPVPRI_OVER
Over priority - the voice should mix its audio with all other audio on the system with no synchronization. SPVPRI_OVER voices only mix their audio on Windows 2000.


SPVSKIPTYPE

SPVSKIPTYPE specifies the type of item to skip in an ISpVoice::Skip call.

typedef enum SPVSKIPTYPE
{
    SPVST_SENTENCE
} SPVSKIPTYPE;

Elements

SPVST_SENTENCE
Specifies that the structure to be skipped is a sentence.


SPWAVEFORMATTYPE

SPWAVEFORMATTYPE is used in the ISpRecognizer::GetFormat method (as SPSTREAMFORMATTYPE) to request either the input format for the original audio source or, the format actually arriving at the speech engine. SAPI may be performing on the fly conversion using an SpStreamFormatConverter inbetween which will cause the two formats to differ.

typedef enum SPWAVEFORMATTYPE
{
    SPWF_INPUT,
    SPWF_SRENGINE
} SPSTREAMFORMATTYPE;

Elements

SPWF_INPUT
Request for the original audio input source information.
SPWF_SRENGINE
Request for the SR engine input source information.


SPWORDINFOOPT

SPWORDINFOOPT specifies the options for a grammar word.

typedef enum SPWORDINFOOPT
{
    SPWIO_NONE
    SPWIO_WANT_TEXT
} SPWORDINFOOPT;

Elements

SPWIO_NONE
Flag specifying the SPWIO_NONE option.
SPWIO_WANT_TEXT
Flag specifying the SPWIO_WANT_TEXT option.


SPWORDPRONOUNCEABLE

See also ISpRecoGrammar::IsPronounceable and ISpSREngine::IsPronounceable.

typedef enum SPWORDPRONOUNCEABLE
{
    SPWP_UNKNOWN_WORD_UNPRONOUNCEABLE
    SPWP_UNKNOWN_WORD_PRONOUNCEABLE
    SPWP_KNOWN_WORD_PRONOUNCEABLE
} SPWORDPRONOUNCEABLE;

Elements

SPWP_UNKNOWN_WORD_UNPRONOUNCEABLE
Specifies an unrecognized word that does not have an available pronunciation.
SPWP_UNKNOWN_WORD_PRONOUNCEABLE
Specifies an unrecognized word that has an available pronunciation.
SPWP_KNOWN_WORD_PRONOUNCEABLE
Specifies a recognized word that has an available pronunciation.


SPWORDTYPE

SPWORDTYPE specifies the change state of a word/pronunciation combination in a lexicon. Using this enumeration, an engine can determine what word pronunciation changes have occurred since it last checked using the ISpLexicon::GetGenerationChange method.

typedef enum SPWORDTYPE
{
   eWORDTYPE_ADDED
   eWORDTYPE_DELETED
} SPWORDTYPE;

Elements

eWORDTYPE_ADDED
The word has been added to the lexicon.
eWORDTYPE_DELETED
The word has been deleted from the lexicon.


Helper Functions

Helper functions are available as a convenience to programming. Since many of the procedures use the same few methods or calls in the same sequence each time, these functions are available to consolidate those standard sequences. In all cases, the functions represent nothing more than the individual steps combined into one call; no additional features have been added or removed. Programmers are free to either use these functions or include the original lines of code. The function name attempts to clearly identify the purpose of the function itself.

The following helper functions are used with SAPI 5.0.

Token Helpers

  • SpCreateBestObject
  • SpCreateDefaultObjectFromCategoryId
  • SpCreateNewToken (by token ID)
  • SpCreateNewToken (by category ID)
  • SpCreateNewTokenEx (by token ID)
  • SpCreateNewTokenEx (by category ID)
  • SpCreateObjectFromSubToken
  • SpCreateObjectFromToken
  • SpCreateObjectFromTokenId
  • SpCreatePhoneConverter
  • SpEnumTokens
  • SpFindBestToken
  • SpGetCategoryFromId
  • SpGetDefaultTokenFromCategoryId
  • SpGetDefaultTokenIdFromCategoryId
  • SpGetDescription
  • SpGetSubTokenFromToken
  • SpGetTokenFromId
  • SpGetUserDefaultUILanguage
  • SpSetCommonTokenData
  • SpSetDefaultTokenForCategoryId
  • SpSetDefaultTokenIdForCategoryId
  • SpSetDescription

Other Helpers

  • SPFEI
  • SPBindToFile
  • SpClearEvent
  • SpConvertStreamFormatEnum
  • SpEventSerializeSize
  • SpInitEvent

Helper Classes

  • CSpDynamicString
  • CSpStreamFormat
  • CSpEvent


CSpStreamFormat

CSpStreamFormat Methods | Description
CSpStreamFormat | The class constructors.
~CSpStreamFormat | The class destructor.
AssignFormat | Assigns (or copies) the instance's current format to a new format (or a new stream).
Clear | Clears values from an instance.
CopyTo | Copies the instance's wave format to a new stream.
Deserialize | Deserializes a stream and passes back a new stream.
DetachTo | Makes a copy of the instance's stream and frees (or detaches) the instance's stream.
FormatId | Returns the instance's format ID.
IsEqual | Compares a specified stream format and format ID to the instance's format and format ID.
ParamValidateAssignFormat | Validates the format ID and wave format and creates the stream.
ParamValidateCopyTo | Validates the pointers for the format ID and wave format parameters.
Serialize | Seralizes the stream.
SerializeSize | Determines the serialized size of the instance's stream.
WaveFormatExPtr | Returns the instance's wave format.


CSpStreamFormat Constructor Methods

 The Microsoft.com Speech website

CSpStreamFormat::Constructor

The following methods may be used to construct the instance.

Initializes the class members to null values

CSpStreamFormat( void );

Parameters

None.

Formats the instance into a wave format structure. The new format and format ID are passed back from the class' public members, m_pCoMemWaveFormatEx and m_guidFormatId respectively.

CSpStreamFormat(
   SPSTREAMFORMAT   eFormat,
   HRESULT         *phr
);

Parameters

eFormat
[in] The requested stream format. Must be a valid SPSTREAMFORMAT value of SPSF_8kHz8BitMono or greater.
phr
The return value for the method.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.
E_INVALIDARG | Either class member m_guidFormatId or m_pCoMemWaveFormatEx is invalid or bad. Alternatively, the current format is not recognized.
FAILED(hr) | Appropriate error message.

Formats the instance according to the format structure specified.

CSpStreamFormat(
   const WAVEFORMATEX *pWaveFormatEx,
   HRESULT            *phr
);

Parameters

pWaveFormatEx
[in] Address of the WAVEFORMATEX structure containing the wave file format information.
phr
The return value for the method.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.


CSpStreamFormat Destructor Method

 The Microsoft.com Speech website

CSpStreamFormat::Destructor

CSpStreamFormat::Destructor is the class destructor. CoMemTaskFree() is used to deallocate the instance.

~CSpStreamFormat( void );

Parameters

None.

Return values

No value is returned.


AssignFormat

 The Microsoft.com Speech website

CSpStreamFormat::AssignFormat

CSpStreamFormat::AssignFormat assigns (or copies) the instance's current format to a new format (or a new stream).

Converts the instance's stream format into a wave format structure.

HRESULT AssignFormat(
   SPSTREAMFORMAT   eFormat
);

Parameters

eFormat
[in] The requested stream format. Must be a valid SPSTREAMFORMAT value of SPSF_8kHz8BitMono or greater.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.
E_INVALIDARG | Either class member m_guidFormatId or m_pCoMemWaveFormatEx is invalid or bad. Alternatively, the current format is not recognized.
FAILED(hr) | Appropriate error message.

Converts instance's stream format into a cached format. Class member m_pCoMemWaveFormatEx will be NULL if an error occurred.

HRESULT AssignFormat(
   ISpStreamFormat  *pStream
);

Parameters

pStream
[in] An ISpStreamFormat object.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | Either class member m_guidFormatId or m_pCoMemWaveFormatEx is invalid or bad.

Converts instance's stream format into the specified wave format. Class member m_guidFormatId will be GUID_NULL if an error occurred.

HRESULT AssignFormat(
   const   WAVEFORMATEX   *pWaveFormatEx
);

Parameters

pWaveFormatEx
[in] Address of the WAVEFORMATEX structure containing the wave file format information.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.

Assigns the instance's format according to a reference GUID and a wave format.
HRESULT AssignFormat(
   REFGUID                 rguidFormatId,
   const   WAVEFORMATEX   *pWaveFormatEx
);

Parameters

rguidFormatId
[in] The reference ID. If specified as SPDFID_WaveFormatEx, pWaveFormatEx is a WAVEFORMATEX data structure. Otherwise this is set to GUID_NULL.
pWaveFormatEx
[in] If rguidFormatId is not set to SPDFID_WaveFormatEx, this is passed back as NULL.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | pWaveFormatEx is invalid or bad.
E_OUTOFMEMORY | Exceeded available memory.

Assigns (or copies) the instance's stream to a specific stream.
HRESULT AssignFormat(
   const   CSpStreamFormat & Src
);

Parameters

Src
[out] The stream to copy to.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.


Clear

 The Microsoft.com Speech website

CSpStreamFormat::Clear

CSpStreamFormat::Clear clears all existing values from the instance.

void Clear( void );

Parameters

None.

Return values

No value is returned.


CopyTo

 The Microsoft.com Speech website

CSpStreamFormat::CopyTo

CSpStreamFormat::CopyTo copies the instance's wave format to a new stream.

HRESULT CopyTo(
   GUID            *pFormatId,
   WAVEFORMATEX   **ppCoMemWFEX
   );

Parameters

pFormatId
The new format ID based on the class member m_guidFormatId's ID. If class member m_pCoMemWaveFormatEx is invalid, pFormatId is set to NULL.
ppCoMemWFEX
The new wave format. If ppCoMemWFEX could not be successfully created, pFormatId is set to zero.

This methods copies the instance's stream to an existing stream.
HRESULT CopyTo(
   wCSpStreamFormat &Other
);

Parameters

Other
The existing stream to copy to. The stream is freed using CoTaskMemFree() first and the instance is then copied to it.

Return values

The return values are the same for both methods.
Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.


Deserialize

 The Microsoft.com Speech website

CSpStreamFormat::Deserialize

CSpStreamFormat::Deserialize deserializes a stream and passes back a new stream.

HRESULT Deserialize(
   const   BYTE   *pBuffer,
   ULONG          *pcbUsed
);

Parameters

pBuffer
[in out] Buffer containing the serialized stream. After successfully completing the methods, pBuffer will also be set to class member m_pCoMemWaveFormatEx. If unsuccessful, class member m_guidFormatId is set to GUID_NULL.
pcbUsed
The number of bytes used for the stream.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.


DetachTo

 The Microsoft.com Speech website

CSpStreamFormat::DetachTo

CSpStreamFormat::DetachTo copies the instance's stream and frees (or detaches) the instance's stream. The instance's stream is set to NULL and the format ID is set to zero.

Copies the instance's stream to another existing stream.

void DetachTo(
   CSpStreamFormat   &Other
);

Parameters

Other
The existing stream to copy to. The stream is freed using CoTaskMemFree() first and the instance is then copied to it.

Makes a copy of the instance's stream with the specified format ID and wave format.

void DetachTo(
   GUID            *pFormatId,
   WAVEFORMATEX   **ppCoMemWaveFormatEx
   )

Parameters

pFormatId
The new format ID based on the class member m_guidFormatId.
ppCoMemWaveFormatEx
The new wave format based on the class member m_pCoMemWaveFormatEx.

Return values

Neither method returns a value.


FormatId

 The Microsoft.com Speech website

CSpStreamFormat::FormatId

CSpStreamFormat::FormatId returns the instance's format ID.

const GUID& FormatId( void );

Parameters

None.

Return values

The format ID of instance is returned by the class member m_guidFormatId.


IsEqual

 The Microsoft.com Speech website

CSpStreamFormat::IsEqual

CSpStreamFormat::IsEqual compares a specified stream format and format ID to the instance's format and format ID.

BOOL IsEqual(
   REFGUID                 rguidFormatId,
   const   WAVEFORMATEX   *pwfex
)

Parameters

rguidFormatId
The format ID of the stream to compare with.
pwfex
The wave format of the stream to compare with.

Return values

Value | Description
TRUE | The two streams are identical.
FALSE | The reference IDs of the streams are not the same, or the streams have different contents.


ParamValidateAssignFormat

 The Microsoft.com Speech website

CSpStreamFormat::ParamValidateAssignFormat

CSpStreamFormat::ParamValidateAssignFormat validates the format ID and wave format. If successful, creates the stream using AssignFormat.

HRESULT ParamValidateAssignFormat(
   REFGUID               rguidFormatId,
   const WAVEFORMATEX   *pWaveFormatEx,
   BOOL                  fRequireWaveFormat = FALSE
)

Parameters

rguidFormatId
The reference format ID of the requesting stream.
pWaveFormatEx
The wave format of the requesting stream.
fRequireWaveFormat
Optional Boolean indicating to create the stream only if the wave format is of a standard type. It is FALSE by default and may be omitted. TRUE, allows the creation of custom formats.

Return values

Value | Description
E_INVALIDARG | At least one of pWaveFormatEx is NULL, pWaveFormatEx is bad or invalid, rguidFormatId is not SPDFID_WaveFormatEx, or pWaveFormatEx is a non-standard format and is disallowed by fRequireWaveFormat.


ParamValidateCopyTo

 The Microsoft.com Speech website

CSpStreamFormat::ParamValidateCopyTo

CSpStreamFormat::ParamValidateCopyTo validates the pointers for the format ID and wave format parameters. If successful, copies the stream using CopyTo.

HRESULT ParamValidateCopyTo(
   GUID            *pFormatId,
   WAVEFORMATEX   **ppCoMemWFEX
   );

Parameters

pFormatId
The proposed pointer for the format ID.
ppCoMemWFEX
The proposed pointer for the stream.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.
E_POINTER | Either pFormatId or ppCoMemWFEX is invalid or bad.


Serialize

 The Microsoft.com Speech website

CSpStreamFormat::Serialize

CSpStreamFormat::Serialize serializes the stream.

ULONG Serialize(
   BYTE   *pBuffer
   );

Parameters

pBuffer
The buffer accepting the serialized results.

Return values

The length, in bytes, of the serialized stream and stream contents. Class member m_pCoMemWaveFormatEx may be NULL and therefore contributes zero bytes to the total length.


SerializeSize

 The Microsoft.com Speech website

CSpStreamFormat::SerializeSize

CSpStreamFormat::SerializeSize determines the serialized size of the instance's stream.

ULONG SerializeSize( void );

Parameters

None.

Return values

Returns the serialized size of the stream, in bytes.


WaveFormatExPtr

 The Microsoft.com Speech website

CSpStreamFormat::WaveFormatExPtr

CSpStreamFormat::WaveFormatExPtr returns the instance's wave format.

const WAVEFORMATEX* WaveFormatExPtr( void );

Parameters

None.

Return values

The wave format of instance is returned by the class member m_pCoMemWaveFormatEx.


CSpDynamicString

The following methods are available.

ISpDataKey Methods | Description
CSpDynamicString | The class constructors.
~CSpDynamicString | The class destructor.
Length | Returns the length of the instance.
Append | Appends a string or strings to the current instance.
Copy | Makes a copy of the instance.
Attach | Attaches or assigns a string to the instance.
Detach | Detaches a string from the instance.
Clear | Clears the text from an instance.
CopyToBSTR | Allocates a system BSTR.
TrimToSize | Truncates the instance to a specific size.
Compact | Compacts the instance by reallocating it.
ClearAndGrowTo | Clears the instance and reallocates it.
LTrim | Trims the white space starting from the left.
RTrim | Trims the white space starting from the right.
TrimBoth | Trims the white space starting from both the right and left sides.


CSpDynamicString Constructor Methods

 The Microsoft.com Speech website

CSpDynamicString::Constructor

The following methods may be used to construct the instance.

Creates and sets the instance to NULL.

void CSpDynamicString( void )

Parameters

None. Initializes string to NULL.

Creates and allocates an instance of the specified number of WCHARs.
void CSpDynamicString(
   ULONG   cchReserve
);

Parameters

cchReserve
[in] The length of the string to allocate.

Creates and initializes and copies the source string into it.

void CSpDynamicString(
   const WCHAR  *pSrc
)

Parameters

cchReserve
[in] Initializes the string to pSrc.

Creates and initializes and copies the source string into it.

void CSpDynamicString(
   const char  *pSrc
)

Parameters

pSrc
[in] Initializes the string to pSrc.

Makes a copy of the CSpDynamicString class string. The current instance is CoMemtaskFree() first, if needed.

void CSpDynamicString(
   const CSpDynamicString  &src
)

Parameters

src
[in] Initializes the string by copying the string of src.

Allocates an instance and copies the reference GUID into it.

void CSpDynamicString(
   REFGUID   rguid
)
Parameters
rguid
[in] Initializes the string from the CLSID of rguid.


CSpDynamicString Destructor Methods

 The Microsoft.com Speech website

CSpDynamicString::Destructor

CSpDynamicString::Destructor is the class destructor. CoMemTaskFree() is used to deallocate the instance.

~CSpDynamicString( void )

Parameters

None


CSpDynamicString Append Methods

 The Microsoft.com Speech website

CSpDynamicString::Append

The following methods append null-terminated strings together.

Appends the null-terminated source string to the end of the instance and returns the resulting string.

WCHAR* Append(
   const WCHAR  *pszSrc
)

Parameters

pszSrc
[in] The null-terminated source string to append.

Appends the null-terminated source string to the end of the instance and returns the resulting string. The size to append is specified.

WCHAR* Append(
   const WCHAR  *pszSrc,
   const ULONG   lenSrc
)

Parameters

pszSrc
[in] The null-terminated source string to append.
lenSrc
[in] Size, in WCHARs, to append.

Appends up to two null-terminated source strings to the end of the instance and returns the resulting string. At least one of the to strings must be non-NULL.

WCHAR* Append2(
   const WCHAR  *pszSrc1,
   const WCHAR  *pszSrc2
)

Parameters

pszSrc1
[in] The first source string to append.
pszSrc2
[in] The second source string to append.


CSpDynamicString Attach Methods

 The Microsoft.com Speech website

CSpDynamicString::Attach

CSpDynamicString::Attach attaches or assigns a string to the instance.

void Attach
   WCHAR  *pszSrc
)

Parameters

pszSrc
[in] The source string to attach or assign.


CSpDynamicString Clear Methods

 The Microsoft.com Speech website

CSpDynamicString::Clear

CSpDynamicString::Clear clears the text from an instance. The instance is set to NULL.

void Clear( void );

Parameters

None


CSpDynamicString ClearAndGrowTo Methods

 The Microsoft.com Speech website

CSpDynamicString::ClearAndGrowTo

CSpDynamicString::ClearAndGrowTo clears the existing instance and reallocates it to the specified size. The subsequent instance is returned.

WCHAR* ClearAndGrowTo(
   ULONG   cch
)

Parameters

cch
[in] The new size of the instance.


CSpDynamicString Compact Methods

 The Microsoft.com Speech website

CSpDynamicString::Compact

CSpDynamicString::Compact compacts the instance by reallocating it to the actual number of characters it currently contains. The subsequent instance is returned.

WCHAR* Compact( void )

Parameters

None


CSpDynamicString Copy Methods

 The Microsoft.com Speech website

CSpDynamicString::Copy

CSpDynamicString::Copy copies the instance.

WCHAR* Copy( void );

Parameters

None


CSpDynamicString CopyToBSTR Methods

 The Microsoft.com Speech website

CSpDynamicString::CopyToBSTR

CSpDynamicString::CopyToBSTR allocates a system BSTR.

HRESULT CopyToBSTR(
   BSTR  *pbstr
)

Parameters

pbstr
[in] The source string to allocate.


CSpDynamicString Detach Methods

 The Microsoft.com Speech website

CSpDynamicString::Detach

CSpDynamicString::Detach detaches and returns a string from the instance. The instance is set to NULL.

WCHAR* Detach( void );

Parameters

None


CSpDynamicString Length Methods

 The Microsoft.com Speech website

CSpDynamicString::Length

CSpDynamicString::Length returns the length of the string instance. If the instance is NULL, zero is returned.

Length( void )

Parameters

None


CSpDynamicString LTrim Methods

 The Microsoft.com Speech website

CSpDynamicString::LTrim

CSpDynamicString::LTrim trims the white space starting from the left. The subsequent instance is returned.

WCHAR* LTrim( void );

Parameters

None


CSpDynamicString RTrim Methods

 The Microsoft.com Speech website

CSpDynamicString::RTrim

CSpDynamicString::RTrim trims the white space starting from the right. The subsequent instance is returned.

WCHAR* RTrim( void );

Parameters

None


CSpDynamicString TrimBoth Methods

 The Microsoft.com Speech website

CSpDynamicString::TrimBoth

CSpDynamicString::TrimBoth trims the white space starting from both the right and left sides. The subsequent instance is returned.

WCHAR* TrimBoth( void );

Parameters

None


CSpDynamicString TrimToSize Methods

 The Microsoft.com Speech website

CSpDynamicString::TrimToSize

CSpDynamicString::TrimToSize truncates the instance to a specific size.

void TrimToSize(
   ULONG  *ulNumChars
)

Parameters

ulNumChars
[in] The new size of the instance. The value must be less than less than or equal to the current size.


CSpEvent

CSpEvent Methods | Description
CSpEvent | The class constructor.
~CSpEvent | The class destructor.
Clear | Clears an event instance.
AddrOf | Returns the address of the event instance.
CopyTo | Copies the event instance and sets the lparam accordingly.
GetFrom | Clears the current instance and retrieves the next event from the event queue.
CopyFrom | Clears the current instance and copies the next event in the event queue to this instance.
Detach | Clears the current instance and optionally makes a copy.
SerializeSize | Calculates the required size of a buffer to serialize an event.
Serialize | Copies event instance and then serializes the new instance.
Deserialize | Sets the current instance to the deserialzed version of the specified event.
Phoneme | Returns the event as a cast type of SPPHONEID.
Viseme | Returns the event as a cast type of SPEI_VISEME.
InputWordPos | Returns the event as a cast type of ULONG.
InputWordLen | Returns the event as a cast type of ULONG.
InputSentPos | Returns the event as a cast type of ULONG.
InputSentLen | Returns the event as a cast type of ULONG.
ObjectToken | Returns the event as a cast type of ISpObjectToken pointer.
VoiceToken | Returns the event as a cast type of SpObjectToken pointer.
PersistVoiceChange | Returns the state of the voice change.
Object | Returns the event as a cast type of an IUnknown pointer.
RecoResult | Returns the event as a cast type of an ISpRecoResult pointer.
IsPaused | Returns the pause state.
IsEmulated | Returns the emulation state.
String | Returns the string from the event's lParam.
BookmarkName | Returns the bookmark string from the event's lParam.
RequestTypeOfUI | Returns the IU type string from the event's lParam.
RecoState | Returns the event's recognition state as a cast type of SPRECOSTATE.
PropertyName | Returns the property name string from the event's lParam.
PropertyNumValue | Returns the property number value as cast to LONG.
PropertyStringValue | Returns the property string value from the event's lParam.
Interference | Returns the event's interference value from lParam and cast SPINTERFERENCE.
EndStreamResult | Returns the event's end stream result from the event's lParam.
InputStreamReleased | Returns the state from releasing the stream.


CSpEvent

 The Microsoft.com Speech website

CSpEvent::Constructor

CSpEvent::Constructor is used to construct the instance. The SPEVENT instance is cleared to zero.

CSpEvent( void );

Parameters

None.

Return values

None.


~CSpEvent

 The Microsoft.com Speech website

CSpEvent::Destructor

CSpEvent::Destructor is the class destructor and is used to clear an event instance. This method is identical to CSpEvent Clear.

~CSpEvent( void );

Parameters

None.

Return values

None.

See Also

SpClearEvent


AddOf

 The Microsoft.com Speech website

CSpEvent::AddrOf

CSpEvent::AddrOf returns the address of the event instance. No ASSERT is made in the case of an error. For additional validation, call GetFrom.

CSpEvent*   AddrOf (void)

Parameters

None.

Return values

Returns the address of the event instance.


BookmarkName

 The Microsoft.com Speech website

CSpEvent::BookmarkName

CSpEvent::BookmarkName returns the bookmark string from the event's lParam. The caller must make sure the event ID is SPEI_TTS_BOOKMARK.

const WCHAR* BookmarkName( void ) const;

Parameters

None.

Return values

Returns the bookmark string from the event's lParam.


Clear

 The Microsoft.com Speech website

CSpEvent::Clear

CSpEvent::Clear clears an event instance. This method is identical to CSpEvent ~CSpEvent.

void Clear( void );

Parameters

None.

Return values

None.

See Also

SpClearEvent


CopyFrom

 The Microsoft.com Speech website

CSpEvent::CopyFrom

CSpEvent::CopyFrom clears the current instance and copies the next event in the event queue to this instance.

HRESULT CopyFrom(
   const SPEVENT   *pSrcEvent
)

Parameters

pSrcEvent
[ in ] The event of which to copy from.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.


CopyTo

 The Microsoft.com Speech website

CSpEvent::CopyTo

CSpEvent::CopyTo copies the event instance and sets the lparam accordingly.

HRESULT CopyTo(
   SPEVENT   *pDestEvent
);

Parameters

pDestEvent
The event to copy to. The member elParamType is copied according to the follow parameter value:
SPET_LPARAM_IS_POINTER copies wparam into lparam if lparam is currently valid. On error, pDestEvent->eEventId is set to SPEI_UNDEFINED.
SPET_LPARAM_IS_STRING copies lparam into lparam if it is currently valid. On error, pDestEvent->eEventId is set to SPEI_UNDEFINED.
SPET_LPARAM_IS_TOKEN or SPET_LPARAM_IS_OBJECT calls AddRef() for lparam.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.


Deserialize

 The Microsoft.com Speech website

CSpEvent::Deserialize

CSpEvent::Deserialize sets the current instance to the deserialzed version of the specified event.

The current instance is cleared first. The original event is not modified.

template <class T>
HRESULT Deserialize(
   const   T   *pSerEvent,
   ULONG       *pcbUsed = NULL
   );

Parameters

pSerEvent
The serialized event.
pcbUsed
Optional parameter passing back the number of bytes of the instance. The default NULL does not pass back any value; TRUE, does.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.


Detach

 The Microsoft.com Speech website

CSpEvent::Detach

CSpEvent::Detach clears the current instance and optionally makes a copy.

void  Detach(
   SPEVENT   *pDestEvent = NULL
);

Parameters

pDestEvent
The event struture to copy to. If NULL, no copy is made before clearing the instance.

Return values

None.


EndStreamResult

 The Microsoft.com Speech website

CSpEvent::EndStreamResult

CSpEvent::EndStreamResult returns the end stream result from the event's lParam. The caller must make sure the event ID is SPEI_END_SR_STREAM.

HRESULT EndStreamResult( void ) const;

Parameters

None.

Return values

Returns the end stream result from the event's lParam.


GetFrom

 The Microsoft.com Speech website

CSpEvent::GetFrom

CSpEvent::GetFrom clears the current instance and retrieves the next event from the event queue.

HRESULT GetFrom(
   ISpEventSource   *pEventSrc
)

Parameters

pEventSrc
The event object from which to get the next event.

Return values

Value | Description
S_OK | Function completed successfully and all requested events were returned.
S_FALSE | Success, but less than the requested amount of events were returned.
E_POINTER | pEventArray is invalid.
FAILED(hr) | Appropriate error message.


InputSentLen

 The Microsoft.com Speech website

CSpEvent::InputSentLen

CSpEvent::InputSentLen returns the event as a cast type of ULONG. The caller must make sure the event ID is SPEI_SENTENCE_BOUNDARY.

ULONG   InputSentLen( void ) const;

Parameters

None.

Return values

The event cast type of ULONG.


InputSentPos

 The Microsoft.com Speech website

CSpEvent::InputSentPos

CSpEvent::InputSentPos returns the event as a cast type of ULONG. The caller must make sure the event ID is SPEI_SENTENCE_BOUNDARY.

ULONG   InputSentPos( void ) const;

Parameters

None.

Return values

The event cast type of ULONG.


InputStreamReleased

 The Microsoft.com Speech website

CSpEvent::InputStreamReleased

CSpEvent::InputStreamReleased returns the state from releasing the stream. The caller must make sure the event ID is SPEI_END_SR_STREAM.

See also SPENDSRSTREAMFLAGS for more information.

BOOL InputStreamReleased( void ) const;

Parameters

None.

Return values

Returns the state from releasing the stream. TRUE indicates the stream was successfully released; FALSE otherwise.


InputWordLen

 The Microsoft.com Speech website

CSpEvent::InputWordLen

CSpEvent::InputWordLen returns the event as a cast type of ULONG. The caller must make sure the event ID is SPEI_WORD_BOUNDARY.

ULONG InputWordLen( void ) const;

Parameters

None.

Return values

The event cast type of ULONG.


InputWordPos

 The Microsoft.com Speech website

CSpEvent::InputWordPos

CSpEvent::InputWordPos returns the event as a cast type of ULONG. The caller must make sure the event ID is SPEI_WORD_BOUNDARY.

ULONG   InputWordPos( void ) const;

Parameters

None.

Return values

The event cast type of ULONG.


Interference

 The Microsoft.com Speech website

CSpEvent::Interference

CSpEvent::Interference returns the event's interference value from lParam and cast SPINTERFERENCE. The caller must make sure the event ID is SPEI_INTERFERENCE.

SPINTERFERENCE Interference( void ) const;

Parameters

None.

Return values

Returns the interference value as cast SPINTERFERENCE.


IsEmulated

 The Microsoft.com Speech website

CSpEvent::IsEmulated

CSpEvent::IsEmulated returns the emulation state. The caller must make sure the event ID is SPEI_RECOGNITION.

BOOL IsEmulated( void );

Parameters

None.

Return values

The event cast type of SPPHONEID.


IsPaused

 The Microsoft.com Speech website

CSpEvent::IsPaused

CSpEvent::IsPaused returns the pause state. The caller must make sure the event ID is either SPEI_RECOGNITION or SPEI_SR_BOOKMARK.

BOOL IsPaused( void )

Parameters

None.

Return values

Returns the pause state.


Object

 The Microsoft.com Speech website

CSpEvent::Object

CSpEvent::Object returns the event as a cast type of an IUnknown pointer. The caller must make sure the event ID is SPET_LPARAM_IS_OBJECT.

IUnknown* Object( void ) const;

Parameters

None.

Return values

The event cast type of an IUnknown pointer.


ObjectToken

 The Microsoft.com Speech website

CSpEvent::ObjectToken

CSpEvent::ObjectToken returns the event as a cast type of ISpObjectToken pointer. The caller must make sure the event type is SPET_LPARAM_IS_TOKEN.

ISpObjectToken*   ObjectToken( void ) const;

Parameters

None.

Return values

The event cast type of ISpObjectToken pointer.


PersistVoiceChange

 The Microsoft.com Speech website

CSpEvent::PersistVoiceChange

CSpEvent::PersistVoiceChange returns the state of the voice change. The caller must make sure the event ID is SPEI_VOICE_CHANGE.

BOOL PersistVoiceChange( void ) const;

Parameters

None.

Return values

The event cast type of BOOL.


Phoneme

 The Microsoft.com Speech website

CSpEvent::Phoneme

CSpEvent::Phoneme returns the event as a cast type of SPPHONEID. The caller must make sure the event ID is SPEI_PHONEME.

SPPHONEID   Phoneme( void ) const;

Parameters

None.

Return values

The event cast type of SPPHONEID.


PropertyName

 The Microsoft.com Speech website

CSpEvent::PropertyName

CSpEvent::PropertyName returns the property name string from the event's lParam. The caller must make sure the event ID is either SPEI_PROPERTY_NUM_CHANGE and event type is SPET_LPARAM_IS_STRING, or event ID is SPEI_PROPERTY_STRING_CHANGE and event type is SPET_LPARAM_IS_POINTER.

const WCHAR* PropertyName( void ) const;

Parameters

None.

Return values

Returns the property name string from the event's lParam.


PropertyNumValue

 The Microsoft.com Speech website

CSpEvent::PropertyNumValue

CSpEvent::PropertyNumValue returns the property number value as cast to LONG. The caller must make sure the event ID is SPEI_PROPERTY_NUM_CHANGE.

const LONG PropertyNumValue( void ) const;

Parameters

None.

Return values

Returns the property number value as cast to LONG.


PropertyStringValue

 The Microsoft.com Speech website

CSpEvent::PropertyStringValue

CSpEvent::PropertyStringValue returns the property string value from the event's lParam. The caller must make sure the event ID is SPEI_PROPERTY_STRING_CHANGE.

const WCHAR* PropertyStringValue( void ) const;

Parameters

None.

Return values

Returns the property string value from the event's lParam.


RecoResult

 The Microsoft.com Speech website

CSpEvent::RecoResult

CSpEvent::RecoResult returns the event as a cast type of an ISpRecoResult pointer. The caller must make sure the event ID is SPEI_RECOGNITION, SPEI_FALSE_RECOGNITION, or SPEI_HYPOTHESIS.

ISpRecoResult* RecoResult( void ) const;

Parameters

None.

Return values

The event cast type of an ISpRecoResult pointer.


RecoState

 The Microsoft.com Speech website

CSpEvent::RecoState

CSpEvent::RecoState returns the event's recognition state as a cast type of SPRECOSTATE. The caller must make sure the event ID is SPEI_RECO_STATE_CHANGE.

SPRECOSTATE RecoState( void ) const;

Parameters

None.

Return values

Returns the event's recognition state as a cast type of SPRECOSTATE.


RequestTypeOfUI

 The Microsoft.com Speech website

CSpEvent::RequestTypeOfUI

CSpEvent::RequestTypeOfUI returns the UI type string from the event's lParam. The caller must make sure the event ID is SPEI_REQUEST_UI.

const WCHAR* RequestTypeOfUI( void ) const;

Parameters

None.

Return values

Returns the UI type string from the event's lParam.


Serialize

 The Microsoft.com Speech website

CSpEvent::Serialize

CSpEvent::Serialize copies and serializes an event.

Copies event instance and then serializes the new instance. The new instance must be based on SPSERIALIZEDEVENT or SPSERIALIZEDEVENT64

void Serialize(
    T    *pSerEvent
);

Parameters

pSerEvent
[ out ] The event for the passed back serialization. Member elParamType must not be SPET_LPARAM_IS_OBJECT.

Return values

None, however member SerializedlParam is set to the size of the serialized structure.

Copies an existing event instance and then serializes the new instance.

HRESULT Serialize(
   T      **ppCoMemSerEvent,
   ULONG   *pcbSerEvent
   );

Parameters

ppCoMemSerEvent
[ out ] The event for the passed back serialization. It is allocated and serialized. When no longer required, it must be manually freed with CoMemTaskFree().
pcbSerEvent
[ out ] The number of bytes allocated for the serialization. On an error, it will be zero.

Return values

Value | Description
S_OK | Serialization completed successfully.
E_OUTOFMEMORY | Exceeded available memory.


SerializeSize

 The Microsoft.com Speech website

CSpEvent::SerializeSize

CSpEvent::SerializeSize calculates the required size of a buffer to serialize an event. The instance must be either SPSERIALIZEDEVENT or SPSERIALIZEDEVENT64.

ULONG SerializeSize( void );

Parameters

None.

Return values

Size, in bytes, required to serialize the event.


String

 The Microsoft.com Speech website

CSpEvent::String

CSpEvent::String returns the string from the event's lParam. The caller must make sure the event type is SPET_LPARAM_IS_STRING.

const WCHAR* String( void ) const;

Parameters

None.

Return values

Returns the string from the event's lParam.


Viseme

 The Microsoft.com Speech website

CSpEvent::Viseme

CSpEvent::Viseme returns the event as a cast type of SPEI_VISEME. The caller must make sure the event ID is SPEI_VISEME.

SPVISEMES   Viseme( void ) const;

Parameters

None.

Return values

The event cast type of SPVISEMES.


VoiceToken

 The Microsoft.com Speech website

CSpEvent::VoiceToken

CSpEvent::VoiceToken returns the event as a cast type of ISpObjectToken pointer. The caller must make sure the event ID is SPEI_VOICE_CHANGE. This is an additional check for the helper ObjectToken.

ISpObjectToken*   VoiceToken( void ) const;

Parameters

None.

Return values

The event cast type of ISpObjectToken pointer.


Helper SpCreateBestObject

 The Microsoft.com Speech website

SpCreateBestObject

SpCreateBestObject creates an object from tokens that best match a set of criteria from required and optional attributes.

Found in: sphelper.h

SpCreateBestObject(
   const WCHAR   *pszCategoryId,
   const WCHAR   *pszReqAttribs,
   const WCHAR   *pszOptAttribs,
   T            **ppObject,
   IUnknown      *IUnknown = NULL,
   DWORD          dwClsCtxt = CLSCTX_ALL,
);

Parameters

pszCategoryId
[in] The null-terminated string category ID on which to base the new token.
pszReqAttribs
[in] The null-terminated string of required attributes for the token.
pszOptAttribs
[in] The null-terminated string of optional attributes for the token.
ppObject
[out, iid_is(riid)] Address of pointer variable that receives the interface pointer requested in riid. Upon successful return, ppObject contains the requested interface pointer. If the object does not support the interface specified in riid, the implementation must set ppObject to NULL.
IUnknown
[in] Optional parameter used for creating aggregate objects. pUnkOuter is the data for the object. If not specified, the value defaults to NULL.
dwClsCtxt
[in] Context in which the code that manages the newly created object will run and should be one of the following values. If not specified, the value defaults to CLSCTX_ALL.
  • CLSCTX_INPROC_SERVER
  • CLSCTX_INPROC_HANDLER
  • CLSCTX_LOCAL_SERVER
  • CLSCTX_REMOTE_SERVER

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpCreateDefaultObjectFromCategoryId

 The Microsoft.com Speech website

SpCreateDefaultObjectFromCategoryId

SpCreateDefaultObjectFromCategoryId creates the object instance from the default object token of a specified category.

Found in: sphelper.h

SpCreateDefaultObjectFromCategoryId(
   const WCHAR   *pszCategoryId,
   T            **ppObject,
   IUnknown      *pUnkOuter = NULL,
   DWORD          dwClsCtxt = CLSCTX_ALL
);

Parameters

pszCategoryId
[in] The type of object token to create.
ppObject
[out] The object being created.
pUnkOuter
[in] Optional parameter used for creating aggregate objects. pUnkOuter is the data for the object. If not specified, the value defaults to NULL.
dwClsCtxt
[in] The type of aggregate object being created. If pUnkOuter is not NULL, this must be supplied. If not specified otherwise, the value defaults to all object types.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.

Example

    CComPtr cpAudio;

    hr = SpCreateDefaultObjectFromCategoryId(SPCAT_AUDIOIN, &cpAudio);


Helper SpCreateNewToken

 The Microsoft.com Speech website

SpCreateNewToken (by CategoryId)

SpCreateNewToken creates a token coercively. The token is created with the specified name, if provided. Otherwise it will automatically generate both a keyname and name.

Found in: sphelper.h

SpCreateNewToken(
   const WCHAR       *pszCategoryId,
   const WCHAR       *pszTokenKeyName,
   ISpObjectToken   **ppToken
);

Parameters

pszCategoryId
[in] The null-terminated string indicating the category ID.
pszTokenKeyName
[in out] The token name being created. If NULL, a unique token key name will be generated. If a name is provided it will append "Tokens" before it.
ppToken
[out] The newly created token. The token will be created, if one does not currently exist.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.

Example

The following code snippet illustrates the use of SpCreateNewToken (by Category Id) with the SPCAT_RECOPROFILE category.

    HRESULT hr = S_OK;

    // create a new recognition profile
    hr = SpCreateNewToken(SPCAT_RECOPROFILES, NULL, &cpObjectToken);
    // Check hr


Helper SpCreateNewToken

 The Microsoft.com Speech website

SpCreateNewToken (by TokenId)

SpCreateNewToken creates a token forcefully.

Found in: sphelper.h

inline HRESULT SpCreateNewToken(
   const WCHAR       *pszTokenId,
   ISpObjectToken   **ppToken
);

Parameters

pszTokenId
[in] The null-terminated string indicating the token IDw.
ppToken
[out] The token being created. It is created with CoCreateInstance and must be manually freed when no longer required.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpCreateNewTokenEx

 The Microsoft.com Speech website

SpCreateNewTokenEx (by CategoryId)

SpCreateNewTokenEx creates a new token using the category ID and a token name. It subsequently completes the information with the helper function SpSetCommonTokenData.

Found in: sphelper.h

SpCreateNewTokenEx(
   const WCHAR     *pszCategoryId,
   const WCHAR     *pszTokenKeyName,
   const CLSID     *pclsid,
   const WCHAR     *pszLangIndependentName,
   LANGID           langid,
   const WCHAR     *pszLangDependentName,
   ISpObjectToken **ppToken,
   ISpDataKey     **ppDataKeyAttribs
);

Parameters

pszCategoryId
[in] The null-terminated string indicating the category ID.
pszTokenKeyName
[in out] The token name being created. If NULL, a unique token key name will be generated. If a name is provided it will append "Tokens" before it.
pclsid
[in] Sets the token's CLSID, if specified.
pszLangIndependentName
[in] Sets the null-terminated token language dependent name.
langid
[in] The language ID of the word. May be zero to indicate the word can be of any LANGID.
pszLangDependentName
[in] Sets the null-terminated token language dependent name.
ppToken
[out] The newly created token. The token is created if it currently does not exist.
ppDataKeyAttribs
[in] Opens the attributes key. The key is created if it does not currently exist. May be NULL if not needed.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpCreateNewTokenEx

 The Microsoft.com Speech website

SpCreateNewTokenEx (by TokenId)

SpCreateNewTokenEx creates a new token using the category ID and a token name. It subsequently completes the information with the helper function SpSetCommonTokenData.

Found in: sphelper.h

SpCreateNewTokenEx(
   const WCHAR     *pszTokenId,
   const CLSID     *pclsid,
   const WCHAR     *pszLangIndependentName,
   LANGID           langid,
   const WCHAR     *pszLangDependentName,
   ISpObjectToken **ppToken,
   ISpDataKey     **ppDataKeyAttribs
);

Parameters

pszTokenId
[in] The null-terminated string indicating the token ID.
pclsid
[in] Sets the token's CLSID, if specified.
pszLangIndependentName
[in] Sets the null-terminated token language dependent name.
langid
[in] The language ID of the word. May be zero to indicate that the word can be of any LANGID.
pszLangDependentName
[in] Sets the null-terminated token language dependent name.
ppToken
[out] The newly created token. The token is created if it currently does not exist.
ppDataKeyAttribs
[in] Opens the attributes key. The key is created if it does not currently exist. May be NULL if not needed.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpCreateObjectFromSubToken

 The Microsoft.com Speech website

SpCreateObjectFromSubToken

SpCreateObjectFromSubToken creates an object from a specified subtoken.

Found in: spddkhlp.h

SpCreateObjectFromSubToken(
   ISpObjectToken   *pToken,
   const WCHAR      *pszSubKeyName,
   T               **ppObject,
   IUnknown          pUnkOuter = NULL,
   DWORD            *dwClsCtxt = CLSCTX_ALL
);

Parameters

pToken
[in] Address of a pointer to an ISpObjectToken object containing the information associated with the tokens being added.
pszSubKeyName
[in] Address of a null-terminated string specifying the name of the subkey of the pToken's corresponding datakey to open.
ppObject
[out, iid_is(riid)] Address of pointer variable that receives the interface pointer requested in riid. Upon successful return, ppObject contains the requested interface pointer. If the object does not support the interface specified in riid, the implementation must set ppObject to NULL.
IUnknown
[in] Optional parameter used for creating aggregate objects. pUnkOuter is the data for the object. If not specified, the value defaults to NULL.
dwClsCtxt
[in] Context in which the code that manages the newly created object will run and should be one of the following values. If not specified, the value defaults to CLSCTX_ALL.
  • CLSCTX_INPROC_SERVER
  • CLSCTX_INPROC_HANDLER
  • CLSCTX_LOCAL_SERVER
  • CLSCTX_REMOTE_SERVER

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpCreateObjectFromToken

 The Microsoft.com Speech website

SpCreateObjectFromToken

SpCreateObjectFromToken creates an object instance from a specified object token.

Found in: sphelper.h

SpCreateObjectFromToken(
   ISpObjectToken  *pToken,
   T              **ppObject,
   IUnknown        *pUnkOuter = NULL,
   DWORD            dwClsCtxt = CLSCTX_ALL
);

Parameters

pToken
[in] The type of object token to create.
ppObject
[out] the object instance being created.
pUnkOuter
[in] Optional parameter used for creating aggregate objects. pUnkOuter is the data for the object. If not specified, the value defaults to NULL.
dwClsCtxt
[in] The type of aggregate object being created. If pUnkOuter is not NULL, this must be supplied. If not specified otherwise, the value defaults to all object types.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpCreateObjectFromTokenId

 The Microsoft.com Speech website

SpCreateObjectFromTokenId

SpCreateObjectFromTokenId creates an object instance from a specified object token ID.

Found in: sphelper.h

SpCreateObjectFromTokenId(
   const WCHAR   *pszTokenId,
   T            **ppObject,
   IUnknown      *pUnkOuter = NULL,
   DWORD          dwClsCtxt = CLSCTX_ALL
);

Parameters

pszTokenId
[in] The type of object token to create.
ppObject
[out] The object being created.
pUnkOuter
[in] Optional parameter used for creating aggregate objects. pUnkOuter is the data for the object. If not specified, the value defaults to NULL.
dwClsCtxt
[in] The type of aggregate object being created. If pUnkOuter is not NULL, this must be supplied. If not specified otherwise, the value defaults to all object types.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpCreatePhoneConverter

 The Microsoft.com Speech website

SpCreatePhoneConverter

SpCreatePhoneConverter creates a directly converted phone. Calls the helper function SpCreateBestObject with the category of SPCAT_PHONECONVERTERS.

Found in: sphelper.h

SpCreatePhoneConverter(
   LANGID            *langid,
   const WCHAR       *pszReqAttribs,
   const WCHAR       *pszOptAttribs,
   ISpPhoneConverter  ppPhoneConverter
);

Parameters

langid
[in] The language ID of the word. May not be NULL or zero.
pszReqAttribs
[in] The null-terminated string of required attributes for the token.
pszOptAttribs
[in] The null-terminated string of optional attributes for the token.
ppPhoneConverter
[out] The converted phone interface.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | langid equals zero.
FAILED (hr) | Appropriate error message.


Helper SpEnumTokens

 The Microsoft.com Speech website

SpEnumTokens

SpEnumTokens enumerates the tokens for the specified category.

Found in: sphelper.h

inline HRESULT SpEnumTokens(
   const WCHAR            *pszCategoryId,
   const WCHAR            *pszReqAttribs,
   const WCHAR            *pszOptAttribs,
   IEnumSpObjectTokens   **ppEnum
);

Parameters

pszCategoryId
[in] The null-terminated string category ID on which to base the enumerations.
pszReqAttribs
[in] The null-terminated string of the required attributes for the token.
pszOptAttribs
[in] The null-terminated string of the optional attributes for the token. The order in which the tokens are listed in ppEnum is based on the order that they match pszOptAttribs.
ppEnum
[out] The enumerated list of tokens found.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpFindBestToken

 The Microsoft.com Speech website

SpFindBestToken

SpFindBestToken enumerates the token category and finds the single best match (if any) based on the required and optional attributes.

Found in: sphelper.h

SpFindBestToken(
   const WCHAR       *pszCategoryId,
   const WCHAR       *pszReqAttribs,
   const WCHAR       *pszOptAttribs,
   ISpObjectToken   **ppObjectToken
);

Parameters

pszCategoryId
[in] The null-terminated string category ID on which to base the enumerations.
pszReqAttribs
[in] The null-terminated string of the required attributes for the token.
pszOptAttribs
[in] The null-terminated string of the optional attributes for the token. The order in which the tokens are listed in ppObjectToken is based on the order they match pszOptAttribs.
ppObjectToken
[out] The single best matched token found.

Return values

Value | Description
S_OK | Function completed successfully.
SPERR_NOT_FOUND | No items match the given attributes.
FAILED (hr) | Appropriate error message.


Helper SpGetCategoryFromId

 The Microsoft.com Speech website

SpGetCategoryFromId

SpGetCategoryFromId creates an object of CLSID_SpObjectTokenCategory. This function assists in locating and creating an object token category without having to search or change the registry directly.

Found in: sphelper.h

SpGetCategoryFromId(
   const WCHAR       *pszCategoryId,
   ISpObjectToken   **ppCategory,
   BOOL              *fCreateIfNotExist = FALSE
);

Parameters

pszCategoryId
[in] The string indicating the CategoryId.
ppCategory
[out] The token being created. It is created with CoCreateInstance and must be manually freed when no longer required.
fCreateIfNotExist
[in] An optional parameter allowing the object to be created if one does not currently exist. The default is FALSE unless otherwise specified.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpGetDefaultTokenFromCategoryId

 The Microsoft.com Speech website

SpGetDefaultTokenFromCategoryId

SpGetDefaultTokenFromCategoryId gets the default token for the specified category ID.

Found in: sphelper.h

inline HRESULT SpGetDefaultTokenFromCategoryId(
   const WCHAR       *pszCategoryId,
   ISpObjectToken   **ppToken,
   BOOL              *fCreateCategoryIfNotExist = TRUE
);

Parameters

pszCategoryId
[in] The null-terminated string for the category ID.
ppToken
[out] The default token for the category ID.
fCreateCategoryIfNotExist
[in] An optional parameter allowing the category to be created if one does not currently exist. By default, the value is TRUE unless otherwise indicated.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpGetDefaultTokenIdFromCategoryId

 The Microsoft.com Speech website

SpGetDefaultTokenIdFromCategoryId

SpGetDefaultTokenIdFromCategoryId gets the default token ID for the specified category.

Found in: sphelper.h

inline HRESULT SpGetDefaultTokenIdFromCategoryId(
   const WCHAR   *pszCategoryId,
   WCHAR        **ppszTokenId
);

Parameters

pszCategoryId
[in] The null-terminated string indicating the category ID.
ppszTokenId
[out] The null-terminated string ID of the owning default token.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpGetDescription

 The Microsoft.com Speech website

SpGetDescription

SpGetDescription passes back the textual description associated with the specified token.

Found in: sphelper.h

SpGetDescription(
   ISpObjectToken   *pObjToken,
   WCHAR           **ppszDescription,
   LANGID           *Language = SpGetUserDefaultUILanguage()
);

Parameters

pObjToken
[in] The object token of the target resource.
ppszDescription
[out] A null-terminated string containing the resource description.
Language
[in] The language ID for the resource. Language is optional and if omitted the default language will be used.

Return values

This helper function calls ISpDataKey::GetStringValue. See those return values.


Helper SpGetSubTokenFromToken

 The Microsoft.com Speech website

SpGetSubTokenFromToken

SpGetSubTokenFromToken creates a subtoken from a token.

Found in: spddkhlp.h

SpGetSubTokenFromToken(
   ISpObjectToken   *pToken,
   const WCHAR      *pszSubKeyName,
   ISpObjectToken  **ppToken,
   BOOL              fCreateIfNotExist = FALSE
);

Parameters

pToken
[in] The object token from which to create the subtoken. If not present, this token will be created if fCreateIfNotExist is TRUE.
pszSubKeyName
[out] The name of the subtoken to use.
ppToken
[in] The newly created subtoken.
fCreateIfNotExist
[in] Optional Boolean indicating that the token is to be created if one does not currently exist. TRUE allows the creation. The default value FALSE does not.

Return values

Value | Description
S_OK | Function completed successfully.
E_POINTER | At least one of the pointers pToken, pszSubKeyName, or ppToken is invalid or bad.
FAILED (hr) | Appropriate error message.


Helper SpGetTokenFromId

 The Microsoft.com Speech website

SpGetTokenFromId

SpGetTokenFromId creates an object token of CLSID_SpObjectToken. This function assists in locating and creating an object token without having to search or change the registry directly.

Found in: sphelper.h

inline HRESULT SpGetTokenFromId(
   const WCHAR       *pszTokenId,
   ISpObjectToken   **ppToken,
   BOOL              *fCreateIfNotExist = FALSE
);

Parameters

pszTokenId
[in] The null-terminated string indicating the token ID.
ppToken
[out] The token being created. It is created with CoCreateInstance and must be manually freed when no longer required.
fCreateIfNotExist
[in] An optional parameter allowing the object to be created if one does not currently exist. The default is FALSE unless otherwise specified.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpGetUserDefaultUILanguage

 The Microsoft.com Speech website

SpGetUserDefaultUILanguage

SpGetUserDefaultUILanguage returns the default user interface language.

Found in: sphelper.h

inline LANGID SpGetUserDefaultUILanguage ( void );

Parameters

None.

Return values

Returns the default language. If the attempt fails to find the default language from the specific operating system, the default language from SAPI is returned instead.


Helper SpSetCommonTokenData

 The Microsoft.com Speech website

SpSetCommonTokenData

SpSetCommonTokenData fills in the token data with the information provided by the parameters. Unused parameters must be NULL.

Found in: sphelper.h

SpSetCommonTokenData(
   ISpObjectToken   *pToken,
   const CLSID      *pclsid,
   const WCHAR      *pszLangIndependentName,
   LANGID            langid,
   const WCHAR      *pszLangDependentName,
   ISpDataKey      **ppDataKeyAttribs
);

Parameters

pToken
[in] Address of a pointer to an ISpObjectToken object containing the information associated with the tokens being added.
pclsid
[in] Sets the token's CLSID, if specified.
pszLangIndependentName
[in] Sets the token's language independent name.
langid
[in] The language ID of the word. May be zero to indicate the word can be of any LANGID.
pszLangDependentName
[in] Sets the token's language dependent name.
ppDataKeyAttribs
[in] Opens the attributes key. The key is created if it does not currently exist. May be NULL if not needed.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpSetDefaultTokenForCategoryId

 The Microsoft.com Speech website

SpSetDefaultTokenForCategoryId

SpSetDefaultTokenForCategoryId sets the default token for the specified category ID.

Found in: sphelper.h

inline HRESULT SpSetDefaultTokenForCategoryId(
   const WCHAR      *pszCategoryId,
   ISpObjectToken   *pToken
);

Parameters

pszCategoryId
[in] The null-terminated string for the category ID.
pToken
[in] The token to be set as the default for the category ID.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SpSetDefaultTokenIdForCategoryId

 The Microsoft.com Speech website

SpSetDefaultTokenIdForCategoryId

SpSetDefaultTokenIdForCategoryId sets a specific token ID as the default for the specified category ID.

Found in: sphelper.h

inline HRESULT SpSetDefaultTokenIdForCategoryId(
   const WCHAR    *pszCategoryId,
   const WCHAR    *pszTokenId
);

Parameters

pszCategoryId
[in] The null-terminated token ID string.
pszTokenId
[in] The null-terminated token ID that will be set as default token ID for this specific category.

Return values

Value | Description
S_OK | Function completed successfully.
FAILED (hr) | Appropriate error message.


Helper SPFEI

 The Microsoft.com Speech website

SPFEI

SPFEI casts a specified value into a 64-bit type.

For a list of the supported SPEI_ord event types, see SPEVENTENUM

Found in: sapi.idl

SPFEI(
   void   *SPEI_ord
);

Parameters

SPEI_ord
[in, out] The value to re-cast. The new value is passed back.

Return values

No error code is returned.

Related Helper Macros

Macro | Description
SPFEI_FLAGCHECK | Retrieve reserved flags for the event interest enum.
SPEI_ALL_EVENTS | Retrieve all possible events flags.
SPFEI_ALL_TTS_EVENTS | Retrieve all possible TTS event flags.
SPFEI_ALL_SR_EVENTS | Retrieve all possible SR event flags.

Remarks

The SPFEI_FLAGCHECK macro retrieves the flags that must always be included in any event interest. SAPI utilizes the SPFEI_FLAGCHECK macro to help developers avoid mistakes by using event interest related methods (see ISpEventSource::SetInterest and ISpRecoContext::SetVoicePurgeEvent) with the actual enumeration instead of the SPFEI() macro.

Example

Here is an example of the SPFEI macro used to set SR event interest

hr = g_cpRecoCtxt->SetInterest(SPFEI(SPEI_RECOGNITION), SPFEI(SPEI_RECOGNITION));

Here is an example of the SPFEI_ALL_SR_EVENTS macro used to set SR event interest in all events. Note this is an example and not recommended practice.

hr = g_cpRecoCtxt->SetInterest(SPFEI_ALL_SR_EVENTS, SPFEI_ALL_SR_EVENTS);


Helper SPBindToFile

 The Microsoft.com Speech website

SPBindToFile

SPBindToFile binds the audio stream to the specified file.

Found in: sphelper.h

SPBindToFile(
   LPCWSTR               pFileName,
   SPFILEMODE            eMode,
   ISpStream           **ppStream,
   const GUID           *pFormatId = NULL,
   const WAVEFORMATEX   *pWaveFormatEx = NULL,
   ULONGLONG            *ullEventInterest = SPFEI_ALL_EVENTS
);

Parameters

pFileName
[in] Address of a null-terminated string containing the file name of the file to bind the stream to.
eMode
[in] Flag of the type SPFILEMODE to define the file opening mode. When opening an audio wave file, this must be SPFM_OPEN_READONLY or SPFM_CREATE_ALWAYS, otherwise the call will fail.
ppStream
[in, out] The address of an ISpStream pointer. If the function succeeds, this value is filled in with the newly created ISpStream interface.
pFormatId
[in] The data format identifier associated with the stream. This can be NULL (default) and the format will be determined from the supplied wave file, if the file has the '.wav' extension. If it doesn't, the file is assumed to be a text file.
pWaveFormatEx
[in] Address of the WAVEFORMATEX structure that contains the wave file format information. If guidFormatId is SPDFID_WaveFormatEx, this must point to a valid WAVEFORMATEX structure. For other formats, it should be NULL.
ullEventInterest
[in] Flags of type SPEVENTENUM for the events wanted.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | One or more arguments are invalid.
SPERR_ALREADY_INITIALIZED | The object has already been initialized.


Helper SpClearEvent

 The Microsoft.com Speech website

SpClearEvent

SpClearEvent clears an event structure. May be used by clients not using CSpEvent class.

Found in: sphelper.h

inline void SpClearEvent(
   SPEVENT   *pe
);

Parameters

pe
[in] The event to clear. Events of types SPET_LPARAM_IS_POINTER, SPET_LPARAM_IS_STRING, SPET_LPARAM_IS_TOKEN, or SPET_LPARAM_IS_OBJECT have the associated data in pe->lParam deallocated first.

Return values

No error code is returned.


Helper SpConvertStreamFormatEnum

 The Microsoft.com Speech website

SpConvertStreamFormatEnum

SpConvertStreamFormatEnum converts the specified stream format into a fully populated wave format structure.

Found in: sphelper.h

SpConvertStreamFormatEnum(
   SPSTREAMFORMAT   eFormat,
   GUID            *pFormatId,
   WAVEFORMATEX   **ppCoMemWaveFormatEx
);

Parameters

eFormat
[in] The requested stream format. Must be a valid SPSTREAMFORMAT value of SPSF_8kHz8BitMono or greater.
pFormatId
[in, out] The GUID of the new format. May be GUID_NULL if an error occurred.
ppCoMemWaveFormatEx
[out] The populated WAVEFORMATEX structure specified by the supplied SPSTREAMFORMAT.

Return values

Value | Description
S_OK | Function completed successfully.
E_OUTOFMEMORY | Exceeded available memory.
E_INVALIDARG | Either pFormatId or ppCoMemWaveFormatEx is invalid or bad. Alternatively, the specified format is not recognized.
FAILED(hr) | Appropriate error message.


Helper SpEventSerializeSize

 The Microsoft.com Speech website

SpEventSerializeSize

SpEventSerializeSize calculates the required size of a buffer to serialize an event. The call must specify which type of serialized event.

Found in: sphelper.h

template <class T>
inline ULONG SpEventSerializeSize(
   const SPEVENT   *pEvent
);

Parameters

pEvent
[in] The event structure to calculate the size of. pEvent must be either SPSERIALIZEDEVENT or SPSERIALIZEDEVENT64.

Return values

Size, in bytes, required to serialize the event.


Helper SpInitEvent

 The Microsoft.com Speech website

SpInitEvent

SpInitEvent clears the event structure.

Found in: sphelper.h

inline void SpInitEvent(
   SPEVENT   *pe
);

Parameters

pe
[in] The event to clear.

Return values

No error code is returned.


Helper SpSetDescription

 The Microsoft.com Speech website

SpSetDescription

SpSetDescription sets the string value for the specified token.

Found in: sphelper.h

inline HRESULT SpSetDescription(
   ISpObjectToken  *pObjToken,
   const WCHAR     *pszDescription,
   LANGID           Language = SpGetUserDefaultUILanguage(),
   BOOL             fSetLangIndependentId = TRUE
);

Parameters

pObjToken
[in] The object token of the target resource.
pszDescription
[in] A null-terminated string containing the resource description.
Language
[in] The language ID for the resource. Language is optional and if omitted the default language will be used.
fSetLangIndependentId
[in] Boolean indicating whether the language independent ID is also changed. TRUE, changes it; FALSE does not. If fSetLangIndependentId omitted the default TRUE will be used.

Return values

Value | Description
S_OK | Function completed successfully.
E_INVALIDARG | Either pszDescription or the language description is invalid or bad.
FAILED(hr) | Appropriate error message.


SAPI Object classes

This section covers the following topics:

  • SAPI Application Object Classes
  • SAPI DDK Object Classes


SAPI Application Object Classes

Object class | Related interfaces
SpNotifyTranslator | ISpNotifySink ISpNotifyTranslator
SpObjectTokenCategory | ISpObjectTokenCategory
SpObjectTokenEnum | IEnumSpObjectTokens
SpObjectToken | ISpObjectToken ISpDataKey
SpDataKey | ISpDataKey
SpResourceManager | IServiceProvider ISpResourceManager ISpTaskManager
SpStreamFormatConverter | IStream ISpStreamFormat ISpAudio
SpMMAudioEnum | IEnumSpObjectTokens
SpMMAudioIn | ISpAudio IStream ISpStreamFormat ISpEventSource ISpEventSink ISpObjectWithToken ISpMMSysAudio
SpMMAudioOut | ISpAudio IStream ISpStreamFormat ISpEventSource ISpEventSink ISpObjectWithToken ISpMMSysAudio
SpRecPlayAudio | ISpAudio IStream ISpStreamFormat ISpEventSource ISpEventSink ISpObjectWithToken
SpStream | IStream ISpStreamFormat IStream
SpVoice | ISpVoice ISpEventSource ISpNotifySource
SpSharedRecognizer | ISpRecognizer
SpInprocRecognizer | ISpRecognizer
SpRecoContext | ISpRecoContext ISpEventSource ISpNotifySource
SpSharedRecoContext | ISpRecoContext ISpEventSource ISpNotifySource
SpRecoGrammar | ISpRecoGrammar ISpGrammarBuilder
SpRecoResult | ISpRecoResult ISpPhrase
SpPhraseAlt | ISpPhraseAlt ISpPhrase
SpLexicon | ISpLexicon
SpUnCompressedLexicon | ISpLexicon ISpObjectWithToken
SpCompressedLexicon | ISpLexicon ISpObjectWithToken
SpPhoneConverter | ISpPhoneConverter ISpObjectWithToken


SPNotifyTranslator

SPNotifyTranslator |
 | ISpNotifySink
 | ISpNotifyTranslator


SpObjectTokenCategory

SpObjectTokenCategory |
 | ISpDataKey
 | ISpObjectTokenCategory


SpObjectTokenEnum

SpObjectTokenEnum |
 | IEnumSpObjectTokens


SpObjectToken

SpObjectToken |
 | ISpObjectToken
 | ISpDataKey


SpDataKey

SpDataKey |
 | ISpDataKey


SpResourceManager

SpResourceManager |
 | IServiceProvider
 | ISpResourceManager
 | ISpTaskManager


SpStreamFormatConverter

SpStreamFormatConverter |
 | IStream
 | ISpStreamFormat
 | ISpAudio


SpMMAudioEnum

SpMMAudioEnum |
 | IEnumSpObjectTokens


SpMMAudioIn

The SAPI implementation of the SpMMAudioIn object supports the following UI through the ISpTokenUI interface if the object has a Windows mixer associated with it:

  • SPDUI_AudioProperties - Displays advanced UI allowing user selection of the input line.
  • SPDUI_AudioVolume - Displays the Windows mixer allowing user adjustment of audio volume.
SpMMAudioIn |
 | IStream
 | ISpStreamFormat
 | ISpEventSource
 | ISpObjectWithToken
 | ISpMMSysAudio


SpMMAudioOut

The SAPI implementation of the SpMMAudioOut object supports the following UI through the ISpTokenUI interface if the object has a Windows mixer associated with it:

  • SPDUI_AudioVolume - Displays the Windows mixer allowing user adjustment of audio volume.
SpMMAudioOut |
 | IStream
 | ISpStreamFormat
 | ISpEventSource
 | ISpObjectWithToken
 | ISpMMSysAudio


SpRecPlayAudio

SpRecPlayAudio |
 | ISpAudio
 | IStream
 | ISpStreamFormat
 | ISpEventSink
 | ISpObjectWithToken


SpStream

SpStream |
 | ISpStream
 | ISpStreamFormat
 | IStream


SpVoice

SpVoice |
 | ISpVoice
 | ISpEventSource
 | ISpNotifySource

An application creates the SpVoice object and uses the ISpVoice interface to submit and control speech synthesis. Applications can speak text strings, text files, and audio files. Although this object is named the "SpVoice", it is actually a much higher-level object than a single voice. Conceptually, it is an object which accepts input data streams which are then rendered to the specified output, potentially using multiple speech synthesis voices in the process. Each SpVoice instance contains its own queue of input streams (usually just text) and its own output stream (usually an audio device). When an application calls ISpVoice::Speak, another item is added to the end of the SpVoice's queue.

Basic Synthesis

The main speech synthesis method is ISpVoice::Speak. Almost everything having to do with controlling synthesis (for example, rate, pitch, and volume) is performed by this single function. This function can speak plain text, or the application can mark up the text using synthesis markup tags. The speak method allows the application to specify weather the call should be synchronous or asynchronous. If the call is synchronous, the Speak method will not return until all of the text has been rendered. Speak returns immediately for asynchronous Speak calls, and the text is rendered on a background thread.

ISpVoice::SpeakStream is similar to the Speak method, but it allows streams of text or audio data to be added to the rendering queue.

Overriding Defaults

SAPI will automatically use the default voice and default audio output device if the application does not specify otherwise. The output can be controlled by the application through ISpVoice::SetOutput. The default voice can be overridden in one of two ways: The application can call ISpVoice::SetVoice or it could speak a <VOICE> synthesis markup tag.

Audio Device Sharing

When an SpVoice object is rendering to an audio device (as opposed to a stream), it will attempt to cooperate with other SpVoice objects that are sharing the same device based on the SpVoice's priority. By default, a voice is set to SPVPRI_NORMAL which means that it will wait until other voices in the system have completed before it will begin rendering its input queue. A voice set to SPVPRI_ALERT will interrupt a normal priority voice by stopping the normal voice, rendering its own queue, and then restarting the normal priority voice. An SpVoice with a priority of SPVPRI_OVER will simply render its data immediately even if another voice is currently speaking (they would both speak at the same time).

Applications can control the priority of a voice by calling ISpVoice::SetPriority.

Rendering to Streams

The SpVoice can render data to any object that implements ISpStreamFormat, which is a simple derivative of the COM standard IStream. The SpStream object is provided to allow easy conversion of existing IStreams to support ISpStreamFormat or to read or write .WAV or other files. Applications can ISpVoice::SetOutput to force the SpVoice to render to a stream. When rendering to a stream, the voice will render the data as quickly as possible.

Synthesis Events

The SpVoice implements the ISpEventSource interface. It forwards events back to the application when the corresponding audio data has been rendered to the output device. Examples of events are reaching a word boundary, speaking a phoneme, reaching a bookmark, etc. Some applications can simply be notified when events occur and then call ISpVoice::GetStatus to determine the current stat of the SpVoice object. More complex applications may need to queue events. See the documentation for ISpEventSource for information on setting interest in events.

How Created

Create the SpVoice object by calling ::CoCreateInstance with CLSID_SpVoice.


SpSharedRecognizer

SpSharedRecognizer |
 | ISpRecognizer


SpInprocRecognizer

SpInprocRecognizer |
 | ISpRecognizer


SpRecoContext

SpRecoContext |
 | ISpRecoContext
 | ISpEventSource
 | ISpNotifySource


SpSharedRecoContext

SpSharedRecoContext |
 | ISpRecoContext
 | ISpEventSource
 | ISpNotifySource


SpRecoGrammar

SpRecoGrammar |
 | ISpRecoGrammar
 | ISpGrammarBuilder


SpRecoResult

SpRecoResult |
 | ISpRecoResult
 | ISpPhrase


SpPhraseAlt

SpPhraseAlt |
 | ISpPhraseAlt
 | ISpPhrase


SpLexicon

SpLexicon |
 | ISpLexicon

The Lexicon database is a repository of words and word-related information such as pronunciations and parts of speech. The SAPI lexicon interface provides application, CSR engine, and TTS engine developers a standard method with which to create, access, modify, and synchronize with lexicons.

Types of Lexicons

There are two types of custom lexicons supported by lexicon interface: user and application. The user lexicon stores words specific to a user. It is a read/write lexicon and is shared among all applications. The application lexicon is supplied by the application and stores words specific to the application. The application supplied lexicons are read-only. Application lexicons ensure that the vocabulary used by the application is well represented in the lexicon.

Apart from custom lexicons, the lexicon interface provides access to vendor, morph, and letter-to-sound lexicons that Microsoft ships with SAPI. Vendor lexicons are large vocabulary lexicons holding words and their pronunciations and parts of speech. The morph lexicons derive pronunciations using the data in the vendor lexicon. The letter-to-sound lexicon computes the pronunciation of a word from its spelling.

User lexicons override application lexicons and engine private lexicons. You cannot change application lexicons from the SpLexicon object.

Modifying and Viewing the Contents of a Lexicon

An application can modify the user lexicon by using the calls ISpLexicon::AddPronunciation and ISpLexicon::RemovePronunciation. The function ISpLexicon::GetWords allows the caller to see what words are in the user or app lexicon. To obtain the pronunciation of a given word, the client would call ISpLexicon::GetPronunciations. There is not a standard method for applications to access the lexicons that are supplied by the engine.

Synchronizing Changes to a Lexicon

The lexicon interface provides methods to synchronize changes in lexicons using a lexicon generation ID, which is a sort of time-stamp on the lexicon. These changes in the lexicon are a result of modifications to user lexicons or for the installation or uninstallation of application lexicons. The client can get the current generation by calling ISpLexicon::GetGeneration and can see the change history since a given generation by calling ISpLexicon::GetGenerationChange. A speech recognition engine might want to use the synchronization to update its private stores with the changes made to the custom lexicons while the client has been offline. For example, SR engines can update their language models with changes made to the custom lexicons while the SR engine had been off-line.

How Created

An SpLexicon can be created by calling ::CoCreateInstance with CLSID_SpLexicon.


SpUnCompressedLexicon

SpUnCompressedLexicon |
 | ISpLexicon
 | ISpObjectWithToken


SpCompressedLexicon

SpCompressedLexicon |
 | ISpLexicon
 | ISpObjectWithToken


SpPhoneConverter

SpPhoneConverter |
 | ISpPhoneConverter
 | ISpObjectWithToken


SAPI DDK Object classes

Object class | Related interfaces
SpDataKey | ISpDataKey ISpRegDataKey
SpObjectTokenEnum | ISpObjectTokenEnumBuilder IEnumSpObjectTokens
SpPhraseBuilder | ISpPhraseBuilder ISpPhrase
SpITNProcessor | ISpITNProcessor
SpGrammarComplier | ISpGrammarCompiler
SpGramCompBackend | ISpGramCompBackend ISpGrammarBuilder
SpSREngineSite | ISpSREngineSite
SpTTSEngineSite | ISpTTSEngineSite

Abstract objects for engine developers

Object class | Related interfaces
SpSREngine | ISpSREngine ISpObjectWithToken
SpTTSEngine | ISpTTSEngine ISpObjectWithToken
SpSRAlternates | ISpSRAlternates
SpRecoExtension | _ISpPrivateEngineCall Implements extended interface(s)
SpTokenUI | ISpTokenUI


SpDataKey (DDK)

SpDataKey |
 | ISpDataKey
 | ISpRegDataKey


SpObjectTokenEnum (DDK)

SpObjectTokenEnum |
 | ISpObjectTokenEnumBuilder
 | IEnumSpObjectTokens


SpPhraseBuilder (DDK)

SpPhraseBuilder |
 | ISpPhraseBuilder
 | ISpPhrase


SpITNProcessor (DDK)

SpITNProcessor |
 | ISpITNProcessor


SpGrammarComplier (DDK)

SpGrammarComplier |
 | ISpGrammarCompiler


SpGramCompBackend (DDK)

SpGramCompBackend |
 | ISpGramCompBackend
 | ISpGrammarBuilder


SpSREngineSite (DDK)

SpSREngineSite |
 | ISpSREngineSite


SpTTSEngineSite (DDK)

SpTTSEngineSite |
 | ISpTTSEngineSite


SpSREngine (DDK)

SpSREngine |
 | ISpSREngine
 | ISpObjectWithToken


SpTTSEngine (DDK)

SpTTSEngine |
 | ISpTTSEngine
 | ISpObjectWithToken


SpSRAlternates (DDK)

SpSRAlternates |
 | ISpSRAlternates


SpRecoExtension (DDK)

SpRecoExtension |
 | ISpPrivateEngineCall


SpTokenUI (DDK)

SpTokenUI |
 | ISpTokenUI


Miscellanea

The follow sections cover supporting speech functions:

  • Global Parameters
  • COM Class ID List
  • Token Category IDs
  • COM Interface IUnknown
  • Further Reading


Enumerations

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Global Variables

The following variables are of special importance to SAPI developers.

  • SP_MAX_WORD_LENGTH
  • SP_MAX_PRON_LENGTH

The following variables are SAPI-defined UI types

See also ISpTokenUI for a description of how to query if an object supports this UI type (see ISpTokenUI::IsUISupported) or how to display the UI (see ISpTokenUI::DisplayUI).

API-Defined UI Types | Description
SPDUI_EngineProperties | Defines the string for displaying the UI for changing TTS or SR engine's properties on a per-user basis.
SPDUI_AddRemoveWord | Defines the string for displaying UI to modify the lexicon.
SPDUI_UserTraining | Defines the string for displaying an SR Engine's User Training UI.
SPDUI_MicTraining | Defines the string for displaying a speech recognition engine's microphone training UI.
SPDUI_RecoProfileProperties | Defines the string for displaying the properties associated with a specific recognition profile.
SPDUI_AudioProperties | Defines the string for displaying an audio object's properties UI.
SPDUI_AudioVolume | Defines the string for displaying an audio object's volume UI.


SP_MAX_WORD_LENGTH

SP_MAX_WORD_LENGTH defines the maximum length of a word.

const ULONG SP_MAX_WORD_LENGTH = 128;
Functions with input word strings matching or exceeding this limit will return E_INVALIDARG.


SP_MAX_PRON_LENGTH

SP_MAX_PRON_LENGTH defines the maximum length of a phoneme pronunciation. This limit applies to zero-terminated lists of SPPHONEID elements.

const ULONG SP_MAX_PRON_LENGTH = 384;
Functions with SPPHONEID arguments matching or exceeding this limit will return the E_INVALIDARG error.


SPDUI_EngineProperties

SPDUI_EngineProperties defines the string for displaying the UI for changing TTS or SR engine's properties on a per-user basis.

It is not a SAPI 5 compliance requirement for a speech engine to implement UI for SPDUI_EngineProperties.

The Microsoft SR Engine that ships in the SAPI 5 SDK does not support SPDUI_EngineProperties.

When to Implement

When writing a speech engine for the desktop or a graphical environment, users can change settings that should affect all of their recognition profiles, but be specific to each user. For example, the Microsoft Text-To-Speech Engine exposes some inverse-text-normalization (ITN) rules (e.g., comma versus period number delimiter, date format) in their engine properties.

Use Speech properties in Control Panel to change settings for all installed SAPI 5-compliant Text-To-Speech and Speech Recognition Engines. Click Settings to change settings on a per-user/per-engine basis. Use Settings to directly accesses each Engine's Settings UI using SPDUI_EngineProperties. If the engine does not support the Engine Properties UI (see ISpTokenUI::IsUISupported), the Settings button will be unavailable.

When to Access

The application could display a button or menu item for advanced engine properties that accessed SPDUI_EngineProperties (see ISpVoice::DisplayUI). Changes made within the engine properties UI will affect only one engine, and will not affect other users.

#define SPDUI_EngineProperties        L"EngineProperties"

Example

The following code snippet illustrates the use of ISpTokenUI::IsUISupported using SPDUI_EngineProperties.

    HRESULT hr = S_OK;

    // get the default text-to-speech engine token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_VOICES, &cpObjectToken);
    // Check hr

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // check if the default text-to-speech engine has UI for its properties
    hr = cpTokenUI->IsUISupported(SPDUI_EngineProperties, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then default speech text-to-speech engine has UI for its properties

The following code snippet illustrates the use of ISpVoice::DisplayUI using SPDUI_EngineProperties.

    HRESULT hr = S_OK;

    // display engine properties UI for the current TTS engine
    hr = cpVoice->DisplayUI(MY_HWND, MY_APP_VOICE_PROPERTIES, SPDUI_EngineProperties, NULL, NULL);
    // Check hr


SPDUI_AddRemoveWord

SPDUI_AddRemoveWord defines the string for displaying UI to modify the lexicon.

It is not a SAPI 5 compliance requirement for a speech recognition engine to implement UI for SPDUI_AddRemoveWord.

If a speech engine is being written for the desktop or a graphical environment, users can modify the set of words that will be recognized or synthesized (see ISpLexicon).

The SDK sample application Dictation Pad allows the user to modify the lexicon using the Voice menu-->Add/Delete Words... When the user selects this menu item, Dictation Pad directly accesses the default SR Engine's graphical lexicon editor through SPDUI_AddRemoveWord. If the SR Engine does not support the Training UI (see ISpTokenUI::IsUISupported), the menu item will be unavailable.

When to Access

The application could monitor the user's speech experience (correction type and frequency). If the user corrects a recognition by typing a word that is missing from the lexicon (see ISpLexicon), the application could prompt the user to add it to the lexicon using ISpRecognizer::DisplayUI and SPDUI_AddRemoveWord.

#define SPDUI_AddRemoveWord        L"AddRemoveWord"

Example

The following code snippet illustrates the use of ISpTokenUI::IsUISupported using SPDUI_AddRemoveWord.

    HRESULT hr = S_OK;

    // get the default speech recognizer token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_RECOGNIZERS, &cpObjectToken);
    // Check hr

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // check if the default speech recognizer has UI for editing the lexicon
    hr = cpTokenUI->IsUISupported(SPDUI_AddRemoveWord, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then default speech recognizer has UI for modifying the lexicon

The following code snippet illustrates the use of ISpRecognizer::DisplayUI using SPDUI_AddRemoveWord.

    HRESULT hr = S_OK;

    // display lexicon editing UI for the current recognizer
    hr = cpRecognizer->DisplayUI(MY_HWND, MY_APP_MIC_TRAINING, SPDUI_AddRemoveWord, NULL, NULL);
    // Check hr


SPDUI_UserTraining

SPDUI_UserTraining defines the string for displaying an SR Engine's User Training UI.

It is not a SAPI 5 compliance requirement for a speech recognition engine to implement UI for SPDUI_UserTraining.

When to Implement

When writing a speech engine for the desktop or a graphical environment, users can train the engine before the first use, or when recognition accuracy is poor.

Use Speech properties in Control Panel to change settings for all installed SAPI 5-compliant SR Engines. Click Train Profile to change settings on a per-user/per-engine basis. Use Train Profile to directly accesses each Engine's Settings UI using SPDUI_UserTraining. If the SR Engine does not support the training UI (see ISpTokenUI::IsUISupported), the Train Profile button will be unavailable.

When to Access

The application could monitor the user's speech experience by recognition accuracy. If a user encounters too many false recognitions or recognition corrections, the application could ask the user if they would like to perform more recognizer training.

Also, an SR Engine can send a SPEI_REQUEST_UI event to the application if it determines that the user needs to perform additional recognizer training. (see also ISpRecognizer::IsUISupported and ISpRecognizer::DisplayUI)

#define SPDUI_UserTraining        L"UserTraining"

Example

The following code snippet illustrates the use of ISpTokenUI::IsUISupported using SPDUI_UserTraining.

    HRESULT hr = S_OK;

    // get the default speech recognizer token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_RECOGNIZERS, &cpObjectToken);
    // Check hr

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // check if the default speech recognizer has UI for performing User Training
    hr = cpTokenUI->IsUISupported(SPDUI_UserTraining, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then default speech recognizer has UI for User Training

The following code snippet illustrates the use of ISpRecognizer::DisplayUI using SPDUI_UserTraining.

    HRESULT hr = S_OK;

    // display user training UI for the current recognizer
    hr = cpRecognizer->DisplayUI(MY_HWND, MY_APP_USER_TRAINING, SPDUI_UserTraining, NULL, NULL);
    // Check hr


SPDUI_MicTraining

SPDUI_MicTraining defines the string for displaying a speech recognition engine's microphone training UI.

It is not a SAPI 5 compliance requirement for a speechrecognition engine to implement UI for SPDUI_MicTraining.

When to Implement

When writing an SR engine for the desktop or a graphical environment, users might want to adapt the engine to the microphone before first use, or if recognition accuracy is poor. Microphone quality can greatly benefit recognition accuracy, even for a speaker-independent engine.

Use Speech properties in Control Panel to change settings for all installed SAPI 5-compliant SR Engines. Click Configure Microphone on the SR tab to adapt the microphone and directly access each SR Engine's Microphone Training UI using SPDUI_MicTraining. If the SR Engine does not have support for training UI (see ISpTokenUI::IsUISupported), Configure Microphone will be unavailable.

When to Access

The application could monitor the user's speech experience by recognition accuracy. If a user encounters too many false recognitions or recognition corrections, the application could ask the user if they would like to re-adapt the SR engine to their microphone (see ISpRecognizer::DisplayUI).

Also, an SR Engine can send a SPEI_REQUEST_UI event to the application if the engine determines that the input is too quiet or too loud and would like the user to perform microphone training. Typically the UI type will be SPDUI_MicTraining to ensure that the SR Engine is adapted to the current input audio settings.

#define SPDUI_MicTraining        L"MicTraining"

Example

The following code snippet illustrates the use of ISpTokenUI::IsUISupported using SPDUI_MicTraining.

    HRESULT hr = S_OK;

    // get the default speech recognizer token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_RECOGNIZERS, &cpObjectToken);
    // Check hr

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // check if the default speech recognizer has UI for performing Microphone Training
    hr = cpTokenUI->IsUISupported(SPDUI_MicTraining, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then default speech recognizer has UI for Microphone Training

The following code snippet illustrates the use of ISpRecognizer::DisplayUI using SPDUI_MicTraining.

    HRESULT hr = S_OK;

    // display microphone training UI for the current recognizer
    hr = cpRecognizer->DisplayUI(MY_HWND, MY_APP_MIC_TRAINING, SPDUI_MicTraining, NULL, NULL);
    // Check hr


SPDUI_AudioVolume

 The Microsoft.com Speech website

SPDUI_RecoProfileProperties

SPDUI_RecoProfileProperties defines the string for displaying the properties associated with a specific recognition profile.

It is not a SAPI 5 compliance requirement for a speech recognition engine to implement UI for SPDUI_RecoProfileProperties.

For more information about Recognition Profiles, see the Object Tokens and Registry Settings White Paper.

When to Implement

When writing a speech engine for the desktop or a graphical environment, users can modify settings for specific runtime environments (e.g., mobile versus desktop, noisy versus quiet, fast versus slow computer, etc.).

Use Speech properties in Control Panel to change settings for all installed SAPI 5-compliant Text-To-Speech and Speech Recognition Engines. Click Settings to change settings on a per-user/per-engine basis. Use SPDUI_RecoProfileProperties to directly access each Engine's Recognition Profile Settings UI. If the engine does not support the Recognition Profile Properties UI (see ISpTokenUI::IsUISupported), the Settings button will be unavailable.

When to Access

The application could display a button or menu item for environment profile settings that accessed SPDUI_RecoProfileProperties. If the application can be used in either noisy or quiet environments, the user could be prompted to update the current recognition profile to reflect their specific environment (see ISpRecognizer::DisplayUI. Changes made within the engine properties UI will affect only one engine, and will not affect other users.

For information on creating new recognition profiles, see the helper function SpCreateNewToken (by Category Id).

#define SPDUI_RecoProfileProperties        L"RecoProfileProperties"

Example

The following code snippet illustrates the use of ISpTokenUI::IsUISupported using SPDUI_RecoProfileProperties.

    HRESULT hr = S_OK;

    // get the default speech recognizer token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_RECOGNIZERS, &cpObjectToken);
    // Check hr

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // check if the default speech recognizer has UI for its portion of the recognition profile
    hr = cpTokenUI->IsUISupported(SPDUI_RecoProfileProperties, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then the default speech recognizer has UI for its portion of the recognition profile

The following code snippet illustrates the use of ISpRecognizer::DisplayUI using SPDUI_RecoProfileProperties.

    HRESULT hr = S_OK;

    // display recognition profile properties UI for the current recognizer
    hr = cpRecognizer->DisplayUI(MY_HWND, MY_APP_RECO_PROFILE_PROPERTIES, SPDUI_RecoProfileProperties, NULL, NULL);
    // Check hr


SPDUI_AudioProperties

SPDUI_AudioProperties defines the string for displaying an audio object's properties UI.

#define SPDUI_AudioProperties        L"AudioProperties"

When to Implement

An application can modify the object's implementation-specific properties when writing a custom audio object. For example, the SAPI implementation of the multimedia audio object has a UI to allow the user to select the multimedia device line (e.g., microphone input, line input, etc.).

A custom audio object that performed noise reduction on the input stream could have UI for selecting the audio input object to read data from initially. It could also display UI that allows the user to adjust how much noise reduction is performed.

The Speech Control Panel allows the user to select the default audio input and output objects. Click Properties to directly access each audio object's properties UI using SPDUI_AudioProperties. If the audio object does not support the Properties UI (see ISpTokenUI::IsUISupported), Properties will be unavailable.

Example

The following code snippet illustrates the use of ISpTokenUI::IsUISupported using SPDUI_AudioProperties.

    HRESULT hr = S_OK;

    // get the default input audio object token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_AUDIOIN, &cpObjectToken);
    // Check hr

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // check if the default audio input object has UI for Properties
    hr = cpTokenUI->IsUISupported(SPDUI_AudioProperties, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then default audio input object has UI for Properties


SPDUI_AudioVolume

SPDUI_AudioVolume defines the string for displaying an audio object's volume UI.

When to Implement

An application can modify the object's volume settings when writing a custom audio object. For example, the SAPI implementation in the multimedia audio object displays the Windows Mixer associated with the audio device.

Using Speech properties in Control Panel, select the default audio input and output objects. Click Volume to directly access each audio object's volume UI using SPDUI_AudioVolume. If the audio object does not support the Volume UI (see ISpTokenUI::IsUISupported), Volume will be unavailable.

When to Access

The application accesses the audio object's volume UI if the output is too loud, or the input is too quiet.

Also, an SR Engine can send an SPEI_REQUEST_UI event to the application if it determines that the user should perform microphone training. Typically the UI type will be SPDUI_MicTraining to ensure that the SR Engine is adapted to the current input audio settings. For example, if the audio input volume is very low, it would be preferable to raise the audio input volume, rather than ask the SR Engine to amplify a poor audio input signal.

SAPI will generally not recognize changes to the Windows Mixer settings. This Mixer is made available solely as a last resort in adjusting the sound system when the Microphone Training wizard fails to set the volume suitably.

#define SPDUI_AudioVolume        L"AudioVolume"

Example

The following code snippet illustrates the use of ISpTokenUI::IsUISupported using SPDUI_AudioVolume.

    HRESULT hr = S_OK;

    // get the default input audio object token
    hr = SpGetDefaultTokenFromCategoryId(SPCAT_AUDIOIN, &cpObjectToken);
    // Check hr

    // get the object token's UI
    hr = cpObjectToken->QueryInterface(&cpTokenUI);
    // Check hr

    // check if the default audio input object has UI for Volume
    hr = cpTokenUI->IsUISupported(SPDUI_AudioVolume, NULL, NULL, NULL, &fSupported);
    // Check hr

    // if fSupported == TRUE, then default audio input object has UI for Properties


Class IDs

The following Class IDs are used with SAPI 5.

CLSID | Interface on which CLSID can be cocreated
CLSID_SpNotifyTranslator | ISpNotifyTranslator
CLSID_SpObjectTokenCategory | ISpObjectTokenCategory
CLSID_SpObjectToken | ISpObjectTokenInit
CLSID_SpResourceManager | ISpResourceManager
CLSID_SpStreamFormatConverter | ISpStreamFormatConverter
CLSID_SpStreamFormatConverter | ISpEventSource
CLSID_SpStreamFormatConverter | ISpEventSink
CLSID_SpStreamFormatConverter | ISpAudio
CLSID_SpMMAudioEnum | ISpObjectWithToken
CLSID_SpMMAudioEnum | IEnumSpObjectTokens
CLSID_SpMMAudioIn | ISpMMSysAudio
CLSID_SpMMAudioOut | ISpMMSysAudio
CLSID_SpRecPlayAudio | ISpAudio
CLSID_SpRecPlayAudio | ISpObjectWithToken
CLSID_SpStream | ISpStream
CLSID_SpStream | ISpEventSource
CLSID_SpStream | ISpEventSink
CLSID_SpStream | ISpTranscript
CLSID_SpVoice | ISpVoice
CLSID_SpVoice | ISpThreadTask
CLSID_SpSharedRecoContext | ISpRecoContext
CLSID_SpSharedRecoContext | _ISpPrivateEngineCall
CLSID_SpInprocRecognizer | ISpRecognizer
CLSID_SpSharedRecognizer | ISpRecognizer
CLSID_SpLexicon | ISpContainerLexicon
CLSID_SpUnCompressedLexicon | ISpLexicon
CLSID_SpUnCompressedLexicon | ISpObjectWithToken
CLSID_SpCompressedLexicon | ISpLexicon
CLSID_SpCompressedLexicon | ISpObjectWithToken
CLSID_SpPhoneConverter | ISpPhoneConverter
CLSID_SpNullPhoneConverter | ISpPhoneConverter
CLSID_SpResourceManager | ISpTaskManager


Category IDs

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Token Category IDs

The following token category IDs are used with SAPI 5.

Category ID | Purpose
SPCAT_AUDIOOUT | Available audio output devices.
SPCAT_AUDIOIN | Available audio input devices.
SPCAT_VOICES | Available voices.
SPCAT_RECOGNIZERS | Available recognizers.
SPCAT_APPLEXICONS | Available application lexicons.
SPCAT_PHONECONVERTERS | Available phoneme converters
SPCAT_RECOPROFILES | Available recognition profiles.


IUnknown

The IUnknown interface is a common interface supported by all COM objects and, therefore, by all speech objects. The IUnknown interface has these member functions:

  • QueryInterface
  • AddRef
  • Release

An application uses IUnknown to obtain pointers to other interfaces supported by an object and to manage the interface pointers after obtaining them.

QueryInterface

HRESULT QueryInterface(
   REFIID riid,			//Identifier of the requested interface
   LPVOID FAR   *ppvObj		//Address of output variable that receives the
                          //interface pointer requested in iid
);

Retrieves the address of a specified interface on a particular object so that an application can query an object to determine what interfaces it supports.

Returns NOERROR, if successful, or one of these error values:

  • CO_E_OBJNOTCONNECTED
  • E_NOINTERFACE
  • E_OUTOFMEMORY
  • E_INVALIDARG
  • E_UNEXPECTED
  • REGDB_E_IIDNOTREG
Parameter | Description
riid | [in] Interface identifier of the interface to be retrieved.
ppvObj | [out] Address of a variable that receives the address of the specified interface on the object. If the interface specified in riid is not supported by the object, the function returns E_NOINTERFACE. All errors set *ppvObj to NULL.

AddRef

ULONG AddRef(void);
  • Increments a reference count for every new copy of an interface pointer to a specified interface on a particular object.
  • Returns the value of the reference count.

When an interface is fully released, the reference count is zero. This information should be used only for diagnostics and testing.

Release

ULONG Release(void);
  • Decrements the reference count for the specified interface on a particular object.
  • Returns the value of the reference count.

When an interface is fully released, the reference count is zero. This information should be used only for diagnostics and testing.

If the object reference count goes to zero as a result of calling Release, the object is freed from memory.

If the AddRef member function has been called on this object's interface n times and this is the n+1th call to Release, the interface pointer frees itself. An object frees itself if the released pointer is the only pointer and if the object supports multiple interfaces through the QueryInterface member function.


Further Reading

The descriptions for the following items are contained in the Microsoft® Platform Software Development Kit (SDK). Non-SAPI SDK link

IStream
IServiceProvider


SDK Tools

The following sections cover tools, samples, and tutorials supporting speech functions:

  • SDK Sample Application Users Guides
  • SDK Tutorials


SDK Overview

This section provides as SDK overview.

The SDK provides examples of applications using SAPI 5.0 within a representative range of uses. These examples may contain only the executable files and are provided for illustrative purposes only. Most however, contain source code. You are free to model applications on these samples.

Since SDK samples are meant to demonstrate basic speech technologies, not all all options or contingencies are checked for. In some instances with the SDK samples, words may display incorrectly if the engine language is not the same as the system language. In such cases, the application is behaving as expected and is not an error of the speech system.

The following topics are available:

Demonstration only (no source code)

  • Age of Empires II Speech Interface
  • Reco (a recognition test tool)

Samples with source code

  • CoffeeS0 Sample Application
  • CoffeeS1 Sample Application
  • CoffeeS2 Sample Application
  • CoffeeS3 Sample Application
  • CoffeeS4 Sample Application
  • CoffeeS5 Sample Application
  • CoffeeS6 Sample Application
  • Dictation Pad
  • MKVoice
  • TTS Application
  • Talkback
  • Simple Telephony

Engine Samples with source code

  • Speech Recognition Engine
  • Text-to-Speech Engine

Tools

  • GC (gc.exe) grammar compiler for SAPI 5.0 XML conversion


AOESAPI

Introduction

AOESAPI is an application adding a speech interface to Microsoft's Age of Empires II: The Age of Kings (AOE II). You do not need a special version of AOE II; it is run normally. The speech interface passes keyboard commands to the game. That is, you can replace most keyboard tasks with speech commands. Some commands, such as scrolling and changing game speeds, are not currently supported by the speech interface. Also, you cannot implement certain mouse operations such as selecting and moving units, or placing buildings with speech commands.

Start AOESAPI to enable speech operations. In the main AOESAPI dialog window, select Listening to enable recognition, then launch AOE II. After selecting the options and a scenario, the game will begin. At that point, you may speak commands for game play. Illegal commands or those incorrectly recognized will not work and no action will be taken. You will not need additional speech and command training to play; however, speak the commands clearly, slowly, and deliberately. Your familiarity at playing the game using speech commands will increase over time and your proficiency will improve.

You will find that a complete list of commands and their syntax is available through Commands. Use speech in the same manner as the keyboard for issuing commands. Some commands are global and you may issue them at any time. These include chat commands and pause game, or menu items such as diplomacy, objectives, or display game time. Other commands are specific to units or buildings. With the target selected, issue a command. If the command is appropriate, it will be executed. For example, select a barrack; you may then delete it, set a gather point or train specific units. However, if you attempt to train a cavalry unit there, this command will not work and no action will be taken.

Options

Several options are presented in the dialog window. By default, all of them are initially on except for Listening. Selecting various options will enable or disable entire categories of commands. Check each option for complete access to all commands.

Activate game commands

Activates a series of administrative commands and menus. These include diplomacy, chat and objective windows as well as game pause. It also displays the technology tree. You can initiate, forward, and send chat by speech commands, but you must still type the actual content.

Activate unit commands

Activates commands for villager units. You must activate the unit with the mouse, but once selected, you can issue speech commands. If a command is not applicable for the unit or the group of units, no action will be taken. Examples include the villager unit attempting to pack, stop, set gather points and garrison, or attempting to select economic or military buildings.

Military unit commands

Activates commands for military units. You must activate the unit with the mouse, but once selected, you can issue speech commands. If a command is not applicable for the unit or the group of units, no action will be taken. Examples include the military unit to take an aggressive, defensive, or stand ground posture, form a line, patrol an area, or guard a building.

Activate training commands

Activates the build, create and train commands. This command allows the creation of many items including types of buildings, villagers, different military units such as swordsmen, knights, and archers. You can also use this command to activate weapons including fire ships, trebuchets, and battering rams.

Activate view commands

Activates the go to and view commands. The commands may be simple such as "go to barracks," or "go to last notification." They may also be more complex such as "go to next idle villager unit" although you may omit the final "unit."

Activate idle commands

Activates the idle command for villagers or military units.

Activate villager build commands

Activates a series of villager/ship build commands. You must activate the unit with the mouse, but once selected, you can issue speech commands. You can issue the build order directly without selecting it from the build list. For example, a selected villager may be told to "build house." An image of the dimmed house will appear on the screen, then use the mouse to place it. Other examples of villager build commands include building the market, blacksmith, farm, fish trap, and military objects such as archery range, barracks and stone wall.

Automatically view before training

Displays the building prior to issuing the build or train command. This allows you to view the building centered on the screen. Unlike the other categories, this one does not effect a series of commands.

Listening

Enables speech recognition. Once activated, AOESAPI attempts to issue keystrokes if a command is recognized. AOE II is intended to be the recipient of the commands. However, if another application capable of accepting keyboard input is active and in the Windows foreground, it is possible that the keystrokes will be passed along to that window instead. Take care to avoid unintended input to the foreground application. As mentioned above, launch AOE II immediately after AOESAPI.


Reco

Introduction

Reco is a speech tool that allows you to examine and test the speech process. You may speak using a microphone for either dictation or command and control. Alternatively, you may directly type a command and control order in an edit box and submit it to the speech recognition engine.

During the recognition process, events will be displayed in the top window as they occur. An event log is kept and to examine the details of each one, double-click the item. You may use this log to trace the sequence and number of events. You may also examine speech results. Double-click the text or phrase to display more detailed information about the text in the bottom window.

Options

The dialog box presents several options. By default, all options are initially on. The following options display detailed information suited for speech testing.

Create Recognition Context

Creates the recognition context for the selected engine. This must be off before selecting a different engine. Select this to activate the engine. You must select it before any speech is recognized.

Retain Reco Audio

Keeps the recorded sound of the actual spoken content in memory. Select this option and double-click the text in the display area of either the Events or Properties window to play back the spoken text. If it is not selected, the playback will be in a synthesized text-to-speech voice.

Activate Microphone

Controls the microphone's input. Select this option to enable the microphone to accept sound input. If it is not selected, the microphone will be blocked and no sound will be registered.

Load Dictation

Controls loading of dictation grammar. Select it to load the grammar for dictation and allow the speech recognition engine to process free-formed speech. You must load this grammar before selecting Activate Speech. If you do not select Load Dictation, will the grammar will be unloaded.

Activate Dictation

Controls the activation status of dictation features. If you have loaded the dictation grammar (see Load Dictation), select this to allow speech attempts to be recognized as free-formed dictation.

Load Command and Control

Controls loading of command and control grammar. Select this to load the grammar for dictation and allow the speech recognition engine to process command-oriented speech. You must load this grammar before selecting Activate Command and Control. If you do not select Load Command and Control, the grammar will be unloaded.

Activate Command and Control

Controls the activation status of command and control features. If you have loaded the command and control grammar, (see Load Command and Control), select this to allow speech attempts to be recognized as commands according to the rules defined the in current grammar. By default, the Solitaire grammar is installed. However, you may change the grammar using the Command and Control->Load Grammar menu.

Load Spelling

Enables or disables the letter-by-letter spelling feature. You must enable spelling before you select Activate Spelling. If Load Spelling is not selected, it will disable spelling.

Activate Spelling

Controls the activation status of the letter-by-letter spelling function. If you have loaded the spelling grammar, (see Load Spelling), select this to activate the spelling capability of the speech recognition engine. This allows you to spell words in a letter-by-letter fashion while in dictation mode. For example, you can say, "Let's take the dogs on a w-a-l-k," and the engine will attempt to recognize the sentence as "Let's take the dogs on a walk."

Shared Recognizer

Selects the engine as a shared engine. This allows other speech-enabled applications running at the same time to use the microphone. To do this:
  1. In Control Panel, double-click the Speech icon.
  2. On the Speech Recognition tab, in the Speech Engine window, select an engine.

InProc

Selects an engine as an in-process (InProc) or non-shared engine. InProc restricts speech-enabled applications running at the same time from using the microphone. Because the engine will not be shared among other applications, this option allows you to select different engines from the drop down menu rather than using the Speech icon. Selecting an InProc engine will not change the engine used by shared applications or the one selected in the Speech Engine window.

Event Window

Displays events and any associated results. The name of the event is displayed inside brackets. Certain events such as recognition or a hypothesis, have text associated with them and that text is displayed immediately under the event. Double-click the text to hear it spoken. If Retain Reco Audio is currently selected, the playback will be in your voice, recorded when it was spoken; otherwise, the text-to-speech voice will be played back. You may change the synthesized voice using the TTS tab in Speech Properties. Double-click the event to display additional information about it. The Results window will update to the new information.

Results Window

Displays details of the properties and values associated with the recognition result. This information includes the applicable rule, the text of the recognition, the parsing of information, and the individual elements of each word.

Alternates

Command and Control Edit Box

Allows you to directly enter a command and control phrase. If any text appears in the box, Submit activates it. Click Submit to enter a command, bypassing the microphone. In this way, the microphone does not need to be active or even present to test or watch the recognition process.

Submit

Submits any text in the command and control edit box, see Command and Control Edit Box.


CoffeeS0

Introduction

CoffeeS0 is the first sample application in a series called Coffee. It uses a consistent coffee shop motif. You will eventually be able to enter the shop, go to the order counter to order drinks, go to the gift store, or speak to management. The samples are intended to demonstrate adding speech recognition to an application. They are designed for the application-level (API) programmer and for those not familiar with speech technology. Writing engines such as speech recognition or text to speech, also called device driver programming, will be covered separately. Each sample will progressively add new features and increase in complexity. The tutorial chapters explain in detail the particulars of the code. You are encouraged to read each chapter.

As the introductory sample, CoffeeS0 is a simple application. There is one window and a limited vocabulary from which to speak. After launching CoffeeS0, you may speak any of the commands. If successfully recognized, the window displays the response, "Please order when ready!" To keep the application simple at this point, no other commands may be used. If the command was not successfully recognized, no action will take place. Due to the speed and processing capabilities of some computers, there might be a slight delay before the CoffeeS0 responds. If after a moment nothing happens, try the command again.

Commands may not be recognized for two reasons. One, the speech may not have been clear. Perhaps the words were not spoken clearly enough or distinctively enough. Try speaking the command again more slowly and clearly. The other reason is that the words may not have been in the command list. Look at the available commands and speak again. CoffeeS0 has a limited command list and if a word is used which is not in the list, the command will not be recognized.

To quit CoffeeS0, click the Close button in the upper right of the window frame.

Commands

Commands are formed by choosing one word from each line of a category. Commands in parenthesis are optional and do need to be included. Words or phrases separated by slashes indicate any of the listed choices may be used although only one may be selected. Sections marked RULEREF indicates words or phrases may be chosen from the corresponding rule ID. Rule names are the same as listed in the corresponding XML configuration file.

For example, you can say, "enter counter," "please enter counter," or "please enter the counter." All three are recognizable commands. Since CoffeeS0 can only take you to the ordering counter, "please enter the shop," and "please enter the store," will have the same effect. Partial phrases or words not listed below may not be used. "Please enter the restaurant," or simply "counter" will not be recognized. The following commands are available:

Command List

XML rule ID: VID_Navigation

  • (please)
  • enter/go to
  • RULEREF: VID_Place

XML rule ID: VID_Place

  • (the)
  • counter/shop/store


CoffeeS1

Introduction

CoffeeS1 is the second sample application in a tutorial series named Coffee. It uses a consistent coffee shop motif. You will eventually be able to enter the shop, go to the order counter to order drinks, go to the gift store, or speak to management. The samples are intended to demonstrate speech recognition capabilities within an application. They are designed for the application-level (API) programmer and for those not familiar with speech technology. Writing engines such as speech recognition or text to speech, also called device driver programming, will be covered separately. Each sample will progressively add new features and increase in complexity. The tutorial chapters explain in detail the particulars of the code. You are encouraged to read each chapter.

As the second example, CoffeeS1 builds on the framework of CoffeeS0. There is one window and although there is a limited vocabulary from which to speak, it is expanded from CoffeeS0. CoffeeS0 restricted you to moving to the counter only. In contrast, CoffeeS1 now lets you order one of a variety of coffee drinks. After launching CoffeeS1, you may speak any of the commands. If successfully recognized, the window displays the response, "Please order when ready!" If the command was not successfully recognized, no action will take place. Due to the speed and processing capabilities of some computers, there might be a slight delay before CoffeeS1 responds. If after a moment nothing happens, try the command again.

Commands may not be recognized for two reasons. One, the speech may not have been clear. Perhaps the words were not spoken clearly enough or distinctively enough. Try speaking the command again more slowly and clearly. The other reason is the words may not have been in the command list. Look at the available commands and speak again. CoffeeS0 has a limited command list and if a word is used which is not in the list, the command will not be recognized.

To quit CoffeeS1, click the Close button in the upper right of the window frame.

Commands

Command sequences are built using at least one word from amongst the various categories. Commands are grouped into two major (or top-level) categories: navigation and drink ordering. Navigation allows you to move around the coffee shop to places such as the counter, shop, or store. In CoffeeS1, the effects of going to any location are minimal and any navigation command takes you to the counter. The distinction allows the code sample to demonstrate additional SR features. Subsequent Coffee examples will expand on this. Drink ordering allows you to place drink requests.

In either case, words or phrases may be selected from each category. Some are marked as Required and others are marked as Optional. You must use at least one word or phrase from the list marked Required to successfully initiate the command. Optional words, though not required, provide a more natural speech. As a special case, the drink orders may include words from up to seven of the categories and may be spoken in any order. However, at least one category is required to successfully match command requirements.

For example, at the minimum you can say “get me mocha.” This satisfies the command rules of the required phrase (“get me”) from the VID_EspressoDrinks required list and at least one of the seven other drink categories, in this case VID_DrinkType. Likewise, you could also say, “I would like a tall hazelnut two percent latte.” Words not appearing in the list may not be used. “Give me a latte,” would fail since “give me” is not an allowable phrase. Orders not fulfilling the requirements will not be recognized. “Please get me a” will also fail. While the required top-level phrase is correctly used, a subsequent drink was not ordered. The following commands are available:

Command List

Commands are formed by choosing one word from each line of a category. Commands in parenthesis are optional and do need to be included. Words or phrases separated by slashes indicate any of the listed choices may be used although only one may be selected. Sections marked RULEREF indicates words or phrases may be chosen from the corresponding rule ID. Rule names are the same as listed in the corresponding XML configuration file.

Navigation

Requires a match from each VID_Navigation and VID_Place.

XML rule ID: VID_Navigation

· Optional: Please

· Enter

· Go to

XML rule ID: VID_Place

· Optional: The

· Counter

· Shop

· Store

Drink Ordering

Requires a match from VID_EspressoDrinks and at least one (but no more than seven) of the subsequent categories.

XML rule ID: VID_EspressoDrinks

· May I have

· Can I have

· Can I get

· Please get me

· Get me

· I'd like

· I would like

· Optional:a

XML rule ID: VID_Iced

· Iced

XML rule ID: VID_Decaf

· Decaf

· Decalf

· Decaffeinated

XML rule ID: VID_Shots

· Single

· Double

· Triple

· Quad

XML rule ID: VID_Size

· Short

· Tall

· Grande

XML rule ID: VID_Syrup

· Hazelnut

· Irish cream

· Almond

· Peppermint

XML rule ID: VID_Milk

· Nonfat

· Two percent

· Whole

XML rule ID: VID_DrinkType

  • Latte/Mocha/Espresso/Americana/Cappuccino


CoffeeS2

Introduction

CoffeeS2 is the third sample application in a tutorial series named Coffee. It uses a consistent coffee shop motif. Customers enter the shop, go to the service counter, speak to order drinks or to enter the front office.

The samples are intended to demonstrate speech recognition capabilities within an application. They are designed for the application-level (API) programmer and for those not familiar with speech technology. Each sample will progressively add new features and increase in complexity. The tutorial chapters explain in detail particulars of the code. You are encouraged to read each chapter. Writing engines such as speech recognition or text-to-speech, also called device driver programming, will be covered separately. The samples can use engines provided by the SAPI SDK or third party SAPI-compliant engines.

Using CoffeeS2

As the third example, CoffeeS2 builds on the framework of its predecessors. There is one window and an expanded vocabulary from which to order drinks and move around the business. In addition to placing an order, you can move to both the counter and the office. However, drinks may not be requested while in the office.

CoffeeS2 allows you may speak any of the CoffeeS1 commands. If successful, the window displays the action such as "please order when ready!" If the command was not successfully recognized, no action will take place.

Due to the speed and processing capabilities of some computers, there might be a slight delay before CoffeeS1 responds. If after a moment nothing happens, try the command again. Commands may not be recognized for two common reasons. One, the speech may not have been clear. Perhaps the words were not spoken clearly enough or distinctively enough. Try speaking the command again more slowly and clearly. The other reason is the words may not have been in the command list. Look up the commands available and speak again. The Coffee examples have a limited command list and if a word is used but is not from those commands, it will not be recognized.

To quit CoffeeS2, click the Close button in the upper right of the window frame.

New Features

See CoffeeS1 users note for a detailed description about using the application.

A new navigation command has been added allowing you to visit the office. You may request "go to the office." The screen will change to "Welcome to the SAPI coffee shop office." Drinks may not be ordered while in the office. As a result, drink commands are unavailable while in the office. To leave, navigate to another part of the business. Like CoffeeS1, navigation commands involving the counter, store, and shop take you to the same place.

A second new feature asks you to repeat the order if it is not understood. If a recognition takes place but the word or phrase is not in the command list, CoffeeS2 will display "Sorry, I didn't get that order. Please try again."

New Commands

Commands are formed by choosing one word from each line of a category. Commands in parenthesis are optional and do need to be included. Words or phrases separated by slashes indicate any of the listed choices may be used although only one may be selected. Sections marked RULEREF indicates words or phrases may be chosen from the corresponding rule ID. Rule names are the same as listed in the corresponding XML configuration file.

XML rule ID: VID_Place
  • office


CoffeeS3

Introduction

CoffeeS3 is the fourth sample application in a tutorial series named Coffee. It uses a consistent coffee shop motif. Customers enter the shop, go to the service counter, speak to order drinks or to enter the front office.

The samples are intended to demonstrate speech recognition capabilities within an application. They are designed for the application-level (API) programmer and for those not familiar with speech technology. Each sample will progressively add new features and increase in complexity. The tutorial chapters explain in detail particulars of the code. You are encouraged to read each chapter. Writing engines such as speech recognition or text-to-speech, also called device driver programming, will be covered separately. The samples can use engines provided by the SAPI SDK or third party SAPI-compliant engines.

Using CoffeeS3

CoffeeS3 is identical to the previous CoffeeS2 sample. See the CoffeeS2 guide for detailed instructions for using this.

However, a synthesized voice has been used for this module. At certain times, the text-to-speech engine will speak the command, an order, or a greeting. For example, the initial screen will not only display the greeting, but will also say "Welcome to the SAPI coffee shop. Speak for service."

New Commands List

There are no new commands for CoffeeS3.


CoffeeS4

Introduction

CoffeeS4 is the fifth sample application in a tutorial series named Coffee. It uses a consistent coffee shop motif. Customers enter the shop, go to the service counter, speak to order drinks or to enter the front office.

The samples are intended to demonstrate speech recognition capabilities within an application. They are designed for the application-level (API) programmer and for those not familiar with speech technology. Each sample will progressively add new features and increase in complexity. The tutorial chapters explain in detail particulars of the code. You are encouraged to read each chapter. Writing engines such as speech recognition or text-to-speech, also called device driver programming, will be covered separately. The samples can use engines provided by the SAPI SDK or third party SAPI-compliant engines.

Using CoffeeS4

CoffeeS4 introduces the concepts of resources and resource management. SAPI stores information in the form of tokens. These tokens are used later to instantiate features such as voices and recognizers. However, programmers can query SAPI for the presence of tokens to learn more about available features. For example, each available voice is kept as a token.

CoffeeS4 displays available voices and may even speak using the currently active voice. Three new commands are used. To do so, enter the office by saying, “go to the office” or “enter office.” Once there, display the voice list by saying “manage the employees.” A list of available voices will display on the right side of the screen. The active voice will be indicated in red.

To have the employee speak, say “hear them speak.” The statement “I will be the best employee you've ever had. Let me work” will be spoken in the current voice. The voice may be changed through the Speech properties of the Control Panel.

New Commands List

Commands are formed by choosing one word from each line of a category. Commands in parenthesis are optional and do need to be included. Words or phrases separated by slashes indicate any of the listed choices may be used although only one may be selected. Sections marked RULEREF indicates words or phrases may be chosen from the corresponding rule ID. Rule names are the same as listed in the corresponding XML configuration file.

XML rule ID: VID_Manage

  • (please)
  • manage
  • (the)
  • RULEREF: VID_ThingsToManage

XML rule ID: VID_ThingsToManage

  • employees

XML rule ID: VID_HearTheVoice

  • hear them speak


CoffeeS5

Introduction

CoffeeS5 is the sixth sample application in a tutorial series named Coffee. It uses a consistent coffee shop motif. Customers enter the shop, go to the service counter, speak to order drinks or to enter the front office.

The samples are intended to demonstrate speech recognition capabilities within an application. They are designed for the application-level (API) programmer and for those not familiar with speech technology. Each sample will progressively add new features and increase in complexity. The tutorial chapters explain in detail particulars of the code. You are encouraged to read each chapter. Writing engines such as speech recognition or text-to-speech, also called device driver programming, will be covered separately. The samples can use engines provided by the SAPI SDK or third party SAPI-compliant engines.

Using CoffeeS5

CoffeeS5 expands the concepts of resources and resource management introduced in CoffeeS4. Using information learned by polling tokens about available voices, CoffeeS5 allows users to change the active voice. In doing so, a dynamic grammar is used. In the pervious Coffee samples, all the speech commands were determined ahead of time and could not be changed. For example, the drinks were limited to five basic types and a new one could not be added. A dynamic grammar allows adding or removing commands during the program execution.

To change the voices, enter the office by saying, “go to the office” or “enter office.” Once there, display the voice list by saying “manage the employees.” A list of available voices will display on the right side of the screen. The active voice will be indicated in red. To have the employee speak, say “hear them speak.” The statement “I will be the best employee you've ever had. Let me work” will be spoken in the current voice.

To change the voice, say the voice name as it appears on the screen. For example, if "Microsoft Mary" is displayed, say "Microsoft Mary." The highlighting will change to the selected voice. Having the employee speak will do so in the voice. Additionally, the list of available voices may be filtered by gender. The left side of the screen displays available commands for this. Say "Show males only," for example, will display only the male voices.

Some voices may not be applicable to this example. For instance, Sample TTS Voice is a composite voice for use with the SDK application MkVoice. The voice contains only seven words with an eighth word being the default for all other words. As a result, it will say "blah" most of the time. In the same way, the MS Simplifying Chinese Voice will spell the content rather than speak it.

New Commands List

Commands are formed by choosing one word from each line of a category. Commands in parenthesis are optional and do need to be included. Words or phrases separated by slashes indicate any of the listed choices may be used although only one may be selected. Sections marked RULEREF indicates words or phrases may be chosen from the corresponding rule ID. Rule names are the same as listed in the corresponding XML configuration file.

XML rule ID: VID_OtherRules

  • (show) males only / (show) females only / (show) both genders

Rule ID: DYN_TTSVOICERULE

This is dynamic rule generated during runtime. No xml code is present. After being generated it contains all the display name for the available voices. The contents of the rule is displayed on the right side of the screen in the CoffeeS5 office after issuing the "manage the employees" command. The rule is generated the time that command is issued and is destroyed after leaving the office afterward.


CoffeeS6

Introduction

CoffeeS6 is the seventh and final sample application in a tutorial series named Coffee. It uses a consistent coffee shop motif. Customers enter the shop, go to the service counter, speak to order drinks or to enter the front office.

The samples are intended to demonstrate speech recognition capabilities within an application. They are designed for the application-level (API) programmer and for those not familiar with speech technology. Each sample will progressively add new features and increase in complexity. The tutorial chapters explain in detail particulars of the code. You are encouraged to read each chapter. Writing engines such as speech recognition or text-to-speech, also called device driver programming, will be covered separately. The samples can use engines provided by the SAPI SDK or third party SAPI-compliant engines.

Using CoffeeS6

CoffeeS6 expands the concepts of grammars and dictations as they have been presented in the previous examples. The early Coffee examples used a fixed grammar to explicitly include a select set of words. CoffeeS5 introduced dynamic grammars, allowing you to add new words to an existing word list. CoffeeS6 goes one additional step and uses dictation. For dictation, you are no longer limited to an explicit list but may now use almost any word or words. However, instead of demonstrating this as a free-formed dictation application, CoffeeS6 uses it in association with existing grammars

To showcase this ability, CoffeeS6 allows you to rename the coffee shop. From the office, a new option is presented: Manage Store Name. Speak this command and the screen changes again. A new order is displayed on the screen allowing you to rename the shop. Say "Rename the coffee shop to" and a name. CoffeeS6 then speaks the new name of the store. For example, by saying "Rename the coffee shop to My Coffee Emporium," the new name will be "My Coffee Emporium." The name will also be displayed throughout CoffeeS6. After saying the command, you may then navigate to a another location or continue to rename the store any number of times.

New Commands List

Commands are formed by choosing one word from each line of a category. Commands in parenthesis are optional and do need to be included. Words or phrases separated by slashes indicate any of the choices listed may be used although only one may be selected. Sections marked RULEREF indicate words or phrases may be chosen from the corresponding rule ID. Rule names are the same as those listed in the corresponding XML configuration file.

The following rule is used to rename the coffee shop. However, because the shop name is not limited to a particular element from list, the asterisk acts as a wildcard. Any word or words are permitted. Additionally, the plus sign forces a greater level of confidence. See VID_ThingsToManage for a more detailed explanation of the sign. Requiring greater confidence forces the speech recognition engine to spend additional time processing the word. Greater confidence results in either better recognition of the word, or a higher confidence of the word returned by the speech recognition engine.

XML rule ID: VID_Rename

  • Rename the coffee shop to *+

The last two items of the next rule are new to CoffeeS6. As expected, they allow you to say "shop" and "store." However, the command allows for a change of emphasis on words. The plus or minus sign changes the required confidences of the word recognition. By increasing the required confidence level, the speech recognizer demands a higher quality of the word being recognized. In a similar way, decreasing the confidence allows for a greater latitude of the word's recognized quality. This way, certain words can be emphasized. For instance, the following rule ensures that the word "name" is recognized. Because regional accents or background noise may detract from the quality of the word, the new emphasis makes certain that the sound heard was actually "name." In turn, the rule places less emphasis on the word "shop" or "store." Although these words are still required, it is not as important to be certain.

XML rule ID: VID_ThingsToManage

  • employees
  • -shop +name
  • -store +name

Additionally, two existing rules have new list elements. The ellipsis disregards any words spoken for the current element. For example, VID_EspressoDrinks allows you to say "May I have a coffee," or "Get me coffee," as expected. However by including the ellipsis in the rule, you may also say "I dunno, how about a coffee." The statement will be recognized since everything before the drink name can essentially be ignored. Including the ellipsis as a list element significantly reduces the constraint of listing all words individually.

XML rule ID: VID_EspressoDrinks

XML rule ID: VID_OrderList

  • ...


Dictation Pad

Dictation Pad is an example of a speech-enabled word processor. This sample application is intended to demonstrate many of the features for SAPI 5.0 in a single coherent application. It is not a full featured speech-enabled application, although the foundations of many of options are present.

Dictation Pad allows you to speak into a microphone and, following successful speech recognition (SR), Dictation Pad will display the sentence on the screen as text. It also allows the words to be spoken back in a text-to-speech (TTS) voice, highlighting words as they are spoken. Features include:

  • Dictation - Recognizes words in any context.
  • Command and control - Recognizes a limited selection of words and applying them to control the flow of Dictation Pad. This includes using speech to select items from menus and changing the SR mode from dictation to command and control.
  • Playback - Plays back words appearing on the screen in a TTS voice.
  • Speakback - Keeps an audio record of the actual spoken content. You can play it back to confirm or verify speech recognition.
  • Phrase tracking - Maintains a list of phrase element information. This can locate the parts of an SR phrase even if the dictation becomes broken or disjointed. It also demonstrates text replacement such as inverse text normalization. This is the process of converting text to numbers such as "one two three" into "1-2-3" or "first" into "1."
  • Word Alternates - Displays a list of alternates for the recognized text and allows you to select a replacement for the original text.
  • Adding words to a grammar - Demonstrates the SR engine's capability to add words or phrases to an existing grammar or word database. Adding a word allows it to be recognized on subsequent occurrences.
  • Document management - Saves documents and opens them retaining the data associated with the recognized SR results.

The complete code base for Dictation Pad is included with the SDK and you are encouraged to look at and examine the code freely. It is intended to be a training aid and to demonstrate as many features as possible.

Note About SR/TTS Engines

Dictation Pad supports common SAPI features such as various user interface calls. However, SR or TTS engines are not required to provide all the features. The current engine will be queried for features that Dictation Pad supports. If available, Dictation Pad will use a feature; otherwise, the feature will not be available or the menu item will be inactive.

For example, Dictation Pad uses the SAPI feature of the Add/Remove Words interface. The Microsoft ASR Version 5 engine supports this feature and it is available to Dictation Pad. The SAPI 5 Sample Engine from the SAPI SDK does not support it; hence, Dictation Pad deactivates the Add/Remove dialog menu item.

Dictation Pad Menu/Toolbar

The main window of the Dictation Pad contains both a toolbar and a menu bar which you can use to control all the application's functions. The toolbar is a convenience feature and you may access many of its functions through the menu.

File Menu

File menu items control the documents that are used in the application.

New
Creates a new document. Multiple documents cannot be open at the same time so the existing version must be saved and closed, or discarded before creating the new one.
Open
Opens a previously saved document. Similar to creating a new document, any current one must be saved first.
Save
Saves the current file. Files are saved as a proprietary *.dpd format.
Save As
Saves the current file under a new or different name.
Exit
Quits Dictation Pad. Unsaved files may be saved before exiting.

Edit Menu

Edit menu items control the copy and pasting of text for the current document.

Cut
Copies the text to the clipboard and removes the selected text from the document.
Copy
Copies the text to the clipboard.
Paste
Copies the text from the clipboard to the document. The text is either placed starting with the cursor insertion point or, if text is selected, replaces the selection with the new text.

Voice Menu

Voice menu items control the speech enabling aspect of the application.

Listen for Dictation
Enables Dictation Pad to receive speech in dictation form rather than for command and control. This allows you to speak any words or combination of words and for them to be recognized. This option is mutually exclusive from Listen for Commands.
Listen for Commands
Enables Dictation Pad to receive speech in command and control form rather than as dictation. You are limited to words defined for application control only. This includes a word equivalent for most menu items or buttons. This option is mutually exclusive from Listen for Dictation.
Playback
Reads back the text. To have a portion of the text read back, select the text you want. If you do not select any text, reading begins at the cursor insertion point. If the insertion point is at the end of the document, the entire document will be read.

If the text was originally dictated, the playback will be the recorded audio of your voice. If you did not dictate the text, but rather typed or pasted it into the document, the TTS voice will read the text.

Grammar Activation
Turns grammars on or off. Words or phrases added through the Add/Delete Word(s) option will neither be available nor recognized if the grammar is turned off. By default, this option is turned off.
Add/Delete Word(s)
Brings up the SR engine-specific user interface allowing you to add words to or delete words from the lexicon (or dictionary).
Select Whole Words
Sets the selection state for word highlighting. If selected, the entire word will be automatically highlighted during the selection process. Otherwise, only the words and letters actually selected will remain selected.
Shared recognition engine
Sets resource sharing. By default, Dictation Pad uses the "In process" (also referred to in SAPI 5.0 as InProc) resource model that causes the SR engine to exist in the same process as Dictation Pad and disallows other applications from using resources required by this application. Other resources include the microphone, so that all audio input is given to Dictation Pad rather than another application currently running. If selected, the SR engine may reside in a separate process.
Voice Training
This brings up Speech Training Recognition Wizard for training or additional training. This is the same wizard as through the Speech Properties in Control Panel. On the SR tab, click Train Profile to bring up the voice training wizard.
Microphone Setup
This brings up the Microphone Wizard for adjusting the microphone set up. This is the same wizard as through the Speech Properties in Control Panel. On the SR tab, click Configure Microphone to bring up the microphone wizard.

Using Dictation Pad

Speech Recognition

By default, Dictation Pad starts in dictation mode with the microphone off. To start speech recognition, click Microphone, select the Voice->Microphone menu item or use control-m. Begin speaking. To indicate processing, ellipses ("...") display in the window. During the recognition process as SAPI starts returning words or phrases, the text appears dimmed until a final recognition is made. When a final recognition is determined, the text will darken and the insertion point will advance.

Below the insertion point is a small box. Click this box to display a list of alternate words. SAPI 5.0 places the final result of its word search on the screen, but you can choose another word by selecting it from the alternate list. This new choice replaces the existing word.

Text-to-Speech

Text may be read back using the TTS voice. Select Voice->Playback or click Play to hear the text.

If no text is selected, Dictation Pad will begin reading from the insertion point to the end of the document. If you select specific text, only the highlighted portion will be read. In either case, the word currently being read will be highlighted. The words will continue to be highlighted until the selection or portion is read. Any text selected prior to being read will remain selected afterward. If the insertion point is at the end of the document, the entire document will be read.

To stop or interrupt playback, click Play again from either the toolbar, menu, or use control-p. You may also click anywhere in the edit window or press Esc to stop playback.

You may change voices characteristics from the TTS tab of Speech Properties. For example, you may change voices or change the speaking rate of the voice. The newly selected voice will automatically be previewed so you can confirm the choice. You can change the speaking rate with the Speed slider bar.

Command and Control

You may use speech to control program flow rather than using it as dictation. In this way, the menus, menu items, and the cursor may be controlled by speech and are collectively referred to as command and control. This is fundamentally different from dictation both programmatically and functionally. One difference is that the word selection is severely limited. The allowable words are restricted to essentially coincide with the menu items, buttons, or serve as logical cursor commands. Other words will not be recognized.

You can switch to command mode in one of three ways. The first is through the Voice->Listen for Commands menu item. The second is to click Command on the toolbar. The third way to switch is during dictation by saying "command." Regardless, Command on the toolbar will automatically depress as visual confirmation of the current mode.

Once in command mode, you can speak the commands. If it is recognized successfully, the action takes place. A menu command will drop down the appropriate menu. The action for a menu item will be directly applied. A cursor command will move the cursor. If the command was not understood, no action will take place. A command may not be recognized for several reasons. The word itself may not be allowable, or the word may not have been spoken clearly, or background noise may have obscured it. Similarly, the menu item may not be applicable to the situation. If the reason is unclear, you should repeat the command clearly and perhaps more slowly. There might also be a slight lag time in response depending on the computer system's capability.


MkVoice

Introduction

MkVoice creates text-to-speech (TTS) voice fonts for sample use. It combines a series of individually spoken words into a single file. The resulting file is automatically loaded and you may use it in any TTS application recognizing SAPI 5.0 voices.

MkVoice is intended to demonstrate making sample voice fonts. The limited scope is not ideal to create larger, more robust voice samples. Additionally, extensive error checking or error prevention routines are not included so that the resulting file may contain conditions not optimal to superior performance.

Running MkVoice

MkVoice is command line application that accepts three parameters:

mkvoice WordListFile VoiceFile VoiceName
WordListFile

This is a document list of words to concatenate and form the output file. The file needs to be saved as text only or created using a simple editor such as NotePad. No character formatting is allowed. The list requires only one word per line and that the line terminate with a carriage return.

The list can be of any size but each word must have a corresponding wav file with the same name. That is, if you use the word "enter," there needs to be another file named enter.wav. The first entry in the list is used as the default word. If a word is encountered that is not otherwise in the list, use this default word instead.

For example, if using the SDK example TTSApp with the "Sample TTS Voice," the text initially displayed will be spoken: enter text to be spoken here. If you change the text by adding a word not in the file list such as "enter text now," it will be spoken as "enter text blah." "Now" is not a part of word list.txt. However, since "blah" is the first entry, it will be used for all unknown words.

VoiceFile

This is the resulting output file. By SAPI 5.0 convention, the vce name is the recommended suffix, although it is not required. If successfully generated, MkVoice automatically loads the sample. The new voice will be displayed with the name "Sample TTS Voice." This replaces any previous voice fonts. The voice will be defined as an English-speaking male.

Finally, to run MkVoice, the application must be in the same folder as the wav files and word list. If run successfully, the application creates an output file; otherwise, it will display appropriate error messages.

VoiceName

This is the name associated with the voice by the object token.

Creating Voice Fonts

Voice fonts are a collection of words spoken by a person and assembled into a phonetic dictionary. When SAPI encounters a word, this database looks it up. A successful match plays a portion of the sound file of the word. By contrast, a synthesized voice uses mathematical algorithms to produce the word. The voice fonts produce what is often considered better and more natural prosody--the way the word sounds.

Voice fonts require two components. First, a word list must be generated. This is the same list as the first parameter described above. In the SDK, the MkVoice example uses wordlist.txt.

Second, individual wav files are needed for each word. The first part of the file name must correspond to an entry in the word list. The name suffix must be wav. Additionally, the wav file must contain the following characteristics:

Value | Description
Audio format | PCM
Sample rate | 11.25 kHz
Audio sample size | 16 bit
Channels | 1 (mono)
Files should contain only one word each. Silence leading or trailing a word should be minimal to provide the best playback. You may not use punctuation marks. Replace any marks with underscores ("_") in both the file name and word list. For instance, MkVoice provides a file named computer_s.wav and this matches the corresponding entry in wordlist.txt. A simple way to generate the wav files is to use Sound Recorder provided by the Windows operating system. The file characteristics may have to be changed to the above requirements.


TTSApp

TTSApp is an example of a Text-to-Speech (TTS) enabled application. This sample application is intended to demonstrate many of the features for SAPI 5.0 in a single coherent application. It is not a full featured TTS-enabled application although the foundations of many of options are present.

TTSApp allows you to hear the resulting audio output from the TTS process for text entered in the main window. Alternatively, you can open a file and TTSApp will speak the contents of that file.

Each word is highlighted in the text window to indicate the current TTS processing position. Features include:

SAPI5 TTSApp | The main display window of the TTSApp sample application.
Text window | TTSApp speaks the text contained in this window using TTS.
Speak | Initiates the TTS process.
Voices | Selects the voice for the audio output.
Rate | Selects the rate of speech.
Volume | Selects the volume level of the audio output stream.
Open File | Enables TTSApp to open and speak the contents of a stored text file.
Pause | Pauses the TTSApp text phrase speaking process.
Resume | Resumes the TTSApp text phrase speaking process.
Stop | Stops the TTSApp text phrase speaking process.
About | Displays the About TTApp information dialog box.
Format | Selects the audio format.
Skip | Specifies the number of text elements to skip in the phrase speaking process.
Speak .wav | Speaks the contents of a stored .wav file.
Reset | Resets TTSApp to its original configuration setting.
Save to .wav | Saves the contents of the TTSApp audio output stream to a .wav file.
Show all events | Displays all TTSApp SAPI events.
Process XML | Specifies that the TTS voice will speak the XML tags and their contents in the TTS process.
Mouth Position | Displays mouth shapes for phrase elements as they are spoken.

SAPI5 TTSApp main window.
Use the main TTSApp window to select the configuration settings that affect the TTS process. The elements of TTSApp are listed above. Click the text in the left column for additional information.

Text window
The text content of this window is spoken by TTSApp. All text entered in this window is processed and spoken by TTSApp voice.
By default, the text content of this window is, "Enter the text you wish spoken here."

Speak
Click Speak to initiate the Text-to-Speech process.

Voices
Select a voice using the drop-down list. TTSApp uses the selected voice when speaking a .wav file or the contents of the text window.

Rate
Move the slide control and move it to the right to increase the speech rate, and to the left to decrease the speech rate. The Rate level determines the number of text units spoken per minute.

Volume
Move the slide control to the right to increase the volume level, and to the left to decrease the volume level.

Open File
Click Open File to access the Windows Open dialog box. Select the file, and then click Open.

Pause
Click Pause to interrupt the TTS process.

Resume
Click Resume to continue the TTS process.

Stop
Click Stop to stop the TTS process.

About
The About window displays information related to TTSApp. Click OK to close the About window.

Format
Use the drop-down list in Format to select one of the following format rates.
Selectable format rates
8kHz | 8 Bit Mono | 8 Bit Stereo | 16 Bit Mono | 16 Bit Stereo
11kHz | 8 Bit Mono | 8 Bit Stereo | 16 Bit Mono | 16 Bit Stereo
12kHz | 8 Bit Mono | 8 Bit Stereo | 16 Bit Mono | 16 Bit Stereo
16kHz | 8 Bit Mono | 8 Bit Stereo | 16 Bit Mono | 16 Bit Stereo
22kHz | 8 Bit Mono | 8 Bit Stereo | 16 Bit Mono | 16 Bit Stereo
24kHz | 8 Bit Mono | 8 Bit Stereo | 16 Bit Mono | 16 Bit Stereo
32kHz | 8 Bit Mono | 8 Bit Stereo | 16 Bit Mono | 16 Bit Stereo
44kHz | 8 Bit Mono | 8 Bit Stereo | 16 Bit Mono | 16 Bit Stereo
48kHz | 8 Bit Mono | 8 Bit Stereo | 16 Bit Mono | 16 Bit Stereo

Skip
Use the spin box to select the number of skipped sentences. Skip functions only while text is being spoken.

Speak .wav
Speak .wav enables TTSApp to speak the contents of a .wav file. Click Speak.wav to access the Windows Open dialog box. Select a .wav file from the dialog box, and then click Open.

Reset
Click Reset to reset TTSApp to its original configuration state.

Save to .wav
Click Save to .wav to save the TTSApp audio output stream to a .wav file.

Show all events
Select Show all events to display SAPI related events in the event display window as the input text is processed by TTSApp.

Process XML
Select Process XML to include the XML tags and their contents in the audio output stream from TTSApp. When this option is selected, the application will parse and interpret the XML tags literally.

For example, if the Process XML option is selected, the application could be paused for the specified number of milliseconds in the SILENCE tag.

Process XML | XML tag | Result
<SILENCE MSEC = "3000"/> | The application would speak 3000 milliseconds of silence.
<SILENCE MSEC = "3000"/> | The application will speak the phrase, "less than silence msec equals quote three thousand quote slash greater than."

Mouth Position
The mouth position displays the various mouth shapes and positions as TTSApp processes the input text stream.


Talkback

Introduction

TalkBack demonstrates speech recognition (SR) and text-to-speech (TTS) capabilities. You can speak any word or phrase and TalkBack will attempt to recognize it, display it on the screen and even play back the actual spoken phrase, allowing you to confirm the transcription.

Its unsophisticated interface belies the richness of the underlying tool suite. Talkback does everything that a major SR application needs to do and does it in only a few lines. It initializes the SR and TTS engines, accepts input from the microphone, and provides audio playback through speakers. It also recognizes words, displays them on the screen, and speaks them back with the computer's TTS voice.

You are encouraged to look at the code. To keep it simple, a DOS interface is used. Therefore, it needs very little code for maintaining the user interface. The few messages that exist are posted to the screen using the printf() command. The Coffee Tutorial samples provide a more detailed explanation for writing speech-enabled applications.

Using TalkBack

TalkBack is a console application but may be treated as a Windows one. Double-clicking its application icon launches it and it will be run in a new console window. You may also launch it from within a console window by typing "Talkback" along with any command line parameters. The application will then run in the current console window.

TalkBack accepts up to two optional parameters:

Talkback -noTTS -noReplay

-noTTS disables the TTS voice from speaking the recognized result.

-noReplay disables the playback of your actual spoken word. By default, neither function is disabled.

Once started, you will be prompted on the screen with:

I will repeat everything you say.
Say "Stop" to exit.

You may talk into the microphone. Any word or words may be spoken, although it works best with only a few words at a time. This recognition may take a few moments depending the system's capabilities. The SR engine processes the phrase and displays it on the screen:

I heard:  though

If replay is not disabled, it plays back the recorded word by speaking "…when you said:" and the actual word. This may be repeated any number of times. As part of the demonstration, the SR engine will force a match for the word even if the match is an unlikely one. For example, a nonsense word will return a match. A more robust application would likely question or mark the word to warn you that no reasonable match was found.

To exit the application, you may say "stop." Upon a successful recognition, TalkBack will quit. You may also use the close box on the window or, if running from the command line, use CTRL+C.


Simple Telephony

Introduction

Simple Telephony is a speech recognition (SR) and text-to-speech (TTS) engine that uses telephony (TAPI) interfaces. It is intended for use with a phone system, allowing you to dial in and talk to Simple Telephony.

Using Simple Telephony

The functionality of Simple Telephony is very similar to the SDK sample, TalkBack. When prompted, you may speak a word, phrase, or a short sentence and Simple Telephony will attempt a recognition. It then speaks the result using a synthesized voice. It will also play back your recorded voice allowing you to confirm the accuracy of the recognition. Please see the TalkBack User's Guide for more information.

Setting Up Simple Telephony

Simple Telephony has two requirements. First, it only runs on the Windows 2000 operating system. Second, you must have a modem, since it is a telephony application. There is no restriction on the type or speed of modem, although it must be properly connected and able to process data. Follow the instructions that came with the modem for specific loading and installation details.

Using Simple Telephony

The application's interface is simple. The phone line or connection displays the status. Commonly it will display "Waiting for a call..." or "Answering..." or "Connected." Several error messages may also be displayed.

Simple Telephony will not automatically launch if a phone call is made to it. It must be already running prior to answering calls. If Auto Answer is selected, Simple Telephony will pick up the call otherwise, you must click the Answer button. Simple Telephony will greet you and ask you to speak something. After the recognition and subsequent play backs, the call will automatically terminate. Only one recognition is allowed per session.

Options

Auto Answer

Directs Simple Telephony to automatically answer incoming calls. Select this to have calls picked up on the second ring. If Auto Answer is not selected, you must click Answer to answer all calls manually.

Answer

Answers incoming calls manually. If the Auto Answer feature is not selected, you must click Answer to answer all calls manually.

Exit

Quits Simple Telephony. Any current call is disconnected and the application exits cleanly.


Sample Speech Recognition Engine

Introduction

The sample speech recognition (SR) engine demonstrates the design, compilation, installation, and testing for engines. The following is basic information you should know about the SR sample engine:

  1. It is not necessary to compile the sample SR engine project (sreng.dsp) to install the .dll.
    The sample SR engine (sreng.dll) is installed with the Speech SDK 5.0.
  2. The path name is: C:\Program Files\Microsoft Speech SDK5.0\bin\sreng.dll (if you install the SDK under C:\Program Files)
  3. The C++ project file C:\Program Files\Microsoft Speech SDK5.0\Samples\CPP\Engines\SR\sreng.dsp

How to Register the Sample Engine dll from the Command Line:

Although applicable to different engines, this demonstration uses the sample sreng.dll. You will need to register the engine with the operating system. At the command line type:

 regsvr32 C:\Program Files\Microsoft Speech SDK5.0\bin\sreng.dll

After the .dll registration process, the sample SR engine is available on the current computer.

How to Set the Sample Engine to the Default Engine

  1. In Control Panel, double-click the Speech icon.
  2. On the Speech Recognition tab, select the SAPI 5.0 Sample Engine from the list of available engines.
  3. Click OK.

Sample Engine Basic Behavior

The sample engine randomly generates context free grammar (CFG) recognition results based on the CFG grammar you select. The sample engine will also generate other events, such as interference and requestui etc.

Sample SR engine notes

  1. The engine does not perform the recognition based on an acoustic or language model. Instead, it retrieves the CFG grammar information from SAPI and constructs random results.
  2. The sample engine provides a basic idea about how to develop an SR engine to interact with SAPI.
  3. The sample engine does not pass the compliance tests.
  4. If you experience unexpected results for the real SR activity, make sure that the sample engine is not in use and that the sample engine has not been set as the default engine.


Sample Engines

Introduction

Sample text-to-speech (TTS) engines are provided to demonstrate the design, compilation, installation, and testing of engines.

Installing TTS Engines

There are two ways to install the engines: command line and compiler.

Command Line Installation

Although applicable to different engines, this demonstration uses the sample ttseng.dll. You will need to register the engine with the operating system. At the command line type:

regsvr32 ttseng.dll
mkvoice wordlist.txt voiceFile voicename

The second command registers the accompanying voice file samplevoice. The voice must be compiled in the same directory as the sample engine. From the SAPI5SDK directory:

Copy bin\mkvoice.exe samples\CPP\Engines\TTS\MkVoice
cd samples\cpp\engines\tts\mkvoice
mkvoice wordlist.txt samplevoice.vce samplevoice
The SDK tool MkVoice creates this voice file (these are also called voice fonts) and it is documented separately. Speech SDK 5.0 provides the sample file samplevoice although you may create your own by following the MkVoice directions.

Compiler Installation

Engines will automatically be registered after compiling if you use the SDK's project workspace. Installation occurs as part of the post-build commands. Likewise, the associated voice fonts will also be registered at the same time. To verify installation, you may examine the registry at:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Voices\Tokens

If successfully loaded, you will see an entry with the file name such as SampleVoice. Please note it is important that you do not manually change or edit any of these registry entries. Only SAPI may modify speech registries and improperly changed ones could irrevocably damage the system. This is provided only for verification reasons.

Setting a Default Engine

Once installed, you can set the TTS engine as the default engine.
  1. In Control Panel, double-click the Speech icon.
  2. On the Text-to-Speech tab, from the list box, select the engine you want. In this case, select the Sample TTS Voice.
  3. A green check mark will appear in the check box, indicating that the Sample TTS Voice has been selected as the voice for the default engine.
All speech applications will now use this voice. As a confirmation of the selection, you will hear the voice speak, "You have selected the Sample TTS voice as the computer's default voice."

Confirming the Default Engine

To test the TTS, select any application that uses a voice. SDK provides a convenient application--TTSApp.exe.


Grammar Compiler

The SAPI grammar compiler (gc.exe) creates binary grammars from extensible markup language (XML) defined grammars.

The following topics are discussed in this section:

  • Introduction
  • Using the Grammar Compiler
  • Grammar compiler custom build settings

Introduction

The SAPI grammar compiler is divided into two parts, the front-end section and the back-end section. The front end parses the grammars described in XML and optimizes the XML text formatted grammar if requested by the application. For example, removal of left recursion. The front end then calls the back-end compiler to convert the internal representation into the SAPI binary format.

return to the top of this page Back to top

Using the grammar compiler

The SAPI 5.0 grammar compiler (gc.exe) can be used from the command line, or added to the Microsoft Visual C++ custom build environment.

Using GC from the command line

When compiling XML grammars from the command line, the following options are available:

Command line argument | Definition
/O | The file name and path information associated with the compiled grammar output file. (CFG) For example: My_CFG_Grammar.cfg
/H | The file name and path information associated with the header file that receives the # define information. For example: My_Header.h See specifying the grammar compiler /h option for more information.
file_name.xml | The file name and path information associated with the grammar file. (XML) For example: My_XML_Grammar.xml

return to the top of this page Back to top

Specifying the grammar compiler /h option

The SAPI grammar compiler can add the XML ID tag NAME and VAL contents to a specified header file. By choosing the /h compile option, the contents of the XML ID tags are added to the specified header file in the standard C-style #define format.

The following illustration shows the result of the /h command line argument while compiling the dictmode.xml file.
Notice the contents of the XML ID tags NAME and VAL are added to the dictmode.h file as c-style #define statements.

return to the top of this page Back to top

Grammar compiler custom build settings

The SAPI 5.0 grammar compiler (gc.exe) can be added to the Microsoft Visual C++ custom build environment. The XML grammar compile settings can be pre-configured in the project custom build settings. Each time the XML grammar is compiled, the grammar compiler uses the settings that are specified in the custom build settings.

The Speech SDK 5.0 sample application Dictation Pad illustrates the use of the grammar compiler in the project custom build settings. Compile the dictpad.dsp project and review the Dictation Pad sample application project settings to learn more about how the grammar compiler can be added to the Microsoft Visual C++ custom build environment.
The Dictation Pad SDK sample application can be found in the following location: C:\Program Files\Microsoft Speech SDK5.0\Samples\CPP\DictPad\dictpad.dsp.

Custom build settings

return to the top of this page Back to top


SDK Tutorial

This section provides an SDK Tutorial overview. Two topics are available. The Coffee series a complete example with documentation. Code projects are available for each Coffee topic and are installed in the SDK. The TTS section has a two brief tutorial overviews for using TTS. While code examples are listed directly in the documentation, no separate project is available.

The following topics are available:

Coffee Examples
  • Setting Up SAPI 5.0
  • CoffeeS0 Tutorial
  • CoffeeS1 Tutorial
  • CoffeeS2 Tutorial
  • CoffeeS3 Tutorial
  • CoffeeS4 Tutorial
  • CoffeeS5 Tutorial
  • CoffeeS6 Tutorial
TTS Examples
  • TTS Tutorial Example
  • TTS Events Explanation


MSDN Authoring Template

Chapter 1

Setting Up SAPI 5.0

Introduction

The SAPI 5.0 Reference API is an excellent guide for programming speech applications. With the large number of methods, interfaces, structures, and enumerations SAPI 5.0 offers, the reference API is a required document. However, those who are new to speech applications may be a little lost at first, as the reference API makes little attempt to weave all the parts together.

The goal of this book is to help you write properly structured SAPI 5.0 applications using a series of examples called Coffee. The application uses a coffee shop motif designed so that you may place orders, talk to management, or buy items at the store. Since each application builds upon the previous one, you need to understand each step before moving on to the next one.

What you need to know

There are two prerequisites for using these examples. First, that you generally understand graphical interface programming and specifically understand native Windows programming. Although you will concentrate on SAPI 5.0 topics, the Coffee examples are Windows applications. The majority of the code in the samples is a framework and is used only for running the application. It handles keyboard, mouse, screen updates, and other processing messages. Unless there is a relevance to SAPI 5.0, much of the code is not going to be discussed.

Second, you should have some experience with C/C++ programming. The intent is to keep programming simple and consistent. There are other models and languages for programming Windows including Visual Basic, JAVA and Microsoft Foundation Class (MFC). MFC adds a layer of complexity that you do not need at the moment. After you gain proficiency with SAPI 5.0 code, theory, and implementation, feel free to change approaches.

Additionally, SAPI 5.0 is common object model (COM) based. Although COM proficiency is not required, some understanding of it is. To further simplify COM, SAPI 5.0 uses active template library (ATL) functions to complement COM. You need a basic understanding of COM smart pointer CComPtr. Additional information about these topics may be found in MSDN or in the myriad of books available through popular bookstores.

Using this book

Each chapter describes the important concepts introduced. The discussion follows the code and provides examples from the Coffee code itself. The first Coffee example is the foundation of the entire process and is slightly longer than other chapters. The narrative is a brief overview and presents enough material to complete the example, though not enough to exhaust the subject.

Each Coffee chapter builds on the previous examples. Since no code is ever removed, changes from chapter to chapter may be found by comparing files or entire folders using a difference engine. Windiff is a convenient application since it is available through the command window of the operating system. However, in general, any application may be used.

What you need to get started

VC 6.0 with Service Pack 3 or later. In general, any 32-bit C compiler will work. However, the samples assume Microsoft Visual Studio 6.0 SP3 or later. Platform SDK (PSDK) is also required. To save disk space, you can load a minimal configuration. See Getting Started in the reference API for more detailed information.

SAPI 5.0 Installation source. This may be a SAPI developer’s CD or the installer through another source (if available) such as the Microsoft Web site. Regardless, you are encouraged to install SAPI 5.0 only through an installation package. The SAPI CD installs all the required components including the required dynamic link libraries (DLL), headers, registry entries, and other resources.

Speech SDK 5.0

In addition to having up-to-date source files and examples, you need the reference API to look up interfaces and methods. Even though the examples are well commented and the tutorials contain narratives about the calls and approaches, the reference API explains each call in detail and to a greater extent than is possible from other sources.

Setting Up

To configure your system for speech recognition, load SAPI 5.0 if you have not already done so. To test the installation, go to Speech Recognition tab of Speech Properties, and speak into the microphone. The bar in the Microphone window registers the volume level. If it registers, the microphone works. Click the Text-to-Speech tab. To test the output voice, click Preview Voice. The text in this section will be spoken, highlighting the words as they are spoken. If this is the case, the output also works. If neither works, see the Troubleshooting section.

It is also recommended that you use the microphone training wizard. From the Speech Recognition tab, click Train Profile. The training wizard instructs you in microphone placement and input level adjustment so that SAPI is able to recognize your commands. For the Coffee examples, the list of commands is quite limited and this training is not required. Speak clearly and deliberately into the microphone and Coffee should be able to recognize your words.


MSDN Authoring Template

Chapter 2

CoffeeS0

Introduction

This first example represents a basic speech recognition (SR) application. The foundation is presented and over the next series of examples, progressively complex features will be added.

The example will focus on setting the foundation of speech-enabled applications in general. The following topics will be discussed:

· Initialization: Setting up engines and grammars

· Events: Definition of events, notifications, interests, expanding events

· Phrases: Definition of phrases, grammar rules, accepting phrases

Running the Example

Run the application by executing CoffeeS0.exe. A window appears and along the top you are greeted: Welcome to the SAPI coffee shop. Speak for service! Since this is a simple example, your only option is to say, “Please go to the counter.” Variations are also accepted such as, “Please go to counter,” or “enter counter.” For this example, even “go to shop” and “enter the store” are recognized for reasons examined later. Speak clearly to activate the voice recognizer and the screen will change to “Please order when ready!”

Perhaps limited in functionality (you cannot actually order coffee in this coffee shop) it does represent the fundamentals of speech recognition. That is, commands are spoken, recognized, and executed. Look at the Coffee code in Coffee.cpp. It looks very much like a simple Windows application, because, in fact, that is what it is. The application brings up one window, and draws two strings to it. To keep the code to a minimum, it does not support keyboard operations, a mouse, or a menu. To exit the application, click the Close button in the upper right of the screen.

The speech API commands are interspersed with the Windows API. Chances are, if you do not recognize a command as belonging to the Windows API, it does not. It belongs either to SAPI 5.0 or it is part of the structure needed to process speech. For instance, there is a new message, WM_RECOEVENT, which defines the application. It is not a part of SAPI but the application structure needs this message to process speech. In the same manner, several routines are also defined by the application (InitSAPI(), CleanupSAPI(), ProcessRecoEvent(), and ExecuteCommand()). Inside these Coffee-defined functions are the SAPI 5.0 method calls. There are also a few #defines at the top of the file for including SAPI 5.0-specific headers.

Header Files

Before starting out with any coding, the proper headers need to be introduced.

#include "stdafx.h"

// Contains definitions of SAPI functions
#include <sphelper.h>

// Contains common defines
#include "common.h"

// Forward declarations and constants
#include "coffee.h"

// This header is created by our grammar compiler and has the rule ids
#include "cofgram.h"

Of the five headers, two are application-specific to the Coffee series. Coffee.h contains the function prototypes and global variables for the application and common.h lists #defines for the application’s windows and other features. As the samples change and expand, these two files are updated. However, this has no effect on the speech aspects of the program. A third file, stdafx.h, is related to COM programming and it is maintained by the compiler. This file can also change and have no impact on speech-related issues.

The other two files are speech related. Sphelper.h is provided by SAPI 5.0 and is the header file for the helper functions. For the programmer’s convenience, there is a list of functions consolidating a series of SAPI API methods into a single call. You are not required to use helper functions for SAPI programming, although doing so may simplify programming. The Coffee samples use helper functions whenever possible. Finally, the grammar compiler uses cofgram.h. A grammar is a list of words available to the application. The topics of grammars and the compiler in general will be discussed in later chapters.

For SAPI 5.0 to work, it needs to be initialized. This is done in four basic steps with each step depending on the success of the previous one. You should always check the return values just as in Windows programming.

Initialization

Step one: COM

First, initialize COM making sure it is present and active. Use the COM command CoInitialize() and later CoUninitialize(). To ensure COM is active throughout the application’s session, these commands are usually nested around the main message loop.

    // Only continue is COM is successfully
initialized
if ( SUCCEEDED( CoInitialize( NULL ) ) )
{
// Main message loop:
while (GetMessage(&msg, NULL, 0, 0))
{
//Code here
}
CoUninitialize();
}

Step two: Recognizer Object

Second, create the recognizer object. This object provides access to the recognition engine.

//Global Definition
CComPtr<ISpRecognizer> g_cpEngine;

// create a recognition engine
hr = g_cpEngine.CoCreateInstance(CLSID_SpSharedRecognizer);
if ( FAILED( hr ) ) //Leave application

A single instance of a recognizer object is created. There are two options for setting up this object: shared and in-process (InProc). Use the following class identifiers (CLSIDs) to set up the instance:

CLSID_SpSharedRecoContext | Creates a shared resource instance
CLSID_SpInprocRecognizer | Creates an InProc or non-shared resource instance.

Shared Instance

Shared instances allow resources such as recognition engines, audio input (microphones), and output devices to be used by several applications at the same time. This is the preferred option for most desktop applications. It is common for a desktop system to have only one microphone and shared instances allow different applications such as a browser, word processor, and a game to use the microphone. Any application using a shared instance will start the SAPI server process. This is an executable program running in the background. It delivers events to the owning application.

InProc Instance

InProc instances, however, allow one and only one application to control the resources. This includes the microphone and speech recognition engine. Using an InProc procedure is very restrictive and you should use it only in special circumstances. For example, you would use InProc if you wanted the entire microphone input to be channeled through one application. Telephony applications are a good example of the need to restrict use to one microphone or audio input source.

Step three: Recognition Context

Third, create a recognition context for the engine.

       //Global Definition
CComPtr<ISpRecoContext>
g_cpRecoCtxt;

// create the command recognition context
hr = g_cpEngine->CreateRecoContext(
&g_cpRecoCtxt );
if ( FAILED( hr ) ) //Leave application

A context is any single area of the application needing to process speech. A simple case (like CoffeeS0) assigns the entire application to only one recognition context. No matter where you are in the application, all speech events and messages are handled by the same procedure. Alternatively, each part of the application may have a different context. Individual windows, dialog boxes, menu bars, or even menu items (such as the Open or Print menu items) may have their own context. Events or messages generated from these areas are processed by their own procedures. This is similar to the way individual windows process events and messages in standard Win32 applications. That is, each window is assigned a window procedure that handles all the events and messages. In the same way, each recognition context is assigned a procedure as well. This way, you have greater control over the program and the handling of speech events. Contexts are created dynamically the moment they are needed and destroyed afterward. Alternatively, you may create them once and retain them throughout the application’s life. However, Coffee is a simple example and uses only one context.

IspRecoContext is an important interface and will be the primary means for recognition. From the interface, the application can load and unload grammars as well as get and respond to events.

Step four: Loading Grammars and Rules

The last major part of the startup sequence is loading the grammar. A grammar specifies what the speech recognizer will recognize.

// Load our grammar
// user defined ("SRGRAMMAR") resource type.
hr = g_cpRecoCtxt->CreateGrammar(GRAMMARID1, &g_cpCmdGrammar);
if ( FAILED( hr ) ) //Leave application

hr = g_cpCmdGrammar->LoadCmdFromResource(
NULL,
MAKEINTRESOURCEW(IDR_CMD_CFG),
L"SRGRAMMAR",
MAKELANGID( LANG_NEUTRAL, SUBLANG_NEUTRAL), TRUE);
if ( FAILED( hr ) ) //Leave application

Essentially there are two types of grammars. One is for dictation and the other is command and control. The dictation grammar is a more free-formed approach to speech. You are able to draw on a very large portion of the body of words for the language. Command and Control is a much more limited list of words. For Coffee examples, you only need certain words and only then to move around the store and to order drinks. It makes no sense for Coffee to know about the word “opisthognathous” so why even attempt to find it? Besides, you are going to sip your coffee anyway.

The Coffee list is a pregenerated set of commands stored as a resource internal to the application. Coffee.xml saves this in a human-readable format. Extensible Markup Language (XML) is the markup language used to generate the grammar and the format that the file uses as defined by SAPI. You need to compile this file into a binary version so SAPI 5.0 can use it. You can do this ahead of time or on the fly. Here it has been precompiled it so that the grammar is delivered inside the application. The SAPI 5.0 SDK has a grammar compiler called GramComp, delivered in the tool suite. Grammars are covered in more detail in the CoffeeS1 example.

IspRecoContext, as mentioned in Step 3, creates the grammar. Once made, you populate the grammar with words from command list. Use ::LoadCmdFromResource, since it is stored as an application resource. Another way you could load it is from other sources, including an external file, memory, or an existing object. After you have retrieved the grammar, you need to set the rules. As a convenience, the XML itself activates the initial set of rules. Specifically the TOPLEVEL=”ACTIVE” tag in coffee.xml does this. The following is an example of an explicit application setting:

// Set rules to active, we are now listening for commands
hr = g_cpCmdGrammar->SetRuleState( NULL, NULL, SPRS_ACTIVE );

The method explicitly sets any rules it encounters to become active. Because the two NULL parameters values did not exclude any rules, all of them were activated. You can also deactivate rules using this method. If the call fails, the application posts a message for the most likely reason.

Events

CoffeeS0 is now an application that can take speech input. It is processing speech in the background. When SAPI has information, it returns it back to the application. SAPI will notify you when an event happens. In short, an event is a condition of special interest to SAPI. Examples of events include, when a sound is first detected on the microphone (SPEI_SOUND_START), when it ends (SPEI_SOUND_END), or when it successfully completes a word recognition (SPEI_RECOGNITION). SAPI maintains several types of events, of which the enumerated type SPEVENTENUM, maintains a complete list. Two important concepts tie events into the application: Notifications and Interests.

Notifications

A notification indicates that a SAPI event has occurred and the application might want to react. It does not relay exactly what happened – for that you will have to dig a little deeper.

To react to notifications, the application has to associate them with specific procedures. There are several ways to do this. The ISpNotifySource interface has four methods: SetNotifyCallbackFunction, SetNotifyCallbackInterface, SetNotifyWin32Event, and SetNotifyWindowMessage. An additional one, ISpNotifySink::Notify, provides a generic method allowing for special or unusual conditions. You can use any or all of these methods depending on your needs. For instance, it might be easier for an application to handle a notification by directly calling a function (::SetNotifyCallbackFunction) such as bringing up a new dialog box or automatically logging the activity in a file. Three of the methods, ::SetNotifyCallbackFunction, ::SetNotifyCallbackInterface, ::SetNotifyWindowMessage, require a message loop and therefore may only be used by Windows applications. You can use the other two, ::SetNotifyWin32Event and ::ISpNotifySink::Notify, without a message loop to provide additional flexibility.

CoffeeS0 chooses to send the notification through a Window procedure. Since SAPI messages are not system level messages, you have to tell the application explicitly about them.

hr = g_cpRecoCtxt->SetNotifyWindowMessage( hWnd, WM_RECOEVENT, 0, 0 );
if ( FAILED( hr ) ) //Leave application

This method associates a message to a specific window. Afterward, any events SAPI passes back will be received by the application in the singular form of a WM_RECOEVENT message, and then sent to the window pointed to by hWnd. You have the option of directing the wParam and lParam window parameters as well. Because CoffeeS0 is not concerned about them here, the application has set them to zero or NULL.

Interests

An interest is a flag allowing or restricting the kind of events SAPI passes back. By default, SAPI sends all events back to the application. So far that is more than 30 different kinds of events. You cannot be concerned about all of them. In reality, CoffeeS0 truly cares about one type: the successful word recognition or SPEI_RECOGNITION event. You can tell SAPI to pass back only this one event. To filter these events, use ::SetInterest.

hr = g_cpRecoCtxt->SetInterest(
SPFEI(SPEI_RECOGNITION), SPFEI(SPEI_RECOGNITION)
);
if ( FAILED( hr ) ) //Leave application

This sets the interest to just one message, SPEI_RECOGNITION. That is, only a successful recognition event generates a notification. SAPI will not notify the application on any other event.

You can define multiple interests using the exclusive OR operator. Two values are set. The first parameter lists the interests in general. That is, it defines all the events you are, or could be, concerned with for the time being. The second parameter lists the events to be queued so that the application can handle them in due time. In this tutorial, you are interested in every occurrence of SPEI_RECOGNITION, even if they come so quickly that the application cannot handle them at one time. Often, these two parameters will be identical but, obviously, they don’t have to be. SPFEI() is a helper function used to reformat the enumerated events into a ULONGLONG number.

Is the application now fully functional? Again, almost. It is true, the application can initialize SAPI, accept speech from a microphone, attempt to recognize it, and send back an event to the application if a word is matched. You still need to put the message in the event loop in order for the application to handle the event. Because it is already defined and known by application, WM_RECOEVENT can be put in like any other message. The following code fragment is from main window procedure WndProc:

// This is our application defined window message to let us know that a
// speech recognition event has occurred.
case WM_RECOEVENT:
ProcessRecoEvent( hWnd );
break;

Each time SAPI is ready to return a word, it sends out an event. The application receives this message as WM_RECOEVENT. The main message loop picks it up and, in this case, sends it to the ProcessRecoEvent() for routine processing.

For speech recognition, this event is handled in a slightly different manner than the way Windows approaches it. Normally, Windows sends the exact message based on the event. In this way you do not have to determine if it were a mouse or keyboard event; the message provides this information. SAPI, on the other hand, does not. You still need to know the exact nature of the event as well as the number of events waiting in the queue.

For this reason, SAPI introduces its own event description system. See the following code for an example:

void ProcessRecoEvent( HWND hWnd )
{
// Event helper class
CSpEvent event;

// Loop processing events while there are any in the queue
while (event.GetFrom(g_cpRecoCtxt) == S_OK)
{
// Look at recognition event only
switch (event.eEventId)
{
case
SPEI_RECOGNITION:
ExecuteCommand(event.RecoResult(), hWnd);
break;
}
}
}

Three items are needed to fully describe the event. The first item is CspEvent. This is a helper class function that contains several useful functions and is an SPEVENT structure. One method, ::GetFrom, does two things at the same time. It retrieves the next event from the queue and loads the corresponding information into the SPEVENT structure making it ready for your inspection.

The second item is to determine which SAPI event actually took place. At this point you only have to look at the member eEventId. If it is an event you are not interested in, skip it and keeping looking or waiting for other events.

The last item is to match the event with your needs. In this example you are interested only in SPEI_RECOGNITION. If there is a match, you are closer to your goal of finding out which word was spoken. The switch statement handles that for you.

Phrases

Determining the actual phrase is the last part of the process. Having initialized SAPI, received a notice that SAPI has identified a word, and had the application process the message, you now need to isolate the word.

SAPI returns the word information in a list or a series of lists. These lists contain not only the word but also additional information about the word, words or the entire phrase. Examine the following code:

void ExecuteCommand(ISpPhrase *pPhrase, HWND hWnd)
{
SPPHRASE *pElements;

// Get the phrase elements, one of which is the rule id we
specified in
// the grammar.  Switch on it to figure out which
command was recognized.
if (SUCCEEDED(pPhrase->GetPhrase(&pElements)))
{
switch ( pElements->Rule.ulId )
{
case
VID_Navigation:
{
switch( pElements->pProperties->vValue.ulVal )
{
case VID_Counter:
PostMessage(
hWnd, WM_GOTOCOUNTER, NULL, NULL
);
break;
}
}
break;
}
// Free the pElements memory which
was allocated for us
::CoTaskMemFree(pElements);
}
}

Without knowing too much about phrase structures, it is obvious that the code drills down into it. This function takes an IspPhrase interface, extracts an exact phrase element (in the form of pElements) and determines which rule has been invoked. Other examples will go one step further and get the exact words spoken. Phrases and rules will be discussed in languishing detail [CoffeeS2].

To understand rules better, look at coffee.xml. Notice that there are two rules defined. RULE ID tags delineate both. The main rule is VID_Navigation. The second one is VID_Place, but this is of lesser importance because it is subservient to VID_Navigation. In essence, coffee.xml defines two sets of phrases. The first set uses the commands “Enter” and “Go To,” and the second set uses the words “counter,” “shop,” and “store.” The recognizer mixes and matches words, selecting one from the first set and one from the second set. Therefore, sentences such as “enter store” and “go to counter” invoke a SAPI rule. There are also optional words that may be used or not used. “Please enter the store” is not only more polite but it is also a valid match. However, the recognizer ignores “please” and “the.” This allows for a more natural approach to speaking and at the same time does not encumber SAPI.

However, this example does not go quite that far. The code stops at the rule level in the switch statement with “case VID_Counter.” Once you have said, “please enter the store,” SAPI invokes the appropriate rule (VID_Navigation). That is why you could have said “counter” or “shop” and still have gone to the same place. In contrast, had you issued “please enter the restaurant,” no rule would have been invoked since no definition includes “restaurant.” In later examples, the exact word will be of more interest.

At this point, a rule has been invoked and caught by the application. Now you can process it as you see fit. CoffeeS0 is interested only in providing you with textual feedback and so it passes a PostMessage() back to the owning window with instructions to change the text. “Please order when ready!” appears on the screen.

Conclusion

Hopefully you are not overwhelmed at this point. SAPI 5.0, like all other systems, requires a certain amount of overhead programming. The intent was to minimize this overhead and allow you to concentrate on the function of the application rather than working with the SAPI code. Most of the code introduced here is initialization and should only happen once during the application launch. With that done, you are free to add additional features.


MSDN Authoring Template

Chapter 3

CoffeeS1

Introduction

With the basic structure for speech recognition firmly in place, CoffeeS1 expands word recognition capabilities. You are not only allowed to navigate around the coffee shop, but also to place orders for different kinds of coffee drinks.

The example will focus on grammars generally and, on Command and Control grammar specifically. The following topics will be discussed

· Grammars: Command and Control, dictation.

· Phrase: Phrase structure, word recognition.

· Grammar Files: XML tagging.

Grammar Types

Command and Control

The last example (CoffeeS0) was not robust. You were limited to about five words. However, they are five words of special interest and they allowed you to move around the application. Using grammar in the Command and Control function limits the use of words and bestows upon them specialized meanings. This is convenient for some uses in applications. In the last chapter, you learned that grammars could have limited recognition contexts such as for menus. As an example, you would want the application to respond, or even to attempt to respond, to certain words relating to menu items or to the menu bar such as “file,” “open,” or “print.”

These words are provided in an exclusive list. If the word is not found, it is not recognized. Also, word order matters in some cases. In CoffeeS0, “Go to counter” was understood and “Counter go to” was not. Using a list approach is also called rule-based or context-free grammar. Words are evaluated according to a fixed set of rules. In short, the word is either in the list or it is not. There is no attempt to figure out the intent of the word based on the words that came before or after it. That is, there is no context for the words.

SAPI 5.0 uses extensible markup language (XML) to create this list. The file may be generated ahead of time or compiled during program execution. Because Command and Control deals mostly with lists, words can be added dynamically and can accommodate new situations easily.

CoffeeS1 addresses word order and the ability to sequence words.

Dictation

Command and Control has obvious shortcomings. As mentioned, it is limited in the words used. Someone has to spend the time to manually define the command set. Often, you will want to speak any word and have it recognized. This is what a traditional speech recognition (SR) program does. That is, allow you to dictate any word, no matter how esoteric, into a word processor and have that word translated into text. For this use of a speech recognition engine, you must move from Command and Control to a dictation grammar. Instead of an XML-based vocabulary, dictation grammar uses a much more extensive range of words and determines each word based on context. The words immediately before and after it are studied and dictation grammar chooses the most likely outcome. For this reason, this is also called a statistical language model (SLM).

SR engines have wide latitude of vocabularies. The Microsoft SR engine that SAPI 5.0 includes 60,000 English words and provides an adequate engine for most people. Other engines are specialized for the legal and medical professions, for example. These can be massive databases generated by commercial firms. In addition, different languages including Japanese, Chinese, German, and Russian are also available.

For as widely disparate as these languages and usages seem, SAPI 5.0 handles them in the same way. The programming approach is very similar. Two other samples provide a dictation approach to speech recognition: SimpleDictation and DictationPad. These may be found on the SAPI 5.0 SDK and are documented separately. Coffee on the other hand, limits itself solely to Command and Control usage.

Phrases

SAPI returns the actual recognized words through a series of structures collectively called phrases. You have seen evidence of this with SPEI_PHRASE_START event indicating the start of the recognition process. For Command and Control uses, it is a two-step process: Determine the activated rule, and then inspect the elements (or words) within that phrase.

CoffeeS0 briefly introduced the first step. While processing a recognition event, you discovered which rule was activated but stopped there. CoffeeS1 takes the next logical step to recognize the exact words used so patrons can get their drinks.

This examination takes place in the ExecuteCommand() routine. As in CoffeeS0, one of the parameters is the phrase. Remember, this phrase is the final result, rather than a hypothesis, so assume SAPI is savvy enough to translate exactly what you said. You will be depending on this phrase for navigation. At this point, you are only interested in which grammar rule was activated. That means the patrons still cannot go to different places in the shop even though they might request to do so. All navigation statements always lead to the counter.

However, CoffeeS1 introduces a new grammar rule: VID_EspressoDrinks. Defined in coffee.xml, this rule lists all drinks available to the customers. Actually, it is several rules bound together that will be discussed later. Again, you are only concerned in the top-level rule of VID_EspressoDrinks. If you place an order that matches this rule, the rule activates and the result is passed back from SAPI. In typical demanding coffee shop fashion, this could be “Get me a iced decaf single tall peppermint whole espresso.” Orders could even include “single triple short tall grande,” and still be valid SAPI grammar although it might raise an eyebrow (if that were possible).

With the order placed and recognition successful, CoffeeS1 now gets to the task of dissecting the phrase. From the original phrase, an IspPhrase interface contains the method GetPhrase()to construct the elements (or word) list.

SPPHRASE *pElements;
if (SUCCEEDED(pPhrase->GetPhrase(&pElements)))

If successful, pElements contains all the information required to construct the sentence. To determine which rule activated and then to learn more about it, look at the Member Rule. This is a structure (SPPHRASERULE) but one that fully describes the rule. The rule ID is found in its member ulId. CoffeeS1 numerically defines the rule VID_EspressoDrinks in the XML file, so that matching becomes easy. Use a simple switch statement in the code to determine the more specific handling routines.

Two things need to be pointed out about the upcoming word list. First, the words are represented numerically rather than by a string. Associating the value of the word to the string itself uses a look-up table. In this case, CoffeeS1 stores the words as a resource in the application.

Second, the actual words are formed by a link list with each word represented by a member in the sequence. The first element is a structure (of type SPPHRASEPROPERTY) pointed to by pElements->pProperties and each subsequent structure uses the SPPHRASEPROPERTY’s pNextSibling member. Traveling this chain is standard link list operation.

            case VID_EspressoDrinks:
{
// This memory will be
freed when the WM_ESPRESSOORDER
ULONG *pulIds = new
ULONG[MAX_ID_ARRAY];

const SPPHRASEPROPERTY *pProp = NULL;
int iCnt = 0;

if ( pulIds )
{
ZeroMemory( pulIds,
MAX_ID_ARRAY * sizeof(ULONG) );
pProp =
pElements->pProperties;
// Fill in an array with the drink
properties received
while ( pProp
&& iCnt < MAX_ID_ARRAY )
{
pulIds[iCnt]
= static_cast< ULONG >(pProp->vValue.ulVal);
pProp = pProp->pNextSibling;
iCnt++;
}
PostMessage(
hWnd, WM_ESPRESSOORDER, NULL, (LPARAM) pulIds );

The bold lines indicate the critical portion of the link list operation. To inspect the elements, the code steps through the links one node at a time until the next node is NULL (meaning there are no more nodes to transverse) or it has already visited at least MAX_ID_ARRAY number of nodes. CoffeeS1 imposes this MAX_ID_ARRAY limitation.

Besides stepping through the link list, this code also stores the words in an internal array for later processing. This not only keeps a record of the words but also helps with sorting. Remember, don’t worry about word order. Customers can say “get me a mocha two percent tall,” and still end up with a tall two percent mocha. However, if you do change the word order then you need the ability to sort internally. To indicate empty array elements, flag them with a zero, hence the Win32 ZeroMemory()call. You can use other methods; this one was just convenient for this example.

After going through the list, CoffeeS1 is ready to display the newly derived information. A message is passed to the owning window (WM_ESPRESSOORDER) indicating the application has additional processing. At this point, SAPI is no longer involved. SAPI will even free the objects it created although CoffeeS1 must manually free pElements since it manually created it. Even so, COM is smart enough to delete any nodes in the link list associated with the list. The rest of the processing is on CoffeeS1’s part and mostly to update the screen. When you speak again, the whole process above is repeated.

Grammar Files

As mentioned, the Coffee examples use Command and Control grammar. This is a discrete list of words associated with certain rules. Coffee keeps this list in two forms. An XML-based file allows you to maintain this list. Ultimately, SAPI can only read a binary or compiled version of that file. This is a grammar configuration that is saved with the cfg file suffix. It was by clever design that CFG not only means “configuration,” but also “context free grammar.” Approbation aside, grammar files may be generated dynamically during the application’s run time. If it is provided with only an xml file, SAPI will compile the file automatically and use the resulting grammar. On the other hand, the grammar may also be compiled ahead of time by the programming team. This restricts access to the vocabulary so users may not change them unexpectedly. This method is also faster for applications since no compiling time is required during operation. The SAPI SDK application provides a compiler called GramComp. Use of this tool to compile grammars is documented separately.

SAPI defines the XML tags and their uses and lists them in Reference API. For a more complete discussion, see Text grammar format. As a brief overview of the structure, look at coffee.xml in the CoffeeS1 project. There are several rules defined but only two are considered top level: VID_EspressoDrinks and VID_Navigation. These are the significant rules for SAPI. When a rule match is made, it is one of these IDs that is passed back to the application. Also, look at ExecuteCommand(). The two case statements coincide with the top-level rule names.

The TOPLEVEL tag within the RULE statement gives these rules their special status. Not only does this identify the rule as being top level, but it also sets the activation state. Only top-level rules may be activated or deactivated. SAPI recognizes active rules and conversely does not recognize deactivated ones. The application may change the state of the rules during execution. If a rule is no longer needed, it may be deactivated. This allows you to turn rules on and off based on the current recognition context. For example, if you have a menu or menu item deactivated, SAPI will not need to attempt to recognize the words associated with it. When the menu is active again, the rule will likewise be activated.

The allowable words or phrases are listed inside the rule. The words or phrases may be optional or required. As the name implies, optional words are not required for a successful rules match. SAPI adds them as a convenience to the speaker. “Please enter the shop” is natural and pleasant sounding as opposed the demanding version of the statement. Required words are, of course, required. However, you can present an alternative word list from which any one word can be used to complete the match. In the case of VID_Navigation, you can say either “enter,” or “go to,” but not both.

In the same manner, you may reference other rules but not other top-level ones. Continuing the VID_Navigation example, the last portion of the requirement is that the rule VID_Place must be successfully matched. The three alternatives, “counter,” “shop,” and “store” are defined as VID_Place. If you say one of these three words, the rule is successfully matched. Upon successful completion of all the requirements, the top-level rule, VID_Navigation matches, and an SPEI_RECOGNITION event passes back to the application.

Additional study of coffee.xml allows you to see how complex rules are constructed. The other rules are basically the same format and follow the same structure. Look up unfamiliar tags in the “Text grammar format” section of the reference API. Curiously enough, the program is case sensitive. “The” and “the” may be duplicated as entries. They may even have the same ID such as <ID NAME=”The” VAL=”1” /> and <ID NAME=”the” VAL=”1” />. While this case has the same pronunciation, consider other words such as “Polish” and “polish.” This applies equally to rule names. There is no requirement for engines to recognize the words as different; however, engine vendors may want to do so. By making the word case sensitive, it allows newer engines to take advantage of these differences.

The first portion of the file assigned numeric values to the individual elements. SAPI does not require this, although in the CoffeeS1 example, it allows you to sort the words. The sorting is based on the “VAL=” tag. Remember to keep the words actually found in the array pulIds for this purpose.

Activating the rules is the same as in CoffeeS0.


MSDN Authoring Template

Chapter 4

CoffeeS2

Introduction

This chapter introduces two new concepts: navigation and politeness. Not only will you be able to go to the counter (this is the same as in CoffeeS1) but the manager will also open the office to you. However, you will not be able to order drinks from the office. The second feature will ask the patron to repeat the order if it was not clearly understood.

The following topics will be discussed:

· Grammars: Rules activation and deactivation.

· Events: Expanding events, SPEI_FALSE_RECOGNITION.

Grammar Rule Activation

CoffeeS0 and CoffeeS1 introduced the concept of rule activation/deactivation in only the broadest sense. In fact, those rules made two assumptions. The first is that the grammar rule itself wanted the rules on by default and the second assumption was that the application wanted them on by default.

The Command and Control grammar sets the initial (or default) state of a particular rule to either active or inactive. In the XML file, the top-level rule is defined as:

<RULE ID="VID_Navigation" TOPLEVEL="ACTIVE">

If nothing changes this status, it remains in that state for the duration of the application’s lifespan. Setting “INACTIVE” will set the state to “off” by default.

However, you will need to react to the changes made by the users. In some situations, the grammar rule may no longer be appropriate. In CoffeeS1, you ordered drinks from any location. That is, from the counter, store or office. CoffeeS2 is more discriminating and limits drink order placement to the counter. As another example (and perhaps a more practical one), if you dictated to a speech-enabled word processor, the Command and Control words would no longer require special actions. If you spoke the word “menu,” it would confuse you to suddenly see a menu drop down.

In either case, you need a way to disable the grammar. A drastic method would be to simply unload the entire grammar when it was no longer required. However, in CoffeeS2’s instance, both the navigation and drink order rules are in the same file. If you unloaded the file, you would lose both rules altogether. Rather, selectively activate or deactivate individual top-level rules.

In the past, Coffee has simply accepted the rules as they were.

hr = g_cpCmdGrammar->SetRuleState( NULL, NULL, SPRS_ACTIVE );

Passing NULL for both the rule name and rule value, you accept all the rules in the file as they stand. CoffeeS2 initially places caffeine-seeking patrons outside the shop. Their first command has to be a navigational one. Because drinks cannot be ordered at any place other than the counter, you must deactivate the rule. From InitSAPI() after LoadCmdFromResource():

hr = g_cpCmdGrammar->SetRuleIdState( VID_EspressoDrinks,
SPRS_INACTIVE );
hr = g_cpCmdGrammar->SetRuleIdState( VID_Navigation, SPRS_ACTIVE );

The second line is included for completeness. If not included, the state would still be activated since that particular top-level rule is active. In the same vein, you could use the CoffeeS1 method and activate all the rules at one time and then selectively change rule states. CoffeeS2 has only two rules so either way you have the same number of lines of code.

If you activate a rule, you can also deactivate it. Deactivation is set in the CounterPaneProc() procedure, WM_INITPANE message. The placement of the call will be discussed in a moment. When the time comes, the rule deactivates with:

g_cpCmdGrammar->SetRuleIdState( VID_EspressoDrinks, SPRS_INACTIVE );

In order to choose which of the two locations (the counter or office), CoffeeS2 has to check its ExecuteCommand(). The matched grammar rule holds this information. By contrast, CoffeeS1 only wants to know if any navigation rule was invoked, not necessarily which one. For this new information, you can determine the rule almost identically to the way in which the case statement VID_EspressoDrinks determines drink orders. Given an IspPhrase interface with ExecuteCommand(), use GetPhrase() to retrieve the phrase returned by SAPI.

The next step between CoffeeS1 and CoffeeS2 appears to be different but fundamentally it is the same. Look at the same value structure that CoffeeS1 used to determine the rule :

switch( pElements->pProperties->vValue.ulVal )

Remember that in the XML grammar file (coffee.xml), you not only numerically defined each drink characteristic individually, but you also defined each rule and sub-expression. That is, although “Decaffeinated” is defined (VID_Decaf) as a drink characteristic. You also have the OrderList rule (VID_OrderList) allowing you to use multiple characteristics and a top-level navigation rule (VID_Navigation). This convenience allows you to look in one location rather than drilling down into structures. The difference in the two switch statements for ExecuteCommand() is that case VID_EspressoDrinks goes one step further and moves along a possible link list where case VID_Navigation needs to look at this information only once.

Expanding Events

Up until now you have been interested in only one event: SPEI_RECOGNITION. SAPI currently handles more than 30 different kinds of events. To better understand the process, suppose you wanted to add two new events to your repertoire, a sound start (SPEI_SOUND_START) and a sound end (SPEI_SOUND_END). The speech recognition engine triggers these events when the microphone detects or stops detecting sound. In the best case scenario, it is your voice triggering this event. However, it may be another sound such as a phone, a cough, or any one of a myriad of cacophonous noises.

It is the event SPEI_SOUND_START that initiates SAPI’s recognition attempts. Once detected, SAPI will start processing on the audio stream. It will listen to and process the stream concurrently. That is, you don’t have to stop speaking before it attempts to recognize your voice. After SAPI detects a sound start, it begins processing and sends an SPEI_PHRASE_START event. During the recognition process, interim SPEI_HYPOTHESIS events indicate attempts are being made. In brief, a hypothesis is the current best guess about what you have spoken up to that moment. Having spoken “Please go to the counter,” SAPI might return with five SPEI_HYPOTHESIS events (one for each new word spoken added) along with a phrase structure with the actual words in it.

An SPEI_SOUND_END occurs after a pre-determined amount of time passes during which SAPI detects no useful sounds. After enough silence, SAPI assumes you have stopped, or are pausing between phrases or sentences. SAPI then finishes the recognition process and returns the final decision about what was spoken. Rather than sending back one last SPEI_HYPOTHESIS, it returns one (and only one) of three values.

An SPEI_RECOGNITION event indicates a word match (from one of the available grammars) and with a sufficiently high confidence value to consider recognition successful. As an example, in the CoffeeS0 application you might have said, “Please go to the counter.” This matches a grammar rule and SAPI returns SPEI_RECOGNITION.

An SPEI_FALSERECOGNITION indicates that you probably spoke words (as opposed to a sound of coughing) but SAPI could not find a close enough match to either existing words or grammar rules. For a CoffeeS2 example, you could have said, “please go to the veranda,” or have mumbled inaudibly as morning coffee drinkers are prone to do. Since the former case does not match an existing rule and the latter case implies no word could be recognized, SAPI returns SPEI_FALSERECOGNITION.

An SPEI_RECO_OTHER_CONTEXT indicates a successful recognition was made but that another other application currently running claims it. This is a useful event if there are multiple shared instances running at the same time. For example, if you had said, “please go to the veranda.” CoffeeS2 does not have a rule covering this but suppose another application did. The second application, even if not currently the active one, receives an SPEI_RECOGNITION event and CoffeeS2 gets SPEI_RECO_OTHER_CONTEXT. In a way it offers closure for CoffeeS2.

Since there is a range of events possible and that not all of them are relevant all applications at all times, SAPI allows you to filter them. The ISpEventSource::SetInterest method allows you to determine which events you want to see. Those events not included in the call will not get generated. By default, that is if you never even call ::SetInterest, only SPEI_RECOGNITION events are generated. To allow other events, you would have to modify the ::SetInterest call and explicitly include them for SAPI. For instance, in InitSAPI():

hr = g_cpRecoCtxt->SetInterest(
SPFEI(SPEI_RECOGNITION)|SPFEI(SPEI_SOUND_START)|SPFEI(SPEI_SOUND_END),
SPFEI(SPEI_RECOGNITION)|SPFEI(SPEI_SOUND_START)|SPFEI(SPEI_SOUND_END)
);
//Check return value

As expected, SPEI_SOUND_START and SPEI_SOUND_END are added. Notice if ::SetInterest is called, you have to explicitly add SPEI_RECOGNITION along with any other events. The second parameter is the list of events you want queued. Events can happen faster than even the processor can handle. In this case, rather than losing them, the events get queued and wait to be processed. Most of them the time you would want the two parameters identical. If you were interested in an event to see it in the first place, you would also be interested in doing something with it. However, this is not always the case and SAPI lets you decide.

If you see the events, you will also want to handle them. You can add the two events to the recognition event loop in ProcessRecoEvent().

case SPEI_SOUND_START:
PostMessage( hWnd, WM_STARTEDTALKING, 0, 0 );
break;

case SPEI_SOUND_END:
PostMessage( hWnd, WM_STOPPEDTALKING, 0, 0 );
break;

These post messages back to the hWnd window. CoffeeS1 does not handle these messages past this point but it does demonstrate how it could be done.

Event SPEI_FALSE_RECOGNITION

Moving from the hypothetical to the practical, CoffeeS2 introduces a new event, SPEI_FALSE_RECOGNITION. As explained earlier, the event indicates that you spoke a word but that word was not found in the Command and Control list. If you simply made a noise or if the microphone picked up spurious noises, SPEI_FALSE_RECOGNITION will not be returned. In fact, in those two cases, no recognition takes place because SAPI is smart enough to tell the difference between words and noises. Only sounds close enough to real or acceptable words will trigger an SPEI_RECOGNITION or SPEI_FALSE_RECOGNITION event. At least that’s the intent. It is the responsibility of the speech recognition engine attempts to detect the difference. However, due to the wide latitude of sounds possible and differences among vendors, an occasional SPEI_FALSE_RECOGNITION may be returned instead.

In addition to SPEI_FALSE_RECOGNITION, CoffeeS2 also introduces a time element. You can determine if the patron had spoken a legitimate command and also if that command was spoken over a predetermined amount of time. This allows you to further determine if the utterance was intentional. The temperamental barristas that CoffeeS2 employs ignore a false recognition if spoken too quickly. Otherwise, they will politely ask for the order to be repeated.

First you need to set the interest. In CounterPaneProc(), the WM_INITPANE message actually controls this.

hr = g_cpRecoCtxt->SetInterest(
SPFEI(SPEI_RECOGNITION)|SPFEI(SPEI_FALSE_RECOGNITION),
SPFEI(SPEI_RECOGNITION)|SPFEI(SPEI_FALSE_RECOGNITION)
);

Notice that the interests are chained together with bitwise (rather than logical) OR statements. This way you can add multiple events at the same time. However, the method is in an odd place located in CounterPaneProc() instead of the initialization routine of InitSAPI(). Remember, you can order drinks only in the counter area. Therefore, you want this event available only after the patron has entered the counter area. Although there are several ways to approach the programming logic of this problem, CoffeeS2 sets the events as the patron enters different areas. In contrast, the WM_GOTOOFFICE of the same CounterPaneProc() routine sets the interests again.

hr = g_cpRecoCtxt->SetInterest( SPFEI(SPEI_RECOGNITION),SPFEI(SPEI_RECOGNITION) );

The same logic is used for rule activation and deactivation described above.

When you set the interests at the time of entry to a room, you can now have one message handling routine for all speech events. This is the same process as in the previous Coffee examples, that is, ProcessRecoEvent() handles the messages at that point, and each event is assigned to an action in the subsequent switch statement.

Use HandleFalseReco() in the case of SPEI_FALSE_RECOGNITION. You may use IspRecoResult::GetResultTimes to determine the time element. This passes back a SPRECORESULTTIMES structure containing different timing information for the event. Specifically, dwTickCount keeps the time from the start of the event. Subtracting this from the system’s time yields the duration of the recognition. If the false recognition took longer than the arbitrarily determined value of MIN_ORDER_INTERVAL, the patron is asked to repeat the request.


MSDN Authoring Template

Chapter 5

CoffeeS3

Introduction

TTS! We’re finally going to do text to speech. Up until now, the Coffee examples have limited themselves to simply accepting speech. You could talk to your CoffeeS3 minions and expect to get drinks. Now you can add the second of the two major components of SAPI – that of text to speech.

With all the excitement from the first two examples, CoffeeS3S3 slows the pace down for the moment. You will be pleasantly surprised at how easy it is to add this feature. The design stage placed emphasis on making things simple. In contrast, SAPI 4 required 200 lines to make a so-called simple “hello world” speak. SAPI 5 requires as few as two. This remarkable reduction in code was possible due to consolidation of overhead. SAPI marshals the required elements for you so your programs have less material to access directly. Also, SAPI uses intelligent defaults whenever possible. You may set many of the elements using existing defaults. The Speech Recognition tab of Speech Properties accesses most of these elements, such as voice and speaking rate. Therefore, at the simplest, it is very simple. Naturally, you may override any of these assumptions or defaults. However, for CoffeeS3S3, you will start with simple tasks. Don’t worry; additional features will be addressed in the next few chapters.

The following topic will be discussed

· Text-to-speech: Initialization, implementation and speaking text

Initialization

The initialization routine is almost anticlimactic. It is essentially two lines. This is very similar to setting up a speech recognition (SR) engine as you did in CoffeeS0: declare the interface and create the instance from the class ID.

CComPtr<ISpVoice> cpVoice;
hr = cpVoice.CoCreateInstance(CLSID_SpVoice);

To actually speak something takes one more line of code.

hr = cpVoice->Speak(L" Hello, world ", 0, NULL);

Any application may initialize TTS this way and this is often the preferred method. However, CoffeeS3S3 takes a slightly different approach. SAPI realizes that because SR and TTS are commonly used together, the initialization routine takes a short cut. Basically, the SR engine provides this capability for you. The following is in CoffeeS3S3’s InitSAPI():

CComPtr<IspVoice> g_cpVoice;
hr = g_cpRecoCtxt->GetVoice(&g_cpVoice);

Although it takes the same number of lines of code, TTS is available through the IspRecoContext interface. In fact, this makes the same call to CoCreateInstance(CLSID_SpVoice). The difference is that this method automatically provides the ability to interrupt TTS whenever you start speaking again. This is appropriately known as “barge in.” Without this capability, the TTS voice would continue to speak in the background even if you were talking. This might cause audio feedback. Of course, you can still write your own TTS interrupt routine, but the barge in service is provided as a convenience.

Speech Properties usually provides the defaults. That is, when SAPI is properly installed, Speech Properties will have defaults for all parameters and will use those defaults. These defaults include the voice, speaking rate, and the language used. This is how TTS can get away with being two lines. However, it is possible that the defaults may not be available or valid, and the application must always check the return value.

That’s about it for TTS. You have seen how to initiate a voice and how to speak something. The rest of the code just implements these instances. For example, CoffeeS3S3 talks on five occasions and you need ::Speak at those five times.


MSDN Authoring Template

Chapter 6

CoffeeS4

Introduction

With the completion of the first four Coffee samples, you covered the basics. You were shown how to place orders by speaking and you were able to hear the request spoken back. In both cases, SAPI used its defaults. Specifically, the barrister talked back to you with the voice set using Properties in the Speech Control Panel. In truth, SAPI usually offers much more than single voice. Other voices may be available as well as different languages, engines, or other resources. Many of these can be changed either dynamically (always through the Speech properties and sometimes by the application itself) and programmatically (through the application’s code).

CoffeeS4 introduces the fundamentals of resource management. As in CoffeeS3, you can enter the manager’s office. In the next two tutorials, you are going to manage employees. CoffeeS4 displays the voices available. Like any good coffeehouse help, you will not be able to manage them too much, but you will able to hear one of them speak in the currently selected voice. CoffeeS5 lets you change the voice. In doing this simple task, you prepare for bigger things.

The following topic will be discussed

· Resources

· Managing Resources

Resources

To provide the robust range of options, such as many different voices, recognizers, languages, and user interface dialog boxes, SAPI needs to store this information for later use. This stored information is collectively referred as resources. Resource management, therefore, is the ability to query SAPI for the availability of resources, gather information about the resource, initiate, instantiate, or remove resources as needed. CoffeeS4 finds and displays voices stored as resources, and determines which one is active. By the same techniques, you can filter the voices for language, gender, age, or some other criterion and display only those.

SAPI relies on two closely associated terms and concepts: objects and tokens. COM and object oriented programmers will recognize objects. They are the functional implementation of a class. That is, an implemented object will have memory allocated to it and have its members and methods initialized. Once implemented or instantiated, the object may be used by the application.

However, SAPI only instantiates objects when they are needed. It is a waste of the computer’s memory to have unneeded objects allocated. Therefore, SAPI stores the information needed to create those objects including the voice’s name, language, and GUID. This stored information is referred to as a token. A token is the textual representation of the parameters needed to fully implement that resource. Stated another way, tokens are the parameters for the object. SAPI needs only to read a token at the right time and instantiate an object based on that information.

In addition, there can be many tokens and SAPI organizes them into related groups called categories. Currently SAPI maintains about eight general categories. Of particular interest to CoffeeS4 is SPCAT_VOICES. This category contains all the voice-related tokens.

Each type of token is different and is documented separately. That is, a voice token will have different values and key entries from a recognizer token because each has different functions. Tokens of the same type (such as voice tokens) will contain the same required entries (such as language, gender, age) although vendors may provide additional and non-standard information for their resource.

The following is a sample representation of a voice token. Not all entries are listed.

Token | ValueName | Sample Value | Comments
MSMary |  |  | Required entry. This is the token name.
 | (Default) | MS Mary | Required entry. This is the language independent name.
 | 409 | MS Mary | There may be multiple entries; one for each language the voice supports. At least one entry is required. The numeric value is the standard Windows language code and is in hexadecimal with no leading notation such as “0x.”
 | 809 | MS Mari
 | CLSID | {65DBDDEF-0725-11D3-B50C-00C04F797396} | Required entry. This is the class ID (CLSID) for the object.
MSMary/Attributes |  |  | Required entry. Additional information is available through the attributes although not all attributes are required.
 | Language | 409;809 | Required. This is the language the voice supports.
 | Age | Adult | Required. This is the age for the voice.
 | Gender | Female | Required. This is the gender for the voice.

Methods

Resources alone would be only marginally useful if you could not work with them. Often you will need to know what resources are available. CoffeeS4, for instance, searches for all the voices it can use. Once found, you may need to use that resource either to instantiate an object or to know more about the resource. There is a rich set of interfaces and helper functions available to do just that. Many of these interfaces are found in the Resource Manager set in Application Level Interfaces of the reference API document. In the same manner, the helper functions provide convenient ways to perform a task without having to know all the underlying details at this point.

For instance, the heart of the matter for CoffeeS4 is getting a list of all the available voices. This task can be done by one helper function SpEnumTokens. At the simplest, the call would look like this:

CComPtr<IEnumSpObjectTokens>    cpEnum;          //
Pointer to token enumerator
// Get a token enumerator for tts voices available
HRESULT hr = SpEnumTokens(SPCAT_VOICES, NULL, NULL, &cpEnum);
// check hr result

SPCAT_VOICES restricts the search to the voice category. The middle two parameters (both of which are set to NULL) allow you to filter the resources. In this case, you want all voice resources so no criteria were set. Just as easily, you could have searched for female voices or narrowed it down even more with just the adult, English-speaking female voices.

cpEnum is an interface pointer to IEnumSpObjectTokens. SpEnumTokens does all the initialization work and returns a complete list for you. In this case, the list is a complete set of tokens in SPCAT_VOICES. You can think of IEnumSpObjectTokens as a link list with built-in support functions. Using IenumSpObjectTokens, you can find the next item in the list, skip several items, make a copy of the list, or go back to the beginning of it, among other things. Although some of the methods will be described here, see the reference API section for additional methods and details.

You are on your way to finding and listing all voices. The algorithm CoffeeS4 uses in the code is:
1) Finds all the voices.
2) Searches the list one by one and retrieves the name of each voice.
3) Stores the display names of the voices. CoffeeS4 needs this indexed array to refresh the screen during updates.
4) Makes one extra step and displays the current voice in red. Like the display name, CoffeeS4 stores the token name in an indexed array for later use. In both cases, storing the names is not a requirement, rather a convenience. For screen updates, CoffeeS4 could also poll resources again but that seems like a waste of time. For both steps 3 and 4, this involves looping through the list, extracting the appropriate name and assigning it to the array.

All the work is done in the CoffeeS4 ManageEmployeesPaneProc() procedure and specifically the WM_INITPANE case. This initialization is logically placed here because the information must be present at the time the window is rendered.

In one sense, the hard part is done for you. Finding the available voices is accomplished in the one-line SpEnumTokens that passes back a list of all the voices and even provides the means to navigate that list. It also provides a method to find out how many items were found using ::GetCount.

static ULONG ulNumTokens;
hr = cpEnum->GetCount( &ulNumTokens );

Knowing the total number of items, CoffeeS4 may now allocate the two indexed arrays.

static CSpDynamicString*
ppcDesciptionString;
ppcDesciptionString = new CSpDynamicString [ulNumTokens];
//Check hr result

static WCHAR**  ppszTokenIds;
ppszTokenIds = new WCHAR* [ulNumTokens];
//Check hr result
ZeroMemory( ppszTokenIds, ulNumTokens*sizeof( WCHAR* ) );

CspDynamicString is a helper function for handling string arrays. It is a string class similar to other object oriented string classes. The subsequent allocation and release of each of its elements is automatic. You do not have to remember to do it manually. On the other hand, ppszTokenIds is simpler array of pointers for storing GUIDs of the token. CoffeeS4 manually allocates it because it is needed throughout the application. As a result, it must also be manually freed when no longer required. This is done in ManageEmployeesPaneCleanup(). The ZeroMemory() confirms all the values are initialized to zero. No valid GUID will be zero.

The next step of looping through the array is equally easy. CoffeeS4 navigates the list item by item to find the voice’s name. As mentioned earlier, IEnumSpObjectTokens has such a method named ::Next.

IspObjectToken     *pToken
= NULL
while (cpEnum->Next(1, &pToken, NULL) == S_OK)
{
//Code here
}

The list represented by cpEnum is traversed one item at a time as the first parameter indicates. The information is passed back in pToken, which is an interface to IspObjectToken. You might correctly guess that you will be looking at this interface in a moment. The last parameter is the number of items actually read. Since it is possible to read more than one at a time, it is also possible that not many items are left to read. If that were the case, it would return the number of items it could read. If this parameter simply cannot read any more items, it returns an error. In this case, CoffeeS4 stops looping through the “while” statement.

As CoffeeS4 steps through the list one at a time, it retrieves the names of the resources. There are two names for the particular token: the token name (also called the token ID) and the display name. The two could be the same but not necessarily. Also the display name can vary by language. It is the display name CoffeeS4 shows in the management window. Again, a helper function is available to simplify this task. SpGetDescription retrieves the display name and assumes the current language. In the case of the sample token, that name would be MSMary/409 value of “MS Mary.”

At the same time, the token ID is also retrieved. This is needed since CoffeeS4 also figures out the voice currently used and will need it shortly. No helper function is provided, as this is a straightforward call to get token ID. In the sample token, this would be “MSMary.” In both cases, the information is stored in indexed arrays for later use.

while (cpEnum->Next(1, &pToken, NULL) == S_OK)
{
// Get a string which describes the
token, in our case, the voice name
hr = SpGetDescription( pToken,
&ppcDesciptionString[ulIndex] );

// Get the token id, for a low
overhead way to retrieve the token later
// without holding on to the object itself
hr = pToken->GetId(
&ppszTokenIds[ulIndex] );

ulIndex++;

// Release the token itself
pToken->Release();
pToken = NULL;
}

When no longer needed, the token must be explicitly released. CoffeeS4 resets the point to NULL and is ready for the next loop. With all the information stored, the last task is figure out which is the currently active voice. This too, is a simple task. CoffeeS4 loops through the token ID array (ppszTokenIds) and compares the each location of the array with the system voice. If there is a match, it breaks out of the loop and stores the index position in ulCurToken.

// Get the token representing the current voice
HRESULT hr = g_cpVoice->GetVoice( &pToken );
if ( SUCCEEDED( hr ) )
{
// Get the current token ID, and
compare it against others to figure out
// which desciption string is the
one currently selected.
hr = pToken->GetId(
&pszCurTokenId );
if ( SUCCEEDED( hr ) )
{
ulIndex = 0;
while ( ulIndex <
ulNumTokens &&
0 != _wcsicmp(
pszCurTokenId, ppszTokenIds[ulIndex] ) )
{
ulIndex++;
}

// We found it, so set the
current index to that of the current token
if ( ulIndex <
ulNumTokens )
{
ulCurToken = ulIndex;
}

CoTaskMemFree( pszCurTokenId
);
}

pToken->Release();

The key to this is the ::GetVoice call. This passes back the current voice. Compare this value against the current voice. The current voice, stored in UlCurToken, is used in ManageEmployeesPanePaint() to highlight the active voice in a different color.

The rest of the codes for CoffeeS4 are relatively simple and are basically modifications of previously discussed techniques. Two new rules have been added: Please Manage the Employees and Hear them Speak. The former displays the list of voices once you are in the office and the latter speaks the current voice. Arrogant perhaps, but the employee states confidently “I will be the best employee you've ever had. Let me work.” In order to hear those words every employer loves, a new case must be added to ProcessRecoEvent(), that of VID_Manage.

The rest of the new code essentially handles events to the screen. OfficePaneProc(), ManageEmployeesPaneProc() and ManageEmployeesPaneCleanup() do the rest of the work.


MSDN Authoring Template

Chapter 7

CoffeeS5

Introduction

The concepts of resource management were introduced in the CoffeeS4. There, the sources of information that SAPI needed (such as voices and recognizers) were stored as tokens waiting to be used. When needed, the tokens were read and used as parameters to create objects. In turn, the objects were the actual implementation of those resources in working form. As objects, they have methods, validated data and communication paths to other parts of the speech system.

CoffeeS4 displayed the names of the available voices. It could even speak something in the current voice. However, as far as managing employees, there was little else it could do. CoffeeS5 addresses managing employees a little bit further. In addition to displaying voice names, it allows you to pick a different voice for each employee. You can also choose the type of voice you want displayed and spoken. You can choose feminine, masculine, or all voices.

The following topic will be discussed

· Dynamic Grammars

Dynamic Grammars

Up until now Coffee has only worked with static grammars. That is, for Command and Control issues, the word list has been both explicit and static. Explicit in that a list of exact words has been defined and therefore no words outside of this list are recognized. For instance, the grammar rule VID_DrinkType lists five types of drinks. Latte is included and may be ordered, but cola is not. Any order including cola would not be properly recognized. Static means this list is defined ahead of time and is not subject to change. Although the .xml file may be edited independently of the application, it may not be changed during execution.

As the name implies, a dynamic grammar is the opposite of a static one. First, words may be added and deleted during runtime and the list does not need to be predetermined. This allows much greater latitude for applications. CoffeeS5 demonstrates this by dynamically adding voices. Because the list of available voices may change not only from machine to machine, but also by user profile, there is no way to predetermine the available voices. The last example of CoffeeS4, started the process by polling the system for voice tokens-- the definitive list of available voices. The process expands now and a grammar of those voices will be constructed and used.

The logic to initializing the grammar is straightforward although there are a few subtleties. First, the rule is retrieved or created. CoffeeS5 creates a new rule but other applications could modify existing ones. Second, unwanted words are removed. Since CoffeeS5 wants to add all the words itself, for simplicity, the entire existing rule is erased. Again, another application could examine each word first before removing it. Regardless, SAPI needs to know explicitly to make the changes. As a last step, the new commands are added and the changes committed.

The magic happens in the ManageEmployeesPaneProc() WM_InitPane case. Specifically, because of the ::GetRule call.

SPSTATEHANDLE hDynamicRuleHandle;

hr = g_cpCmdGrammar->GetRule(NULL, DYN_TTSVOICERULE,
SPRAF_TopLevel | SPRAF_Active | SPRAF_Dynamic,
TRUE, &hDynamicRuleHandle);

Words may be added and deleted freely now. That is, up to the point of removing all words from a grammar. Conversely, and in this case, a completely empty grammar may be used and words added in; there is nothing wrong with an empty grammar. As an example, CoffeeS5 can actually add a cola drink to the order list or remove latte as the milk runs out. However, CoffeeS5 starts the rule from scratch. ::GetRule not only retrieves existing rules but can also create new ones. The fCreateIfNotExist member (TRUE in this case) permits this creation.

The first two parameters are often mutually exclusive. The first one, requests a rule by name while the second one searches by ID. In practice, knowing either the name or the ID is enough and you can leave the unneeded one NULL or zero respectively. Since you are creating a new rule, it will not have an ID to search for (hence the NULL value) and you must specify the name (DYN_TTSVOICERULE). The attributes may also be set here. Some attributes may (and will be) set later. This includes activating and deactivating grammars. Other attributes must be set at the time of creation. The rule is being made to be top-level, dynamic and active. Obviously, the values may be strung together with logical operators to get the exact nuance you want. And last, a handle (hDynamicRuleHandle) is passed back.

It is possible that there was already a rule existing for DYN_TTSVOICERULE. It might not have been properly destroyed before or you just forgot to remove it. ::ClearRule removes any doubt by removing the state information for the rule. In essence, all the words are removed and it effectively guarantees a clean start.

// Clear the rule first
hr = g_cpCmdGrammar->ClearRule( hDynamicRuleHandle );

// Commit the changes
hr = g_cpCmdGrammar->Commit(0);

After making any changes to the grammar, SAPI must be notified explicitly. ::Commit submits the changes to SAPI and thereafter the grammar is considered in the new state, with the new words. The parameter must be set to zero or it returns an error. In the example above, CoffeeS5 simply clears out the words and commits the changes. The grammar is now in a pristine state.

Words are added to the grammar using ::AddWordTransition.


MSDN Authoring Template

Chapter 8

CoffeeS6

Introduction

In CoffeeS6, the last of the tutorial chapters, no new programming code is introduced, per se. Rather, CoffeeS6 makes a variation on an existing theme. The past several Coffees demonstrated working with context free grammars, also called grammar rules. In short, they were predetermined lists of words that needed to be matched exactly. Even dynamic grammars, though more flexible, still had to match exact words once the word list was determined. For all the promise of speech recognition, the models so far have not allowed for dictation, or free-formed speech. CoffeeS6 now allows you to use unrestrained speech in your applications. It uses the simple case of renaming the coffee shop to anything you want.

The following topic will be discussed

· Embedded Dictation

· Grammar Modifiers

Embedded Dictation

As mentioned in the introduction, it would be limiting if you could not speak anything; that is, dictate to your application. Remember, the Coffee samples showcase a Command and Control model. As users, you are ordering the application to perform the following: get coffee, hire and fire employees, and make them talk to you. There is little room to get chatty with the other customers or employees. In fact, the Dictation model is better covered in both width and depth in the SDK sample applications DictPad and SimpleDict. By the same token, CoffeeS6 would be remiss in omitting this entirely. For that reason, the coffee shop manager now allows you to change the name of the shop.

For instance, go to office and give the order “manage store name.” A new screen displays and if you follow the instructions, you can say, “Rename the coffee shop to” and provide any name you want. CoffeeS6 will echo back the new name as “Welcome to the X coffee shop,” X, of course, being the moniker. The name will even display in all in the subsequent windows.

This is called embedded dictation because it combines the two forms. CoffeeS6 is still in Command and Control mode and using the same grammar rules as before. The advantages are that Coffee users can easily expand their drink offerings with only a minor change to the grammar file. Embedded dictation is not dynamic grammar. A rule is not created right before use and seeded with the words you want. In fact, embedded dictation would not handle this case appropriately. In renaming the coffee shop, you as the programmer, will have no idea what the user may choose for a name. The only real limitation is if the new name is in the dictation or not. Even this can be worked around.

Keep in mind, embedded dictation does not create new rules. CoffeeS6 has explicit support for renaming the store because a rule provides for that case. Instead, embedded dictation lessens restrictions on existing grammars, whether they are static or dynamic. You need not even change the code. Your application might want to handle the words elements differently, but that change is not really SAPI-related.

Grammar Modifiers

Programmatically, there is no new code to support this. CoffeeS6 simply modifies existing grammar rules. Technically these modifiers are not XML tags in the same way <P> and <l> are, for example. They appear inside the rule and are associates to other text or as the element itself. Each is explained below.

Wildcard: …

The ellipsis is a wildcard symbol indicating that any word or words may be accepted in this position. Informally it is also known as a garbage collector because it is used to accept words that the application may not explicitly care about. In this role, four things happen. First, the user may say anything including a series of words. As expected, the engine attempts to recognize the words, although not much is done with the words afterward. Coffee needs to know that a word was actually spoken and that the user was not just coughing or sneezing. Second, the rule will still be matched and activated. If the rule uses several parts (such as VID_EspressoDrinks), the wildcard words successfully match the rule requirements. Third, the parsed phrase returns only one element for the ellipsis regardless of the number of words actually spoken. For the most part, Coffee’s only interest is that legitimate words were spoken but not what the words actually were. And fourth, the element in the returned phrase itself will be the ellipsis rather than any useful word. Again, since Coffee is interested only that something was spoken, it makes sense not to return the actual word. If you are interested in the word, then the wildcard marker is not the right one for the rule; see Dictation below. In short, it really is a wildcard because any word may be used and still activate the rule.

The Coffee.xml (for CoffeeS6) snippet uses it in the following role in VID_EspressoDrinks:

<L>

<P>May I have</P>

<P>Can I have</P>

<P>Can I get</P>

<P>Please get me</P>

<P>Get me</P>

<P>I'd like</P>

<P>I would like</P>

<P>...</P>

</L>

In previous Coffee examples, the wording of the request was limited to one of the first seven requests. That is, the user had to begin by saying, “please get me,” or “can I get,” a drink. Because of the wildcard, the CoffeeS6 user may say almost anything and still get the drink. “Gimme a mocha,” will work (mimicking real life to boot), Even “gee-I-dunno-I-suppose-I’d-like a mocha” will also work provided the customer slurs the words together enough. Remember the rules are phrased-based and a sufficiently long pause between words never activates rules in the same way that the indecisive customer will never gets drinks by saying “may (pause) I (pause) have (pause) a (pause) mocha.”

So why even have the other phrases if the ellipsis is present? There are some subtleties to that answer. First, astute programmers may notice that they do not care what is spoken here. In the code, nothing is ever actually dependent on the fact that the customer said “please” or not. Yet, a rule consisting of only “coffee” may fire inappropriately. For example, a customer simply saying, “coffee is good,” might fire a too-simple rule. In this case, some sort of introductory clause is needed. Second, the additional words speed up the recognition process. The engine is much more likely to recognize “please” or “may” because it is described exactly in the rule. It also increases the confidence rating for the rule overall. Though both of the following phrases would activate the drink rule, “please get me a mocha,” returns a much higher confidence rating than would “aardvark a mocha.”

Dictation: *

The asterisk is a dictation indictor. Like the wildcard ellipsis, any word (or possibly words, see next entry) will validate the rule. In the same manner, the engine will attempt to recognize the word. The difference is that the actual word is returned back to the user in the phrase element. This is the key to renaming the coffee store.

The new rule VID_Rename is defined:

<RULE ID="VID_Rename" TOPLEVEL="ACTIVE">

<P PROPNAME="Named SAPI Coffee Shop to"> Rename the coffee shop to *+ </P>

</RULE>

Ignoring the plus sign for the moment, VID_Rename is activated on upon successfully matching “rename the coffee shop to,” followed by any word in the engine’s dictionary. The parsed phrase returns an element containing the actual word. With the actual word available, CoffeeS6 can use it to display the new name as it would with any stored variable.

Multiple dictation: +

The dictation entry in VID_Rename has a plus sign after the asterisk. This indicates that multiple words may be accepted in the rule. This allows for longer dictation phrases. Over-zealous customers may now rename the coffee store to virtually any name they want. By saying, “rename the coffee shop to Billy Bob Joe’s and Sally Jean Ann’s Coffee Emporium on the Highway,” they have successfully changed the name.

Confidence Increase: +
Confidence Decrease: –

One of these two signs placed in front of words respectively increases or decreases the required confidence for a successful recognition. Obviously increasing the required confidence means that the Speech Recognition engine will have to be much more certain that the word it hears really is the expected word. For example, if the user is responding to an important question such as “Reform hard disk?” you want to make an extra effort that what is recognized as “yes” really is “yes.” To make sure, the rule is noted as “+yes”.

Likewise, the minus sign decreases the required confidence for the word. This allows you to de-emphasize some words. Although the word is required for the rule, it is not important to verify that the user actually said it. It is a case of “close enough is good enough.”

In CoffeeS6 this is seen in the VID_ThingsToManage rule:

<RULE ID="VID_ThingsToManage" >

<L PROPID="VID_ThingsToManage">

<P VAL="VID_Employees">employees</P>

<P VAL="VID_ShopName">-shop +name</P>

<P VAL="VID_ShopName">-store +name</P>

</L>

</RULE>

The last two phrase elements allow the store name to be changed. However, Coffee is de-emphasizing the words “shop” and “store.” It is not important that users speak this word precisely; it just needs to be reasonably close. However, “name” has to be recognized clearly.

Code Modifications

In terms of code support for embedded dictation, there are very few changes. The case of handling a new rule is added, of course. The real work of CoffeeS6 is in case VID_Rename of ExecuteCommand(). Notice there is no extra effort required to implement the dictation itself.

if ( 5 <= pElements->Rule.ulCountOfElements )
{
if ( SUCCEEDED(
pPhrase->GetText( 5, pElements->Rule.ulCountOfElements - 5, FALSE,
&wszCoMemNameText, NULL )
) )
{
int ilen = wcslen(
pElements->pProperties->pszName );
ilen = (ilen + wcslen( wszCoMemNameText
) + 2) * sizeof(WCHAR);
wszCoMemValueText = (WCHAR *)
CoTaskMemAlloc( ilen );
if ( wszCoMemValueText )
{
wcscpy(
wszCoMemValueText, pElements->pProperties->pszName );
wcscat( wszCoMemValueText,
L" " );
wcscat(
wszCoMemValueText, wszCoMemNameText );
// Copy new shop name to
global shop name
_tcsncpy( g_szShopName,
W2T(wszCoMemNameText), NORMAL_LOADSTRING - 1 );
PostMessage( hWnd,
WM_RENAMEWINDOW, 0, (LPARAM) wszCoMemValueText );

CoTaskMemFree(
wszCoMemNameText );
}
}
}

This code filters through the phrase elements to retrieve the dictated text. The phrase elements are examined and retrieved. In practice, it is usually better not to assume that the sixth element is always the one you want but in this case CoffeeS6 does. Since the rename rule allows multiple words, CoffeeS6 starts at the sixth element (since it knows the first five elements have to be “rename the coffee shop to” and strings together the rest of the words for the new name.


Text-To-Speech Tutorial

This tutorial covers a very basic text to speech example. The console application is among the most simple demonstration allowing speech and is the Hello World equivalent for TTS.

The sample builds up from the most simple (but effectively non-functional) COM framework to to speaking a sentence. Steps are provided for each new function. The sample even goes one step beyond to demonstrate using XML tags to modify speech. The Complete Sample Application is at the bottom of the page.

Step 1: Initialize COM
Step 2: Setting Up Voices
Step 3: Speak!
Step 4: Modifying Speech

Step 1: Initialize COM

SAPI is a COM-based application and COM must be initialized before use and during the time SAPI is active. In most cases, this is for the life time of the host application. The following code (from Listing 1) initializes COM. Of course the application does not do anything beyond that but it does ensure COM is successfully started.

#include <windows.h>
#include <sapi.h>

int main(int argc, char* argv[])
{
    if (FAILED(::CoInitialize(NULL)))
        return FALSE;

    ::CoUninitialize();
    return TRUE;
}

Code Listing 1

Step 2: Setting Up Voices

Once COM is running, the next step is the create the voice. Voices are simply a COM object. SAPI uses intelligent defaults and during the initialization of the object, most values are assigned so that the object is ready to use directly afterward. The defaults are from the Speech properties in Control Panel and include such information as the voice to use (if more than one is available on your system), and the language (English, Japanese, etc.). While some defaults are obvious, others are not (speaking rate, pitch, etc.). Nevertheless, all defaults may be changed either through Speech properties or programmatically.

#include <windows.h>
#include <sapi.h>

int main(int argc, char* argv[])
{
    ISpVoice * pVoice = NULL;

    if (FAILED(::CoInitialize(NULL)))
        return FALSE;

    HRESULT hr = CoCreateInstance(CLSID_SpVoice, NULL, CLSCTX_ALL, IID_ISpVoice, (void **)&pVoice);
    if( SUCCEEDED( hr ) )
    {
        pVoice->Release();
        pVoice = NULL;
    }

    ::CoUninitialize();
    return TRUE;
}

Code Listing 2. Bold text represents new code for this example.

Step 3: Speak!

The actual speaking of the phrase is an equally simple task: one line calling Speak. When the instance of the voice is no longer needed, release the object. The setting of the pointer to NULL is not required but is useful.

#include <windows.h>
#include <sapi.h>

int main(int argc, char* argv[])
{
    ISpVoice * pVoice = NULL;

    if (FAILED(::CoInitialize(NULL)))
        return FALSE;

    HRESULT hr = CoCreateInstance(CLSID_SpVoice, NULL, CLSCTX_ALL, IID_ISpVoice, (void **)&pVoice);
    if( SUCCEEDED( hr ) )
    {
        hr = pVoice->Speak(L"Hello world", 0, NULL);
        pVoice->Release();
        pVoice = NULL;
    }

    ::CoUninitialize();
    return TRUE;
}

Code Listing 3. Bold text represents new code for this example.

Step 4: Modifying Speech

Voices may be modified by a variety of means. The most direct way is by applying XML commands directly to the stream. The commands are outlined in XML Schema. In this case, the pitch of the voice will be lowered by a relative rating of 10.

#include <windows.h>
#include <sapi.h>

int main(int argc, char* argv[])
{
    ISpVoice * pVoice = NULL;

    if (FAILED(::CoInitialize(NULL)))
        return FALSE;

    HRESULT hr = CoCreateInstance(CLSID_SpVoice, NULL, CLSCTX_ALL, IID_ISpVoice, (void **)&pVoice);
    if( SUCCEEDED( hr ) )
    {
        hr = pVoice->Speak(L"Hello world", 0, NULL);

        // Change pitch
        hr = pVoice->Speak(L "This is a <pitch middle = '-10'/> demonstration of SAPI 5", SPF_IS_XML, NULL );

        pVoice->Release();
        pVoice = NULL;
    }
    ::CoUninitialize();
    return TRUE;
}

Code Listing 4. Bold text represents new code for this example. This is the complete code sample.


TTS Events Explanation

Events are structures used to pass information from the TTS engine back to the application. SAPI fires events when the audio data corresponding to them is being output. This allows applications to react to audio output as it is happening - examples include animating a face appropriately as viseme events are received, or highlighting text as it is spoken (see the sample application, TTSApp, for an example of both of these).

Applications should call ISpEventSource::SetInterest to inform SAPI about the types of events they are interested in receiving. Applications may also call this through ISpVoice, because it inherits from ISpEventSource. Applications may then call ISpEventSource::GetEvents to retrieve fired events from SAPI.

Following is the set of event types generated by TTS engines (this is a subset of the SPEVENTENUM enumeration):

typedef enum SPEVENTENUM
{
    //--- TTS engine
    SPEI_START_INPUT_STREAM     = 1,
    SPEI_END_INPUT_STREAM       = 2,
    SPEI_VOICE_CHANGE           = 3,   // LPARAM_IS_TOKEN
    SPEI_TTS_BOOKMARK           = 4,   // LPARAM_IS_STRING
    SPEI_WORD_BOUNDARY          = 5,
    SPEI_PHONEME                = 6,
    SPEI_SENTENCE_BOUNDARY      = 7,
    SPEI_VISEME                 = 8,
    SPEI_TTS_AUDIO_LEVEL        = 9
} SPEVENTENUM;

The SPEVENT structure contains varying information depending on which of these event types it represents.

typedef [restricted] struct SPEVENT
{
    WORD         eEventId;
    WORD         elParamType;
    ULONG        ulStreamNum;
    ULONGLONG    ullAudioStreamOffset;
    WPARAM       wParam;
    LPARAM       lParam;
} SPEVENT;

You can analyze the various fields of the SPEVENT structure as to which event type they correspond to. For all event types, ulStreamNum corresponds to the stream number returned using the ISpVoice::Speak or ISpVoice::SpeakStream call.

The SPEI_START_INPUT_STREAM event indicates that the ouput object has begun receiving output for a specific stream number. The rest of the fields are not of any interest for this event type.

The SPEI_END_INPUT_STREAM event indicates that the output object has finished receiving output for a specific stream number. The rest of the fields are not of any interest for this event type.

The SPEI_VOICE_CHANGE event indicates that the voice responsible for speaking the input text (or stream) has changed because of a <Voice> XML tag. This event is also fired at the beginning of each Speak call. For more information on using object tokens, see the Object Tokens and Registry Settings white paper.

SPEVENT Field | Voice Change event
eEventId | SPEI_VOICE_CHANGE
elParamType | SPET_LPARAM_IS_TOKEN
wParam |
lParam | Object token of the new voice.

The SPEI_TTS_BOOKMARK event indicates that it has reached a bookmark. These can be inserted into the input text using the <Bookmark> XML tag.

SPEVENT Field | Bookmark event
eEventId | SPEI_TTS_BOOKMARK
elParamType | SPET_LPARAM_IS_STRING
wParam | Value of the bookmark string when converted to a long (_wtol(...) can be used).
lParam | Null-terminated copy of the bookmark string.

The SPEI_WORD_BOUNDARY event indicates that it has reached the beginning of a word.

SPEVENT Field | Word Boundary event
eEventId | SPEI_WORD_BOUNDARY
elParamType | SPET_LPARAM_IS_UNKNOWN
wParam | Character offset at the beginning of the word being synthesized.
lParam | Character length of the word in the current input stream being synthesized.

The SPEI_SENTENCE_BOUNDARY event indicates that it has reached the beginning of a sentence.

SPEVENT Field | Sentence Boundary event
eEventId | SPEI_SENTENCE_BOUNDARY
elParamType | SPET_LPARAM_IS_UNKNOWN
wParam | Character offset at the beginning of the sentence being synthesized.
lParam | Character length of the sentence in the current input stream being synthesized.

The SPEI_PHONEME event indicates that it has reached the phoneme.

SPEVENT Field | Phoneme event
eEventId | SPEI_PHONEME
elParamType | SPET_LPARAM_IS_UNKNOWN
wParam | The high word is the duration, in milliseconds, of the current phoneme. The low word is the PhoneID of the next phoneme.
lParam | The low word is the PhoneID of the current phoneme. The high word is the SPVFEATURE value associated with the current phoneme.

The SAPI 5 American English phoneme set can be found here. The SAPI 5 Chinese phoneme set can be found here. The SAPI 5 Japanese phoneme set can be found here.

SPVFEATURE contains two flags: SPVFEATURE_STRESSED and SPVFEATURE_EMPHASIS SPVFEATURE_STRESSED means that the phoneme is stressed relative to the other phonemes of a word (stress is usually associated with the vowel of a stressed syllable). SPVFEATURE_EMPHASIS means that the phoneme is part of an emphasized word. That is, stress is a syllabic phenomenon within a word, and emphasis is a word-level phenomenon within a sentence.

The SPEI_VISEME event indicates that it has reached the viseme.

SPEVENT Field | Viseme event
eEventId | SPEI_VISEME
elParamType | SPET_LPARAM_IS_UNKNOWN
wParam | The high word is the duration, in milliseconds, of the current viseme. The low word is the code for the next viseme.
lParam | The low word is the code of the current viseme. The high word is the SPVFEATURE value associated with the current viseme (and phoneme).

See SPVISEMES for a listing of the SAPI 5 viseme set.

The SPEI_TTS_AUDIO_LEVEL event indicates the audio has reached the level of the synthesis at any given point.

SPEVENT Field | Audio Level event
eEventId | SPEI_TTS_AUDIO_LEVEL
elParamType | SPET_LPARAM_IS_UNDEFINED
wParam | TTS audio level (ULONG).
lParam | NULL

For an example of how to use TTS events in an application, see the Text-To-Speech Tutorial.


Microsoft Speech SDK (SAPI 5.0) White Papers

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

White Papers

The following items are covered in this section:

  • SR Properties White Paper
  • TTS Engine Vendor Porting Guide White Paper
  • SR Engine Vendor Porting Guide White Paper
  • Object Tokens and Registry Settings White Paper
  • Compliance Tests White Paper
  • Setup White Paper
  • XML Schema : Grammar
  • XML Schema : SAPI
  • Text Normalization


Setup Specification for SAPI 5

SAPI 5.0 SR Properties White Paper

Introduction

This document describes the ISpProperties elements for SAPI 5 compliant SR engines. This spec will serve to define these attributes only for SR engines. Application developers hoping to build a SAPI 5 compliant engine should reference this document. For more information, developers should refer to the SAPI SDK help documents.

ISpProperties

ISpProperties is an interface that enables the SR and TTS engines to get or set various attributes for an object. The attributes are passed to the engine via the ISpProperties interface. ISpProperties are identified by a unique LONG value. SAPI defines certain attributes known as system attributes. The range of these attributes is from 0x0001 to 0xffff. Vendor ISpProperties attributes are defined by a unique high word value (two ANSI Characters that identify the engine vendor).

Attributes may be LONGs, strings, or memory addresses.

SR Properties

The following table lists the SR properties that are set by the application and passed to the SR engine via SAPI. These attributes are not required for SAPI compliance. However, the ranges accompanied by the attributes are required values and the exact interpretation of the values is left to the SR engine. The different implementation is defined by each property. The SAPI ranges and defaults for each property are also shown.

NOTE: The attributes are associated with a user profile and written in the registry by SAPI. SAPI detects the correct settings. The application should not write attribute changes to the registry.

WCHAR Value

dwAttrib Value

Meaning

Range

SPPROP_RESOURCE_USAGE

ResourceUsage

The ResourceUsage specifies the engine CPU consumption. As the resource usage increases, so does the required CPU power.

0 – 100

default = 50

SPPROP_HIGH_CONFIDENCE_THRESHOLD

SPPROP_NORMAL_CONFIDENCE_THRESHOLD

SPPROP_LOW_CONFIDENCE_THRESHOLD

HighConfidenceThreshold

NormalConfidenceThreshold

LowConfidenceThreshold

The threshold values are used to divide a confidence scale into four portions: rejected, low, medium, and high. The location of the low confidence, normal confidence, and high confidence markers control how a word’s confidence is labeled. The HighConfidenceThreshold (HCT) separates the high and medium confidence range. The NormalConfidenceThreshold (NCT) separates the medium and the low confidence thresholds. The LowConfidenceThreshold (LCT) separates the low and rejected confidence range.

If the all three confidences are equal to 0, then all words will have high confidence. If all three confidences are equal to 100, then all words will have low confidence.

0 – 100

default

LCT = 20

NCT = 50

HCT = 80

SPPROP_RESPONSE_SPEED

ResponseSpeed

This indicates the amount of silence the engine looks for before completing a recognition. This attribute is used when the recognition is not ambiguous. For example, in the case of a context free grammar (CFG) which has two sentences: 1) new game please and 2) new game, a non-ambiguous recognition would be “new game please.”

0 – 10,000 ms

default = 150 ms

SPPROP_COMPLEX_RESPONSE_SPEED

ComplexResponseSpeed

This indicates the amount of silence that the engine will look for before completing a recognition. This attribute is used when the recognition is ambiguous. For example, in the case of a CFG which has two sentences: 1) new game please and 2) new game, an ambiguous recognition would be “new game.” This property’s value must be greater than the ResponseSpeed value.

ResponseSpeed – 10,000

Default = 500 ms

SPPROP_ADAPTATION_ON

AdaptationOn

Indicates whether the recognition engine should adapt the acoustic model.

1 or 0

default = 1


TTS Engine Vendor Porting Guide White Paper

Table of Contents

Overview of SAPI 5.0 Architecture

SAPI Objects and Interfaces

Creating and Initializing the Engine - ISpObjectWithToken

Receiving Calls from SAPI - ISpTTSEngine

GetOutputFormat

Speak

Fragment List Example

Writing Data Back to SAPI - ISpTTSEngineSite

Getting Real-Time Action Requests

Volume

Rate

Skip

Queueing Events

Bookmarks

Word Boundaries

Sentence Boundaries

Phonemes

Visemes

Queueing Audio Data

Creating an Engine Properties UI - ISpTokenUI

Using SAPI Lexicons

Appendix A - SAPI 5 Phonemes

Appendix B - SAPI 5 Visemes

Overview of SAPI 5.0 Architecture

The Microsoft Speech API (SAPI) is a layer of software which sits between applications and speech engines, allowing them to communicate in a standardized way. One of its main goals is enabling application developers to use speech technology in a simple and straightforward way. Another goal is solving some of the more basic complications of developing speech engines, such as audio device manipulation and threading issues, thus allowing engine developers to focus on speech.

From an engine vendor’s point of view, there are a number of technical advantages to using SAPI 5 over SAPI 4:

§ The SAPI 5 DDI has been greatly simplified.

§ SAPI 5 can handle all audio format conversion for the TTS engine.

§ SAPI 5 parses SAPI 5 XML for the TTS engine. Engine proprietary tags are passed to the engine untouched, allowing the engine to interpret them.

§ SAPI 5 performs parameter validation for the engine.

§ SAPI 5 has lexicon management features.

SAPI Objects and Interfaces

There are two main objects of interest to a TTS Engine developer: the SpVoice object (SAPI) and the TTS Engine object (refer to figure 2). The third object in the figure is a UI component which an engine may or may not implement.

The SpVoice object implements two interfaces which we will be concerned with – ISpVoice, which is the interface which the application uses to access TTS functionality, and ISpTTSEngineSite, which the engine uses to write audio data and queue events. The TTS Engine must implement two interfaces as well – ISpTTSEngine, which is the interface through which SAPI will call the engine, and ISpObjectWithToken, which is the interface through which SAPI will create and initialize the engine. The UI object, if it exists, must implement ISpTokenUI, through which it will be accessed by the SAPI control panel (or, potentially, other applications).

For the most part this document is not concerned with ISpVoice, and so it won’t be covered in any detail. Each of the other interfaces, however, will be discussed in depth.

Creating and Initializing the Engine - ISpObjectWithToken

One important thing to realize about the SAPI 5 architecture is that while SAPI knows about TTS Engines, applications only know about TTS voices. The difference between these two is fairly obvious - one engine implementation can potentially support any number of different voices, with the only differences being data files, parameters, etc. What this means at the engine level is that an engine will be created by one of its voices, in a certain sense.

SAPI 5 uses tokens to represent resources available on a machine (see the Object Tokens and Registry Settings White Paper for more details), including TTS voices. These tokens contain the CLSID of the objects they represent, as well as various attributes of those objects. When an application wishes to use a TTS voice, SAPI will get that voice's token from the registry. Through the voice token, an engine will be cocreated via its CLSID. The SpVoice object then queries the engine for the ISpObjectWithToken interface, through which it calls SetObjectToken .

Here is an example of what a voice token might look like in the registry (voices are located under HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\Voices\Tokens):

The SetObjectToken call gives the TTS Engine a pointer to the token (and thus the voice) from which it was created, which gives the Engine a chance to initialize itself based on information stored in the token. In the example token above, the VoiceData and VoiceDef keys in the token allow the TTS engine to load the appropriate voice data, once it has a pointer to the token. Similarly, the Lex and LTS subkeys allow the TTS engine to load the appropriate lexicon and letter-to-sound rules.

Again, for more details on registering a TTS engine, see the Object Tokens and Registry Settings White Paper.

Receiving Calls from SAPI - ISpTTSEngine

Once an engine has been created SAPI will begin calling the engine via ISpTTSEngine. ISpTTSEngine has only two methods – GetOutputFormat and Speak.

GetOutputFormat is used to query the engine about a specific output format – the engine should examine the desired output format and return to the SpVoice object the closest format which it supports. This function may potentially be called many times during the life of the engine.

HRESULT GetOutputFormat(

[in] const GUID * pTargetFmtId,

[in] const WAVEFORMATEX * pTargetWaveFormatEx,

[out] GUID * pOutputFormatId,

[out] WAVEFORMATEX ** ppCoMemOutputWaveFormatEx

);

In the normal case, pTargetFmtId will be SPDFID_WaveFormatEx, and pTargetWaveFormatEx will be a pointer to a WAVEFORMATEX structure describing the desired output format. In this case, the engine should set pOutputFormatId to SPDFID_WaveFormatEx, allocate space (via ppCoMemOutputWaveFormatEx) for a WAVEFORMATEX structure, and set it to the closest format to pTargetWaveFormatEx it supports.

If pTargetFmtId is NULL, the engine should simply return to SAPI its default format.

NOTE: If pTargetFmtId is SPDFID_Text, engines can do whatever they please. Essentially, this format type if provided for debugging purposes – it is not required that any engine support this for SAPI 5.0 compliance, nor is it required that engines to do anything specific with this format if they do support it.

See the Sample TTS Engine's GetOutputFormat implementation for more details.

Speak is the main function of the interface – it passes the engine the text to be rendered, an output format to render it in, and an output site to which the engine should write audio data and events. A Speak call should return when either all of the input text has been rendered, or the engine has been told to abort the call by the SpVoice object. Let’s look at the parameters in more detail.

HRESULT Speak(

[in]DWORD dwSpeakFlags,

[in]REFGUID rguidFormatId,

[in]const WAVEFORMATEX * pWaveFormatEx,

[in]const SPVTEXTFRAG* pTextFragList,

[in]ISpTTSEngineSite* pOutputSite,

);

The first parameter of the Speak call, dwSpeakFlags, is a DWORD which will have one of two values – 0, or SPF_NLP_SPEAK_PUNC (all other flags in the SPEAKFLAGS enumeration are masked out, since they are handled by SAPI). If the value is SPF_NLP_SPEAK_PUNC, the engine should speak all punctuation (e.g. “This is a sentence.” should be expanded to “This is a sentence period”).

The second and third parameters of the Speak call will specify the output format which the engine should use for rendering the text passed in for this call. This format is guaranteed to be one which the engine told SAPI it supports via a previous GetOutputFormat call. Again, if this rguidFormatId is SPDFID_Text, it is not required that engines support this format, nor is it required that engines do anything specific with this format if it is supported.

The fourth parameter is the text to be rendered in the form of a linked list of SPVTEXTFRAGs. Let’s look at this structure in more detail.

typedef struct SPVTEXTFRAG

{

struct SPVTEXTFRAG *pNext;

SPVSTATE State;

LPCWSTR pTextStart;

ULONG ulTextLen;

ULONG ulTextSrcOffset;

} SPVTEXTFRAG;

pTextStart is a pointer to the beginning of the text associated with the fragment. ulTextLen is the length of this text, in WCHARs. ulTextSrcOffset is the offset of the first character of the text associated with the fragment. Finally, State is the SAPI 5.0 XML state associated with this fragment (see the XML Schema : SAPI white paper for more details on SAPI 5 XML markup).

typedef [restricted] struct SPVSTATE

{

SPVACTIONS eAction;

LANGID LangID;

WORD wReserved;

long EmphAdj;

long RateAdj;

ULONG Volume;

SPVPITCH PitchAdj;

ULONG SilenceMSecs;

SPPHONEID *pPhoneIds;

SPPARTOFSPEECH ePartOfSpeech;

SPVCONTEXT Context;

} SPVSTATE;

eActions is an enumerated value which tells the engine what it should do with this fragment.

typedef enum SPVACTIONS

{

SPVA_Speak = 0,

SPVA_Silence,

SPVA_Pronounce,

SPVA_Bookmark,

SPVA_SpellOut,

SPVA_Section,

SPVA_ParseUnknownTag

} SPVACTIONS;

SPVA_Speak (the default value) means that the engine should process the text associated with the fragment and render it in the proper output format. SPVA_Silence means that SAPI was passed a <Silence> SAPI 5.0 XML tag, and that the engine should write SilenceMSecs (see struct SPVSTATE) milliseconds of silence. SPVA_Pronounce means that SAPI was passed a <Pron> SAPI 5.0 XML tag, and that the engine should use pPhoneIds (see struct SPVSTATE) as the pronunciation of the associated text, or just insert the pronunciation if there is no associated text. SPVA_Bookmark means that SAPI was passed a <Bookmark> SAPI 5.0 XML tag, and that the engine should write a Bookmark event (see below for information on writing events). SPVA_SpellOut means that the engine should spell out the associated text letter by letter, including punctuation and miscellaneous characters (and render this expanded version of the text in the proper output format). SPVA_Section is currently unused. SPVA_ParseUnknownTag means that a non-SAPI 5.0 XML tag was passed to SAPI – if the engine supports additional tags, it should attempt to parse this tag. Otherwise, it should just ignore it.

LANGID will be zero, unless a language was specified to SAPI via a <Lang> SAPI 5.0 XML tag.

EmphAdj will be zero, unless SAPI was passed an <Emph> SAPI 5.0 XML tag.

RateAdj will be 0, unless SAPI was passed a <Rate> SAPI 5.0 XML tag. This gives the absolute rate which the engine should use to render the text associated with this fragment. NOTE: the engine should combine these values with values obtained through ISpTTSEngineSite::GetRate calls to arrive at a final value.

Volume will be 100, unless SAPI was passed a <Volume> SAPI 5.0 XML tag. This gives the absolute volume which the engine should use to render the text associated with this fragment. NOTE: the engine should combine these values with values obtained through ISpTTSEngineSite::GetVolume calls to arrive at a final value.

PitchAdj will have a MiddleAdj of zero and a RangeAdj of zero, unless SAPI was passed a <Pitch> SAPI 5.0 XML tag. This gives the absolute pitch middle and range which the engine should use to render the text associated with this fragment (the pitch middle is used to raise or lower the overall pitch of the voice, the pitch range is used to expand or contract the pitch range of the voice, making it more or less monotone).

typedef struct SPVPITCH

{

long MiddleAdj;

long RangeAdj;

} SPVPITCH;

ePartOfSpeech will be SPPS_Unknown (see SPPARTOFSPEECH) unless SAPI was passed a <PartOfSp> SAPI 5.0 XML tag. This part of speech should be used for the text associated with this fragment (e.g. to disambiguate a word with multiple pronunciations).

Finally, the pointers within Context will be NULL unless SAPI was passed a <Context> SAPI 5.0 XML tag.

typedef [restricted] struct SPVCONTEXT

{

LPCWSTR pCategory;

LPCWSTR pBefore;

LPCWSTR pAfter;

} SPVCONTEXT;

This field can be used to disambiguate items in the text associated with this fragment (e.g. ambiguous date formats).

Let’s look at an example of a fragment list. Imagine this text is passed to SAPI:

"This is a
<PITCH MIDDLE = '6'> sample piece of <PARTOFSP PART = 'Noun'> text
</PARTOFSP> which will <BOOKMARK MARK = '1'/> demonstrate <VOLUME
LEVEL = '30'> what a <VOLUME LEVEL = '90'> fragment </VOLUME>
list </VOLUME> looks like </PITCH> conceptually."

This will be the resulting linked list of SPVTEXTFRAGs passed to the TTS Engine:

SPVTEXTFRAGs

Element 1

Element 2

pNext

Element 2

Element 3

State

eAction

SPVA_Speak

SPVA_Speak

LangId

0

0

EmphAdj

0

0

RateAdj

0

0

Volume

100

100

PitchAdj

MiddleAdj

0

6

RangeAdj

0

0

SilenceMSecs

0

0

pPhoneIds

NULL

NULL

ePartOfSpeech

SPPS_Unknown

SPPS_Unknown

Context

pCategory

NULL

NULL

pBefore

NULL

NULL

pAfter

NULL

NULL

pTextStart

“This is a <PITCH …“

“sample piece of <PART…”

ulTextLen

10

16

ulTextSrcOffset

0

31

SPVTEXTFRAGs

Element 3

Element 4

Element 5

pNext

Element 4

Element 5

Element 6

State

eAction

SPVA_Speak

SPVA_Speak

SPVA_Bookmark

LangId

0

0

0

EmphAdj

0

0

0

RateAdj

0

0

0

Volume

100

100

100

PitchAdj

MiddleAdj

6

6

6

RangeAdj

0

0

0

SilenceMSecs

0

0

0

pPhoneIds

NULL

NULL

NULL

ePartOfSpeech

SPPS_Noun

SPPS_Unknown

SPPS_Unknown

Context

pCategory

NULL

NULL

NULL

pBefore

NULL

NULL

NULL

pAfter

NULL

NULL

NULL

pTextStart

“text </PART…”

“which will <B…”

“1’/> demonstrate…”

ulTextLen

5

11

1

ulTextSrcOffset

72

89

100

SPVTEXTFRAGs

Element 6

Element 7

Element 8

pNext

Element 7

Element 8

Element 9

State

eAction

SPVA_Speak

SPVA_Speak

SPVA_Speak

LangId

0

0

0

EmphAdj

0

0

0

RateAdj

0

0

0

Volume

100

30

90

PitchAdj

MiddleAdj

6

6

6

RangeAdj

0

0

0

SilenceMSecs

0

0

0

pPhoneIds

NULL

NULL

NULL

ePartOfSpeech

SPPS_Unknown

SPPS_Unknown

SPPS_Unknown

Context

pCategory

NULL

NULL

NULL

pBefore

NULL

NULL

NULL

pAfter

NULL

NULL

NULL

pTextStart

“demonstrate <V…”

“what a <VOL…”

“fragment </VOL…”

ulTextLen

12

7

9

ulTextSrcOffset

123

157

186

SPVTEXTFRAGs

Element 9

Element 10

Element 11

pNext

Element 10

Element 11

Element 12

State

eAction

SPVA_Speak

SPVA_Speak

SPVA_Speak

LangId

0

0

0

EmphAdj

0

0

0

RateAdj

0

0

0

Volume

30

100

100

PitchAdj

MiddleAdj

6

6

0

RangeAdj

0

0

0

SilenceMSecs

0

0

0

pPhoneIds

NULL

NULL

NULL

ePartOfSpeech

SPPS_Unknown

SPPS_Unknown

SPPS_Unknown

Context

pCategory

NULL

NULL

NULL

pBefore

NULL

NULL

NULL

pAfter

NULL

NULL

NULL

pTextStart

“list </VOL…”

“looks like </PIT…”

“conceptually.”

ulTextLen

5

11

14

ulTextSrcOffset

205

220

240

The last parameter of the Speak call is an ISpTTSEngineSite pointer – pOutputSite. This pointer should be stored by the engine, as it will be used to write audio data and events back to the SpVoice object, as well as to poll the SpVoice object for real-time action requests.

Writing Data Back to SAPI - ISpTTSEngineSite

Getting Real-Time Action Requests

Within a Speak call, an Engine should call ISpTTSEngineSite::GetActions as often as possible to ensure near real-time processing of SAPI actions. This is an inexpensive call – it simply returns a DWORD which will contain one or more values from the SPVESACTIONS enumeration.

DWORD GetActions( void );

typedef enum SPVESACTIONS

{

SPVES_CONTINUE = 0,

SPVES_ABORT = ( 1L << 0 ),

SPVES_SKIP = ( 1L << 1 ),

SPVES_RATE = ( 1L << 2 ),

SPVES_VOLUME = ( 1L << 3 )

} SPVESACTIONS;

SPVES_CONTINUE is the default case (no actions) – it means to continue processing normally. SPVES_ABORT means that the engine should abort the Speak call and return immediately. The other three cases require a bit more explanation.

SPVES_VOLUME – the engine should call ISpTTSEngineSite::GetVolume, which will return a new volume level. The engine should adjust its volume level accordingly. Note that when no XML volume has been specified, the level returned by GetVolume should be exactly the level used by the engine, but if the volume is already affected by an XML tag, the final volume should be a combination of the two.

HRESULT GetVolume(

[out] USHORT *pusVolume

);

SPVES_RATE – the engine should call ISpTTSEngineSite::GetRate, which will return a new rate level. The engine should adjust its rate level accordingly. Note that, similarly to volume, XML rate levels and GetRate rate levels should be combined to produce the final rate.

HRESULT GetRate(

[out] long *pRateAdjust

);

SPVES_SKIP – the engine should call ISpTTSEngineSite::GetSkipInfo, which will return a type of unit to skip (currently only sentences are supported) and the number of such units to skip. This number can be positive (skip forward in the text), negative (skip backward in the text), or zero (skip to the beginning of the current item). The engine should stop writing data to SAPI, skip the appropriate number of units (or as many as it can) and then call ISpTTSEngineSite::CompleteSkip to tell SAPI how many units it was able to successfully skip. If it was able to successfully skip the entire number returned by GetSkipInfo, the engine should then continue rendering text at the appropriate point. Otherwise, it should abort the current Speak call and return immediately.

HRESULT GetSkipInfo(

[out] SPVSKIPTYPE *peType,

[out] long *plNumItems

);

HRESULT CompleteSkip(

[in] long ulNumSkipped

);

As an example, imagine an engine was passed this text:

“This is sentence one. This is sentence two. This is sentence three.”

Now suppose that the engine was currently rendering the second sentence when it discovered, via GetActions and GetSkipInfo, that it was being asked to skip +1 sentence. The engine should stop rendering the second sentence, skip forward to the third sentence, call CompleteSkip with a parameter of +1, and begin rendering the third sentence. Now imagine that the engine was asked to skip –2 sentences. The engine should again stop rendering the second sentence, and then skip backwards until it discovers that it cannot skip the appropriate number. It would then call CompleteSkip with a parameter of –1 and abort its Speak call.

Queueing Events

Events are structures which are used to pass information from the engine back to the application. The engine is responsible for generating certain types of events, and then handing them to SAPI through the function ISpTTSEngineSite::AddEvents. SAPI will then take care of firing the events at the appropriate times.

HRESULT AddEvents(

[in] const SPEVENT* pEventArray,

[in] ULONG ulCount

);

Engines should call the function ISpTTSEngineSite::GetEventInterest, which will tell them which events the application (and/or SAPI) is interested in receiving.

HRESULT GetEventInterest(

[out] ULONGLONG * pullEventInterest

);

This function will return (via pullEventInterest) a ULONGLONG which will contain one or more values from the TTS subset of the SPEVENTENUM enumeration:

§ SPEI_TTS_BOOKMARK

§ SPEI_WORD_BOUNDARY

§ SPEI_SENTENCE_BOUNDARY

§ SPEI_PHONEME

§ SPEI_VISEME

The engine must then generate the appropriate types of events. Here is the structure of an SPEVENT:

typedef [restricted] struct SPEVENT

{

WORD eEventId;

WORD elParamType;

ULONG ulStreamNum;

ULONGLONG ullAudioStreamOffset;

WPARAM wParam;

LPARAM lParam;

} SPEVENT;

Note that SAPI is responsible for setting ulStreamNum – the engine need not worry about this field. ullAudioStreamOffset should in each case be the byte (not sample) offset in the audio stream at which the event should be fired. NOTE: this offset should correspond to a sample boundary.

Let’s go through what the various fields of the SPEVENT structure correspond to for each event type.

The SPEI_TTS_BOOKMARK event indicates that the TTS engine has reached a bookmark. Here is the format for the fields of the Bookmark event:

SPEVENT Field | Bookmark event
eEventId | SPEI_TTS_BOOKMARK
elParamType | SPET_LPARAM_IS_STRING
wParam | Value of the bookmark string when converted to a long (_wtol(...) can be used)
lParam | Null terminated copy of the bookmark string

For example, if an engine was passed a bookmark corresponding to this XML marked up text:

“<BOOKMARK MARK=”this is a bookmark”/>”

The engine would need to generate an event whose lParam was “this is a bookmark”. If the engine was passed a bookmark corresponding to this XML marked up text:

“<BOOKMARK MARK='1'/>”

The engine would need to generate an event whose wParam was equal to the integer, one.

The SPEI_WORD_BOUNDARY event indicates that the TTS engine has started synthesizing a word. Here is the format for the fields of the word boundary event:

SPEVENT Field | Word Boundary event
eEventId | SPEI_WORD_BOUNDARY
elParamType | SPET_LPARAM_IS_UNKNOWN
wParam | Character offset of the beginning of the word being synthesized.
lParam | Character length of the word in the current input stream being synthesized

The SPEI_SENTENCE_BOUNDARY event indicates that the TTS engine has started synthesizing a sentence. Here is the format for the fields of the sentence boundary event:

SPEVENT Field | Sentence Boundary event
eEventId | SPEI_SENTENCE_BOUNDARY
elParamType | SPET_LPARAM_IS_UNKNOWN
wParam | Character offset of the beginning of the sentence being synthesized.
lParam | Character length of the sentence in the current input stream being synthesized

The SPEI_PHONEME event indicates that the TTS engine has synthesized a phoneme. Here is the format for the fields of the phoneme event:

SPEVENT Field | Phoneme event
eEventId | SPEI_PHONEME
elParamType | SPET_LPARAM_IS_UNKNOWN
wParam | The high word is the duration in milliseconds of the current phoneme. The low word is the PhoneID of the next phoneme.
lParam | The low word is the PhoneID of the current phoneme. The high word is the SPVFEATURE value associated with the current phoneme.

See Appendix A for the SAPI 5.0 phoneme set.

SPVFEATURE contains two flags – SPVFEATURE_STRESSED, which means that the phoneme is stressed relative to the other phonemes of a word (stress is usually associated with the vowel of a stressed syllable), while SPVFEATURE_EMPHASIS means that the phoneme is part of an emphasized word. That is, stress is a syllabic phenomenon within a word, while emphasis is a word-level phenomenon within a sentence.

The SPEI_VISEME event indicates that the TTS engine has synthesized a viseme. Here is the format for the fields of the viseme event:

SPEVENT Field | Viseme event
eEventId | SPEI_VISEME
elParamType | SPET_LPARAM_IS_UNKNOWN
wParam | The high word is the duration in milliseconds of the current viseme. The low word is the code for the next viseme
lParam | The low word is the code of the current viseme. The high word is the SPVFEATURE value associated with the current viseme (and phoneme).

The SAPI visemes are based off the Disney 13 Visemes and are described in Appendix B for the SAPI American English phoneme set.

Queueing Audio Data

After an engine has queued events, it should write audio data to the output site in the appropriate format. NOTE: the order of these two events is important – events should not be queued after their associated audio data has already been written or they cannot be fired at the proper times. The function ISpTTSEngineSite::Write is used to write audio data.

HRESULT Write(

const void* pBuff,

ULONG cb,

ULONG *pcbWritten

);

This function is straightforward – pBuff points to a buffer of audio data to be written to the output site, cb is the number of bytes (not samples) to be written, and pcbWritten will return the number of bytes actually written (which should be the same as cb, assuming nothing has gone wrong). NOTE: only complete samples should be written. If the Write function returns SP_AUDIO_STOPPED the audio device has been stopped and the Speak call should abort immediately.

It should be noted that if an engine (from the application's perspective, a voice) is paused (via ISpVoice::Pause), SAPI will block an ISpTTSEngineSite::Write call until the engine is to resume. The same thing will happen if an alert priority voice interrupts a normal priority voice (see ISpVoice::SetPriority for more information on voice priorities).

Creating an Engine Properties UI - ISpTokenUI

TTS Engines may wish to supply various UI components - one example is an Engine Properties component which users can access through the SAPI 5.0 control panel. SAPI provides mechanisms for engines to describe what UI components they have, and for applications to request the display of these components.

The UI components that an engine supports should be contained within the engine voice's objet tokens (refer to the Object Tokens and Registry Settings White Paper for more discussion of tokens) within a UI subkey. Within this key should be subkeys for each UI component the engine implements. For example, an engine properties component would be in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Voices\Tokens\{Voice Name}\UI\EngineProperties. The EngineProperties key would then contain the CLSID of the class to be created when this UI component is displayed. The engine setup should install and register this class, and the class must implement the interface ISpTokenUI.

An application can then see if a particular UI component is supported by an engine by calling ISpTokenUI::IsSupportedUI on the engine's object token.

[local] HRESULT IsUISupported(

[in] const WCHAR *pszTypeOfUI,

[in] void *pvExtraData,

[in] ULONG cbExtraData,

[in] IUnknown *punkObject,

[out] BOOL *pfSupported

);

Here is an example implementation of IsUISupported:

STDMETHODIMP EnginePropertiesUI::IsUISupported(

const WCHAR* pszTypeOfUI,

void * /*pvExtraData*/,

ULONG /*cbExtraData*/,

IUnknown * /*punkObject*/,

BOOL *pfSupported )

{

*pfSupported = false;

if ( wcscmp( pszTypeOfUI, SPDUI_EngineProperties ) == 0 )

{

*pfSupported = true;

}

return S_OK;

}

SPDUI_EngineProperties is just the string, "EngineProperties" - this is the string which the SAPI 5.0 control panel uses to query engines for UI components to be disaplayed when the user clicks the "Settings" button. If this function call returns true (via pfSupported), the application can then call ISpTokenUI::DisplayUI to disaply the UI component.

[local] HRESULT DisplayUI(
[in] HWND hwndParent,
[in] const WCHAR * pszTitle,
[in] const WCHAR * pszTypeOfUI,
[in] void * pvExtraData,
[in] ULONG cbExtraData,
[in] ISpObjectToken * pToken,
[in] IUnknown * punkObject

);

Here is an example implementation of DisaplyUI:

STDMETHODIMP SpTtsEngUI::DisplayUI(

HWND hwndParent,
const WCHAR * pszTitle,
const WCHAR * pszTypeOfUI,
void * /* pvExtraData */,
ULONG /* cbExtraData */,
ISpObjectToken * pToken,
IUnknown * /* punkObject */)
{
HRESULT hr = S_OK;

if ( SUCCEEDED( hr ) )
{
if ( wcscmp( pszTypeOfUI, SPDUI_EngineProperties ) == 0)
{
EnginePropertiesDialog dlg;

dlg.hInstance = g_hInstance;

dlg.hwndParent = hwndParent;

hr = dlg.Run();
}
}

return hr;
}

Using SAPI Lexicons

SAPI provides lexicons so that users and applications may specify pronunciation and part of speech information for words important to them. As such, all SAPI compliant TTS engines should use these lexicons to guarantee uniformity of pronunciation and part of speech information.

There are two types of lexicons in SAPI:

§ User Lexicons: Each user who logs onto a machine will have a User Lexicon. These are initially empty, but can have words added to them either programmatically, or via an engine's add/remove words UI component (for example, the sample application Dictation Pad provides an Add/Remove Words dialog).

§ Application Lexicons: Applications can create and ship their own lexicons of specialized words - these are read only.

Each of these lexicon types implements the ISpLexicon interface and can be created directly, but SAPI provides a Container Lexicon class which combines the user lexicon and all application lexicons into a single entity, making manipulating the lexicon information much simpler. Here is an example of how to create a Container Lexicon (which will contain the user lexicon and all the application lexicons):

CComPtr<ISpContainerLexicon> cpContainerLexicon;

cpContainerLexicon.CoCreateInstance( CLSID_SpLexicon );

The main lexicon function engines will want to use is ISpLexicon::GetPronunciations:

HRESULT GetPronunciations(

[in] const WCHAR *pszWord,

[in] LANGID LangId,

[in] DWORD dwFlags,

[out][in] SPWORDPRONUNCIATIONLIST *pWordPronunciationList

);

Here is an example of how to get pronunciations out of a Container Lexicon:

HRESULT hr = S_OK;

DWORD dwLexFlags = eLEXTYPE_USER | eLEXTYPE_APP;

SPWORDPRONUNCIATIONLIST SPList;

ZeroMemory( &SPList, sizeof( SPWORDPRONUNCIATIONLIST ) );

hr = cpContainerLexicon->GetPronunciations( pszWord, 1033, dwLexFlags, &SPList );

if ( SUCCEEDED( hr ) )

{

    for (
SPWORDPRONUNCIATION *pWordPron = SPList.pFirstWordPronunciation;
pWordPron;

pWordPron = pWordPron->pNextWordPronunciation )

{

//--- Do something with each pronunciation

}

}

if ( SPList.pvBuffer )

{

::CoTaskMemFree( SPList.pvBuffer );

}

SPWORDPRONUNCIATIONLIST is the structure SAPI uses to return a list of pronunciations for a word:

typedef struct SPWORDPRONUNCIATIONLIST

{

ULONG ulSize;

BYTE *pvBuffer;

SPWORDPRONUNCIATION *pFirstWordPronunciation;

} SPWORDPRONUNCIATIONLIST;

This struct should be initialized to zeroes before GetPronunciations is called (see the ZeroMemory call in the sample code, above). Furthermore, the memory allocated for the pronunciations which are returned in this structure must be freed by the engine after GetPronunciations is called - this memory is all pointed to by pvBuffer, hence a single ::CoTaskMemFree call will free all of the allocated memory (see the sample code, above). SPWORDPRONUNCIATIONLIST is just a linked list of SPWORDPRONUNCIATIONs:

typedef [restricted] struct SPWORDPRONUNCIATION

{

struct SPWORDPRONUNCIATION *pNextWordPronunciation;

SPLEXICONTYPE eLexiconType;

LANGID LangID;

WORD wReserved;

SPPARTOFSPEECH ePartOfSpeech;

SPPHONEID szPronunciation[1];

} SPWORDPRONUNCIATION;

eLexiconType indicates which type of lexicon this pronunciation came from - in the above sample code, eLexiconType will be either eLEXTYPE_USER or eLEXTYPE_APP for each returned SPWORDPRONUNICATION. szPronunciation is a NULL-terminated array of SPPHONEIDs which runs of the end of the SPWORDPRONUNCIATION structure into the pvBuffer member of SPWORDPRONUNCIATIONLIST;

If a word has a pronunciation in the User Lexicon, that pronunciation should take precedence over pronunciations in engine internal lexicons and pronunciations in Application Lexicons. Application Lexicon pronunciations should similarly take precedence over pronunciations in engine internal lexicons.

For more information on SAPI Lexicons, including adding and removing words from the User Lexicon, or using the basic SAPI Lexicon classes (SpCompressedLexicon, SpUncompressedLexicon) for an engines internal lexicons, see the Lexicon Manager section).

Appendix A - SAPI 5 Phonemes

SYM | Example | PhoneID
- | syllable boundary (hyphen) | 1
! | Sentence terminator (exclamation mark) | 2
& | word boundary | 3
, | Sentence terminator (comma) | 4
. | Sentence terminator (period) | 5
? | Sentence terminator (question mark) | 6
_ | Silence (underscore) | 7
1 | primary stress | 8
2 | secondary stress | 9
aa | father | 10
ae | cat | 11
ah | cut | 12
ao | dog | 13
aw | foul | 14
ax | ago | 15
ay | bite | 16
b | big | 17
ch | chin | 18
d | dig | 19
dh | then | 20
eh | pet | 21
er | fur | 22
ey | a te | 23
f | f ork | 24
g | g ut | 25
h | help | 26
ih | fill | 27
iy | feel | 28
jh | j oy | 29
k | cut | 30
l | l id | 31
m | m at | 32
n | n o | 33
ng | sing | 34
ow | go | 35
oy | toy | 36
p | put | 37
r | r ed | 38
s | s it | 39
sh | she | 40
t | talk | 41
th | th in | 42
uh | book | 43
uw | too | 44
v | v at | 45
w | with | 46
y | y ard | 47
z | z ap | 48
zh | pleasure | 49

Appendix B - SAPI 5 Visemes

VISEME | Described SAPI Phonemes
SP_VISEME_0 | Silence
SP_VISEME_1 | ae, ax, ah
SP_VISEME_2 | aa
SP_VISEME_3 | ao
SP_VISEME_4 | ey, eh, uh
SP_VISEME_5 | er
SP_VISEME_6 | y, iy, ih, ix
SP_VISEME_7 | w, uw
SP_VISEME_8 | ow
SP_VISEME_9 | aw
SP_VISEME_10 | oy
SP_VISEME_11 | ay
SP_VISEME_12 | h
SP_VISEME_13 | r
SP_VISEME_14 | l
SP_VISEME_15 | s, z
SP_VISEME_16 | sh, ch, jh, zh
SP_VISEME_17 | th, dh
SP_VISEME_18 | f, v
SP_VISEME_19 | d, t, n
SP_VISEME_20 | k, g, ng
SP_VISEME_21 | p, b, m


SAPI 5 SR Engine Porting Guide

1 Contents

SAPI Speech Recognition Engine Guide.. 1

1 Contents. 2

2 Summary.. 3

3 Introduction.. 4

3.1 SAPI SR Objects and Interfaces. 4

3.2 Sample SR Engine. 5

4 Engine Initialization and Setup. 5

4.1 Engine Creation.. 5

4.2 Object token layout. 6

4.3 SetObjectToken.. 6

4.4 RecoProfiles. 7

4.5 RecoContexts. 7

4.6 Recognizer Properties. 7

5 Grammar handling.. 8

5.1 Grammar Creation and Deletion.. 8

5.2 CFG Grammars. 9

5.2.1 Introduction and terminology. 9

5.2.2 Grammar Notifications. 9

5.2.3 Word Notifications. 9

5.2.4 Rule Notifications. 10

5.2.5 States. 10

5.2.6 Transitions. 11

5.2.7 Special Transitions. 11

5.2.8 Semantic Properties. 12

5.2.9 Additional topics. 12

5.3 Dictation Grammars. 13

5.3.1 Language model adaptation. 14

5.4 Proprietary grammars. 14

5.4.1 Porting other grammar formats. 14

6 Lexicon handling.. 15

6.1 Using lexicons. 15

6.2 Phone Converters. 16

7 Recognition and audio.. 16

7.1 RecognizeStream... 17

7.1.1 Active always state. 17

7.2 Reading audio.. 17

7.2.1 How audio formats are represented in SAPI17

7.2.2 Setting the audio format17

7.2.3 Reading data. 18

7.2.4 Information about the audio input18

7.2.5 Setting the input gain. 19

7.3 Threading model. 19

7.4 Synchronization.. 20

7.4.1 Pause and auto-pause. 20

7.5 Events and Recognitions. 21

7.5.1 Standard events. 21

7.5.2 Event ordering. 22

7.5.3 Other events. 22

7.6 Completion of processing.. 23

8 Recognition Results. 23

8.1 Recognition Call. 24

8.2 Dictation Phrases. 25

8.3 CFG Phrases. 26

8.4 Confidence Scoring and Rejection.. 27

8.4.1 Word Confidence. 27

8.4.2 Property and Rule Confidence. 27

8.4.3 Required Confidence and Rejection. 28

8.4.4 Ambiguous Results. 28

8.5 Inverse Text Normalization (ITN)29

8.6 Interpreters. 30

9 Alternates. 30

9.1 Returning alternates in a Recognition.. 30

9.2 Alternates Analyzer.. 31

10 User-Interface.. 31

11 Engine extensions. 32

2 Summary

This document describes fully the Speech Recognition engine interface in SAPI 5.0. Speech Recognition engines use this interface in order to be able to connect to SAPI and be used by applications. This document is aimed at engine vendors wishing to port their SR engine to using SAPI 5.0, and to general developers who are interested in understanding more about SAPI. This document explains which interfaces and objects SAPI implements, and which interfaces an SR engine should implement. It describes how engines are registered and initialized; how grammar and lexicon information is communicated to the engine; how engines read data and perform recognition; and how engines return events and results back to the application.

3 Introduction

The Microsoft Speech API (SAPI) is a software layer allowing speech-enabled applications to communicate with speech engines (Speech Recognition (SR) engines and Text to Speech (TTS) engines). SAPI includes an Application Programming Interface (API) and a Device Driver Interface (DDI). Applications communicate with SAPI via the API layer and speech engines communicate with SAPI via the DDI layer.

A speech-enabled application and an SR engine do not directly communicate with each other – all communication is done via SAPI. SAPI takes responsibility for a number of aspects of a speech system, such as:

· Controlling audio input, whether from a microphone, files, or a custom audio source; and converting audio data to a valid engine format.

· Loading grammar files, whether dynamically created or created from memory, URL or file; and resolving grammar imports and grammar editing.

· Compiling standard SAPI XML grammar format, and conversion of custom grammar formats, and parsing semantic tags in results.

· Sharing of recognition across multiple applications using the shared engine. All marshalling between engine and applications is done by SAPI.

· Returning results and other information back to the application and interacting with its message loop or other notification method. This allows an engine to have a much simpler threading model than in SAPI 4, as SAPI 5 does much of the thread handling.

· Storing audio and serializing results for later analysis.

· Ensuring that applications do not cause errors – preventing applications from calling the engine with invalid parameters, and dealing with applications hanging or crashing.

The SR engine is responsible for:

· Using SAPI grammar interfaces and loading dictation.

· Performing recognition.

· Polling SAPI in order to learn about grammar and state changes.

· Generating recognitions and other events to provide information to the application.

3.1 SAPI SR Objects and Interfaces

In order for an SR engine to be a SAPI 5 engine, it must implement at least one COM object. Each instance of this object represents one SR engine instance. The main interface this object must implement is the ISpSREngine interface. SAPI calls the engine using the methods of this interface to pass details of recognition grammars and tell the engine to start and stop recognition etc. SAPI itself implements the interface ISpSREngineSite. A pointer to this is passed to the engine and the engine calls SAPI using this interface to read audio, return recognition results etc.

ISpSREngine is the main interface that needs implementing, but there are other interfaces that an engine may implement. The SR engine can implement the ISpObjectWithToken interface. This provides a mechanism for the engine to query and edit information about the object token in the registry used to create the engine. Information about object tokens is provided in the “Object Tokens and Registry Settings White Paper” and in SetObjectToken.

There are two other interfaces that the engine can also implement. Each needs to be implemented in a separate COM object, because SAPI needs to create and delete them independently of the main engine. These interfaces are:

· ISpSRAlternates, which can be used by the SR engine to generate alternates for dictation results. It is possible to generate alternates without this interface, but this interface allows alternates to be generated off-line, after the result has been serialized. (See Alternates).

· ISpTokenUI allows engines to implement UI components that can be initialized from an application. These can be used to perform user training, add and remove user words, calibrate the microphone etc. (See User-Interface)

The engine can also implement another COM object enabling engine-specific calls between the application and the engine. This object can implement any interface that the application is able to QueryInterface for. (See Engine extensions)

3.2 Sample SR Engine

The SAPI 5.0 SDK contains working Microsoft speech recognition engines for US English, Japanese and Chinese. These engines are not shipped with source. The SDK also contains a Sample SR Engine, which is shipped with source (In directory Microsoft Speech SDK5.0\Samples\CPP\Engines\SR). This is a “fake” engine – it implements all the functionality of an SR engine and can be created and used in applications, but it does not actually recognize – instead it generates valid, but random, results. This is very useful example code for understanding how a real SR engine might be implemented.

4 Engine Initialization and Setup

4.1 Engine Creation

When an application wants to perform recognition, it can create a recognizer in one of two ways. The application can create an in-process (InProc) ISpRecognizer object. In this case, SAPI creates the SR engine COM object from the object token representing an engine. Alternatively, an application can create the shared recognizer. In this case, SAPI will create the SR engine in a separate process (named sapisvr.exe) and all applications will share this recognizer. Note that this process is completely invisible to the SR engine – all marshaling etc. is handled by SAPI.

In order to create the SR engine, SAPI uses the SetRecognizer call to look at the object token of the default recognizer or the object token that the application has specified it wants. The class ID (CLSID) of the main SR engine object is contained in the object token, and this class is created. The SR engine COM classes must register themselves with “ThreadingModel = Both” or they may not be successfully created.

The SetSite method on the ISpSREngine interface is then called to give the engine a reference to the ISpSREngineSite interface it will use to call back to SAPI on. Like all COM interfaces, the engine should use AddRef to maintain the correct reference count.

4.2 Object token layout

The SR engine install is responsible for adding an object token that represents the engine into the user’s system. Otherwise, SAPI will have no information about the SR engine. Add an object token by adding a key to this point in the registry:

HKEY_LOCAL_MACHINE\Software\Microsoft\Speech\Recognizers\Tokens

On a machine with SAPI 5 installed, there are several registry keys here for the Microsoft English, Chinese, Japanese and Sample SR Engine.

Inside this key there must be the value CLSID containing the CLSID of the main SR engine class. This is how SAPI determines how to create the engine. There are values for the other CLSIDs that the engine can implement – AlternatesCLSID for the class implementing the ISpSRAlternates alternates analyzer, and RecoExtension for the class implementing any engine specific private call interfaces. The key should also contain a {Default} value set with the name of the engine so that the Speech Control Panel displays it.

In this key there can also be a sub-key Attributes, used by applications to query for engines matching certain attributes. Typically, the engine sets a value Language to indicate which languages the engine supports; Dictation to indicate the engine supports dictation; CommandAndControl to indicate the engine supports command and control, and so on. (See the “Object Tokens and Registry Settings White Paper” in the Help for more information on object tokens).

There can also be a key UI showing the types of user-interface components the engine supports (See User-Interface).

4.3 SetObjectToken

Once SAPI creates the SR engine COM object, it determines if the engine supports the ISpObjectWithToken interface. If it does, then the SetObjectToken method is called. This passes a pointer to the object token this engine was created from. This is useful for two reasons:

· The engine can use the token to store information. It can store information directly in the object token using the ISpDataKey methods, or it can store file paths to other engine data. The ISpObjectToken interface has a method, GetStorageFileName, which provides an easy way for an engine to find a file path to store data. This path is stored in the Files sub-key of the engine object token.

· The engine can also read information from the token, such as file paths set during install, or user options set using an engine properties UI component. It is also possible for several object tokens to share the same engine CLSID – for example, for different language engines or for telephony or desktop variants of the engine. In this case, the engine needs to know from which object token it is being created.

4.4 RecoProfiles

The SetRecoProfile method is called next to give the engine an ISpObjectToken pointer referring to the current user profile. RecoProfiles are added or removed using Speech properties in Control Panel. The engine can create a sub-key under the profile object token and use it to store any data. The engine must store the data in a sub-key named after its CLSID. This avoids other engines on the system recognizing the same profile.

To provide user enrollment, the engine implements a UI component (See User-Interface) with the name UserTraining (SPDUI_UserTraining is defined as this in sapi.idl). This is instantiated by using Control Panel->Speech properties->SR Tab->Train Profile. Engines can also request that an application display the UI using the AddEvent method (See Events and Recognitions) to request UI.

The user-training UI might produce some adapted model files. These can be saved and their location stored in the RecoProfile object token. The engine can read the location of these files from the object token later.

4.5 RecoContexts

Each application using speech has at least one RecoContext object implementing ISpRecoContext. It is from this interface that the application creates and loads grammars and activates recognition. SAPI informs the SR engine of each RecoContext associated with it using the methods OnCreateRecoContext and OnDeleteRecoContext. The SR engine returns a pointer to SAPI from the OnCreateRecoContext function, which is then passed to the engine in any future calls that need to refer to the RecoContext. It is not essential for an engine to keep track of each RecoContext unless it is using private calls or proprietary grammars.

4.6 Recognizer Properties

SAPI provides a means for applications to set certain settings and configurations on the SR engine. This is done using the application calling methods on the ISpProperties interface, implemented on the RecoContext objects. There are four methods on this interface to get and set string and integer values. When these methods are called, SAPI calls equivalent methods on the SR engine - GetPropertyString, SetPropertyString, GetPropertyNum, SetPropertyNum.

In SAPI 5.0, each method has a SPPROPSRC parameter, which is always set to SPPROPSRC_RECO_INST, and a pointer pvSrcObj, which is always set to NULL.

A number of these properties are already defined by SAPI (See the SR Properties White Paper in the Help). Ideally, the engine should implement these if they have equivalent parameters that can be controlled.

When an application tries to set one of these values, and the SR engine is called, it returns S_OK if it supports this property and the value was updated, and S_FALSE if it does not.

Note that these properties exist to alter run-time settings for this instance of the engine, and are thus reset every time the engine is deleted. For permanent changes to engine performance, use an engine properties UI component, or include additional values in its object token that applications can read and set.

5 Grammar handling

Each speech application can have one or more ISpRecoGrammar objects associated with it. Within each grammar object there are several types of grammar:

· Command and control grammars. These are context-free grammars (CFG) created from either a SAPI XML grammar, or dynamically from the application, or from some other grammar format using the SpGramCompBackend object. In all cases, SAPI reports the contents of the grammars to the engine using the same mechanism.

· Dictation grammars. Here the engine loads and unloads its own dictation language model.

· Proprietary grammars. There are various calls in SAPI to support engine-specific grammar formats.

Each grammar object can contain a dictation grammar and either a CFG or proprietary grammar.

Each application can have several grammars and in the shared recognizer case multiple applications can be connected to one recognizer. Thus, grammars can be loaded and unloaded, modified, activated, and deactivated independently of each other. However, the SR engine is able to control when it is informed of these grammar state changes during recognition (See Synchronization).

5.1 Grammar Creation and Deletion

When an application creates a grammar object, this is reported to the engine using the OnCreateGrammar method. This passes to the engine a grammar handle, as well as the pointer the SR engine returned from the call to OnCreateRecoContext. From this method, the engine must also return a pointer, which is used to identify the grammar in later calls from SAPI. Use the OnDeleteGrammar method to delete grammars.

5.2 CFG Grammars

5.2.1 Introduction and terminology

Each CFG grammar contains one or more rules. Rules can be top-level, indicating that they can be activated for recognition. Each rule has an initial state and additional states, connected by transitions. Each transition can be one of several types:

· A word transition indicating a word to be recognized

· A rule transition indicating a reference to a sub-rule

· An epsilon (null) transition

· Some special transitions for such features as embedding dictation within a CFG.

References to sub-rules can be recursive – i.e., rules can reference themselves, either directly or indirectly. Left recursion is not supported and SAPI will reject such grammars upon loading. Inside a grammar, transitions can have semantic properties, although the engine does not normally need to recognize these.

SAPI takes full control of loading a grammar when an application asks it to. SAPI can load from a file, a URL, a resource, or from memory, and can load either binary or XML forms of the grammar, and resolve imports. SAPI then notifies the SR engine about the contents of the grammar through various DDI methods.

5.2.2 Grammar Notifications

To be notified about CFG grammar information, the engine implements two methods: WordNotify and RuleNotify. SAPI calls both before recognition begins, when a grammar is first loaded, and during recognition within a Synchronize call if grammars change (See Synchronization).

5.2.3 Word Notifications

The WordNotify call informs the engine about the words in the grammar. A single call is made to either add or remove words. SAPI keeps a reference count internally so that each word will be added only if it is not present in any existing grammar. Each word is represented by an SPWORDENTRY structure:

typedef struct SPWORDENTRY

{

SPWORDHANDLE hWord;

LANGID LangID;

WCHAR *pszDisplayText;

WCHAR *pszLexicalForm;

SPPHONEID *aPhoneId;

void *pvClientContext;

} SPWORDENTRY;

The hWord is a unique handle identifying the word. The pvClientContext is an arbitrary pointer that the SR engine sets with a call to SetWordClientContext. Subsequent calls to the method GetWordInfo will return the same structure with this field filled in, which may be helpful to the engine. The LangID field represents the language of the word. Currently, this will be the same for all words in a grammar, but in the future SAPI may support multi-lingual grammars.

The pszDisplayText and pszLexicalForm fields give the text of the word. Words can be defined in a grammar to have a different textual display form to the actual spoken lexical form used to look up the words in a lexicon. The grammar can also specify the pronunciation of the word. This is given as an array of SPHONEIDs. See Phone Converters for more detail on phones and phone converters.

5.2.4 Rule Notifications

The RuleNotify call informs the engines when rules are added, changed or removed. There are five actions that are performed on rules:

· New rules can be added or existing rules removed.

· Rules can be activated or deactivated for recognition.

· Rule can be invalidated, which means the rule has been edited by the application and thus the engine needs to reread the contents of the rule.

Each rule is represented by an SPRULEENTRY structure:

typedef struct SPRULEENTRY

{

SPRULEHANDLE hRule;

SPSTATEHANDLE hInitialState;

DWORD Attributes;

void * pvClientRuleContext;

void * pvClientGrammarContext;

} SPRULEENTRY;

The hRule is a unique handle identifying the rule. The pvClientRuleContext is a pointer that the engine sets using SetRuleClientContext. Subsequent calls to the GetRuleInfo method return the same structure but with the pvClientRuleContext field filled in. The pvClientGrammarContext is the pointer that the engine set in the OnCreateGrammar method. This indicates which grammar the rule belongs to. The Attributes field, of type SPCFGRULEATTRIBUTES, contains some flags giving extra information about the rule:

· SPRAF_TopLevel if the rule is top-level and thus can be activated for recognition.

· SPRAF_Active if the rule is currently activated.

· SPRAF_Interpreter if the rule is associated with an Interpreter object for semantic processing (See Interpreters).

· SPRAF_AutoPause if the rule is auto-pause (See Pause and auto-pause).

The hInitialState gives the initial state of the rule.

5.2.5 States

The SR engine determines the full contents of the rule (either immediately, or later during recognition), using the GetStateInfo method. This method passes information about all the subsequent states following on from any given state. The engine passes a state handle into this method (starting with the hInitialState of the rule), and a pointer to an SPSTATEINFO structure (with all its fields initially zeroed). This structure gets filled out with information on all of the transitions out of that state in the pTransitions array. SAPI uses CoTaskMemAlloc to create this array. The engine can call this method again on each of the states following the current state, and so on, in order to get information about all of the states in the rule. Note that loop-back transitions are possible in a rule so the engine needs to check that it has not visited the current state before.

When the engine calls GetStateInfo subsequent times, it can call it with the cAllocatedEntries and pTransitions fields unchanged. SAPI will then re-use the memory from the transition array if possible rather that re-allocating it. Alternatively, the engine can use CoTaskMemFree to free the pTransitions memory, and set these fields to NULL. SAPI will then re-allocate the memory every time.

5.2.6 Transitions

Each transition represents a link from one state to another state and is represented by an SPTRANSITIONENTRY structure. This structure contains an ID field that uniquely identifies the transition, an hNextState handle that indicates the state the transition is connected to, and a Type field that indicates what type of transition this is.

There are three common types of transition, which all engines need to support:

· Word transitions (SPTRANSWORD). These represent single words that the recognizer recognizes before advancing to the next state. Inside the SPTRANSITIONENTRY structure the handle to the word is supplied, which the engine uses to find the full text of the word with GetWordInfo. Also, the engine’s word pointer is supplied. To produce recognition results the engine needs to keep track of the transition IDs of word transitions as they are used in the ParseFromTransitions method.

· Rule transitions (SPTRANSRULE). These represent transitions into sub-rules. This transition is only passed when a path through the sub-rule has been recognized. The rule handle, engine’s rule pointer and initial state of the sub-rule are supplied. Rules can be recursive, but not left recursive.

· Epsilon transitions (SPTRANSEPSILON). These are “null” transitions that can be traversed without recognizing anything.

The end of a rule is indicated by a state having a transition to a null state handle. There can also be “void” states, which are blocking and indicate that there is no recognition path from this state. These void paths are indicated by a state having zero transitions out of it.

5.2.7 Special Transitions

There are a number of special transitions that may not be supported by all engines. Attributes in the engine object token indicate whether these are supported:

· Wildcard transition (SPTRANSWILDCARD). This indicates a transition that matches any word or words (sometimes called a “garbage” model). The engine does not have to try and recognize the spoken words. The engine includes the string value WildcardInCFG as an attribute in its object token to indicate to the application that it is capable of supporting this.

· Dictation transition (SPTRANSDICTATION). This is used to embed dictation within a CFG. Each transition means one word should be recognized. The attribute DictationInCFG in the engine object token indicates support for this feature.

· Text buffer transition (SPTRANSTEXTBUF). This indicates that the engine is to recognize a sub-string of words from the text-buffer, if it has been set. (See Text-buffers).

5.2.8 Semantic Properties

Application developers are able to put properties (also known as semantic tags) within a grammar. This provides a powerful means for semantic information to be easily embedded inside a grammar.

By default, the engine does not see these properties. Typically, an engine simply recognizes the speech from the words in the grammar, and SAPI parses and adds the property information in the ParseFromTransitions call. However, it is possible for an engine to receive this information by calling GetTransitionProperty on any transition. If there is a property on this transition, the property name or id and value are returned in the SPTRANSITIONPROPERTY structure, which needs to be freed using CoTaskMemFree after use.

5.2.9 Additional topics

5.2.9.1 Ordering of actions

SAPI notifies engines about the contents of CFGs in a logical order. When a grammar is loaded, all new words are notified first with a WordNotify call, then all rules with a RuleNotify. Then rules are activated and deactivated. When a grammar is deleted all rules are removed first and then all words.

All the rules and words for each grammar are added with single calls to RuleNotify and WordNotify. If there are a number of grammars being loaded and rules being activated then separate calls are made for each grammar. For engines that have some time-consuming internal grammar compilation to do before starting recognition, care needs to be taken to avoid unnecessary recompilations. Where possible, applications should try and combine separate CFG grammars into one to minimize this issue.

5.2.9.2 Grammar Weights

Each transition has a Weight field that can be used by grammar designers to change the likelihood of certain paths being taken. This Weight field is a probability – the range of values is 0.0 to 1.0, and the values of the transitions out of any state sum to 1.0. A value of 0.0 should always be interpreted as making this transition impossible to pass during recognition. Engines may or may not incorporate the other weight values into their recognition search. By default, grammars do not have weights set, so each transition weight is 1.0 divided by the number of transitions out of the preceding state.

5.2.9.3 Required Confidence

Each transition also contains a RequiredConfidence field. Grammar designers use this field to set how easily recognitions are to be accepted or rejected. For example, this field is used to avoid false positives on critical actions (“delete file”), but not affect less critical actions (“scroll down”). See Required Confidence and Rejection for more information on how this field can be used for rejection.

5.2.9.4 Text-buffers

This feature allows an application to define a text buffer. When a text-buffer transition is reached in a CFG, the engine should try and recognize a sub-string of words from the text buffer.

The text buffer is set by the application using the method ISpRecoGrammar::SetWordSequenceData, and reported to the engine using the ISpSREngine::SetWordSequenceData method. The format of the buffer is a sequence of one or more null-terminated strings, with a double null-termination at the end. The engine can recognize any sub-string of words from any of the strings in the buffer. This provides a very simple way for applications to allow selection from a set of text.

It is also possible for the application to alter the areas of the buffer are to be used for recognition. This is done using the SetTextSelection method with the structure SPTEXTSELECTIONINFO. The ulStartActiveOffset and cchActiveChars show which area of the buffer should be active for recognition.

The other two fields of the SPTEXTSELECTIONINFO, ulStartSelection and cchSelection are used with dictation, to indicate which area of the buffer is currently selected on screen. If cchSelection is zero, then this shows where the insertion point currently is. The engine could use this insertion point to get extra language model context from the preceding words in the dictated text.

This text buffer feature is optional for engines, and support for it is shown in the TextBuffer attribute.

5.2.9.5 Rule invalidations

When a rule is invalidated by a RuleNotify call with SPCFGN_INVALIDATE action, the engine needs to discard any cached information it has regarding this rule and parse the rule again from the new initial state. If a sub-rule is edited in a grammar, then only that rule is invalidated, and not other rules referring to this rule.

5.2.9.6 Grammar Resources

It is possible for resource data to be included in a grammar. Each rule can contain one or more named strings containing arbitrary data. The engine recovers this data using the GetResource method, passing in the rule handle and resource name.

5.3 Dictation Grammars

The mechanism used for handling dictation grammars is considerably simpler than for CFG grammars. SAPI tells the engine to load a dictation grammar (or Statistical Language Model) with the LoadSLM method. This method gives the engine’s pointer to the grammar, and a topic name. The topic name is by default NULL, meaning use the standard dictation language model, but an application can request a specific topic. Currently, the only topic SAPI defines is Spelling for a spelling-mode grammar, although engines are free to define others. Dictation is unloaded with UnloadSLM. Dictation is activated or deactivated for recognition with the SetDictationState method.

5.3.1 Language model adaptation

An application can supply text data to the engine for language model adaptation with the SetAdaptationData method. The engine is free to do nothing or anything with this data, and to either persist the adaptation in the current RecoProfile or reset every session. Because some engines take considerable processing to do adaptation, it is recommended that applications submit adaptation data in chunks to the engine. When the engine is ready to receive more data, it fires an event SPEI_ADAPTATION. If an engine does not have this performance issue, it can send the event immediately.

5.4 Proprietary grammars

It is possible for an engine and application use a grammar format different from the standard SAPI binary or XML formats. The methods LoadProprietaryGrammar and UnloadProprietaryGrammar are used to load and unload such grammars. LoadProprietaryGrammar is called when the application calls ISpRecoGrammar:: LoadProprietaryGrammar. The application can supply the engine with either string data, binary data, or a GUID, or some combination of these. SAPI does not touch this data in any way apart from marshaling it between the application and shared engine.

To activate rules in a proprietary grammar, call the SetProprietaryRuleState or SetProprietaryRuleIdState. The first of these methods takes an optional string pointer, where NULL is interpreted to mean activate all top-level rules. This order is used so that ISpRecoGrammar::SetRuleState works consistently on SAPI CFGs and Proprietary grammars. The engine must set the pcRulesChanged value to inform SAPI how many rules have been activated or deactivated.

There are some extra methods that an SR engine needs to recognize if it is using proprietary grammars. The SetGrammarState is called when a grammar gets activated or deactivated, and the SetContextState does the same for a context grammar. With CFG grammars, SAPI will automatically deactivate or activate all relevant rules so these methods are not needed.

5.4.1 Porting other grammar formats

A disadvantage of proprietary grammar formats is that they are not engine or application independent. It is possible to write code that will convert any finite-state or CFG-based grammar format into the compiled SAPI binary format. For example, SAPI XML grammars are compiled using the object CLSID_SpGrammarCompiler. This object uses the CLSID_SpGramCompBackend object to actually add the correct rules, states, and transitions into the grammar, and then saves the grammar to an IStream using the SetSaveObjects and Commit methods.

Exactly the same approach can be used for other formats. A new COM object could be created using the compiler back-end methods to add the correct information and save the grammar (See the ISpGrammarBuilder interface documentation for more info). The grammar produced is a standard SAPI-compiled grammar and can be used by any SAPI engine or application, without using the proprietary grammar support.

6 Lexicon handling

SAPI provides a means for users or applications to specify new words and their pronunciations in lexicons. An engine should look at these words and pronunciations and use them during recognition.

There are two types of lexicons in SAPI:

· User Lexicon. There is one of these for the current user logged onto the machine. This is initially empty but can be added to by the user, either programmatically, or by using an engine’s add/remove words UI component. For example, running the Dictation Pad sample application and choosing Add/Remove Words will bring up the Microsoft engine UI where the user can add new words to their user lexicon.

· Application lexicons. Applications can create and ship their own lexicons of specialized words. These are read-only.

Both types of lexicons are represented by object tokens. (Application Lexicons are stored in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\AppLexicons\ and the current user lexicon in HKEY_CURRENT_USER\SOFTWARE\Microsoft\Speech\CurrentUserLexicon).

Each lexicon object implements the ISpLexicon interface. Lexicons can be individually created from their object tokens. However, SAPI provides a Container Lexicon that combines the user and application lexicons into one single entity, making manipulating the lexicon information simpler. This class is SpLexicon (CLSID_SpLexicon).

6.1 Using lexicons

Typically, when an engine runs dictation, it will use CoCreate to create the SpLexicon object and determine what words and pronunciations are in it. The engine adds these to its dictation vocabulary and language model if it supports this feature. If the lexicon contains words with no pronunciation, the engine should try and generate pronunciations for them if it supports this feature. Similarly, if a CFG grammar is loaded containing words that are in the lexicon, the engine acknowledges these pronunciations.

Applications can call IsPronounceable, on the RecoContext objects, to determine if the engine can generate a pronunciation for a word. Engines set the BOOL pointer passed in to TRUE, if a pronunciation was found, otherwise, FALSE.

An engine also adds words into the user lexicon using AddPronunciations on the SpLexicon. (See the Lexicon Manager section in the Help for more information).

As an option, an engine also uses the basic SAPI lexicon classes (CLSID_SpCompressedLexicon for read-only lexicons and CLSID_SpUncompressedLexicon for read-write lexicons) for its own internal lexicons.

6.2 Phone Converters

For each language, SAPI defines a phone-set describing which phones can be used in defining the pronunciation of a word. Currently phone sets are defined for US English, Japanese and Chinese, and more will be added in later updates. Phone converters are represented as object tokens in the registry. Each phone converter object implements the interface ISpPhoneConverter. This has two methods, PhoneToId and IdToPhone, to convert between phone strings and phone ids.

The pronunciations returned from a lexicon are returned as null-terminated arrays of SPPHONEID. All the methods that use pronunciations also use these arrays of phone ids and the engine may never need to use a phone converter directly. However, to create a phone converter for a specific language, an engine uses the SpCreatePhoneConverter method in sphelper.h:

ISpPhoneConverter *pPhoneCon;

HRESULT hr = SpCreatePhoneConverter(409, NULL, NULL, &pPhoneCon);

If a vendor wishes to implement an engine in a language for which Microsoft does not currently define a phone converter, then either it will be impossible to use lexicons or grammars with pronunciations in them, or the engine vendor will need to define their own phone converter, which would be engine specific.

7 Recognition and audio

SAPI indicates that the engine should start recognition by calling the RecognizeStream method on the SR engine. From that point on, the engine can read data, perform recognition, and send results and events back to SAPI. When all the data to be recognized has been exhausted or the application has deactivated recognition, the engine finishes processing and returns from the RecognizeStream call.

The basic actions that take place are as follows:

· SAPI calls RecognizeStream on the engine.

· The Engine starts reading data and recognizing.

· The Engine calls Synchronize and UpdateRecoPos methods to be informed of grammar changes.

· The Engine returns events, hypotheses, and recognitions to SAPI.

· Recognition continues until the stream is terminated and the engine returns from RecognizeStream method.

7.1 RecognizeStream

RecognizeStream will normally get called after grammars have been loaded and activated. The engine is expected to recognize from the rules that are active. If multiple rules or dictations are active, the engine is expected to recognize from all things in parallel, i.e., the user is able to say something from any rule or dictation that is active.

7.1.1 Active always state

There is one case when RecognizeStream can be called when there are no active rules. This is when the application has set the RecoState to SPRST_ACTIVE_ALWAYS (with the ISpRecognizer::SetRecoState method). This might be done by an application when it wants audio running to display a VU-meter (by listening to SPEI_SR_AUDIO_LEVEL events). In this case, RecognizeStream is called regardless of whether there are any active rules. The engine is free to throw the data away; although engines can use this data to perform environmental adaptation or noise level estimation.

7.2 Reading audio

7.2.1 How audio formats are represented in SAPI

Two fields are used to define audio formats in SAPI: A GUID defining the class of format; and for wave format types, a WAVEFORMATEX structure that contains type, sample rate, bits per sample etc. All wave format types have the GUID SPDFID_WaveFormatEx. Engines can use other GUIDs for any engine-specific formats they have. A helper class CSpStreamFormat in sphelper.h converts to and from this format and an enumeration, SPSTREAMFORMAT, commonly used formats.

The WAVEFORMATEX structure has the following definition:

typedef struct WAVEFORMATEX

{

WORD wFormatTag;

WORD nChannels;

DWORD nSamplesPerSec;

DWORD nAvgBytesPerSec;

WORD nBlockAlign;

WORD wBitsPerSample;

WORD cbSize;

} WAVEFORMATEX;

For example, the format for mono, PCM linear, 16-bit, 16kHz audio would be indicated by:

GUID guid = SPDFID_WaveFormatEx and

WAVEFORMATEX wfx = { WAVE_FORMAT_PCM, 1, 16000, 32000, 2, 16, 0 }

7.2.2 Setting the audio format

To determine what audio formats the engine supports, SAPI calls the method GetInputAudioFormat on the SR engine. The first pair of parameters of this method indicate the format SAPI is determining if the engine supports; the engine fills in the second pair of parameters to indicate which format it supports. Alternatively, SAPI calls this method with the first format set to NULL, in which case the engine sets the second pair of parameters to its preferred format.

After this is done, the RecognizeStream call specifies the current audio format in the rguidFmtId and pWavFormatEx parameters. This will not change during the RecognizeStream.

7.2.3 Reading data

The Read method is used to read data from the audio source. The SR engine requests the amount of data to be read. SAPI returns this data immediately if it is available, or will block until it is available. If the Read call returns a failure code or it reports that less data has been read than requested, the stream has ended and the engine can return from the RecognizeStream call once it has processed any data it has buffered.

It is possible to see how much data is available for reading immediately with the DataAvailable call. This call is used to read only the data that is already available without blocking. There is also a mechanism to signal the engine when a certain amount of data is available. A Win32 event parameter hDataAvailable on RecognizeStream is set when a certain amount of data is available. The amount of data before this happens can be set by the SetBufferNotifySize method.

The engine should try and read data as close to real-time as possible. SAPI has only a finite buffer of audio data, so if the engine’s reading lags by more than a certain amount of time (approx. 30 seconds currently), SAPI terminates the stream. The engine has the option of keeping its own buffer of data that has been read but has not yet processed.

Note that the amount of data requested in the Read call is in bytes. All stream positions used in sending events, recognition information etc. back to SAPI, are all in bytes also.

7.2.4 Information about the audio input

The SR engine does not directly have access to the audio input device. SAPI handles this so that the SR engine is consistent regardless of the type of input. However, there is some information about the stream that the engine receives from parameters in the RecognizeStream call:

· The rguidFmtId and pWavFormatEx parameters indicate the format of the stream.

· The pAudioObjectToken points to the object token that represents the audio input device.

· The fRealTimeAudio Boolean indicates whether the input is real-time or not. Real-time inputs in SAPI are those that implement the ISpAudio interface. An example of this is the standard multi-media microphone input. Non-real time streams are those that only implement ISpStreamFormat. For example, inputting wave files using the ISpStream object. With non real-time streams, all the data is available for reading immediately. The hDataAvailable event is always set and the DataAvailable method always returns INFINITE.

· The fNewAudioStream specifies whether this call to RecognizeStream is a restarting of an existing stream or a new stream. For example, if an application deactivates the active rules and the RecognizeStream returns, and then later the application activates some rules, the RecognizeStream call will have this parameter set as FALSE. Only if the application does a new SetInput will this return TRUE. This is useful for preserving environmental information between calls to RecognizeStream. Reset fNewAudioStream only when the input is really changed.

7.2.5 Setting the input gain

The engine can alter the gain of the input, if the input is from an audio device passing through the Windows Mixer. The engine stores a value in the RecoProfile for each device indicating what the gain should be, and SAPI will set the gain on this device in the mixer every time the audio is opened. The engine can set this value in the RecoProfile either when calibrating a microphone within its Microphone Training UI component, or at any other point.

The value stored in the profile must be in a sub-key with the same name as the CLSID of the main engine object. The value name should have the token ID of the audio input object currently being used (found by calling GetId on the audio input object token). The value should be a DWORD value between 0 and 10000 indicating the mixer level to set.

7.3 Threading model

With SAPI, engines use a very simple a threading model. When recognition does not occur (i.e., when the RecognizeStream method is not being called on the engine), SAPI calls the engine on only one thread. During the RecognizeStream, SAPI only calls the engine when the engine itself calls Synchronize. Thus, the engine can control when it is called back. These call backs also occur only on one thread.

Because the engine does not return from the RecognizeStream method until all recognition is completed, SAPI has effectively given the engine one thread on which to operate. It is possible to write an engine to do all its work on this one thread and thus require no additional threads or critical sections or other thread-locking. This one thread system works best if the engine is not blocked unnecessarily during Read calls when it could be performing recognition. This can be achieved by using the DataAvailable method.

An alternative arrangement would be to have one additional thread. In this case, one thread could read the data, and possibly perform feature extraction, while another thread could do the actual recognition processing. Other threading arrangements are possible, and SAPI makes no restrictions about which threads call which methods or whether the methods are called simultaneously.

7.4 Synchronization

The Synchronize method informs SAPI that the engine is ready to receive any pending actions, such as grammar changes, rule activations/deactivations, private calls etc. When the engine calls Synchronize, the engine is called from SAPI to perform these actions. For example, if the application has changed a grammar and called Commit, then the next time the engine calls Synchronize, SAPI will call the engine back on the RuleNotify and WordNotify methods to give the engine the details of the modified grammar. When the engine returns from these methods SAPI returns back from Synchronize (as long as the engine is not in a paused state). Note that when the engine sends a final recognition, (See Events and Recognitions) Synchronize is also called internally by SAPI.

The engine has complete control over when Synchronize is called. For example, engines may only want to handle grammar changes when the user is not speaking; and not handle changes when they are actually doing recognition. There is also a Win32 event fRequestSync that is passed as a parameter in the RecognizeStream call. This event is set when an action is queued for the engine to respond to when it calls Synchronize. An engine can either call Synchronize only when this event is set, or regularly call Synchronize.

The more quickly the engine responds to queued tasks, the more responsive it will seem to a user. An engine should not wait too long before calling Synchronize because in some cases an application will hang until the engine does so. For example, the “fake” SR Sample engine does not normally call Synchronize when it has detected speech. However, if speech has been detected for a long time and the hRequestSync event is set, then the engine will always call Synchronize to prevent hanging.

The stream position given as a parameter to Synchronize indicates the point where the engine thinks it is sure it won’t fire any events before. SAPI discards its stored audio before this point and thus the engine cannot fire any events or report recognitions before this point. This position does not need to be exactly where the engine is currently recognizing, it can be further back to a point where the engine fire any events.

There is another method that an engine should call regularly during recognition. This is UpdateRecoPos and is used to inform SAPI of the engine’s position in recognizing the stream. This is currently used to ensure that Bookmarks are fired correctly, and to keep the application informed of the recognizer position (using the ullRecognitionStreamPos field returned from ISpRecognizer::GetStatus).

7.4.1 Pause and auto-pause

It is possible for an engine to be put into a paused state. This happens for one of three reasons:

· The application called ISpRecoContext::Pause.

· A rule, which the application activated as SPRS_ACTIVE_WITH_AUTO_PAUSE, was recognized.

· A bookmark event of type SPBO_PAUSE has been reached.

When in the paused state, SAPI does not return to the engine from a call to Synchronize or a final recognition. Instead, control sits inside SAPI and it calls back into the engine to inform it of any grammar changes etc. that may occur. In fact this result is identical to what would happen in a normal call to Synchronize or final recognition, except that SAPI waits inside the call until the application resumes the engine. While in the paused state, the engine is still able to Read data if it has another thread running. If the engine also performs sound start/end detection in this thread, it could fire those events to the application.

The paused state allows an application to make grammar and state changes at specific points. Normally grammar changes only occur the next time an engine calls Synchronize. Thus, if an application wanted to make a number of changes, then some might occur during one Synchronize call and some in a different one, which might not be desired. The pause functionality allows the application to make changes: after each recognition (using Auto-Pause); at a specific point in the stream (using Bookmarks); or just as soon as possible (using Pause); and make as many grammar changes as needed. Then when the application calls Resume, the engine continues recognizing, without having lost any data, but with all the grammar changes having been reported.

7.5 Events and Recognitions

7.5.1 Standard events

In order to report to the application information about what is being recognized, there are several events the engine can report. These indicate, for example, that the engine has detected the start or end of speech, or that it has a hypothesis or a completed recognition result. The main events used to report the progress of recognition are as follows:

· Sound Start. Used to indicate that the start of some speech-like sound has been detected. Reported by calling AddEvent with event type SPEI_SOUND_START.

· Sound End. Used to indicate that the end of some speech-like sound has been detected. Reported by calling AddEvent with event type SPEI_SOUND_END.

· Phrase Start. Used to report the start of some speech that the engine believes is an utterance matching the currently active grammar. Reported by calling AddEvent with event type SPEI_PHRASE_START.

· Final recognition. Used to return results of the recognition of an utterance. Reported by calling Recognition.

· False recognition. Used to indicate that the engine attempted recognition of the utterance but rejected it on the basis of low confidence scores, inability to find a valid path, etc. Indicated by calling Recognition with the eResultType having the SPRT_FALSE_RECOGNITION flag set.

· Hypothesis. Used to report a partial recognition of the utterance. Indicated by calling Recognition with the fHypothesis flag set.

The AddEvent method takes as parameters an SPEVENT structure, and an SPRECOCONTEXTHANDLE which should be set to NULL. The SPEVENT has the following fields:

SPEVENTENUM        eEventId;

SPEVENTLPARAMTYPE  elParamType;

ULONG       ulStreamNum;

ULONGLONG   ullAudioStreamOffset;

WPARAM      wParam;

LPARAM      lParam;

The elParamType should be set to SPET_LPARAM_IS_UNDEFINED and the lParam and wParam are set to NULL to show no extra information is returned with these events. The ulStreamNumber can also be set to 0 as SAPI fills this field in before returning the event to the application. The eEventId indicates the type of event (SPEI_SOUND_START, SPEI_SOUND_END, or SPEI_PHRASE_START). The stream position indicates the position in the audio stream where the engine decides this event has happened.

The Recognition method is used to send hypotheses and final or false recognitions.

7.5.2 Event ordering

There are various requirements for the chronological ordering and stream positions of how these events are reported:

· Sound start and sound end events form a pair. Every sound start call must later have a sound end call with a later stream position.

· Each phrase start event forms a pair with either a final or false recognition. The recognition must be fired later and have stream positions later than the phrase start event. Between the phrase start and recognition some hypotheses can optionally be located.

· Each phrase start/recognition pair must have stream positions inside a sound start/sound end pair, i.e. if part of the stream has been determined to be a phrase it must also be speech. Zero, one, or more than one phrase start/recognition pairs can sit between a sound start/end. Different phrase start/recognition pairs cannot overlap. If part of the stream is determined to be in one utterance, it cannot belong to other utterances.

· Although the phrase starts and recognitions must have stream positions within sound start/end events, the actual time sequence of the firing of these events can vary. For example, if an engine has an independent speech detector, which can determine the end of speech before the recognition has completed, it can fire the sound end event before a recognition event.

7.5.3 Other events

There are several other event types the engine can fire to indicate other information to SAPI and applications:

· SPEI_ADAPTATION. This is used to show that the engine is ready to receive more text adaptation data (See Language model adaptation).

· SPEI_REQUEST_UI. This is used to request a display of one of the engine's UI components. Both these events are called with a stream position that is either valid or zero (See User-Interface).

· SPEI_INTERFERENCE. This is used to indicate to applications that there is a problem with the audio stream or user speech. This event is called with the LPARAM set to one of the SPINTERFERENCE values to indicate the nature of the problem (no signal, clipping, user speaking too fast etc). Applications have the choice of whether or not to respond to these events.

· SPEI_SR_PRIVATE. This is an event type that engines use for engine specific communication with applications. Engines should include some unique identifier in the data sent with the event to distinguish the use of this event from another engine's. The SPRECOCONTEXTHANDLE parameter in the AddEvent call can also be used to send this event to a particular RecoContext, rather than to all contexts.

7.6 Completion of processing

An engine should continue recognizing as long as data is available and it is not told to finish. Typically, after an engine reports a recognition, it will check for grammar changes and then continue reading data and recognizing.

The main indication that the engine should finish recognition is when there is no more data to read from the stream with the Read call. The engine should finish processing the data it has, sending events and recognitions as necessary, and return from the RecognizeStream method. There is also a Win32 event, passed as the hExit parameter to RecognizeStream, which is set to indicate that the recognizer should exit immediately, without necessarily completing reading all data. This condition is also shown by the Recognition or Synchronize methods returning S_FALSE rather than S_OK.

Recognition is complete for one of several reasons:

· All active rules in the applications connected to the engine were deactivated.

· An application set the recognition state (with SetRecoState) to inactive or inactive with purge.

· There is no data left in the stream (e.g., for wav file streams).

· There was an error or buffer overflow in the stream reading.

Note that the default model in SAPI is for an engine to continue recognizing unless it is explicitly informed to stop. This corresponds to, say, a desktop application where everything the user says is being listened to and acted upon. For some systems an utterance-by-utterance method is required, where recognition stops after each thing the user says. This is best implemented by the application using the auto-pause feature when it activates rules. Then, after every recognition, the engine will pause, and the application can do any processing it likes and then terminate recognition by deactivating all active rules.

8 Recognition Results

In this chapter, the process of returning results is explained in more detail.

8.1 Recognition Call

The Recognition call itself takes a pointer to the following structure as a parameter:

typedef struct SPRECORESULTINFO

{

    ULONG             cbSize;

    SPRESULTTYPE      eResultType;

    BOOL              fHypothesis;

    BOOL              fProprietaryAutoPause;

    ULONGLONG         ullStreamPosStart;

    ULONGLONG         ullStreamPosEnd;

    SPGRAMMARHANDLE   hGrammar;

    ULONG             ulSizeEngineData;

    void             *pvEngineData;

    IspPhraseBuilder *pPhrase;

    SPPHRASEALT*      aPhraseAlts;

    ULONG             ulNumAlts;

} SPRECORESULTINFO;

These fields are set as follows:

· cbSize is simply size of(SPRECORESULTINFO).

· eResultType is either SPRT_CFG, SPRT_SLM, or SPRT_PROPRIETARY. The SPRT_FALSE_RECOGNITION flag is set to indicate a false recognition.

· fHypothesis indicates whether the result is a hypothesis or final recognition.

· fProprietaryAutoPause is set false unless using auto-pause with proprietary grammars.

· ullStreamPosStart/End provides the start and end positions of the result. This informs the application of the position of the result, and controls what audio data is retained in the result. The start position must be equal or later than the stream position reported for the phrase start corresponding to this recognition, and later than the stream positions reported in previous Synchronize calls.

· hGrammar is set to the grammar handle for dictation or proprietary results, and NULL for CFG results.

· pvEngineData allows arbitrary data of size ulSizeEngineData to be returned with the result. This is used for generating alternates if an alternates analyzer object is being used.

· The pPhrase structure contains the main information about the result. This is created in different ways for CFG, dictation, or proprietary results. This can also be NULL for false recognitions if the engine has no phrase to report. See below for details of how to create

· aPhraseAlts is an array containing ulNumAlts alternates for this recognition. These are optional (See Returning alternates in a Recognition).

8.2 Dictation Phrases

The standard method to construct a phrase builder object to hold a dictation result is for the engine to create an SPPHRASE structure with the desired information and then CoCreate an SpPhraseBuilder object (CLSID_SpPhraseBuilder). The SPHRASE information is then added with ISpPhraseBuilder->InitFromPhrase.

The SPPHRASE has the following fields:

· cbSize. This is the size of (SPPHRASE).

· LangID. The LANGID of the result.

· ullGrammarID, wReserved, ftStartTime, ulRetainedSizeBytes, ulAudioSizeTime. Set by SAPI and can be left as 0.

· ullAudioStreamPosition and ulAudioSizeBytes. Indicate the position of the result in the audio stream. The position is in bytes, relative to the start of the stream. The part of the stream spanned by this phrase must be the same as or less than the range in the ullStreamPosStart/End fields in the SPRECORESULTINFO.

· Rule. Set to zero for dictation results apart from the ulCountOfElements, which must be filled in with the number of words in the result.

· pProperties. The array of semantic properties, which will be NULL for a dictation result.

· pElements. The array of actual words in the result.

· pReplacements. This holds an array of size cReplacements of ITN text replacements the may fill in (See Inverse Text Normalization (ITN)).

· SREngineID. An arbitrary GUID that could be the CLSID of the engine, for example.

· pSREnginePrivateData. This allows arbitrary engine-specific data to be returned with the phrase object, of size ulSREnginePrivateDataSize.

The words of the result are represented by an array of SPPHRASEELEMENT structures in the pElements field. The rule ulCountOfElements field indicates the number of words. The elements are filled in either directly before the InitFromPhrase call, or afterward with an AddElements call.

Each SPPHRASEELEMENT contains the following information:

· Result times: ulAudioTimeOffset and ulAudioSizeTime and retained audio format details, ulRetainedStreamOffset and ulRetainedSizeBytes. These are all filled in by SAPI and are set to 0.

· Audio stream start position and size for this word: These are relative to the start stream position of the parent phrase. These can be left 0, although the application then cannot obtain word position information.

· Display text, lexical form, and pronunciation information for the word. The display text must be set so that the application displays the result. Optionally, the lexical form and pronunciation can also be set.

· Display attributes information. This is of type SPDISPLAYATTRIBUTES and provides information to SAPI about how to format the result. For European languages, this would usually be set to SPAF_ONE_TRAILING_SPACE so that each word is printed with a space between it.

· RequiredConfidence. This is filled in by SAPI.

· Actual and SR confidence. (See Confidence Scoring and Rejection)

Once the PhraseBuilder has been filled in, it can be passed to SAPI as the pPhrase field in the Recognition call.

Results for proprietary grammars are completed in exactly the same way as for dictation results.

8.3 CFG Phrases

The engine does not directly create a phrase object for a CFG result, but by a call to ParseFromTransitions. The engine provides to this method information about the words in the result, and SAPI parses the active rules to fill in semantic property and other information correctly. The engine then passes the returned phrase builder object to SAPI as the pPhrase pointer in the Recognition method.

· The ParseFromTransitions method takes as a parameter an SPPARSEINFO structure containing the following informationcbSize. Set to size of (SPPARSEINFO).

· hRule. The handle of the top-level rule this result refers to.

· ullAudioStreamPosition and ulAudioSize indicate the position of the result in the audio stream. The position is in bytes, relative to the start of the stream. The part of the stream spanned by this phrase must be the same as or less than the range given in the ullStreamPosStart/End fields in the SPRECORESULTINFO.

· pPath. An array, of size cTransitions, of SPPATHENTRY structures.

· An SREngineID GUID that can be used by the application to identify the engine.

· Optional private engine data pSREnginePrivateData of size ulSREnginePrivateDataSize.

· A flag fHypothesis indicating if the result is to be for a hypothesis or final recognition.

The SPPATHENTRY array contains information about the words in the result. There needs to be an entry for each word transition in the result path. The engine does not include rule or epsilon transitions, as the ParseFromTransitions method is able to process the result without these. Each SPPATHENTRY contains a transition ID for the word transition, and an SPPHRASEELEMENT structure, which is filled in the same way as for dictation results (Note that neither the display text, lexical form nor pronunciation need to be filled in for word transitions. ParseFromTransitions will automatically do this).

There should also be entries for any special transitions:

· For a wildcard transition, the engine sets the transition ID to the value it received for the wildcard transition, and fills in the phrase element information as it would for a normal word.

· For a dictation or text-buffer transition the engine includes an SPPATHENTRY for each word recognized in the dictation or text-buffer. The display text needs to be set to the text for each word. Optionally, the lexical form and pronunciation can be set also.

8.4 Confidence Scoring and Rejection

8.4.1 Word Confidence

It is possible for confidence score information to be included in recognition results. On each phrase element there are two confidence fields that the engine can set. The first, ActualConfidence, is a three level value to indicate low, medium or high confidence (SP_LOW_CONFIDENCE, SP_NORMAL_CONFIDENCE, SP_HIGH_CONFIDENCE). This is designed to give applications a simple, and engine-independent, confidence value. The second value, SREngineConfidence is a positive floating-point value. This can be used by engines to give more detailed confidence information, but is not necessarily engine-independent. SAPI defines that this value should be positive, with Zero indicating the lowest confidence. If engines do not wish to use this field they should set this to –1.0.

8.4.2 Property and Rule Confidence

It is also possible for confidences to be associated with rules and semantic properties in CFG results. This allows an application to, rather than look at confidence on individual words, look at confidence on groups of words. The confidence for a rule is the overall confidence for all the words in the phrase contained within that rule. Thus, for the top-level rule this gives an overall confidence for the whole phrase. The confidence for a semantic property is the confidence for all the words within the rule, if the property is on a rule or rule reference; or the confidence for the word, if the property is on a word transition.

These confidences are found in the SPPHRASERULE and SPPHRASEPROPERTY structures. These have both a Confidence (three-level) field and an SREngineConfidence (floating-point) field. If the engine does not explicitly set any of these values, SAPI will try and produce reasonable default values for them. It will produce the Confidence values by averaging the levels for each of the words in the phrase or property, and it will set the SREngineConfidence values to –1.0.

It is possible for the engine to override these settings if it has an alternative method of estimating phrase confidences. Since these fields cannot be directly manipulated on the ISpPhraseBuilder interface, it is necessary to convert them back to an SPPHRASE structure first. This is done by the following sequence of actions:

· Calling GetPhrase on the SpPhraseBuilder object to get an SPPHRASE.

· Modifying the desired fields in the SPPHRASE (Note this may require casting the fields away from const to do this).

· Calling InitFromPhrase on the original SpPhraseBuilder to set the modified phrase information in the object.

· The engine can then use the SpPhraseBuilder in Recognition calls.

Note this method can also be used to override other information in a CFG result phrase after ParseFromTransitions has been called.

8.4.3 Required Confidence and Rejection

Each transition on a CFG contains a RequiredConfidence field. This is set to one of three values like the ActualConfidence field. This field is set to SP_NORMAL_CONFIDENCE by default, and is changed in the XML grammar by preceding words with “+” or “-”. The purpose of this field is to indicate how much confidence in the recognition an application requires for the result to be returned. In principle, if the engine has a result and the confidence of any of the words in the phrase is lower than the required confidence, then the engine should reject the result. Note that the engine may have a different mechanism for rejecting phrases so SAPI does not enforce this behavior. The engine should, however, take account of the required confidence fields if possible.

It is assumed that most applications will listen for and act only upon final recognitions (SPEI_RECOGNITION events), and probably do not analyze the confidence scores of these results. Thus engines are expected to send these events only after they have determined that the confidence is high enough to accept the result.

When a result is rejected, the engine sends a false recognition rather than a final recognition. A false recognition can include exactly the same phrase information as a final recognition, or it can have a NULL phrase. Returning a phrase with a false recognition could be useful to applications so that they can analyze why the phrase was rejected, and recover the audio from the phrase. Advanced applications may wish to perform their own rejection, and thus could look at both final and false recognition events and analyze the confidence scores returned in each result.

8.4.4 Ambiguous Results

Sometimes the words that have been recognized may match more than one possible path, either with dictation or CFG grammars. SAPI does not currently provide a means to resolve ambiguity or send the result to multiple contexts, so the engine must select one path to return the result as. The guidelines for this are as follows:

· If several CFG paths match the recognized words, set the result to be for the most recently activated rule. This should mean that applications with focus should receive the rule over those in the background.

· If the rule matches both a dictation and a CFG, pick the CFG (unless the path has a very low CFG weight score indicating that the dictation path should be chosen).

· If several dictations are active, pick the most recently activated dictation.

Note that this kind of ambiguity is in general a bad thing, as there is no way for engines or SAPI to determine what the user meant to say. Thus applications are encouraged to try and avoid potentially ambiguous grammars.

8.5 Inverse Text Normalization (ITN)

For dictation results, it is possible for the SR engine to specify a normalized form as well as the raw text of the recognized words. This is done by adding one or more SPPHRASEREPLACEMENT structures into the result phrase, either with the AddReplacements method or by directly setting the pReplacements and cReplacements fields in the SPPHRASE. It is possible to have more than one replacement because each can refer to a sub-set of the full text.

SAPI does not provide an automatic inverse-normalization (ITN) facility. However, it does provide a mechanism for engine vendors to write a grammar describing their ITN rules which can be automatically parsed.

SAPI provides an object SpITNProcessor that implements ISpITNProcessor. The engine can CoCreate this (CLSID_SpITNProcessor), and then call LoadITNGrammar on this object. The engine must pass in the CLSID of an object that implements the ISpCFGInterpreter interface. Engine vendors must implement this object, which has two methods. The first method, InitGrammar, is called by SAPI when the LoadITNGrammar is called. This method should load a SAPI binary grammar containing the ITN information and return it as serialized data.

The ITN grammar that the engine implements should have rules for each of the phrase fragments that need to be normalized. These rules need to have the attributes “TOPLEVEL=ACTIVE” so that SAPI will activate them for parsing, and “INTERPRETER=1” so that SAPI calls the engine’s ISpCFGInterpreter object when the rule is fired.

When the engine has a result phrase it wants to normalize, it should call ITNPhrase on the SpITNProcessor object. This will parse the grammar, and if a fragment of text matches any of the rules in the grammar, the ISpCFGInterpreter::Interpret method will be called. This method will be passed in a phrase containing the result text and matching rule information, and an ISpInterpreterSite pointer. The engine’s implementation of this should look at the reported text and call ISpInterpreterSite::AddTextReplacement to add the normalized text in.

Because the ITN grammar is a standard SAPI grammar, it is possible to use all the features of such a grammar. For example, it may be useful to include the normalized text as a semantic property. This will get included in the phrase passed to the Interpret method so that this method could just set the replacement text to be the property string. For example, the following is a simple grammar rule to convert dollar and pound signs to their respective symbols:

<RULE NAME="currency_type" INTERPRETER="1" TOPLEVEL="ACTIVE">

<L PROPNAME="CURRENCY_TYPE" PROPID="CURRENCY_TYPE">

<P VALSTR="$">dollar</P>

<P VALSTR="$">dollars</P>

<P VALSTR="£">pound</P>

<P VALSTR="£">pounds</P>

</L>

</RULE>

When trying to re-order symbols (e.g., so that the dollar sign comes before the currency amount) more sophisticated processing would have to be done in the Interpret method.

8.6 Interpreters

Sometimes an application may want to do post-processing of CFG results in order to extract semantic information etc. It is possible for a grammar to be associated with an object to allow this to happen. The object would implement ISpCFGInterpreter, and would get called every time a result matching a rule in the grammar was passed into ParseFromTransitions. Then, rather than filling in the property information statically from the grammar, this object would do so with the ISpInterpreterSite::AddProperty method.

This process is basically invisible to the SR engine. It recognizes the words in the grammar and calls ParseFromTransitions and then Recognition. The one difference is that the GetTransitionProperty method may report that there are no properties on a transition, which later are added into the result in the ParseFromTransitions calls. Rules with an associated interpreter have the SPRAF_Interpreter flag set in the SPRULEENTRY Attributes field.

9 Alternates

There are two ways the engine can supply alternates back to the application. It either supplies the alternates directly in the Recognition call, or produces alternates using an alternates analyzer object.

9.1 Returning alternates in a Recognition

The field aPhraseAltsin the SPRECORESULTINFO structure can contain an array of alternates. The size of the array is given by ulNumAlts. Each entry in the array is a SPPHRASEALT structure. This contains an ISpPhraseBuilder structure generated from a ParseFromTransitions or directly depending on whether the alternate is for a CFG or dictation result. This contains the full phrase of the alternate, not just the words that are different to the main result. The ulStartElementInParent and cElementsInParent show which words in the main phrase are different in this alternate, and the cElementsInAlternate shows which words these are being replaced with in this alternate (There does not need to be a ulStartElementInAlternate because it would be the same as ulStartElementInParent). Some engine-specific extra data can be added to the alternate with the pvAltExtra and cbAltExtra fields.

The number of alternates that an application requires is obtained with either the GetMaxAlternates or GetContextMaxAlternates methods. The first gives the maximum number of alternates for any rule, and the latter, the maximum for any RecoContext.

Sometimes when multiple applications are using the shared recognizer and all have rules active, the engine may generate alternates from different applications. SAPI (currently) cannot process such alternate lists and so it is not possible for the engine to send alternates referring to different RecoContexts in the main result. To simplify detecting which context an alternate belongs to, the engine can use the method IsAlternate, giving the handle of the main and alternate top-level rule. SAPI returns S_OK if the alternate is valid and S_FALSE otherwise.

9.2 Alternates Analyzer

The engine can also implement a separate alternates analyzer COM object. This must implement the interface ISpSRAlternates. The CLSID for this object is stored in the engine’s object token in the “AlternatesCLSID” string value. This type of analyzer can only be used for dictation alternates, not for CFG alternates.

When an application asks for alternates with the ISpRecoResult::GetAlternates method, and if the engine has already supplied alternates within the Recognition call, SAPI supplies these to the application. Otherwise, if the engine has an alternates analyzer object, SAPI creates this and requests the alternates from it, using the GetAlternates method.

As well as passing in the main ISpPhrase result and the number of alternates requested, the alternates analyzer is also passed in any extra data the engine supplied in the pvEngineData field of the SPRECORESULTINFO structure in the Recognition call. It is expected that engines will store serialized lattice information or similar information here, which the alternates analyzer can use to generate the alternates.

The analyzer is also passed a pointer to the IspRecoContext, which asks for the alternates. The analyzer can use this to query for an engine-specific extension interface (See Engine extensions) to make a private call to the main engine object (Note: These objects may be in different processes). Because the result object can be serialized and then recreated later, it may not be possible to provide a RecoContext referring to the SR engine. In this case NULL is passed in.

If a user makes a correction after looking at the alternates, the application can commit that alternate with ISpRecoResult::Commit. The Commit method is called on the analyzer, giving the engine the SPPHRASEALT that the application selected. The engine can use this to perform adaptation to hopefully improve subsequent recognition performance.

10 User-Interface

There are a variety of user-interface components an engine may wish to supply along with the main engine object itself. Examples of these are user enrollment, microphone set-up, adding and removing user exception words and engine properties. Engines should not create UIs directly. Instead, SAPI provides mechanisms for engines to describe what UI components they have and to request the display of these components.

The engine’s object token contains the details of the UI that an engine supports. The token can contain a “UI” sub-key. Within this key can be sub-keys for each component type the engine implements. For example the Sample SR engine (Token: “HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Recognizers\Tokens\SAPI5SampleEngine”) has a key “UI\AddRemoveWord”. This contains a value “CLSID”, which holds the CLSID of the class to be created when this UI component is displayed. The engine setup installs and registers this class. This class must implement the interface ISpTokenUI.

An SR engine can request that a certain UI be displayed by calling the AddEvent method with an eEventId of SPEI_REQUEST_UI. The name of the UI component requested is in the lParam field, with the elParamType set to SPET_LPARAM_IS_STRING. To cancel the UI request, the engine calls this method again with a NULL lParam.

This event is sent to all applications connected to the engine that are listening for the event. An application can either ignore the engine’s request, or respond to it. An application can ask to create an engine UI at any time, not just after it has received a request.

An application determines if an engine object token supports a particular UI component by calling IsUISupported, either on the ISpObjectToken itself, or on the ISpRecognizer. If the object token has a CLSID for this type of UI then SAPI will create the object and call ISpTokenUI::IsUISupported on the engine’s UI class. To actually display the UI, the application calls DisplayUI, which leads to ISpTokenUI::DisplayUI being called on the engine’s class.

Both the DisplayUI and IsUISupported methods on ISpTokenUI take an IUnknown parameter punkObject. If an application calls these methods on the IspRecognizer, then this parameter will point to this ISpRecognizer object. This allows the UI component to find the RecoContext and then make a private call to the main SR engine object. If the methods are called by the application directly from the object token, then this parameter may be NULL or point to a different object. If the engine requires its parent SR engine to be created in order to run the UI, then it may not be able to display the UI.

11 Engine extensions

SAPI 5 aims to provide interfaces for all the main things SR engines will wish to do. However there are often additional features that SR engines want to be able to implement when connected to certain applications. As mentioned already, there are a number of places where engine-specific data can be passed to an application. For example, there are engine-specific data fields on results phrases, the SPEI_SR_PRIVATE event, methods to support proprietary grammars, new object token attributes and properties etc. There is also a mechanism for an engine to implement additional interfaces for an application to use if it wants to be specifically tied to a particular engine. This is done by the engine implementing a new COM object. The CLSID for this object is stored in the engine object token, in the string value “RecoExtension”. This object can implement any interfaces that the engine vendor wants to implement. To use this object, an application must QueryInterface on the RecoContext for an interface that it supports. SAPI then creates the engine extension object as an aggregate, and queries it for the interface. The application is then free to call any methods on this interface and the methods will be passed to the extension object. For example, the Sample SR Engine implements the interface ISampleSREngine, which an application can query for from a RecoContext.

The extension object can make calls to its main SR engine class. It does this by querying on the RecoContext for the _ISpPrivateEngineCall interface. The RecoContext is the outer unknown of the extension object. The sample engine does this (in srengext.h) by:

hr = OuterQueryInterface(IID__ISpPrivateEngineCall, (void **)&m_pEngineCall);

Note that this querying for _ISpPrivateEngineCall must be done in the constructor or ATL FinalConstruct method of the engine extension object. It cannot be done later.

This gives a pointer to an _ISpPrivateEngineCall interface. See 11.1 Important notes regarding handling interface pointers in a reco extension. This has two methods, CallEngine and CallEngineEx. Both these methods call the main SR engine object, the first on ISpSREngine::PrivateCall and the second on PrivateCallEx. Both methods allow a serialized chunk of data to be sent to the engine. The difference between these two methods is that CallEngineEx can return a chunk of data back from the engine that is larger than the data passed in.

It is possible for applications to know what engine they are connected to. If the engine is not one that supports the extension interface, the QueryInterface will fail. Alternatively, the application can look at the CLSID of the engine using the ISpRecognizer::GetStatus method. Thus, applications that plan to use special features of a particular engine will be able to detect if they do not have this interface and either perform with limited functionality or fail gracefully. This should allow maximum interoperability between engines, while still allowing applications to take advantage of engine-specific information.

11.1 Important notes about COM interface pointer handling by reco extension aggregates

A reco extension can only query for the IID__ISpPrivateEngineCall during creation of the object (for example, in the FinalConstruct call of an ATL object). The interface is only made visible by the RecoContext during this time. This interface does not use AddRef for the QueryInterface call, so the extension should never call Release on this interface.

Because the reco extension is created as a COM aggregate, if it were to hold a reference to its outer IUnknown interface (the IUnknown of the reco context), it would prevent the context from ever being released. If the extension object calls QueryInterface on the reco context for any interface except for _ISpPrivateEngineCall, then Release() must be called immediately on the outer unknown object to prevent a self-reference. Continue to use this interface even though Release has already been called. This is because the lifetime of the reco context and the extension object are guaranteed to be the same.

The sample engine reco extension shows an example of how to handle these cases in the FinalConstruct method.


SAPI 5 Registry Usage

1 Contents

1 Contents. 1

2 Summary.. 2

3 Overview: Tokens, Categories and the Registry.. 2

3.1 Tokens. 2

3.2 Categories. 5

3.3 TokenIDs and CategoryIDs. 7

3.4 User Defaults. 7

3.5 Token Enumerators. 8

4 Using Tokens and Categories. 9

4.1 Helper Function Examples. 9

4.2 Enumerating tokens. 11

4.3 Instantiating an Object from a Token.. 14

5 Tokens and Categories For Engine Developers. 15

5.1 Making Resources Available Through SAPI15

5.2 Associating Files with Tokens. 15

5.3 Inspecting Underlying Keys of a Token.. 16

5.4 Creating New Keys in the Registry.. 16

6 Registry Settings. 17

6.1 Category: Voices. 18

6.2 Category: Recognizers. 19

6.3 Category: RecoProfiles. 22

6.4 Category: AudioInput. 23

6.5 Category: AudioOutput. 23

6.6 Category: AppLexicons. 23

6.7 Category: PhoneConverters. 23

6.8 UserLexicons. 24

7 Index of Tables. 24

2 Summary

This document is intended to help developers of speech-enabled applications discover and use resources (Voices/Recognizers) on a machine that has SAPI installed. A speech-enabled application is one that attempts to either recognize or synthesize speech.Developers of speech recognition (SR) and speech synthesis (Text to Speech or TTS) engines make their resources available to applications.

This spec answers the following questions:

· What are Tokens and Categories in SAPI?

· Where is information about tokens stored in the Registry?

· How does an application find tokens and initialize resources (i.e., Voices or Recognizers) from them?

· What are the SAPI-defined attributes that engines should document in the registry?

· How are files associated with tokens?

Note:

The Speech SDK documentation section on Object Tokens, which provides a complete description of the ISpObjectToken and ISpObjectTokenCategory and their methods, complements this document.

3 Overview: Tokens, Categories and the Registry

3.1 Tokens

A token is an object representing a resource that is available on a machine, such as a voice, recognizer, or an audio input device. A token provides an application an easy way to inspect the various attributes of a resource without having to instantiate it. The Vendor of a Recognizer, and Gender of a Voice are examples of attributes of resources. In many cases, applications should use SAPI-provided helper functions for common scenarios. For example, an application can use the SpCreateBestObject helper function to rapidly create the object, given a certain type of resource. The application can also query for tokens meeting certain criteria without using the helper function. To do this, the application calls the EnumTokens method on the ISpObjectTokenCategory interface to get an enumerator, and inspect the tokens in the enumerator further if it chooses to. Finally, the application selects one of the tokens in the enumerator to instantiate a resource. Once the resource (such as SR Engine) is instantiated, if it implements the ISpObjectWithToken interface, then it is handed a pointer to the token that was used to create it. This way, the resource contains a handle to more information about itself.

Conceptually, a token contains the following information:

· The language-independent name. This is the name that should be displayed wherever the name of the token is displayed. It is marked as (Default) in the registry. The implementer of the token may also choose to provide a set of language-dependent names in several languages.

· The CLSID used to instantiate the object from the token.

· A set ofAttributes, which are the only set of queriable values in a token. This means SAPI provides a mechanism to query for tokens whose attributes match certain values. Details on how to query for tokens that match a set of attributes are in Sections 4.1 and 4.2.

A token may also contain the following:

· If a token has user interfaces (UIs), such as the properties of a Recognizer or a wizard to customize a Voice to display, then the token will also contain the CLSID for the COM object used to instantiate each type of UI.

· The set of Files from which SAPI returns the paths to all the associated files for the token.

SAPI stores information about tokens in the registry. A token is represented in the registry by a key and the key’s underlying keys and values. When an application queries SAPI for tokens of all the female voices on the machine, SAPI will look at the HKEY_LOCAL_MACHINE\Software\Microsoft\Speech\Voices area. This corresponds to a Category and categories are discussed in the Section 3.2. SAPI searches for tokens that match the criteria (in this case, a voice with the Gender attribute set to female) and uses one of these matching tokens to initialize the voice. The application may also specify a different fully qualified registry path to specify any non-standard (from a SAPI) location in the registry for SAPI to search for a token. In addition to the keys SAPI recommends, the entry for the token may contain any other bits of information that the implementer of the token can store here. In the registry, a token looks like this:

Table 1 Parts of a Token in the Registry

RegKey

ValueName

Sample Value

Comments

SampleTokenKey

Required - This is the RegKey for the Token.

(Default)

Joe

Required - Language Independent Name.

409

Joe

Name in Hex LangID 409, which is English. There may be several of these rows, one for each LangID in which the Token has a name. Note, no leading 0x before the LangID.

809

Joe

CLSID

{8021D50E-D93C-4075-8504-FC4E124D64E9}

Required - Sample CLSID for object which instantiates the token.

SampleTokenKey/Attributes

Attributes for the token are under this key.

Language

409;809

There may be several of these rows, one for each attribute that is queriable. See Section 4 for an explanation of each of the attributes.

Vendor

VoiceVendor

In the registry, this looks like:

Figure 1 A Token Key in the Registry

The Attributes key contains all the queriable values for the token. Section 4.2 discusses in detail how an application queries a token.

Figure 2 Attributes of a Token

If the token is capable of displaying UI, then each UI has its own key under the token. Fig 3 shows the token for a Recognizer that supports four types of UI: AddWord, EngineProperties, MicTraining and UserTraining, as well as the CLSID underlying each UI type.

Figure 3 A Token that supports UI has a token for each UI type

SAPI provides a comprehensive set of helper functions for the common scenarios using tokens. Section 4.1 provides a number of examples. SAPI also provides a way for engines and applications to implement tokens in their own proprietary manner. See Section 3.4 on token enumerators, for further discussion. Sections 4 and 5 explore common scenarios using these interfaces from application and engine coding perspectives.

3.2 Categories

A ObjectTokenCategory (hereafter referred to as category) is the highest level of grouping of registry entries in SAPI. A category is a class of tokens (or of resources, since each token represents an actual resource on the machine). Intuitively, a category is a type of SAPI resource. It is represented in the registry by a key containing one or more token keys under it. It is created and manipulated using helper functions such as SpCreateDefaultObjectFromCategoryID or methods on the ISpObjectTokenCategory interface. Please refer to the SAPI documentation for details on either of these. Examples of categories are Recognizers and Voices. Figure 4 shows the default SAPI categories, with the category Voices selected.

Figure 4 The Category Voices

SAPI organizes tokens in the Registry under seven categories.

By default, the following tokens for six of the SAPI categories are located under HKEY_LOCAL_MACHINE\Software\Microsoft\Speech (HKLMS). This is where all system-specific SAPI keys and values should be stored as recommended by Windows Application guidelines. Examples include settings and files for Voices and the Recognizers ( also known as Speech Recognition engines) installed on a machine, as shown in Figure 1.

1. Voices

2. Recognizers

3. AppLexicons

4. AudioInput

5. AudioOutput

6. PhoneConverter

The tokens for the other category, Recoprofiles, are located under HKEY_CURENT_USER\Software\Microsoft\Speech (HKCUS).HKCUS also contains all other user-specific keys and values in the registry, such as user defaults for Voices, Recognizers, as well the location of the user lexicon file.

Categories contain the following items:

· A single key called Tokens, and the keys for the tokens that belong to that category under it. For example, the Voices category has a key for the voice called Manuel. All the keys and values for Manuel are located under HKLMS/Tokens/Manuel.

· Keys for token enumerators. A token enumerator is a special type of token that generates other tokens for the same category. This token provides a way for Vendors to generate tokens that are generated in non-standard way, such as, reading data from a stored file stored. Those engine vendors following SAPI guidelines for registering resources (Sections 4 and 5) can safely ignore these and regard them as generators for another set of tokens. Section 3.4 explains token enumerators in more detail.

3.3 TokenIDs and CategoryIDs

A CategoryID uniquely identifies a category in the registry. For SAPI defined categories they take the form of HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\{CategoryName}. For example, HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Recognizers\ for the Recognizers category. All SAPI CategoryIDs should be referenced using the constants defined in sapi.idl file:

1. SPCAT_AUDIOOUT

2. SPCAT_AUDIOIN

3. SPCAT_VOICES

4. SPCAT_RECOGNIZERS

5. SPCAT_APPLEXICONS

6. SPCAT_PHONECONVERTERS

7. SPCAT_RECOPROFILES

Similarly, TokenIDs uniquely identify tokens in the registry. For tokens located in SAPI defined categories, they take the form of:

· CATID\Tokens\TokenKeyName - a static token from the registry. For example, HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Recognizers\MSASREnglish

· CATID\TokenEnums\TokenEnumKeyName - a static token from the registry that represents a token enumerator. This token instantiates a token enumerator used to enumerate dynamic tokens. SAPI uses this for its own implementation of audio input and output to list just the channels available on the computer at runtime. Token enumerators can also read tokens from other areas of the registry, or from remote computers. For example, HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\AudioOutput\DSoundAudioIn

· CATID\TokenEnums\TokenEnumKeyName\ - a dynamic token representing the default token that the specified token enumerator generates. For example, SPDSOUND_AUDIO_IN_TOKEN_ID creates the default Dsound audio in an object. For example: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\AudioOutput\DSoundAudioIn\

· CATID\TokenEnums\TokenEnumKeyNameEnumExtra… - a specific dynamic token from the specified token enumerator. For example: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\AudioOutput\DSoundAudioIn\Direct Sound Crystal WDM Audio, which generates the Direct Sound Crystal WDM audio object.

3.4 User Defaults

In addition to the category defaults mentioned in Section 3.2, the categories Voices, Recognizers, AudioInput, AudioOutput and RecoProfile, also have user defaults and settings. As shown in Figure 5, these are located in the HKCUS area, under their respective category keys. Section 6 explains each category of tokens. This section also lists out the user-specific entries in the HKCUS and the system-wide entries in HKLMS.

Figure 5 The User category for Recognizers

3.5 Token Enumerators

Note: This section is relevant only forEngine or Application developers who need to store tokens in a separate part of the registry or even on the file system, and dynamically enumerate them.

SAPI provides a way for third parties to store their registry settings without following any of the SAPI-recommended guidelines. SAPI can find these tokens as long as the parties have implemented token enumerators. Token enumerators are COM objects that enumerate the necessary entries for the tokens under it. All token enumerators are stored under CategoryName/TokenEnums. Each token enumerator listed under a category needs to have the CLSID of the COM object that implements it under the token enumerator.

The token enumerator

· Must implement the methods Next, Skip, Reset, Clone, Item, GetCount on the IEnumSpObjectToken interface.

· May choose to implement methods SetObjectToken and GetObjectToken on ISpObjectWithToken interface. As mentioned in the end of Section 3.1, these give a resource a handle to the token that was used to instantiate it.

These tokens can be located in a separate part of the registry or somewhere else (possibly on the flusters). It is the responsibility of the token enumerator to return correctly on the above methods so an application does not know the difference between tokens coming from the token enumerator and tokens coming from the SAPI-specified part of the registry.

SAPI itself uses token enumerators only for the AudioInput and AudioOutput categories. Refer to Sections 6.4 and 6.5 for more details. Note that the token enumerator for the MMSYS audio object creates its tokens from keys that are under it.

The following is an example of what a TokenID for a token located under a token enumerator looks like: CategoryName/TokenEnums/TE1/XXX where (i) TE1 is a sample token enumerator and (ii) XXX is a reference to one of the tokens generated by TE1. On a call to the helper function SpCreateCreateNewToken giventhe TokenID above, the IEnumSpObjectToken returned by the token enumerator TE1 to SAPI includes all tokens. SAPI then goes through each token (those returned by token enumerators and those under the tokens key) to find the one that has a Token name matching XXX.

Table 2 Parts of the AudioInput token enumerator

RegKey

ValueName

Sample Value

Comments

HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\AudioInput\

This is the category.

DefaultDefaultTokenID

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\AudioInput\TokenEnums\MMAudioIn\

This is the TokenID for the default token for this category. If the DefaultTokenID is present, it will supercede this default token for the category. Details in section 4.2

HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\AudioInput\TokenEnums

HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\AudioInput\TokenEnums\MMSys

This is the MMSys token enumerator

CLSID

{GUID}

This is the CLSID for the COM object that implements the MMSound token enumerator.

Figure 6 AudioInput token enumerator in the registry

Figure 6 illustrates how the AudioInput token enumerator looks in the registry.

4 Using Tokens and Categories

4.1 Helper Function Examples

A SAPI 5 application needs to find tokens and instantiate objects that meet certain criteria from the resources available on a machine. Helper functions distributed in the sphelper.h file are the recommended way for applications to interact with tokens and categories whenever possible. Table 3 provides a list of helper functions and the scenarios they address. The helper functions have been broken up into Common Helper Functions and Engine Developer Helper Functions based on likelihood of use. If the specific helper function is not found in either section, refer to the SAPI documentation for the comprehensive listing.

Table 3 Common Helper Functions

Helper Function | Action | Example Helper Function Call
SpGetDefaultTokenFromCategoryID | Creates the default token from a CategoryID. The last argument tells SAPI to create the token if it does not currently exist. | CcomPtr<ISpObjectToken> m_cpEngineToken; hr = SpGetDefaultTokenFromCategoryId(SPCAT_RECOGNIZERS, &m_cpEngineToken);
SpFindBestToken | Finds the most appropriate token given a set of required and optional criteria. For details on attribute matching see Section 4.2 | CComPtr<ISpObjectToken> cpTokenEng; hr = SpFindBestToken(SPCAT_RECOGNIZERS, L"Language=409", L"VendorPreferred", &cpTokenEng);
SpEnumTokens | Returns a token enumerator containing all tokens meeting a set of required and optional attributes. Tokens in the enumerator are sorted in the order specified in the Section 4.2. | CcomPtr<IEnumSpObjectTokens> cpIEnum; hr = SpEnumTokens(SPCAT_VOICES, L"Gender=Female;Language=409", L"Vendor=VoiceVendor1;Age=Child” , &pEnum);
SpCreateDefaultObjectFromCategoryID | Creates the default object in a category, such as AudioInput or Recognizer | CComPtr<ISpVoice> cpVoice; SpCreateDefaultObjectFromCategoryID(SPCAT_VOICES, &cpVoice);
SpCreateBestObject | Instantiates a resource that best matches a set of required and optional criteria. For details on attribute matching see Section 4.2 | CComPtr<ISpVoice> cpVoice; SpCreateBestObject(SPCAT_VOICES, L"Vendor=VoiceVendor1;Age=Child", L”Gender=Female”, &cpVoice);
SpCreateObjectFromToken | Creates an object from a token. | CComPtr<ISpVoice> cpVoice; CComPtr<ISpObjectToken> cpVoiceToken; //--like last step SpFindBestToken(SPCAT_VOICES, L"Language=409", L"VendorPreferred", &cpVoiceToken); /--now create object SpCreateObjectFromToken(cpVoiceToken, &cpVoice); }

Table 4 Engine Developer Helper Functions

Helper Function | Action | Example Helper Function Call
SpCreateNewToken | Creates a new object token in the registry with CategoryID, but without specifying a keyname. This creates a token with a GUID as its registry key. | CComPtr<ISpObjectToken> cpUserToken; hr = SpCreateNewToken(SPCAT_RECOPROFILES, L"", &cpUserToken); cpUserToken;
SpGetTokenFromID | Creates a token from a TokenID of an enumerator or a new token if the token does not already exist. The last argument of FALSE tells SAPI not to create the token if it doesn’t already exist. | CComPtr<ISpObjectToken> cpAudioInTok; hr = SpGetTokenFromID(SPCAT_AUDIOIN, &cpAudioInTok, FALSE)))
SpCreateObjectFromSubToken | Creates an object from a subtoken of a token. In this case, the engine token pEngineToken has the Lts key under it, which in turn has a CLSID value under it. This CLSID is used to instantiate the object. | CComPtr<ISpObjectToken> m_cpEngineToken; hr = SpGetDefaultTokenFromCategoryId(SPCAT_RECOGNIZERS, &m_cpEngineToken); ISpLexicon * m_pLtsLex; HRESULT hr = SpCreateObjectFromSubToken(pEngineToken, L"Lts", &m_pLtsLex);
SpGetSubTokenFromToken | Creates a subtoken under a token. This is useful, for example, when an Engine vendor would like to create a subtoken for custom data under its Recognizer token. | CComPtr<ISpObjectToken> cpSubSubToken; hr = SpGetSubTokenFromToken(&m_cpEngineToken, L"EngineProperties", &cpSubSubToken, TRUE );

4.2 Enumerating tokens

The principal tasks related to tokens and categories that an application needs to accomplish are:

· Enumerating tokens

· Inspecting and instantiating tokens

The two primary ways to enumerate tokens are by the helper function SpEnumTokens, or by the method ISpObjectTokenCategory::EnumTokens. Both methods allow the caller to specify a category and a set of required and optional attributes. The call then returns a token enumerator containing all the tokens matching those criteria. The method is defined as:

HRESULT EnumTokens(

        [in] const WCHAR *pszCatName,

        [in, string] const WCHAR *pReqAttrs,

        [in, string] const WCHAR *pOptAttrs,

        [out] IEnumSpObjectTokens **ppEnum);

When identifying matching tokens under in a category, an application needs to specify a fully qualified category identifier (FQCID). An FQCID is the full registry path to a category, such as H KEY_CURRENT_USER\Software\Microsoft\Speech\Voices. It is recommended that these categories be referenced using the constants defined in the sapi.idl file below, and not using the full string to minimize typos in commonly used registry paths. SAPI maps the constant to the correct hive in the registry and returns matching tokens from the category. For instance, the SAPI defined AudioInput constant (from the sapi.idl file) is:

//--- Categories for speech resource management

const WCHAR SPCAT_AUDIOOUT[]    = L"HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Speech\\AudioOutput";

Similarly, there are constants for the AudioInput, Voices, Recognizer, Applexicon, PhoneConverter, and RecoProfile categories.

An application may also specify a non-standard registry location by simply providing its FQCID, such as HKEY_CURRENT_USER\Software\TTSVendor1\Speech\Voices

In both SpEnumTokens and ISpObjectTokenCategory::EnumTokens the following clauses are permitted in the ReqAttrs and OptAttrs strings, separated by semicolons.

Table 5 Query Operators

Condition | Example | Explanation
Exists | Telephony;Dictation | The valuenames Name and Dictation exist in the list of attributes for this token.
One of | Language=409 | At least one of the values of the Valuename Language is 409. There may be other values, like 809, 512 as well.
Not Equals | Age!=Child;Age!=Teen | Values of Age that are neither “Child” nor “Teen”.

The tokens are sorted “best matches” first using the following intuitive rules:

1. Only tokens matching the required attributes are returned.

2. Those tokens matching the optional attributes as well will be before those that just match the required attributes.

3. If there are no required or optional attributes (i.e., both are set to NULL), the first token is the default token for that category. If there is a valid DefaultTokenID in HKLMS/Category, that is returned as the default tokenID. If not, if there is a default tokenID in HKCUS/Category, that is returned. If none of these exist, SAPI searches for a DefaultdefaultTokenID in HKLMS/CategoryName, and that is returned.

4. Matching Rules: If a token matches an optional attribute, it gets a score of 1, otherwise, 0 for that attribute. The optional attributes mentioned earlier in the query string are more significant. These scores are concatenated as shown in Table 7. The tokens are then placed in descending order. This is illustrated in Tables 6 and 7.

5. Tokens having the same score are returned in random order in the enumerator.

A call to EnumTokens could look like:

CComPtr<IEnumSpObjectTokens>  cpEnum;

CComPtr<ISpObjectTokenCategory> cpVoiceCat;

HRESULT hr = cpTokenCategory.CoCreateInstance(CLSID_SpObjectTokenCategory);

const WCHAR Req_Attrs[ ]=L"LanguagesSupported=409";

const WCHAR Opt_Attrs[]=L"Vendor=VoiceVendor1;Age=Child;Gender=Female”;

HRESULT hr = cpVoiceCat->EnumTokens(SPCAT_VOICES, ReqAttrs , OptAttrs , &cpEnum);

// SPCAT_VOICES is defined in sapi.idl

If the following voices are installed on a machine as shown in Table 6:

Table 6 Voices installed on a machine

Voice | Vendor | Age | LanguagesSupported | Gender
Michelle | VoiceVendor1 | Child | 409; 411 | Female
Mary | VoiceVendor1 | Adult | 409 | Female
Jane | VoiceVendor2 | Child | 409 | Female
Frank | VoiceVendor2 | Adult | 411 | Male
Anna | VoiceVendor2 | Adult | 411 | Female

Then the order of the Voices returned in cpEnum will be as shown in Table 7:

Table 7 Scoring of tokens matching optional criteria

Optional Criteria -> | Vendor | Age | Gender | Net Score
Michelle | 1 | 1 | 1 | 111
Mary | 1 | 0 | 1 | 101
Jane | 0 | 1 | 1 | 011

The final order is:

1. Michelle (meets all required criteria, scored 111 on optional criteria)

2. Mary (meets all required criteria, scores 101 on optional criteria)

3. Jane (meets only required criteria, score 11 optional criteria)

If the call to EnumTokens is changed to:

HRESULT hr = cpVoiceCat->EnumTokens(SPCAT_VOICES, NULL, NULL, &cpEnum);

and the users default token in HKCUS\Voices\DefaultTokenID is set to: HKEY_LOCAL_MACHINE\Software\Microsoft\Speech\Voices\Tokens\Jane

then the enumerator cpEnum will contain all the tokens, with Jane being the first token.

What does SAPI do when ISpObjectTokenCategory::EnumTokens is called?

Consider a fictitious category that has both tokens and token enumerators under it. When an application calls the SAPI ISpObjectTokenCategory::EnumTokens, the following things happen:

1. SAPI creates an enumerator called IEnumSpObjectTokens that can enumerate all the matching tokens from these keys under HKLMS/Voices/Tokens.

2. Token enumerators Step (skip this step if not using token enumerators).

a. SAPI searches for a CategoryName/TokenEnums key. If found, it instantiates a token enumerator from each of the tokens under this key, one by one.

b. Each of the token enumerators return an IEnumSpObjectToken containing matching tokens under it that is merged with the IEnumSpObjectToken created in (i).

3. SAPI applies the required attributes so that the IEnumSpObject enumerator contains only those tokens that match these Attributes, then it sorts them according to how well they match the optional attributes (exact rules earlier in Section 4.2).

4. The application searches for an appropriate token and until one is found, it steps through each token, and further checks attributes and strings of each token with ISpObjectToken methods GetData, GetStringValue, and GetDWORD (inherited from ISpDataKey).

5. The application identifies the token it is interested in and calls ISpObjectToken::CreateInstance and QIs the newly created object to see if the newly created object supports the ISpObjectWithToken interface. If it does, SAPI calls ISpObjectWithToken::SetDataKey to give the newly instantiated object a pointer to the token it was instantiated from.

4.3 Instantiating an Object from a Token

Continuing with this example, the application now has a pointer to the enumerator IEnumSpObjectTokens. An application may choose to step through the enumerator with the methods Next, Skip or Reset to find an ISpObjectToken that best meets its needs. Assume that the application is searching for a voice that sounds clear over a telephone. Also assume that such voices typically have a ValueName called SupportsTelephony, which is set to 1. There is no such protocol in SAPI; this is for illustration only. Because this is not a value under Attributes, it cannot be picked up by the standard query mechanism of required attributes. The variable pCurVoiceToken represents a token for that category. In the example below, the category is populated with tokens in cpEnum until a voice is found that also supports Telephony.

ISpObjectToken         *pCurVoiceToken;

bool             fFeature = false;

while (cpEnum->Next(1, &pToken, NULL) = S_OK)

    {

// At this point, all we know is that pToken is a pointer to a Voice token.

      hr = pToken->GetData(L"SupportsTelephony", fFeature);

// Note, ISpObjectToken inherits from ISpDataKey

      if (( SUCCEEDED( hr ) ) && fFeature )

      {

        // this is the token for the Voice we want

        pCurVoiceToken =pToken;

        break;

      }

    }

At this point, store the selected Voice token in pCurVoiceToken. Now create the voice object from this token, so that Speak and other methods on it may be called. To create a voice object, ISpVoice must be created.

EXTERN_C const CLSID CLSID_SpVoice;

CComPtr<ISpVoice>      cpVoice;

// The Application may want to check to see if the token has any associated UI that it needs to display

hr = pCurVoiceToken->IsUISupported(SPDUI_EngineProperties, NULL, 0, NULL, &fSupported);

// The Application calls the UI, or maybe enables a button in its own UI so the user can call the UI

// Next, CoCreate an instance of SpVoice called cpVoice

hr = cpVoice.CoCreateInstance(CLSID_SpVoice);

    if( SUCCEEDED( hr ) )

    {

      // set cpVoice to our selected voice token

      hr = cpVoice->SetVoice(pCurVoiceToken);

    }

At this point the cpVoice object (of type ISpVoice) has been instantiated and is ready to speak, with a call such as:

hr = cpVoice->Speak( L"This audio file was created using SAPI five text to speech.", 0, NULL);

5 Tokens and Categories For Engine Developers

In addition to the enumerating and instantiating tokens, an engine vendor also needs to be able to:

· Create new tokens

· Associate files with tokens

5.1 Making Resources Available Through SAPI

There are several straightforward steps for an SR or TTS engine to be discoverable by SAPI:

1. Make an appropriate entry under the correct CategoryID/Tokens in the registry (details in Section 6).

2. Make an entry under CategoryID/TokenEnums if the vendor prefers dynamic tokens (i.e., the engine registry information is already stored in some other registry location or file). The enumerator should implement the interfaces outlined in Section 3.4.

3. Look at the standard attributes for a category in SAPI and identify the characteristics of the engines so that applications can query the engine for these properties.

4. Hand the SR engine a pointer to the recognition profile token once the RecoInstance has been created.

5.2 Associating Files with Tokens

One of the key issues for an engine vendor is to associate files with tokens in the registry, such as the language model files for a Recognizer or a RecoProfile token. A token can query for all the files under its Files key using the ISpObjectToken::GetStorageFileName method. SAPI searches for the file in a number of known locations. Because of the possibility of roaming, SAPI does not store fully qualified file paths in the registry (such as C:/Documents and Settings/JoeUser/Local Settings/Application Data), but stores paths such as %1c%\Microsoft\Speech\Files\MSASR\SP_81738BE4B81F42F0BFC4BB98B72EB81A.spz instead. SAPI queries the ShGetFolderPath .dll for the user’s non-roaming directory on the individual machines. The calling application can specify (i) the specific name of the file if any, and (ii) the subdirectory to put the file in. Refer to the GetStorageFileName documentation for the exact interfaces. The engine may append any additional vendor-identifying directory names to indicate engine-specific data. Deleting the tokens with which the files are associated by calling ISpObjectToken::RemoveStorageFileName, will remove files from the filesystem as well.

Caveat: If roaming is enabled, the user’s RecoProfiles in the HKCUS hive of the registry will roam (because the entire HKCUS hive roams); the associated files, situated in a non-roaming directory will not. This causes two unexpected effects:

1. When the Recognizer is initiated on the second machine, the Recoprofiles are likely to be missing. The Recognizer needs to be able to handle this and copy the necessary new-profile files. Known issue: Upon roaming the Microsoft SR Engine currently creates a new set of files, but these have entirely different names from the names on the original machine. As a result, when the registry is roamed back to the original machine, the original profile files become orphaned.

2. Subsequently, upon deleting the Recoprofiles from one machine, all the associated files and registry entries on that machine will be deleted. The rest will become orphans, that is, files without pointers to them.

5.3 Inspecting Underlying Keys of a Token

Besides helper functions, keys under a token can be inspected using a recognizer token, and opening the attributes key under it as a DataKey. Then all the ISpDataKey methods are available to inspect the values under the Attributes key. The sample below goes from the Recognizer token, to the attributes key under it, and finally to the “Desktop” and “Telephony” strings under that.

hr = SpGenericSetObjectToken(pToken, m_cpEngineObjectToken);

  if(FAILED(hr))

  {

    return hr;

  }

  // Read attribute information

  CComPtr<ISpDataKey> cpAttribKey;

  hr = pToken->OpenKey(L"Attributes", &cpAttribKey);

  if(SUCCEEDED(hr))

  {

    WCHAR *psz = NULL;

    hr = cpAttribKey->GetStringValue(L"Desktop", &psz);

::CoTaskMemFree(psz);

    if(SUCCEEDED(hr))

    {

      // This instance of the engine is for doing desktop recognition

    }

    else if(hr = SPERR_NOT_FOUND)

    {

      hr = cpAttribKey->GetStringValue(L"Telephony", &psz);

      ::CoTaskMemFree(psz);

      if(SUCCEEDED(hr))

      {

        // This instance of the engine is for doing telephony recognition

      }

    }

  }

5.4 Creating New Keys in the Registry

Below is another snippet of code where the Microsoft Sample Engine creates a new entry under a recognition profile. If the Recognition Profile does not exist for the engine (pszCLSID contains a pointer to the Engine GUID), it needs to be created it as well as the Gender and Age values under it.

// Read attribute information from Engine key;pProfile is the RecoProfile token we obtain by calling GetRecoProfile on the Recognition Instance.

hr = pProfile->OpenKey(pszCLSID, &dataKey);

  if(hr = SPERR_NOT_FOUND)

  {

    // This user profile has not been seen before, so create a new registry key to hold info for it

    hr = pProfile->CreateKey(pszCLSID, &dataKey);

    // Now set some default values

    if(SUCCEEDED(hr))

    {

      hr = dataKey->SetStringValue(L"GENDER", L"UNKNOWN");

    }

    if(SUCCEEDED(hr))

    {

      hr = dataKey->SetStringValue(L"AGE", L"UNKNOWN");

    }

    // Now create some temporary file storage for trained models

    // this will create a valuename called SampleEngTrainingFiles and value C:\Documents and Settings\username\application data\microsoft

speech\files\MSASR\LM7454901D23334AAF87707147726EC235.dat

    if(SUCCEEDED(hr))

    {      hr = pProfile->GetStorageFileName(CLSID_SampleSREngine, L"SampleEngTrainingFile", "MSASR\LM%d.dat", CSIDL_FLAG_CREATE, &pszPath);

    }

    // and request a UI for user training or properties - SPDUI_RecoProfileProperties

    hr = AddEventString(SPEI_REQUEST_UI, 0, SPDUI_UserTraining);

6 Registry Settings

This section documents in some detail, the registry settings of each category of tokens in both the HKCUS and the HKLMS hives. Each token entry needs to have the required keys and values for a token as outlined in Table 1. To find the most suitable token on the machine for the Recognizers, Voices, and Phone Converters categories of tokens, an application needs to define a standard set of attributes that applications can query for. It is important for engine vendors to implement these keys exactly as specified because the engines/voices must be discoverable through SAPI to applications.

It is important to note that in addition to the specified keys and values, a vendor may create any keys and values necessary to use as a resource in the registry. SAPI will ignore these values and not disturb them in any way, unless SAPI is uninstalled from the machine.

6.1 Category: Voices

The Voices category enumerates every voice installed on the machine by all TTS engines. The voice tokens should be located under the key:

HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\Voices\Tokens

The requirements for a voice token are listed below, with some sample values.

· Each voice token should meet the requirements for a standard token.

· Voices should document the SAPI-specified attributes that describe them so applications can search for them. Table 8 contains a full listing of Voices attributes and their locations. All Voice attributes are required. Section 3.1 and Section 4.2 have more information about attributes and querying them.

· Voices may have their own Vendor-specific UI implemented by the TTS Engine rendering the voice. If such UI is present, then the UI needs a separate token in the location described in Table 8. The minimum requirement is that the token contain the CLSID of the COM object implementing the UI. Click Properties on the Text-to-Speech tab of the Speech Control Panel to access the Vendor-specific UI. The Properties button will be unavailable if the EngineProperties token for the current default Voice is not supported

Table 8 provides a detailed listing of the registry entries that constitute a sample voice token called VoiceToken1 under HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\Voices\Tokens

Table 8 Voice Registry and Attributes

RegKey

ValueName

Comments

VoiceToken1

Required - This is the RegKey for the Token.

(Default)

Required - language independent name.

409

Name in Hex_LangID 409, which is English. There may be several of these rows, one for each LangID in which the token has a name. Note, no leading 0x before the LangID

809

CLSID

Required - Sample CLSID for object which instantiates the voice.

VoiceToken1/Attributes

Attributes for the Token are under this key.

Age

Required - Value should be “Child,” “Teen,” “Adult,” or “Senior” depending on Age of TTS Voice. Senior indicates an elderly voice. Vendors may choose to classify some voices as both “Senior” and “Adult”.

Vendor

Required – TTS engine Vendor name.

Language

Required - The LCID in hex of language this engine speaks.

Gender

Required - Value should be “Male” if Male voice, “Female” if female.

VendorPreferred

Required - If this is the Default voice for the vendor named in vendor.

Name

Required - String representing language independent name

VoiceToken1\UI

Required, if the Voice has UI - UI tokens for the voice token will be stored under this key.

VoiceToken1\UI\EngineProperties

The only SAPI-specified UI token is EngineProperties. Called when the user clicks Properties on the Text-to-speech tab.

CLSID

Required - Sample CLSID for object which instantiates engine-specific UI from the Speech Control Panel.

Note: Please refer to the registry entries of the Microsoft recognizer and the Sample Engine, which ship in the SAPI 5 SDK, as an example of how the are entries are created.

There is also a Voices category in the HKCUS hive that stores the following:

· The default TTS rate selected by the user in the Speech Control Panel.

· The default voice selected by the user.

Table 9 provides a listing of the user registry entries that constitute a voice token in

HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\Speech\Voices\

Table 9 Voices - User Registry Settings

ValueName

Value

Comments

VoiceToken1

DefaultTokenID

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Voices\Tokens\MSMary

This points to the default voice selected by the user in the Speech Control Panel. It is empty at installation time and gets populated when the user selects a voice.

TTSRate

5

{Scale is 0 to +10, with 0 being slowest}. This applies to all voices on the system for the currently logged on user.

Note: The TTS engine does not need to store any of these values- SAPI takes care of that.

Vendors may choose to store any additional keys and values in the same areas of the registry. Following is additional information relating to voice tokens:

· User specific entries for the voice (such as volume, pitch, rate, and any other information) should be stored in keys and values under HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\Speech\Voices\Tokens\VoiceToken1

This creates a structure in the HKCUS hive parallel to the one in the HKLMS hive.

· Entries applying to all the voices using an engine should be stored under HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\Speech\Voices\Tokens\EngineGUID1

· Non-user entries (pertaining to all users on the machine) for a voice should be stored in keys and values under the category HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\Voices\Tokens

6.2 Category: Recognizers

SAPI enumerates all the SR Engines installed on the machine from the tokens and token enumerators under

HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\Recognizers

Below are the guidelines for registering recognizer tokens:

· Each recognizer token should meet the requirements for a standard token (Table 1).

· Each speech recognition engine installed on the machine should have a recognizer token. If vendors use a single recognizer for recognition in multiple languages (with different acoustic models), or a discrete and a continuous recognizer, they may choose to store the relevant data files and other initialization information under separate tokens, but use the same value for the CLSID. For example, a vendor may use the same recognition engine to recognize both Japanese and English. In this case, there are two tokens, both containing the CLSID of the same recognizer, but associated with different language and acoustic model files stored with the token.

· Recognizers should document the SAPI-specific attributes shown in Table 10 so that applications can search for them. Required attributes are also indicated Table 10.

· Most speech applications written with SAPI will be tested for a specific engine, or a few specific engines, if the application has a clear need for multiple engines. Typically applications will query for and use this engine by default. Use attributes when the application cannot find its preferred engine (or doesn’t have one), and needs to locate the most suitable engine installed on the machine for its needs.

· Recognizer tokens may have an Alternate CLSID if they implement alternates.

· Recognizer tokens may have a RecoExtension CLSID for objects that extend SAPI’s recognition context.

· The Recognizer may also have a number of engine-specific UIs that it exposes to SAPI. There should be a separate key under {Recognizer TokenID}/UI/ for each such UI supported. The keys are listed and documented in Table 10 below.

Table 10 provides a detailed listing of the registry entries that constitute a sample voice token called VoiceToken1 under HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\Recognizers\Tokens

Table 10 Sample Entries of a Recognizer token

RegKey

ValueName

Comments

RecognizerToken1

Required - This is the RegKey for the Token.

(Default)

Required - Language Independent Name

409

Name in Hex_LangID 409, which is English There may be several of these rows, one for each LangID in which the token has a name. Note, no leading 0x before the LangID

AlternatesCLSID

CLSID for the object that implements alternates.

RecoExtension

CLSID for the object that extends the recognition context provided by SAPI.

RecognizerToken1\Attributes

Vendor

Required - Company Name

Language

Required - The LCIDs in hex of the language(s) this engine recognizes. Typically this is only one per recognizer token. Recognizers recognizing multiple languages can expose that in the manner shown. In addition, a value of “409;9” indicates that the Recognizer recognizes generic English (American, British, Australian, etc.).

SpeakingStyle

Required - Value should be “Discrete” if the engine requires pauses between words for recognition. Recognizers without this requirement should have “Discrete;Continuous” for this value.

Dictation

Required - If the engine supports dictation, it must contain this value.

CommandAnd Control

Required - If the engine supports command and control. This value must be there if the engine supports Command and Control.

Desktop

The engine supports desktop audio.

Telephony

The engine is configured to recognize audio coming in from a telephony channel.

VendorPreferred

This token is the vendor’s default token.

Alternates

Value is “CC” if the engine supports Command and Control alternates. Value is “Dictation” if it supports dictation alternates. If both types of alternates are supported, then the value should be “CC;Dictation”

Hypotheses

The engine returns hypotheses before final recognition.

WordSequences

Value is “Trailing” if the engine supports word sequence elements in CFGs, at the end of rule. Value should be “Anywhere;Trailing” if word sequence elements is supported anywhere in a rule.

DictationInCFG

Value is “Trailing” if the engine supports dictation element in CFGs, as defined by SAPI at the end of rule. Value should be “Anywhere;Trailing” if dictation element is supported anywhere in a rule. Dictation elementisan element in a CFG that loads a SLM and returns the recognition result to the application. Refer to SAPI documentation for additional information.

WildcardInCFG

Value should be “Trailing” if the engine supports garbage elements at the end of a rule in a CFGs, as defined by SAPI. Value should be “Anywhere;Trailing” if garbage elements are supported anywhere in a rule. Garbage is an element in a rule that the engine doesn’t recognize but recognizes the rest of the phrase from a CFG. Refer to SAPI documentation for additional information on Garbage.

RecognizerToken1\UI

Required - If the Voice has UI –this key stores UI tokens for the voice token.

RecognizerToken1\UI\EngineProperties

Required - If engine supports a separate UI for Engine properties. A dialog box appears when the user clicks the Properties button next to the list of Engines on the Speech Recognition tab. If the engine does not support a UI for Engine properties, the Properties button will be unavailable.

CLSID

Required - Sample CLSID for object which instantiates engine-specific UI from the Speech Control Panel.

RecognizerToken1\UI\ AddRemoveWord

Required - If engine provides a separate UI for Engine properties. This brings up the dialog box for the user to add a word to the user lexicon and language model, typically from within an application.

CLSID

Required - Sample CLSID for object which instantiates an engine-specific UI from the Speech Control Panel.

RecognizerToken1\UI\MicTraining

Required - If the engine provides a separate UI for Microphone wizard. The microphone wizard sets the microphone gain. It appears from within the Speech Control Panel or any application that calls it. SAPI checks to see if RecognizerToken1\UI\MicTraining is populated. Click Configure Microphone on the Speech Recognition tab to invoke the CLSID below, which invokes the microphone training wizard. If the MicTraining token is not supported on the current default engine, the button is unavailable in the Speech Control Panel.

CLSID

Required - Sample CLSID for object which instantiates engine-specific UI from Speech Control Panel.

RecognizerToken1\UI\UserTraining

Required - If the engine provides an engine-specific training wizard. The training wizard trains the current default Recognition profile. The training wizard appears from within the Speech Control Panel or any application that calls it. SAPI checks to see if RecognizerToken1\UI\UserTraining is populated. Click Train Profile on the Speech Recognition tab to invoke the CLSID below, which invokes the training wizard. If the UserTraining token under the current default engine is not supported, the button is unavailable in the Speech Control Panel.

CLSID

Required - Sample CLSID for object which instantiates an engine-specific UI from the Speech Control Panel.

RecognizerToken1\UI\RecoProfileProperties

Required - If the engine-specific profile wizard exists. To set any additional properties on the Recognition Profile such as sensitivity, background adaptation, etc. Click Properties on the Speech Recognition tab next to the list of recognition profiles.

CLSID

Required - Sample CLSID for object which instantiates engine-specific UI from Speech Control Panel.

There is also a Recognizers category in the HKCUS hive that stores the selected default Recognizer. This is done exactly as for Voices, as shown in Table 8. The CategoryID is:

HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\Speech\Recognizers\

6.3 Category: RecoProfiles

RecoProfiles (RP) is a user-specific, engine-specific data file in which an SR Engine stores the user-specific acoustic and language data. The RP can be thought of as a bag of information that only the engine knows about. The RP also stores the attributes in a few keys under the RP’s key in the registry (this is current Profiles Tab of the Speech Control Panel).

There are two key reasons for a user to have multiple acoustic profiles:

1. In a shared login case (for example, with a Win98 or Millennium home computer where the users typically press cancel to the login dialog box to enter the machine), multiple files allow two or more users to keep languages and acoustic data separate. In this case, the user will need to manually change the profile to the correct one in the Speech Control Panel before starting recognition (or an application may provide its own UI to do this).

2. On a laptop, to offer the user the choice of having different acoustic profiles for different acoustic settings, such as home and office.

A typical RP token is located in the user hive in the following location in the registry

HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\Speech\RecoProfiles\Tokens\{ProfileGUID 1}

Initially, SAPI creates only one GUID, called Default User, for the RecoProfile. When the Recognizer is used for the first time, it should create a key under this GUID token of the Recognizer. For instance, if the default recognizer has the GUID XXX, the token HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\Speech\RecoProfiles\Tokens\{ProfileGUID 1}\XXXis created. RecoProfile stores all the files and settings under this key. These settings may include paths to the acoustic and language model files for the profile that are modified during speaker enrollment and subsequently during recognition. It may also contain additional data about the profile that may improve the recognizer accuracy, such as age, gender, microphone gain setting and so on.

Under the Recoprofile token, there is a key for the GUID of each engine that has a profile. When keeping the profile the same, a user switches the default engine (say to YYY) in the Speech Control Panel. The new engine, on instantiation (or termination of the session) should create thekey HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\Speech\RecoProfiles\Tokens\{ProfileGUID 1}\YYY FullPathToFil A

All engine-specific (YYY-specific for instance) settings for its RecoProfile should be stored under this key.

6.4 Category: AudioInput

The HKEY_LOCAL_MACHINE/SOFTWARE/MICROSOFT/Speech/AudioInput category contains token enumerators that enumerate all the AudioInput devices present on the machine. There is a token enumerator for each class of AudioInput Device. By default, SAPI 5 will have only a single token enumerator for the MMSys technology. This token enumerator will create an audio token for each AudioInput device (microphone) on the machine and return it when an application or engine calls SpEnumTokens or IenumSpObjectTokens.

The AudioInput category does not have standard attributes, and if multiple technologies are installed, an application needs to inspect each token to find the most suitable one.

Any additional AudioIn token enumerators must meet the requirements for a token enumerator laid out in Table 2. Example of the AudioInput category at:

HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\Recognizers\

Table 11 AudioInput Category

RegKey | ValueName | Comments
TokenEnums\MMSys |  | This is the category.
 | DefaultTokenID | This Default can point to a token enumerator or token.

6.5 Category: AudioOutput

The HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\AudioOutput category contains token enumerators that enumerate all the audio output devices present on the machine. Exactly as in the AudioInput category, there is a token enumerator for each technology of audio output Devices. By default, there will be a single token enumerator for MMsys. Under this, there will be entries for each audio output device installed on the machine.

6.6 Category: AppLexicons

The AppLexicons category stores all the application lexicons SAPI knows about. As in other categories, the lexicons are located under HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\Applexicons\Tokens. When called, the SpLexicon interface enumerates all the applexicons. Applexicons have no attributes, and therefore, there is no way to load only specific Applexicons. These keys will be created by applications to make their own lexicons available through SAPI.

6.7 Category: PhoneConverters

The ISpPhoneConverter interface enables the application to convert from the SAPI character phoneset to the ID phoneset. Phone Converter keys should go under HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Speech\PhoneConverters,

SAPI has a single phoneconverter for each language. An engine can query for the phoneconverter whose language attribute matches the application’s language of interest.

6.8 UserLexicons

SAPI stores the userlexicon keys under the HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\Speech\UserLexicon key. UserLexicon is not a category by itself. There is no interaction between an application or engine with the UserLexicon token. It is mentioned here only for the sake of completeness of the registry documentation.

7 Index of Tables

Table 1: Parts of a Token in the Registry. 3

Table 2 Parts of the AudioInput token enumerator9

Table 3 Common Helper Functions. 10

Table 4 Engine Developer Helper Functions. 11

Table 5 Query Operators. 12

Table 6 Voices installed on a machine. 13

Table 7 Scoring of tokens matching optional criteria. 13

Table 8 Voice Registry and Attributes. 18

Table 9: Voices - User Registry Settings. 19

Table 10 Sample Entries of a Recognizer token. 20

Table 11: AudioInput Category. 23


SAPI 5 Registry Usage

1 Contents

1 Contents. 1

2 Table of Tables. 1

3 Compliance Testing Overview.. 3

3.1 SAPI Compliance Required Tests 3

3.2 SAPI Compliance Feature List Tests 3

3.3 Minimum Requirements 3

4 Using the compliance testing tool4

5 Compliance Tests. 5

5.1 Test Result Log:5

6 Compliance Testing Configuration Options. 5

6.1 SAPI 5.0 Compliance Testing Application Toolbar6

6.2 SAPI 5.0 Compliance Testing Application Menu Choices 7

6.3 SAPI 5.0 Compliance Testing Logging Options 9

6.4 SAPI 5.0 Compliance Testing Run Options 10

6.5 SAPI 5.0 Compliance Test Selection Options 11

7 SAPI Compliance: SR.. 13

7.1 Required Tests 13

7.2 Feature Tests 23

7.3 SR Sample Engine 27

7.4 Compliance Test Customization 29

7.5 Multilingual Support29

7.6 OS Language Incompatibility 31

8 SAPI Compliance: TTS. 32

8.1 Required Tests 32

8.2 Feature Tests 34

8.3 TTS Sample Engine 35

8.4 Multilingual Support36

8.5 OS Language Incompatibility 43

2 Table of Tables

Table 1: Events Compliance Test14

Table 2: Lexicon Compliance Test16

Table 3: Command and Control Compliance Test20

Table 4: Required Compliance Tests. 22

Table 5: Events Feature Compliance Test23

Table 6: Grammar Feature Compliance Test25

Table 7: Feature Compliance Tests. 26

Table 8: Sample Engine Required Compliance Test results. 28

Table 9: Sample Engine Feature Compliance Test results. 28

Table 10: Strings to be localized. 31

Table 12: Speak Flag Tests. 32

Table 13: Speak Tests. 33

Table 14: Lexicon Tests. 33

Table 15: SAPI XML tests. 33

Table 16: Events Tests. 34

Table 17: Sample Engine Required Test Results. 36

Table 18: Sample Engine Feature List Test Results. 36

Table 19: Strings to be localized for compliance tests. 42

Table 21: Required Compliance Tests Failed. 43

Table 22: Feature Compliance Tests Not Supported. 43

3 Compliance Testing Overview

This paper, directed toward engine vendors, describes the SAPI 5.0 compliance testing tool by answering the following questions:

· What does SAPI compliance for SAPI 5.0 imply?

· What are the SAPI compliance tests?

· What does each test look for?

The goals of the compliance tool are to help engine vendors test their speech engines for SAPI compliance and port these speech engines to SAPI 5.0. The tests also help vendors to support various SAPI features that are not required for compliance. These tests do not test the speech or performance quality of the engines. All compliance tests assume that SAPI will do parameter validation, and as such, they do not check the engine’s ability to handle invalid parameters such as null, bad pointers, or values out of range.

To run the compliance tests, the SPcomp.exe tool is used and either the Text-to-Speech (TTS) or the Speech Recognition (SR) test suite is selected. This tool generates a log report indicating the results of the compliance tests.

There are two types of SAPI 5 compliance tests:

1) required tests

2) feature list tests

The compliance tests do not necessarily test the DDI directly, instead, the use the SAPI API function calls to test the engine’s response to the DDI. The default engine is always used as the engine in the compliance test. Currently, the supported languages for the compliance tests are English, Japanese and Simplified Chinese[1]. Please check www.microsoft.com/speech/ for language pack updates and information.

3.1 SAPI Compliance Required Tests

The results of the required tests are of a pass/fail nature. These tests were designed to help an engine reach a minimal amount of functionality with the SAPI DDI layer. In order to be SAPI compliant, the engine must pass all required SAPI tests.

3.2 SAPI Compliance Feature List Tests

The results of the feature list tests are either “Supported” or “Unsupported”. Feature list tests were designed to help engine vendors port advanced features to SAPI 5.0. To be SAPI compliant, the engine does not need to pass any feature list test, although it is recommended that all features be implemented if possible.

3.3 Minimum Requirements

The minimum requirements for speech recognition for dictation are a 200 mhz Pentium with 64 MB for win 95/98 or 96 MB for NT. The recommended machine is a 300 Mhz Pentium II with 128 megs of RAM or better.

4 Using the compliance testing tool

The SAPI 5.0 compliance test tool, SPcomp.exe, enables you to load compliance test suites and determine the test result logging options. Please see Compliance Tests for more options.

The SPcomp.exe test tool creates the .pro file for a given test suite. Perform compliance tests by starting the SPcomp.exe application and loading a test suite from a .pro file. The .pro file loads the associated dynamic link library (.dll), which contains the SR or TTS compliance tests.

To start the SAPI 5.0 compliance test tool SPcomp.exe from Windows Explorer, double-click the compliance tool icon. Alternatively, you can perform each compliance test from the command line by running the compliance test tool and command line syntax.

For example, the command line syntax for running the SRcomp compliance test in the srcompreq.pro test suite is as follows:

C:> SPcomp.exe srcompreq.pro

SPcomp.exe srcompreq.pro

Starts the compliance test tool and loads the speech recognition (SR) required tests from the SRcomp.dll.

SPcomp.exe srcompopt.pro

Starts the compliance test tool and loads the speech recognition (SR) feature list tests from the SRcomp.dll.

SPcomp.exe ttscompreq.pro

Starts the compliance test tool and loads the text-to-speech (TTS) required tests from the TTScomp.dll.

SPcomp.exe ttscompopt.pro

Starts the compliance test tool and loads the text-to-speech (TTS) feature list tests from the TTScomp.dll.

5 Compliance Tests

The SAPI 5.0 compliance tests verify that you have successfully implemented the required features to be considered compatible with SAPI 5.0. Your engine must successfully complete each of the following four compliance tests with 100 percent pass rate to be compliant with SAPI 5.0.

1. srcompreq.bat
—Speech Recognition (SR) required test batch file.

2. srcompopt.bat
—Speech Recognition (SR) feature list test batch file.

3. ttscompreq.bat
—Text-to-Speech (TTS) required test batch file.

4. ttscompopt.bat
—Text-to-Speech (TTS) feature list test batch file.

5.1 Test Result Log:

The SAPI 5.0 compliance tool generates a result log and you can configure it to display the result log information or you can save it to a file. The test result log contains pass or fail state information for each segment of the test suite. If a compliance test fails, you can review the result log to determine the origin of the failure.

Example result log with a 100% pass rate for all tests.

Total:
=============================================================================
     PASS    FAIL    SUPPORTED    UNSUPPORTED    ABORTED    SKIPPED
-----------------------------------------------------------------------------
     1       0       0            0              0          0
=============================================================================
Status: PASS

You can select the detailed logging option from the Set Logging configuration choices; the SAPI 5.0 compliance tool creates a detailed result log. Information about setting the logging options is available at: SAPI5 compliance testing logging options.

6 Compliance Testing Configuration Options

The SAPI 5.0 compliance test application user interface (UI) enables you to configure the testing options. The following section provides additional compliance test configuration information.

· SAPI 5.0 Compliance Testing Application Toolbar

· SAPI 5.0 Compliance Testing Application Menu Choices

· SAPI 5.0 Compliance Testing Logging Options

· SAPI 5.0 Compliance Testing Run Options

· SAPI 5.0 Compliance Test Selection Options

6.1 SAPI 5.0 Compliance Testing Application Toolbar

The main window of the SAPI 5.0 compliance testing application contains a toolbar from which you can access the configuration options. Additionally, the configuration options are also available from the menu bar located at the top of the compliance testing application window.

Pause on an icon to display tooltip text. Click an icon to view the information associated with the feature.

Load the test DLL

Loads a test dynamic-link library (DLL).

You can run compliance tests by using one of the following methods:

1. From the SAPI Engine Compliance Tool, click File, and then click Load Test DLL.

2. Load the test DLL into SPcomp.exe from the command line.
For more information, see Using the compliance testing tool.

Note: loading a compliance test with either method results in automatically unloading any previously loaded compliance tests.

Load the test settings

Loads one of the pre-configured test suites.

You can run compliance tests by using one of the following methods:

1. From the SAPI Engine Compliance Tool, click File, and then click Load Settings.

2. Load the test DLL into SPcomp.exe from the command line.
For more information, see Using the compliance testing tool.

Note: loading a compliance test with either method results in automatically unloading any previously loaded compliance tests.

Save settings

Saves the configuration settings for the compliance test application.

Copy

Selects and copies content from the display log.

Clear Window

Clears the display contents of the result log.

Find

Searches for a specific word or phrase within the result log.

Find Next

Searches for the next occurrence of a specific word or phrase within the result log.

Run Test

Begins the compliance test.

Stop Test

Stops the compliance test.

Set Run Options

Configures the compliance test options.

Select Tests

Chooses which compliance test contained in the test suite to.

Set Logging

Determines location of the compliance test log information.

6.2 SAPI 5.0 Compliance Testing Application Menu Choices

The SAPI 5.0 compliance testing application configuration choices are accessible through the menu system. The following items are covered in this section:

· File menu

· Edit menu

· Test menu

· Options menu

· Help menu

6.2.1 File menu

Click File to set configuration options to load settings, save settings, or load the appropriate test DLL. Use the arrow keys to view various menu choices. Press ENTER to select a menu choice.

6.2.2 Edit menu

Click Edit to copy text from the result log and search for text within the result log. Use the arrow keys to view various menu choices. Press ENTER to select a menu choice.

6.2.3 Test menu

Click Test to run the test or select a test. Use the arrow keys to view various menu choices. Press ENTER to select a menu choice.

6.2.4 Options menu

Click Options to view the various configuration settings. Use the arrow keys to view various menu choices. Press ENTER to select a menu choice.

6.2.5 Help menu

Click Help and then click About to display the SAPI 5.0 Engine Compliance Tool Version dialog box. Use the arrow keys to view the various menu choices. Press ENTER to select a menu choice.

6.3 SAPI 5.0 Compliance Testing Logging Options

From the Options menu, choose Logging Settings to set SAPI 5.0 compliance test result log configuration options.

Window

Displays the test result information in the main window of the compliance testing application.

Log File

Saves the test result information as text in a log file.

The log file is located at the same directory as SPcomp.exe tool and the file name will be the following style:

spcomp@442.log

The numbers "442" in the file name are generated by the SPcomp.exe tool and will be incremented by one each time you restart SPcomp.exe tool and run the test. A new log file is generated each time you start SPcomp.exe tool and run a compliance test.

Detailed

Specifies detailed result log information.

Summary

Specifies summary result log information.

6.4 SAPI 5.0 Compliance Testing Run Options

From the Options menu, click Run Options to configure SAPI 5.0 compliance testing run options.

Random

Randomizes the test order.

Close after execution

Closes the compliance testing application after the test sequence.

Stress

This option should not be selected for compliance tests.

Run count

Specifies the number of interations the selected test should run.

Disable screen saver

Disable the screen saver.

Quiet

Runs the selected test in quiet mode.

Random Seed

The random seed value set here is used for the next time you run the compliance test.

Note: When troubleshooting a failed compliance test, you need to enter the same seed value information that was used for the failed compliance test before you repeat the compliace test proceedure.

You can obtain the compliance test seed value from the "Random Seed" field information in the SPcomp@xxx.log file that was generated during the unsuccessful compliance test.

6.5 SAPI 5.0 Compliance Test Selection Options

From the Test menu, click Select Test to configure SAPI 5.0 compliance test choices.

Test Cases

Displays the current test suite.

Selected Test Cases

Displays the current selected tests.

Add Case(s)

Adds test items to the list of selected test cases.
Alternatively, to add test cases, right-click the test case in the test case display window and click Add Item.

Remove Case

Removes the selected test case from the current test. However, removing the selected test does not affect the need to successfully pass this test case to satisfy SAPI compliancy.

Alternatively, to remove test cases, right-click the test cases in the selected test case display window and click Remove Case.

Remove All

Removes all test cases.

7 SAPI Compliance: SR

SAPI compliant SR engines must be able to perform the following[2]:

§ Generate certain SR events

§ Interact with the SAPI lexicon

§ Handle Command and Control (C&C) grammars

§ Generate Phrase Elements

§ Support auto pause on recognition

§ Support rule synchronization

§ Support multiple instances of the engine

§ Support multiple application contexts

7.1 Required Tests

7.1.1 Events

Events will be checked for with .wav files. The test will feed the wav file to the engine and expect a specific event notification to occur. Please note that whether or not the engine can fire a specific event depends on the confidence threshold of the engine. Engine vendors could change the .wav quality to meet their requirement.

For English:

Test | Description | Resource IDs | Description
SoundStart | Test will check if a sound start event occurs. | IDS_WAV_SOUNDSTART | Input .wav file, tag_l.wav
 |  | IDR_L_GRAMMAR | Input CFG grammar
SoundEnd | Test will check if a sound end event occurs. | IDS_WAV_SOUNDEND | Input .wav file, tag_l.wav
 |  | IDR_L_GRAMMAR | Input CFG grammar
PhraseStart | A .wav file with audio the engine can do recognition on. Test insures that a phrase start event occurs | IDS_WAV_PHRASESTART IDR_L_GRAMMAR | Input .wav file, tag_l.wav Input CFG grammar
Recognition | A .wav with audio that the engine can do recognition on. Test insures that a recognition event occurs. | IDS_WAV_RECOGNITION_1 IDR_L_GRAMMAR | Input .wav file, tag_l.wav Input CFG grammar
False Recognition | A wav file and a mismatching C&C grammar are loaded. Test insures that false recognition event occurs. | IDS_WAV_RECOGNITION_1 IDR_RULE_GRAMMAR | Input .wav file, tag_l.wav Input CFG grammar
SoundStart/ SoundEnd | Test will check that the sound start event occurs before the sound end event. | IDS_WAV_SOUNDSTARTEND IDR_L_GRAMMAR | Input .wav file, tag_l.wav Input CFG grammar
PhraseStart/ Recognition | Test will check that the phrasestart event occurs before the recognition event. | IDS_WAV_RECOGNITION_1 IDR_L_GRAMMAR | Input .wav file, tag_l.wav Input CFG grammar
SoundStart/ PhraseStart/ Recognition/ SoundEnd/ | A wav file with audio that the engine can do recognition on. Test insures that the audiooffsets of these events are correct in terms of value comparison. | IDS_WAV_RECOGNITION_1 IDR_L_GRAMMAR | Input .wav file, tag_l.wav Input CFG grammar

Table 1: Events Compliance Test

7.1.2 Lexicon

It is expected that changes in the user and application lexicon will be synchronized with the engine both when the engine starts up and after it has loaded a command and control grammar.

Test | Description | Resource IDs | Description
User Lexicon Before C&C Grammar Loaded | A made-up word with its customized pronunciation is added to the user lexicon. After command and control grammar is loaded, audio will be sent with the word added and the expected result is checked for. | IDS_WAV_SYNCH_BEFORE_LOAD IDR_SNORK_GRAMMAR IDS_RECO_SYNCH_BEFORE_LOAD IDS_RECO_NEWWORD_PRON | Input .wav file, lexicon.wav Input CFG grammar The lexicon form of new word The pronunciation of the new word in user lexicon
User Lexicon After C&C Grammar Loaded | After command and control grammar is loaded, a made-up word with its customized pronunciation is added to the user lexicon. Audio will be sent with the word added and the expected result is checked for. | IDS_WAV_SYNCH_AFTER_GRAM IDR_SNORK_GRAMMAR IDS_RECO_SYNCH_AFTER_GRAM IDS_RECO_NEWWORD_PRON | Input .wav file, lexicon.wav Input CFG grammar The lexicon form of new word The pronunciation of the new word in user lexicon
Application Lexicon and C&C Grammar | A made-up word with its customized pronunciation is added to the application lexicon. After command and control grammar is loaded, audio will be sent with the word added and the expected result is checked for. | IDS_WAV_APPLEX IDR_SNORK_GRAMMAR IDS_APPLEX_WORD IDS_APPLEX_PROP | Input .wav file, lexicon.wav Input CFG grammar The lexicon form of new word The pronunciation of the new word in application lexicon
User lexicon before application lexicon | A made-up word is added to both user lexicon andapplication lexicon using the different customized pronunciations. After command and control grammar is loaded, audio will be sent with the word’s pronunciation in user lexicon and the expected result is checked for. | IDS_WAV_USERLEXBEFOREAPPLEX IDR_SNORK_GRAMMAR IDS_USERLEXBEFOREAPPLEX_WORD IDS_USERLEXBEFOREAPPLEX_USERPROP IDS_USERLEXBEFOREAPPLEX_APPPROP | Input .wav file, lexicon.wav Input CFG grammar The lexicon form of new word The pronunciation of the new word in user lexicon The pronunciation of the new word in application lexicon

Table 2: Lexicon Compliance Test

7.1.3 Command and Control Grammar

Testing the engine for grammar compliance is perhaps the most complex set of tests. The engine must process a grammar correctly. Each test will use a grammar specifically tailored for the particular feature.

Test | Description | Resource IDs | Description
L Tag | A three-element list grammar is loaded. Audio with the middle item to be recognized with the sent to the engine and the result checked for this item. | IDS_RECO_L_TAG IDS_WAV_L_TAG IDR_L_GRAMMAR | Expected Result Input .wav file, tag_l.wav Input CFG grammar
Expected Rule | A grammar with two identical rules is loaded. The first rule will be activated. Audio that triggers this rule is sent and test verifies that the engine uses the first rule. The first rule is then de-activated and the second rule is activated. The same audio is sent and the test verifies that the engine uses the second rule. | IDS_RECO_EXPRULE_FIRSTRULE IDS_RECO_EXPRULE_SECONDRULE IDS_WAV_EXPRULE_TAG IDR_EXPRULE_GRAMMAR | First Rule’s Name Second Rule’s Name Input .wav file, tag_exprule.wav Input CFG grammar
P Tag | A simple grammar with a single phrase. Audio is sent and recognition is expected. Audio that does not contain the phrase is sent and no recognition is expected. | IDS_RECO_P_TAG IDS_WAV_P_TAG IDR_P1_GRAMMAR | Expected Result Input .wav file, tag_p.wav Input CFG grammar
O Tag | A grammar will be defined with a phrase and an optional phrase preceding and following it. Three audio streams will be sent. One with the first optional phrase, one for the second, and the third that does not contain any optional phrases. The appropriate recognition result is checked for in each case. | IDS_RECO_O_TAG_1 IDS_RECO_O_TAG_2 IDS_RECO_O_TAG_3 IDS_WAV_O_TAG_1 IDS_WAV_O_TAG_2 IDS_WAV_O_TAG_3 | FirstOptionalWord Required word Second Optional word Input .wav file containing the first optional word, tag_o1.wav Input .wav file containing the second optional word, tag_o2.wav Input .wav file without optional words, tag_o3.wav
RULEREF Tag | A grammar with a phrase with a rule reference and a rule defined will be loaded. Audio that triggers the rule will be sent and the result checked. | IDS_RECO_RULE_TAG IDS_WAV_RULE_TAG IDR_RULE_GRAMMAR | Expected Result Input .wav file, tag_rule.wav Input CFG grammar
/Disp/lex/pron format | Test ensures engine can support customized pronunciation provided in the command and control grammar file. | IDS_CUSTOMPROP_NEWWORD_PRON IDS_CUSTOMPROP_NEWWORD_DISP IDS_CUSTOMPROP_NEWWORD_LEX IDS_CUSTOMPROP_RULE IDS_WAV_CUSTOMPROP | The customized pronunciation form of the new word The customized display form of the new word The customized lexicon form of the new word The dynamic grammar rule name Input .wav file, lexicon.wav

Table 3: Command and Control Compliance Test

7.1.4 Phrase Elements, Auto Pause, Rule invalidation, multiple instances and contexts.

Test | Description | Resource IDs | Description
Phrase Elements | The audio offsets of SPPHRASEELEMENTs in one SPPHRASE are correctly filled in, which means that the audio offset of the first SPPHRASEELEMENT is less than the audio offset of the second SPPHRASEELEMENT, the audio offset of the second SPPHRASEELEMENT is less than the third one, etc. | IDS_WAV_RULE_TAG IDR_RULE_GRAMMAR | Input .wav file, tag_rule.wav Input CFG grammar
Auto Pause | The test makes sure engine can support auto pause feature provided by SAPI. | IDS_AUTOPAUSE_DYNAMICWORD1 IDS_AUTOPAUSE_DYNAMICWORD2 IDS_AUTOPAUSE_DYNAMICRULE1 IDS_AUTOPAUSE_DYNAMICRULE2 IDS_WAV_AUTOPAUSE | The word in the first rule The word in the second rule The name of the first rule The name of the second rule Input .wav file, autopause.wav
Top-level rule invalidation | Test verifies that engine can synchronize the rule information after SAPI notifies engine of top-level rule invalidation. | IDS_INVALIDATETOPLEVEL_DYNAMICWORDS IDS_INVALIDATETOPLEVEL_DYNAMICRULE IDS_WAV_INVALIDATETOPLEVEL_OLD IDS_INVALIDATETOPLEVEL_DYNAMICNEWWORDS IDS_WAV_INVALIDATETOPLEVEL_NEW | The words in the dynamic grammar The rule name in the dynamic grammar Input .wav file used before invalidation, tag_exprule.wav The new words in the dynamic grammar Input .wav file used after invalidation
None-top-level rule invalidation | Test verifies that engine can synchronize the rule information after SAPI notifies engine of non-top-level rule invalidation. | IDS_INVALIDATENONTOPLEVEL_RULE1 IDS_INVALIDATENONTOPLEVEL_RULE2 IDS_INVALIDATENONTOPLEVEL_TOPLEVELRULE IDS_INVALIDATENONTOPLEVEL_OLDWORD1 IDS_INVALIDATENONTOPLEVEL_OLDWORD2 IDS_WAV_INVALIDATENONTOPLEVEL_OLD IDS_INVALIDATENONTOPLEVEL_NEWWORD1 IDS_INVALIDATENONTOPLEVEL_NEWWORD2 IDS_WAV_INVALIDATENONTOPLEVEL_NEW | The first rule name The second rule name The top-level rule name The word in the first rule used before invalidation The word in the second rule used before invalidation Input .wav file used before invalidation, tag_exprule.wav The word in the first rule used after invalidation The word in the second rule used after invalidation Input .wav file used after invalidation, tag_rule.wav
Multiple recognition contexts | Multiple recognition contexts will be created with different grammars. The test will verify that the recognition event is generated by the correct recognition contexts. | IDS_RECO_P_TAG IDS_WAV_MULT_RECO IDR_P1_GRAMMAR IDR_P2_GRAMMAR | The result expected in the second grammar Input .wav file, multireco.wav The first grammar used by the first recocontext The second grammar used by the second recocontext
Multiple recognition engine instances | Basic tests are run separately on different threads to see if engine can support multi instances. | NA | NA

Table 4: Required Compliance Tests

7.2 Feature Tests

Some of the features exposed through SAPI are useful from a competitive advantage point of view. Features are not required by SAPI compliance, but may be an attractive function for engine vendors to implement. SAPI features are:

§ Interference and hypothesis events

§ Dictation functionalities

§ Advanced command and control features

§ Command and control alternate

§ Engine properties

§ Inversed text normalization

7.2.1 Events

Events will be checked for with .wav. The test will feed the .wav to the engine and expect a specific event notification to occur. Please note that whether or not the engine can fire a specific event depends on the confidence threshold of the engine. Engine vendors may change the .wav files if it is felt that the .wav quality does not meet their requirements (Refer to Section 7.4).

Test | Description | Resource IDs | Descriptions
Interference | A wav file with noises. Test will check that an interference event occurs. | IDS_WAV_INTERFERENCE IDR_L_GRAMMAR | Input .wav file, tag_l.wav Input CFG grammar
Hypothesis | A .wav file with audio that engine can do recognition on. Test insures a hypothesis event occurs. | IDS_WAV_HYPOTHESIS IDR_EXPRULE_GRAMMAR | Input .wav file, tag_exprule.wav Input CFG grammar

Table 5: Events Feature Compliance Test

7.2.2 Dictation functionalities

This the required features if Engine wants to support dictation grammar. This include some basic functionalities for dictation grammar. This includes lexicon, dictation tag, dictation alternates.

Test | Description | Resource IDs | Descriptions
User Lexicon Before dictation Grammar Loaded | A made-up word with its customized pronunciation is added to the user lexicon. After dictation grammar is loaded, audio will be sent with the word added and the expected result is checked for. | IDS_WAV_SYNCH_BEFORE_LOAD IDS_RECO_SYNCH_BEFORE_LOAD IDS_RECO_NEWWORD_PRON | Input .wav file, lexicon.wav The lexicon form of new word The pronunciation of the new word in user lexicon
User Lexicon After dictation Grammar Loaded | After dictation grammar is loaded, a made-up word with its customized pronunciation is added to the user lexicon. Audio will be sent with the word added and the expected result is checked for. | IDS_WAV_SYNCH_AFTER_DICT IDS_RECO_SYNCH_AFTER_DICT IDS_RECO_NEWWORD_PRON | Input .wav file, lexicon.wav The lexicon form of new word The pronunciation of the new word in user lexicon
Dictation Tag | A rule with dictation tag is loaded. Audio is feed and the test verifies the recognition event is generated. | IDS_DICTATIONTAG_WORDS IDS_DICTATIONTAG_RULE IDS_WAV_DICTATIONTAG | The word before the dictation tag The dynamic grammar rule name Input .wav file, tag_exprule.wav
Dictation alternates | Test ensures that engine can generate alternate results for dictation grammar. The test makes sure that engine has its own alternate object and the object can generate some alternate results. | IDS_WAV_EXPRULE_TAG | Input .wav file, tag_exprule.wav

Table 5: Dictation Compliance Test

7.2.3 Grammar

Each test will use a grammar specifically tailored for the particular feature. Some tests would use dynamic grammar instead of the static grammar.

Test | Description | Resource IDs | Descriptions
WildCard Tag | A rule with wildcard tag is loaded. Audio is feed and the test verifies the recognition event is generated. | IDS_WILDCARD_WORDS IDS_WILDCARD_RULE IDS_WAV_WILDCARD | The word before the wildcard tag The dynamic grammar rule name Input .wav file, tag_rule.wav
TextBuffer Tag | A grammar with <TextBuffer> tag will be loaded. Test fills in the content of TextBuffer on the fly. Audio with both static part and dynamic part of the grammar would be feed and the result would be checked. | IDS_CFGTEXTBUFFER_WORDS IDS_CFGTEXTBUFFER_BUFFERWORD IDS_CFGTEXTBUFFER_RULE IDS_WAV_CFGTEXTBUFFER | The words before TEXTBUFFER tag The word for TEXTBUFFER tag The rule name Input .wav file, tag_exprule.wav
Use the correct grammar | Two unambiguous grammars are loaded to test if engine can use the correct grammar to do recognition. | IDS_RECO_RULE_TAG IDS_WAV_RULE_TAG IDR_L_GRAMMAR IDR_RULE_GRAMMAR | Expected result for the second grammar Input .wav file, tag_rule.wav The first grammar The second grammar
Use the most recently activated grammar | Two ambiguous grammars are loaded to test if engine can use the most recently activated grammar to do the recognition. | IDS_WAV_RULE_TAG IDR_RULE_GRAMMAR | Input .wav file, tag_rule.wav Input CFG grammar

Table 6: Grammar Feature Compliance Test

7.2.4 Alternates, engine properties, inversed text normalization

Test | Description | Resource IDs | Descriptions
Command and Control alternates | Test ensures that the engine can generate alternate results for command and control grammar | IDS_ALTERNATESCFG_BESTWORD IDS_ALTERNATESCFG_ALTERNATE1 IDS_ALTERNATESCFG_ALTERNATE2 IDS_ALTERNATESCFG_WORDS IDS_WAV_ALTERMATESCFG | The best choice of the CFG grammar Alternate word in CFG grammar Alternate word in CFG grammar Others words in the CFG grammar tag_exprule.wav
Engine numeric properties | If engine supports the numeric properties specified by SAPI | NA |
Engine text properties | If engine can return S_FALSE on the text properties that are not supported. | NA |
Inversed Text Normalization | The test uses a wav file and expects engine to pass back a result containing digits together with the normal result. Please note that this is a very specific ITN test and is not coverage of ITN related issues. | IDS_RECO_GETITNRESULT IDS_WAV_GETITNRESULT IDR_RULE_GRAMMAR | Expected ITN result Input .wav file, tag_rule.wav Input CFG grammar

Table 7: Feature Compliance Tests

7.3 SR Sample Engine

The sample engine is not fully SAPI compliant due to the fact that it does not have the full range of functionality that a true SR engine would have. Table 8 indicates which compliance tests will pass. Table 9 indicates which features are supported.

Test | Result | Description
Events |  |
SoundStart | Pass |
SoundEnd | Pass |
PhraseStart | Pass |
FalseRecognition | Fail | The sample engine doesn’t generate this event based on the real SR job.
Recognition | Pass |
SoundStart/SoundEnd order | Pass |
PhraseStart/Recognition order | Pass |
Event offset | Pass |
Lexicon |  |
User Lexicon Before C&C Grammar Loaded | Fail | The sample engine doesn’t use user lexicon.
User Lexicon After C&C Grammar Loaded | Fail | The sample engine doesn’t use user lexicon.
App Lexicon | Fail | The sample engine doesn’t use application lexicon.
Use user lexicon before application lexicon | Fail | The sample engine does not use either a user lexicon or an application lexicon.
Grammar |  |
L Tag | Fail | The result might be sometimes fail and sometimes pass. The sample engine randomly generates results based on the given grammar. It doesn’t do any real recognition.
Expected Rule | Pass |
P Tag | Fail | The result might be sometimes fail and sometimes pass. The sample engine randomly generates results based on the given grammar. It doesn’t do any real recognition.
O Tag | Fail | The result might be sometimes fail and sometimes pass. The sample engine randomly generates results based on the given grammar. It doesn’t do any real recognition.
Ruleref Tag | Fail | The result might be sometimes fail and sometimes pass. The sample engine randomly generates results based on the given grammar. It doesn’t do any real recognition.
/Disp/lex/pron format | Fail | The result might be sometimes fail and sometimes pass. The sample engine randomly generates results based on the given grammar. It doesn’t do any real recognition.
Other |  |
Phrase Elements | Pass |
Auto Pause | Pass |
Top-level rule invalidation | Fail | The sample engine randomly generates results based on the given grammar. It doesn’t do any real recognition.
Non-top-level rule invalidation | Fail | The sample engine randomly generates results based on the given grammar. It doesn’t do any real recognition.
Multiple recognition contexts | Pass |
Multiple recognition engine instances |  | The sample engine randomly generates a cfg result based on the given grammar. It doesn’t do any real recognition.

Table 8: Sample Engine Required Compliance Test results

Test | Result | Description
Events |  |
Hypothesis | SUPPORTED |
Interference | UNSUPPORTED | The sample engine doesn’t generate the event correctly.
Dictation |  |
User lexicon synchronize before dictation grammar loaded | UNSUPPORTED | The sample engine doesn’t use user lexicon.
User lexicon synchronize after dictation grammar loaded | UNSUPPORTED | The sample engine doesn’t use user lexicon.
Dictation Tag | SUPPORTED |
Dictation alternates | SUPPORTED |
Grammar |  |
Wildcard Tag | SUPPORTED |
TextBuffer Tag | SUPPORTED |
Use the correct grammar | UNSUPPORTED | The sample engine randomly generates results based on the given grammar. It doesn’t do any real recognition.
Use the most recently activated grammar | UNSUPPORTED | The sample engine randomly generates results based on the given grammar. It doesn’t do any real recognition.
Other |  |
Command and Control alternates | UNSUPPORTED | The compliance test only uses one rule while the sample engine needs at least two rules.
Engine numeric properties | SUPPORTED |
Engine text properties | SUPPORTED |
Inversed Text Normalization | UNSUPPORTED | The sample engine doesn’t have this functionality.

Table 9: Sample Engine Feature Compliance Test results

7.4 Compliance Test Customization

Many of the tests do require that a specific recognition result be returned to verify proper handing of such things as the grammar format. To accommodate different engines variability with recognition of different voices and to support non-English engines, these tests will allow the engine vendor to supply a sound file that passes the test (Refer to Section 7.5). Since some tests might share the same .wav file, it is recommended to supply a .wav file with different name. Additionally the grammars can be changed to accommodate words that the engine is able to recognize better (Refer to Section 7.6).

7.5 Multilingual Support

The compliance tests will tests engines for the supported languages[3]. To test an SR engine that uses another language, one must:

§ Ensure that the correct language pack is installed. For Windows 2000 and Millennium Edition, this may be done by installing the language pack from the Windows 2000 or Windows Millennium CD. For Windows 98 and Windows NT 4.0, install the language pack from the Windows update web site (http://windowsupdate.microsoft.com/)

§ Select the engine as the default engine using the Speech Control Panel SR tab.

§ Create and insert a string table in the sapi5sdk\tools\comp\sr\srcomp.rc that is localized for the language. (Refer to Table 10)

§ Create the .wav files[4] according to the new string table and place this under the specified directory (according to the search path precedence (Refer to Section 7.5.2)).

§ Create and compile the appropriate XML files using a grammar editor and complier.

§ Include the CFG binaries into the .dll by importing the CFG file names into srcomp.rc[5].

§ Recompile the sr.dsp.

§ Run the compliance tests.

7.5.1 Example:

If you want to add resource for test SoundStart for language 888:

1. create and insert copy a string table in srcomp.rc for language 888.

2. Change the string “IDS_WAV_SOUNDSTART” to the new .wav file you want to use.

3. Insert the xml grammar file you want to use into the project. Modify the IDR_L_GRAMMAR reference to your cfg binary.

NOTE: If the default engine supports multiple languages, then the compliance test will only run on the first language specified in string “Language” under key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Recognizers\Tokens\MSASREnglish\Attributes. In other words you need to change the order of the languages in the attributes key under your speech recognizer token for each language you wish to test. (Refer to Section 7.6)

The strings that need to be localized are shown in Table 10.

String Number | English Text |
IDS_RECO_L_TAG | put | Translate string
IDS_RECO_EXPRULE_TAG_1 | play | Translate string
IDS_RECO_EXPRULE_TAG_2 | the | Translate string
IDS_RECO_P_TAG | white | Translate string
IDS_RECO_O_TAG_1 | please | Translate string
IDS_RECO_O_TAG_2 | walk | Translate string
IDS_RECO_O_TAG_3 | slowly | Translate string
IDS_RECO_RULE_TAG | seven | Translate string
IDS_RECO_LN_TAG | red | Translate string
IDS_RECO_NEWWORD_PRON | s n ao 1 r k | Translate phonemes
IDS_AUTOPAUSE_DYNAMICWORD1 | put | Translate string
IDS_AUTOPAUSE_DYNAMICWORD2 | red | Translate string
IDS_AUTOPAUSE_DYNAMICRULE1 | Action | Translate string
IDS_AUTOPAUSE_DYNAMICRULE2 | color | Translate string
IDS_INVALIDATETOPLEVEL_DYNAMICWORDS | play the oboe | Translate string
IDS_INVALIDATETOPLEVEL_DYNAMICRULE | Play | Translate string
IDS_INVALIDATETOPLEVEL_DYNAMICNEWWORDS | please play the seven | Translate string
IDS_INVALIDATENONTOPLEVEL_RULE1 | option | Translate string
IDS_INVALIDATENONTOPLEVEL_RULE2 | Thing | Translate string
IDS_INVALIDATENONTOPLEVEL_TOPLEVELRULE | play | Translate string
IDS_INVALIDATENONTOPLEVEL_OLDWORD1 | empty | Translate string
IDS_INVALIDATENONTOPLEVEL_OLDWORD2 | Oboe | Translate string
IDS_INVALIDATENONTOPLEVEL_NEWWORD2 | Seven | Translate string
IDS_INVALIDATENONTOPLEVEL_TOPLEVELWORDS | Play the | Translate string
IDS_CFGTEXTBUFFER_WORDS | Play the | Translate string
IDS_CFGTEXTBUFFER_BUFFERWORD | oboe | Translate string
IDS_CFGTEXTBUFFER_RULE | play | Translate string
IDS_ALTERNATESCFG_BESTWORD | play | Translate string
IDS_ALTERNATESCFG_ALTERNATE1 | played | Translate string
IDS_ALTERNATESCFG_ALTERNATE2 | pay | Translate string
IDS_ALTERNATESCFG_WORDS | The oboe | Translate string
IDS_ALTERNATESCFG_RULE | play | Translate string
IDS_RECO_GETITNRESULT | please play the 7 | Translate string
IDS_CUSTOMPROP_NEWWORD_PRON | s n ao 1 r k | Translate phonemes
IDS_CUSTOMPROP_RULE | play | Translate string
IDS_CUSTOMPROP_NEWWORD_DISP | abc | Translate string
IDS_CUSTOMPROP_NEWWORD_LEX | play | Translate string
IDS_DICTATIONTAG_WORDS | Play the | Translate string
IDS_DICTATIONTAG_RULE | play | Translate string
IDS_WILDCARD_WORDS | Please play | Translate string
IDS_WILDCARD_RULE | play | Translate string
IDS_APPLEX_PROP | s n ao 1 r k | Translate phonemes
IDS_USERLEXBEFOREAPPLEX_USERPROP | s n ao 1 r k | Translate phonemes
IDS_USERLEXBEFOREAPPLEX_APPPROP | P l ey | Translate phonemes
IDS_INVALIDATENONTOPLEVEL_NEWWORD1 | Please | Translate string

Table 10: Strings to be localized

7.5.2 Search Path Precedence

The compliance tests use a search path precedence to find the various .wav files needed for the compliance tests. The order of search is:

§ current directory

§ ‘..\resources’,

§ ‘..\..\..\resources’

§ ‘..\..\resources’

If the compliance tests cannot find the .wav files in these directories, the test will pop up a dialog asking the user to enter the customized dir to open the file. The new directory will be added to the search order and the new search order will persist for the life of SpComp.

It is important to note that for languages that do not have a SAPI standard phoneme set (i.e. languages which are not supported in this version of SAPI), the engine will fail the following required compliance tests:

Test | Result
Lexicon |
User Lexicon Before C&C Grammar Loaded | Fail
User Lexicon After C&C Grammar Loaded | Fail
App Lexicon | Fail
Use user lexicon before application lexicon | Fail
User lexicon synchronize before dictation grammar loaded | Fail
User lexicon synchronize after dictation grammar loaded | Fail
/Disp/lex/pron format | Fail

7.6 OS Language Incompatibility

The compliance tests are not based on the language of the OS. They are based on the first language in the token of the default engine. In other words, a Japanese engine on an English OS will cause the compliance tests to load the Japanese resources and run the compliance tests expecting a Japanese engine. In order to run the compliance test for an engine that has a different language than the OS, you will need to set the default engine through the control panel.

NOTE: If the default engine supports multiple languages, then the compliance test will only run the first language. In other words you need to change the order of the languages in the attributes key under your speech recognizer token for each language you wish to test. For example, if your engine token supports both Japanese and English, to test English, , the string “Language” must be like “409; 411” under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Recognizers\Tokens\MSASREnglish\Attributes

To test Japanese, the string “Language” must be like “411; 409” under the same key.

8 SAPI Compliance: TTS

SAPI compliant TTS engines must be able to perform the following:

§ Speak with SAPI defined speak flags

§ Return the supported audio output formats

§ Interact with the SAPI lexicon

§ Interpret SAPI XML tags

§ React to programmatic volume changes

§ React to programmatic rate changes

§ Synthesize certain SAPI events

§ Skip forwards and backwards through a segment of text

§ Support multiple instances

8.1 Required Tests

8.1.1 Speak

The speak tests test the engine ability to interact with ISpTTSEngine::Speak as well as process and react to actions. Before SAPI passes a speak call to the engine, it Query Interfaces (QIs) the engine object for ISpTTSEngine and ISpObjectWithToken interfaces. If either of these are not implemented correctly, the speak call will fail. If both of the QIs pass, then SAPI will pass the speak call to the engine. The speak call contains a number of speak flags. Table 1 describes the individual tests.

Test | Description
SPF_DEFAULT | This is the normal, default speak flag. The engine should be able to render text to the output site under normal conditions. The engine should also be able to continue rendering when passed the continue action from SAPI
SPF_PURGEBEFORESPEAK | The engine should be able to stop rendering when passed to abort action
SPF_IS_XML | The engine should be able to interpret the SAPI XML when passed to it by SAPI.
SPF_ASYNC | The engine should be able to render text to the output site. The asynch environment should not affect the engine since SAPI handles the multithreading issues.
SPF_SPEAK_NLP_PUNC | the engine should be able to speak punctuation.

Table 12: Speak Flag Tests

The compliance tests also test a number of combination scenarios to help ensure that the engine is able to stop speaking and start speaking in various combinations that are shown in table 2.

Test | Description
Speak Destroy | SAPI sends the engine a speak call followed by an abort action.
Speak Stop | SAPI sends the engine a speak call followed by a purge call. This is to ensure that the engine is able to stop speaking and clear memory.

Table 13: Speak Tests

8.1.2 Output Format

The output format test checks to ensure that the engine is capable of passing its supported output format to SAPI. This is testing the ISpTTSEngine::GetOutputFormat function.

8.1.3 Lexicon

The lexicon tests check the interaction of the engine and SAPI. These tests add a word to the user and application lexicons and check to ensure that the engine is able to detect the words in the lexicon and is using the word pronunciation from the user lexicon first, and then the application lexicon second if the word is present in both lexicons. There are two separate tests as shown:

Test | Description
User lexicon | A word is added to the user lexicon and the engine is requested to pronounce this word. The engine should be aware of the case sensitivity. The engine must support the SAPI phoneme, SAPI part of speech, as well the lexicon APIs.
Application lexicon | A word is added to the application lexicon and the engine is requested to pronounce this word. The engine must support the SAPI phoneme, SAPI part of speech, as well the lexicon APIs.

Table 14: Lexicon Tests

8.1.4 XML Tags

The SAPI XML tags are required for compliance. The tags tests shown are required:

Test | Description
Bookmark | Tests if the engine is able to process the bookmark tag and write the appropriate bookmark event.
Silence | Tests if the engine outputs the correct amount of silence as indicated by the silence tag.
Volume | Tests if the engine can change the volume by the correct amount.
Spell | Tests if the engine can handle the tag and spell out the text.
Pron | Tests if the engine can handle the SAPI defined phoneme set.
Rate | Tests if the engine can change the rate by the correct amount.
Pitch | Tests if the engine can change the pitch by the correct amount.
Context | Tests if the engine can handle the SAPI defined context tag.
Engine proprietary SAPI tags | Test if the engine can handle non-SAPI tags.

Table 15: SAPI XML tests

8.1.5 SetVolume

This test checks to see if the engine is capable of processing the volume change action. When the engine received this action, it should call the GetVolume function from SAPI to get the new volume, and reflect the change in the audio output.

8.1.6 SetRate

This test checks to see if the engine is capable of processing the rate change action. When the engine received this action, it should call the GetRate function from SAPI to get the new volume, and reflect the change in the audio output.

8.1.7 Events

The events test checks to ensure that the engine is writing the correct data, especially wParam and lParam, to the event structure. For the sentence boundary event, wParam is the character length of the sentence including punctuation in the current input stream being synthesized. lParam is the character position within the current text input stream of the sentence being synthesized. For the word boundary event, wParam is the character length of the word in the current input stream being synthesized. lParam is the character position within the current text input stream of the word being synthesized. Any leading and ending spaces will not be included in the length of the word or the sentence. There are three events which are required for SAPI compliance:

Test | Description
SPEI_TTS_BOOKMARK | Checks the engine’s ability to properly fire bookmarks embedded in the text.
SPEI_WORD_BOUNDARY | Checks the engine’s ability to generate word boundaries given a segment of text.
SPEI_SENTENCE_BOUNDARY | Checks the engine’s ability to detect and generate sentence boundaries.

Table 16: Events Tests

8.1.8 Skip

This test examines the engine’s ability to interact with the skip action. Once the engine receives this action, it should call the ISpTTSEngineSite::GetSkipInfo function. After it has completed the skip, it should call the ISpTTSEngineSite::CompleteSkip function.

8.1.9 Multi-Instance

The multiple instances test checks to ensure that the engine can handle multiple calls at the same time from SAPI. The tests contains a total of 4 threads and each thread has its own ISpVoice object and the test runs a random combination of the following tests 20 times consecutively:

§ Speak

§ Skip

§ GetOutPutFormat

§ SetRate

§ SetVolume

§ Check SAPI required Event

§ XML Bookmark

§ XML Silence

§ XML Spell

§ XML Pron

§ XML Rate

§ XML Volume

§ XML Pitch

§ Real time Rate changes

§ Real time Volume changes

§ Speak Stop

§ Lexicon

§ XML context

§ Engine proprietary SAPI tags and other combination of XML tags

8.2 Feature Tests

Some of the features exposed through SAPI are useful from a competitive advantage point of view. Features are not required by SAPI compliance, but may be an attractive function for engine vendors to implement. SAPI features are:

§ Generation of phoneme events (determines if the engine can generate a phoneme event for a given string of text. The phonemes must correspond to the SAPI defined phonemes).

§ Generation of viseme events (determines if the engine can generate a viseme for a given string of text. The viseme must correspond with the SAPI defined visemes).

§ XML emphasis tag (the engine should change the volume, rate, or pitch of the audio rendered).

§ XML PartOfSp tag (the engine should handle the SAPI defined part of speech – the engine will need to implement this for the lexicon compatibility tests)

8.3 TTS Sample Engine

The sample engine is not fully SAPI compliant due to the fact that it does not have the full range of functionality that a true TTS engine would have. Table 6 indicates which compliance tests will pass. Table 7 indicates which features are supported.

Test | Result | Description
Speak |  |
SPF_DEFAULT | Pass |
SPF_PURGEBEFORESPEAK | Pass |
SPF_IS_XML | Pass |
SPF_ASYNC | Pass |
SPF_SPEAK_NLP_PUNC | Pass |
Speak Destroy | Pass |
Speak Stop | Pass |
GetOutput Format | Pass |
Lexicon |  |
User Lexicon | Fail | The sample engine does not use a lexicon.
App Lexicon | Fail | The sample engine does not use an application lexicon.
XML Tags |  |
Bookmark | Pass |
Silence | Pass |
Volume | Fail | The sample engine uses pre-recoded .wav files and cannot adjust the volume of the .wav files.
Spell | Fail | The sample engine uses pre-recorded .wav files and cannot spell each word.
Pron | Fail | The sample engine uses pre-recorded .wav files and cannot interpret the SAPI phonemes.
Rate | Fail | The sample engine uses pre-recoded .wav files and cannot adjust the rate of the .wav files.
Pitch | Fail | The sample engine uses pre-recoded .wav files and cannot adjust the pitch of the .wav files.
Context | Pass |
Engine proprietary SAPI tags | Pass |
SetVolume | Fail | The sample engine uses pre-recoded .wav files and cannot adjust the rate of the .wav files.
SetRate | Fail | The sample engine uses pre-recoded .wav files and cannot adjust the volume of the .wav files.
Events |  |
SPEI_TTS_BOOKMARK | Pass |
SPEI_WORD_BOUNDARY | Pass |
SPEI_SENTENCE_BOUNDARY | Pass |
Skip | Fail | The sample engine uses pre-recoded .wav files and cannot skip sentences since it does not have a sentence breaker.
Multiple Instances Test | Fail | The sample engine uses pre-recoded .wav files and cannot skip sentences, change rate, pitch, and volume, or use lexicons.

Table 17: Sample Engine Required Test Results

Test | Result | Description
Phoneme events | UNSUPPORTED | The sample engine uses pre-recoded .wav files and cannot synthesize the phoneme events.
Viseme events | UNSUPPORTED | The sample engine uses pre-recoded .wav files and cannot synthesize the viseme events.
XML Emph Tag | UNSUPPORTED | The sample engine uses pre-recoded .wav files and cannot adjust the emphasis of the .wav files.
XML PartOfSp | UNSUPPORTED | The sample engine uses pre-recoded .wav files and cannot adjust the part of speech of the .wav files.

Table 18: Sample Engine Feature List Test Results

8.4 Multilingual Support

To test an engine that uses language aside for the supported languages, one must:

§ Ensure that the correct language pack is installed. For Windows 2000 and Millennium Edition, this may be done by installing the language pack from the Windows 2000 or Windows Millennium CD. For Windows 98 and Windows NT 4.0, install the language pack from the Windows update web site (http://windowsupdate.microsoft.com/)

§ Select the engine as the default engine using the control panel TTS tab.

§ Create a string table in the \sapi5sdk\tools\comp\tts\ttscomp.rc which is localized for the language. (Refer to Table 19)

o       Go to ResourceView in the ttscomp workspace, right click the mouse on “String Table”, and select “Insert Copy”. The following window will appear. From the window, select the language that the engine supports, and then click OK.

o       Open \sapi5sdk\tools\comp\tts\ttscomp.rc to your editor and edit your language resources (Refer to Table 19), and then save ttscomp.rc

o       The following is an example how to support GetOutputFormat test in Korean by using Microsoft FrontPage editor:

§ Open ttscomp.dsw and create a string table in Korean, save ttscomp.rc

§ Go to below table 19 and find strings used in GetOutputFormat test. Only one string, IDS_STRING65, is found corresponding to the test.

§ Open ttscomp.rc in Notepad and find IDS_STRING65 under Korean resources

§ Launch Microsoft FrontPage and select File | New and Normal tab

§ Translate “This is the TTS Compliance Test” to Korean.

§ Select Preview tab, right click your mouse, select Encoding | Western European (Windows), and then cut/paste the string from MS FrontPage to IDS_STRING65 under Korean resources in your notepad

§ Save ttscomp.rc

§ Recompile the tts.dsp.

§ Run the compliance tests.

The strings that need to be localized are:

Test Name | String Number | English Text |
Speak Destroy | IDS_STRING6 | This is a long string of text that will not complete because we release it in the next line of code. We expect the engine to clean-up correctly and not fault. | Translate string
Speak | IDS_STRING8 | Hello <BOOKMARK MARK= “12”>World | Translate “Hello World”
 | IDS_STRING10 | This is a test. | Translate string
 | IDS_STRING11 | Blah blah … | May need to translate if engine does not understand phonemes
Phoneme & Viseme Events | IDS_STRING10 |  |
SetVolume | IDS_STRING10 |  |
SetRate | IDS_STRING10 |  |
Check SAPI required Events | IDS_STRING20 | Bookmark <BOOKMARK MARK= “123”/>test | Translate “Bookmark test”
XML Bookmark | IDS_STRING20 |  |
XML Silence | IDS_STRING23 | Hello World | Translate string
 | IDS_STRING24 | Hello <SILENCE MSEC = “8000”/> World | Translate “Hello World”
XML Spell | IDS_STRING26 | <SAPI> ENGLISH LANGUAGE</SAPI> | May need to translate “ENGLISH LANGUAGE” if letters are not known (Refer to IDS_STRING27)
 | IDS_STRING27 | <SPELL> ENGLISH LANGUAGE </SPELL> | May need to translate “ENGLISH LANGUAGE” if letters are not known
XML Rate | IDS_STRING30 | <RATE SPEED= ‘-5’> hello world </RATE> | Translate “hello world”
 | IDS_STRING31 | <RATE SPEED= ‘5’> hello world </RATE> | Translate “hello world”
XML Volume | IDS_STRING33 | <VOLUME LEVEL = ‘100’> hello </VOLUME> | Translate “hello”
 | IDS_STRING34 | <VOLUME LEVEL = ‘1’> hello </VOLUME> | Translate “hello”
XML Pitch | IDS_STRING37 | <PITCH MIDDLE =’-10’> a </PITCH> | Translate the letter ‘a’
 | IDS_STRING38 | <PITCH MIDDLE =’+10’> a </PITCH> | Translate the letter ‘a’
XML PartOfSp | IDS_STRING48 | H l ow | Translate phonemes
 | IDS_STRING52 | N ow n p r ow n ow aa aa ah ao aw b ch eh er | Translate phonemes
 | IDS_STRING76 | test | Translate “test”
Real time volume change | IDS_STRING53 | This <BOOKMARK MARK=’1234’/>string is used in the real time rate and volume tests. It’s rate and volume are adjusted mid stream. Engines should pick these changes up. | Translate “This string is used in the real time rate and volume tests. It’s rate and volume are adjusted mid stream. Engines should pick these changes up.”
Real time rate change | IDS_STRING53 |  |
XML Pronounce | IDS_STRING56 | A | Translate the word “a”
 | IDS_STRING57 | <PRON SYM=”aa n th ow p ow l ow jh iy aa n th ow p ow l ow jh iy aa n th ow p ow l ow jh iy”>a</PRON> | Translate phonemes
Skip | IDS_STRING63 | <SAPI>one. Two. Three. Four. Five. Six. Seven. Eight. Nine. Ten. <BOOKMARK MARK=”123”/>bookmark event. One. Two. Three. Four. Five. Six. Seven. Eight. Nine. Ten. </SAPI> | Translate “one. Two. Three. Four. Five. Six. Seven. Eight. Nine. Ten. Bookmark event. One. Two. Three. Four. Five. Six. Seven. Eight. Nine. Ten.”
User Lexicon Test | IDS_STRING64 | Computer | Translate “computer”
 | IDS_STRING71 | dh aa n th ow p ow l ow jh iy aa n th ow p ow l ow jh iy aa n th ow p ow l ow jh ch ow ao ah ow ow p ow l ow jh ch ow ao ah ow | Translate phonemes
 | IDS_STRING76 |  |
 | IDS_STRING94 | h eh l ow w er l d h eh l ow w er l d h eh l ow w er l d h eh l ow w er l d | Translate phonemes
GetOutputFormat | IDS_STRING65 | This is the TTS Compliance Test | Translate “This is the TTS Compliance Test”
XML Non-SAPI tags | IDS_STRING67 | <SOMEBOGUSTAGS> Non-SAPI tags test </SOMEBOGUSTAGS> | Translate “Non-SAPI tags test”
XML Emph | IDS_STRING68 | <SAPI>Do you hear me?</SAPI> | Translate “Do you hear me?”
 | IDS_STRING69 | <SAPI><EMPH>Do you hear</EMPH>me?</SAPI> | Translate “Do you hear me?”
App Lexicon Test | IDS_STRING76 |  |
 | IDS_STRING94 |  |
XML Context | IDS_STRING99 | <context id=’date_mdy’>12/21/99</context><context id=’date_mdy’>12.21.00</context> <context id=’date_mdy’>12-21-9999</context> | May need to translate dates
 | IDS_STRING100 | <context id=’date_dmy’>21/12/00</context><context id=’date_dmy’>21.12.33</context><context id=’date_dmy’>21-12-1999</context> | May need to translate dates
 | IDS_STRING101 | <context id=’date_ymd’>99/12/21</context><context id=’date_ymd’>99.12.21</context> <context id=’date_ymd’>1999-12-21</context> | May need to translate dates
 | IDS_STRING102 | <context id=’date_ym’>99-12</context><context id=’date_ym’>1999.12</context><context id=’date_ym’>99/12</context> | May need to translate dates
 | IDS_STRING103 | <context id=’date_my’>12-99</context><context id=’date_my’>12.1999</context><context id=’date_my’>12/99</context> | May need to translate dates
 | IDS_STRING104 | <context id=’date_dm’>21.12</context> <context id=’date_dm’>21-12</context> <context id=’date_dm’>21/12</context>” | May need to translate dates
 | IDS_STRING105 | <context id=’date_md’>12-21</context> <context id=’date_md’>12.21</context> <context id=’date_md’>12/21</context> | May need to translate dates
 | IDS_STRING106 | <context ID = ‘date_year’> 1999</context> <context ID = ‘date_year’> 2001</context> | May need to translate dates
 | IDS_STRING107 | <context id=’time’>12:30:10</context><context id=’time’>12:30</context><context id=’time’>1’21”</context> | May need to translate times
 | IDS_STRING108 | <context id=’number_cardinal’>3432</context> | May need to translate number
 | IDS_STRING109 | <context id=’number_digit’>3432</context> | May need to translate number
 | IDS_STRING110 | <context id=’number_fraction’>3/15</context> | May need to translate number
 | IDS_STRING111 | <context id=’number_decimal’>423.12433</context> | May need to translate number
 | IDS_STRING112 | <context id=’phone_number’>(425)706-2693</context> | May need to translate phone number
 | IDS_STRING113 | <context id=’currency’>$12312.90</context> | May need to translate currency
 | IDS_STRING116 | <context ID = ‘address’>One Microsoft Way, Redmond, WA, 98052</context> | May need to translate address
 | IDS_STRING117 | <context ID = ‘address_postal’> A2C 4X5</context> | May need to translate postal address
 | IDS_STRING118 | <CONTEXT ID = ‘MS_My_Context’> text </CONTEXT> | Translate “text”
Multiple-Instance Test | IDS_STRING41 | Hello <SILENCE MSEC = “%d”/> World | Translate “Hello World”
 | IDS_STRING43 | <PRON SYM = “aa n th ow p ow l iw jh iy”> hello </PRON> | Translate SYM phonemes and “hello”
 | IDS_STRING44 | <RATE SPEED = “%d”> hello </RATE> | Translate “Hello”
 | IDS_STRING45 | <VOLUME LEVEL = ‘%d’> hello </VOLUME> | Translate “Hello”
 | IDS_STRING46 | <PITCH MIDDLE = ‘%d’> hello </PITCH> | Translate “hello”
 | IDS_STRING8 | Hello <BOOKMARK MARK= “12”>World | Translate “Hello World”
 | IDS_STRING27 |  |
 | IDS_STRING67 |  |
 | IDS_STRING76 |  |
 | IDS_STRING94 |  |
 | IDS_STRING119 | My name is Frances Sandy Joseph Hill Alam Jr. I was born in <context id=’date_mdy’>12/1/2006</context>, <context id=’TIME’>12:30:11</context>pm on 123 St., St. Luis. I weight 6.23456789 lb I got my B.S., M.S., and Ph.D from Flintstone Univ. in Mars planet. I have created many 3D pictures. I have to pay $123123123123.5 tax every year. It costs about 85% of my income. Anyway, life is a box of M & M chocolate. Usually I watch DVD and listen MTV on VH1 between 3pm to 12am w/o Mathew. If you like, Go to http://www.DoesNotExist.com/ and send your comments to ttsengines@hotmail.com or call me at <context id=’phone_number’>(888)888-CONTEXT</context>, or mail to: <context ID=’address’>1845666 9th Ave. NE Earth, WA 98188 </context>. | Translate “My name is Frances Sandy Joseph Hill Alam Jr. I was born in 12/1/2006 12:30:11 pm on 123 St., St. Luis. I weight 6.23456789 lb I got my B.S., M.S., and Ph.D from Flintstone Univ. in Mars planet. I have created many 3D pictures. I have to pay $123123123123.5 tax every year. It costs about 85% of my income. Anyway, life is a box of M & M chocolate. Usually I watch DVD and listen MTV on VH1 between 3pm to 12am w/o Mathew. If you like, Go to http://www.DoesNotExist.com/ and send your comments to ttsengines@hotmail.com or call me at (888)888-CONTEXT, or mail to: 1845666 9th Ave. NE Earth, WA 98188.”

Table 19: Strings to be localized for compliance tests

It is important to note that for languages which do not have a SAPI standard phoneme set (i.e. languages which are not supported in this SAPI release), the engine will fail the following required compliance tests:

Test | Result
Lexicon |
User Lexicon | Fail
App Lexicon | Fail
XML Tags |
Pron | Fail

Table 21: Required Compliance Tests Failed

The engine will also fail the following feature tests:

Test | Result
Phoneme events | UNSUPPORTED
Viseme events | UNSUPPORTED
XML PartOfSp | UNSUPPORTED

Table 22: Feature Compliance Tests Not Supported

8.5 OS Language Incompatibility

The compliance tests are not based on the language of the OS. They are based on the first language in the token of the default engine. In other words, a Japanese engine on an English OS will cause the compliance tests to load the Japanese resources and run the compliance tests expecting a Japanese engine. In order to run the compliance test for an engine that has a different language than the OS, you will need to set the default engine through the control panel.

[1] For SR Simplified Chinese, the .wav files, cfg files, and resources do not ship with the SAPI 5.0 SDK. To request these localized files, please send mail to sapi5@microsoft.com.

[2] All SR tests use the default recognition profile. To increase the accuracy of the tests, you may wish to change the .wav files used (Refer to Section 7.4) or train the reconition profile.

[3] The Simplified Chinese SR resource files, cfg files and .wav files are not included in the SAPI 5 SDK. Please e-mail sapi5@microsoft.com to obtain these files.

[4] These wave files MUST have different names than the original wave files. The new wave file names should be reflected in the string table.

[5] A basic assumption of the compliance tests is that the CFG files are included in the dll whereas the wave files are external.


Microsoft Speech SDK Setup Whitepaper

Introduction

The Microsoft Speech SDK setup is built on the Windows Installer technology. The SAPI 5.0 core components are only available for distribution through Windows Installer merge modules and are called .msm files. These .msm files should be included in an applications setup and packaged in a Microsoft Installer .msi file. The .msi files consume the merge modules and handle the actual installation process. All other application specific installable files should be included in this module. The .msi is run by the setup.exe file. The setup.exe file determines if the Windows Installer is present on the user's system and installs it if necessary. SAPI 5.0 is redistributable by ISVs or individuals by including the Speech SDK MSM's in their setup process and using the Windows Installer merge module technologies.

The following topics are covered in this section:

  • SAPI 4.0 and SAPI 5.0 co-existence
  • Speech SDK merge modules (MSM)
  • Speech SDK core modules
  • Speech Recocnition (SR) modules
  • Text to Speech (TTS) modules
  • Speech SDK modules
  • Speech SDK file locations
  • Speech SDK files
  • Quiet Install
  • Registry Settings
  • Glossary of Terms and Abbreviations

SAPI 4.0 and SAPI 5.0 co-existence

In order to ensure that SAPI 4.0 and SAPI 5.0 can co-exist on a single machine, the following four steps have been taken:

  1. SAPI 5.0 dlls have different names from SAPI 4.0 dlls
  2. SAPI 5.0 registry keys are registered in different locations
  3. SAPI 5.0 GUIDs are different from SAPI 4.0 GUIDs
  4. SAPI 5.0 registry keys are different from SAPI 4.0 registry keys

There is one known issue: SAPI 4.0 and SAPI 5.0 cannot share a common microphone. If one version of SAPI has control of the microphone, the other cannot use it. However, the functionality of TTS is not affected.

The engines that will be shipped with the Microsoft Speech SDK are shown below:

Engine | Language
SR | English
 | Japanese
 | Chinese
TTS | English
 | Chinese

return to the top of this page Back to top

Speech SDK merge modules (MSM)

The Speech SDK setup will produce merge modules for use with MSI to support the following configurations:

  • Speech SDK core modules
  • Speech Recocnition (SR) modules
  • Text to Speech (TTS) modules
  • Speech SDK modules

Speech SDK core modules

sp5.msm
The SAPI 5.0 runtime (SAPI.DLL ,SAPISVR.exe, sapi.cpl, sapi.cpl help files) only: We will make this available as a re-distributable component for ISVs, both application vendors and engine vendors. The ISV will be expected to install their engines in the proper location and make the proper registry settings for the engines. For more information, please see the Speech SDK file locations section below.
Sp5intl.msm
These modules contain localized resource .dlls needed for the control panel, sapi.cpl. (Note - Sapi.cpl contains English resources so that if none of the sp5intl.msm's are installed we default to English). The control panel is the GUI which is used to select the various speech engines and setting for a voice enabled application . The control panel has been localized for other languages. Currently, there are three separate language dependent modules, as listed below:
  • English
  • Japanese
  • Chinese (simplified)

If the number of localized languages increases, there will be separate msms for each language.

This module has a dependency on sp5.msm.

return to the top of this page Back to top

Speech Recocnition (SR) modules

Sp5sr.msm
The Microsoft SAPI 5.0 SR engine files are contained within this module (spsreng.dll and spsrx.dll). There are no speech data files located within this module, as the Microsoft SR engine is language independent. This implies that one engine can perform SR processing for multiple languages by loading different speech data files.

This module has a dependency on sp5.msm and sp5intl.msm

Sp5itn.msm
The language specific SAPI 5.0 ITN components are located within this file. The ITN modules allow developers to include Inverse Text Normalization in the SR applications. There are three separate language dependent modules:
  • English
  • Japanese
  • Chinese (simplified)

This module has a dependency on sp5sr.msm, sp5intl.msm and sp5.msm.

Sp5ccint.msm
The SAPI 5.0 command and control (C&C) module contains the SR data needed for developers that want to add command and control to their applications. Game developers that need to provide a C&C module that is small enough to be efficiently downloaded over the web, is another example of using this module. This module also contains localized resource .dlls for the Microsoft Speech Recognition Engine (spsrx.dll). These resource .dlls contain User Interfaces for the Training Wizard and Microphone Wizard. (Note - Spsrx.dll contains English resources, if none of the sp5ccint.msm's are installed we default to English). There are three separate language dependent modules:
  • English
  • Japanese
  • Chinese (simplified)

This module has a dependency on sp5sr.msm, sp5intl.msm and sp5.msm.

Sp5dcint.msm
The Microsoft SAPI 5.0 SR engine's entire acoustic and language modules are rather large files to download over the web. These will be in three separate languages dependent modules:
  • English
  • Japanese
  • Chinese (simplified)

This module has a dependency on sp5ccint.msm, sp5sr.msm, sp5intl.msm and sp5.msm.

return to the top of this page Back to top

Text to Speech (TTS) modules

Sp5ttint.msm
The Microsoft SAPI 5.0 TTS English engine is a language dependent TTS engine. This implies that the TTS engine module will contain the engine as well as the data files for the engine. Currently, Microsoft is shipping the following language TTS engines:
  • English
  • Chinese (simplified)

This module has a dependency on sp5.msm and sp5intl.msm.

spcommon.msm
Contains files that are common to both the Microsoft SAPI 5.0 TTS and SR engine. Currently, this is shipped for the following languages:
  • English

return to the top of this page Back to top

Microsoft Speech SDK modules

Sp5sdk.msi
The full installation of the Microsoft Speech SDK includes the following modules:
  • sp5.msm
  • sp5intl.msm
  • sp5sr.msm
  • sp5ccint.msm
  • sp5dcint.msm
  • Sp5ttint.msm

The Speech SDK samples and help documentation for SAPI 5.0 API/DDI will also be included the installation. However, the L&H engines are not included.

return to the top of this page Back to top

Speech SDK file locations

Setup will verify the versions of the various installed components. Setup will detect if the operating system one of the following:

Supported Operating Systems

  • Microsoft Windows(r) NT Workstation 4.0, service pack 6a, English, Japanese or Simplified Chinese edition.
  • Microsoft Windows 2000 Professional Workstation, English edition or English edition with Japanese or Simplified Chinese Language support.
  • Microsoft Windows 98. However, Windows 95 is not supported.
  • Microsoft Windows Millennium edition.

When attempting to install the Microsoft Speech SDK on a non-supported operating system, a dialogue will be presented to the user with the string "SAPI5 is currently not supported on this Operating System. You must upgrade to Windows98 or higher." After the Speech SDK has been installed on a user's machine, the footprint will not be deleted from the drive. If the user accidentally deletes the sapi.dll and then tries to run one of the applications, then the footprint file will be able to get the sapi.dll file and install it. All engine files should follow the 8.3 naming convention. NOTE: For SAPI 5.0, the user cannot install on any drive as indicated below. This may be revisited in a later release of SAPI.

SAPI.DLL
The sapi.dll file is the main dll for SAPI 5.0. This file should be independent of engine vendor and language. As a result, this file should be located in the system directory.
Control Panel
The control panel file is the file for the control panel. This file should be independent of engine vendor and language. As a result, this file should be located in the system directory.
Lexicons
The SAPI 5.0 user lexicons should be placed in the user's profile directory under the speech directory.
For example, for Windows 2000 installations, this would be Documents and Settings\<user name>\Speech.
SR Engine
The SR engines may be installed on any drive on the user's computer. The SR engine is language independent (i.e. the same engine is loaded with different data to create a different language engine). As a result, the SR engine files by default should be located in a language independent path, as follows:

Microsoft SR Engine

The SR engines may be installed on any drive on the user's computer. The SR engine is language independent (i.e. the same engine is loaded with different data to create a different language engine). As a result, the SR engine files by default should be located in a language independent path, as follows: program files\common files\speechengines\Microsoft\sr

The SR data files contain the language specific information. The SR data files (including the files needed for C&C and ITN) should be located in the following path: program files\common files\speechengines\Microsoft\sr\<LCID> where the <LCID> is 1033 (English), 2052 (Chinese (simplified)) and 1041 (Japanese)

TTS Engine

Microsoft TTS Engine

The TTS engines may be installed on any drive on the user's computer. The Microsoft TTS engine is not language independent (i.e. currently, each language is based on a different TTS code base). As a result, the TTS engine by default should be placed under the LCID it represents. The TTS engine files should be located in the following path: program files\common files\speechengines\Microsoft\TTS\LCID where the LCID is 1033 (English), 2052 (Chinese (simplified)) and 1041 (Japanese)

return to the top of this page Back to top

Speech SDK files

The Microsoft Speech SDK contains a number of samples and tools. These samples should be located in the following directories:

Executable Files
All compiled executables of the samples and the grammar compiler are located in the following directory: \Microsoft Speech SDK5.0\bin
Help Documentation
The reference file SAPI5SDK.chm is located in: \Microsoft Speech SDK5.0\docs\help
SAPI 5.0 IDL
The sapi.idl contains all of the API function declarations in SAPI 5.0. This is the main file used by application developers when developing speech enabled applications. \Microsoft Speech SDK5.0\idl
SAPI5ddk idl
The sapi5ddk.idl contains the DDI function declarations for SAPI. This is the main file used by engine developers when developing speech engines. \Microsoft Speech SDK5.0\idl
Header files
The header files for SAPI 5.0 should be located in: \Microsoft Speech SDK5.0\include
Misc
The following folder contains the sapiguid.lib \Microsoft Speech SDK5.0\lib\i386
Samples
The following table outlines the location of the source code for the various samples application and tools.
Name | Path
Dictation Pad | \Microsoft Speech SDK5.0\samples\cpp\dictpad
Simple Dictation | \Microsoft Speech SDK5.0\samples\cpp\simpledic
TTSApp | \Microsoft Speech SDK5.0\samples\cpp\TTSApp
Tutorial - Coffee S0 | \Microsoft Speech SDK5.0\samples\cpp\tutorials\CoffeeS0
Tutorial - Coffee S1 | \Microsoft Speech SDK5.0\samples\cpp\tutorials\CoffeeS1
Tutorial - Coffee S2 | \Microsoft Speech SDK5.0\samples\cpp\tutorials\CoffeeS2
Tutorial - Coffee S3 | \Microsoft Speech SDK5.0\samples\cpp\tutorials\CoffeeS3
Tutorial - Coffee S4 | \Microsoft Speech SDK5.0\samples\cpp\tutorials\CoffeeS4
Tutorial - Coffee S5 | \Microsoft Speech SDK5.0\samples\cpp\tutorials\CoffeeS5
Tutorial - Coffee S6 | \Microsoft Speech SDK5.0\samples\cpp\tutorials\CoffeeS6
Talkback | \Microsoft Speech SDK5.0\samples\cpp\talkback
Telephony Application | \Microsoft Speech SDK5.0\samples\cpp\telephony
SR Engine (Null engine) | \Microsoft Speech SDK5.0\samples\cpp\engines\SR
TTS engine (Null engine) | \Microsoft Speech SDK5.0\samples\cpp\engines\TTS
SPComp | \Microsoft Speech SDK5.0\bin
SRComp | \Microsoft Speech SDK5.0\tools\comp\SR
TTSComp | \Microsoft Speech SDK5.0\tools\comp\TTS
Grammar Editor | \Microsoft Speech SDK5.0\bin
Make Voice | \Microsoft Speech SDK5.0\samples\cpp\engines\TTS\mkVoice

return to the top of this page Back to top

Quiet Install

The command used for quite install is "msiexec /i "Microsoft Speech SDK 5.0.msi" /qn". "setup.exe /S /v/qn" is also another usable option.

return to the top of this page Back to top

Registry Settings

For engine specific registry settings, please see the Object Tokens and Registry Settings white paper. All registry keys will be manually created and deleted upon installation and un-installation respectively. This means that there will be nothing in the setup procedure that will use self-registration. We will not handle the lazy init for the speech user profiles.

Setup is not able to know about the individual users who will be using speech features after installation. The lazy registration information will continue to be built up by sapi.dll at runtime.

return to the top of this page Back to top

Glossary of Terms and Abbreviations

API
Application programming interface, the "top" side of the SAPI 5.0 middleware
C&C
Command and control
CSR
Continuous speech recognition, also called dictation
DDI
Device Driver Interface. In SAPI 5.0, this is the interface speech engine providers code to (the underside of the middleware)
MSI
Microsoft Installer, also called Darwin
MSM
Microsoft Merge Module
SAPI
Microsoft Speech Application Programming Interface. SAPI 5.0 is middleware that provides an API for applications and a DDI for speech providers.
SR
Speech Recognition (includes both CSR and C&C)
TTS
Text-to-Speech, also called speech synthesis

return to the top of this page Back to top


Text Normalization

You can perform simple text normalization for voice training using the text buffer provided to the engine. Text normalization is the process of changing the input buffer that allows the engine to use preferred word units. The engine word units affect how words are expected to be pronounced as well as how they appear to you in the voice training wizard.

The text provided to the engine is called an article. An article is composed of multiple phrases, each separated by a new-line character. The voice training wizard displays one phrase at a time.

<article> ::= { <phrase> "\n" }

A phrase is a sequence of word units, separated by white space characters. In this context, white space characters are all characters for which the C runtime function iswspace() returns TRUE.

<phrase> ::= { <word> | <literal_symbols> | <numeric_expression> }

Word Units

Literals

The following symbols are recognized as units. They should be separated from adjacent text with white space; they will "snuggle" to the words appropriately when presented to the user.

<literal_symbols> ::=
"!\exclamation-point" | "\"\end-quote" | "\"\quote" | "#\pound-sign" | "$\dollar" | "%\percent" | "&\ampersand" | "'\end-quote" | "'\quote" | "(\paren" | ")\close-paren" | "*\asterisk" | "+\plus" | ",\comma" | "--\double-dash" | "-\hyphen" | "...\ellipsis" | ".\dot" | ".\period" | "/\slash" | ":\colon" | ";\semicolon" | "<\less-than" | "=\equals" | ">\greater-than" | "?\question-mark" | "@\at-sign" | "[\bracket" | "\\back-slash" | "]\close-bracket" | "^\circumflex" | "_\underscore" | "`\back-quote" | "{\left-brace" | "| \vertical-bar" | "}\right-brace" | "~\tilde"

Numerics

Numbers can be the following form:

<digit> ::= "0"-"9"

<non_zero_digit> ::= "1"-"9"

<numeric_expression> ::= <integer_expression> | <integer_expression> <cardinal_suffix> | <floating_expression>

<integer_expression> ::= ["-"] <non_zero_digit>[<digit>[<digit>]] { [","] <digit><digit><digit> }

<floating_expression> ::= <integer_expression> "." <digit> [{ <digit> }]

<cardinal_suffix> ::= "st" | "nd" | "rd" | "th"

Collections

The remainder of the buffer will be treated as a collection of words:

<alpha_char> ::= "a"-"z"| "A"-"Z"

<word_char> ::= <alpha_char> | "-" | "_" | "0"-"9"

<word> ::= <word_0> | <word_1> | <word_2> | <word_3>

<word0> ::= <alpha_char> [{<word_char>}]

<word1> ::= <alpha_char> [{<word_char>}] "s'"|"in'"

<word2> ::= <alpha_char> [{<word_char>}] "." <word2>

<word3> ::= <abbreviation_string> "."

<abbreviation_string> ::=
"al" | "apr" | "assn" | "assoc" | "atty" | "aug" | "bef" | "bldg" | "ch" | "chg" | "co" | "com" | "cont" | "corp" | "dec" | "def" | "det" | "dev" | "div" | "doc" | "etc" | "ext" | "feb" | "gov" | "in" | "ins" | "int" | "intl" | "jan" | "jr" | "jul" | "jun" | "mar" | "messrs" | "mos" | "mph" | "mr" | "mrs" | "ms" | "mt" | "no" | "nov" | "oct" | "oz" | "par" | "pct" | "pfc" | "pp" | "pres" | "prov" | "pt" | "qtr" | "ref" | "reg" | "rep" | "rev" | "sdn" | "sec" | "sep" | "sq" | "sr" | "tech" | "vol" | "wm"


Welcome to the Microsoft Speech SDK

Welcome to the Microsoft Speech SDK, version 5.0


SAPI text to speech

 Intelligent Interface Technologies Home Page Microsoft Speech SDK

Text-to-Speech Engine Manager

The following section covers:

  • ISpTTSEngine
  • ISpTTSEngineSite


XML Schema

 The Microsoft.com Speech website

SAPI 5.0 XML Schema

The SAPI 5.0 provides support for the following XML tags and their associated attributes.

The following grammar elements are defined in the grammar.xsd file.

Elements | Attributes
<DEFINE> | DISP
<DICTATION> | DYNAMIC
<GRAMMAR>(document element) | EXPORT
<ID> | ID
<L> | INTERPRETER
<LIST> | LANGID
<O> | LEXDELIMITER
<OPT> | MAX
<P> | MIN
<PHRASE> | NAME
<RESOURCE> | OBJECT
<RULE> | PRON
<RULEREF> | PROPID
<TEXTBUFFER> | PROPNAME
<WILDCARD> | REFID
 | TEMPLATE
 | TOPLEVEL
 | URL
 | VAL
 | VALSTR
 | WEIGHT
 | WORDTYPE
Schema Attributes Reference, Grammar | Datatype Reference, Grammar

The following elements are defined in the sapi.xsd file.

Elements | Attributes
<BOOKMARK> | ABSMIDDLE
<CONTEXT> | ABSSPEED
<EMPH> | ID
<LANG> | LANGID
<PARTOFSP> | LEVEL
<PITCH> | MARK
<PRON> | MIDDLE
<RATE> | MSEC
<SAPI> (document element) | OPTIONAL
<SILENCE> | PART
<SPELL> | REQUIRED
<VOICE> | SPEED
<VOLUME> | SYM
Schema Attributes Reference, Sapi | Datatype Reference, Sapi