Expect to spend time and effort in testing and refining the voice recognition features of a 511 system.
An Arizona Department of Transportation experience in 511 implementation.
Although voice recognition is a proven telephone user interface, its implementation in a 511 system can be challenging and 511 deployers should expect to spend considerable effort testing and refining the voice-recognition system. The ADOT experience provides several suggestions for 511 deployers:
- Capture user comments to identify and resolve problems with the voice-recognition system. In addition to its own testing, ADOT discovered many problems from user comments. Many users of the early ADOT voice recognition system reacted quite negatively and were able to express their frustration via the caller comment feature (a voice mailbox on the 511 menu). Most problems fell into one of two categories: (1) errors in menu system and message programming, i.e., requests for "roads" being sent to the transit menu, or information for Interstate 15 being given in response to a request for information on Interstate 10; (2) the system misinterpreting user inputs (i.e., utterances) or interpreting background noises as utterances.
- Be sure to include testing and refinement in the project schedule. ADOT found that it took considerable time to discover and correct problems. The second type of malfunction noted above, that is the system misinterpreting user utterances, proved especially challenging. Many users continued to find the voice recognition system unsatisfactory: 10 months after implementation, 35% of surveyed users were dissatisfied. ADOT found the complexity of the voice-recognition implementation resulted in between 1 and 3 months schedule delay.
- Be sure to budget for testing and refining the voice-recognition system. ADOT estimates that they spent approximately $560,000 on the voice recognition related enhancement, about 61% more than they had estimated.
- Beware that the larger the transportation system covered by 511, the more problems will be encountered with voice-recognition. As the system covered by 511 increases, especially in the number of roadways covered, so too does the number of valid utterances the system must distinguish. In this respect, large 511 systems like the statewide Arizona 511 system strain basic voice recognition performance much more than systems that limit users to a few distinct utterances at each prompt, e.g., "arrivals" or "departures" in an airline customer information system, or systems which list the options and, for each, ask the user to say "yes" or "no".
Implementing and fine-tuning voice recognition performance is one of the more challenging, labor intensive activities associated with developing or enhancing a 511 system. Deployers should allocate significant time and resources to this activity, with the understanding that the amount of effort will increase with the number of possible user inputs the system must recognize. Systems covering larger areas, such as statewide systems, will require more effort to implement and fine-tune. Systems that accept many different specific names for transportation facilities and services (rather than listing the options and asking only for users to confirm with "yes" or "no") will also require more effort.
Author: Battelle and the University of Arizona
Published By: Federal Highway Administration, U.S. DOT
Prepared by Battelle and the University of Arizona for the U.S. DOT
Source Date: 30 September 2005
EDL Number: 14248
Other Reference Number: Report No. FHWA-JPO-06-013URL: http://ntl.bts.gov/lib//jpodocs/repts_te//14248.htm
Average User Rating