KB119: Advanced debugging techniques for the Character Server
While the Character Builder is a great way to generate raw character animation, it is also a
platform for building characters that respond to buttons, speak dynamic text from a database,
and even chat with users on the web. This article contains some tips for debugging solutions
based on the Character Server.
The most important thing to do when debugging an application that involves the Character Server is to
ensure that extended logging is turned on in the CharacterServer.ini file (it is turned on by default).
This will result in an Extended.log file containing detailed input, output, and timing information.
Typically you will clear the entire contents of the log directory, then run a test, and then view the contents
of the extended log.
If Extended.log is missing, even after you run the test, then your Builder-generated Flash is likely not pointing to the
correct URL, or the Character Server is not set up correctly. Be sure that the self-test in the Character Server Readme
is operational, and also be mindful of the fact that the Builder-generated Flash will always access the cs.exe CGI process
on the same domain as the domain that the Flash file resides on, even if a different domain is specified in the External Message
Build External Message URL dialog.
In applications involving dynamic speech, a failed request may result in a character that appears "stuck". Typically the
Extended.log, and in this case also the Error.log, will contain an error that describes the problem, such as a missing character
If the Character Server works, but appears slow, or has a watermark, then the Extended.log file may also be able to tell you why - it
will insert warnings if the server license is incorrect, or if the number of character licenses has been exceeded and execution has
been delayed. There will be no warning in the log if the number of Text-to-Speech licenses has been exceeded, but this will manifest
itself as significantly longer processing times on the second of two back-to-back Speech Server requests. You can see the processing
times for individual stages:
01/12/2011 09:02:31 (1) -Speak-> (request SRQC5 - C:\Program Files\Character Server\Wave\A553938.wav)
Microsoft Mike: I am ready for your questions.
01/12/2011 09:02:31 (1) <-Speak- (request SRQC5 - C:\Program Files\Character Server\Wave\A553938.wav
- elapsed 16 queue + 30 processing = 46 ms)
This trace indicates that a Speech Server request was received and processed on channel 1 (more on channels in the Server Guide). The request took a total of 46ms.
If a first request returns within 100ms, but a second request, soon after, takes, say, 3 seconds, then you will know that the voice is being
port-limited. This can be addressed by upgrading the speech engine to add a second port. This is only an issue with higher quality
voices, as only these tend to be port-limited, but you can also simulate port-limiting on free voices in order to do performance testing.
You may also see other .tmp files appear in the Logs directory, as this directory also serves as the "scratchpad" where the
various components communicate their requests. For example a file named SRQC5.tmp will be created temporarily during
execution of the trace shown above. If you see a buildup of SRQ (Speech Request) files then the Speech Server service may
be stopped. If you see a buildup of ARQ (AI Request) files, then the AI service may be stopped. CRQ (Character Request)
files are generated for each cs.exe request.
If you suspect that a problem is occurring on the client-side, then you can install the Flash Debug Player (aka Content Debugger). The
Flash Debug Player is a version of Flash that allows you to obtain a debug trace of the operation of your Character
Builder-generated Flash file(s). You can normally find this download at http://www.adobe.com/support/flashplayer/downloads.html.
You may need to uninstall your current Flash player before you can install the Debug Flash Player, and you will need
to restart the browser.
You can then verify that you have the Debug Player installed by right-clicking on a swf. You should have the
"Debug" option in the context menu.
Once the Debug Flash Player is installed, there are additional steps you can take to enable tracing in the mm.cfg file. These
steps are error-prone, and vary depending on the version of Windows installed. We recommend
a free tool called Vizzy that will both configure your Debug Player and let you view the Flash log output.
The tool is available at http://code.google.com/p/flash-tracer/.
Finally, before seeing any debug output, you will need to re-render your project using the Debug loader.
Click the Advanced tab on the Project object, and select Debug as the Loader setting, then Render to Flash.
After running your project you can see a trace of the loader. The trace is somewhat
verbose - to get a full understanding of the trace you will want to see the Loader source files, which are available
as part of the Extensibility Kit.
If you are rendering to HTML5, then you can also use your browser's "developer" option to obtain a log window. By default
only high-level messages and errors are logged, but a more detailed trace is also available by recompiling your project using
the the Debug setting above.
There are several levels of caching that help boost the throughput of the Character Server, however caching can also
mask or exacerbate problems during development. For example a speech request might return with an audio "license warning" message,
even though you have just added licensing for that speech engine. The likely cause is that the Character Server is happily
reusing a previously-generated audio file from the Wave directory.
There are 5 types of caching that you should be aware of:
- Client-side cache. Requests that arrive at the client
are normally cached locally by the browser for a few minutes minutes. This allows a client to queue up requests to the
Character Server so that playback overlaps with generation.
- Flash cache. Requests with the same parameters will normally result in no extra work on the server since the
previous flash file is returned from the Flash cache. You can clear the Flash cache by
deleting all the files in the Flash subdirectory of the Character Server.
- Audio cache. Speech requests with the same parameters will normally result in no extra TTS work since the
previous audio (.wav) file is returned from the audio cache, in the Wave directory. You can clear the audio cache by
deleting all the files in the Wave subdirectory of the Character Server. You should do this after applying speech engine licensing, as the
Character Server is unable to determine if the license status of a speech engine has changed.
- Image cache. Realistic and Photofit characters ship with a smaller database of images that can be just-in-time
recolored and layered to produce a given custom appearance. These files are placed in an image cache to make the operation
of a character with a particular set of customizations comparable in speed to that of a non-customizable character, once
the image cache is populated. You can clear the image cache by
deleting all the files in the Image subdirectory of the Character Server.
- HTML5 cache. The HTML5 cache works like the Flash cache, but for HTML5 generation. It contains audio, image, and js (data) files. You can clear the HTML5 cache by
deleting all the files in the HTML5 subdirectory of the Character Server.