UCIF's Ad Hoc Test: How'd They Do?UCIF's Ad Hoc Test: How'd They Do?
About two dozen participants uncovered some significant issues, especially when trying to communicate internationally. Still, there were also a few promising areas.
August 19, 2010
About two dozen participants uncovered some significant issues, especially when trying to communicate internationally. Still, there were also a few promising areas.
The ad hoc interoperability test staged by the UCIF (Unified Communications Interoperability Forum) on Tuesday showed, unsurprisingly, some significant areas of concern, but also a few surprisingly good results as well, according to Bernard Aboba of Microsoft, who chairs UCIF's board.
The test drew about "two dozen" participants, Bernard told me in an interview yesterday. Participants came from all over the world, which wound up revealing one of the main weaknesses in UC interoperability--what UCIF calls "internationalization" of UC functionality.
One of the major stumbling blocks in trying to connect international participants was that systems seemed to be unable to simultaneously handle right-to-left text (e.g., Arabic and Hebrew) and left-to-right (English, etc.). That issue hasn't gotten a lot of attention in the past, but Bernard told me it's something that standards bodies need to consider. "Our motto in UCIF is that UC is for everyone," and that means people with languages whose script is something other than the typical Western languages.
UCIF produced a document summarizing the results of Tuesday's test. Here's its bulleted highlights:
* Internationalization is an area which appears to require more attention if the UC industry is to reach its potential. The issues encountered (such as the mixing of RTL [right-to-left] and LTR [left-to-right] scripts) appeared to involve more than implementation bugs; some standardization work will probably be necessary.
* Mobility is an area which also requires more attention. The rapid growth in smartphone shipments implies that a significant fraction of UC endpoints will be mobile. Yet, web conferencing solutions can be difficult to utilize on a mobile device or may be dependent on plugins not available on popular mobile platforms.
* Audio, while generally more mature than other UC modes, may still encounter issues with audio devices, such as headsets and microphones.
* Instant Messaging & Presence clients may not necessarily support the entire set of core capabilities needed to carry out common tasks. Profiling the required capabilities may therefore provide value.
* Jingle, though promising, requires additional interoperability testing.
* Mobility is an area which also requires more attention. The rapid growth in smartphone shipments implies that a significant fraction of UC endpoints will be mobile. Yet, web conferencing solutions can be difficult to utilize on a mobile device or may be dependent on plugins not available on popular mobile platforms.
* Audio, while generally more mature than other UC modes, may still encounter issues with audio devices, such as headsets and microphones.
* Instant Messaging & Presence clients may not necessarily support the entire set of core capabilities needed to carry out common tasks. Profiling the required capabilities may therefore provide value.
* Jingle, though promising, requires additional interoperability testing.
Here's the document's section on "What Was Tested:"
Audio: An audio bridge (hosted by a major carrier) was available, and was tested using multiple VOIP implementations. Quality was very good. Audio was also available on the Web conferencing platforms, though audio quality was noticeably worse than on the bridge. The best audio quality was observed during a successful Jingle audio test (wideband codec was negotiated). The biggest problem during the audio tests related to devices (sound cards, headsets and microphones), as well as volume adjustment and silence detection algorithms.
Web conferencing: Two different web conferencing systems were tested with Web browsers on multiple operating systems (Windows, Mac, Linux) and Browsers (Mozilla, Chrome, Safari, IE, Opera). One conferencing platform provided all features on all browsers and PC platforms (with a plugin), but other had issues on some platform/browser combinations. None of the web conferencing solutions were usable on a mobile handset. Issues were observed with parsing of some file formats (e.g. pptx vs. ppt), in some cases cryptic error messages were generated and the presentations failed to display. Internationalization was tested to a limited extent; most browsers today support IDNA [Internationalized Domain Name] to varying degrees. Both web conferencing systems appeared to be capable of sending and receiving basic audio and "postage stamp" video. Support for mobile platforms is an area which appears to require more attention from the industry.
Instant Messaging, Presence and Multi-User Chat: A wide range of XMPP clients were tested on multiple OS platforms and services. Basic IM&P operations such as subscription, rosters and instant messaging appeared to work satisfactorily, but some discovery and setup operations could not be carried out by all clients (e.g. setting conference parameters). In a number of situations, settings required adjustment before connectivity could be achieved. This resulted in some cryptic error messages. MUC was not completely supported on some clients (e.g. one client could accept an invite to a multi-user chat, but could not send an invitation or join from the native UI).
With respect to internationalization, while some basic operations were successful (we were able to create a user with a username and password in Arabic, as well as to conduct conversations in non-latin scripts), major issues were found, such as inability to correctly display or enter characters on some implementations. In some cases, cut/paste of non-latin characters worked, but attempting to enter them on a keyboard did not. Mixing of Right to Left and Left-to-Right scripts provided some astonishing behavior.... Internationalization is an area which appears to require more attention if the UC industry is to reach its potential.
Jingle: Interoperability of Jingle was tested between multiple XMPP clients across multiple operating systems and services. Multiple failure modes were observed, including inability to negotiate video codecs, and in one situation, Jingle sessions could only be initiated in one direction, but not the other (potentially due to NAT traversal issues). Jingle audio operation was demonstrated in one direction with excellent sound quality between multiple platforms, services and clients, but attempts to upgrade to video generated unusual failure modes (e.g. video appeared to be sent but not received, receiver received no indication of video being sent, sender received no indication that video was not being received). Jingle appears to be a feature that could benefit from an increased focus on compliance and interoperability.
As Bernard told me the day before the test, this was not intended to be a comprehensive test, but, as the name suggests, ad hoc. Before the test, Bernard said they were aiming to get a general sense of whether interoperability is closer to the top or the bottom of a 2-to-10 scale. When I asked him yesterday what score he'd give based on the test, he said the differing implemenations they tried ranged from a 2 to an 8. The emphasis going forward definitely appears to be on the international issues.
Web conferencing: Two different web conferencing systems were tested with Web browsers on multiple operating systems (Windows, Mac, Linux) and Browsers (Mozilla, Chrome, Safari, IE, Opera). One conferencing platform provided all features on all browsers and PC platforms (with a plugin), but other had issues on some platform/browser combinations. None of the web conferencing solutions were usable on a mobile handset. Issues were observed with parsing of some file formats (e.g. pptx vs. ppt), in some cases cryptic error messages were generated and the presentations failed to display. Internationalization was tested to a limited extent; most browsers today support IDNA [Internationalized Domain Name] to varying degrees. Both web conferencing systems appeared to be capable of sending and receiving basic audio and "postage stamp" video. Support for mobile platforms is an area which appears to require more attention from the industry.
Instant Messaging, Presence and Multi-User Chat: A wide range of XMPP clients were tested on multiple OS platforms and services. Basic IM&P operations such as subscription, rosters and instant messaging appeared to work satisfactorily, but some discovery and setup operations could not be carried out by all clients (e.g. setting conference parameters). In a number of situations, settings required adjustment before connectivity could be achieved. This resulted in some cryptic error messages. MUC was not completely supported on some clients (e.g. one client could accept an invite to a multi-user chat, but could not send an invitation or join from the native UI).
With respect to internationalization, while some basic operations were successful (we were able to create a user with a username and password in Arabic, as well as to conduct conversations in non-latin scripts), major issues were found, such as inability to correctly display or enter characters on some implementations. In some cases, cut/paste of non-latin characters worked, but attempting to enter them on a keyboard did not. Mixing of Right to Left and Left-to-Right scripts provided some astonishing behavior.... Internationalization is an area which appears to require more attention if the UC industry is to reach its potential.
Jingle: Interoperability of Jingle was tested between multiple XMPP clients across multiple operating systems and services. Multiple failure modes were observed, including inability to negotiate video codecs, and in one situation, Jingle sessions could only be initiated in one direction, but not the other (potentially due to NAT traversal issues). Jingle audio operation was demonstrated in one direction with excellent sound quality between multiple platforms, services and clients, but attempts to upgrade to video generated unusual failure modes (e.g. video appeared to be sent but not received, receiver received no indication of video being sent, sender received no indication that video was not being received). Jingle appears to be a feature that could benefit from an increased focus on compliance and interoperability.
As Bernard told me the day before the test, this was not intended to be a comprehensive test, but, as the name suggests, ad hoc. Before the test, Bernard said they were aiming to get a general sense of whether interoperability is closer to the top or the bottom of a 2-to-10 scale. When I asked him yesterday what score he'd give based on the test, he said the differing implemenations they tried ranged from a 2 to an 8. The emphasis going forward definitely appears to be on the international issues.
