Speaking of Standards

For those tracking the the “RTCWEB/WebRTC” initiative the next “virtual interim meeting” for the IETF side of the effort will be held on Thursday, September 8th. From the announcement email:

The RTCWEB working group plans to hold an interim meeting meeting. It will be held on September 8, 2011 at 7:00 A.M. PDT. We’re currently allocating 4 hours for the meeting. Details of participation will follow on the working group mailing list (http://www.ietf.org/mail-archive/web/rtcweb/).

If you are already on the mailing list, you’ll get the announcements and agenda… if not, it’s a good time to join the mailing list!

P.S. And if you don’t know what any of this is about, see my last post.

Sadly, this XKCD comic can seem all too true sometimes…

In just 13 days, the 81st meeting of the Internet Engineering Task Force (IETF) will kick off up in the beautiful world of Quebec City in Canada. It will be the usual jam-packed week of meetings on the very wide range of topics covered by the IETF.

Of particular interest to those of us in the real-time communications area will be the ongoing discussions of the RTCWEB initiative that is looking at how to bring real-time communications directly into web browsers.

There is still time to register to attend IETF 81 if you have not already done so.

Voxeo’s head of standards, Dan Burnett, will be attending on our behalf. Please contact him if you would like to meet up with him at the event. You can expect to find him in most of the various SIP and RAI events. (Sadly, my schedule does not permit me (Dan York) to attend this event… I’m looking forward to getting back to one of these IETF meetings soon!)

Want to learn about how IPv6 affects telecommunications? Want to understand how the SIP protocol is impacted by IPv6? If so, join in this free webinar:

IPv6 and Telecom Networks
Thursday, June 2, 2011
1:00pm US Eastern

Registration is free.

The webinar is hosted by US Telecom and the description is:

The networks that make up the Internet and IP communications are in the middle of a sea-change with the transition to IPv6. What impact will IPv6 have on telecom and communications networks?

Join USTelecom and Voxeo for a look at the various challenges that telecom and broadband services providers face in keeping their communication services working while transitioning to IPv6.

I’ll be explaining briefly why there is all the attention on IPv6 then getting into the basics of IPv6 addressing. After a brief overview, I’ll then dive into how IPv6 affects the Session Initiation Protocol (SIP) and get into some technical detail. I’ll then wrap up with some resources about how to learn more and get started with IPv6 and finish with a Q&A session.

If you attended the Developer Jam Session I presented back in May on IPv6, I’m going to be covering basically the same material although with a vendor-neutral perspective (i.e. I won’t be explaining and demonstrating how Voxeo Prophecy and PRISM now natively support IPv6). Obviously the live Q&A session will be new, too, and I find the questions around IPv6 always quite fun to discuss.

Please feel free to join us at 1pm US Eastern tomorrow. Registration is free – and if you can’t join live the session will be archived and available for viewing on US Telecom’s website for 90 days. With World IPv6 Day coming up on June 8th, it’s a great time to learn about what is going on with IPv6!

P.S. And if you are interested in IPv6 in general, don’t forget to check out our IPv6 Resource Page at:

http://bit.ly/voxeoipv6

Yesterday the W3C published a Proposed Recommendation for “Voice Browser Call Control: CCXML Version 1.0” and also asked for any final comments to be received by June 10, 2011. The CCXML Proposed Recommendation can be found at:

http://www.w3.org/TR/2011/PR-ccxml-20110510/

The list of changes in the proposed recommendation is at:

http://www.w3.org/TR/2011/PR-ccxml-20110510/#changes-pr

and the W3C has very helpfully provided a diff version showing the changes from the past revision.

Given that Voxeo CTO RJ Auburn is the Editor-in-Chief of the CCXML specification and that we do quite a good bit with CCXML (and indeed many other folks in the industry actually license our CCXML engine to use in their products), we’re very pleased to see this Proposed Recommendation come out. Within the W3C process, this is nearing the final stage where CCXML will become a full “Recommendation”. Kudos to RJ and all the folks involved with the W3C Voice Browser Working Group for moving the specification along to where it is.

If you have never worked with CCXML, we have a lengthy set of CCXML tutorials over on our documentation site and have also written a good bit about CCXML over the years on our Voxeo Developers Corner blog. If you aren’t familiar with “state machines”, you might want to start with the post I wrote about state machines and CCXML (and James Bond) as that may help explain a bit about how CCXML works.

You can of course try out CCXML for free either in our hosted Evolution developer portal or by downloading Prophecy to your local system (for Windows, Linux or Mac OS X).

Besides using MediaGroup to provide multimedia functions, another common use of media server is multi-party conference calls. MediaMixer is the object in Java Media Control that provides sound mixing capability.

Overview

Similar to MediaGroup, a MediaMixer is a Joinable MediaObject. To connect an endpoint to MediaMixer, simply joining the endpoint’s NetworkConnection to the MediaMixer object, as illustrated in the following diagram from JSR 309 specification.

NC1.join(DUPLEX, theMediaMixer);
NC2.join(DUPLEX, theMediaMixer);
NC3.join(DUPLEX, theMediaMixer);

Sample Application

Here is a simple conference application. When a call comes in, the application asks the caller to input a conference number (max 9 digits). Based on the conference number, the application either create or locate a MediaMixer. Then join the caller’s NetworkConnection to the MediaMixer.

The following is a simplified sequence diagram that shows the interaction among different objects.

Participant Object

The Participant is an abstraction that capture all the signaling and media states and operations on the per caller (conference participant) basis. This is a design pattern I often use in SIP Servlet and Java Media Control based applications.

When an initial SIP INVITE message comes, the MainServlet creates a new Participant object. During the construction, Participant initializes a MediaSession and create a NetworkConnection and a MediaGroup from the MediaSession. Then Participant uses SdpPortManager of the NetworkConnection to negotiate the SDP with the client. See SDP Negotiation for more information about the SdpPortManager.

Collect Conference ID

Once the call is setup, Participant asks the caller to input the conference ID. Instead of using Player to ask the question and use SignalDetector to collect input, I use the PROMPT in the SignalDetector to ask and collect in a single receiveSignal call, as shown below.

 SignalDetector detector = _mg.getSignalDetector();
 Parameters options = _factory.createParameters();
 options.put(SignalDetector.INTER_SIG_TIMEOUT, 5000);
 options.put(SignalDetector.PROMPT, URI.create("data:" + ANNOUCEMENT));
 detector.receiveSignals(9, SignalDetector.NO_PATTERN, RTC.NO_RTC, options);

I also set INTER_SIG_TIMEOUT to 5 seconds. This allows SignalDetector to collect any conference ID that is less 9 digits long when the user stops inputting for 5 seconds. Otherwise, SignalDetector will keep waiting until the user inputs 9 digits.

Join the Conference

Once the conference ID is collected, Participant calls MixerManager to create a MediaMixer based on the ID. The MixerManager maintains a mapping between ID and MediaMixer and only creates a MediaMixer when needed.

 public synchronized MediaMixer createMixer(Configuration config, Parameters options, String key) throws MsControlException {
   MediaMixer mixer = _mixers.get(key);
   if (mixer == null) {
      MediaSession session = _factory.createMediaSession();
      mixer = session.createMediaMixer(config, options);
      _mixers.put(key, mixer);
    }
    return mixer;
 }

Once Participant obtains the MediaMixer, it will join its NetworkConnection to the MediaMixer.

  public void join(String id) throws MsControlException {
    _mixer = _mgr.createMixer(MediaMixer.AUDIO, Parameters.NO_PARAMETER, id);
    _id = id;
    _nc.join(Direction.DUPLEX, _mixer);
  }

Leave the Conference

When the caller hangs up the call, SIP BYE is received by the MainServlet. Participant, retrieved from the SipSession, unjoin its NetworkConnection from the MediaMixer.

  public void unjoin() throws MsControlException {
    if (_mixer != null) {
      _mixer.unjoin(_nc);
      _ms.release();
      _mgr.removeMixer(_id);
    }
  }

The MixerManager only removes the MediaMixer when no Participant is joined.

  public synchronized MediaMixer removeMixer(String key) throws MsControlException {
    MediaMixer mixer = getMixer(key);
    if (mixer.getJoinees().length == 0) {
      _mixers.remove(key);
      mixer.release();
      mixer.getMediaSession().release();
      return mixer;
    }
    return null;
  }

SipListener

I implemented a SipListener as both SipSessionListener and SipErrorListener to handle the following events.

• SipErrorListener.noAckRecieved. Because MediaSession and related resources are created when INVITE is created, we need clean it up when call setup is failed because of no ACK.
• SipSessionListener.sessionDestroyed. Similar, we need clean up  the MediaSession when the SipSession is destroyed.

Test Sample

As other samples, simply download the WAR and deployed it under <prism>/apps directory. You can use two softphones dialing into the server to form a two party conference call. Please note Voxeo Prism free trial only comes with 2 media ports. Please contact Voxeo Support to request additional licenses.

Upon the completion of SDP negotiation, the media server essentially gets a media pipe to the client, represented as the NetworkConnection. Now, to have some fun, we need to connect that pipe to a multimedia studio that can play, record, and recognize different sound.

MediaGroup

This multimedia studio is called MediaGroup in Java Media Control API.  MediaGroup contains four Resources — Player, Recorder, SignalDetector, and SignalGenerator. Once a MediaGroup is connected to a NetworkConnection, you can use these Resources to, e.g. play music to the client.

MediaGroup can be created from a MediaSession based on a Configuration and, optionally, Parameters.

MediaSession.createMediaGroup(Configuration);

The Configuration indicates to the media server what Resources that the application might be using (or not using). Thus the media server can allocate appropriate resources for this MediaGroup. Typically you will use one of the pre-defined Configurations

• PLAYER: Use this when the application will only use Player in this MediaGroup.
• PLAYER_SIGNALDETECTOR: Use this when the application will only use Player and SignalDetector in this MediaGroup.
• PLAYER_RECORDER_SIGNALDETECTOR: Use this when the application will use all Resources but SignalGenerator in this MediaGroup.
• PLAYER_RECORDER_SIGNALDETECTOR_SIGNALGENERATOR: Use this when the application will use all Resources in this MediaGroup.

To use the MediaGroup, simply joining the NetworkConnection with the MediaGroup. (See Media Object Composition about join concept.) Now you can provide multimedia functions for the client by using different Resource in the MediaGroup.

NetworkConnection nc = ...
MediaGroup mg = session.createMediaGroup(MediaGroup.PLAYER_RECORDER_SIGNALDETECTOR);
nc.join(mg);
mg.getPlayer().play(URI.create("http://myserver.com/mysong.wav"), RTC.NO_RTC, Parameters.NO_PARAMETER);

Let’s take a look what you can do with each Resource in the MediaGroup.

Player

Player can be used to play media streams from a URI (or a set of URIs). For example, the following code snippet shows how to play a song from a Web server.

mg.getPlayer().play(URI.create("http://myserver.com/mysong.wav"), RTC.NO_RTC, Parameters.NO_PARAMETER);

If the media server supports text-to-speech, you can use SSML to render text into synthesized speech. For example,

final static String HELLO_WORLD_SSML = URLEncoder.encode("application/ssml+xml, <?xml version=\"1.0\"?><speak><voice>Hello World!</voice></speak>", "UTF-8");
mg.getPlayer().play(URI.create("data:" + HELLO_WORLD_SSML), RTC.NO_RTC, Parameters.NO_PARAMETER);

You can ask the Player to play multiple items at once by giving an array of URIs.

To stop playing, simply call stop operation to stop either the current item being played or all the items in the queue.

You can also pause and resume the play, as well as adjusting speed and volume.

Recorder

Recorder can be used to record media to URI. For example, the following code snippet shows how to record your voice into a file on a Web server.

mg.getRecorder().record(URI.create("http://myserver.com/mysong.wav"), RTC.NO_RTC, Parameters.NO_PARAMETER);

To stop recording, simply call stop operation.

You can also pause and resume the recording.

SignalDetector

SignalDetector can be used to collect user input based on a set of patterns. For example, the following code snippet tries to receive one DTMF input from the user.

mg.getSignalDetector().receiveSignals(1, SignalDetector.NO_PATTERN, RTC.NO_RTC, Parameters.NO_PARAMETER);

The use of Parameter and pattern label in Java Media Control API is a bit akward. I will explain more in the future blog.

Similarly, you can call stop() to stop the receiveSignal operation.

SignalGenerator

SignalGenerator can be used to generate signal into the media server. This is typically used when the DTMF input are received via SIP INFO.

Sample Application

Here is a sample application that will announce “Hello World” after you dial into it. Then it will ask you to input “1″ for replay. Any other key input will determinate the call.

To run this sample, assuming you have Voxeo Prism downloaded and installed, simply drop the Tutorial.3.war to <prism>/apps directory. Once you have Voxeo Prism (both Application Server and Media Server) started, simply dial “sip:user@localhost” from your SIP phone such as SJPhone or XLite.

Source code is included in the sample .

To continue my Java Media Control blogs, before we can talk about media operations, we need first talk about media object composition concept. Media objects are composed together for media processing and functions. E.g. a NetworkConnection, which represents the client, connects to a MediaMixer to join a conference.

Join

A composable media object is called Joinable in Java Media Control API. When a joiner joins with a joinee, the media streams are connected between them. You can even specify the directions of the media streams when joining.

• DUPLEX means the media streams can flow both ways.
• RECV means the media streams can flow from joinee to joiner only.
• SEND means the media streams can flow from joiner to joinee only.

Here is code sinppet that shows how to join a NetworkConnection and a MediaGroup.

      NetworkConnection nc = ....
      MediaGroup mg = ....
      nc.join(Joinable.Direction.DUPLEX, mg));

To decompose, you simply unjoin the joinee from the joiner.

Asynchronous Join

join won’t return until the composition is completed. If you don’t want to wait for the the completion, you can use the asynchronous joinInitiate method, which simply initiates the composition. A Joinable object that supports asynchronous join must be a JoinEventNotifiers so it can send the composition result back as a JoinEvent. This is very similar to media event model, as illustrated in the following digram.

Similarly, you can also use asynchronous unjoinInitiate method to start the decomposition and get notified by the event for the decomposition result.

Re-Join

A pair of joined Joinables can be re-joined, potentially change the existing media stream directions.

E.g. the following code snippet shows how to change a fully connected NetworkConnection to listen-only mode.

     nc.join(Joinable.Direction.DUPLEX, mg));
     ...
     nc.join(Joinable.Direction.RECV, mg));

Multiple Joins

A media object can be joined to more than one media object to form a graph of composition. However, one important rule here is any media object, except MediaMixer, can talk to many other media objects but only listen to one media object.

A simple use case of this is illustrated by the following diagram from Java Media Control API spec.

In this diagram, the Fred’s NetworkConnection NC2 joins to both Mark’s NetworkConnection NC1 and a MediaGroup. Please note, in this case, you can not use the Player of the MediaGroup since it violates the talk-to-many/listen-to-one rule as Fred can only listen to Mark right now.

Join Degradation

Please note in the above example, NC2 is explicitly joined with SEND direction with the MediaGroup. What happens if NC2 joins the MediaGroup with DUPLEX? I.e.

      NC1.join(DUPLEX, NC2);
      NC2.join(DUPLEX, MGASR);

This will cause so called Join Degradation — the join between NC1 and NC2 will be degraded from DUPLEX to RECV only. I.e. Fred can speak to Mark but not heard from him. Rather Fred will hear from whatever plays in the MediaGroup. The Join Degradation basically means the last joined DUPLEX pair wins and all the previous joined pairs will change according to the talk-to-many/listen-to-one rule.

For those of you following the ongoing “RTCWEB” work to bring about standards for real-time communications from within web browsers, there are now two new public mailing lists that you can join – and are where all the RTCWEB activity is now intended to happen.

On the IETF side, as part of the new charter for the RTCWEB work, there is now a mailing list at:

https://www.ietf.org/mailman/listinfo/rtcweb

You can subscribe and also view the archive from that page.

On the W3C side, a working group is still forming there, but the mailing list is now up for subscription at:

http://lists.w3.org/Archives/Public/public-webrtc/

The W3C list was literally created today, so there has not yet been any activity on that list.

The work of the RTCWEB initiative is now moving into the standards bodies… and so if you want to monitor or participate in the work, you’ll need to join those mailing lists.

Carrying on a fine tradition, the IETF today released RFC 6214, which adapts the venerable RFC 1149 (from 1990) for IPv6.

If you are not aware, RFC 1149 defines, of course, IP-over-carrier-pigeon.

RFC 6214 does outline some of the challenges with IPv6 with regard to avian carriers:

As noted in RFC 1149, the MTU is variable, and generally increases with increased carrier age. Since the minimum link MTU allowed by RFC 2460 is 1280 octets, this means that older carriers MUST be used for IPv6. RFC 1149 does not provide exact conversion factors between age and milligrams, or between milligrams and octets. These conversion factors are implementation dependent, but as an illustrative example, we assume that the 256 milligram MTU suggested in RFC 1149 corresponds to an MTU of 576 octets. In that case, the typical MTU for the present specification will be at least 256*1280/576, which is approximately 569 milligrams. Again as an illustrative example, this is likely to require a carrier age of at least 365 days.

Furthermore, the MTU issues are non-linear with carrier age. That is, a young carrier can only carry small payloads, an adult carrier can carry jumbograms [RFC2675], and an elderly carrier can again carry only smaller payloads. There is also an effect on transit time depending on carrier age, affecting bandwidth-delay product and hence the performance of TCP.

I did very much enjoy this part of section 5:

Routing carriers through the territory of similar carriers, without peering agreements, will sometimes cause abrupt route changes, looping packets, and out-of-order delivery. Similarly, routing carriers through the territory of predatory carriers may potentially cause severe packet loss. It is strongly recommended that these factors be considered in the routing algorithm used to create carrier routing tables. Implementers should consider policy-based routing to ensure reliable packet delivery by routing around areas where territorial and predatory carriers are prevalent.

There is evidence that some carriers have a propensity to eat other carriers and then carry the eaten payloads. Perhaps this provides a new way to tunnel an IPv4 packet in an IPv6 payload, or vice versa. However, the decapsulation mechanism is unclear at the time of this writing.

All in all a fun bit of work… enjoy!