Speaking of Standards

Besides using MediaGroup to provide multimedia functions, another common use of media server is multi-party conference calls. MediaMixer is the object in Java Media Control that provides sound mixing capability.

Overview

Similar to MediaGroup, a MediaMixer is a Joinable MediaObject. To connect an endpoint to MediaMixer, simply joining the endpoint’s NetworkConnection to the MediaMixer object, as illustrated in the following diagram from JSR 309 specification.

NC1.join(DUPLEX, theMediaMixer);
NC2.join(DUPLEX, theMediaMixer);
NC3.join(DUPLEX, theMediaMixer);

Sample Application

Here is a simple conference application. When a call comes in, the application asks the caller to input a conference number (max 9 digits). Based on the conference number, the application either create or locate a MediaMixer. Then join the caller’s NetworkConnection to the MediaMixer.

The following is a simplified sequence diagram that shows the interaction among different objects.

Participant Object

The Participant is an abstraction that capture all the signaling and media states and operations on the per caller (conference participant) basis. This is a design pattern I often use in SIP Servlet and Java Media Control based applications.

When an initial SIP INVITE message comes, the MainServlet creates a new Participant object. During the construction, Participant initializes a MediaSession and create a NetworkConnection and a MediaGroup from the MediaSession. Then Participant uses SdpPortManager of the NetworkConnection to negotiate the SDP with the client. See SDP Negotiation for more information about the SdpPortManager.

Collect Conference ID

Once the call is setup, Participant asks the caller to input the conference ID. Instead of using Player to ask the question and use SignalDetector to collect input, I use the PROMPT in the SignalDetector to ask and collect in a single receiveSignal call, as shown below.

 SignalDetector detector = _mg.getSignalDetector();
 Parameters options = _factory.createParameters();
 options.put(SignalDetector.INTER_SIG_TIMEOUT, 5000);
 options.put(SignalDetector.PROMPT, URI.create("data:" + ANNOUCEMENT));
 detector.receiveSignals(9, SignalDetector.NO_PATTERN, RTC.NO_RTC, options);

I also set INTER_SIG_TIMEOUT to 5 seconds. This allows SignalDetector to collect any conference ID that is less 9 digits long when the user stops inputting for 5 seconds. Otherwise, SignalDetector will keep waiting until the user inputs 9 digits.

Join the Conference

Once the conference ID is collected, Participant calls MixerManager to create a MediaMixer based on the ID. The MixerManager maintains a mapping between ID and MediaMixer and only creates a MediaMixer when needed.

 public synchronized MediaMixer createMixer(Configuration config, Parameters options, String key) throws MsControlException {
   MediaMixer mixer = _mixers.get(key);
   if (mixer == null) {
      MediaSession session = _factory.createMediaSession();
      mixer = session.createMediaMixer(config, options);
      _mixers.put(key, mixer);
    }
    return mixer;
 }

Once Participant obtains the MediaMixer, it will join its NetworkConnection to the MediaMixer.

  public void join(String id) throws MsControlException {
    _mixer = _mgr.createMixer(MediaMixer.AUDIO, Parameters.NO_PARAMETER, id);
    _id = id;
    _nc.join(Direction.DUPLEX, _mixer);
  }

Leave the Conference

When the caller hangs up the call, SIP BYE is received by the MainServlet. Participant, retrieved from the SipSession, unjoin its NetworkConnection from the MediaMixer.

  public void unjoin() throws MsControlException {
    if (_mixer != null) {
      _mixer.unjoin(_nc);
      _ms.release();
      _mgr.removeMixer(_id);
    }
  }

The MixerManager only removes the MediaMixer when no Participant is joined.

  public synchronized MediaMixer removeMixer(String key) throws MsControlException {
    MediaMixer mixer = getMixer(key);
    if (mixer.getJoinees().length == 0) {
      _mixers.remove(key);
      mixer.release();
      mixer.getMediaSession().release();
      return mixer;
    }
    return null;
  }

SipListener

I implemented a SipListener as both SipSessionListener and SipErrorListener to handle the following events.

• SipErrorListener.noAckRecieved. Because MediaSession and related resources are created when INVITE is created, we need clean it up when call setup is failed because of no ACK.
• SipSessionListener.sessionDestroyed. Similar, we need clean up  the MediaSession when the SipSession is destroyed.

Test Sample

As other samples, simply download the WAR and deployed it under <prism>/apps directory. You can use two softphones dialing into the server to form a two party conference call. Please note Voxeo Prism free trial only comes with 2 media ports. Please contact Voxeo Support to request additional licenses.

Upon the completion of SDP negotiation, the media server essentially gets a media pipe to the client, represented as the NetworkConnection. Now, to have some fun, we need to connect that pipe to a multimedia studio that can play, record, and recognize different sound.

MediaGroup

This multimedia studio is called MediaGroup in Java Media Control API.  MediaGroup contains four Resources — Player, Recorder, SignalDetector, and SignalGenerator. Once a MediaGroup is connected to a NetworkConnection, you can use these Resources to, e.g. play music to the client.

MediaGroup can be created from a MediaSession based on a Configuration and, optionally, Parameters.

MediaSession.createMediaGroup(Configuration);

The Configuration indicates to the media server what Resources that the application might be using (or not using). Thus the media server can allocate appropriate resources for this MediaGroup. Typically you will use one of the pre-defined Configurations

• PLAYER: Use this when the application will only use Player in this MediaGroup.
• PLAYER_SIGNALDETECTOR: Use this when the application will only use Player and SignalDetector in this MediaGroup.
• PLAYER_RECORDER_SIGNALDETECTOR: Use this when the application will use all Resources but SignalGenerator in this MediaGroup.
• PLAYER_RECORDER_SIGNALDETECTOR_SIGNALGENERATOR: Use this when the application will use all Resources in this MediaGroup.

To use the MediaGroup, simply joining the NetworkConnection with the MediaGroup. (See Media Object Composition about join concept.) Now you can provide multimedia functions for the client by using different Resource in the MediaGroup.

NetworkConnection nc = ...
MediaGroup mg = session.createMediaGroup(MediaGroup.PLAYER_RECORDER_SIGNALDETECTOR);
nc.join(mg);
mg.getPlayer().play(URI.create("http://myserver.com/mysong.wav"), RTC.NO_RTC, Parameters.NO_PARAMETER);

Let’s take a look what you can do with each Resource in the MediaGroup.

Player

Player can be used to play media streams from a URI (or a set of URIs). For example, the following code snippet shows how to play a song from a Web server.

mg.getPlayer().play(URI.create("http://myserver.com/mysong.wav"), RTC.NO_RTC, Parameters.NO_PARAMETER);

If the media server supports text-to-speech, you can use SSML to render text into synthesized speech. For example,

final static String HELLO_WORLD_SSML = URLEncoder.encode("application/ssml+xml, <?xml version=\"1.0\"?><speak><voice>Hello World!</voice></speak>", "UTF-8");
mg.getPlayer().play(URI.create("data:" + HELLO_WORLD_SSML), RTC.NO_RTC, Parameters.NO_PARAMETER);

You can ask the Player to play multiple items at once by giving an array of URIs.

To stop playing, simply call stop operation to stop either the current item being played or all the items in the queue.

You can also pause and resume the play, as well as adjusting speed and volume.

Recorder

Recorder can be used to record media to URI. For example, the following code snippet shows how to record your voice into a file on a Web server.

mg.getRecorder().record(URI.create("http://myserver.com/mysong.wav"), RTC.NO_RTC, Parameters.NO_PARAMETER);

To stop recording, simply call stop operation.

You can also pause and resume the recording.

SignalDetector

SignalDetector can be used to collect user input based on a set of patterns. For example, the following code snippet tries to receive one DTMF input from the user.

mg.getSignalDetector().receiveSignals(1, SignalDetector.NO_PATTERN, RTC.NO_RTC, Parameters.NO_PARAMETER);

The use of Parameter and pattern label in Java Media Control API is a bit akward. I will explain more in the future blog.

Similarly, you can call stop() to stop the receiveSignal operation.

SignalGenerator

SignalGenerator can be used to generate signal into the media server. This is typically used when the DTMF input are received via SIP INFO.

Sample Application

Here is a sample application that will announce “Hello World” after you dial into it. Then it will ask you to input “1″ for replay. Any other key input will determinate the call.

To run this sample, assuming you have Voxeo Prism downloaded and installed, simply drop the Tutorial.3.war to <prism>/apps directory. Once you have Voxeo Prism (both Application Server and Media Server) started, simply dial “sip:user@localhost” from your SIP phone such as SJPhone or XLite.

Source code is included in the sample .

Micromethod acquisition brings the Voxeo developer community a new set of programming interfaces – Java based APIs for call controls and potentially media controls.

These APIs are based on specifications defined by Java Community Process (JCP). JCP is a process for the Java community to develop specificiations for different Java technologies, such as APIs, languages, virtual machines, etc.

A specification is typically started with a Java Specification Request (JSR) by one or more JCP members. Once accepted by JCP Executive Committee, the JSR is assigned with a number, such as JSR 289. The JSR will be developed within the community in the following phases.

Here is a list of all JSRs that have been developed so far.

SIPMethod Application Server is a JSR-116 (SIP Servlet 1.0) and JSR-289 (SIP Servlet 1.1) based SIP application server. I will talk more about how to develop SIP Servlet based applications in the future blogs.