Audio

The multimedia architecture is used to perform audio operations, such as recording and playing. A variety of different audio formats ”for example, .wav and .midi ”are supported for use within applications.

To handle the various formats a plug-in architecture is used to match the file format to an appropriate codec ( coder /decoder). Besides this, it is also possible to explicitly select a codec, which is necessary when dealing with raw audio data.

Utility classes are available for carrying out the essential audio tasks , and provision is made for recording, streaming, converting between audio formats, tone playing and audio file playing. From a technical standpoint, an observer mechanism is used, with each utility having a dedicated observer class and callbacks. This allows applications to respond to the asynchronous nature of interacting with the audio utility classes.

Consequently, in a similar vein to the approach used in managing image manipulation, it can be useful to write an "Adapter" class, deriving from the observer, which implements the callbacks. The libraries of interest when developing applications that use sounds are listed in Table 11-13.

Table 11-13. Series 60 Audio Libraries

Library (.lib)	Role	Key Elements
MediaClientAudio.lib	Playing and recording of sounds. Conversion between audio formats.	Audio sample editor, audio sample player and audio tone player.
MediaClientAudioStream.lib	Allows multimedia clients to stream audio data.	Audio streaming.

Unlike image manipulation, the changes between Series 60 1.x and Series 2.x, have been more "behind-the-scenes," so differences will be highlighted as and when they appear. The various aspects of Series 60 sound are illustrated across two example applications, AnsPhone and Audio .

Recording

Recording is made possible through the CMdaAudioRecorderUtility class. In addition to recording in different audio formats, this class also offers a playback facility. While this can save resources, by removing the need to instantiate a separate player object, use of one of the player objects discussed later in this section, such as CMdaAudioPlayerUtility , is recommended for dedicated playback.

In terms of physical storage, recording can take place to a file, a descriptor and in Series 60 2.x, a URL. Recording is essentially a three-stage process of gaining a handle to a data store, setting the desired configurations and then recording.

Accessing the data store and physically recording are asynchronous operations, and the MMdaObjectStateChangeObserver::MoscoStateChangeEvent() function informs applications of the completion of each operation. This architecture requires the class calling the asynchronous functions to derive from MMdaObjectStateChangeObserver and implement MoscoStateChangeEvent() .

The AnsPhone example application has a class CAnsPhoneEngine , which derives from MMdaObjectStateChangeObserver and implements MoscoStateChangeEvent() :

[View full width]

 
 [View full width] 
 class CAnsPhoneEngine :  public CBase,    public MMdaObjectStateChangeObserver,    public MAnsPhoneTimerObserver,    public MAnsPhoneCallWatcherObserver,    public MAnsPhoneCallMakerObserver,    public MAnsPhoneCallLogObserver    {    ... public:   // MMdaObjectStateChangeObserver    virtual void MoscoStateChangeEvent(CBase* aObject, TInt aPreviousState, TInt  aCurrentState, TInt aErrorCode);    ...    CMdaAudioRecorderUtility*   iSound;    ...    // Recording message settings    TMdaFileClipLocation   iMessageLocation;    TMdaAudioDataSettings   iMessageSettings;    TMdaWavClipFormat       iMessageFormat;    TMdaPcmWavCodec         iMessageCodec;    ...    }; 

CMdaAudioRecorderUtility provides an OpenL() function that opens a channel to the data store that should be used as the destination for the recording or, if the class is being employed for playback, the audio data that will be played . As the correct codec for handling raw audio data cannot be selected automatically, an explicit reference to a codec may be necessary.

In addition to OpenL() , functions are also available for specific destination types. Series 60 1.x offers OpenFileL() and OpenDesL() for files and descriptors, respectively, while Series 60 2.x also has OpenUrlL() .

In the AnsPhone application, CMdaAudioRecorderUtility::OpenL() is used in the RecordMessageL() method:

 iSound = CMdaAudioRecorderUtility::NewL(*this); ... iSound->OpenL(&iMessageLocation, &iMessageFormat, &iMessageCodec, &iMessageSettings); ... 

Once the open operation has completed, the function MoscoStateChangeEvent() (from the MMdaObjectStateChangeObserver mixin) will be called.

Before recording, a variety of configuration settings are available, including: volume, sample rate and balance. These settings are extensively covered in the SDK documentation. The AnsPhone application sets the gain, volume and recording position before recording.

Once the required configuration is set, CMdaAudioRecorderUtility::RecordL() can be invoked, remembering that MoscoStateChangeEvent() will be called once more when recording has finished:

[View full width]

 
 [View full width] 
 void CAnsPhoneEngine::MoscoStateChangeEventL(CBase* /*aObject*/, TInt /*aPreviousState*/,  TInt aCurrentState, TInt aErrorCode)    {    ...    switch (iState)       {       case ERecordInit:          // Record from the telephony line          // and set to max gain          if (iIsLocal)             {             iSound->SetAudioDeviceMode(CMdaAudioRecorderUtility::ELocal);             iSound->SetGain(iSound->MaxGain());             }          else             {             iSound->SetAudioDeviceMode(CMdaAudioRecorderUtility::ETelephonyNonMixed);             TInt maxGain = iSound->MaxGain();             iSound->SetGain(maxGain / 2);             }          // Delete current audio sample from beginning of file          iSound->SetPosition(TTimeIntervalMicroSeconds(0));          iSound->CropL();          // start recording          iSound->RecordL();          iState = ERecord;          break;          ...    } 

Note that the above function is actually called from MoscoStateChangeEvent() within a trap harness, as it may leave.

Another important consideration is the tdeviceMode enumeration, which determines how the audio device handles playback and recording. This enumeration is defined in the file \epoc32\include\MdaAudioSampleEditor.h in the root directory of your SDK. Table 11-14 examines the options available.

Table 11-14. Audio Device Modes

Enumeration Value	Effect
EDefault = 0	If a call is currently taking place, then the line and the device microphone will be recorded, and playback will be made down the line. If there is no call, then the device microphone will be recorded, and playback will be through the device speaker.
ETelephonyOrLocal = EDefault	As with EDefault .
ETelephonyMixed = 1	If a call is in progress, then the audio source is the device microphone, mixed with the telephony line. Playback will be both through the device speaker and down the phone line. If there is no call, then recording is not attempted to the device microphone, and playback will not take place to the device speaker.
ETelephonyNonMixed = 2	If there is a call in progress, then only the telephone line is used for recording and playback ”it is not mixed with the device microphone or speaker.
ELocal = 3	The telephone line is ignored. The device microphone will be recorded, and the device speaker is used for playback.

There is a known issue with Series 60 1.x and some early devices ”the default mode will always be used by CMdaAudioRecorderUtility , despite any values passed into SetAudioDeviceMode() . An up-to-date list of known Series 60 issues can be found at www.forum.nokia.com.

Although primarily used for recording, as highlighted earlier, CMdaAudioRecorderUtility can be used for playback purposes. Opening the data source and configuring the playback options are the same as for recording; however, the CMdaAudioRecorderUtility::PlayL() function is used to play the audio data.

[View full width]

 
 [View full width] 
 void CAnsPhoneEngine::MoscoStateChangeEventL(CBase* /*aObject*/, TInt /*aPreviousState*/,  TInt aCurrentState, TInt aErrorCode)    {    ...    switch (iState)       {       ...       case EPlayInit:          {          // Play through the device speaker          // and set to max volume          if (iIsLocal)             {             iSound->SetAudioDeviceMode(CMdaAudioRecorderUtility::ELocal);             }          else             {             iSound->SetAudioDeviceMode(CMdaAudioRecorderUtility::ETelephonyOrLocal);             }          iSound->SetVolume(iSound->MaxVolume());          // Set the playback position to the start          // of the file          iSound->SetPosition(TTimeIntervalMicroSeconds(0));          iSound->PlayL();          ...    } 

Tones

The CMdaAudioToneUtility class plays audio tones. Tones can be sine waves, or Dual Tone Multi Frequency ( DTMF ) telephony signals (tone-dialing "touch-tones"). As with recording, an observer is used for notifications and callbacks ”in this case MMdaAudioToneObserver .

For notification purposes, MMdaAudioToneObserver provides two pure virtual methods : MatoPrepareComplete() , to inform the client that the tone player is configured properly to play, and MatoPlayComplete() , which is called once the tone has finished playing. Both of these functions accept a TInt parameter to indicate if any errors have occurred. If KErrNone is passed into either function, then the action was successful; otherwise the error value must be interpreted, and appropriate action taken.

Following the construction of CMdaAudioToneUtility , a tone can be generated by the player through one of the variants of the CMdaAudioToneUtility::PrepareToPlayXXX() function. These variants cover playing DTMFs and tone sequences from files and descriptors. The PrepareToPlayTone() function will play a single tone based on the frequency and duration parameters provided to it.

 void CAudioPlayerEngine::PlayToneL()    {    iState = EPlaying;    iPlayerTone = CMdaAudioToneUtility::NewL(*this);    iPlayerTone->PrepareToPlayTone(KToneFrequency, TTimeIntervalMicroSeconds(KToneDuration));    } 

Once MatoPrepareComplete() has been called, the player can be configured in many ways ”for example, setting the volume, and in Series 60 2.x, the balance. A fuller description of the configuration options can be found in the SDK documentation.

CMdaAudioToneUtility::Play() can now be called, and MatoPlayComplete() will inform the application that playing has completed. Playing can be aborted before completion by calling CMdaAudioToneUtility::CancelPlay() .

 // implementation of MMdaAudioToneObserver virtual function void CAudioPlayerEngine::MatoPrepareComplete(TInt aError)    {    if (aError)       {       Stop();       }    else       {       iPlayerTone->SetVolume(iPlayerTone->MaxVolume() / KToneVolumeDenominator);       iPlayerTone->Play();       }    } 

Audio Data

The playing of audio data, such as .wav and .midi , is made possible by the CMdaAudioPlayerUtility class and its associated observer, MMdaAudioPlayerCallback . This observer has two functions to inform the client application of the current status of CMdaAudioPlayerUtility : MapcInitComplete() indicates that the data source has been opened successfully, while MapcPlayComplete() signals that playing has concluded. As with the callback functions used in tone playing, both accept a TInt parameter to indicate if any errors have taken place.

While the differences between Series 60 1.x and Series 60 2.x are minimal in the other audio utility classes, they are much more noticeable with CMdaAudioPlayerUtility , as its feature set has greatly increased.

Under Series 60 1.x, CMdaAudioPlayerUtility has three instantiation functions for reading audio data from files, descriptors and read-only descriptors, named NewFilePlayerL() , NewDesPlayerL() , and NewDesPlayerReadOnlyL() , respectively. These functions are asynchronous, and on completion will invoke MapcInitComplete() .

 void CAudioPlayerEngine::PlayWavL()    {    TFileName wavFile(KToPlayFileWav);    User::LeaveIfError(CompleteWithAppPath(wavFile));    iState = EPlaying;    iPlayerFile = CMdaAudioPlayerUtility::NewFilePlayerL(wavFile, *this);    } 

In addition to the functions above, when targeting Series 60 2.x there is also CMdaAudioPlayerUtility::NewL() , which does not require a specific data source ”however, on completion, it too will call MapcInitComplete() . Series 60 2. x-specific "open" and "open and play" functionality allows a single player instance to open multiple data sources at different times. Note that in addition to standard files and descriptors, URLs can also be opened for playing:

[View full width]

 
 [View full width] 
 OpenUrlL(const TDesC& aUrl, const TDesC8& aMimeType = KNullDesC8); OpenAndPlayUrlL(const TDesC& aUrl, const TDesC8& aMimeType = KNullDesC8, TInt aPriority =  EMdaPriorityNormal, TMdaPriorityPreference aPref = EMdaPriorityPreferenceTimeAndQuality); 

Using Series 60 1.x, it is possible to achieve a similar effect to the "open and play" facility of Series 60 2.x by calling CMdaAudioUtility::Play() from within the implementation of MapcInitComplete() :

[View full width]

 
 [View full width] 
 void CAudioPlayerEngine::MapcInitComplete(TInt aError, const TTimeIntervalMicroSeconds&  /*aDuration*/)    {    if (aError == KErrNone)       {       iPlayerFile->SetVolume(iPlayerFile->MaxVolume() / KToneVolumeDenominator);       iPlayerFile->Play();       }    else       {       Stop();       }    } 

Remember, though, that this is optional, and an application might have an explicit "play" feature, which could be invoked at any time, not just immediately after opening the data source.

An audio clip can be stopped at any time by calling CMdaAudioPlayerUtility::Stop() . Additionally on Series 60 2.x CMdaAudioPlayerUtility::Close() should be invoked, as this will allow new audio data to be played with the same CMdaAudioPlayerUtility object.

In Series 60 1.x, MIDI files are not supported for playback in the emulator and will cause a panic. They will, however, play on most Series 60 1.x-based hardware. This is illustrated in the example application, Audio .

The player can be configured like the other audio utilities ”for example, setting the volume and balance. Other functions associated with playing, such as getting the duration of a clip and setting the play position within a clip, can be particularly useful when implementing a player control that displays the length of the clip, or has a graphical slider which increments during playback of the audio track. Full details of the APIs available in each version of the platform can be found in the relevant SDK documentation.

Streaming

Audio streams allow applications to play audio without having the entire contents of the audio data. Audio data is accessed and buffered incrementally, and every attempt is made to ensure smooth and continuous playback. To perform this task, the CMdaAudioOutputStream and MMdaAudioOutputStreamCallback classes are used. These two classes provide the mechanisms to manage the flow of buffered audio data within the multimedia architecture, and to pass it on to the lower-level sound device.

Data must be presented to the player in descriptor format, and the only supported audio format that the streamed player can support is 16-bit pulse-code modulation ( PCM ). Usually PCM data contains a 44-byte header that contains information about the audio data payload. Since the streaming audio player accepts only 16-bit PCM, it does not require this header information.

The desired properties of the audio device can be set up in different ways through the CMdaAudioOutputStream class. The SetAudioPropertiesL() method can be used to set the sample rate and number of channels used. Additional attributes, such as the volume, can be controlled by passing a TMdaAudioDataSettings object into the Open() method ”calling Open() prepares the player object for streaming:

 void CAudioPlayerEngine::PlayStreamL()    {    // open the file and load it into the buffers    RFs fs;    CleanupClosePushL(fs);                     // PUSH    User::LeaveIfError(fs.Connect());    RFile file;    CleanupClosePushL(file);                   // PUSH    TFileName streamFile(KToPlayFileStream);    User::LeaveIfError(CompleteWithAppPath(streamFile));    User::LeaveIfError(file.Open(fs, streamFile, EFileRead  EFileShareReadersOnly));    TInt fileSize = 0;    file.Size(fileSize);    iStreamData = new (ELeave) TUint8[fileSize];    iStreamBuffer = new (ELeave) TPtr8(iStreamData, fileSize, fileSize);    file.Read(*iStreamBuffer);    CleanupStack::PopAndDestroy(2);            // file & fs    iState = EPlaying;    iPlayerStream = CMdaAudioOutputStream::NewL(*this);    iPlayerStream->Open(&iStreamSettings);    } 

MMdaAudioOutputStreamCallback is used to handle the asynchronous nature of buffering data for streaming audio. It provides three virtual functions to respond to opening the player, copying a buffer of data to the player, and the termination of playing:

• MaoscOpenComplete() notifies the client that the stream has been opened following a call to CMdaAudioOutputStream::Open() and indicates that the audio output stream is ready for use:

   void CAudioPlayerEngine::MaoscOpenComplete(TInt aError)    {    if (aError == KErrNone)       {       TRAPD(err, iPlayerStream->WriteL(*iStreamBuffer));       }    else       {       Stop();       }    } 

  
  

• MaoscBufferCopied() is called after data has been placed in the buffer by CMdaAudioOutputStream::WriteL() . This informs the client application that the audio data was written successfully. The copied buffer can now be destroyed , freeing up memory, or more data can be written by calling WriteL() again:

   void CAudioPlayerEngine::MaoscBufferCopied(TInt aError, const TDesC8& /*aBuffer*/)    {    if (aError == KErrNone)       {       TRAPD(err, iPlayerStream->WriteL(*iStreamBuffer))       iPlayerStream->SetVolume(iPlayerStream->MaxVolume() / KToneVolumeDenominator);       }    } 

  
  

• MaoscPlayComplete() informs the client that streaming has been terminated :

   void CAudioPlayerEngine::MaoscPlayComplete(TInt /*aError*/)    {    iState = EStopped;    delete iPlayerStream;    iPlayerStream = NULL;    delete iStreamBuffer;    iStreamBuffer = NULL;    delete [] iStreamData;    iStreamData = NULL;    TRAPD(err, iObserver.HandlePlayingStoppedL());    } 

  
  

Be aware that, if the buffer size is too small, it could cause the audio to sound as if it were stuttering. This is because the audio device is consuming the buffer faster than the new buffers are being copied.