Section 6.1. Working with MicrophoneAudio Input | Programming Flash Communication Server

6.1. Working with Microphone/Audio Input

Flash Player 6 and higher support the intrinsic Microphone class, enabling Flash to accept audio input from the client. The Microphone class does not interface directly with FlashCom, but it does provide the means by which you can capture the input and send it over a stream to the FlashCom application on the server. In the simplest scenario, working with microphone input requires very minimal code. Of course, many other scenarios will require more sophistication. In the following sections, you'll get the chance to learn how to use the Microphone class to create applications of varying complexity.

6.1.1. Getting the Microphone/Audio Input

Before you can do anything with microphone input, you need to open a Microphone instance that references the device. The static Microphone.get( ) method returns a reference to a Microphone instance, which creates a connection to a specific device attached to the computer:

 // Create a   Microphone   instance, which creates a  // connection to the default audio device. user_mic = Microphone.get( );

A computer may not have an audio input device, in which case get( ) returns null . If an audio device is present but currently locked exclusively by another application, the Flash Player won't be able to use it, though it won't report null either. Most devices on Windows do not enable exclusive locks. Macintosh audio devices aren't exclusively locked either. Furthermore, it's very unlikely that two applications would try to gain exclusive use of an audio device at the same time. Therefore, it's not likely to be an issue.

A computer may have more than one audio device connected. In most scenarios, you will want to use the default device. The user can and should select the default device from the Macromedia Flash Player Settings panel's Microphone tab. (This panel is accessible by choosing Settings from the Flash Player context menu.) This not only puts the user in control, it ensures that the same device is active for all applications that use a microphone. When you call get( ) with no parameters, Flash creates an object (a Microphone instance) that connects to the default device.

However, in other, less common scenarios, you may want to programmatically select a different device. Each audio device connected to the computer has an integer index. You can pass the corresponding index to get( ) in order to instruct the Flash Player to create a Microphone instance that references the specified device:

 // Create a   Microphone   instance that creates a  // connection to the audio device with index 1. user_mic = Microphone.get(1);

The indexes you can use with get( ) represent elements of the Microphone. names array, which contains a list of audio devices detected by Flash. For example, if your computer has two sound cards recognized by Flash, then the following code would display 2 in the Output panel:

 trace(Microphone.names.length);

Once you've created a Microphone instance, you can also retrieve the index and name of the device to which it connects, using the index and name properties:

 trace(user_mic.index); trace(user_mic.name);

Note that the first device is index 0, not 1.

6.1.2. Understanding Microphone Permissions

The Flash Player allows the user to make decisions about what happens with regard to his computer, including the Player's access to audio devices. When you invoke Microphone.get( ) , Flash creates an object that contains information about an audio device connected to the computer. However, before you can do anything useful with the device, the user has to grant permission.

Users grant (or deny) permission via the Macromedia Flash Player Settings panel, as shown in Figure 6-1. The Settings panel opens automatically when the Flash application attempts to obtain data from a Microphone object, such as getting Microphone.activityLevel or attaching a microphone to a published stream. However, attaching a Microphone object to a stream that is not being published will not open the Settings panel. If the stream is published afterward, the panel will open.

Figure 6-1. Asking permission to access the local microphone

The Settings panel appears only if the dimensions of the Flash Player exceed 215 x 138. Otherwise, the panel will not display, and there will be no further notification to the user.

When the user is presented with the Settings panel, she has the option to either allow or deny access to the audio device. If she allows access, Flash will have access to the microphone input. However, your program must also deal gracefully with the user denying access to the audio devices. Therefore, you need a way to detect what option the user selected. Fortunately, the Microphone.onStatus( ) method is called automatically when the user changes the permission for the device via the Settings panel. Flash passes a single parameter to the methodan information object with a code property of "Microphone.Muted" or "Microphone.Unmuted". The following is an example of an onStatus( ) event handler as you might define it for testing purposes on a Microphone object named user_mic :

 user_mic.onStatus = function (info) {   if (info.code == "Microphone.Muted") {     trace("You have denied access to the following audio device: " + this.name);   }   else if (info.code == "Microphone.Unmuted") {     trace("You have allowed access to the following audio device: " + this.name);   } };

The Settings panel opens automatically when you attempt to use microphone audio in the Flash application. However, the panel can be opened in two other ways. The user can manually open the Settings panel by choosing Settings from the Flash Player context menu (available by right-clicking atop the Flash Player window on Windows or Ctrl-clicking on Macintosh). Alternatively, Flash can programmatically open the Settings panel using the System.showSettings( ) method:

 System.showSettings(  );

When the Settings panel opens automatically, it merely enables the user to grant or deny permission, as shown in Figure 6-1. When opened either programmatically or by way of the context menu, the panel offers more options, as shown in Figure 6-2.

Figure 6-2. The full-fledged Macromedia Flash Player Settings panel

By default, showSettings( ) opens the Settings panel to the tab that was last open. You can also specify an integer from 0 to 3 as a parameter to showSettings( ) , which tells Flash which tab to display:

: Opens the Privacy tab where the user can allow or deny access to the camera and microphone devices. The Advanced button opens the Settings Manager URL on Macromedia's site where the user can configure privacy and storage settings on a per-web site basis, plus set global preferences for privacy, storage, security, and automatic notifications.
1: Opens the Local Storage tab, allowing the user to control the amount of local storage allocated to the domain.
2: Opens the Microphone tab, allowing the user to select an audio input device, set recording volume, and control echo suppression.
3: Opens the Camera tab, allowing the user to select a video input device.

Many users find it invasive for a program to open the Settings panel without the user initiating it. Instead, provide a button that allows the user to open the panel. The following code assigns an onRelease( ) event handler method to a button instance so that it opens the Settings panel when clicked. The button is initially made invisible, but if the user denies access to the audio device, the button is made visible:

 settings_btn.onRelease = function (  ) {   System.showSettings(0); }; settings_btn._visible = false; user_mic.onStatus = function (info) {   if (info.code == "Microphone.Muted") {     settings_btn._visible = true;   }   else if (info.code == "Microphone.Unmuted") {     settings_btn._visible = false;   } };

6.1.3. Detecting the Microphone Privacy Setting

If your Flash application requires access to the microphone, you'll need a way to detect whether the user has allowed such access.

Your application can check the read-only, Boolean Microphone.muted property to determine whether the user has granted access to the microphone. The property is false if access is allowed. However, the default is true (indicating access denied) until the user grants permission or if the user has remembered a deny status.

So let's imagine that your application attempts to publish audio input from the microphone.

If the user has not yet been asked for permission to access the microphone:

The initial value of the muted property is true .
The Settings panel shown in Figure 6-1 opens automatically, prompting the user to either allow or deny access.
After the user makes a selection, Flash invokes the Microphone.onStatus( ) method.
You can check the muted property from within onStatus( ) to determine whether the user has allowed access and act accordingly .

But as discussed earlier, if the Settings panel is opened programmatically or via the context menu, the user can also choose whether to remember the Allow or Deny setting, as shown in Figure 6-2. If a user selects the Remember option, Flash remembers that privacy setting for other movies run from the same domain.

If the user previously remembered an "Allow" setting (allow state), the muted property is false and access to the audio device is allowed, so your Flash movie can access the microphone.

But if the user previously remembered a "deny" setting (deny state), the muted property is true and the Settings panel will not open automatically. Because Flash invokes onStatus( ) only when there is a change in the privacy status, onStatus( ) won't be invoked. However, you shouldn't assume from the muted property alone whether the user has yet to be prompted for permission or whether he has remembered a deny state!

If the user has chosen to remember a deny state, it may be rude to display the Settings panel programmatically and ask him to reconsider. We need a more graceful way to determine what to do next .

One possible solution is as follows :

Before attaching or publishing the audio, check the value of the muted property. If muted is false , the user has granted access to the microphone, so your application can skip the remaining steps and proceed as normal.
Otherwise, the muted property is TRue , so continue with Steps 3 through 6.
Use setInterval( ) to create a timeout after a specified interval. If the timeout is reached, assume the user had previously selected that the Player should remember a deny state. Therefore, the timeout handler should decide whether to abort the application, prompt the user to change his mind, or continue running the application without microphone input.
Add an onStatus( ) event handler method that clears the interval created in Step 3. The handler should also check the muted property to determine whether the user chose to allow or deny access to the audio device and proceed accordingly.
Attach or publish the audio.
If the timeout is reached, assume the user had previously selected that the Player should remember a deny state, and proceed as described in Step 3. If onStatus( ) is called, it can check the muted property as described in Step 4.

The following code illustrates the basic idea:

 // Get a microphone object. var user_mic = Microphone.get( ); // Declare a variable to hold the interval identifier. var mutedInterval; // If   muted   is false, the user has remembered an allow state, // so proceed as normal. Otherwise, set an interval for the timeout. if (!user_mic.muted) {   okay( ); } else {   // Set an interval for the timeout. This example uses a 5-second   // timeout after which the   muted( )   function is called.   mutedInterval = setInterval(muted, 5000); } // Define an   onStatus( )   method. user_mic.onStatus = function (info) {   // Clear the interval to prevent the timeout handler from executing.   clearInterval(mutedInterval);   // Call the appropriate function based on the user's selection.   if (info.code == "Microphone.Muted") {     muted( );   }   else {     okay( );   } }; // Attach the audio (may trigger the Settings panel, depending on remember status). this.attachAudio(user_mic); /* The following functions simply display messages in the Output panel so that you  * can see the way in which the code works. In an actual application you would  * likely want to have the functions route the user's experience appropriately  * depending on whether the user has allowed access to the audio device.  */ function okay ( ) {   trace("okay"); } function muted ( ) {   trace("muted"); }

The preceding solution is not without possible flaws. The key is determining the optimal length for the timeout interval. If the timeout is too long, the user will have to wait for a long time before she is notified of the status. But if the timeout is too short, the application could make a decision while the user is still choosing to either allow or deny access.

6.1.4. Attaching and Publishing Audio

Once you've created a Microphone instance, you can attach it to a movie clip so that the audio plays back locally or attach it to a NetStream object and publish it to a FlashCom server.

To play the audio locally, pass the Microphone instance to the MovieClip.attachAudio( ) method:

 controller_mc.attachAudio(user_mic);

To publish the input from a Microphone instance, first attach it to a stream using the NetStream.attachAudio( ) method, as follows:

 publisher_ns.attachAudio(user_mc);

You can then publish the stream using NetStream.publish( ) . The overhead of a net stream without any data is negligible, so you can start publishing the stream without having to check whether the user has granted permission to the audio device. See the previous section for details on determining whether the user has granted permission.

6.1.5. Adjusting Microphone Settings

Once you've created a Microphone instance and the user has granted permission to use the device, you can make some programmatic adjustments to the settings, including gain, rate, minimum activity level, and echo suppression. The gain and echo suppression properties correspond to options within the Settings panel. The rate and minimum activity-level properties are settable via ActionScript but do not have any corresponding options in the Settings panel.

6.1.5.1 Working with gain

The gain determines the audio recording level. The default value is 50, midway between the minimum value of 0 and a maximum value of 100. The higher the value, the louder the audio will sound. Keep in mind that the gain is not exactly the same thing as the volume level. Increasing the gain boosts the incoming signal, but it has the potential to increase feedback and distortion. If the user has the volume at the device level too low, he should adjust those settings. For example, if the user has the volume for the "mic in" channel of the sound card set to the minimum value, increasing the gain in the Flash Player is more likely to result in distortion and feedback than if the sound card level is set at higher values instead. Ideally, the user should set the volume of the microphone in the microphone driver's setting dialog box, because the appropriate volume usually applies to all applications. Unfortunately, some users will not set the microphone volume level properly, so setting the gain programmatically may be the only option to improve their audio input.

You can set the gain using the setGain( ) method, passing a value from 0 to 100:

 user_mic.setGain(50);

You can also retrieve the current gain with the read-only gain property:

 trace(user_mic.gain);

The following code uses a NumericStepper component named cnsGain to allow the user to control the gain for a Microphone instance named user_mic (a NumericStepper is simply a numeric entry box with arrows to increase or decrease the values):

 var gainListener = new Object(  ); gainListener.change = function (event) {   user_mic.setGain(event.target.value); }; cnsGain.addEventListener("change", gainListener); cnsGain.minimum = 0; cnsGain.maximum = 100; cnsGain.value = user_mic.gain;

6.1.5.2 Working with rate

You can also programmatically control the sampling rate (in kHz) at which the audio is retrieved. Use the setRate( ) method to set the rate to a value of 5, 8, 11, 22, or 44. Any other value is rounded to the nearest supported value:

 user_mic.setRate(44);

Use the read-only rate property to retrieve the current rate:

 trace(user_mic.rate);

The following code uses a combo box named ccbRate to allow the user to select a rate and apply it to the audio from a Microphone instance named user_mic :

 // Populate the combo box with the possible rates. ccbRate.dataProvider = [5, 8, 11, 22, 44]; // Set the initial value of the combo box, by selecting  // the item that matches the current rate. for (var i:Number = 0; i < ccbRate.length; i++) {   if (ccbRate.getItemAt(i) == user_mic.rate) {     ccbRate.selectedIndex = i;     break;   } } // Set up a listener to update the rate when the combo box selection changes. var rateListener = new Object( ); rateListener.change = function (event) {   user_mic.setRate(event.target.value); }; ccbRate.addEventListener("change", rateListener);

6.1.5.3 Working with activity level

The activity level of a Microphone instance ranges from 0 (no activity) to 100 (maximum activity). Low-level ambient noise can register little to no activity (levels of 0 to 2) if the gain is low to normal. At high gain settings (or when there is loud computer equipment or nearby air conditioners), ambient noise can cause readings of 20 or more. You can retrieve the current activity level for a Microphone instance using the read-only activityLevel property. The following code uses setInterval( ) to poll the activity level every 100 milliseconds and display the value in a text input component named ctiLevel :

 setInterval(showLevel, 100); function showLevel (  ) {   ctiLevel.text = user_mic.activityLevel; }

Different applications may require different specifications with regard to how microphones function. In some scenarios, you may want to record every sound over a duration of time. In other applications, you may want to publish audio only when the activity level is above a certain threshold (voice-activated recording).

Also keep in mind that, when publishing audio, bandwidth can be a major issue. Publishing audio, even when the audio consists of just ambient noise, requires a lot of bandwidth. Consider the audio in a conferencing applicationonly one or two people are likely to be speaking at a time, yet tens or even hundreds of users may be participating. Flash provides you with the setSilenceLevel( ) method in order to adjust the minimum activity level required for the microphone to set itself to active.

The default minimum activity level to activate a microphone is 10.

With a two-person conference, each user can adjust the listening volume on the receiving end, but with three or more participants , the volumes must be set correctly on the sending end, since they are mixed together on the receiving end.

Ideally, each user should set her microphone gain in the Microphone tab of the Settings panel so that the default silence level of 10 is appropriate. If all users in a multiuser conference normalize their settings this way, their audio should all be at about the same volume (enabling everyone to hear everyone else).

However, you can adjust the minimum activity level programmatically to any value from 0 to 100. The following code sets the minimum activity level to 0, effectively making the microphone active regardless of activity level:

 user_mic.setSilenceLevel(0);

A microphone remains active as long as the activity level remains above the minimum. After the activity level drops below the minimum, Flash does not deactivate the microphone until a timeout is reached. The default timeout is 2000 milliseconds (2 seconds). You can adjust the timeout as well as the minimum activity level using setSilenceLevel( ) . Simply pass a second parameter to the method indicating the number of milliseconds you want Flash to wait before deactivating the microphone:

 user_mic.setSilenceLevel(50, 1000);

When the microphone is deactivated, no audio is published over a NetStream . Instead, a single message is sent to the server to indicate that the audio stream is silent (a "silent" user consumes no bandwidth). Once the minimum activity level is surpassed again, the microphone starts sending audio data again.

Each time the microphone is activated or deactivated, the Microphone.onActivity( ) event handler method is called. Flash passes the method a Boolean parameter indicating whether the microphone is active ( true ) or inactive ( false ). The following code writes a value to a text area component instance named ctaActivity each time the microphone is activated or deactivated:

 user_mic.onActivity = function(active) {   ctaActivity.text += active + newline; };

You can use the read-only silenceLevel and silenceTimeout properties to retrieve the current values for the minimum activity level required to activate a microphone and the timeout before the microphone is deactivated.

6.1.5.4 Working with echo management

Microphones and speakers often cause unwanted echo, resulting in feedback. Feedback is normally a symptom of the microphone being placed so that it picks up audio from speakers. The best way to avoid speaker audio feeding back into the microphone is to advise users to use a closed headset instead of speakers . Most users have headsets from portable audio devices if not headsets purchased specifically for use with their computers. Some headsets have a built-in microphone and often provide the best echo protection, but even a standard headset will usually work well.

If a user must use speakers rather than headphones, no amount of programmatic attempts will take the place of good microphone placement. The microphone should be placed facing the user and away from the audio speakers. Assuming the speakers and the microphone are in front of the user, the microphone should be nearer to the user and the speakers should be further away. Furthermore, the microphone should be directional. If the microphone is omnidirectional, it will pick up speaker output no matter how it is placed relative to the speakers.

Though the effect is fairly minimal, you can manage echo and feedback slightly using the Microphone.setUseEchoSuppression( ) method. You can call the method with a value of TRue to turn on echo suppression (the default value is false ):

 user_mic.setUseEchoSuppression(true);

The read-only useEchoSuppression property reports the current value.

The echo suppression feature works by temporarily halving the gain when audio is playing through the speakers/headphones.