Audio Effects on MIDI SequencesThere are four ways of applying audio effects to MIDI sequences:
PrecalculationAs with sampled audio, using Java at execution time to modify a sequence can be time-consuming and difficult to implement. Several tools allow you to create or edit MIDI sequences, though you do need an understanding of music and MIDI to use them. Here are some of packages I've tinkered with:
Sequence ManipulationFigure 9-2 shows the internals of a sequence. Figure 9-2. The internals of a MIDI sequenceThe regularity of the data structure means that it can be easy to modify at runtime, but you're going to need to understand the MIDI specification. Doubling the sequence volumeHere is the basic code for playing a sequence: Sequence seq = MidiSystem.getSequence(getClass( ).getResource(fnm)); // change the sequence: double its volume in this case doubleVolumeSeq(seq); sequencer.setSequence(seq); // load changed sequence sequencer.start( ); // start playing it
The sequence is modified after being loaded with getSequence( ) and before being assigned to the sequencer with setSequence( ). Volume doubling is applied to every track in the sequence: private void doubleVolumeSeq(Sequence seq) { Track tracks[] = seq.getTracks( ); // get all the tracks for(int i=0; i < tracks.length; i++) // iterate through them doubleVolume(tracks[i], tracks[i].size( )); } doubleVolume( ) examines every MidiEvent in the supplied track, extracting its component tick and MIDI message. If the message is a NOTE_ON, then its volume will double (up to a maximum of 127): private void doubleVolume(Track track, int size) { MidiEvent event; MidiMessage message; ShortMessage sMessage, newShort; for (int i=0; i < size; i++) { event = track.get(i); // get the event message = event.getMessage( ); // get its MIDI message long tick = event.getTick( ); // get its tick if (message instanceof ShortMessage) { sMessage = (ShortMessage) message; // check if the message is a NOTE_ON if (sMessage.getCommand( ) == ShortMessage.NOTE_ON) { int doubleVol = sMessage.getData2( ) * 2; int newVol = (doubleVol > 127) ? 127 : doubleVol; newShort = new ShortMessage( ); try { newShort.setMessage(ShortMessage.NOTE_ON, sMessage.getChannel( ), sMessage.getData1( ), newVol); track.remove(event); track.add( new MidiEvent(newShort,tick) ); } catch ( InvalidMidiDataException e) { System.out.println("Invalid data"); } } } } } // end of doubleVolume( ) Each MIDI message is composed from three bytes: a command name and two data bytes. ShortMessage.getCommand( ) is employed to check the name. If the command name is NOTE_ON, then the first byte will be the note number, and the second its velocity (similar to a volume level).
The volume is obtained with a call to ShortMessage.getData2( ) and then doubled with a ceiling of 127 since the number must fit back into a single byte. A new ShortMessage object is constructed and filled with relevant details (command name, destination channel ID, note number, new volume): newShort.setMessage(ShortMessage.NOTE_ON, sMessage.getChannel( ), sMessage.getData1( ), newVol); The old MIDI event (containing the original message) must be replaced by an event holding the new message: a two-step process involving track.remove( ) and track.add( ). The new event is built from the new message and the old tick value: track.add( new MidiEvent(newShort,tick) ); The tick specifies where the event will be placed in the track. MIDI Channel ControllersFigure 9-3 shows the presence of 16 MIDI channels inside the synthesizer; each one acts as a "musician," playing a particular instrument. As the stream of MIDI messages arrive (individually or as part of a sequence), each message is routed to a channel based on its channel setting. Each channel has a set of controllers associated with it. The set depends on the particular synthesizer; controllers defined in the General MIDI specification should be present, but there may be others. For example, controllers offering the Roland GS enhancements are found on many devices. General MIDI controllers include controls for volume level, stereo balancing, and panning. Popular Roland GS enhancements include reverberation and chorus effects. Each controller is identified by a unique ID, between 0 and 127.
Figure 9-3. A MIDI sequencer and synthesizerThe FadeMidi and PanMidi examples illustrate how to use channel controllers to affect the playback of an existing sequence. They both reuse several methods from PlayMidi.java, shown in Chapter 7. Making a sequence fade awayFadeMidi.java (located in SoundExamps/SoundPlayer/) plays a sequence, gradually reducing its volume level to 0 by the end of the clip. The volume settings for all 16 channels are manipulated by accessing each channel's main volume controller (the ID for that controller is the number 7).
The incremental volume reduction is managed by a VolChanger thread, which repeatedly lowers the volume reduction until the sequence has been played to its end. Figure 9-4 gives the class diagrams for FadeMidi and VolChanger, showing only the public methods. The main( ) method initializes FadeMidi and starts VolChanger: public static void main(String[] args) { if (args.length != 1) { System.out.println("Usage: java FadeMidi <midi file>"); System.exit(0); } Figure 9-4. Class diagrams for FadeMidi and VolChanger// set up the player and the volume changer FadeMidi player = new FadeMidi(args[0]); VolChanger vc = new VolChanger(player); player.startVolChanger(vc); // start volume manipulation } VolChanger is passed a reference to FadeMidi so it can affect the synthesizer's volume settings. startVolChanger( ) starts the VolChanger thread running and supplies the sequence duration in milliseconds. The thread needs it to calculate how often to change the volume: public void startVolChanger(VolChanger vc) { vc.startChanging( (int)(seq.getMicrosecondLength( )/1000) ); } The FadeMidi constructor looks similar to the one in PlayMidi: public FadeMidi(String fnm) { df = new DecimalFormat("0.#"); // 1 dp filename = SOUND_DIR + fnm; initSequencer( ); loadMidi(filename); play( ); /* No need for sleeping to keep the object alive, since the VolChanger thread refers to it. */ } initSequencer( ) and loadMidi( ) are identical to the methods of the same name in PlayClip, and play( ) is slightly different. The most significant change is the absence of a call to sleep( ), which keeps PlayMidi alive until its sequence has finished. Sleeping is unnecessary in FadeMidi because the object is referred to by the VolChanger tHRead, which keeps calling its setVolume( ) method. play( ) initializes a global array of MIDI channels: private static final int VOLUME_CONTROLLER = 7; // global holding the synthesizer's channels private MidiChannel[] channels; private void play( ) { if ((sequencer != null) && (seq != null)) { try { sequencer.setSequence(seq); // load MIDI into sequencer sequencer.start( ); // play it channels = synthesizer.getChannels( ); // showChannelVolumes( ); } catch (InvalidMidiDataException e) { System.out.println("Invalid midi file: " + filename); System.exit(0); } } } private void showChannelVolumes( ) // show the volume levels for all the synthesizer channels { System.out.println("Syntheziser Channels: " + channels.length); System.out.print("Volumes: {"); for (int i=0; i < channels.length; i++) System.out.print( channels[i].getController(VOLUME_CONTROLLER) + " "); System.out.println("}"); }
Channels in the array are accessed using the indices 0 to 15 though the MIDI specification numbers them 1 to 16. For instance, the special percussion channel is MIDI number 10, but it is represented by channels[9] in Java. In showChannelVolumes( ), MidiChannel.getController( ) obtains the current value of the specified controller. Supplying it with the ID for the volume controller (7) will cause it to return the current volume setting. A controller stores the data in a single byte, so the returned value will be in the range 0 to 127. Getting and setting the volumeFadeMidi contains two public methods for getting and setting the volume, both used by VolChanger: public int getMaxVolume( ) // return the max level for all the volume controllers { int maxVol = 0; int channelVol; for (int i=0; i < channels.length; i++) { channelVol = channels[i].getController(VOLUME_CONTROLLER); if (maxVol < channelVol) maxVol = channelVol; } return maxVol; } public void setVolume(int vol) // set all the controller's volume levels to vol { for (int i=0; i < channels.length; i++) channels[i].controlChange(VOLUME_CONTROLLER, vol); } getMaxVolume( ) returns a single volume, rather than all 16; this keeps the code simple. setVolume( ) shows how MidiChannel.controlChange( ) is used to change a specified controller's value. The data should be an integer between 0 and 127. Changing the volumeVolChanger gets started when its startChanging( ) method is called. At this point, the sequence will be playing, and the MIDI channel controllers are available for manipulation: // globals // the amount of time between changes to the volume, in ms private static int PERIOD = 500; private FadeMidi player; private int numChanges = 0; public void startChanging(int duration) /* FadeMidi calls this method, supplying the duration of its sequence in ms. */ { // calculate how many times the volume should be adjusted numChanges = (int) duration/PERIOD; start( ); } // end of startChanging( ) VolChanger adjusts the volume every PERIOD (500 ms), but how many times? The duration of the sequence is passed in as an argument to startChanging( ) and is used to calculate the number of volume changes. run( ) implements a volume reduction/sleep cycle: public void run( ) { /* calculate stepVolume, the amount to decrease the volume each time that the volume is changed. */ int volume = player.getMaxVolume( ); int stepVolume = (int) volume / numChanges; if (stepVolume == 0) stepVolume = 1; System.out.println("Max Volume: " + volume + ", step: " + stepVolume); int counter = 0; System.out.print("Fading"); while(counter < numChanges){ try { volume -= stepVolume; // reduce the required volume level if ((volume >= 0) && (player != null)) player.setVolume(volume); // change the volume Thread.sleep(PERIOD); // delay a while } catch(InterruptedException e) {} System.out.print("."); counter++; } System.out.println( ); } The MIDI volume bugFadeMid.java doesn't work with J2SE 5.0 due to a bug associated with the volume adjustment of a sequencer. The offending line is in initSequencer( ): sequencer = MidiSystem.getSequencer( ); The sequencer is retrieved, but subsequent volume changes have no effect. The solution is to explicitly request the sequencer by finding it in on the list of available MIDI devices for the machine. This is packaged inside obtainSequencer( ): private Sequencer obtainSequencer( ) { MidiDevice.Info[] mdi = MidiSystem.getMidiDeviceInfo( ); int seqPosn = -1; for(int i=0; i < mdi.length; i++) { System.out.println(mdi[i].getName( )); if (mdi[i].getName( ).indexOf("Sequencer") != -1) { seqPosn = i; // found the Sequencer System.out.println(" Found Sequencer"); } } try { if (seqPosn != -1) return (Sequencer) MidiSystem.getMidiDevice( mdi[seqPosn] ); else return null; } catch(MidiUnavailableException e) { return null; } } // end of obtainSequencer( ) The position of the sequencer in the MIDI device information array, mdi[], will vary depending on the audio devices attached to a given machine and the J2SE version, so some searching is required. The list printed on a test machine running J2SE 5.0 is shown in Example 9-2. Example 9-2. MIDI device information in J2SE 5.0Roland MPU-401 MIDI Mapper Microsoft GS Wavetable SW Synth Roland MPU-401 Real Time Sequencer Found Sequencer Java Sound Synthesizer The list generated on a different machine, using J2SE 1.4.2, is shown in Example 9-3. Example 9-3. MIDI device information in J2SE 1.4.2Java Sound Synthesizer Java Sound Sequencer Found Sequencer MIDI Mapper Microsoft GS Wavetable SW Synth The sequencer is obtained in initSequencer( ) by calling obtainSequencer( ): sequencer = obtainSequencer( ); The problem, which has been reported by several users in the Java Sound forums (e.g., at http://archives.java.sun.com/cgi-bin/wa?A0=javasound-interest), only seems to occur when the volume needs to be changed. For example, this extra work isn't required in PanMidi (the next example): the sequencer it obtains with MidiSystem.getSequencer( ) does respond to panning changes.
Panning the sequencePanMidi repeatedly switches its sequence from the left to the right speaker and back again. A PanChanger thread switches the pan settings in all the channel controllers at periodic intervals during the playing of the sequence.
The class diagrams for PanMidi and PanChanger are given in Figure 9-5. Figure 9-5. Class diagrams for PanMidi and PanChangerThe main( ) method initializes the player and the thread, and then it calls PanMidi's startPanChanger( ) to start the thread running. startPanChanger( ) passes the duration of the sequence to the thread, so it can calculate the number of changes it will make. The PanMidi pan methods used by PanChanger are getMaxPan( ) and setPan( ): // global constants // private static final int BALANCE_CONTROLLER = 8; //not working? private static final int PAN_CONTROLLER = 10; public int getMaxPan( ) // return the max value for all the pan controllers { int maxPan = 0; int channelPan; for (int i=0; i < channels.length; i++) { channelPan = channels[i].getController(PAN_CONTROLLER); if (maxPan < channelPan) maxPan = channelPan; } return maxPan; } public void setPan(int panVal) // set all the controller's pan levels to panVal { for (int i=0; i < channels.length; i++) channels[i].controlChange(PAN_CONTROLLER, panVal); } The only real difference in PanMidi from FadeMidi is the use of the PAN_CONTROLLER controller number.
Changing the pan valueUnlike VolChanger, PanChanger carries out a cyclic series of changes to the pan value. However, the core of run( ) is still a loop repeatedly calling setPan( ) and sleeping for an interval. The series of pan values that make up a single cycle are defined in a panVals[] array: // time to move left to right and back again private static int CYCLE_PERIOD = 4000; // in ms // pan values used in a single cycle // (make the array's length integer divisible into CYCLE_PERIOD) private int[] panVals = {0, 127}; // or try // private int[] panVals = {0, 16, 32, 48, 64, 80, 96, 112, 127, // 112, 96, 80, 64, 48, 32, 16}; The run( ) method cycles through the panVals[] array until it has executed for a time equal to the sequence's duration: public void run( ) { /* Get the original pan setting, just for information. It is not used any further. */ int pan = player.getMaxPan( ); System.out.println("Max Pan: " + pan); int panValsIdx = 0; int timeCount = 0; int delayPeriod = (int) (CYCLE_PERIOD / panVals.length); System.out.print("Panning"); while(timeCount < duration){ try { if (player != null) player.setPan( panVals[panValsIdx] ); Thread.sleep(delayPeriod); // delay } catch(InterruptedException e) {} System.out.print("."); panValsIdx = (panValsIdx+1) % panVals.length; // cycle through the array timeCount += delayPeriod; } System.out.println( ); } Sequencer MethodsThe Sequencer has methods that can change the tempo (speed) of playback. The easiest to use is probably setTempoFactor( ), which scales the existing tempo by the supplied float: sequencer.setTempoFactor(2.0f); // double the tempo Tempo adjustments only work if the sequence's event ticks are defined in the PPQ (ticks per beat) format since tempo affects the number of beats per minute. Sequencer.getTempoFactor( ) can be employed after calling Sequencer.setTempoFactor( ) to check whether the requested change has occurred. The Sequence class offers getdivisionType( ), which returns a float representing the sequence's division type. Sequence.PPQ for PPQ, or one of the many Society of Motion Picture and Television Engineers (SMPTE) types, use ticks per frame. This information can be used to determine if setTempoFactor( ) would work on the sequence. Sequencer has two methods that act upon the sequence's tracks: setTrackMute( ), and setTrackSolo( ). Here's a fragment of code that sets and tests the mute value: sequencer.setTrackMute(4, true); boolean muted = sequencer.getTrackMute(4); if (!muted) // muting failed |