Audio Synthesis Libraries


MIDI Synthesis

I'll consider three approaches to synthesizing MIDI sound at runtime:

  • Send note-playing messages to a MIDI channel. The MidiChannel class offers noteOn( ) and noteOff( ) methods that transmit NOTE_ON and NOTE_OFF MIDI messages.

  • Send MIDI messages to the synthesizer's receiver port. This is a generalization of the first approach. The advantages include the ability to deliver messages to different channels, and the ability to send a wider variety of messages.

  • Create a sequence, which is passed to the sequencer. This is a generalization of the second approach. Rather than send individual notes to the synthesizer, I build a complete sequence.

These approaches are labeled in the MIDI devices diagram in Figure 10-4.

Figure 10-4. Different MIDI synthesis approaches


There is a good Java Tech Tip on these topics at http://java.sun.com/jdc/JDCTechTips/2003/tt0805.html.


Sending Note-Playing Message to a MIDI Channel

The MidiChannel class offers noteOn( ) and noteOff( ) methods that correspond to the NOTE_ON and NOTE_OFF MIDI messages:

     void noteOn(int noteNumber, int velocity);     void noteOff(int noteNumber, int velocity);     void noteOff(int noteNumber);

The note number is the MIDI number assigned to a musical note, and velocity is equivalent to the loudness. A note will keep playing after a noteOn( ) call until it's terminated with noteOff( ). The two-argument form of noteOff( ) can affect how quickly the note fades away.

MIDI notes can range between 0 and 127, extending well beyond the piano's scope, which includes 88 standard keys. This means that the note-naming scheme gets a little strange below note 12 (C0) since we have to start talking about octave -1 (e.g., (see the table at http://www.harmony-central.com/MIDI/Doc/table2.html). Additionally, a maximum value of 127 means that note names only go up to G9; there is no G#9. Table 10-2 shows the mapping of MIDI numbers to notes for the fourth octave.

Table 10-2. MIDI numbers and note names

MIDI number

Note name

60

C4

61

C#4

62

D4

63

D#4

64

E4

65

F4

66

F#4

67

G4

68

G#4

69

A4

70

A#4

71

B4


A table showing the correspondence between MIDI numbers and note names can be found at http://www.phys.unsw.edu.au/tildjw/notes.html.


A channel is obtained in the following way:

     Synthesizer synthesizer = MidiSystem.getSynthesizer( );     synthesizer.open( );     MidiChannel drumChannel = synthesizer.getChannels( )[9];

Channel 9 plays different percussion and audio effect sounds depending on the note numbers sent to it.

Playing a note corresponds to sending a NOTE_ON message, letting it play, and then killing it with a NOTE_OFF message. This can be wrapped up in a playNote( ) method:

     public void playNote(int note, int duration)     {       drumChannel.noteOn(note, 70);  // 70 is the volume level       try {         Thread.sleep(duration*1000);   // secs --> ms       }       catch (InterruptedException e) {}       drumChannel.noteOff(note);     }

The following will trigger applause:

     for (int i=0; i < 10; i++)       playNote(39, 1);  // 1 sec duration for note 39

Note 39, used here as an example, corresponds to a hand clap sound. A list of the mappings from MIDI numbers to drum sounds can be found at http://www.midi.org/about-midi/gm/gm1sound.shtml.

MidiChannel supports a range of useful methods aside from noteOn( ) and noteOff( ), including setMute( ), setSolo( ), setOmni( ), and setPitchBend( ). The two MidiChannel.programChange( ) methods allow the channel's instrument to be changed, based on its bank and program numbers:

     synthesizer.getChannels( )[0].programChange(0, 15);      /* change the instrument used by channel 0 to         a dulcimer - located at bank 0, program 15 */

Instruments and soundbanks are explained in more detail later in this chapter.


Sending MIDI Messages to the Synthesizer's Receiver Port

This approach is functionally similar to the channel technique in the last section, except that I use MIDI messages directly. The advantages include the ability to direct messages to different channels and send more kinds of messages than just NOTE_ON and NOTE_OFF.

Lists of MIDI messages can be found at http://www.borg.com/tildjglatt/tech/midispec.htm and http://users.chariot.net.au/tildgmarts/midi.htm.


The receiver port for the synthesizer is obtained first:

     Synthesizer synthesizer = MidiSystem.getSynthesizer( );     synthesizer.open( );     Receiver receiver = synthesizer.getReceiver( );

As before, sending a note is two messages, separated by a delay to give the note time to play. You can conclude this logic in another version of the playNote( |) method:

     public void playNote(int note, int duration, int channel)     {       ShortMessage msg = new ShortMessage( );       try {         msg.setMessage(ShortMessage.NOTE_ON, channel, note, 70);                                  // 70 is the volume level         receiver.send(msg, -1);  // -1 means play immediately         try {           Thread.sleep(duration*1000);         } catch (InterruptedException e) {}             // reuse the ShortMessage object         msg.setMessage(ShortMessage.NOTE_OFF, channel, note, 70);         receiver.send(msg, -1);       }       catch (InvalidMidiDataException e)       {  System.out.println(e.getMessage( ));  }     }

The receiver expects MIDI events, so the MIDI message must be sent with a time-stamp. -1, used here, means that the message should be processed immediately.

The following sets up more applause:

     for (int i=0; i < 10; i++)       playNote(39, 1, 9); // note 39 sent to the drum channel, 9

A drawback with this technique, and the previous one, is the timing mechanism, which depends on the program sleeping. It would be better if the synthesizer managed the time spacing of MIDI messages by working with MIDI events that use real timestamps (called tick values). This approach is explained later in the chapter.

Control change messages

The FadeMidi and PanMidi examples in Chapter 9 show how to access channel controllers via the synthesizer and MIDI channels, such as in this example:

     MidiChannel[] channels = synthesizer.getChannels( );     // Set the volume controller for channel 4 to be full on (127)     int channelVol = channels[4].getController(VOLUME_CONTROLLER);     channels[4].controlChange(VOLUME_CONTROLLER, 127);

Another approach is to construct a MIDI message aimed at a particular channel and controller and to send it to the synthesizer's receiver.

     // Set the volume controller for channel 4 to be full on (127)     ShortMessage volMsg = new ShortMessage( );     volMsg.setMessage(ShortMessage.CONTROL_CHANGE, 4, VOLUME_CONTROLLER, 127);     receiver.send(volMsg, -1);

The second argument of the ShortMessage.setMessage( ) is the channel ID (an index between 0 and 15, not 1 and 16), the third argument is the channel controller ID, and the fourth is the message value itself.

Creating a Sequence

Rather than send individual notes to the synthesizer, the SeqSynth application creates a complete sequence that is passed to the sequencer and then to the synthesizer.

The generation of a complete sequence is preferable if the music is going to be longer than just a few notes. However, this technique requires the programmer to understand the internals of a sequence. A graphical representation of a sequence's structure is given in Figure 10-5.

Figure 10-5. The internals of a MIDI sequence


SeqSynth plays the first few notes of "As Time Goes By" from the movie Casablanca. The application can be found in the directory SoundExamps/SynthSound/.

The original MIDI note sequence was written by Heinz M. Kabutz (see http://www.javaspecialists.co.za/archive/Issue076.html).


The application constructs a sequence of MidiEvents containing NOTE_ON and NOTE_OFF messages for playing notes, and PROGRAM_CHANGE and CONTROL_CHANGE messages for changing instruments. The speed of playing is specified in terms of the ticks per beat (also called pulses per quarter [PPQ] note) and beats/minute (the tempo setting). The sequence only communicates with channel 0 (i.e., it only uses one musician), but this could be made more flexible.

Notes can be expressed as MIDI numbers or as note names (e.g., F4#). See http://www.phys.unsw.edu.au/tildjw/notes.html for a chart linking the two. This support for note names by SeqSynth is the beginning of an application that could translate a text-based score into music.

Here's SeqSynth's constructor:

     public SeqSynth( )     {       createSequencer( );       // listInstruments( );       createTrack(4);       // 4 is the PPQ resolution       makeSong( );       // makeScale(21);     // the key is "A0"       startSequencer(60);   // tempo: 60 beats/min       // wait for the sound sequence to finish playing       try {         Thread.sleep(600000);   // 10 mins in ms       }       catch(InterruptedException e)       { System.out.println("Sleep Interrupted"); }       System.exit(0);     } // end of SeqSynth( )

createSequencer( ) is nothing new: It initializes the sequencer and synthesizer objects, which are assigned to global variables.

Instruments and soundbanks

listInstruments( ) is a utility for listing all the instruments currently available to the synthesizer. The range of instruments depends on the currently loaded soundbank. The default soundbank is soundbank.gm, located in $J2SE_HOME/jre/lib/audio and $J2RE_HOME/lib/audio. It's possible to change soundbanks, for example, to improve the quality of the instruments. This is explained in the Java Tech Tip at http://java.sun.com/developer/JDCTechTips/2004/tt0309.html.

A soundbank, which is shown as a gray rectangle in Figure 10-4, can be viewed as a 2D-array, as in Figure 10-6.

Figure 10-6. A soundbank in more detail


Each box in the soundbank is an instrument (represented by an Instrument object), with its array location stored in a Patch object. To utilize an instrument at runtime, it must be referred to using its Patch details. A patch holds two values: a bank number and a program number.

The General MIDI specification defines a set of instrument names that must be supported in bank 0, for program numbers 0 to 127 (e.g., see http://www.midi.org/about-midi/gm/gm1sound.shtml). These will be available on all MIDI synthesizers. The contents of banks 1, 2, etc., can vary.

Even within bank 0, only the names are prescribed, not the actual sound, so the output can differ from one synthesizer to another.


The General MIDI specification actually talks about banks 1-128 and programs 1-128, while Java uses 0-127 for bank and program numbers. For example, the dulcimer is in bank 1, program 16 in the specification, but it is accessed using <0,15> in Java.

Listing instruments

listInstruments( ) prints out the names and patch details for the extensive set of instruments in the default soundbank:

     private void listInstruments( )     {       Instrument[] instrument = synthesizer.getAvailableInstruments( );       System.out.println("No. of Instruments: " + instrument.length);       for (int i=0; i < instrument.length; i++) {         Patch p = instrument[i].getPatch( );         System.out.print("(" + instrument[i].getName( ) +                   " <" + p.getBank( ) + "," + p.getProgram( ) + ">) ");         if (i%3 ==0)           System.out.println( );       }       System.out.println( );     } // end of listInstruments( )

The output on my machine reports on four banks (0 to 3), holding a total of 411 instruments.

Making a sequence

createTrack( ) creates a sequence with a single empty track and specifies its MIDI event timing to be in ticks per beat (PPQ). This allows its tempo to be set in startSequencer( ) using Sequencer.setTempoInBPM( ). (BPM stands for beats per minute.) It permits the tempo to be changed during execution with methods such as Sequencer.setTempoFactor( ):

     private void createTrack(int resolution)     { try {         sequence = new Sequence(Sequence.PPQ, resolution);       }       catch (InvalidMidiDataException e) {         e.printStackTrace( );       }       track = sequence.createTrack( );  // track is global     }

The other common timestamp format is based on ticks per frame and FPS.


makeSong( ) fills the sequence's single track with MIDI events. In this case, the code is concerned with reproducing the first few notes of "As Time Goes By":

     private void makeSong( )     { changeInstrument(0,33);    // set bank and program; bass       addRest(7);       add("F4"); add("F4#"); add("F4"); add("D4#");       add("C4#"); add("D4#", 3);  add("F4"); add("G4#");       add("F4#"); add("F4"); add("D4#"); add("F4#", 3);       add("G4#"); add("C5#"); add("C5"); add("A4#");       add("G4#"); add("A4#", 4); add("G4", 4); add("G4#", 2);       changeInstrument(0,15);   // dulcimer       addRest(1);       add("C5"); add("D5#"); add("C5#"); add("C5"); add("A4#");       add("C5", 2); add("C5#", 2); add("G4#", 2); add("G4#", 2);       add("C4#", 2); add("D4#", 2); add("C4#", 2);       addRest(1);     }

changeInstrument( ) is supplied with bank and program numbers to switch the instrument. addRest( ) inserts a period of quiet into the sequence, equal to the supplied number of ticks. add( ) adds a note, with an optional tick duration parameter.

Commented out in SeqSynth.java is a simpler example; makeScale( ) plays a rising scale followed by a falling one:

     private void makeScale(int baseNote)     {       for (int i=0; i < 13; i++) {   // one octave up         add(baseNote);         baseNote++;       }       for (int i=0; i < 13; i++) {    // one octave down         add(baseNote);         baseNote--;       }     }

makeScale( ) is called with the MIDI number 21 (note A0), and subsequent notes are calculated using addition and subtraction. This version of add( ) takes an integer argument rather than a string.

For the musically adept out there, this is a useful feature. In any key, you can calculate the notes of the scale numerically and not worry about note names. For example, a major scale is whole step (+2 from the root of the scale), whole step (+2), half step (+1), whole step (+2), whole step (+2), whole step (+2), half step (+1). Using those numerical values is a lot easier than remembering if E# is part of the C# major scale.


Playing the sequence

startSequencer( ) is the final method called from the constructor. It plays the sequence built in the preceding call to makeSong( ) (or makeScale( )):

     private void startSequencer(int tempo)     /* Start the sequence playing.        The tempo setting is in BPM (beats per minute),        which is combined with the PPQ (ticks / beat)        resolution to determine the speed of playing. */     {       try {         sequencer.setSequence(sequence);       }       catch (InvalidMidiDataException e) {         e.printStackTrace( );       }       sequencer.addMetaEventListener(this);       sequencer.start( );       sequencer.setTempoInBPM(tempo);     } // end of startSequencer( )     public void meta(MetaMessage meta)     // called when a meta event occurs during sequence playing     {       if (meta.getType( ) == END_OF_TRACK) {         System.out.println("End of the track");         System.exit(0);    // not required in J2SE 5.0       }     }

startSequence( ) sets the tempo and adds a meta-event listener. The listener calls meta( ) when the track finishes playing, allowing the application to exit immediately instead of waiting for the full 10 minutes allocated by the constructor.

The add( ) methods

The add( ) methods must deal with note name or MIDI number input and with an optional note-playing period:

     // global used to timestamp the MidiEvent messages     private int tickPos = 0;         private void add(String noteStr)     {  add(noteStr, 1);  }     private void add(int note)     { add(note, 1);  }     private void add(String noteStr, int period)     // convert the note string to a numerical note, then add it     { int note = getKey(noteStr);       add(note, period);     }     private void add(int note, int period)     { setMessage(ShortMessage.NOTE_ON, note, tickPos);       tickPos += period;       setMessage(ShortMessage.NOTE_OFF, note, tickPos);     }     private void addRest(int period)     // this will leave a period of no notes (i.e., silence) in the track     { tickPos += period; }

The note name is converted into a MIDI number with getKey( ). The core add( ) method takes a MIDI number and tick period, and it creates two MIDI events with setMessage( )one a NOTE_ON message and the other a NOTE_OFF. These events are timestamped, so they are separated by the required interval.

setMessage( ) builds a MIDI message, places it inside a MIDI event, and adds it to the track:

     // globals     private static final int CHANNEL = 0;  // always use channel 0     private static final int VOLUME = 90;  // fixed volume for notes     private void setMessage(int onOrOff, int note, int tickPos)     {       if ((note < 0) || (note > 127)) {         System.out.println("Note outside MIDI range (0-127): " + note);         return;       }       ShortMessage message = new ShortMessage( );       try {         message.setMessage(onOrOff, CHANNEL, note, VOLUME);         MidiEvent event = new MidiEvent(message, tickPos);         track.add(event);       }       catch (InvalidMidiDataException e) {         e.printStackTrace( );       }     } // end of setMessage( )

Changing an instrument

changeInstrument( ) is supplied with the bank and program numbers of the instrument that should be used by the channel from this point on:

     private void changeInstrument(int bank, int program)     {       Instrument[] instrument = synthesizer.getAvailableInstruments( );       for (int i=0; i < instrument.length; i++) {         Patch p = instrument[i].getPatch( );         if ((bank == p.getBank( )) && (program == p.getProgram( ))) {          programChange(program);          bankChange(bank);            return;         }       }       System.out.println("No instrument of type <" + bank +                                               "," + program + ">");     }

The validity of these two numbers are checked before they're processed.


Program and bank change

programChange( ) places a PROGRAM_CHANGE MIDI message onto the track:

     private void programChange(int program)     {       ShortMessage message = new ShortMessage( );       try {              message.setMessage(ShortMessage.PROGRAM_CHANGE, CHANNEL, program, 0);                               // the second data byte (0) is unused         MidiEvent event = new MidiEvent(message, tickPos);         track.add(event);       }       catch (InvalidMidiDataException e) {          e.printStackTrace( );       }     }

bankChange( ) is similar but uses the bank selection channel controller (number 0), so a CONTROL_CHANGE message is placed on the track:

     // global     // channel controller name for changing an instrument bank     private static final int BANK_CONTROLLER = 0;     private void bankChange(int bank)     {       ShortMessage message = new ShortMessage( );       try {         message.setMessage(ShortMessage.CONTROL_CHANGE,                                     CHANNEL, BANK_CONTROLLER, bank);         MidiEvent event = new MidiEvent(message, tickPos);         track.add(event);       }       catch (InvalidMidiDataException e) {          e.printStackTrace( );       }     }

From note name to MIDI number

The note name syntax used by SeqSynth is simple, albeit nonstandard. Only one letter-single octave combination is allowed (e.g., "C4," "A0"), so it's not possible to refer to the -1 octave. A sharp can be included, but only after the octave number (e.g., "G4#"); the normal convention is that a sharp follows the note letter. No notation for flats is included here though you can represent any flatted note with the "sharped" version of the note below it; for example, D flat is equivalent to C sharp.

The calculations done by getKey( ) use several constants:

     private static final int[] cOffsets =  {9, 11, 0, 2, 4, 5, 7};                                          // A   B  C  D  E  F  G     private static final int C4_KEY = 60;           // C4 is the "C" in the 4th octave on a piano     private static final int OCTAVE = 12;    // note size of an octave

The note offsets in cOffsets[] use the C Major scale, which is ordered C D E F G A B, but the offsets are stored in an A B C D E F G order to simplify their lookup by getKey( ).

getKey( ) calculates a MIDI note number by examining the note letter, octave number, and optional sharp character in the supplied string:

     private int getKey(String noteStr)     /* Convert a note string (e.g., "C4", "B5#" into a key. */     {       char[] letters = noteStr.toCharArray( );       if (letters.length < 2) {         System.out.println("Incorrect note syntax; using C4");         return C4_KEY;       }       // look at note letter in letters[0]       int c_offset = 0;       if ((letters[0] >= 'A') && (letters[0] <= 'G'))         c_offset = cOffsets[letters[0] - 'A'];       else         System.out.println("Incorrect: " + letters[0] + ", using C");           // look at octave number in letters[1]       int range = C4_KEY;       if ((letters[1] >= '0') && (letters[1] <= '9'))         range = OCTAVE * (letters[1] - '0' + 1);       else         System.out.println("Incorrect: " + letters[1] + ", using 4");       // look at optional sharp in letters[2]       int sharp = 0;       if ((letters.length > 2) && (letters[2] == '#'))         sharp = 1;    // a sharp is 1 note higher       int key = range + c_offset + sharp;       return key;     }  // end of getKey( )

Extending SeqSynth

SeqSynth would be more flexible if it could read song operations (i.e., a score) from a text file instead of having those operations hard-coded and passed into methods such as makeSong( ).

The range of musical notation understood by SeqSynth could be enlarged. For example, David Flanagan's PlayerPiano application from Java Examples in a Nutshell (O'Reilly) covers similar ground to SeqSynth and supports flats, chords (combined notes), volume control, and the damper pedal (http://www.onjava.com/pub/a/onjava/excerpt/jenut3_ch17/index1.html). The resulting sequence can be played or saved to a file.

Several ASCII notations represent scores, such as the abc language (http://www.gre.ac.uk/tildc.walshaw/abc/). abc is widely used for notating and distributing music. Many tools exist for playing abc notated music, converting it into MIDI sequences or sheet music, and so on. Wil Macaulay has written Skink, a Java application, which supports the abc 1.6 standard with some extensions. It can open, edit, save, play, and display abc files (http://www.geocities.com/w_macaulay/skink.html). Skink generates a MIDI sequence using similar techniques as in SeqSynth.



Killer Game Programming in Java
Killer Game Programming in Java
ISBN: 0596007302
EAN: 2147483647
Year: 2006
Pages: 340

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net