Sound Effects An Easy Way

Sound FX An Easy Way

This tutorial is aimed at any user who is curious as to how to get sound effects into their game. As the title suggests, today we will discuss one way of doing just that. If you are like me, the idea of creating a patch with nothing but your imagination and a text editor seems daunting. Especially if we really don't understand everything about the sound system! We are in luck however, we don't have to resort to that. I will briefly explain how you can have quality sound effects, with little or no trial and error(depending on complexity). I am assuming you have some understanding of what patch commands are, if not skim through this [1]and this[2] first. Also, you will need to figure out how to navigate and do the basic things required with the tools by yourself, otherwise this tutorial will get cluttered.

We will be working with NES sound effects, as the Uzebox kernel is pretty similar in capabilities. You might also be able to use some SNES format sounds. The process would be the same but you would use SPC2MIDI, but let's get back on track. Some NES sound effects could have multiple channels and even PCM data simultaneously.

The Uzebox kernel treats patches as a list of sequential instructions operating on 1 channel at a time. You could split sounds into multiple patches to be played simultaneously (with appropriate frame delays), but this example will be kept simple. You should be able to figure out the former case once you know these basics. You can also ignore some extra details and still end up with a good sound. I have only recently started thinking about how to do this, and this method has made things easy for my purposes. Hopefully you'll get the same mileage or adapt it to your needs.

Required Tools

For this to work we are going to need a few things. All of these are available freely and are adequate for this purpose.

• Anvil Studio[3] - This is a good free MIDI development tool(Windows). Ubuntu users should find Rosegarden or Muse in normal repositories(haven't tried them).

• NSF2MIDI[4] - This tool will convert NSF(NES Sound Format) files to MIDI files. The method I will describe is a way to use the created MIDI as a base for your sound effect.

• NSF File[5] - You will need a NSF file that has the sound effect(s) you want to reproduce. Not all NSF have effects, you may need to try different versions or settle on something else.

• steptable.inc[6] - Get this from kernel\data\. It's a handy reference sheet.

Initial Process

After we have acquired everything we needs it's time to get started. Here's some steps before we dig in.

• Put on some music, gently explore your tools and get to know them a little before we start(just make sure no one is around...)

• Look around for a NSF files with sound effects you like. Check out [7]Zophar's extensive collection. For this demonstration I will be using Adventures of Lolo NSF which can be found here[8].

• Load the NSF into NSF2MIDI. Using your arrow keys navigate until you find your desired sound effect. Keep in mind that the sound effect is being composed with MIDI commands. It may sound quite different than the way it sound's in the actual game. In our example I'm looking for the sound that plays when Lolo picks up a heart. Scrolling through I find that #11 is the effect I'm looking for. Now we go to File->Log to SMF , save the file and open it up with Anvil Studio. Time to see some sound! You will see a display of tracks, click the one that has a bunch of notes(see below). Now click the compose button on the left hand side around the middle. You will see a drop down box, select piano roll and you should see something that looks like this.

Most sound effects have notes that are very close together. You will want to set your grid to 1/128 notes(above)so you can more accurately look at them. Each horizontal line segment represents an MIDI note/event. Secondary click on one of the notes and click properties. You're provided with the time that the note starts, volume, and the duration in midi ticks. For this example the duration is meaningless to us. All the notes are 100 for volume, but this might not always be the case. Keep this in mind for later. Before we get too carried away we should briefly touch on what we are actually trying to do with this information. I'm assuming you understand the most basic patch commands. This is not a complicated patch but thats why I chose it :) Our goal is to create this:

  const char patch06[] PROGMEM ={//heart pickup
  0,PC_WAVE,4,              //Tell the kernel which wave we will be working off.
  1,PC_PITCH,69,            //We want to set the pitch to note 69. The first argument tells the kernel we want this to happen 1 frame after we call TriggerFx()
  1,PC_PITCH,73,            //Change frequency to note 73, 1 frame after the above command.(So 69 ends up playing for 1 frame from the start)
  1,PC_PITCH,70,            //1 frame later...
  1,PC_PITCH,74,
  1,PC_PITCH,71,
  1,PC_PITCH,75,
  1,PC_PITCH,72,
  1,PC_PITCH,76,
  1,PC_PITCH,73,
  1,PC_PITCH,77,
  1,PC_NOTE_CUT,0,         //After the note above plays for 1 frame, stop the effect entirely
  0,PATCH_END              //End of patch
  };

Maybe you don't know where I am pulling these numbers from? Open up [9].\kernel\data\steptable.inc and you will see precomputed list of all the frequencies the kernel will play. Seek until you see this:

  //steptable.inc
  .   
  ..
  ...
  .word 0x06c2 // Note: 68 (G#-6), MIDI note: 68, Freq:415.3046975799451, Step:6.757100577603021
  .word 0x0729 // Note: 69 (A-6) , MIDI note: 69, Freq:440.0000000000000, Step:7.158898686844278
  .word 0x0796 // Note: 70 (A#-6), MIDI note: 70, Freq:466.1637615180899, Step:7.584588954968734
  .word 0x0809 // Note: 71 (B-6) , MIDI note: 71, Freq:493.8833012561241, Step:8.035592083674509
  .word 0x0883 // Note: 72 (C-7) , MIDI note: 72, Freq:523.2511306011972, Step:8.513413253978852    <------
  ...
  ..
  .

Notice the note names after the note number(in parenthesis). Since Anvil Studio displays the notes by musical name(A,A#,B,C,...), we will have to find what number our first note is. Higher notes are higher vertically on the piano roll editor. You should hopefully see a C note somewhere followed by a number. If not you will see an "add sounds" button located below the note names. There you can specify the range of notes that will be displayed. In our example we see C-7 which is the 7th Octave C (based on 12 notes/octave). We see that in the step table this is note 72. Notice that the leftmost note starts at midi tick 1, and is 3 steps down from C-7. So 72-3 would be note 69 (A-6)for the very first note. Notice the first pitch entry in the patch we're trying to make corresponds to this. Checking the next note we see that there is a bit of overlap, but one note always starts slightly before another. Click on properties to check midi start time for each note, you'll see all of them start 1 midi tick after the last one. So in our patch you'll see each frequency change relates to this by happening 1 frame after the last one(since the tempo is 60). Makes sense no?

We can only play one frequency at a time, so the overlap period we will just forget about. This is why duration means nothing in this example, only the time that the next note starts. To make this fast let's count the notes, there are 10. Copy and paste the X,PC_PITCH,XX command so you have 10 copies of it each with a delay of 1 between them. Now all we have to do is change the note number for each of them to correspond to the notes we see on the Piano Roll. Use whatever method is easiest to convert this to note numbers. Personally I just count the number of steps up or down from the previous note I did(which is why we look up the first note). Then I add or subtract that from the note value I calculated last time and repeat process. Then you don't have to go back and forth looking at the steptable. Not all sounds will have so few notes and it can get confusing going back and forth(especially with notes so close together). I recommend after you find the value for a note just delete it. Then you will always be working off the left most note. That's really all there is to it at this point. After this I needed another effect; when Lolo picks up a different kind of heart. It turns out to be the exact same sound only 1 octave higher. In this instance I just work off the existing patch, but add 12 steps to each frequency which brings it up to the same note in the next octave(ie C6 to C7). In the same way you could work off a sound effect that is close to what you want, and then fine tune it. Most likely it would still be easier than making it from scratch. Let's hear what this modification sound likes and compare it to the original sound recorded from an emulator.

TODO Original Sound TODO Our patch

Not bad. I would say this is a lot more accurate than we could have done with just an emulator and a text editor. For this simple of a patch I would call this conversion done. It is very close to the original and the actual frequencies are right on. If only it was that easy for all effects! Now we need to figure out what to do if the effect is not simple, and this approach does not yield an accurate enough sound(though at least your frequencies should be correct at this point).

Detangling Notes

If you sense something is missing, check your source midi for situations where a note starts and runs for a duration that totally contains the start and end of another. The general case will probably be best solved by ending the first long note right before the overlapped note starts. Then as soon as that short note is over, start the long note again until the original endpoint it had. This should usually sound pretty good, but it is up to you to determine which note is more important for the sound and adjust fire accordingly. Here's what this looks like:

Using Waves

At the start of each patch, you will have something like 0,PC_WAVE,4,. Assuming you have read the sound engine documentation, you have at least a concept of what this is. I find this is really where you dial in your sound, because it specifies what wave(included in the kernel) the sound works off. So your frequencies and controllers are modifying this base wave, but of course if you don't have the right source it wont sound right. I just trial and error until it sounds right, there aren't many waves to try. You get a feel for the direction from 0 being a pure sine wave, and onwards gets a bit more square wave up to a distorted "metallic". Almost certainly you will be able to find one that sounds very good with your patch. In the rare case you can't you may wish to fabricate your own source wave. I may make a tutorial on this once I have tried it, let me know if that interests anyone. I also recommend you look at Space_Saving_Tricks for an example of what to do with unused waves, after you are done with all your patches.

Using Envelope And Controllers

• I could be wrong on a few details, I will try to ensure everything is accurate!

So our sound is close but not quite right? This is where some trial and error could kick in, because it is possible that the sound effect in the game is actually applying controllers. If you look at your NSF2MIDI tool under Settings->Patch Editor you will see some check boxes for these. Some of the controllers are not handled, or at least may sound different on the Uzebox. Luckily this doesn't concern us much, since we are just after the notes right now. This is where you will have to use your ears and common sense. You should set up a skeleton project and make your main function look something like this:

  ...
  ...
  #include "YourPatches.inc"

  int main(){
     //...
     //...
     InitMusicPlayer(YourPatches);
     
     while(true){
        TriggerFx(YOUR_PATCH_NUM, 0xFF, true);
        WaitVsync(60);
     } 
     return 1;
  }

Now you have a project that rebuilds quickly and plays your sound effect every second. I recommend using headphones and listen to it several times. Envelopes are instruction that tell the kernel to start doing something continually every frame until we tell it otherwise. As you modify envelope parameters hopefully you get a feel for the changes in sound. How do we use envelopes? Here is an example:

  //....issue some patch commands
  0,PC_ENV_VOL, 100, 

  0,PC_ENV_SPEED,-4,   //Instruct the kernel that our volume should drop by 4 steps every frame that follows
  //1 VOL = 96
  //2 VOL = 92
  //3 VOL = 88
  //4 ...
  //the volume keeps falling 4 each frame until we reach the next command, giving us a "sinking sound"
  
  10,PC_ENV_VOL,80,     //Set the volume of the patch to 80, though it will continue to fall from here due to the previous ENV_SPEED -4
 
  0,PC_ENV_SPEED,0,     //Now we have set the envelop to 0 so the volume will stay at 80 as defined above. We create a period of time where the note holds 1 volume.

  6,PC_ENV_SPEED,2,    //After a 6 frame delay, start increasing the volume by 2 steps each frame.
  
  //more commands....

• And how about a tremolo example:

  0,PC_WAVE,1,

  0,PC_TREMOLO_LEVEL,180, //set up the severity of the tremolo effect     
  0,PC_TREMOLO_RATE,60,   //set the frequency that the tremolo goes up and down

  0,PC_ENV_VOL,44,        //set our volume low
  0,PC_ENV_SPEED,11,      //start raising the volume slowly, this give a slow attack sound to it something like an organ perhaps
  5,PC_ENV_SPEED,0,       //let the volume rise for 5 frames, now hold it at the peak volume 
  5,PC_ENV_SPEED,-25,     //after holding it at peak for 5 frames, create a quick decay and make the sound fall off pretty fast.
  3,PATCH_END             //let it decay for 3 frames then call it done.

Some of this you will just need to get a feel for, eventually you should be able to visualize the sound in your head and have a decent idea of attack/peak dwelling/decay/etc. If you get good enough at this, you might actually be able to use a program that displays waves forms and use that as a basis to create sounds. Hmmm, maybe a tutorial if that works! It doesn't have to be exactly like the original to sound good. Hopefully I will have a better explanation later ;) Till next time, I'll add some more information as I acquire it.

• Anything I didn't touch at all, let me know. Contributions are welcome.