Buildlog.Net Blog

Game Audio for the ESP32

I have been working on some games for the ESP32 and needed some decent quality audio with a minimum number of additional components.  I was bouncing between using the DAC and using the I2S bus. The DAC requires less external parts, so I went that way. I ended up creating a very simple library for use in he Arduino IDE. (Note: This only works with ESP32)

DACs

The ESP32 has (2) internal 8 bit DACs. DACs are Digital to Analog Converters. You give them an 8 bit value and they output an analog voltage. The voltage range of the 0-255 8-bit value is roughly Gnd to 3.3V on the ESP32. If you digitize an audio file, you can then play it back through the DAC. The DAC can’t directly drive a basic speaker, but you can connect it to an amplifier or amplified external speakers. The 8-bit quality is not great, but more than good enough for simple games.

The sound data comes from wave (.wav) files. This uses the 8 bit, unsigned, uncompressed format.  Wave files can be saved with any sampling rate you want. To get a decent sound you need to sample the  analog files at thousands of times per second. I have found that 4000 Hz is about the minimum for anything musical or spoken. 8000 sounds pretty good for most stuff. For reference, CDs are sampled a 44100 Hz (also 16-bit not 8-bit). This means there is a lot of data even for short duration audio. The typical ESP32 board has 2 to 4MB. At an 8000Hz sampling rate that gives you 125 seconds per MB. That is plenty for what I am working on, so I decided to store the data on the ESP32. If that is not good enough, you could use an SD card.

Here is a reference of the wave format. Only a few of the values are used.

Byte	Length	Description
0	4	Always "RIFF"
4	4	File size in Bytes
8	4	Always "WAVE"
12	4	Always "fmt " with a null
16	4	Length of format
20	2	Audio format (1=PCM)
22	2	Number of channels
24	4	Sample rate
28	4	Byte Rate
32	2	Block Align
34	2	Bits per sample (8,16)
36	4	Alaways "data"
40	4	File size
44	...	Start of data

References

I found some good references to help me with this library.

• IoT Sharing: This was a great reference for understanding the basics, but it uses an SD card to save the data.
• Xtronical: This was very helpful and my code is very close to this.
• Wave file links: Wikipedia, A good reference of the header, another one.

I started with the Xtronical XT_DAC_Audio library, but found it got be very unstable when I added it to my game. It often crashed when seemingly unrelated code was added. I think the problem was due to trying to use floating point math in the interrupt. As soon as I took that out, it got very stable. They make reference to the issue in the code and suggest it could get fixed via compiler updates.

How it works

The wave data is stored in large char arrays. An interrupt reads the data and feeds that to the DAC. Using an interrupt allows the audio to play “in the background” while the game is running. The Xtronical library uses a fixed interrupt rate of 50kHz. It does some floating math in the interrupt to adjust to the actual sample rate of the wav file. I decided to change the interrupt rate to the actual sample rate of the wav data.

Here is a snippet of what wave data array looks like.

unsigned char PROGMEM pacman[33850] = {
  0x52, 0x49, 0x46, 0x46, 0x32, 0x84, 0x00, 0x00, 0x57, 0x41, 0x56, 0x45,
  0x66, 0x6D, 0x74, 0x20, 0x10, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00,
  0x40, 0x1F, 0x00, 0x00, 0x40, 0x1F, 0x00, 0x00, 0x01, 0x00, 0x08, 0x00,
  0x64, 0x61, 0x74, 0x61, 0xC6, 0x83, 0x00, 0x00, 0x7F, 0x7F, 0x7F, 0x7F,
  0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F,
  0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F,
  0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7F, 0x7E, 0x80, 0x7E, 0x80, 0x7E, 0x80,
  0x7E, 0x80, 0x7D, 0x84, 0xA2, 0x9F, 0xAF, 0xC0, 0xCE, 0xE1, 0xED, 0xF1,
  0xED, 0xE8, 0xDC, 0xCB, 0xB5, 0xA4, 0x95, 0x92, 0x91, 0x94, 0x9D, 0xB1,
  0xB6, 0xB5, 0xB9, 0xBB, 0x99, 0x7D, 0x66, 0x51, 0x64, 0x65, 0x5E, 0x74,
  0x9D, 0xAF, 0xC9, 0xC4, 0xB8, 0xC1, 0xBB, 0xA3, 0x89, 0x73, 0x54, 0x3B,
  0x2E, 0x30, 0x35, 0x42, 0x56, 0x70, 0x7F, 0x89, 0x8E, 0x8C, 0x86, 0x77,
  0x5E, 0x49, 0x3E, 0x34, 0x31, 0x42, 0x5A, 0x72, 0x91, 0xB1, 0xC2, 0xC1,
  ...... 

 

The sample rate is stored in the header of the wav data. When you play the file, it sets the interrupt rate to match the sample rate, so no math is required. You can use a mix of sample rates in the waves you use. It will adjust as needed before it is played back. It allows 2000Hz to 50000Hz sample rates. If you try to go outside that range it will use 2000 Hz. You will easily know there is a problem if you hear that.

Game Related Features

I added some features that might be needed when using with a game.

• Check to see if the audio player is currently playing anything.
• Interrupt The Current Wave – If something happens like an explosion you need to immediately switch to that sound. This is an option when you send a command to play a wave.
• Change the Frequency – I had some sounds that needed to change based on game speed. There is an option speed up r slow down the play back speed by a multiplier to give.

Using the library

• Download the zip file.
• Install the library using the Sketch…Include Library…Add .ZIP Library menus.
• It will also add an example. Open that using the File…Examples…Game_Audio menus.
• Create a character array of the wav data. See this video from XTronical for how to do that.
• Give the XTronical video a thumbs up and a nice comment 🙂
• Create a Game_Audio_Class object. The first parameter is the DAC pin number you want to use. It should be 25, 26. The second parameter is the timer number to use. Example: Game_Audio_Class GameAudio(25,0);
• Create a Game_Audio_Wav_Class object. The only parameter is name of the char array of you wave data. Example: Game_Audio_Wav_Class pmDeath(pacmanDeath);

Here is the demo that installs with the library

#include "SoundData.h";
#include "Game_Audio.h";

Game_Audio_Class GameAudio(25,0);

Game_Audio_Wav_Class pmDeath(pacmanDeath); // pacman dyingsound
Game_Audio_Wav_Class pmWav(pacman); // pacman theme

void setup() {
Serial.begin(115200);
Serial.println("Begin...");
}

void loop() {

// ------------- Play 2 wavs one after the other ----------------

Serial.print("Pacman Theme Sample Rate (Hz):");
Serial.println(pmWav.getSampleRate());
Serial.print("Duration (secs):");
Serial.println(pmWav.getDuration());

GameAudio.PlayWav(&pmWav, false, 1.0);

// wait until done
while(GameAudio.IsPlaying()){
}

delay(200);

GameAudio.PlayWav(&pmDeath, false, 1.0);

// wait until done
while(GameAudio.IsPlaying())
{
}

// -------- Play a wav then interrupt it after 600ms ------------

GameAudio.PlayWav(&pmWav, false, 1.0);
delay(600); // this is less than the duration of the last wav

// this wav will not play because the other is playing and interruptCurrent is false
GameAudio.PlayWav(&pmWav, false, 1.0);
// this will interrupt the playing file
GameAudio.PlayWav(&pmDeath, true, 1.0);

// --------- Play a wav with increacing sample rate (speeds up playback)
for (int i=0; i<10; i += 2)
{
while(GameAudio.IsPlaying()){// wait for wav to complete
}
GameAudio.PlayWav(&pmWav, false, (1.0) + (float)i / 10.0);
}

while (true) { // stop here program is done...but anything still playing will complete
}

}

Video

Next Steps

• I’ll am working on some games, so new features might be added.
• I’ll probably add the option to use I2S audio rather than the DAC.
• I might add the option to pull files from an SD card.

---

If you want to be notified of future blog posts, please subscribe.