Streaming audio

From dreamcast.wiki
Jump to navigation Jump to search

Author: Andress Barajas

Introduction

This guide will focus solely on streaming sound on the Sega Dreamcast. Unlike sound effects, sound that is streamed is NOT uploaded all-at-once to the Dreamcast AICA RAM. Rather a small buffer of MAX 65536 bytes(per channel) is allocated that we constantly refill while sound is being played. Streamed sound can be either mono or stereo and can be 16-bit uncompressed PCM, 8-bit PCM or 4-bit Yamaha ADPCM. The max amount of streams that can exist at once is 4. This hard limit does not apply to sound effects. You can have 4 sound streams playing at once and have many sound effects depending on how much AICA RAM is left.

The libraries in kos-ports that most people use to stream sound are libmp3, libogg, and libtremor. These sound libraries take care of all the nitty-gritty stuff covered in the API section. So much so that you can skip the API section and go straight to the code example section to get started.

The approach these sound libraries use to stream sound is to create and run a detached thread in the background that constantly polls the AICA, asking it how much sound data it wants so it can keep streaming. This polling executes a callback where we then decode sound data into a buffer and return that buffer to the AICA to read from. This process continues until we reach the end of the sound file that we are playing.

API

The KOS reference manual for this API contains more detail on each function and also has more functions not listed here. The source code for most of these methods can be found within snd_stream.c.

Initialize Sound System

int snd_stream_init()

The snd_stream_init() function initializes the sound system for streaming by allocating some internal stream buffers. It is usually called at the beginning of your program. You can't stream sound before calling this function. This will implicitly call snd_init(), so you can also play sound effects.

Stream Callback

typedef void*(* snd_stream_callback_t)(snd_stream_hnd_t hnd, int smp_req, int *smp_recv)

Functions for providing stream data will be of this type, and can be registered with snd_stream_alloc() or snd_stream_set_callback().

  • hnd The stream handle being referred to.
  • smp_req The number of samples requested.
  • smp_recv Used to return the number of samples available.

Returns: A pointer to the buffer of samples. If stereo, the samples should be interleaved.

Allocate Sound Stream

snd_stream_hnd_t snd_stream_alloc(snd_stream_callback_t callback, int bufsize)

After initializing the sound system you can start allocating sound streams. The snd_stream_alloc() function allocates a stream and sets its parameters. The maximum number of streams that can be allocated at one time is 4 (SND_STREAM_MAX ). This function returns a stream handler that will be used to play/stop streams among other things.

  • callback is the function where we decode and send sound data to AICA.
  • bufsize is the size of the streaming buffer we want to create in AICA RAM. The maximum size of the buffer we can set is SND_STREAM_BUFFER_MAX (65536 bytes...per channel so 65536*2 bytes internally to support stereo) . The smaller the buffer size means the sound buffer is filled up faster, which means the sound begins to play quicker. This however means we are decoding many small portions of sound data at a time rather than in bigger chunks. This causes overhead because we are calling the callback function many more times by doing that. I will explain stream queuing which gives us the benefit of both using the max buffer size and having sound playing instantaneously when you start streaming.

Start Stream

void snd_stream_start(snd_stream_hnd_t hnd, uint32 freq, int stereo)

The snd_stream_start() function starts a stream. This function starts processing the given stream, prefilling the buffers as necessary. In queueing mode, this will not start playback.

  • hnd The stream to start.
  • freq is the frequency of the audio data. Most of the time this is gonna be 44.1Hz.
  • stereo is a boolean saying whether the sound data we are streaming is 1-Stereo or 0-Mono

Poll Stream

int snd_stream_poll(snd_stream_hnd_t hnd)

The snd_stream_poll() function polls the specified stream to load more data if necessary. If more data is requested, the callback function set in snd_stream_alloc() is executed.

  • hnd The stream to poll.

Stop Stream

void snd_stream_stop(snd_stream_hnd_t hnd)

The snd_stream_stop() function stops a stream, stopping any sound playing from it. This will happen immediately, regardless of whether queueing is enabled or not.

  • hnd The stream to stop.

Stream Volume

void snd_stream_volume(snd_stream_hnd_t hnd, int vol)

The snd_stream_volume() function sets the volume of the specified stream.

  • hnd The stream to set volume on.
  • vol The volume to set. Valid values are 0-255.

Set Stream Callback

void snd_stream_set_callback(snd_stream_hnd_t hnd, snd_stream_callback_t callback)

The snd_stream_set_callback() function updates the get data callback function for a given stream, overwriting any old callback that may have been in place.

  • hnd The stream handle for the callback.
  • callback A pointer to the callback function.

Destroy Stream

void snd_stream_destroy(snd_stream_hnd_t hnd)

The snd_stream_destroy() function destroys a previously created stream, freeing all memory associated with it.

  • hnd The stream to clean up.

Shutdown Stream System

void snd_stream_shutdown()

The snd_stream_shutdown() function shuts down the stream system and frees the memory associated with it. This does not call snd_shutdown().

Stream Queuing

The stream queuing system allows you to setup a stream to be played, prefilling the buffers, without actually starting the stream. This can be useful if you want to setup a stream long before actually using it. Since the buffers are pre-filled with sound data, playback of the stream is immediate.

Normally the route to start playback of a stream is to first allocate a stream using snd_stream_alloc() and then use snd_stream_start(). What snd_stream_start() does internally is that it decodes sound data until the buffer (which you set the size of in snd_stream_alloc()) is full and then sound starts playing. This process isn't instant because decoding the sound data and moving it to the AICA takes time. Normally this is not an issue but if you want to sync the playback of the stream with a button press you will get a delay:

if(A_BUTTON_IS_PRESSED)) { 
    snd_stream_start(hnd, freq, stereo); 
}

What you should do instead after allocating a stream using snd_stream_alloc() is to call snd_stream_queue_enable() and then snd_stream_start(). Because we enabled queuing of the stream, the stream will not start playback when we call snd_stream_start(). It just pre-fills the buffers for us. Then in your control input loop:

if(A_BUTTON_IS_PRESSED)) { 
    snd_stream_queue_go(hnd); 
}

Enable Stream Queue

void snd_stream_queue_enable(snd_stream_hnd_t hnd)

The snd_stream_queue_enable() function enables queueing on the specified stream. This will make it so that you must call snd_stream_queue_go() to actually start the stream, after scheduling the start. This is useful for getting something ready but not firing it right away.

  • hnd The stream to enable queueing on.

Disable Stream Queue

void snd_stream_queue_disable(snd_stream_hnd_t hnd)

The snd_stream_queue_disable() function disables queueing on the specified stream. This does not imply that a previously queued start on the stream will be fired if queueing was enabled before.

  • hnd The stream to disable queueing on.

Start Queued Stream

void snd_stream_queue_go(snd_stream_hnd_t hnd)

The snd_stream_queue_go() function makes the stream start once a start request has been queued, if queueing mode is enabled on the stream.

  • hnd The stream to start the queue on.

Stream Filtering

This allows you to apply filters to the decoded sound data that is returned by the callback you set in snd_stream_alloc(). In order to filter the sound you need to create a function with the filter callback(shown below) and adding it to a stream using snd_stream_filter_add(). This can create cool dynamic sound manipulation. Manipulating sound data is out of scope of this tutorial and requires you to study DSP(Digital Signal Processing). Keep in mind that you will only be manipulating small portions of sound data at a time.

Filter Callback

typedef void(* snd_stream_filter_t)(snd_stream_hnd_t hnd, void *obj, int hz, int channels, void **buffer, int *samplecnt)

Functions providing filters over the stream data will be of this type, and can be set with snd_stream_filter_add().

  • hnd The stream being referred to.
  • obj Filter user data.
  • hz The frequency of the sound data.
  • channels The number of channels in the sound data.
  • buffer A pointer to the buffer to process. This is before any stereo separation is done. Can be changed by the filter, if appropriate.
  • samplecnt A pointer to the number of samples/bytes. This can be modified by the filter, if appropriate.

Add Stream Filter

void snd_stream_filter_add(snd_stream_hnd_t hnd, snd_stream_filter_t filtfunc, void* obj)

The snd_stream_filter_add() function adds a filter to the specified stream. The filter will be called on each block of data input to the stream from then forward.

  • hnd The stream to add the filter to.
  • filtfunc A pointer to the filter function.
  • obj Filter function user data.

Remove Stream Filter

void snd_stream_filter_remove(snd_stream_hnd_t hnd, snd_stream_filter_t filtfunc, void* obj)

The snd_stream_filter_remove() function removes a filter that was previously added to the specified stream.

  • hnd The stream to remove the filter from.
  • filtfunc A pointer to the filter function to remove.
  • obj The filter function's user data. Must be the same as what was passed as obj to snd_stream_filter_add().

Code Example

The following code example, which is mostly devoid of the API functions listed above, demonstrates how to stream sound using the libtremor library. The libtremor is a playback library used to play .ogg sound files. liboggvorbisplay is another library that plays .ogg files but uses float arithmetic and consumes more memory. Stick with libtremor. I also recommend that you avoid using libmp3 as a playback library if you want to keep your project closed source since its licensed under GNU General Public License version 2 and requires your project to be open source.

This code example requires that you have libtremor installed. In your kos-ports directory you will find libtremor. Using your terminal, navigate to that folder and execute:

make install clean

Error handling is non-existent in this example. You'll want to check if the sound effects are loaded correctly and that you have a controller plugged in when running this demo.

You can find all the code and resources of this example on Github. Three versions of this example exist for each playback library(libtremor, libmp3, and libwav).


NOTE: This code sample may not work in Sega Dreamcast emulators.

#include <kos.h>
#include <oggvorbis/sndoggvorbis.h>

#define LEFT 0
#define CENTER 128
#define RIGHT 255

#define LOOP 1
#define SHAKER_VOL 200
#define INITIAL_CRY 128
#define LOUDEST_CRY 240

extern uint8 romdisk[];
KOS_INIT_ROMDISK(romdisk);

static void draw_instructions(uint8_t volume);

static cont_state_t* get_cont_state();
static int button_pressed(unsigned int current_buttons, unsigned int changed_buttons, unsigned int button);

int main(int argc, char **argv) {
    uint8_t baby_volume = INITIAL_CRY;
    uint8_t shake_volume = SHAKER_VOL;

    uint64_t start_time = timer_ms_gettime64();
    uint64_t end_time = start_time;

    cont_state_t* cond;

    vid_set_mode(DM_640x480, PM_RGB555);
    // Initialize sound system and OGG
    snd_stream_init();
    sndoggvorbis_init();

    // Load wav files found in romdisk
    sfxhnd_t shake1 = snd_sfx_load("/rd/shake-1.wav");
    sfxhnd_t shake2 = snd_sfx_load("/rd/shake-2.wav");

    sndoggvorbis_volume(baby_volume); // Can put volume before starting 
    sndoggvorbis_start("/rd/baby-whining-01.ogg", LOOP);

    unsigned int current_buttons = 0;
    unsigned int changed_buttons = 0;
    unsigned int previous_buttons = 0;

    for(;;) {
        cond = get_cont_state();
        current_buttons = cond->buttons;
        changed_buttons = current_buttons ^ previous_buttons;
        previous_buttons = current_buttons;
        
        // Play rattle sounds to calm the baby
        if(button_pressed(current_buttons, changed_buttons, CONT_X)) {
            snd_sfx_play(shake1, shake_volume, LEFT);
            if(baby_volume > 40)
                baby_volume -= 10;
        }
        if(button_pressed(current_buttons, changed_buttons, CONT_Y)) {
            snd_sfx_play(shake2, shake_volume, RIGHT);
            if(baby_volume > 40)
                baby_volume -= 10;
        }
        // Wake the baby
        if(button_pressed(current_buttons, changed_buttons, CONT_A)) {
            if(!sndoggvorbis_isplaying()) {
                baby_volume = INITIAL_CRY;
                sndoggvorbis_volume(baby_volume);
                sndoggvorbis_start("/rd/baby-whining-01.ogg", LOOP);
            }
        }
        // Force the baby to sleep
        if(button_pressed(current_buttons, changed_buttons, CONT_B)) {
            baby_volume = 0;
            sndoggvorbis_stop();
        }
        // Exit Program
        if(button_pressed(current_buttons, changed_buttons, CONT_START))
            break;

        // Adjust Volume with time
        end_time = timer_ms_gettime64();
        // Increase baby volume by 15 every second (Max LOUDEST_CRY)
        if((end_time - start_time) >= 1000) {
             baby_volume += 15;
             start_time = end_time;
        }

        if(baby_volume > LOUDEST_CRY) {
            baby_volume = LOUDEST_CRY;
        }

        // If baby volume goes <= 40, put baby to sleep
        if(baby_volume <= 40) {
            baby_volume = 0;
            sndoggvorbis_stop();
        } else {
            sndoggvorbis_volume(baby_volume);
        }

        draw_instructions(baby_volume);
    }

    // Unload all sound effects from sound RAM
    snd_sfx_unload_all();	

    sndoggvorbis_stop();
    sndoggvorbis_shutdown();
    snd_stream_shutdown();

    return 0;
}

static void draw_instructions(uint8_t baby_volume) {
    int x = 20, y = 20+24;
    int color = 1;
    char baby_status[50];
    memset(baby_status, 0, 50);
    
    if(baby_volume == 0) {
        snprintf(baby_status, 50, "%-50s", "Baby is asleep!!! Finally...");
    }
    else if(baby_volume > 40 && baby_volume <= 90) {
        snprintf(baby_status, 50, "%-50s", "Baby is almost asleep. Keep rattling!!");
    }
    else if(baby_volume > 90 && baby_volume <= 180) {
        snprintf(baby_status, 50, "%-50s", "You can do better than that!");
    }
    else if(baby_volume > 180 && baby_volume <= LOUDEST_CRY) {
        snprintf(baby_status, 50, "%-50s", "Are you even rattling!?!?!");
    }
    
    bfont_draw_str(vram_s + y*640+x, 640, color, "Press X and/or Y to play rattle sounds to calm");
    y += 24;
    bfont_draw_str(vram_s + y*640+x, 640, color, "down the baby so it can go to sleep");
    y += 48;
    bfont_draw_str(vram_s + y*640+x, 640, color, "Press A to wake the baby up");
    y += 48;
    bfont_draw_str(vram_s + y*640+x, 640, color, "Press B to force baby asleep");
    y += 48;
    bfont_draw_str(vram_s + y*640+x, 640, color, "Press Start to exit program");
    y += 72;
    bfont_draw_str(vram_s + y*640+x, 640, color, baby_status);
}

static cont_state_t* get_cont_state() {
    maple_device_t* cont;
    cont_state_t* state;

    cont = maple_enum_type(0, MAPLE_FUNC_CONTROLLER);
    if(cont) {
        state = (cont_state_t*)maple_dev_status(cont);
        return state;
    }

    return NULL;
}

static int button_pressed(unsigned int current_buttons, unsigned int changed_buttons, unsigned int button) {
    if (changed_buttons & button) {
        if (current_buttons & button)
            return 1;
    }

    return 0;
}