libretro: mix audio the way the standalone build does (#70)

* libretro: do not ask the emulator to stop when presenting a frame it did not

A frame ends when the emulator asks to be let go: render_framebuffer_updated()
hands the frontend the picture and calls system_request_exit(), which is what
returns control from retro_run(). A system that comes back without presenting
anything - the media player, which has no video at all - would leave the
frontend without a frame, so this file presents one on its behalf, and it did
that by calling render_framebuffer_updated() as well.

That request to stop does not expire. It is still standing when the next
retro_run() enters the emulation loop, which is written as

	while (!gen->m68k->should_return)

so the loop ends before a single instruction runs, clears the flag, and returns
with nothing presented - which makes this file present a frame again, and ask
the emulator to stop again. One missed frame and the machine never runs again.

A Mega Drive never hit it because it presents a frame on the very first call. A
Sega CD does not: the call that starts it returns while the BIOS is still
bringing the machine up, and the deadlock closes over it. Sonic CD sat on a
black screen with both 68000s frozen - the main one at 420 cycles, the sub one
still held in reset - for as long as it was left running.

Presenting and asking to stop are two things, so separate them: the frame this
file owes the frontend goes out through present_framebuffer(), and only the
emulator's own frame end asks for the exit. The media player does not need the
request either; it sets should_return itself in the IS_LIB build.

Sonic CD now boots and runs. Measured over 600 frames: 95.7% of the audio the
declared sample rate calls for, where before there was none at all, and the
picture changes rather than staying the black frame it started on.

* libretro: mix audio the way the standalone build does

render_do_audio_ready() was the whole mixer: it swapped a source's buffers and,
once every source had one ready, mixed four frames and handed them over. Four is
what a source's buffer holds in the sync-to-audio path, so with the FM and the
PSG - clocked from the same master clock, resampled to the same rate - the two
arrive together and every buffer is consumed as it is filled.

Nothing else about a Mega Drive holds. A Sega CD adds CD-DA and the PCM chip,
which fill at a pace of their own, so a source can fill twice before another
fills once. The second fill swaps over a buffer nobody mixed: four frames gone,
which is what a click is. The standalone does not have this problem because its
emulation thread waits on the audio callback there, or breaks the wait with
audio_deadlock_hack(), and neither is available to a core that must return from
retro_run().

Since a frame here ends when the VDP hands over a framebuffer, audio does not
have to pace anything, so use the arrangement render_audio.c already has for
that case: report render_is_audio_sync() as false, give each source a ring
buffer of its own, and let render_do_audio_ready() do nothing but publish what
has been written. retro_run() then mixes with mix_and_convert() and hands the
frontend a batch, the same call the standalone's audio callback makes.

How much to mix is what every source that has anything can cover, which
audio_buffered() reports; the rest stays queued rather than being mixed past
what a source has written. Sources with nothing at all are left out of that
count, or a CD-DA source with no track playing - which never writes anything,
ever - would hold up the mixing forever, and taking the busiest source instead
pads the ones behind it with silence.

Measured over 600 frames each. Sonic 1: 100.0% of the frames the declared
sample rate calls for, before and after, in 1200 batches rather than 133305.
Sonic CD, which has all four sources going: 95.7% before, 99.8% after, and no
silent frame in either. Neither shows a step between samples larger than a real
waveform takes.

It also fixes the soft reset, which was reported separately as "reset the
game freezes". Measured with the same harness, Sonic 1, 180 frames either
side of retro_reset(): before this change the core delivers 159876 audio
frames before the reset and 0 after it, and the picture stops changing
altogether - the frontend still gets its frames, but every one of them is
the same. After this change it delivers 159106 audio frames after the
reset and the picture goes on changing.

That follows from the same cause. The old path mixes only when
all_sources_ready() says every source has a buffer, and it reports
render_is_audio_sync() as true, so the emulator throttles itself to audio
being consumed. A reset leaves one source that never fills again, the AND
never holds, nothing is consumed, and the emulation stops waiting for an
audio callback that cannot come. Mixing what each source actually has, and
not pacing on audio at all, removes both halves of that.

libblastem.c

@@ -532,6 +532,28 @@ RETRO_API void retro_set_audio_sample_batch(retro_audio_sample_batch_t rasb)
 	retro_audio_sample_batch = rasb;
 }
 
+//mix_and_convert() mixes into a buffer sized from what render_audio_initialized()
+//was told, so a batch may be no larger than that.
+#define AUDIO_BATCH_FRAMES 512
+
+//Hands the frontend everything every source has ready. Asking for no more than
+//that is what keeps a source from being mixed past what it has written, which
+//would be heard as a gap.
+static void flush_audio(void)
+{
+	if (!retro_audio_sample_batch) {
+		return;
+	}
+	for (uint32_t remaining = audio_buffered(); remaining; )
+	{
+		uint32_t frames = remaining > AUDIO_BATCH_FRAMES ? AUDIO_BATCH_FRAMES : remaining;
+		int16_t buffer[AUDIO_BATCH_FRAMES * 2];
+		mix_and_convert((uint8_t *)buffer, frames * 2 * sizeof(int16_t), NULL);
+		retro_audio_sample_batch(buffer, frames);
+		remaining -= frames;
+	}
+}
+
 static retro_input_poll_t retro_input_poll;
 RETRO_API void retro_set_input_poll(retro_input_poll_t rip)
 {
@@ -558,7 +580,7 @@ const system_media *current_media(void)
 
 RETRO_API void retro_init(void)
 {
-	render_audio_initialized(RENDER_AUDIO_S16, 53693175 / (7 * 6 * 4), 2, 4, sizeof(int16_t));
+	render_audio_initialized(RENDER_AUDIO_S16, 53693175 / (7 * 6 * 4), 2, AUDIO_BATCH_FRAMES, sizeof(int16_t));
 }
 
 RETRO_API void retro_deinit(void)
@@ -624,6 +646,7 @@ static void apply_core_options(void)
 static vid_std video_standard;
 static uint32_t last_width, last_height;
 static uint8_t frame_presented;
+static void present_framebuffer(uint8_t which, int width);
 static uint32_t overscan_top, overscan_bot, overscan_left, overscan_right;
 static void override_overscan(const char *key, uint32_t *dst)
 {
@@ -681,7 +704,7 @@ static void update_av_info(void)
 	av_info.timing.fps = master_clock / (3420.0 * lines);
 	av_info.timing.sample_rate = master_clock / (7 * 6 * 24); //sample rate of YM2612
 	sample_rate = av_info.timing.sample_rate;
-	render_audio_initialized(RENDER_AUDIO_S16, sample_rate, 2, 4, sizeof(int16_t));
+	render_audio_initialized(RENDER_AUDIO_S16, sample_rate, 2, AUDIO_BATCH_FRAMES, sizeof(int16_t));
 	//force adjustment of resampling parameters since target sample rate may have changed slightly
 	current_system->set_speed_percent(current_system, 100);
 }
@@ -721,6 +744,8 @@ RETRO_API void retro_run(void)
 		//its settings menu. Everything else - the model, the region, the clock
 		//divider - is read while a machine is built and so waits for the next
 		//load.
+		//update_av_info() re-runs render_audio_initialized() with the batch size
+		//below, so the re-init this used to do by hand is already covered.
 		update_av_info();
 		if (current_system->config_updated) {
 			current_system->config_updated(current_system);
@@ -735,12 +760,14 @@ RETRO_API void retro_run(void)
 		current_system->start_context(current_system, NULL);
 		started = 1;
 	}
-	//The media player has no video, so it returns without presenting anything.
-	//Hand the frontend the (blank) framebuffer anyway so it still gets one frame
-	//per call and its pacing and audio sync have something to run against.
+	//A system that returned without presenting anything still owes the frontend
+	//a frame, so that it gets one per call and its pacing and audio sync have
+	//something to run against. The media player never has video at all, and a
+	//Sega CD does not reach the end of a frame on the call that starts it.
 	if (!frame_presented) {
-		render_framebuffer_updated(render_get_active_framebuffer(), LINEBUF_SIZE);
+		present_framebuffer(render_get_active_framebuffer(), LINEBUF_SIZE);
 	}
+	flush_audio();
 }
 
 /* Returns the amount of data the implementation requires to serialize
@@ -1106,7 +1133,11 @@ uint32_t *render_get_framebuffer(uint8_t which, int *pitch)
 	}
 }
 
-void render_framebuffer_updated(uint8_t which, int width)
+//Hands the frontend a frame. Separate from render_framebuffer_updated() because
+//the emulator asking to be let go is what ends a frame, and a frame this file
+//presents on its own behalf must not ask for that: the request would still be
+//standing on the next retro_run() and skip the emulation loop entirely.
+static void present_framebuffer(uint8_t which, int width)
 {
 	unsigned height = (video_standard == VID_NTSC ? 243 : 294) - (overscan_top + overscan_bot);
 	width -= (overscan_left + overscan_right);
@@ -1127,6 +1158,11 @@ void render_framebuffer_updated(uint8_t which, int width)
 	}
 	retro_video_refresh(fb + overscan_left + LINEBUF_SIZE * overscan_top, width, height, LINEBUF_SIZE * sizeof(uint32_t));
 	frame_presented = 1;
+}
+
+void render_framebuffer_updated(uint8_t which, int width)
+{
+	present_framebuffer(which, width);
 	system_request_exit(current_system, 0);
 }
 
@@ -1202,9 +1238,13 @@ void process_events()
 
 uint8_t render_is_audio_sync(void)
 {
-	//whether this is true depends on the libretro frontend implementation
-	//but the sync to audio path works better here
-	return 1;
+	//A frame ends when the VDP hands over a framebuffer, so audio is not what
+	//paces emulation here and each source can keep its own ring buffer. That is
+	//the only arrangement that survives sources running at rates of their own:
+	//the Sega CD's CD-DA and PCM chip fill at their own pace, and in the sync
+	//path a source that filled twice before the others filled once had a whole
+	//buffer dropped on the floor.
+	return 0;
 }
 
 uint8_t render_should_release_on_exit(void)
@@ -1235,13 +1275,17 @@ void render_unlock_audio()
 
 uint32_t render_min_buffered(void)
 {
-	//not actually used in the sync to audio path
-	return 4;
+	//Sizes each source's ring: render_audio_source() rounds min_buffered * 4 *
+	//channels up to a power of two. This leaves room for around a tenth of a
+	//second, so a source that runs ahead of the others has somewhere to put it.
+	return 2048;
 }
 
 uint32_t render_audio_syncs_per_sec(void)
 {
-	return 0;
+	//How often a source publishes what it has written. Often enough that a
+	//frame's worth arrives in pieces rather than all at once.
+	return 1000;
 }
 
 void render_audio_created(audio_source *src)
@@ -1250,17 +1294,8 @@ void render_audio_created(audio_source *src)
 
 void render_do_audio_ready(audio_source *src)
 {
-	int16_t *tmp = src->front;
-	src->front = src->back;
-	src->back = tmp;
-	src->front_populated = 1;
-	src->buffer_pos = 0;
-	if (all_sources_ready()) {
-		int16_t buffer[8];
-		int min_remaining_out;
-		mix_and_convert((uint8_t *)buffer, sizeof(buffer), &min_remaining_out);
-		retro_audio_sample_batch(buffer, sizeof(buffer)/(2*sizeof(*buffer)));
-	}
+	//Publish what has been written; the mixing happens once the frame is done
+	src->read_end = src->buffer_pos;
 }
 
 void render_source_paused(audio_source *src, uint8_t remaining_sources)

render_audio.c

@@ -166,6 +166,32 @@ int mix_and_convert(unsigned char *byte_stream, int len, int *min_remaining_out)
 	return min_buffered;
 }
 
+//How much audio there is to hand over, for a backend that pulls whatever is
+//there rather than waiting for a fixed buffer to fill. Only meaningful when
+//audio is not the sync source, since that is when read_end follows what has
+//been written.
+//
+//What every source that has anything to say can cover. Mixing past that would
+//take a source further than it has been written, which is heard as a gap, so
+//whatever is left stays queued for the next time.
+//
+//A source with nothing at all is a different matter: a CD-DA track that is not
+//playing and a PCM chip a game never touches write nothing, ever, and waiting
+//on them would mean waiting forever. Those mix as the silence they are.
+uint32_t audio_buffered(void)
+{
+	uint32_t least_buffered = 0xFFFFFFFFU;
+	for (uint8_t i = 0; i < num_audio_sources; i++)
+	{
+		audio_source *src = audio_sources[i];
+		uint32_t buffered = ((src->read_end - src->read_start) & src->mask) / src->num_channels;
+		if (buffered && buffered < least_buffered) {
+			least_buffered = buffered;
+		}
+	}
+	return least_buffered == 0xFFFFFFFFU ? 0 : least_buffered;
+}
+
 uint8_t all_sources_ready(void)
 {
 	uint8_t num_populated = 0;
@@ -353,6 +379,7 @@ static void interp_sample(audio_source *src, int16_t last, int16_t current)
 }
 
 static uint32_t sync_samples;
+
 void render_put_mono_sample(audio_source *src, int16_t value)
 {
 	value = lowpass_sample(src, src->last_left, value);

render_audio.h

@@ -42,6 +42,7 @@ void render_save_audio(char *path);
 //interface for render backends
 void render_audio_initialized(render_audio_format format, uint32_t rate, uint8_t channels, uint32_t buffer_size, int sample_size);
 int mix_and_convert(unsigned char *byte_stream, int len, int *min_remaining_out);
+uint32_t audio_buffered(void);
 uint8_t all_sources_ready(void);
 void render_audio_adjust_speed(float adjust_ratio);
 //to be implemented by render backend