Real-time syncing the emu - why by video?

The Uzebox now have a fully functional emulator! Download and discuss it here.
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Real-time syncing the emu - why by video?

Post by Jubatian »

Poking around in the SDL2 porting topic and in the code I have the impression that uzem synchronizes itself to real time using the video output, to produce exactly 60 FPS.

I have the feeling that maybe this is not the right approach, and also is the main reason for choppy sound which I see many people experiencing here in various topics.

I would rather attempt to sync to sound if it was up to me (and I actually did in an own emulator project, successfully). Why so?

The sound and the video output is tightly coupled in Uzebox, running from the same clock source (the AVR CPU generating both). On a desktop PC however the video output, the CPU and the sound system all run on separate clocks. So if you sync to one, you are out of sync for the other, you can only come near it.

As far as I know and understand so far, syncing to video on a PC is some kind of arcane magic, you are lucky if you can even get anything, and even that "anything" is usually just that some subsystem will flip the pages for a double-buffered render some time in the future so the thing doesn't tear. The PC's video output might neither run at 60Hz at all. SDL's framerate manager as far as I see now doesn't even do anything about video output, just delays, its time source being some internal clock which is neither derived from the physical audio output, neither the video. I think this is simply the worst choice here.

However it is actually quite simple to sync to the audio. If you set up the audio output to some fixed rate (15734Hz in case of Uzebox), and have callbacks to feed its buffer, the firing rate of those callbacks is exactly what you need (it is a bit more complicated as I figured out with SDL1, since apparently the callback fire rate may vary, requesting different amounts of samples, but you get it: the number of samples you need to fill in tells you how much you have to go forth in emulating Uzebox to produce those). Doing the real time sync in this manner would essentially sync the emulator to the audio hardware's internal clock, and you get absolutely no skipping / distortion (unless the buffer is too small compared to how often Uzebox gets chance from the OS to proceed, or of course if emulation drains all ounces of processing power you have).

For the video I simply wouldn't care apart from eliminating tearing if possible. Uzebox is tightly coupled on audio / video term, it is an impossible problem to solve, audio distortion is worse than, say, an occasional repeated or skipped video frame.

What do you think?

(Maybe I would try an attempt with this, would take a while since the code looks quite convoluted there, but since there is an SDL2 porting going on, trying this on a separate thread would result in two clashing versions of the emu)
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Real-time syncing the emu - why by video?

Post by Artcfox »

Sorry, it actually does sync to the audio, it's just that I had been trying a few different things, and the flag SDL_RENDERER_PRESENTVSYNC slipped into the pull request that I made to Uze. I'll fix that right now, and submit another pull request.

And the SDL2 port is not underway, it's been done.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Real-time syncing the emu - why by video?

Post by Artcfox »

Okay, I just submitted the pull request. Nice catch!
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Real-time syncing the emu - why by video?

Post by Jubatian »

Huh, I might have mixed branches then. In the 1.40 branch there is clearly no frameratelimiter mumbo-jumbo going on in the code, and I wrote this mainly based and researching after those. I even see the code in there which attempts to sync to the audio...

Something went south in that thing, though, at least it seems so since I get quite weird results (22MHz, then occasionally if the constellations align the right way, 28MHz for a couple of seconds).
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Real-time syncing the emu - why by video?

Post by Artcfox »

Code: Select all

			while (audioRing.isFull())SDL_Delay(1);
			SDL_LockAudio();
			audioRing.push(value);
			SDL_UnlockAudio();
That's what limits the CPU. Try running it with the -n command line flag to disable audio (after you remove the SDL_RENDERER_PRESENTVSYNC flag) and it will run as fast as your computer will allow it to.
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Real-time syncing the emu - why by video?

Post by Jubatian »

Currently with SDL_RENDERER_PRESENTVSYNC removed, "while (audioRing.isFull())SDL_Delay(1);" commented out I can get it running occasionally at 36MHz when started for about 15 or so seconds or even a minute, then it falls back to 24MHz. So apparently my machine has the power to run this thing, video card sits at a healthy 45 Celsius all the while, so that neither can be the reason for this weirdness.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Real-time syncing the emu - why by video?

Post by Artcfox »

Jubatian wrote:Currently with SDL_RENDERER_PRESENTVSYNC removed, "while (audioRing.isFull())SDL_Delay(1);" commented out I can get it running occasionally at 36MHz when started for about 15 or so seconds or even a minute, then it falls back to 24MHz. So apparently my machine has the power to run this thing, video card sits at a healthy 45 Celsius all the while, so that neither can be the reason for this weirdness.
That's weird. I don't know what could be causing that. There is an Uzem140_SDL1 tag that you can checkout, which will give you the uzem140 branch as it existed before the SDL2 port. If you still can't get 100% emulation speed with that, then it would rule out SDL2 as the culprit.

BTW: Using your uzem140-hacks branch (on my laptop, nevertheless!) I was able to run the emulator at 100% emulation speed with the -r flag set, which records the a/v output of the emulator into a .mp4 file (scaling, upsampling it to 720p, and H.264 encoding it, and performing a sample rate conversion and mp3 encoding of the audio). Very impressive work you've done with the optimizations! :o
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Real-time syncing the emu - why by video?

Post by Jubatian »

Observing the surrounding code I see problems, though.

I wouldn't do the syncing in a write_io() call, that way the emulator is tied to the kernel's implementation. The audio should rather be output at given cycle intervals, so the wait should be rather tied to elapsed cycles: every 1820 cycles it should do this wait. This way even if the kernel changes or is goofed up the emulator keeps going. Or even if someone decides to process audio twice, outputting a second time mid-line to get ~31KHz. You could even ask a more sane (such as 48KHz) audio sampling rate from SDL then, and tune the elapsed cycles accordingly (597 cycles then).

(This was particularly annoying when goofing up the CPU so the entire emulator got unresponsive simply because it never arrived to try to output a sample, or you could also get a "dead" emulator simply by bad code, and it is not very convenient to kill that)

EDIT: Tomorrow I think I will try, it is very late here already :p
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Real-time syncing the emu - why by video?

Post by Jubatian »

Could someone help me a bit with that branch?

When I try to compile, I get messages like "'HANDLE' does not name a type" or "'LPBYTE' does not name a type", and even grepping the whole project does not reveal where they could have been defined. By any chance, wasn't that version Windows-only? ("LPBYTE" reeks suspiciously of mangled Hungarian notation which Microsoft so loved to use)
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Real-time syncing the emu - why by video?

Post by Artcfox »

Jubatian wrote:Could someone help me a bit with that branch?

When I try to compile, I get messages like "'HANDLE' does not name a type" or "'LPBYTE' does not name a type", and even grepping the whole project does not reveal where they could have been defined. By any chance, wasn't that version Windows-only? ("LPBYTE" reeks suspiciously of mangled Hungarian notation which Microsoft so loved to use)
Look at the commit where I ported it to SDL2, because it has that fix, along with all the other linux compile bugs fixed. (I described all the changes in the commit message.)
Post Reply