CUzeBox - The new official Uzebox emulator

The Uzebox now have a fully functional emulator! Download and discuss it here.
CunningFellow
Posts: 1445
Joined: Mon Feb 11, 2013 8:08 am
Location: Brisbane, Australia

Re: CUzeBox - a new Uzebox emulator in progress

Post by CunningFellow »

I think maybe this is an undocumented feature of many SD card.

It is possible that until the are fully initalised they only have low power drivers built outputing the signal. This makes it marginal for the AVR to get a clear response.

In SD simple I changed it so there was a weak pull up on MISO until after init. Just about all my problems with bad init went away then.

D3thAdd3r, can you modify the ASM code in the mmc.s file to do this and recompile alter ego to see if that helps with your problem cards ?
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: CUzeBox - a new Uzebox emulator in progress

Post by D3thAdd3r »

For AE I will probably just port it to use SimpleSD instead so it can use the stock kernel; there turned out to be no use to modify the kernel in that one. I meant to use SimpleSD since so long ago anyway. There are a couple things I would like to fix besides, and then I'd have significantly more room for a level editor for AE2 and I can feel like the entire thing was down right.

For testing reading I would guess Tornado pushes things to the limits of SD and would probably be the best test. If it can do that it should do anything? For AE of course do not make any provisions for it to be compatible, I will fix it up, but on real hardware I think it works on all of something like 10+ cards tested between Alec and I. So there is some weight to that depending on which is more clean or important: idealism or realism.

But yeah, this has SD now and especially SD writing would convince me to finish TQ(just needs the level editor), and a speed increase I need for ESP8266. So CUzeBox has me quite excited about some things I was stalled on.

Are there plans to do a GUI?
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: CUzeBox - a new Uzebox emulator in progress

Post by Artcfox »

I just tested the Emscripten build on multiple (fast) computers, but the sound was lagging by about a second. I traced the problem to the buffer size. For Emscripten builds, it's better to half the audio buffer size to 1024, rather than double it to 4096, as compared to the native build which uses 2048.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: CUzeBox - a new Uzebox emulator in progress

Post by Artcfox »

Jubatian, I'm guessing you're not going to like this question, because it totally goes against all of the great decoupling you've done with this emulator… but can you think of a clean way to add the ability to optionally frame-lock the emulator to the vsync rate, rather than the doing the drift calculations it does now?

I ask because I'm using an older laptop that ran Uzem natively just fine (even with its window maximized) but when I run CUzeBox natively, at the size it pops up as, with the debug display turned off, it refuses to run at a constant speed (even though the console always says 100%). I noticed it when playing Bugz, because there are two sound effects that are played extremely frequently, and the pitch of each sound effect constantly changes from one instance to the next.

When I disable the debug display and hit F4 to let it run as fast as it can, it hovers around 117 fps, so I know the computer is more than capable. I'm thinking the only way to completely remove that drift and consistently hit 60fps would be to tie the execution of the program to a hardware based interrupt, and using vsync for that is the most straightforward way I can think of.

I can also hear the same drift in the (Uzem-keymapped) web version of Bugz on CUzeBox.
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: CUzeBox - a new Uzebox emulator in progress

Post by Jubatian »

Huh, so it caught on then!

SD card.

Actually Tempest was the first I got running. It respects timings for the initialization, so it could boot up, and it neither used the App. Command way to do the init which was initially buggy. So with a few hacks and tweaks seeing what it does, I could run it while everything else choked.

The timings: In AE notably it is recommended that until you get to data transfer mode, you keep it between 100KHz and 400KHz. My SD emulation attempts to enforce that normally (refusing to init otherwise), and AE completely ignores this, trying to do all the init at full speed (and not even attempting retries). Another thing recommended was a few milliseconds before the init burst after pulling CS high, explained as to wait for voltage stabilization, which my emulator tried to enforce, and AE ignored.

Syncing.

The problem is especially that the emulator is synced to the video Vsync, that is, it is frame-locked to the video so long that it runs about 1,5% within 60Hz by the system clock.

There are three asynchronous timers here to cope with. The system timer, the video VSync rate, and the audio. These are truly asynchronous, that is, it is not possible to know how a nominal millisecond on either would relate to another.

The emulator so primarily runs locked to the video VSync, so long that by the system timer it sees that it runs within 1,5% tolerance to 60Hz. Otherwise it does frame rate management in relation to the system timer. For these to work correctly, the overall jitter has to be below 100ms, that is, there shouldn't be anything in the background which bombs your CPU with bursts of load (on Emscripten, the JavaScript garbage collector occasionally does this, upsetting the frame rate manager).

The audio is produced at 48KHz, that is, there is no attempt to set Uzebox's nominal audio rate since due to the above it may be running anywhere between 58Hz to 62Hz, however the system stabilizes in the longer run (primarily depending on whatever the actual refresh rate of your display is if it is within tolerance). The required actual output rate is determined by a very soft PD controller, which if the frame rate manager can actually work, does its job fine. If it wasn't done this way, the audio would experience annoying skips either repeatedly depleting the buffer (the set audio frequency being too high to the video) or overrunning it (the set audio frequency is too low).

I increased the audio buffer size for Emscripten because as I experienced due to the JavaScript garbage collector's bursts and other various bursts of load there were very frequent buffer underruns. The PD controller within actually tries to detect whether such a load took place to ignore the effect on the audio, more or less this works. Since the output runs at 48KHz, that 4096 samples buffer size means less than 100ms lag, and I didn't experienced it to be a lot more. I think there is something else there beyond the control of the emulator itself which causes a second of lag (all the buffers within the emulator's audio chain for Emscripten wouldn't make up more than 200ms, so it is impossible to lag more by the emulator only).

Possibly if I loosen the main frame rate manager somewhat, it will work better overall, but it is a complex subject. The main problem is that Uzebox works on a single time base while the results have to be distributed among devices running on timers asynchronous to each other, which are impossible to actually measure (that's why the PD controller is necessary for the audio).

The Uzem emulator actually runs either video or audio synced. If the video frame rate is high so more samples are produced than what the audio can swallow, the delay loop in the audio output will kill off frames (the video will skip). If the video frame rate is low, so less samples are produced than required by the audio, it (the audio) would occasionally skip. So it is quite far from ideal, it just works usually. But it is also the reason why the emulator chokes on anything producing improper video signal, and it totally doesn't fit the main loop principle required by Emscripten for optimal operation.

The controls:

Yes, I forgot about the shifts, guessing they weren't that commonly used when I quickly set that part up. I also remapped the buttons for the following reasons:
  • When I was doing my own UCC entry, initially I set the button mapping up for Uzem as I mostly tested it there. Then I fired up the real thing, grabbed the SNES controller... And found the assingments I made were totally impractical. So I mapped the buttons according to the diamond on the controller itself (Q => SNES Y; W => SNES X; A => SNES B; S => SNES A).
  • I shifted them up with a row compared to Uzem since on the bottom row there is the 'Z' key. On many keyboard this is a 'Y' (QWERTZ, like mine), so the actual button to use will end up in a very awkward location. SDL2 has a solution to this, but even that doesn't work in Emscripten, and to be cross-compatible with SDL1, I couldn't use it anyway.
EDIT:

I did some tweaks on the frame rate management. It is ridiculously difficult to make good decisions there. A big problem with video syncing is that you can't add a counter on vsync, so you don't know where you miss one, which due to the vsync locking, results in a permanent 16.67ms lag at that point which the audio has to correct. Such misses can happen for whatever reason, just a little load on the CPU, you don't get scheduled for like 10ms and it is done. Maybe I will try some tricks with the system timer to detect it, so to compensate with a dropped frame (to bypass the sync lock on it), but, just to note the pitfalls: natively the point where you can precisely measure time is right after the SDL_RenderPresent call, however in Emscripten it is at the entry of the main loop.

EDIT2:

Gui... Yes, I have plans for it, a minimalistic one, it is not very essential expect for showing the controls and various debug info useful for developing. The farther plans is that once the SPM instruction is properly implemented, I will just add the bootloader, so there is no need for even a game selector, just have your set of games in the emulator's view, and it is ready to go, behaving just like the real thing (of course the current capability of running a selected game supplied on command line will remain there, it is useful on Windows as well: you can drag and drop a game even on Uzem to run it).
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: CUzeBox - a new Uzebox emulator in progress

Post by Jubatian »

Lee: Are you sure the highscore feature of Toorum's Quest works deterministically?

I got it trying to write, and by it I verified that CMD24 is implemented properly. But it is not consistent, sometimes even though I have a high score and the game requests my name, it fails to even issue a CMD24 (I am logging the entire SPI traffic, so I can't miss it). Now I added the write support to the Virtual FAT, so it should actually write things once it happened, but right now I did the fifth run through that dang game without successfully triggering a high score save attempt.

I committed it with write support enabled, so once somebody manages to issue a CMD24 or CMD25, something should happen.

The implementation is so that it is "sandboxed" within the game's directory (no subdirectories are supported). You can write to existing files, expand them, or you may even create new files, including fragmented stuff, the virtual FAT properly resolves what belongs to what. Shrinking or deleting files from the host is not supported (but you may wipe out their contents). Expanding and creating only works if you build up the FAT first, and write the contents only then (as the module discards any writes which doesn't belong either to the system area of the virtual FAT or a file which it can trace by the FAT).
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: CUzeBox - a new Uzebox emulator in progress

Post by Jubatian »

It now has SPI RAM emulation. It runs this demo seemingly correctly.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: CUzeBox - a new Uzebox emulator in progress

Post by D3thAdd3r »

Damn, probably a bug, but I don't know what causes it. SD Writing was the last thing I was doing on it at the time, but don't recall any known issues I had. The lowest level code for writing SD is simple at least, and I see nothing wrong here:

Code: Select all

void WriteSD(uint32_t offset, uint16_t len, unsigned char *data){

	//this write cannot overlap sectors or data will be corrupted. Sprites must be turned off before calling so that kernel does not corrupt ram_tiles[]
	FRESULT res;
	WORD	br;
	uint32_t first_sector_byte;
	uint32_t inset;

	first_sector_byte = (offset/512);
	first_sector_byte *= 512;//point to first byte of sector this lays in
	inset	= offset-first_sector_byte;//number of bytes in the sector before our data starts

	//this is needed due to the way PFF handles writes(writes 0 before and after data in the sector if data isn't specified, corrupting existing)
	res =	pf_lseek((uint32_t)(first_sector_byte));//round down to first byte of sector
	res |=	pf_read((BYTE *)ram_tiles,512UL,&br);//copy the whole sector
	if(res)
		SD_Crash(7,5);

	for(uint32_t i=inset;i<inset+len;i++) //insert our data
		ram_tiles[i] = data[i-inset];

	res = pf_lseek((uint32_t)(first_sector_byte));
	res |= pf_write((BYTE *)ram_tiles,512UL,&br);//write the entire sector out, including what we added
	res |= pf_write(0,0,0);//finalize write
	if(res)
		SD_Crash(8,4);
}
At the PFF level, it is not reporting any errors from the functions since those are trapped by the SD_Crash() full screen message, which I assume you have not seen. So it is probably something higher level than that which isn't calling it for some reason.

On the high score screen does it show "ALEC,CUNNING,FLECKO,JUBATIAN,..." with 2800 points, 2700, 2600, etc? To access it quicker, just start a game then quit. I am just wondering if it is getting a corrupted read, and using the defaults. Would something that writes different patterns immediately via different button pushes help? Eliminates the possibly unknown bugs in the game, which I will fully admit was thrown together in the quickest dirtiest most untested manner.

Edit - BTW Sandor you are kicking ass to have that SPI ram demo running already. I will check that out later on. AFAIK that is state of the art for SPI ram things so far, very very nice work.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: CUzeBox - a new Uzebox emulator in progress

Post by D3thAdd3r »

Jubatian wrote: In AE notably it is recommended that until you get to data transfer mode, you keep it between 100KHz and 400KHz. My SD emulation attempts to enforce that normally (refusing to init otherwise), and AE completely ignores this, trying to do all the init at full speed (and not even attempting retries). Another thing recommended was a few milliseconds before the init burst after pulling CS high, explained as to wait for voltage stabilization, which my emulator tried to enforce, and AE ignored.

Code: Select all

//..
	pf_mount(&fs);
	for(uint8_t i=0;i<MAX_SD_ATTEMPTS;i++){
		res =	pf_open("aedat.bin");
		WaitVsync(2);
		if(!res)
			break;
	}
	if(res)
		SDCrash(0);

	res =	pf_lseek(0);//the first byte of the header has the number of episodes
	res |=	pf_read((BYTE *)ram_tiles,1,&br);
	num_episodes_available = ram_tiles[0];

	res =	pf_lseek(16);//go to first byte of episode 0 name, read the name of all episodes
	res |=	pf_read((BYTE *)ram_tiles,32*MAX_EPISODES,&br);

	if(res)
		SDCrash(1);
//..
I am trying to understand what could be wrong with the timing, since I only use PFF functions directly and nothing lower level. Unless I misunderstand, all games using PFF following standard conventions should then be affected? This is more or less the same as the example code given for PFF here If we have to do anything with pin states, it seems like that should happen inside PFF. Else there is little reason to use it if you have to mettle with the lower level stuff anyways.
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: CUzeBox - a new Uzebox emulator in progress

Post by Jubatian »

The SPI RAM is actually a quite simple peripheral :) It didn't take any more than about an hour to figure it out and implement (of course given the emulator's modular architecture so it was possible to add it without further considerations).

The Petit Fat FS module itself I think has nothing to do about the problem you experience. It is lower level than that: the SD access. You would have to configure the SPI peripheral to run between 100 and 400 KHz while you init the card, and you could only go to full speed when you successfully left idle state. (At least a quick glance at PFF tells that it only implements the FAT without the low level disk I/O. Downloading it I see just "Put your code here" markers in its disk I/O layer, so I guess the problem lies in whatever ended up there).

Yes, I see Toorum's Quest's high score table, I also checked the file itself: It wasn't touched since I unpacked the game a long while ago (I just dropped the UZE you provided in the game's directory replacing the original. Their sizes were the same, possibly they were even the same). I don't get any error while running it, it visually behaves exactly the same manner when it triggers a CMD24 like when it doesn't. It isn't even really reasonable to test this on the real thing either because of this, it is cumbersome to get feedback.

Just to note: later I will add some visual debug aids for the SD access and the SPI RAM as well. Just essentials, but things which should be quite useful (like the SD card's status, read & write activities, so you see where your code could get with the initialization or an access).
Post Reply