Uzebox Bootloader V5

Topics related to the API, programming discussions & questions, coding tips, bugs, etc. should go here.
Post Reply
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: New bootloader

Post by Jubatian »

uze6666 wrote: Wed Nov 01, 2017 3:45 amI don't want to tone down the overall excellent work, but I'm not fond about the part about allowing a game to flash sectors (if I've read correctly).This makes me quite nervous that a bug in a game (or even on purpose) could brick a chip in no time. As a kit seller, what will I do if peoples ran into such a problematic game? Put a disclaimer to run games at your own perils? Take the hit and send them new chips? It's a risk I was never willing to take. If this is allowed, sadly, I won't be able to use that new bootloader on my kits (and include any games that requires it obviously). My two cents would be to drop this feature for the greater good and avoid "market fragmentation" (because now you will have games that wont run unless you have bootloader X).
True, I also had this concern when developing it and deciding on what to expose to the user. I am open for some discussion on it, and maybe I will drop it. Personally I don't really like the idea and wouldn't use it myself.

My end decision was the following API which is currently tested and works:

The game can request the bootloader to program a new game image. This can be a .uze file as-is, or the complete image of a .uze file within another file. So if the game calls this, the game ends, actually doing a "soft-reboot" into the bootloader, which programs a complete new game. So this isn't as versatile as a page program interface, thus because of its limitations (you can't normally take state across unless you save it somewhere and restore), it needs more consideration for using right. It also makes it a bit more difficult for a single game to goof up and perpetually ask programming stuff (if it keeps asking programming itself, the bootloader will happily do nothing and restart the game, otherwise what it programs is a different game, so it is not very trivial to set up a "ROM killer chain" in this manner).
uze6666 wrote: Wed Nov 01, 2017 3:45 amFor the SD API part, the fallback code is essential. To keep all games backward compatible with all existing kits that runs the 'old' bootloader. Btw, how is it going to work with the emulators? How do they know what bootloader to use? Yiou would not have choice? Could it be a better idea to link the dependency bootloader at the end of game so it's self contained and could "just run" in the emulators?
The API library example I posted above has an appropriate compatibility layer, so the compiled game using it works fine with all the emulators and existing bootloaders (or no bootloader). If it can't detect the new bootloader (it has a signature and version info embedded in its binary), it uses the same MMC code which I found in the original bootloader to interface the card, using a simple no fragmentation FAT-16 layer on top of it (this is smaller than a stand-alone SDHC + FAT32 + fragmentation capable library). The only plus it has is CRC checking if it can turn it on (but it can accept it not being available to keep working on Uzem). So games using this SD library would work fine on all existing Uzeboxes and emulators.

Sadly I can't do anything with old games coded to use SDSC + FAT16, of course they won't work if you have the new bootloader and put them on an SDHC card or one with FAT32. Of course you are free to use a FAT16 formatted SDSC card with the new bootloader.

Thanks for feedback on this by the way! :)
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: New bootloader

Post by Jubatian »

uze6666 wrote: Wed Nov 01, 2017 4:28 am
Yeah, that's true. But IMHO having SDHC and fragmentation support is important.
Agreed. But think that all the regular SD games out there won't work anymore when an SDHC card is plugged in (block addressing is different if I recall). Someone will likely have to recompile games to support both SD and SDHC and hoping the code fits. Or have to maintain two sets of roms (not a nice option IHMO). No show stopper really, just things to consider.
Yes, old games on a new card will fail. Not only the block addressing is different, they requre a different init sequence too (although some games use such an SD library which is almost capable to init an SDHC card). However if you make a game SDHC aware, then it would work with both old and new cards as with SDSC the recommended way to init was already the method SDHC enforces. Having an SDHC init only makes things incompatible with very old MMC cards probably, but possibly there isn't any of them out in the wild nowadays.

The API in this bootloader aims to solve this by providing routines which can perform a proper SD card init and type detection, then if you wanted, you could take over and use SD commands to interface the card directly (such as for games like Tempest). Such a game would work with the old bootloader or no bootloader as the interface library carries compatibility methods (the old bootloader's way of SD init and usage), however if they are used on an Uzebox with the new bootloader, they would use SDHC + FAT32 all right (since the bootloader's code does that part for them), and also fragmentation if you stick to using the filesystem methods provided by the library.

So should I remove the .uze image loading capability? My personal opinion is that if you want to make a big game, use the SPI RAM: It is possibly even easier to solder one onto an existing Uzebox than trying to find an SD card which is fast enough for Tempest or Chickens in Choppers (I don't have any myself), and with new video mode designs, you can have lots of content served from the SPI RAM, lessening the want for reprogramming the ATMega.

Why I would also side with removing this is that then I would have some extra room for making the SD/MMC card init more robust (more likely to work with a wider variety of cards).
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: New bootloader

Post by Artcfox »

My vote would be to keep it, because it opens up more possibilities. I see it as being no more destructive than manually loading a part 2 .uze file, and it wouldn't add to the limit of .uze files able to be stored on the card, and might save reflashes for curious people trying to reflash a visible part 2 .uze file.
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: New bootloader

Post by Jubatian »

By the way by the card init I am talking about normal MMC cards which might be obsolete, but still there could be plenty of them such as this search reveals it on Ebay. They should work with the original bootloader as it does an MMC init, but not with this new loader, possibly neither with some of the existing games (I think Alter Ego is also doing an SD specific init). They might be more common than SDSC cards (and are easier to identify). So I would like to cram in MMC compatibility which needs room, which currently I don't have.

EDIT: I could actually shoehorn in the MMC init, now having just 4 bytes free. I would still like to add a block size selection (CMD16) which might make a few additional cards working which don't have 512 byte sectors by default. MMC would be supported for up to 2Gb (those do byte addressing, and it would need additional code to query the size of an MMC), however larger cards of this type seem to be extremely rare. Does anyone have an MMC by the way to test? (I only have SD cards)
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: New bootloader

Post by D3thAdd3r »

I have no MMC to test with unfortunately. It seems like if the new bootloader supports the cards that get included with the kits, that is the big compatibility point which is already reached. I am curious who is actually using MMC to be honest, and if no one, then maybe it is ok to drop? At least I wouldn't argue against that, especially since cramming that in the remaining bytes sounds pretty laborious.

Random idea on code space, is it possible to create a simple virtual machine that can utilize some basic SPI blocks in native code to save space? I don't know how much free ram you have there, just a weird idea.
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: New bootloader

Post by Jubatian »

D3thAdd3r wrote: Sat Nov 04, 2017 9:42 pm I have no MMC to test with unfortunately. It seems like if the new bootloader supports the cards that get included with the kits, that is the big compatibility point which is already reached. I am curious who is actually using MMC to be honest, and if no one, then maybe it is ok to drop? At least I wouldn't argue against that, especially since cramming that in the remaining bytes sounds pretty laborious.

Random idea on code space, is it possible to create a simple virtual machine that can utilize some basic SPI blocks in native code to save space? I don't know how much free ram you have there, just a weird idea.
I already have MMC in (4 bytes free), but I don't know how well it would work in that environment (as a fall-back from an SD init), I would like to have someone who has an MMC to test with. I would like to add a block size set command, which may be required for some cards to work (that is, they would work if that command was used to set them to use 512 bytes / sector).

I don't think it is possible to crunch that code further. No VM, but just about everything is in subroutines which can reasonably be, I have plenty of stack. And also I use "rcall" (1 word) everywhere instead of "call" (2 words) as the 4K bootloader area is fully covered by the relative instruction. The original bootloader already pushed the limits, I went a lot beyond that with the code capable to deal with SDHC, FAT32 and fragmentation.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: New bootloader

Post by Artcfox »

As far as I know cards ignore the block size set command, and if you use it you have to verify if it actually changed anything. I don't know of any MMC cards, never had one even back in the day. The Uzebox came out in 2008 right? I think it is safe to assume people aren't using MMC.
User avatar
uze6666
Site Admin
Posts: 4801
Joined: Tue Aug 12, 2008 9:13 pm
Location: Montreal, Canada
Contact:

Re: New bootloader

Post by uze6666 »

Totally with Artcfox on this. All Uzeboxes kits I know of: mines, Harty's, Adafruit's, Sparkfun's, ITead's, and Uzebox Jamma's... all used regular SD. So don't bother about MMC please. Drop all support for it if it saves code space or anything, it's totally irrelevant nowadays!
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: New bootloader

Post by Jubatian »

uze6666 wrote: Fri Nov 17, 2017 8:35 am Totally with Artcfox on this. All Uzeboxes kits I know of: mines, Harty's, Adafruit's, Sparkfun's, ITead's, and Uzebox Jamma's... all used regular SD. So don't bother about MMC please. Drop all support for it if it saves code space or anything, it's totally irrelevant nowadays!
Huh, then how I should best use the remaining few bytes?
  • I still need some decision on whether the Game Load API should be in. It is 26-30 bytes, enabling user programs to request loading a new .uze (by file or image within another file).
  • I have 18 bytes free in V.5.0.07. In that I could fit a Block Size Set command (might help a few cards which start with non-512 byte block size but support it) or as I could do it experimentally, probable MMC compatibility.
  • Fitting in SD write is not likely possible (however a library offering compatibility for the no bootloader case wouldn't benefit much anyway from SD write within the bootloader).
  • Anything else amiss?
If V.5.0.07 is OK, I would add it to the Uzebox master repo (as a new bootloader, not replacing the old one), then I would also merge in the changed port inits. If you were testing the new bootloader, that will also work just fine as they are the same. Then I could go on for the next step: finalizing and documenting the API library (which as I mentioned is compatible with the old/no-bootloader & emulator case using a small MMC + FAT16 fallback solution), which would be especially useful for SPI RAM games.
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: New bootloader

Post by nicksen782 »

My two cents. Perhaps having a game request a reflash could be an issue but what about rebooting directly into the bootloader (instead of resetting the game?)

Actually, can that be done with the existing bootloader?
Post Reply