The UZE file format

Topics on software tools like TileStudio, comments on documentation and tutorials (or the lack of) should go here.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: The UZE file format

Post by Artcfox »

D3thAdd3r wrote: Sun Apr 15, 2018 9:05 pm One thing I didn't see discussed, how do I build an UZE with embedded extra data right now?
I would think a simple:

Code: Select all

cat datafile.bin >> gamefile.uze
would suffice, though you may want to pad the beginning of your datafile.bin so any real data you want to read starts on a sector boundary.
User avatar
D3thAdd3r
Posts: 3175
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: The UZE file format

Post by D3thAdd3r »

That would do it, would this work in MinGW?
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: The UZE file format

Post by Artcfox »

D3thAdd3r wrote: Sun Apr 15, 2018 10:55 pm That would do it, would this work in MinGW?
It should.
User avatar
Jubatian
Posts: 1560
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: The UZE file format

Post by Jubatian »

D3thAdd3r wrote: Sun Apr 15, 2018 9:05 pmOne thing I didn't see discussed, how do I build an UZE with embedded extra data right now?
Artcfox wrote: Sun Apr 15, 2018 9:55 pmI would think a simple:

Code: Select all

cat datafile.bin >> gamefile.uze
would suffice, though you may want to pad the beginning of your datafile.bin so any real data you want to read starts on a sector boundary.
This is an okay answer for "right now", but first if we settled on standardization, the header fields governing it (where the data is to be located) should be agreed upon. Later possibly the Packrom tool could be extended to support adding data files into the .UZE along with generating the appropriate header.

So here comes an initial draft for the header:
  • 0x193 - 0x1E2: Reserved (for further Game Description to allow making it up to 144 bytes, as I think 64 bytes isn't quite enough for this purpose if you really wanted to use this field).
  • 0x1E3: HW feature requirement flags. Bit 0: 128Kb SPI RAM required if set. (Bit 1: SNES mouse required, maybe, depreciating the SNES mouse field which one's usages are unknown to me, Whack a mole in particular doesn't have it set).
  • 0x1E4 - 0x1E6: Save state area 512b sector start offset within UZE file, Little Endian.
  • 0x1E7 - 0x1E8: Save state area length in 512b sector units, Little Endian.
  • 0x1E9: Save state area definition complemented checksum (calculated over 0x1E4 - 0x1E8). This can be used by libraries and tools to verify whether the save state area definition is okay, to make writing it more robust.
  • 0x1EA - 0x1EC: Extra data area 512b sector start offset within UZE file, Little Endian.
  • 0x1ED - 0x1EF: Extra data area length in 512b sector units, Little Endian.
The offsets take 3 bytes as the 4th byte would be always zero anyway as the FAT filesystem can not store files larger than 2Gb. The save state area definition ensures that it can be reliably checked whether the UZE contains saves, and where these are. So you can create a generic wipe for them, too (zeroing this region).

Further stuff may be added to the remaining 16 bytes later. I don't think games would demand anything beyond these, all what may come are information fields if there was any software which could use them (we already have more info fields than our tools' capabilities, the 16x16 icon and the game description I think was never used).
User avatar
D3thAdd3r
Posts: 3175
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: The UZE file format

Post by D3thAdd3r »

Sounds good to me.
User avatar
Jubatian
Posts: 1560
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: The UZE file format

Post by Jubatian »

Artcfox wrote: Sun Apr 15, 2018 7:43 pm(...) and then they might inadvertently include ROMs that don't start at the beginning of their games, because they tested them out first using an emulator.
An idea how you could protect your game against this problem regardless of how you store saves on the SD card:

You can still use an EEPROM block for your game. In this EEPROM block, you could store whether the game was ever started on that particular Uzebox, and if you detect that it wasn't, while the game has some save state, you could offer a clean start (but it is important to make this an offer as the player could actually want to continue a game on a different Uzebox or emulator).
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: The UZE file format

Post by Artcfox »

Jubatian wrote: Tue Apr 17, 2018 12:49 pm
Artcfox wrote: Sun Apr 15, 2018 7:43 pm(...) and then they might inadvertently include ROMs that don't start at the beginning of their games, because they tested them out first using an emulator.
An idea how you could protect your game against this problem regardless of how you store saves on the SD card:

You can still use an EEPROM block for your game. In this EEPROM block, you could store whether the game was ever started on that particular Uzebox, and if you detect that it wasn't, while the game has some save state, you could offer a clean start (but it is important to make this an offer as the player could actually want to continue a game on a different Uzebox or emulator).
Interesting idea. Most games that save state offer a choice of New Game or Load Game already right? I wonder if that would be enough to cover the special case you just mentioned without requiring extra checks. A new user starting a game would probably pick New Game, because they wouldn't think a saved game would be bundled with it. I guess only if a game didn't offer this choice would it be a problem, and then there would be no in-game way to start a new game.

I'm wondering if any games would modify the code and/or tilesets/sprites of their own game file. For instance, if you had a set of characters all the same size with complex animations, but you only ever see one character at a time, I can see rewriting the spriteset from data on the SD card to the spot where all of the sprites are stored in your .uze file and then re-loading the game to play as that character. Then you'd get all the optimizations from having a large static spriteset, but you'd be able to swap from essentially a limitless set of sprites. You could even have a character editor inside a game that allows a player to completely design their own character. And for a multi-player game using Uzenet, a game could exchange the custom spritesets with the other player, so the other person would be able to see your custom avatar in their game as well as you seeing theirs in your game.
User avatar
Jubatian
Posts: 1560
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: The UZE file format

Post by Jubatian »

Artcfox wrote: Tue Apr 17, 2018 1:24 pmMost games that save state offer a choice of New Game or Load Game already right?
For one offering such, of course this isn't a great concern at all. Probably rather games like Alter Ego, which unlock levels as the player makes progress: there no such menu is offered, you get the selection of available levels based on your advancement.
Artcfox wrote: Tue Apr 17, 2018 1:24 pmI'm wondering if any games would modify the code and/or tilesets/sprites of their own game file.
Normally no. The point of the save area is that a library can be made for it, which makes write access a lot safer: the library ensures that only the designated area can be written in a UZE file. Of course even now nothing stops you from messing around with UZE files from within something running on the Uzebox (for the kicks, somebody could program a virus :) ).
User avatar
L4rry
Posts: 242
Joined: Sun Dec 28, 2014 7:19 am
Location: Cape Town, South Africa

Re: The UZE file format

Post by L4rry »

OK, I'm going to put this out there. A mutable binary is a dumb idea. Especially for an open platform. There is no good reason to combine your game code and assets with state.

All you will achieve is a fractured an confusing echo system whenever you share your game.

A lot of good ideas here. Mutable binaries is not one of them.

EDIT: Thanks for documenting the UZE file format :)
User avatar
D3thAdd3r
Posts: 3175
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: The UZE file format

Post by D3thAdd3r »

Alter Ego does offer the option to continue or start new(perhaps it gives a warning if selecting new, don't remember). If selecting new, it deletes the previously stored progress, so if you don't beat a level after that[EDIT]AE has a dedicated clear[/EDIT], you are in a totally stock state. I suppose that is a very simple and standard way to handle it.
L4rry wrote: Wed Apr 18, 2018 1:41 am A lot of good ideas here. Mutable binaries is not one of them.
On this, and also CunningFellow's wariness for rogue SD writing, I understand the philosophy here and agree with the intent. I would call some statements "very conservative" though, where in my opinion "moderate" is the right path. Seeing as no outright "liberal use of writing" view seems to have been expressed, I am willing to be the bad guy and play the Devil's Advocate here(with an opinion a bit more pointed than the way I actually feel...for science!) :lol: My example would be a successful use of SD writing. I built a new more difficult episode for Toorum's Quest using the editor, and saw no problems with my SD at any point. That editor feature is probably more impressive than the rest of the game in reality, and I tested it more so than other parts, because I was writing which is dangerous. To be fair, I should add "WARNING THIS PROGRAM WRITES TO THE SD CARD" whenever the user selects the editor. Bigger more creative things can be had in that direction too; sounds interesting!

Now for espousing the Devil's thoughts on flash writing, we are all taking a risk running any rom off the forums since any of them could be a big virus that fries the flash in the '644. If someone new on the forums posted an RPG, I am going to fire it up without a care in the world whether he talked about testing or not(all members active here I am sure would test to the best of their abilities). If I imagine there is a 1% risk of a bug/malice causing hardware failure, and the cost of replacement is $6.00 for a new '644...sorry, hundreds of hours spent on Uzebox over the years, I am not going to limit my fun for $6.00*1% = $0.06 risk. For SD, there is no personal or irreplaceable data there, and this is not life critical nor hard to avoid the pitfalls. Anyway I would write in game if there is good benefit to it(not just for the sake of it, and as minimal as possible), thoroughly test all such things, and employ warnings that the program does such things. That is my humble opinion and I am going to add such a warning to TQ editor as I do think we should all try to honor some general "Best Practices", but definitely I would not just avoid doing something for the sake of philosophy if it makes sense otherwise.

EDIT-also, Devil's Advocate speaks sharply, but there was nothing pointed at anyone there, just a viewpoint.
Post Reply