Bootloader API library with SDHC and FAT32

Topics related to the API, programming discussions & questions, coding tips, bugs, etc. should go here.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Bootloader API library with SDHC and FAT32

Post by Artcfox »

Jubatian wrote: Mon Apr 02, 2018 7:38 am Eh, if you need it so much, here is a solution in an attachment. Add it to the Makefile like any other assembly module (check the lines used for including kernel stuff), then you can use it as described in its header file.

Still, I wouldn't especially recommend this route. When designing the bootloader library, I could experience some of my SD cards being really slow. I mean so slow that even with the normal CRC enabled reading, waiting for the data token took occasionally more than the read itself. So realistically if you wanted to support any SD card, you may still need to spare a full VBlank for a read (it may not always happen, but if it does, be prepared to handle it in a sensible manner).

CunningFellow's streaming (Tempest background) on the other hand may still work fine even on such slow cards. If you issued a CMD18 at the beginning of the VBlank, then left the card to do whatever it needs to do to get the data ready, it may be ready to read by the time the next frame starts, and the cards are designed to be fast with sequential access.

EDIT: What about the write support I submitted as an experimental branch above? If it works okay, I would like to merge it in!
Thank you so much! It works great, and I don't miss vsync, but it looks like I still have a bug in the way I think I'm writing my level file, or the way I think I'm reading it. I verified using hexdump, cut, and diff that the first 1024 sectors of the level file match what I get reading the first 1024 sectors using this function, so the function works properly, I just have to track down my bug.

For the write support, can it only overwrite an existing file? And can I just keep calling FS_Next_Sector to extend the file properly?
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Bootloader API library with SDHC and FAT32

Post by D3thAdd3r »

Pretty sure you cannot correctly extend the file without quite a bit of potential work(and ram probably), as you would have to traverse and fix the cluster chain. That is the only way say, if at the end of your original file size the start of another file began. By following the chain you can access a file size that changes, but it is not necessarily linear with fragmentation. I always thought to not worry about fragmentation on Uzebox as just delete and paste files back on, but with changing file sizes it seems for the most part it is simply going to fragment. I think the streaming style like Cunning's do not handle that, and only touch FAT stuff to correctly find the start of a file and tread it as a linear string of sectors from there. Someone correct me if I am wrong on any of that.
User avatar
Jubatian
Posts: 1563
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Bootloader API library with SDHC and FAT32

Post by Jubatian »

That's correct, you can not extend files, only write within an existing file. Extending would involve additional work on FS level, and at that, you would have to support both FAT16 and FAT32 if you wanted to work with everything the new bootloader can handle.

Fragmentation is not as hairy as it might seem, in most cases it wouldn't be complicated to support it as the FS routines from the bootloader library handle it just fine. For SPI RAM games I think there is absolutely no reason to not support it (there I would say, there is really little need to do anything beyond FS routine access). For the "unexpanded" Uzebox, it depends, mostly on whether you want to stream and what. Games like Alter Ego I think could do well with normal FS access (loads level data, then does not access the SD card during gameplay).

I provided some info on this in the bootloader API guide, basically supporting a 4K cluster size would cover nearly all filesystems which could reasonably occur on SD cards.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Bootloader API library with SDHC and FAT32

Post by Artcfox »

Cool, I'll check it out when I get some time. Right now my life just got super crazy busy.

I can envision reserving N sectors at the beginning of a level file (after the main header) as "memory slots" for saving things.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Bootloader API library with SDHC and FAT32

Post by Artcfox »

I'm making the assumption that I can use this library to get the starting sector of a file (given a filename) and then I require my level file to be contiguous, so I can use that starting sector plus a calculated offset (based on the current camera position) in order to know which sector I need to read.

Are these assumptions correct?

Code: Select all

    uint16_t calculatedSectorOffset = (cyt * c->level->width + cxt);
    SD_Read_Sector_Nr_Nc(&c->sds, FS_Get_Sector(&c->sds) + calculatedSectorOffset);
The reason why I'm asking is because I have verified that my level generator generates the data that I expect, and I can see that it is correct with a hex editor, but for some reason my Uzebox program is not reading the correct data. I suspect it's a bug somewhere in my code, because I was able to verify that all of the sectors of the file match what I see in the hex editor.
User avatar
Jubatian
Posts: 1563
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Bootloader API library with SDHC and FAT32

Post by Jubatian »

Artcfox wrote: Wed Apr 04, 2018 1:18 pm

Code: Select all

    uint16_t calculatedSectorOffset = (cyt * c->level->width + cxt);
    SD_Read_Sector_Nr_Nc(&c->sds, FS_Get_Sector(&c->sds) + calculatedSectorOffset);
Do you really have 512 bytes worth of data for every single X:Y location? That's a quite huge level! :shock:

I guess not. I assume you try to get a byte position there. However what you get is a 512 byte sector position! (I mean like base + sector 1 represents bytes 0x200 - 0x3FF in the file and not bytes 0x1 - 0x200)
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Bootloader API library with SDHC and FAT32

Post by Artcfox »

They will be packed 4 camera positions to a sector, but until I debug this, I'm going with 1 sector per valid camera position (in tiles, not pixels) with known padding bytes to help me debug.

I just wanted to make sure that adding an offset to that API function is okay to do, as long as the file is contiguous, and it seems so.
User avatar
Jubatian
Posts: 1563
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Bootloader API library with SDHC and FAT32

Post by Jubatian »

Artcfox wrote: Wed Apr 04, 2018 7:35 pmThey will be packed 4 camera positions to a sector (...)
So you say the odd looking code is intentional then.

I could only suggest remembering the debug ports then by which you can check your values with relative ease (maybe in multiple passes if it is a larger than 16 bit variable). In C, you can access them by "*((u8*)(0x0039U)) = value;" or "*((u8*)(0x003AU)) = value;".
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: Bootloader API library with SDHC and FAT32

Post by nicksen782 »

Just my two cents.

Code: Select all

static void _emu_whisper(int port, unsigned char val) {
	if(port==0x39 || port == 0){ u8 volatile * const _whisper_pointer1 = (u8 *) 0x39; *_whisper_pointer1 = val; }
	if(port==0x3A || port == 1){ u8 volatile * const _whisper_pointer2 = (u8 *) 0x3A; *_whisper_pointer2 = val; }
}

// Usage:
_emu_whisper(0, nextSlot);
_emu_whisper(1 slotsFree);
I use this very often.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Bootloader API library with SDHC and FAT32

Post by Artcfox »

Yeah, I'm using UZEMCHR and UZEMHEX to do entire hexdumps of the sectors, and 16-bit sector numbers, and comparing them to the output of hexdump -Cv on my level file using diff. That part seems okay, so it must be something else in my code. I'll keep at it until it works.
Post Reply