SD Card Best Practices

Discuss general Uzebox topics here: features, wish list. nice to have, etc.
Post Reply
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

SD Card Best Practices

Post by Artcfox »

Moving this to a new thread because it is off topic from the original thread.
danboid wrote: Mon Feb 20, 2023 11:40 am
Artcfox wrote: Sun Feb 19, 2023 8:25 am
danboid wrote: Sun Feb 19, 2023 8:19 am * A minimal file manager, if only for deleting files off SD but copy and move would be cool too.
Just a FYI, deleting files off the SD card can cause fragmentation, which would break a bunch of games that require their level data to be contiguous (for speed purposes) if any of those games were copied to the SD card after the fragmentation took place.

Edit: The best way to restore full speed of a fragmented SD card is to format it with this proprietary tool: https://www.sdcard.org/downloads/formatter/ which only has a Windows and Mac version, because it does some behind the scenes magic to make sure the filesystem parameters match up with the underlying hardware implementation for that specific card. I guess on Linux, one could create a compressed image of the card before copying any files to it, and then use dd to restore the formatted image back.
I thought I'd read on the wiki that v5 of the bootloader handles fragmentation better?

It seems that the majority of Uzebox games can run entirely from flash without and any data stored on SD card. It seems its only really T2K and playing audio files with Uzeamp that might require a defragged SD card. I've been deleting and adding stuff onto my SD card without issue so far. T2K still seems to play after all my messing without any defrags so maybe its only really Uzeamp that is hyper sensitive to fragmentation?

I would imagine that simply copying all of the files off your SD card, then doing a full mkfs.vfat format or dd'ing the disk and then doing a quick vfat format should both fix fragmentation but there is also at least one tool to defrag FAT32 (and NTFS) disks from Linux:

https://github.com/749/UltraDefrag4Linux

There is at least one more but I'm struggling to find it or remember what it was called.
You are correct that the v5 bootloader handles fragmentation properly when flashing games, however that is just the bootloader. Games are free to use whatever routines they want to read from the SD card. They can use the bootloader's high level filesystem routines to read files and it should handle fragmentation properly, or they can use the low level sector routines directly, enabling or disabling CRC as they see fit, or they can mix and match. For my WIP I am planning on using the bootloader routines to get the starting sector and then use the super-optimized sdBase.S routines to read sectors assuming the file is contiguous, because I am planning on doing this from within HSYNC and there is not that many cycles free, and it needs to be super fast.

T2K works for you because you likely copied it to the SD card before you started deleting files. Had you deleted files first and copied it while you had fragmentation, it might not run correctly. Same with my 8-way scrolling demo.

You are also correct that running mkfs.vfat or using the Windows formatting tool on an SD card would get rid of the fragmentation, but a while back I had discovered a bug in the v5 bootloader (it was not looking far enough for the start of the partition) with newer SD cards. The way the card came from the factory and the way the sdcard.org tool formatted the cards put something like 4 or 16 MB (memory is fuzzy) of free space before the start of the first partition. The reason for this variable amount of free space on different cards is to align the filesystem with hardware-specific implementation details to maximize performance. If you reformat it yourself with a different alignment it can totally kill the performance. In a past project, I personally experienced a massive hit, to the point it was unusable for my purposes until properly reformatted with the sdcard.org tool. Jubatian fixed the bug so the bootloader can properly handle cards with large offsets before the first partition, so we can use the cards as they are from the factory at their highest speeds.
User avatar
danboid
Posts: 1937
Joined: Sun Jun 14, 2020 12:14 am

Re: SD Card Best Practices

Post by danboid »

Thanks for the gory details Artcfox!

I hope Jubatian is OK. Has anyone heard from him at all in the last year or so? He did so much for the Uzebox community.
User avatar
ry755
Posts: 226
Joined: Mon May 22, 2017 6:01 am

Re: SD Card Best Practices

Post by ry755 »

danboid wrote: Tue Feb 21, 2023 12:52 pm I hope Jubatian is OK. Has anyone heard from him at all in the last year or so? He did so much for the Uzebox community.
He posted an update on his website at the end of 2022, so I assume he's just taking a break from everything.

https://jubatian.com/articles/2022-we-wont-miss-you/
Post Reply