Evolution of the SD card API

Topics related to the API, programming discussions & questions, coding tips, bugs, etc. should go here.
User avatar
D3thAdd3r
Posts: 3293
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Evolution of the SD card API

Post by D3thAdd3r »

Unfortunately not yet. If you do the appropriate logic sequences(that which would accomplish a write on real hardware) the console window should output some "not emulated" type of message for each byte written.
User avatar
Jubatian
Posts: 1569
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Evolution of the SD card API

Post by Jubatian »

Just something for testing. Preferably everyone who has some SD cards laying around.

Currently it initializes and reads the first sector from the SD card. What it does in extra compared to the SD libraries we already have are the followings:
  • It has CRC checking enabled (CMD59), calculates properly command & data CRCs.
  • It is SDHC aware.
There are two numbers in the upper left corner: normally if a card is in, they read '0' and '0', then there is an identification of the card: 'SC' for Standard Capacity and 'HC' for High Capacity. In the middle you will see the first sector, and its CRC value as the algorithm calculates it.

The first number is the initialization result (0: Success), the second is a CMD17's result (0: Success). Try to start the program with as many cards as you can, and report whether other numbers pop up indicating failure (including the number, and whether it is a single occasion or deterministic). I got it working with all the cards I have.

If this thing works, then I am going to work further on it aiming towards a FAT16 / 32 supporting library for basic access including support for fragmentation. Later goals are a new bootloader.

(It works in the most recent CUzeBox to which I added data CRC calculation for now, later I will complete it for full SD CRC emulation)
Attachments
sdcard_lib_20170731.tar.bz2
SD library proto
(26.27 KiB) Downloaded 325 times
User avatar
D3thAdd3r
Posts: 3293
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Evolution of the SD card API

Post by D3thAdd3r »

Very cool, sorry I have had no time for Uzebox lately to try this yet, but this is an important development as I see it.

I am not sure what the performamce implications are, but this is very interesting if we could use larger cards especially as it is hard to find non-SDHC cards. Before I might have been "opposed" to any such thing for performance, and probably said as much, but ultimately making all cards work(and CRC is nice as some report possible bad flashes at times)and leaning on SPI ram for guaranteed timing could make "this game requires a fast card" or "why doesn't my card work on Uzebox" a thing of the past.

How would this tie in with simpleSD?
User avatar
Jubatian
Posts: 1569
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Evolution of the SD card API

Post by Jubatian »

This won't be a too fast library as I will rather aim for robustness than speed (as required in a bootloader). It should still be fine for games not wanting to do real-time SD access (including possibly every SPI RAM game which might be created in the future).

I won't aim for compatibility with sdBase as sdBase's interface is not even really suitable for going after supporting fragmentation (I want to target a very tolerant bootloader and high tolerance in games which don't need to push the SD to the edge, rather just have data occasionally from it). Games which demand streaming may still use sdBase.

(I will likely design the interface so it is possible to build a non-fragmentation aware, but SDHC capable real-time layer over it for games which need that)

Full sdBase interface compatibility is impossible with SDHC support as it doesn't fully hide the SDSC interface having byte addresses (which can not cover an SDHC card's storage, so if a game file ends up beyond the 4Gb boundary, it will simply not work even if the routines were made SDHC aware).

I am tempted to go for the bootloader with the library exposed from within it. The uzebox kernel then might also contain the library, but with such wrappers which if a suitable bootloader is there, it would rather use that (so games compiled with such a kernel would work both on old and new bootloaders, with new bootloader they would use the bootloader's possibly upgraded / fixed / enhanced SD support, allowing to improve card compatibility later if necessary).
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Evolution of the SD card API

Post by Artcfox »

I haven't had a chance to check this out yet, but it sounds awesome. SDHC support is one of the things that I have really really wanted, because it is pretty much impossible to find a non-HC card that isn't counterfeit.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Evolution of the SD card API

Post by Artcfox »

Sandisk microSD 2GB - 0 0
Samsung microSD 2GB - 0 0
Sandisk microSDHC 8GB - 0 0
Sandisk microSDHC 4 GB - 0 0

Looks good!
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: Evolution of the SD card API

Post by nicksen782 »

I've been looking around and have not yet found a clear answer yet.

Would it be possible for the Uzebox to mount a file on the SD card and determine if it is fragmented? I'm not talking about fixing the fragmentation. I just want to be able to detect it.

I'm using sDbase.
User avatar
Jubatian
Posts: 1569
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Evolution of the SD card API

Post by Jubatian »

Artcfox wrote: Thu Aug 10, 2017 1:30 am Sandisk microSD 2GB - 0 0
Samsung microSD 2GB - 0 0
Sandisk microSDHC 8GB - 0 0
Sandisk microSDHC 4 GB - 0 0

Looks good!
Thanks, didn't forget about it, just the Mode 3 blitter boost project intervened (I picked that up on seeing Larry's Iros).
nicksen782 wrote: Fri Aug 25, 2017 6:52 pm I've been looking around and have not yet found a clear answer yet.

Would it be possible for the Uzebox to mount a file on the SD card and determine if it is fragmented? I'm not talking about fixing the fragmentation. I just want to be able to detect it.

I'm using sDbase.
You can! You have to locate the file's first sector in the FAT, then walk through it in the FAT checking whether it is contiguous ( https://en.wikipedia.org/wiki/Design_of ... ile_system , it is around the middle, in the chapter "File Allocation Table"). If it isn't contiguous, then the file is fragmented. But if you are at it, you might just support the fragmented file in the end unless you want to stream or slow seeking is prohibitive (at least once you do this, you will understand how a fragmented file can be accessed). Code for determining the first sector by filename is there in SDBase.
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: Evolution of the SD card API

Post by nicksen782 »

Not sure how to do that off the top of my head. I'll need to find the FAT (there are 2 of them and they should be identical) then I need to find the start sector of the file then look for that number IN the FAT?
User avatar
Jubatian
Posts: 1569
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Evolution of the SD card API

Post by Jubatian »

nicksen782 wrote: Fri Aug 25, 2017 8:42 pm Not sure how to do that off the top of my head. I'll need to find the FAT (there are 2 of them and they should be identical) then I need to find the start sector of the file then look for that number IN the FAT?
SDBase by a quick skim doesn't seem to touch the FAT at all, it goes only until finding the first cluster (which is in the Directory Entry, at 0x1A) of the found file. You will need to look at the FAT using this value (the two FATs should be identical, so just use one of them) as an index (it is essentially just a list of 16 bit values making the file's clusters a chained list. The index of the FAT is the cluster number, the 16 bit value stored is the next cluster or a termination marker). CUzeBox generates a proper FAT for the virtual FS (FAT16), so you can test your code with it (I am just mentioning since by the above of course things might just work even without a simulated FAT).
Post Reply