Difference between revisions of "SD Image Creation Tool"

From Uzebox Wiki
Jump to: navigation, search
(Created page with "==Introduction== These tools are intended to save you time and self hair-pulling-out. One of the general problems that is sometimes more difficult than the code itself, is...")
(No difference)

Revision as of 21:00, 23 September 2017


These tools are intended to save you time and self hair-pulling-out. One of the general problems that is sometimes more difficult than the code itself, is to get data into an SD image that is the correct format to be loaded and used by your program. One solution I have used for years, is to manually build the data using a combination of copy/paste and in place editing inside a hex editor...that is every bit as fun as you might imagine ;) The real issue, that is a huge waste of time that is not focused on the fun parts of development. It is also error prone, and potentially a huge pain if formats need to change later or resources need to change size or move to other locations. So far my games done manually, do indeed have spurious information and wasted space as remnants of things moving and formats changing as the game evolves. That leaves a large temptation to leave empty voids between data for future changes, and for bigger things, this space waste adds up even for the size of an SD card.

With the advent of Streaming Music, it became very important to have an automated way to build the data, as often times music will change quite a bit from it's initial conception, just as graphics will. Tack that on with the fact that many will likely want to use SPI Ram to offload multiple types of resources(graphics, music, etc.), there is a clear need for automatic and fast creation of these images. Now, some power users might note that all this could likely be done with good usage of built in things like "cat" on Linux with perhaps some scripting. That is also valid and perfectly good, but not necessarily cross platform or common knowledge. Also, simply having a small tool with easy to understand source code and built for more specific tasks, can have an advantage. And so here it is, since I found it's creation necessary, to save my sanity!!

What It Is And Is Not

This is designed to be a very simple set of tools, that do what they need to do, and nothing else. They are generally agnostic, and do not know at all, what this data is nor do any modification(except for streaming music compressed format). There is already a good and powerful selection of tools available for graphical, music, and SFX generation and there is no sense in duplicating that functionality. These tools are designed to harness the direct output of these well established programs, and performing only mild formatting, turn it into usable data in an SD image without you ever having to open up a hex editor to manually intervene. Then you can focus on programming which is productive and fun.


This tool does what it's name implies. If you have a file with a C array, you can turn this into a binary representation and directly write it to a file, at any position you choose. You can optionally also have it write the offset that the data was written to, at another location. This is designed to make creating directories which contain pointers to find the data, easy. In this way, your program can find different objects/entries, without having to hard code addresses/offsets.

This is rather useful for a multitude of things, as most of the existing utilities will output a C array originally intended to directly include in the program. Instead, all these tools can be used exactly as before, to build an SD image with the data. This allows all existing tools to be useful for SD games without any extra tools or arcane knowledge.

As an example let's imagine that we have used gconvert to generate tile and map data from an image, and we would like to store that ready to use(perhaps in the way described in Flash Free Screens From SD) on the SD card. In order to keep the source code simple, and the command line usage simple as well, it can only do 1 array at a time, but you can choose the array. In gconvert imagine you have tile data followed by map data. Let's first put the map data somewhere, and put a pointer to it in the directory so our program can find it:

./hex2bin gfxin.inc SD_IMAGE.DAT -o512 -d0 -z3

This will put a binary representation of the first C array in the file(the map data), starting at offset 512 in the file "SD_IMAGE.DAT". If the file exists, it will not destroy it, and will overwrite anything starting at offset 512 up to the size of the data, expanding the file if necessary. If it does not exist, it will create it. It will also write a 24 bit(3 byte) value at offsets 0..3(MSB first) that is the offset of that data. This directory entry can be written anywhere(specified by -d0 here), and be any size(specified by -z3 here), though 24 is likely sufficient for most cases. So in this case starting at offset 0 in the file, we would find

0x00,0x02,0x00,//24 bit MSB first hexadecimal representation of decimal 512, the offset we wrote the map data to

Now if we also wanted to put the second array in the file "gfxin.inc" some where else, we would call it again like:

./hex2bin gfxin.inc SD_IMAGE.DAT -o2048 -d3 -z3 -k0

Again we are using a 3 byte/24 bit directory entry specified by "-z3"(could be "-z2", "-z4", for 16 bit or 32 bit entries, any format you like), and we are putting that directory entry at offset 3 in the file(because the first entry used bytes 0..2). We are putting the tile data after the map data we wrote before, at offset 2048 specified by "-o2048". All of this of course is read from "gfxin.inc" and is written to "SD_IMAGE.DAT" like before. We are also specifying "-k1" because by default the program will read the first C array it finds(which was the map data). Since we already did that in the previous step, we tell the program to skip the first array it finds, which will cause it to read the second; in this case that is the tile data. Doing these 2 steps we have total control of putting any graphical data wherever we want, as well as simultaneously creating a directory so our program can find that data. This example has the techniques found in Flash Free Screens From SD and Flash Free Screens From SPI Ram in mind, though they could be used for any other technique you might require. It takes multiple steps, but it should be simple to understand yet powerful enough to handle any need, as opposed to more complex possible solutions.

Another example of using this technique for music can be found at Streaming Music.