Flash Free Screens From SPI Ram: Difference between revisions

From Uzebox Wiki
Jump to navigation Jump to search
mNo edit summary
No edit summary
 
Line 58: Line 58:
Now that we have a handle on the data we need to load, we need to consider where to buffer this data. Because reading 1 byte at a time is extremely slow on an SD card, it is much better to read in some multiple of the sector size to avoid unnecessary inter-sector delays. In this example we will use 512 bytes, because this is a good point of low buffer size requirements and good speed. Bigger buffers will have better speed, but those are hard to come by. In this example, we will not waste that much ram for a buffer that is used only once. Instead we will "multiplex"...basically we will abuse vram, to act as an intermediary buffer to transfer SD card bytes to SPI Ram. Each time we want to access the SPI Ram after the SD card(to write the data we read), or the SD card after the SPI Ram(to read more data), we need to call the appropriate functions that will control and select the appropriate device.
Now that we have a handle on the data we need to load, we need to consider where to buffer this data. Because reading 1 byte at a time is extremely slow on an SD card, it is much better to read in some multiple of the sector size to avoid unnecessary inter-sector delays. In this example we will use 512 bytes, because this is a good point of low buffer size requirements and good speed. Bigger buffers will have better speed, but those are hard to come by. In this example, we will not waste that much ram for a buffer that is used only once. Instead we will "multiplex"...basically we will abuse vram, to act as an intermediary buffer to transfer SD card bytes to SPI Ram. Each time we want to access the SPI Ram after the SD card(to write the data we read), or the SD card after the SPI Ram(to read more data), we need to call the appropriate functions that will control and select the appropriate device.


SetRenderingParameters(FIRST_RENDER_LINE,FIRST_RENDER_LINE+5);//decrease the number of drawn scanlines for more free cycles to load data
SetRenderingParameters(FIRST_RENDER_LINE,FIRST_RENDER_LINE+5);//decrease the number of drawn scanlines for more free cycles to load data
DDRD = 0;//black screen because we are going to abuse vram(otherwise use ram_tiles, songBuf, or some other buffer you can multiplex safely)
DDRD = 0;//black screen because we are going to abuse vram(otherwise use ram_tiles, songBuf, or some other buffer you can multiplex safely)
for(uint8_t i=0;i<32*2;i++){//load 512 bytes at a time from SD into ram, then from ram into SPI ram, for a total of 32K or 64 sectors
for(uint8_t i=0;i<32*2;i++){//load 512 bytes at a time from SD into ram, then from ram into SPI ram, for a total of 32K or 64 sectors
 
sdCardCueSectorAddress((uint32_t)(sectorStart+i));
sdCardCueSectorAddress((uint32_t)(sectorStart+i));
for(uint16_t j=0;j<512UL;j++)//load the SD bytes into ram
for(uint16_t j=0;j<512UL;j++)//load the SD bytes into ram
vram[j] = sdCardGetByte();
vram[j] = sdCardGetByte();
sdCardStopTransmission();
sdCardStopTransmission();
 
SpiRamSeqWriteStart(0,(uint16_t)(i*512UL));
SpiRamSeqWriteStart(0,(uint16_t)(i*512UL));
for(uint16_t j=0;j<512UL;j++)//load the ram bytes into SPI ram
for(uint16_t j=0;j<512UL;j++)//load the ram bytes into SPI ram
SpiRamSeqWriteU8(vram[j]);
SpiRamSeqWriteU8(vram[j]);
SpiRamSeqWriteEnd();
SpiRamSeqWriteEnd();
}
}
ClearVram();//clear the gibberish we wrote over vram
ClearVram();//clear the gibberish we wrote over vram
SetRenderingParameters(FIRST_RENDER_LINE,FRAME_LINES);//increase the number of drawn scanlines so the whole screen shows
SetRenderingParameters(FIRST_RENDER_LINE,FRAME_LINES);//increase the number of drawn scanlines so the whole screen shows
DDRD = 255;//make the screen visible again
DDRD = 255;//make the screen visible again


You are of course not limited to using vram, and in some video modes it might actually crash if putting "random" values there. For Mode 3 and several others, this will cause no problem. We must hide the screen before we do it since it will look like giberish, clear the screen after we use it as a buffer, and unhide the screen when we are all cleaned up and done. You could also use ram_tiles[], or anything else which is acceptable to temporarily corrupt with SD data. Also notice a trick used to greatly increase the speed of the load, which is to set the number of scanlines drawn to a low value, which buys thousands of extra cycles per 1/60 second frame, to perform the loading. Because this example shows a one time load, and after that not doing more loading, this works to simply do it at the beginning of the program where a small delay is no issue.
You are of course not limited to using vram, and in some video modes it might actually crash if putting "random" values there. For Mode 3 and several others, this will cause no problem. We must hide the screen before we do it since it will look like giberish, clear the screen after we use it as a buffer, and unhide the screen when we are all cleaned up and done. You could also use ram_tiles[], or anything else which is acceptable to temporarily corrupt with SD data. Also notice a trick used to greatly increase the speed of the load, which is to set the number of scanlines drawn to a low value, which buys thousands of extra cycles per 1/60 second frame, to perform the loading. Because this example shows a one time load, and after that not doing more loading, this works to simply do it at the beginning of the program where a small delay is no issue.

Latest revision as of 19:57, 23 September 2017

Introduction

This technique is a powerful extension beyond the normal Flash Free Screens From SD. If you have not read through that tutorial I urge you to first do that first as, for brevity and focus, the basic concept will not be repeated. The actual implementation of this technique will be described in a generic as possible sense, though certainly there will be different program specific designs that you might wish to implement that are modified from this.


There are exactly 4 general methods you might use to get useful data into SPI ram. The most common method of all will likely be directly loaded from SD card(with perhaps a mild form of compression). Another option which would only make sense in limited situations, would be to store the data on '644 flash(almost certainly compressed). Yet another option would be to algorithmically generate the data from code, so that only the formulas/algorithms required to generate the data take up space on the '644. Past this, it seems there is no other possible way to put useful data into the SPI ram, except by retrieving it from another machine over the internet. Naturally this opens up the possibility of using a far more powerful machine with more storage to assist the Uzebox in running a game that might be seen as "too big" to be on Uzebox. Because using SPI Ram with an Uzenet module, or the new revisions PCBs that have it built in also implies the inclusion of the ESP8266, this is not a far fetched idea at all. Only the expected most common method of SD->SPI Ram will be covered in this tutorial, though the basic and general technique described here should work no matter where the data actually comes from initially.


Formatting The Data

Before we can draw something to the screen from the SPI ram, we need to load data into it. The most obvious technique, and the one this tutorial will cover, is to retrieve the data from the SD card and write it into SPI Ram. Keep in mind however, that this technique is agnostic to where the data comes from. Once the data is there, however it gets there, it will work.


The format in this example will be a simple possible solution, that will cover most expected uses. There is an arbitrary sized directory that starts at the first byte of SPI ram, and also the first byte of the SD file. The SD image will be copied 1:1 to the SPI Ram in this example, so that it is easy to understand the stored format and the format in ram simultaneously..because they are exactly the same! In this example, we will use the entire first sector of the SD card(512 bytes) to store a sequential series of 3 byte values one after another. These 3 byte values are a 24 bit address, which represents a direct byte offset to the start of each data entry. In this example the data entries are maps to put into vram, and tile data to put into ram tiles. So the offset that the first resource starts at is likely 512 which is immediately after the directory, and so the first 3 bytes of the SD image/SPI Ram would be 0x00, 0x02, 0x00; though it certainly wouldn't have to as this pointer scheme is very flexible. 512 bytes is arbitrary, and so is using a full 24 bit address. If you do not need to access data beyond the limits of a 16 bit address(0-65535, which is half of SPI Ram capacity), then you can fit 50% more entries in the same space. Basically, use a directory size and pointer size that makes sense for your particular game requirements. As example, the first 6 bytes which let you put the vram map and ram tile data anywhere you like:

0x00,0x20,0x00,//address to vram data(map)
0x00,0x60,0x00,//address to ram tile data(tile pixels)

Here we see that for each screen, it takes 6 bytes in the directory to describe where it is. That is 3 bytes for the map pointer, and 3 bytes for the pixel pointer. You can use more compressed version, for instance if you assume the size of the tile map and know that the ram tile data starts directly after that;no need for a separate pointer to map and pixel data then, since you will know where it is based on the location of the map data plus the size. This tutorial is intended to make comfortable the concept, not to describe some arbitrary format that must be followed in all cases. It essentially uses a bit more of the SPI Ram for directory than might be necessary in some cases, so as to allow an "unlimited" approach that is the most flexible possible. I believe this is a good approach in general, as a few bytes out of 128K is not likely to be the difference between having a feature or not...save optimization time for other stuff.


Creating An Index Table

It is easiest and most flexible to not encode data at arbitrary places that the statically compiled program must be aware of; which avoids the annoyance of updating these as things change. This is basically the same concept as shown in Flash Free Screens From SD. This can be done manually, and perhaps it will be easier to understand things having done this once. Past that, I would no longer recommend wasting your time doing this manually as it is inefficient and error prone compared to doing it with a tool. It is also a huge hassle when you change things like screen height, and things take more or less space than they originally did. I created a simple tool specifically for this kind of purpose(basically required for Streaming Music, but it is useful for anything similar). The usage of this tool is described in detail on SD Image Creation Tool, and wont be covered here. Following that tutorial, you should more or less easily be able to create an SD image in the format you require. Now you just need to get it into SPI Ram and go at it!


Loading SPI Ram

SD cards are accessed on the basic of 512 byte sectors, though random seeks are possible by going to the start of a sector and "eating"(reading without using) bytes to get to the offset you require which is not 512 byte aligned. It is simpler, faster, and I daresay better, to just go with the 512 byte nature of the card.


The first thing we must do is initialize the SD card, for which at this point in time I recommend the simpleSD library above anything else(assuming you do not require some of the features that PetitFS and FatFS have, which simpleSD does not..and space is not an issue). This tutorial will assume using simpleSD. First we will need to store the filename of the SD card in flash. Keep in mind that simpleSD is based around finding the first sector of a file which has no fragmentation, and working from there. It is intended for 8.3 file names, where the actual '.' is not stored. The following string will be used to find a file called "SD_SPI_.DAT". We also need a variable to keep track of which sector this is.

const char  fileName[] PROGMEM = "SD_SPI_DAT";
long sectorStart;

Now we need to find the first sector that this file data is stored at:

	sdCardInitNoBuffer();
	SpiRamInit();//this must be done after the SD card is initialized

	if((sectorStart = sdCardFindFileFirstSectorFlash(fileName)) == 0)//can we find the first sector of the file without error?
		Print(0,0,PSTR("FILE SD_SPI_.DAT NOT FOUND ON SD CARD"));
	else{
		sdCardCueSectorAddress(sectorStart);//seek to the first byte of the sector our file starts on
		Print(0,0,PSTR("FOUND SD_SPI_.DAT"));
	}

Now that we have a handle on the data we need to load, we need to consider where to buffer this data. Because reading 1 byte at a time is extremely slow on an SD card, it is much better to read in some multiple of the sector size to avoid unnecessary inter-sector delays. In this example we will use 512 bytes, because this is a good point of low buffer size requirements and good speed. Bigger buffers will have better speed, but those are hard to come by. In this example, we will not waste that much ram for a buffer that is used only once. Instead we will "multiplex"...basically we will abuse vram, to act as an intermediary buffer to transfer SD card bytes to SPI Ram. Each time we want to access the SPI Ram after the SD card(to write the data we read), or the SD card after the SPI Ram(to read more data), we need to call the appropriate functions that will control and select the appropriate device.

	SetRenderingParameters(FIRST_RENDER_LINE,FIRST_RENDER_LINE+5);//decrease the number of drawn scanlines for more free cycles to load data
	DDRD = 0;//black screen because we are going to abuse vram(otherwise use ram_tiles, songBuf, or some other buffer you can multiplex safely)
	for(uint8_t i=0;i<32*2;i++){//load 512 bytes at a time from SD into ram, then from ram into SPI ram, for a total of 32K or 64 sectors

		sdCardCueSectorAddress((uint32_t)(sectorStart+i));
		for(uint16_t j=0;j<512UL;j++)//load the SD bytes into ram
			vram[j] = sdCardGetByte();
		sdCardStopTransmission();

		SpiRamSeqWriteStart(0,(uint16_t)(i*512UL));
		for(uint16_t j=0;j<512UL;j++)//load the ram bytes into SPI ram
			SpiRamSeqWriteU8(vram[j]);
		SpiRamSeqWriteEnd();
	}
	ClearVram();//clear the gibberish we wrote over vram
	SetRenderingParameters(FIRST_RENDER_LINE,FRAME_LINES);//increase the number of drawn scanlines so the whole screen shows
	DDRD = 255;//make the screen visible again

You are of course not limited to using vram, and in some video modes it might actually crash if putting "random" values there. For Mode 3 and several others, this will cause no problem. We must hide the screen before we do it since it will look like giberish, clear the screen after we use it as a buffer, and unhide the screen when we are all cleaned up and done. You could also use ram_tiles[], or anything else which is acceptable to temporarily corrupt with SD data. Also notice a trick used to greatly increase the speed of the load, which is to set the number of scanlines drawn to a low value, which buys thousands of extra cycles per 1/60 second frame, to perform the loading. Because this example shows a one time load, and after that not doing more loading, this works to simply do it at the beginning of the program where a small delay is no issue.


Putting It On Screen

Now we have all the pixel data for our graphics tiles, we have the information for which tiles to draw where, and our program can easily find exactly where all that is in the SPI Ram. The next part is just as simple, and it's where our hard work pays off! Understanding everything we have read so far, the following code should hopefully be self explanatory:


void DrawMapSPIRam(uint8_t id){
	HideSprites();//IMPORTANT! This might take more than 1 frame. Sprite blitting during the read will ruin our buffered data
	//SpiRamSeqReadEnd();//some considerations may be required if you are streaming other things from the SPI ram
 
	uint32_t mapBase, pixelBase;

	mapBase = SpiRamReadU32(0,(id*6)+0);//read from the start of the 2 pointer pair in the directory
	mapBase >>= 8;//we read 4 bytes instead of 3. The 4th byte is the 1st byte of the next pointer, get rid of it
	pixelBase = SpiRamReadU32(0,(id*6)+3);
	mapBase >>= 8;

	SpiRamReadInto(0,mapBase,vram,sizeof(vram));//load all the tile data into vram[]
	SpiRamReadInto(0,pixelBase,ram_tiles,sizeof(ram_tiles));//load all the pixel data into ram_tiles[]
}


There you go that's it. You don't have to worry about anything it entirely loads up all your ram tiles with data and your vram with the map. If you have less unique tiles than you have ram tiles that is fine as those ram tiles will be filled with whatever data is after, but wont show up on screen since your map doesn't point to them. From here on you can draw an entire screen of graphics anywhere you want by simply calling:

DrawMapSPIRam(screen_num);

Your title screen, intermission, level select, credits, game over, ending, everything can be drawn to the screen form only a couple bytes for the function call.


Closing Notes

As previously said, there is more information that will probably be useful on Flash Free Screens From SD, that I skipped for this tutorial. The same limitations, and techniques specific to vram and ram tile usage are the same. However, due to the much faster and entirely predictable performance of the SPI ram, there are more advanced techniques that you could implement. There are definitely hard limits to things like animations, that do not work well with the SD card and it's high latency, that will be realistic to pull off with SPI Ram.

Also, if you are going to use SPI Ram in your new game, I would also recommend considering the information found at Streaming Music. This tutorial was designed to be compatible with that tutorial, so that you can offload both graphical and music data to the SPI ram, which put together is a serious amount of potential content that simply could not fit in flash. I hope this helps someone start using SPI Ram for cool things in their new game!