Legend of Zelda (RPG game)

[D3thAdd3r]

Nice, it's amazing how many iterative optimizations are usually possible in just about everything. Even laying something out pretty well to begin with, once things start to connect together, there is more and more interesting ways to do the same thing better.. In the end all those add up big time, and of courses this is a big game where that is required. Over time I see clearly you are willing to do the work and iterative improvements while staying motivated towards the end goal. All things are motivation anyway, the end point and everything else comes from that. Also cool, that you are doing tools that are not some cryptic thing only you could understand. There are occasionally some users that come and then go, because they wanted to create something like a new quest, but were not interested in the outright programming aspect. I guess those are the "artist types", and I think we lose a lot of those retro-enthusiasts just due to not having anything for them to do that isn't intimately tied with programming. I am bad about making reusable tools, but they really do have appreciable effects.

SD music streaming I forgot about for a bit because for some inexplicable reason anything I build that uses SD fails to work PFF, simpleSD, etc. Reinstalled everything and no dice, and I really hate problems like that. I figure my linux box can build SD stuff so I might take a look at it again soon. A 16 byte buffer is likely required, and I suppose willingness to use a non-standard sound engine until it is stable enough to be #define in the kernel...if ever.

[D3thAdd3r]

Do you have any code for SD streaming yet? I have wanted this forever, don't need it at the moment, but it needs doing for some kind of solution everyone can use. Definitely have some plans for this eventually, but so far all my solutions do not blend too well with the existing song player. Can't find a clean way to have it just #define between stock and SD due to looping issues. When I get a chance I will find what I have and throw it out there to get thinking on it.

[D3thAdd3r]

Lost whatever code I had, but a different approach was needed anyway. At least the first step is to get the music engine reading from a circular buffer, so just to throw some code out there(because I have not and will not have a decent chunk of time for a bit) here is some kind of theoretical first steps towards it:

Code: Select all

	u8 song_buffer[SD_SONG_BUFFER_SIZE];
	u8 song_buffer_head, song_buffer_tail;
	u8 song_buffer_last_byte, song_buffer_state;
	
	void song_buffer_add_byte(uint8_t t);
	u8 song_buffer_read_byte();
	u8 song_buffer_bytes_available();
	u8 song_buffer_fill();

	void song_buffer_add_byte(u8 t){
		if(t == 0xFF && song_buffer_last_byte == 0xFF){//0xFF,0xFF = end of data marker, do not store it, and eat the last 0xFF from the stream(it is not real data for the song player)
			if(!song_buffer_head)
				song_buffer_head = SD_SONG_BUFFER_SIZE-1;//last 0xFF was at the end of the buffer
			else
				song_buffer_head--;//last byte was somewhere before the end of the buffer
			//set state to seek to first sector of loop point
		}else{//more data to add
			song_buffer[song_buffer_head++] = t;
			if(song_buffer_head >= SD_SONG_BUFFER_SIZE)
				song_buffer_head = 0;
		}
	}
	
	u8 song_buffer_read_byte(){
		u8 t;
		if(song_buffer_head == song_buffer_tail){//need to read more bytes
			do{
				if(!song_buffer_fill())
					return;
			}while(!song_buffer_bytes_available());
			
		}
		
		//there is some data in the buffer
		uint8_t t = song_buffer[song_buffer_tail++];
		if(song_buffer_tail >= SD_SONG_BUFFER_SIZE)
			song_buffer_tail = 0;

		return t;
	}
	
	u8 song_buffer_bytes_available(){
		if(song_buffer_head > song_buffer_tail)
			return song_buffer_head-song_buffer_tail;
		else
			return song_buffer_tail-song_buffer_head;
	}
	
	u8 song_buffer_fill(){//this would be called every frame unless you were using the SD for something else
		if(song_buffer_state == 0)//not running a song
			return 0;
		else if(song_buffer_state == 1){//seeking a sector
			//seek to a sector position
			song_buffer_state++;
		else if(song_buffer_state == 2){//reading stuff bytes
			//read stuff bytes
			song_buffer_state++;
		}else{//reading bytes
			uint8_t left = SD_SONG_BUFFER_SIZE-song_buffer_bytes_available();
			while(left){
				if(GetVsncFlag())
					break;
				uint8_t t;//read a byte from the SD
				song_buffer_add_byte(t);
				left--;
			}
		}
		return 1;
	}

That 0xFF,0xFF is some idea I had to simplify the looping issue. That issue exists if you mindlessly fill the buffer, when you actually send a end of loop marker to the song player, you will have a buffer full of garbage from after the song data. Instead, simply pad out the sector with the first bytes of the start of the song(after the last normal song byte). In all cases, when a sector end is found go back to the queue sector mode and it should work...there may be more clever ways to do it but they are only good if the worst case scenario is better.

And this is some preliminary idea of what needs to be modified in the song player:

Code: Select all

		#elif MUSIC_ENGINE == SD

			
			//process all simultaneous events
			while(currDeltaTime==nextDeltaTime){

				c1=song_buffer_read_byte();
				songPos++;

				if(c1==0xff){
					//META data type event
					c1=song_buffer_read_byte();
					songPos++;

				
					if(c1==0x2f){ //end of song
						playSong=false;
						break;	
					}else if(c1==0x6){ //marker
						c1=song_buffer_read_byte();//read len
						songPos++;
						c2=song_buffer_read_byte();//read data
						songPos++;
						if(c2=='S'){ //loop start
							loopStart=songPos;
						}else if(c2=='E'){//loop end
							songPos=loopStart;
						}
					}
				

				}else{

					if(c1&0x80) lastStatus=c1;					
					channel=lastStatus&0x0f;
				
					//get next data byte
					//Note: maybe we should not advance the cursor
					//in case we receive an unsupported command				
					if(c1&0x80){
						c1=song_buffer_read_byte();
						songPos++;
					}

					switch(lastStatus&0xf0){

						//note-on
						case 0x90:
							//c1 = note						
							c2=song_buffer_read_byte()<<1;//get volume
							songPos++;
						
							if(tracks[channel].flags|TRACK_FLAGS_ALLOCATED){ //allocated==true
								TriggerNote(channel,tracks[channel].patchNo,c1,c2);
							}
							break;

						//controllers
						case 0xb0:
							///c1 = controller #
							c2=song_buffer_read_byte();//get controller value
							songPos++;
						
							if(c1==CONTROLER_VOL){
								tracks[channel].trackVol=c2<<1;
							}else if(c1==CONTROLER_EXPRESSION){
								tracks[channel].expressionVol=c2<<1;
							}else if(c1==CONTROLER_TREMOLO){
								tracks[channel].tremoloLevel=c2<<1;
							}else if(c1==CONTROLER_TREMOLO_RATE){
								tracks[channel].tremoloRate=c2<<1;
							}
						
							break;

						//program change
						case 0xc0:
							// c1 = patch #						
							tracks[channel].patchNo=c1;
							break;

					}//end switch(c1&0xf0)


				}//end if(c1==0xff)

				//read next delta time
				
///////////////////////////nextDeltaTime=ReadVarLen(&songPos);			
    				u16 varlen = song_buffer_read_byte();
				songPos++;

				if(varlen & 0x80){
					u8 var_t;
					varlen &= 0x7F;
					do{
						var_t = song_buffer_read_byte();
						songPos++;
						varlen = (varlen<<7)+(var_t & 0x7F);
					}while(var_t & 0x80);
				}

/////////////////////////////////////

				currDeltaTime=0;
		
				#if SONG_SPEED == 1
					if(songSpeed != 0){
						uint32_t l  = (uint32_t)(nextDeltaTime<<8);

						if(songSpeed < 0){//slower
							(uint32_t)(l += (uint32_t)(-songSpeed*(nextDeltaTime<<1)));
							(uint32_t)(l >>= 8);
						}
						else//faster
							(uint32_t)(l /= (uint32_t)((1<<8)+(songSpeed<<1)));

						nextDeltaTime = l;
					}
				#endif

			}//end while
		
			currDeltaTime++;

Totally untested so far, but that is basically my idea for the general workings of it and I can never lose experiment code if I post/back up to the forums. I will get the approach working from flash first, then add in the SD interfacing parts...there is just potentially some speed issues, we will see.

[nicksen782]

I'll need to get to music soon.

For now I've been re-planning data structures to hold all the parts of the game. Right now I have it so that the exit ids of a screen are all provided via the sd card binary (saved at least 384 bytes of flash.) I can control other things such as new screen x/y position and screen draw transition effects.

I'll also need to use the SD card to store things such as already found secrets/bombed walls/unlocked doors/etc.

Zelda work continues! I would like to have another demo version online within a week or two.

[D3thAdd3r]

Excited to see the next demo. For the sd music I want to make it a priority so I am using it in my current game to force the issue.

Some sort of interrupt system meeds to exist I figure. Basically the screen, etc. stuff that needs the card will have to request it from the music player. Then the music player will fill its buffer, and finish out the last SD command it started. Then it will await the other task to finish, then set a flag that it can start buffering music again.

This is the only solurion I could find. Depending on the buffer fullness, there could be some variable delay then, sub-second, of lag before you could write out secrets/etc. and load the next screen. It seems more tolerable than having the music stall. Does that seem like it would work?

[nicksen782]

Hmm.. so, how large of a music buffer will be needed? How many seconds of audio is that? How long does it take to read in X number of seconds of audio from the SD card?

I would have the stuff that needs to be written all stored in ram for the current screen and any changes (I would need a flag to indicate that a change occurred) could be written to the SD card when the player exits the screen. SD data would only be written if needed.

We already have vsync as an interrupt timer, right? Correct me if I'm wrong. We definitely should not interrupt sprite blitting or graphics drawing. How much time is really needed to fill a local music buffer for an acceptable number of seconds of music? What if the music was really simple and therefore took less space? Lets not have the music restart on every new screen.

[D3thAdd3r]

I could imagine 10 note on events a second, across the channels, during some lively moments. Each of those takes 3 bytes, so I with a 32 byte buffer you can only expect to last 1 second which I estimate will be enough for a transition. Probably when the edge of the screen is reached, hide all sprites except Link, then request the SD. Then there is more free cycles available for this stress point.

I said interrupt just for the lack of imagining a better word, but not a real interrupt. It is just something like a flag that the state machine always checks so the SD music engine finishes and stops what it is doing (a real interrupt like vsync will take over in the middle of this if need be, and it continues where it was next frame). I have experimented in the past and found a custom WaitVsync() is the right way to do this because it uses cycles otherwise wasted waiting. The flag would need to be turned off by the level scrolling code once it was done, and this way both sides do not try issuing commands simultaneously or in the middle of completing one. Basically level code doesnt start until music says so, and vice versa.

It is impossible to know exactly how long this lag period will be, and I suppose buffer size needs to be as large is as needed for the worst card you want to support. We will see how it works when we get there, but I think "any card" will work with a reasonable buffer like 32 or so.

[D3thAdd3r]

BTW the music wont restart for each screen, but instead just seek to the last place it was(while playing music that was previously buffered in the mean time) once the level scrolling code is done. I thought more about this, and assuming you are doing it similar to the original, you can indeed hide all sprites except for link. This would allow you to use ram tiles that are not needed during this transition as a buffer for the screen data to scroll, while still being able to draw link in the interim. Why do this? Because it eliminates the need to mix your scrolling code with music code, and you don't have to go back and forth between music reading SD and game reading SD with each step.

Another good thing about this approach, is that you can have more safety margins which means you can have a smaller buffer which potentially removes lag also. I might think it will be totally undetectable, but not sure. The solution is only good if it is generic, and abstracted from the rest of the game. It is simply too complicated to have a game intertwined with this kind of stuff. So essentially you would only need 2 connections into your game. A custom WaitVsync() which runs the SD music engine state machine every frame instead of standard WaitVsync(), very easy. The last part you need connection to is something like:

Code: Select all

//preparing to scroll
	HideSprites(6,MAX_SPRITES);//hide all sprites except Link, saves cycles and gives us a buffer for next screen data

	sd_music_state |= SDM_BUFFER_AND_WAIT;//tell the engine to prepare for a data drought
	do{
		CustomWaitVsync(1);//run the SD music state machine, which will load bytes until VSYNC, until it is full, then wait
	}while(!(sd_music_state & SDM_WAITING));

	RamifyNextScreen();//load SD data into now unused ram tiles
	
	//prepare to write secrets, it might not even be necessary to buffer again if it writes fast enough!

	sd_music_state ^= SDM_WAITING;//make SD engine fill buffer again, then wait. This is just to keep the buffer as full as possible, but may add lag frames.
	do{
		CustomWaitVsync(1);
	}while(!(sd_music_state & SDM_WAITING));

	SDWriteSecrets();//if applicable
	sd_music_state ^= (SDM_WAITING|SDM_BUFFER_AND_WAIT);//set SD music to normal
	ScrollLevel();//assuming this itself will also be calling CustomWaitVsync()

	//run game like normal and use all ram tiles for sprites...

I think that is something that does not require much effort or change to implement into your code, and should be generic enough for anything. Does it seem a reasonable method that would work for your project?

[D3thAdd3r]

Err, triple post to try and clarify. To explain why we are using "free cycles" that are otherwise wasted it's best to look at the standard WaitVsync():

Code: Select all

//Wait for the beginning of next frame (60hz)
void WaitVsync(int count){
	int i;
	//ClearVsyncFlag();
	for(i=0;i<count;i++){
		while(!GetVsyncFlag());
		ClearVsyncFlag();		
	}
}

You can see if your game logic does not run too long, then any cycles until next frame are used in a tight loop checking if it is time to run again. We can still synchronize to 60hz by checking this flag, but in the mean time until it happens, do other things. Naturally, if the game is missing vsync all the time, any other code in this loop is only going to make things worse. Then again on Uzebox, it generally is not much of an option to miss vsync frequently, unless you do some of that stackless black magic Jubatian was talking about a while back...ok back on topic. This would be a hypothetical custom vsync that uses the extra cycles to try and keep the music buffer as full as is possible, using the pseudo code from a few posts back. Keeping in mind that inside the state machine, it will not touch the SD card when it is SDM_WAIT(at state it sets itself, when it received SDM_BUFFER_AND_WAIT and completed filling the buffer):

Code: Select all

//Wait for the beginning of next frame (60hz)
void CustomWaitVsync(int count){
	int i;
	//ClearVsyncFlag();
	for(i=0;i<count;i++){
WAIT_VSYNC_TOP:
		if(GetVsyncFlag()){
			ClearVsyncFlag();
			continue;
		}
		sd_music_buffer_fill();//if there are extra cycles, use them.		
		goto WAIT_VSYNC_TOP;
	}
}

On the wiki I made some graphic so that I could visualize it better, as it really was not obvious to me for a long time what the actual order of events was but became critical for a few things I did:

Renderflow.png (41.33 KiB) Viewed 6351 times

Edit - Fixed broken CustomWaitVsync();

[nicksen782]

Hmm... Interesting idea. I have this at the top of my loop:

Code: Select all

while (!GetVsyncFlag()) {} ClearVsyncFlag();

I've seen other people just do this:

Code: Select all

WaitVsync(1);

Both of these would wait until the next vsync, right? My question is how long of a wait is it? I thought about trying to calculate it and display it somewhere but the code for doing that would affect the timing. Best I can think of is doing something like setting a pin high and then low after each vsync and then timing it with an o-scope. Somehow I figure there must be a better way.

You are suggesting reading from the SD card while in the user-code portion, right? It would be really nice to see how many cycles the main loop actually uses and how many are left. I'm in the process or refactoring (again) and it would be really really nice to see this.

So, you've discussed when to fill the buffer. What about how to get the sound engine to read from that buffer instead of progmem? This buffer would be ram-based and therefore (correct me if I am wrong) would take less cycles/time because of reducing the progmem reads. Does that mean that we could gain some time with the modifications to the sound engine? How much time? Any reason not to put the SD card read portion in there so it is automatic?

Another question, there is the pre and the post vsync call back. Why might you choose to use one vs the other? I'm using the post vsync callback to increment some counters that I use as simple timers (such as when the title screen triforce should change color or how long a character should flash after being hit.) Is that appropriate?

Thanks!