Streaming Music(SD, SPI ram, Network, etc. source)

[nicksen782]

Could you include the sd file too? I saw an error message stating that it was missing.

[D3thAdd3r]

Attached, is the previous version I am still using which I have not yet changed to keep the Channel Volume events so as not to introduce more variables yet. Please let me know if you hear or find anything, I simply cannot notice anything at this point.

[Artcfox]

My reference has a WaitVsync(60); in the beginning before the main loop to give cuzebox time to start up and run the CPU at a constant speed before starting the song.

Otherwise the beginning of the song sounded all messed up, as the CPU speed was stabilizing.

It might be a better comparison if you could add that 1 second pause in the beginning of your test ROM, but I'll take a listen now.

[Artcfox]

Okay, so I tried it again with the new hex file you posted, and it looks (and sounds) like the first and second playthrough sounds correctly, no more stretched out notes in the beginning.

What I did was hack uzem to output a WAV file if you pass the -r flag by changing libmp3lame to pcm_s16le and the .mp3 to .wav:

Code: Select all

diff --git a/tools/uzem/avr8.cpp b/tools/uzem/avr8.cpp
index 4b8d64d..3df649e 100644
--- a/tools/uzem/avr8.cpp
+++ b/tools/uzem/avr8.cpp
@@ -2132,7 +2132,7 @@ bool avr8::init_gui()
                        return false;
                }
 
-               avconv_audio = popen("ffmpeg -y -f u8 -ar 15734 -ac 1 -i - -acodec libmp3lame -ar 44.1k uzemtemp.mp3", "w");
+               avconv_audio = popen("ffmpeg -y -f u8 -ar 15734 -ac 1 -i - -acodec pcm_s16le -ar 44.1k uzemtemp.wav", "w");
                if(avconv_audio == NULL){
                        fprintf(stderr, "Unable to init ffmpeg.\n");
                        return false;

Then I ran:

Code: Select all

uzem -r midisong.hex

and let it play through the song twice, and as it started a third time, I hit Esc to kill it. The above hack to uzem makes it so it can't generate a proper video, so it deletes the video it saved temporarily, but since we renamed the file from uzemtemp.mp3 to uzemtemp.wav, it can't delete the wave file, so we can rename that to ref1.wav after, and we have exactly what we want. I also made a second copy of this wave file called ref2.wav.

Then I ran:

Code: Select all

uzem -r SDMusicDemo.hex

and did the same procedure as above to get a new wave file recording, which I renamed stream1.wav, and made a copy of this called stream2.wav.

I opened Audacity, and imported ref1.wav and ref2.wav, and used the cut tool to remove the second playthrough of ref1.wav, and the first playthrough of ref2.wav. I then imported stream1.wav and stream2.wav and did the same thing to isolate each playthrough.

Next I selected each waveform and went to Effects > Normalize, and accepted the defaults. This makes it easier to see the waveforms on the screen.

Then I zoomed way way in near the beginning and used the Cut tool to precisely line up each waveform with each other. Now I can scroll through and see if the waveforms match up on each playthrough, and between the reference version and the streaming version. I can also mute the tracks to play them at the same time to see if anything sounds weird.

It looks like the latest streaming version you posted is self-consistent, but as you can see from the screenshots, it seems a bit stretched from the reference version. I screenshotted it at various parts while scrolling through, and if you play all the tracks at once, you can see and hear where they start to diverge.

I don't know if this is because the buffer is set too low? I can't increase the buffer fast enough to get repeatable results, so maybe you can make a new version that has a longer buffer, and feeds it more data faster, just to see if that's what is causing it. Did you use the midisong.h file that I sent you (the one that was generated from the Java MidiConvert program) or did you use midiconv to generate your own header? If you used the C++ midiconv, then that may also explain why the length is off between these version, because for some reason the speed factor is not treated exactly the same between the two midi engines.

I've attached the midisong.h that I used directly in my reference version (as generated by the Java MidiConvert program).

waves1.png (109.78 KiB) Viewed 4263 times

waves2.png (118.59 KiB) Viewed 4263 times

[Artcfox]

waves3.png (118.31 KiB) Viewed 4260 times

waves4.png (108.19 KiB) Viewed 4260 times

[D3thAdd3r]

Wow I am pretty confused on this then. Even though the top and bottom sets are self similar with each other, there are clear differences in those wave forms. That seems very off to me, as there has to be a part that is not deterministic there, but I am not aware of which part in Uzem is not deterministic save for maybe a random seed used to support the watchdog based random number seed.

The only thing I can suspect in the code would be the way it calculate delta times, or perhaps there is a bug with removed controllers..let me check that again. But does it make sense to you that these wave forms at least for the same rom, should come out byte for byte exactly the same each time in emulation?

[D3thAdd3r]

BTW I am running off the exact file you gave me originally and I have not modified it.

The timing part in mconvert would be:

Code: Select all

		delta = inBuf[i++]; /* calculate the next delta time, which is possibly encoded as multiple bytes */

		if(delta & 0x80){
			delta &= 0x7F;
			do{
				c3 = inBuf[i++];
				delta = (delta<<7) + (c3 & 0x7F);
			}while(c3 & 0x80);
		}

		if(delta == 0)/* we do not store deltas of 0(it is implied in the format) */
			continue;

		runTime += delta;/* keep track of how long this song is, in 60hz ticks, for statistic display */
			
		while(delta){			
			/* for a non-zero delta, we create a Tick End event */
			if(delta < 8){/* store 1 to 7 frame delays in the same byte as the command, 0 = 1, 1 = 2, etc. */
				outBuf[outSize++] = 0b11000011 | (((delta-1) & 0b00000111)<<2);/* 0b110DDD11, stored as 1 less than actual delay */
				delta = 0;
			}else{ /* we store longer delays in a 2 byte format */			
				outBuf[outSize++] = 0b11011111;/* delta of 7(+1) indicates that the next byte holds the actual delay */				
				if(delta > 254){/* we can store any delay as a series of 8 bit values */
					outBuf[outSize++] = 0b11111111;
					delta -= 254;
				}else{
					outBuf[outSize++] = (delta & 0b11111111);
					delta = 0;
				}
			}
		}

For timing, the stock MIDI player does this:

Code: Select all

				//read next delta time
				nextDeltaTime=ReadVarLen(&songPos);			
currDeltaTime=0;
//.....
//.....
unsigned int ReadVarLen(const char **songPos)
{
    unsigned int value;
    unsigned char c;


    if ( (value = pgm_read_byte((*songPos)++)) & 0x80 )
    {
       value &= 0x7F;
       do
       {
         value = (value << 7) + ((c = pgm_read_byte((*songPos)++)) & 0x7F);
       } while (c & 0x80);
    }


    return value;
}

In the streaming song player, it explains the +1 on delta time:

Code: Select all

else{//Tick End(0b110XXX11)
							nextDeltaTime = ((c1 & 0b00011100)>>2)+1;//possibly short version 1-7 frames, 1 byte
							if(nextDeltaTime == (0b00000111)+1)//longer version, 2 bytes
								nextDeltaTime = SongBufRead();
}

So it seems to me discarding 0 deltas should have no effect. Then subtracting one from the base, and adding it back at play time should also have no effect. I just can't see any part where this can be happening and on my end the mp3s seemed to match. Though I never had the 60 tick delay you mentioned...that is spooky.

Attached is a modified demo which should be overkill on the buffering. It also adds a 60 frame delay before playing.

[Artcfox]

Attached is a modified demo which should be overkill on the buffering. It also adds a 60 frame delay before playing.

WHATEVER YOU DID FIXED IT!!!!!

The middle 2 muted tracks were the old hex file, and the top two are the first 2 plays of my reference song, and the bottom 2 are the latest hex file you uploaded.

I can even play all 4 of those audio tracks at the same time, and it sounds harmonious and the waveforms look the same.

Edit: Interestingly, when zoomed in a lot, the first and second plays look slightly different, but sound the same. The first play of the MIDI player and the first play of new compressed streaming player look identical, and the same with both of their second plays. Looks like you nailed it!

fixed.png (131.05 KiB) Viewed 4246 times

fixed2.png (105.29 KiB) Viewed 4242 times

[Artcfox]

BTW: Can you share the source code to the latest project (the working ROM) and a snapshot of the kernel that goes with it, just so you have an offsite backup for it?

[D3thAdd3r]

Spectacular! I don't even want to ask questions at this point what it was, but most likely versioning again. Before I touch anything, I am putting an archive of the current state here. It is large, I didn't even run make clean on it and the forums seemed to stall trying to upload. Again, I greatly appreciate all your feed back, testing, and help during this process! I has helped a lot.

I am going to very carefully upload all the files I have touched onto github. -edit- done.