Streaming Music(SD, SPI ram, Network, etc. source)

Topics related to the API, programming discussions & questions, coding tips, bugs, etc. should go here.
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by nicksen782 »

Thank you for such a great explanation. I believe that I understand well now. I'll be doing some experiments to test my knowledge.

Thanks again! Looking forward to this being in the official master.
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by D3thAdd3r »

For sure, hope you find it useful for your game. I did merge it, so I would recommend grabbing a new copy of master now.
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by nicksen782 »

I got it.

I was able to use TriggerFx to play some patches. I found that patches that only had PATCH_END would make some sort of noise and then hold forever. Save PC_NOTE_HOLD.

I was using the patches.inc file from one of the music streaming demos. Only one patch actually worked when triggered. So, I then found the patches.inc from Megatris. This works. Some patches still exhibit that strange behavior.

However, I noticed that all the music in Bubble Bobble sounded different as if the instruments were different.

I really don't understand this yet.
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by D3thAdd3r »

That is weird, do the sound effect issues remain if you disable streaming music amd rebuild?
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by nicksen782 »

I will have to test that one.

But, I should also say that with the working sound effects I could play them while music was streaming from SPIRAM. One of the channels would go out and the sound effect would be played over it. I expected this though.

I seem to remember long ago that sounds could crash a game.

I'll test more when I get home. What tests should I make sure to do?
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by D3thAdd3r »

Triggering a bad or non-existent patch can crash a game or otherwise break the running state of anything, which could be the sound engine. So music can break games too just as if TriggerFx (badPatch); if it uses such an out of bounds patch index, since basically it is going to grab something(after the patchStruct) which is not a pointer, treat it as a pointer, and interpret the data there(which is not patch data) as an index to table of a function pointers (to a possibly out of table bounds non-existent function), and pass some "random" values to it. That is by memory and have not looked in a while, but essentially that is what can happen. Broken music could therefore, possibly, behave differently from undetected in 1 kernel/compiler version to breaking something obviously in another, based on what was at the "random" code/data there. Not that this is necessarily the issue, but worth verifying as I can't see what else would do it.

Does the behavior happen without music playing?
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by D3thAdd3r »

also semi-related, if you want to use the 5th channel for sound effects so that they do not clobber music instruments, you can do something similar like Block Boy, TQ, etc:

Code: Select all

void BB_triggerFx(uint8_t patch){//use the 5th channel exclusively to allow music(uses 1-4) to be uninterrupted

	Track* track=&tracks[4];
	tracks[4].flags|=TRACK_FLAGS_PRIORITY;
	track->patchCommandStreamPos = NULL;
	TriggerCommon(track,patch,255,80);
	track->flags|=TRACK_FLAGS_PLAYING;

}
The caveat is that you are basically using a PCM channel to play what would normally be on a "wave channel/track", so you must do something like this in your patch struct:

Code: Select all

...
{2,waves+(256*3),patch32,0,256},//lift
{2,waves+(256*2),patch33,0,256},//throw
{2,waves+(256*3),patch34,0,256},//jump
{2,waves+(256*2),patch35,0,256},//menu
...
So it is actually using a PCM sound effect, just that you are using one of the kernel's default waves as the source of PCM. That I recall, you should not use "PC_WAVE" in any patch which you are going to use on the 5th channel, and there might be a couple extra limitations that I don't remember.

There are more complex schemes you can do with priority systems, where the most important effects don't get interrupted by less important ones...or very important ones will clobber a music patch..basically anything that would be required can be custom coded.

As far as testing I guess see if you can get sounds to be wrong without music playing, and determine if turning the music on breaks it. If so, see if you can use a different known working song(and make sure you have patches for it). Then I suppose see if you can use the song, but not through mconvert, and use the stock MIDI player and compare behaviors against those things. I just figure it is easiest to verify the music and effects themselves first, since false positives are possible with the potential pitfalls I mentioned in the previous post.
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by Jubatian »

If you are experimenting with a game which is already complex, it is possible that with some music engines a crash could happen. This is due to the interrupt hazard related to the music engine called from VSync interrupting some code doing stuff with the audio, causing the music engine to run on a jumbled state (In FoaD I removed all VSync tasks due to this, to eliminate all such hazards: before doing this, audio would randomly produce a very unpleasant burst of noise. Game logic in that takes occasionally more than the time available before VSync).
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by D3thAdd3r »

Just making a note here, I have a 20K+ MIDI file which compresses down to maybe 12K. While this is ok, the Streaming Music format was never made for flash based systems, it was designed only to use the easiest to do compression which was compatible with SD. On SD, there is no way to seek back and forth, and putting big pattern tables in flash would defeat the point. So the existing format is still quite good for that I think, with no need to change.

I went nuts and starting making a new format which is distinctly designed for flash solutions(and applicable to SID compression, more on that later). I'd add it to mconvert with an additional conversion specifier in the config file. Basically it takes an improved mconvert stream, then performs "Patternization" on it at the byte level(not MIDI event, this is universal actually). The most useful(length*frequency) 256 patterns are kept, and the entire format has been geared towards allowing up to 30 patterns to be triggered by the single byte case! The single byte case is by far the most important for average song compression. I think this could rival MOD compression actually.

I looked into "pattern of patterns" stuff, but I don't think the juice is worth the squeeze there since it increases the player size and needs extra compile flag options. Also I'm a bit lazy to actually create that compressor :roll: I'm hoping this can achieve double the compression of the Stream format(which stays intentionally pattern-less), otherwise I wouldn't do it. It creates even more nastiness inside uzeboxSoundEngine.c with now 4 different players(and I'm not kidding about SID either, so...), to the point I think there should be individual player files which get included.

I'll document the details and edit this post with a link. Just if someone comes here, they are probably looking to save space in a flash only setup, and this wont be the optimal solution anymore.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by Artcfox »

D3thAdd3r wrote: Sun Jul 09, 2023 12:09 am Just making a note here, I have a 20K+ MIDI file which compresses down to maybe 12K. While this is ok, the Streaming Music format was never made for flash based systems, it was designed only to use the easiest to do compression which was compatible with SD. On SD, there is no way to seek back and forth, and putting big pattern tables in flash would defeat the point. So the existing format is still quite good for that I think, with no need to change.

I went nuts and starting making a new format which is distinctly designed for flash solutions(and applicable to SID compression, more on that later). I'd add it to mconvert with an additional conversion specifier in the config file. Basically it takes an improved mconvert stream, then performs "Patternization" on it at the byte level(not MIDI event, this is universal actually). The most useful(length*frequency) 256 patterns are kept, and the entire format has been geared towards allowing up to 30 patterns to be triggered by the single byte case! The single byte case is by far the most important for average song compression. I think this could rival MOD compression actually.

I looked into "pattern of patterns" stuff, but I don't think the juice is worth the squeeze there since it increases the player size and needs extra compile flag options. Also I'm a bit lazy to actually create that compressor :roll: I'm hoping this can achieve double the compression of the Stream format(which stays intentionally pattern-less), otherwise I wouldn't do it. It creates even more nastiness inside uzeboxSoundEngine.c with now 4 different players(and I'm not kidding about SID either, so...), to the point I think there should be individual player files which get included.

I'll document the details and edit this post with a link. Just if someone comes here, they are probably looking to save space in a flash only setup, and this wont be the optimal solution anymore.
Woah! That sounds amazing. The song I used in my Laser Puzzle games is super huge, and has a ton of repeating patterns, so that would free up a ton of extra space.
Post Reply