Streaming Music(SD, SPI ram, Network, etc. source)

Topics related to the API, programming discussions & questions, coding tips, bugs, etc. should go here.
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by D3thAdd3r »

I have been too brain dead for quite a while to put any effort towards this; just worn out. I got at it today, and implemented what seems to be a working "Streaming Flash Music Player". Perhaps it is best called "Compressed Music" but I guess "Streaming Music" has sort of stuck by now, and gives a bit more indication to it's general design. Like I have said before, I did not design this to be optimal for flash based playback which is good at random seeks(like MOD makes excellent use of), more, everything was designed to minimize the worst case scenario that might occur during streaming a song(where many random seeks is simply not feasible), versus the stock MIDI player format. It just so happens, that in all cases I have seen, this format comes out significantly smaller than the stock flash player. It would be possible to develop better compression in the flash case, but MOD has already done that. This should be easy for people to use, since most everyone is using MIDI and songs are generally available in that format(I have not successfully even created a MOD song yet).

So I really hope people will not be to shy to try it in their makefile, to trigger mconvert to do all the things necessary. Essentially, if you are willing to follow the wiki tutorial and build a simple configuration file for your music(and integrate your music conversion into the makefile for easier development). You should be able to save ~30% on flash space, versus anything you were doing with the standard MIDI. I have fixed up the tutorial here, and if anything doesn't make sense please let me know. I have stepped through the tutorial to build a flash based demo(which is probably redundant to include in the main branch), and that I can see the tutorial is accurate as written. As another nice side benefit, there should be a nice ram savings over the stock MIDI player(do not need loopEnd, nor songStart, so 4 bytes). Not a ton of testing has been done, but I have successfully converted a song to the compressed format, and simply included it in my project. Then the code to play it is....:

Code: Select all

StartSong(SongName);
That is it, just works like stock without all the complexities in the buffered version :D The buffered version remains the same, except that you must now specify:

Code: Select all

-DSTREAM_MUSIC_RAM
to specify that you will be using the buffering code(otherwise it will assume flash based). I attempted to do this all as cleanly as possible, considering we now have 3 separate music engines(MIDI,MOD,Streaming ram with it's 2 submodes). So just a brief wait before release here because I want it to be done right and not need fixing...I am finalizing it at the moment and consulting with Jubatian, as he is keen to combine this in the main branch along with his nice improvements to the mixer which should really add up to some nice gains for future games. Stay tuned!
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by nicksen782 »

I've integrated it fully into Bubble Bobble at least for the background music. I have well tested method of handling everything. I made no changes to the music kernel.

I could share with you what I have Lee.
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by D3thAdd3r »

Yes that would be great and perhaps there are some things that can be updated in the SPIRamMusicDemo. There should be no changes to the ram based player from when you started using it, except to add the -DSTREAM_MUSIC_RAM to your makefile(this just keeps uzeboxSoundEngine.c more clean I think).

I guess a consideration I haven't put as much thought into, is what is an appropriate demonstration of this. SPIRamMusicDemo IMO shows basically what streaming music is, and probably there needs be no demo just to show flash based streaming music, which works exactly the same as normal MIDI; except the extra conversion step(so basically reading the mconert wiki page is the only difference for the developer). I never did make any real progress towards the direct SD streaming method, but I still think it is doable to a point(at least better than the initial demo I put out). Probably I would not even include that existing demo in the main branch, as really it should handle the SD card in a less blocking way before it is a decent example. That seems more an exercise in building a good state machine for SD handling, but something that should probably get done in the future.
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by nicksen782 »

In my implementation there are routine checks of the buffer state and it gets filled if it needs to be. The fill commands are SPIRAM commands but they could just as easily be SD Card commands. I never bothered checking. I don't think anything would actually stop this from working. The problem you may face is keeping the position on the SD file. SPIRAM can be read from anywhere. My filesystem is a simple as offsets into SPIRAM. A system that you are already using for the SPIRAM streaming. It works.

One of the last things I got completed for Bubble Bobble was a proper music test. It works. You can turn on/off streaming, repeat. Play and stop. Choose a song. It works. It also allowed me to stress test the music engine. I used to be able to crash it by rapidly starting songs.

I would be happy to create a music test program. At the very least, I would like to offer additional thoughts on implementation.

I've been thinking too that you should be able to change the SPIRAM data with new SD data (songs) as long as you have correct offsets.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by Artcfox »

Sweet! With all these savings, maybe this should be the default flash music player once it has had some more use?
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by D3thAdd3r »

Artcfox wrote: Mon Jan 15, 2018 12:19 am With all these savings, maybe this should be the default flash music player once it has had some more use?
It could be feasible, and there is probably some good points to doing so once things are proven a bit more. All the source code I have seen, uses prebuilt MIDI and the only program I have seen build the MIDI during compilation is SPIRamMusicDemo. So in theory if streaming music was default right now, and a specific flag to override a default of streaming was added to each makefile for existing programs(or a config file built to convert it to compressed/streaming), all existing programs should build without a hitch.

The main question in my mind, is how terrible do people find the burden of having to use mconvert? The thought I had since the beginning is to let midiconv do what it already does well, and not add a whole other thing to that. On the other side of the coin, not replicating already proven behavior that midiconv does within mconvert. So this idea(and realistically I don't think there is a fundamentally better alternative) mandates the 2 step midiconv->mconvert process. The only way I originally saw to get away from it, would be to bring mconvert into midiconv, which seems obvious to me as a bad idea. I could certainly update the music tutorials to reflect the change to the 2 step process, if and when it ever would happen. Music is already a fairly in depth process, so probably setting up an extra config isn't too bad. I suppose, mconvert could actually trigger midiconv(adding extra arguments in mconvert config to feed to midiconv), eliminating the makefile calls to midiconv followed by the call to mconvert. That is about all the ideas I have on it, so I am not too sure on any of that yet; but something important to think about I believe.

Long story short, I very slightly lean towards leaving stock MIDI the default in the future. Simply because of the extra config, at which points adding an extra flag to specify compressed/streaming would be the easy part. Not steadfast on any of that, whatever logically works out I am all for.
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by nicksen782 »

I use midiconvert and then mconvert. I do this via a script that is called prior to the actual game compiling. It is really quite automatic once you get things going.

What is the flash/ram cost of the stock music player? Would there really be a benefit of removing it? I would not think there would be much benefit.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by Artcfox »

Using both programs is probably a good idea, because it allows people to learn in stages. Okay you got normal midi working, now if you need extra space, you can put in extra effort on top of something that you already got working to get more savings. Or if normal midi is fine then you don't have to.
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by D3thAdd3r »

@nicksen782 I do not know the flash cost, but I would expect it is pretty similar to stock for the flash streaming version(uses less ram than stock at least). The ram based streaming is probably larger than stock. I will have to try some tests to get an idea of the cost differences there.

I don't think getting rid of the standard/legacy player is an option. Though it's existence costs no extra space if you are using the streaming player (and vice versa), since it is not compiled in that case due to #if structures. The only benefit to removing anything there is to make the source code less crazy looking to someone reading it. The only candidate for removal might be the MIDI_IN thing, but maybe I will leave someone else to make that call.

I suck at github BTW so I am shy to F something up again. I basically have the latest kernel with streaming music combined locally. If someone confident in their github skills could help me merge it to master I would appreciate it. Jubatian said the best approach is to first merge streaming, then he can merge his mixer improvements on top so that is what I need. I am convinced it is working and done and the number of files affected is minimal.
User avatar
Jubatian
Posts: 1561
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Streaming Music(SD, SPI ram, Network, etc. source)

Post by Jubatian »

D3thAdd3r wrote: Wed Jan 17, 2018 7:57 pmI suck at github BTW so I am shy to F something up again. I basically have the latest kernel with streaming music combined locally. If someone confident in their github skills could help me merge it to master I would appreciate it. Jubatian said the best approach is to first merge streaming, then he can merge his mixer improvements on top so that is what I need. I am convinced it is working and done and the number of files affected is minimal.
I tried a merge, it seems to be doable and functional. The only place I am getting conflicts is SPIRamMusicDemo, which was possibly an accidental commit from an earlier state, I dunno. It also generates an SD_MUSIC.DAT file which is directly added on master.

So you should check this demo (and maybe SDMusicDemo as well), and decide upon it. Maybe the best would be ignoring them and using what the streaming-music branch has. You may also make an attempt at using the bootloader API library instead of SdBase for the SPI RAM music demo, it shouldn't be difficult to do so.
Post Reply