Starting the preliminary planning on a port of the audio from Legend of Zelda and came to a realization it is not feasible(space wise) to do a "near perfect" port like it generally is with the tried and true NSF->MIDI->Uzebox method. Zelda uses quite a lot of PCM even for things that I wouldn't expect it to, like stepping down into secret stairs, and of course the various boss sounds. Some of this stuff, it is probably possible to make reasonable approximations by using a 2 part effect, with 1 part wave channel and the other noise channel. Some of the sounds appear to not be PCM in the Japanese version, so these could be used, but I rather find the "wrong" sounds jarring from what I knew as a kid. There is really no way you can ever replicate PCM this way and I think the sound of the original is so iconic and excellent on this title(I believe the North American version is essentially the same as all versions released in areas outside of Japan) that people have a strong concept of what it should sound like. Either a lot of work and a chance it will not sound ideal, though probably passable, or investigating alternative methods. It will probably end up the former, but I am interested in exploring the later.
My thought is this, keeping all the same channels we currently have, adding another compile flag to support PCM streamed from the SPI ram. I think it should be reasonable that the main program is responsible for setting the mode and address to where the sound starts, and also pre-scaling the volume or pitch(even if it required multiple copies in SPI ram)if required. With this method, the cycle cost should still fit in the inline mixer since it is basically an "IN" and an "OUT", and audio wise it gives a nice option for expansion and legitimate use for SPI ram in any game(I think people find making PCM effects easier, in general). For instance, you can detect SPI ram at start up and if present use the better sound. Otherwise use the wave/noise methods only. One problem with this method is that you must plan any SD card usage out appropriately as a SPI PCM sound cannot be playing when you access the SD card, you would need to wait or stop it early since some elaborate state machine for all this is so huge as to not be graceful or even possible with the inline mixer. > 90% of games now use inline mixer it seems. This introduces a small extra trouble, where if you need to cut an SPI ram sound instead of waiting it out, you would need to jump to an address that contains a smooth fade out from the last sample magnitude, as cutting it entirely when you just played a sample with a value of say 252, would be a huge audio click. I don't think any of that is so ridiculous nor takes much code.
SPI streamed MIDI seems to work out fine. Even though you can also stream music off the SD card, there is the advantage(for this and many things) that the latency and delays of the SPI ram are entirely predictable. This is not the case at all with an SD card and even something simple and easy like Alter Ego's animated Uzebox Logo I had to modify, as it had big problems on slow cards, and ran much too quickly on fast cards. For parts where you also have game logic, it becomes basically impossible to test all situation and setups to make sure it never crashes;with the ram you can. I think I will try it, any thoughts?
SPI Ram Audio Sourcing
Re: SPI Ram Audio Sourcing
I don't think this being a very good idea.
The reason is partially what you mentioned with the SD card: putting SPI accesses within the mixer would lock out the SPI bus for just about anything else for the entire duration while such a thing plays (unlike the SD loads which I integrated into Mode 74 as an option: they are only present within the video frame, so properly planned user code in VBlank could freely access SPI peripherals). The problem is that if you do this, you have to only use the SPI RAM for audio since you wouldn't be able to access it elsewhere.
Of course this is OK for a game where the SPI RAM material is planned as an optional improvement (if present) in quality, but not for something which really wants to use that RAM.
If you go on this path you could easily avoid clicks when positioning the read pointer for the RAM by iterleaving an address setup with the regular channels, and adding the SPI RAM loaded byte last. With this approach you can also play the SPI RAM material at arbitrary frequency (since if you interleave an entire address setup, that can be always present). It is 5 * 18 cycles total (Command + 3 address bytes + dummy byte to fetch the targeted byte), so it fits fine. In a ~210 cycle inline mixer you could possibly even cram two such SPI RAM channels.
Alternatively you could design the second access for a controlled load / store (so games may still access SPI RAM material slowly while playing music).
You will likely need two mixer bodies for this approach, one with the SPI RAM accesses and one without. The reason is that you have to be able to turn off SPI RAM streaming to access the SD card or the RAM itself.
The reason is partially what you mentioned with the SD card: putting SPI accesses within the mixer would lock out the SPI bus for just about anything else for the entire duration while such a thing plays (unlike the SD loads which I integrated into Mode 74 as an option: they are only present within the video frame, so properly planned user code in VBlank could freely access SPI peripherals). The problem is that if you do this, you have to only use the SPI RAM for audio since you wouldn't be able to access it elsewhere.
Of course this is OK for a game where the SPI RAM material is planned as an optional improvement (if present) in quality, but not for something which really wants to use that RAM.
If you go on this path you could easily avoid clicks when positioning the read pointer for the RAM by iterleaving an address setup with the regular channels, and adding the SPI RAM loaded byte last. With this approach you can also play the SPI RAM material at arbitrary frequency (since if you interleave an entire address setup, that can be always present). It is 5 * 18 cycles total (Command + 3 address bytes + dummy byte to fetch the targeted byte), so it fits fine. In a ~210 cycle inline mixer you could possibly even cram two such SPI RAM channels.
Alternatively you could design the second access for a controlled load / store (so games may still access SPI RAM material slowly while playing music).
You will likely need two mixer bodies for this approach, one with the SPI RAM accesses and one without. The reason is that you have to be able to turn off SPI RAM streaming to access the SD card or the RAM itself.
Re: SPI Ram Audio Sourcing
Right I agree there is the issue with SPI contention if it is to be used for other things. But simultaneously using it for multiple things, in the scenarios I imagine, puts an absolute requirement on having the SPI ram to play the game anyway. I don't think that is bad at all but some might. PCM is one of the few things I can see a standard and an enhanced version being feasible at the same time and, maybe, is so easy that people start to use the ram...better than not using it at least.
It might be better if the solution is small and generic enough, to have some sort of state machine that is a standardized way to work with the SPI ram while using it as an audio source also. With Zelda in particular I think the pressure is rather low on the SD card as it only needs to be read when a new screen is moved to and also the performance gain of SPI ram over SD seem useless there. Now using it as a sprite source could be interesting, I just don't know how feasible that is cycle wise having never experimented with such a thing.
With the constant HSYNC interrupt happening even during user code, it seems one must constantly check what state the SPI ram is in(so requiring some '644 ram usage) if you plan to use it for multiple things, where 1 is audio and the other is an operation that is expected to be completed atomically in user code. Some lower level system like you speak of could negate those issues though. It just seems there is no 1 solution that can be perfect for all...something like a "UserHsyncRoutine" sounds attractive since it actually lets you build a custom per game system that takes into consideration exactly the plan and requirements the programmer has.
It might be better if the solution is small and generic enough, to have some sort of state machine that is a standardized way to work with the SPI ram while using it as an audio source also. With Zelda in particular I think the pressure is rather low on the SD card as it only needs to be read when a new screen is moved to and also the performance gain of SPI ram over SD seem useless there. Now using it as a sprite source could be interesting, I just don't know how feasible that is cycle wise having never experimented with such a thing.
With the constant HSYNC interrupt happening even during user code, it seems one must constantly check what state the SPI ram is in(so requiring some '644 ram usage) if you plan to use it for multiple things, where 1 is audio and the other is an operation that is expected to be completed atomically in user code. Some lower level system like you speak of could negate those issues though. It just seems there is no 1 solution that can be perfect for all...something like a "UserHsyncRoutine" sounds attractive since it actually lets you build a custom per game system that takes into consideration exactly the plan and requirements the programmer has.
-
nicksen782
- Posts: 714
- Joined: Wed Feb 01, 2012 8:23 pm
- Location: Detroit, United States
- Contact:
Re: SPI Ram Audio Sourcing
For Zelda I would actually prefer to NOT use the SPI-RAM at this point. Mostly because nobody actually has it. If you can make SPI-RAM easy to add to an existing Uzebox then I'm interested or even if it gets combined somehow with UzeNet.
Also, don't the game controllers use typical shift registers? That basically is SPI. Controller port 2 could be used as a sort of RAM expansion. I like the idea of using the second controller port quite a bit actually. Additionally, it is a bus where each individual device has it's own Chip Select.
I've been working on the title screen of Zelda for now (All RAM tiles from SD!) and it looks pretty decent. I would very much like to have an SD based system of having music in the game. Normal sound effects should probably just stay in flash.
Also, don't the game controllers use typical shift registers? That basically is SPI. Controller port 2 could be used as a sort of RAM expansion. I like the idea of using the second controller port quite a bit actually. Additionally, it is a bus where each individual device has it's own Chip Select.
I've been working on the title screen of Zelda for now (All RAM tiles from SD!) and it looks pretty decent. I would very much like to have an SD based system of having music in the game. Normal sound effects should probably just stay in flash.
Re: SPI Ram Audio Sourcing
SPI ram is already part of the Uzenet 2.0 module you can get from Alec for cheap, which simply connects to the standard expansion port with no modifications(EUzebox currently requires modifications, I did it and it is easy enough). I don't know when the new "experimental" revision PCB will come out, but they have existed for maybe a couple years?, I have a few right next to me which has ESP8266 and SPI ram built in. They work perfectly for the few demos that utilize them. I think it is exactly like the SD card situation years back, as there seemed to be some opposition to making games that required SD hardware due to several home made versions not having it. In my opinion that was alleviated when Uzem got SD card support and games started to use it, as CUzebox has SPI ram support now.
It might be worth considering for Zelda anyway, as it would not require the SPI ram at all but simply use it if it is there for better sound. People running the game in CUzebox which I guess is a large portion, see the best possible version right from the start. Just an idea for discussion sake.
Even if there is theoretically an ultimate solution I will probably start using this soon as just a sound enhancement since it is probably the easiest thing to do and should be obvious that(probably in MegaBomber) the sound can be steps above what seemed possible a few years back. Mainly just lots of PCM variety is not something otherwise possible, or at least unreliable if from SD, but it can be used to good effect as you will notice if you have ever played the SNES Bomberman series. I am thinking what Jubatian says is a minimum to do, with the variable frequency playback. That adds a lot more than just PCM sound effects, since by that point you can have full blown PCM instruments sounds. Basically SNES sound versus NES/Uzebox. Probably just a modified kernel and keep it all away from the game code then, a lot easier to coordinate and it should not be too terrible to also stream music simultaneously in that case. I meant to only release games that required SPI ram a while ago but so far fell flat and snuck in a couple small games while I let it all settle in my brain I guess. The time feels right for me anyhow.
It might be worth considering for Zelda anyway, as it would not require the SPI ram at all but simply use it if it is there for better sound. People running the game in CUzebox which I guess is a large portion, see the best possible version right from the start. Just an idea for discussion sake.
Even if there is theoretically an ultimate solution I will probably start using this soon as just a sound enhancement since it is probably the easiest thing to do and should be obvious that(probably in MegaBomber) the sound can be steps above what seemed possible a few years back. Mainly just lots of PCM variety is not something otherwise possible, or at least unreliable if from SD, but it can be used to good effect as you will notice if you have ever played the SNES Bomberman series. I am thinking what Jubatian says is a minimum to do, with the variable frequency playback. That adds a lot more than just PCM sound effects, since by that point you can have full blown PCM instruments sounds. Basically SNES sound versus NES/Uzebox. Probably just a modified kernel and keep it all away from the game code then, a lot easier to coordinate and it should not be too terrible to also stream music simultaneously in that case. I meant to only release games that required SPI ram a while ago but so far fell flat and snuck in a couple small games while I let it all settle in my brain I guess. The time feels right for me anyhow.