WIP: Shadowgate

Use this forum to share and discuss Uzebox games and demos.
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: WIP: Shadowgate

Post by D3thAdd3r »

Have you determined if you are going to use the original NES graphics or remake everything with a higher color depth? That seems to be the very first step in determining what's possible. With so many waiting cycles a line buffer and pallet trickery might be squeezed in there.

I mentioned some exotic SPI ram/644 ram entangled bits concept that I would put money on being possible, maybe not practical. Another weird idea is Hold-And-Modify. If you need higher bit depth, and other tricks fail, it seems you need higher "symbol depth" so some type of compression becomes necessary.
uberlinuxguy
Posts: 86
Joined: Sun Jan 04, 2015 4:38 am

Re: WIP: Shadowgate

Post by uberlinuxguy »

D3thAdd3r wrote:Have you determined if you are going to use the original NES graphics or remake everything with a higher color depth?
I would like to use the original if possible, but I have found it hard to find anything that documents how to extract them from a .rom (with the color palette) let alone a program that runs on a modern OS to do the extraction. I think I looked into Tile Layer Pro, but had trouble understanding what I was doing and I think all the tiles came up in black/white/grayscale.
D3thAdd3r wrote:I mentioned some exotic SPI ram/644 ram entangled bits concept that I would put money on being possible, maybe not practical. Another weird idea is Hold-And-Modify. If you need higher bit depth, and other tricks fail, it seems you need higher "symbol depth" so some type of compression becomes necessary.
I liked your idea but I was trying to cut down on the SPI RAM access because it has a high clock count per byte retrieved. I didn't want to run into a problem with what I was trying to render via an SPIRAM buffer and hitting a clock count barrier in the rendering routines forcing me to start over.

I guess I am really curious how much work can be done "outside" of the rendering routines before you hit point where that too starts to effect the rendering process. (glitching and what not.) I know it's a fine balance, but I don't want to over run any thing. I find this type of pre-planning for software massively fascinating in a masochistic sort of way.
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: WIP: Shadowgate

Post by D3thAdd3r »

I found the NES memory addresses 0x0500 and 0x53D ,if you have an emulator with support for poke/peek, that control the room displayed and...the object layout/pallet? respectively. You have to enter a different room for it to update the display...and the doors are not visible since the logic seems to then work off the new (not yet drawn level) so that method isn't useful yet.

Usually very quick to find where the pallet data is in NES games, simply open up PPU viewer while running the game and check out the values of the pallet. So on the first room I checked and found they were "0f 00 17 10 0f 07 17....etc" open the rom in a hex editor and search there is exactly 1 occurence at offset 0x18331 and matches for all 16 tile color entries. Moving onto the next room I find the pallet matches exactly the 16 byte sequence starting at 0x18321. I didn't investigate further, but I'd say all your pallet data is there. Not sure if every screen uses a unique color pallet, but that sure would be easier. Otherwise there will be some byte of memory in common with different rooms that share the same pallet while warping around. It should be an offset into that table. Hopefully that is a table of sequential bytes instead of at the start of each room data. Then literally rip that table and you know what pallet every room uses. Then rip all the pallet data. Then convert NES colors to Uzebox colors, check out this, there is source code for something like that in the src/data/main.c in that zip. There was somewhere(IMO) a perfect NES to Uzebox color mapping but I don't see it in there...I will look...it must be there since how did it work otherwise? Honestly though a little trial and error and a hex editor can deconstruct formats fast usually. If you look at the code ripping graphics from CV was pretty straight forward and we already know(surprisingly) Shadowgate doesn't use compression.

Edit-used the same fast trick by viewing the nametable, basically it shows NES tile vram as numbers. Searched for that string and it looks like there is no compression on the screen tiles either, nice! It appears the address 0xa8 is the first tile indice byte of the first game screen and it is linear left to right top to bottom. Strangely there appears a duplicate right below it(is it for after you open the skull to get the key?)..so perhaps in the format there is something for tile animation or something, if nothing else you could rip everything and remove unneeded screens manually. The second screen seems to match starting at 0x250 as well so I'm pretty sure about it.

Edit2-the link I posted apparently did not do pallet conversion...I did do something that required me to painstakingly match up all NES colors to Uzebox colors into a PROGMEM table. If I lost it I will be very sad, but that part is also possible to recreate though extremely tedious. So I should say that is enough to get started ripping. Finding the sprite data will probably not be too hard either.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: WIP: Shadowgate

Post by Artcfox »

uberlinuxguy wrote:I was trying to cut down on the SPI RAM access because it has a high clock count per byte retrieved. I didn't want to run into a problem with what I was trying to render via an SPIRAM buffer and hitting a clock count barrier in the rendering routines forcing me to start over.
I'm not sure how you implemented the SPI RAM, but I have a Logic Pirate, and they did something really clever with 2 x QSPI RAM chips, to create an 8-channel logic analyzer that can read and store 8 digital signals at up to 60MHz. It's my understanding that with two chips in parallel, both operating in quad-SPI mode, once you set up the address for the read, all you need to do is provide one clock pulse, and then an entire byte of data is available. If you can dedicate an entire PORT on the ATmega644 for reading/writing to the RAM chips, then you should be able to pull off sequential reads and writes in very few clocks. The Logic Pirate is also an open source software/hardware project, so you can look at their schematics and code to see how they did it. It uses a PIC, rather than an AVR, but the concepts should be the same.

I'm not sure how feasible it would be to use some sort of tri-state buffer chip so you can hook the 2 x QSPI RAM chips directly into the DAC lines, so you wouldn't need to devote an entire PORT on the ATmega644 for that purpose, and then you should be able to stream sequential bytes to the DAC at one per clock. This is just me brainstorming out loud; I'm not sure if it would even work.
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: WIP: Shadowgate

Post by D3thAdd3r »

Forgive me for going overboard tonight, but it is rare I get to use my(most of the time quite useless, and certainly worthless for party conversation :lol: ) "rom hacking" tricks acquired at great waste of time. I really want to see this game take off and I enjoy this type of thing actually; also catching up to your knowledge on SPI ram+emulation things(which I now need for Uzenet it turns out) isn't going to happen so there is an ulterior motive here to your success ;)

Comparing PPU tile data to the rom in TLP, it looks like they directly drop linearly what you see straight into tile ram which is super ideal. They even duplicate the hand cursor graphics for each one set you can see, approximately 50 times I count(so are there 50 unique screens?) in TLP. If that holds true for all screens, which I suspect, ripping the graphics doesn't require much insight into how the code works.

NES tile data is a little weird but simple as described here. The offsets to the graphics and sprites are easy to see from TLP. Read in 16 planar bytes, lookup the appropriate pallet index for each 2 bit pixel, use that index into the NES->Uzebox lookup table and spit that byte out as a color in whatever format is needed per tile. If you were going to do an entire bitmap of screens it could be composited 8x8 pixels at a time too or any other format you might use.

I will try and see if I can rip the first screen and tiles into PROGMEM and display it in mode 3. If so, it should work the same for all of them and I will recreate the NES->Uzebox color lookup table I can't seem to find anywhere.
uberlinuxguy
Posts: 86
Joined: Sun Jan 04, 2015 4:38 am

Re: WIP: Shadowgate

Post by uberlinuxguy »

Artcfox wrote: I'm not sure how feasible it would be to use some sort of tri-state buffer chip so you can hook the 2 x QSPI RAM chips directly into the DAC lines, so you wouldn't need to devote an entire PORT on the ATmega644 for that purpose, and then you should be able to stream sequential bytes to the DAC at one per clock. This is just me brainstorming out loud; I'm not sure if it would even work.
Well, the DAC seems to have lines for RGB, so you would need to clock out the Red Green and Blue bytes (I think they are bytes, didn't look at the data sheet) separately to each line. But an interesting thought.
uberlinuxguy
Posts: 86
Joined: Sun Jan 04, 2015 4:38 am

Re: WIP: Shadowgate

Post by uberlinuxguy »

D3thAdd3r wrote: I will try and see if I can rip the first screen and tiles into PROGMEM and display it in mode 3. If so, it should work the same for all of them and I will recreate the NES->Uzebox color lookup table I can't seem to find anywhere.
When you say "screen" do you mean the whole thing including the inventory and bottom box as well as the room picture? I guess I need to read through your posts and make sure I understand what is going on. More coffee is need.

The ripping part may be easier than I thought.

You realize the next question after graphics is how to rip the audio. :-)
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: WIP: Shadowgate

Post by D3thAdd3r »

The graphics that are the same on each screen are in there somewhere(menu border, etc). By screen I mean the 14x14 tile window that is unique to each room or whatever you want to call it. It's stored sequentially in the rom starting at 0xA8, so you read 14x14 bytes from there and that is the map. Right after that is the attribute table(very luckily!). I didn't realize it was right there and resorted to debugging memory writes to it. After the attribute bytes starts the next screen tiles and so on. The attribute bytes are the tricky part, but, knowing the pallet bytes and the attribute bytes you can get the exact colors for every screen. Basically the 16 planar bytes per tile specify the lower 2 bits of color per pixel for a tile(tile or sprite format is the same there), then the attribute byte uses different bits for different tiles and specifies 2 more bits for a 4x4 section of tiles on screen. Using the 2+2 bits you have the lookup into the current pallet, which seem to be stored sequentially per level followed by the sprite pallet bytes. That sounds a little contorted, and it is. NES attribute bytes are pretty crazy and I'd have to read up on them again to give more detail into that. That is the hard part of implementing the full rip. Ripping the sprites is much easier because sprites never use attribute bytes. However, I have not tried finding the "frame maps" yet that specify which sprites tiles go where for an object....that paragraph sounds crazy I know, but it is the gist of it.

This is a big rom, just the screen maps for 50 levels is about 10k. Each screen looks to have 128 tiles with about 10 of those being the same between them so perhaps ~118*64=7552 bytes of tile graphics if using 8bpp, per screen or ~7552*50 = 377,600 for all 50!. The font is the same between them, but it's clear the SD card has to be involved somehow. I have absolutely no clue what it means for the video mode, but it is.

I did find where the text is and it looks like it's all sequential tables but it is compressed somehow. It looks to be using 2 separate tables at 0x16010 and 16210, but I don't understand the format. Modifying either of those bytes will make the first word on the first screen warped, and changing by just 1 makes a big difference more than just a letter, even causes the window to close instantly or long strings of random letters. That sounds like something to do last since it will probably require debug stepping to figure it out.
uberlinuxguy wrote:You realize the next question after graphics is how to rip the audio.
This part should be way easier. The NSF
ShadowgateNSF.zip
(13.17 KiB) Downloaded 628 times
has all the music and effects and the standard way should work just fine! Basically this and this method. There are some noise channel effects that would have to be done manually but not too bad.
uberlinuxguy wrote:Well, the DAC seems to have lines for RGB, so you would need to clock out the Red Green and Blue bytes (I think they are bytes, didn't look at the data sheet) separately to each line. But an interesting thought.
The concept is interesting, I don't know enough about hardware to really get how it would work. An 8 bit address (dual port?) ram chip in front of the DAC would be interesting too. Set it to write when you want to change pallet entries during game logic, set it to read for hardware pallet look up output. 2bpp,4bpp,etc without bit twiddling by arranging the pallet right.
User avatar
D3thAdd3r
Posts: 3221
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: WIP: Shadowgate

Post by D3thAdd3r »

A quick demonstration to build motivation, 3 songs ripped from the NSF, press Up and Down to change songs. The first one I like and the 2nd and 3rd need some work, but there is definitely ways to get you the resources for this one. There are 13 songs and 13 sound effects used in the game. Most songs are < 25 seconds, some are much shorter as well, so it could be chipped away at until you had all of them. I will say, Shadowgate has some great music and it makes me want to play through it.
Attachments
SHADOWGT.UZE
(32.8 KiB) Downloaded 927 times
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: WIP: Shadowgate

Post by Artcfox »

uberlinuxguy wrote:
Artcfox wrote: I'm not sure how feasible it would be to use some sort of tri-state buffer chip so you can hook the 2 x QSPI RAM chips directly into the DAC lines, so you wouldn't need to devote an entire PORT on the ATmega644 for that purpose, and then you should be able to stream sequential bytes to the DAC at one per clock. This is just me brainstorming out loud; I'm not sure if it would even work.
Well, the DAC seems to have lines for RGB, so you would need to clock out the Red Green and Blue bytes (I think they are bytes, didn't look at the data sheet) separately to each line. But an interesting thought.
I thought that the DAC was already hooked up to a PORT on the ATmega644, and writing a single byte outputs 3 red bits, 3 green bits, and 2 blue bits to the R2R DAC, so once you set up the address you want to stream from on the SRAM, you should be able to write out an entire pixel (all colors) just by pulsing its clock line.
Post Reply