Weber's Rants (tutorials)

Topics related to the API, programming discussions & questions, coding tips, bugs, etc. should go here.
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Weber's Rants (tutorials)

Post by Jubatian »

I just noticed the origin of Toorum's Quest (here). Impressive stuff, and it got a bit weird, that the author (Petri Häkkinen) states he was inspired by Uzebox for his own design, creates an own game console, and now from that "derivative" it is ported back to Uzebox.

The graphics mode of the original as described is a quite impressive monster. Something I also proposed here: double scanning. He does it at 6 cycles / pixel (on his much slower setup, that's how it ends up with 104 pixels horizontally), and achieves a Mode 2 like end result (true sprites). With Uzebox it is perfectly possible to go for 9 cycles per pixel with the same principle, which would result square pixels (and on CunningFellow's micro handheld it could achieve a clean 160x112 resolution if he went for the without CPLD path).

I like the original's music more. It even has a very slight Gordian Tomb feel (the C64 version of course), anyway, I think it fits better for the cave exploration theme.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Weber's Rants (tutorials)

Post by D3thAdd3r »

Yes it is kind of crazy and random how it started. It is an homage to Petri's project basically.

Ideally I would like to use different music before calling it done. It proved to be a difficult task to port the sound engine over which would have allowed the use the original music and sound effects. Doing so requires the vsync mixer which comes at cost of ram tiles already at the edge of potential flickering (cant happen currently). To avoid all that, I ideally would have made a custom low res video mode and leveraged the savings to sound.

All decisions were based around quick development time to be honest. If more games existed for the platform, it is possible to port video and sound at a lower level. For scanline doubling and lower res I think Petri's work really shows that you can have an engaging game at that kind of resolution. On Uzebox even more so it would seem those resources could then be used somewhere else. This game in particular seems most efficient in something mode 2 like, but maybe a mode 3 like with the specs you mention could prove to fill a hole in video modes. Like mode 74 a mixture of paradigms could be powerful.
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Weber's Rants (tutorials)

Post by Jubatian »

Could the 2000 cycle deal help there? It isn't much in comparison to a VSync mixer's demands, but still something to get closer to the goal. Another thing may be hacking around a bit to get more time for the inline mixer, allowing more complex sound generation there.

So it seems like there is some interest in a doublescan mode then! Should I start developing it? Initial goals would be 4 bit colors to get demands even lower and provide nice palette effects (this has uses beyond fading, stuff I discovered while working on the UCC dragon game, again something unseen in Uzebox games so far), H & V scrolling capabilities, maximal resolution of 144x116 or maybe 160x116, attempting to retain the inline mixer (interleaving sprite generation with odd lines).

I just had an other game idea in mind, still undecided, something I talked about in PMs. So 6cy/pixel 2bpp monochrome-ish, either by SPI RAM or normal Uzebox (two entirely different concepts towards different look and feel). For computer RPG stuff. But the double scanned mode with "hardware" sprites may also be a great thing for excessive SD card use (since it doesn't need RAM tiles taking away half the RAM, if it retains the inline mixer, that means even less RAM missing in action; the 256b palette buffer necessary for 4bpp can also be shared with the main stack using Mode 74's reset concept).
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Weber's Rants (tutorials)

Post by D3thAdd3r »

I think it would be worth developing. Tile size would be an interesting issue, perhaps they should be 8x8 or perhaps if cycles allow smaller than that. Either way there is something that can be done with it for original games, though probably not a good video mode for porting existing NES like games more like a re-imagintion of them.

A ram tiles approach should actually scale better than it does at higher resolutions where now the same amount of ram fill much larger screen real estate. Just that simple fact, in my mind, would be extremely powerful for sprites, scrolling games, etc. With the line doubling and the extra cycles it buy at expense of vertical resolution, I don't know what should be done with those extra cycles. More sound channels I guess!

As we discussed you already have several methods basically thought out to different ends, it is almost dizzying possibilities. If you are keen to an RPG, and I'd love to see an Uzebox RPG, I suppose necessity is the mother of invention.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Weber's Rants (tutorials)

Post by D3thAdd3r »

I make occasional but steady progress on ESP emulation and I am thus far pretty happy with how it is going finally. A little update is in order as I have went a very different direction that the previous extremely slow version I posted somewhere on the forums. Essentially I came to the conclusion that the core speed requirement of the previous method was too high and perhaps not within range of any foreseeable consumer hardware even when optimized. I went threaded and it was much more difficult and error prone than I anticipated.

The basic idea is to hopefully force the OS to use 2 cores, and then make sure threads do not have to stall all the time by having separate Tx/Rx buffers for each device. Uzebox writes to it's Tx whenever it wants, and when requested by the ESP thread, dumps this buffer to the ESP Rx thread(which when it placed a request, will not touch the buffer until Uzebox unsets the request flag). It works the same way in the other direction basically like this:
espflow.jpg
espflow.jpg (50.87 KiB) Viewed 4036 times
This takes advantage of the fact that testing shows you cannot make safe code on the Uzebox that relies on any cycle precise response from the module, especially the ones where pushing timing is important:network transfers. So this is a loosely coupled scheme to match how the real hardware acts, unpredictable within a window, where the ESP clock is entirely driven by the Uzebox thread yet it runs free up to a "decoherence limit". Basically it is a counter the ESP constantly sets to 0, and when it is above an arbitrary threshold the Uzebox thread will stop until it catches up so that any level of cycle accuracy is possible(though naturally slower the higher). Importantly it implements the concept of one or the other device not listening for a time and that data being lost. In my previous attempts with naive buffers, if the Uzebox turned off UART for 20 seconds then on, the first byte it would read would be from 20 seconds ago. In reality it would have been a byte long since overwritten; this system emulates all such UART things correctly and as said before has arbitrary precise response times. When the module does make a response, the entire buffer is sent to Uzebox(which requests such whenever it's Rx is empty) and this allows Uzebox to realistically receive bytes at full baud speed(even though the emulated module has long since moved on to other things since it send that payload).

On my crappy 1.6ghz AMD it runs about 14 times faster than my previous single threaded version which had a much lower resolution for UART response. Still only about 14mhz based on the old 1.3 codebase in this case, but as long as the OS uses multiple cores a 3ghz i5 or so should run it fine off the much faster 2.0 code. I ended up cooking my good mobo to death, so I will have to test that theory later. The concern is if the threads will be able to communicate with ~28mhz full speed, and still be able to achieve within 1000 emulated Uzebox cycles. I don't know if it scales up to be true yet. If not then I have no idea how it can be done as I've tried every thing I can make up at this point.

Ran in to lots of problems I never had to deal with before. Race conditions really broke my brain for a while, I never did any real multithreaded program that required any strict synchronization. In the end memory fences on the buffer lock flags(force no reordering of instruction so that flag is not unset before payload is transferred) and volatile flags/buffers were required. The bad thing, there is really no way to escape the use of volatile buffers with this scheme and that is probably the biggest performance problem. The other issue is forcing the OS to use separate cores which as far as I know cannot be done except in .NET. Anyone that knows more about such subjects I'd love to hear from you.

I attempted to do yield() in various location when it does end up locked on a flag, like when the threads are too far out of sync, but it actually decreased performance. I am starting to believe that never yielding makes the OS more likely to use multiple cores. If the OS does not and it is on 1 core, then I am fairly certain the very coarse thread timing even if 1000 times a second makes high performance impossible(and so in that case better single threaded). I so far can't see anyway it could run fast enough without the use of multiple cores unless a very high amount of Uzebox cycles pass before the module updates are performed. That makes the resolution of timing much lower than the real thing, you can realistically expect an answer to "AT\r\n" multiple times a frame in reality. The module thread itself is actually quite a low burden. It is much more important for the Uzebox thread to go faster, since the module thread rarely actually processes any data(because rarely is the data terminated yet), and instead spend most of it's time in a loop looking for received internet data. It would be fine to do this updates less often since it is impossible to predict when internet data will come and the real hardware will be no where near as responsive(getting the packet data out to UART) at receiving as the emulated version is. It feels like the real things swings 10's of milliseconds even with test program you know send packets every millisecond, unless I am misunderstanding test results.

Anyway I am cautiously optimistic it will work out in the end. I might need some help with the Linux side of things as I look into implementing the cross platform aspect of network functions that are already contained in wrapper functions. Sockets programming is generally easy and very similar across platforms at least.

PS. does anyone know anything about real time threads? I wonder if it is possible to force 2 threads real time and if the OS then decides to always use 2 cores. Perhaps that doesn't work well with 2 core systems but 4+ cores machines seem really common at this point.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Weber's Rants (tutorials)

Post by D3thAdd3r »

Haven't talked about this one for a long time because I haven't touched it until recently. There are a few minor things to yet be done, a ladder physics bug, and possibly some level editor issues. The level editor is much better, that is the update here really, as it has the functionality to build the original episode or any possible new one right now(just needs to save to SD!). Instructions will appear, depending on which mode you are in(push select) to describe all editor functions at the bottom of the screen(wait a bit for all messages to cycle). I haven't had the time yet to build and try Jubatian's CUzebox emulator yet. CUzebox is really what got me back interested in the game, since it has the functionality for SD writing the game relies on, where I couldn't really release it until there was emulator support..just a broken feeling game otherwise. I will try it on CUzebox when I get a chance, just wanted to throw an updated version on this so people can check out the level editor or perhaps Jubatian will see something with the writing that makes more sense to him than me. I cannot see anything I am doing wrong according to PFF documentation on writes, it is supposedly dead simple :?

Also assuming this SD writing ever gets sorted out, if anyone takes the time to craft a quality episode or 2 with the editor, I would include it in the official release with credits on the intro screen. I will likely create 1 or 2 myself so that the game has more than the slightly short original episode. Else there was not much use in the drastic and frustrating changes to make everything run off the SD card to begin with. Though, I am very tempted to graft this into Alter Ego when done, since user levels could be fun on that as well(I know there is code space for it now).
Attachments
TOORUMSQ.zip
(83.24 KiB) Downloaded 215 times
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Weber's Rants (tutorials)

Post by Artcfox »

D3thAdd3r wrote:I haven't had the time yet to build and try Jubatian's CUzebox emulator yet.
Do you need a Linux or Windows build? I attached the Windows build I sent CunningFellow earlier tonight. This build uses Uzem's keymapping for A, B, SELECT, and START, and by default the debug display is off (hit F3 to toggle).
Attachments
cuzebox-win32.zip
Win32 version, cross-compiled on Linux
(415.77 KiB) Downloaded 228 times
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Weber's Rants (tutorials)

Post by D3thAdd3r »

Thanks very much!
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Weber's Rants (tutorials)

Post by Artcfox »

D3thAdd3r wrote:Thanks very much!
No problem! I'm trying to play through it now.
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Weber's Rants (tutorials)

Post by Jubatian »

I fixed bugs within the SD write support. Now it should be functional.

I think this Toorum's Quest is buggy though, it definitely writes something on the SD, but it doesn't affect the high score table (not even on the real hardware). I verified the emulator's operation by comparing it to what the game did on the real thing, now it seems to be consistently doing the same, somewhere into that _HISCORE.DAT it writes out the new score (which now behaves the same manner now on both the emulator and the real thing).

I attached a Win32 build along with it, cross compiled from Linux, tested in Wine (you will need an SDL2.dll to start it).
Attachments
cuzebox-win32.zip
SD writes fixed, Cuzebox Win32 build
(28.66 KiB) Downloaded 226 times
Post Reply