Smoother rendering of high res modes

The Uzebox now have a fully functional emulator! Download and discuss it here.
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Smoother rendering of high res modes

Post by Jubatian »

OK, I checked it, just a few notes on the GitHub comments (I commented it there of course, just notes).

The case of the SingleStep variable maybe should be revised, but I simply couldn't find any place where it was set, so associated code could become effective. If it is meant to be set by the debugger itself (and not anywhere in the emulator's code), it is missing a "volatile" qualifier.

There are some "murky" SDL behaviors which may need some further exploration. Such as I would be interested whether fullscreen works at all anywhere (on my PC it doesn't: it just changes the resolution (!!!), the code runs, sound is there, but nothing renders, and at exit SDL forgets to revert the resolution even though the window is destroyed). The mouse's behavior on fullscreen also feels odd (the code which I dropped since it creates an interdependence which hinders modularizing the renderer proper).

Currently there is a screenshot interface there, but I rather plan to add a line request interface function, to build both the capture and the screenshot function on that. While screenshot is small, I think I will also put that in its own module, to be clean. What needs consideration is that I wish to make the interface not dependent on SDL, making coupling less tight among components (so notably the capture / screenshot feature may not depend on SDL at all).
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Smoother rendering of high res modes

Post by Artcfox »

Jubatian wrote:OK, I checked it, just a few notes on the GitHub comments (I commented it there of course, just notes).

The case of the SingleStep variable maybe should be revised, but I simply couldn't find any place where it was set, so associated code could become effective. If it is meant to be set by the debugger itself (and not anywhere in the emulator's code), it is missing a "volatile" qualifier.

There are some "murky" SDL behaviors which may need some further exploration. Such as I would be interested whether fullscreen works at all anywhere (on my PC it doesn't: it just changes the resolution (!!!), the code runs, sound is there, but nothing renders, and at exit SDL forgets to revert the resolution even though the window is destroyed). The mouse's behavior on fullscreen also feels odd (the code which I dropped since it creates an interdependence which hinders modularizing the renderer proper).

Currently there is a screenshot interface there, but I rather plan to add a line request interface function, to build both the capture and the screenshot function on that. While screenshot is small, I think I will also put that in its own module, to be clean. What needs consideration is that I wish to make the interface not dependent on SDL, making coupling less tight among components (so notably the capture / screenshot feature may not depend on SDL at all).
I replied to your comments on Github. I tested your code, and fullscreen works properly, though it doesn't seem to change the resolution for me, it seems to keep my current resolution and scale things up, but it did go fullscreen properly, so I think that something might be weird with your instance.

As far as the mouse warping, maybe add a mouseEvent to your renderer that only gets called if the mouse is moved, and inside there you could check to see if the renderer is fullscreen and then warp it to the center.

The screenshot interface I could live without, since technically when rendering directly to a texture's back buffer the pointer is "write-only" and may not read back the same. With the ability to record a video, is screenshotting really necessary? It would also appear that with SDL2, the OS overrides the PrintScreen key, so screenshotting doesn't actually work anyway (at least on my computer it didn't).

How do you want to collaborate on this, since Github isn't letting me fork your fork?

Edit: I also fixed a bunch of bugs, and fixed most of the warnings in the code. I'm just waiting for Uze to merge in those pull-requests, and then you can sync with the upstream and you'll have those changes too.
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Smoother rendering of high res modes

Post by Jubatian »

Artcfox wrote:I tested your code, and fullscreen works properly, though it doesn't seem to change the resolution for me, it seems to keep my current resolution and scale things up, but it did go fullscreen properly, so I think that something might be weird with your instance.
What do you have there? (Windows? Linux? If the latter, which distribution?) At me it is Debian Jessie, no extras which would interfere here, WindowMaker as window manager, DRI working properly as far as qlxinfo etc. reports.
Artcfox wrote:As far as the mouse warping, maybe add a mouseEvent to your renderer that only gets called if the mouse is moved, and inside there you could check to see if the renderer is fullscreen and then warp it to the center.
It would be nice if it could be solved all right within the renderer, but right now it seems impossible this way since you can not sniff the event stream reliably. You could with PeepEvents without interfering with other components, but that wouldn't be too reliable (since other events might arrive after this call, before the input part consumes all of them, which events would be missed then). Maybe some mechanism on the interface is necessary to allow the renderer for shaping individual input events if necessary.

The screenshot feature doesn't need to be discarded. A proper capture interface would suit all right for a screenshot interface as well (after all it's just single frame versus multiple).

In GitHub, when sending pull requests, you can do it against any fork of the repo, so you can send one for me just as well as for Uze. However I will try to merge the mainline and fix these up in some manner before that, hopefully to tomorrow it will be done. Then you should pull the then-current state of my linebuffer fork, pimp up your part accordingly, and send the pull against my fork then which I will merge in. (For line endings I will try to not forget cleaning it up so already existing files would have consistent Windows line ends, new files Unix line ends, a sane editor should not have problem with that).

Bunch of bugs fixed - good :) So hopefully less warnings, less UB (Undefined Behavior), more consistent compiles. I only skimmed it through, but did I see that right that the SD part was faulty for some UB?
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Smoother rendering of high res modes

Post by Artcfox »

Jubatian wrote:
Artcfox wrote:I tested your code, and fullscreen works properly, though it doesn't seem to change the resolution for me, it seems to keep my current resolution and scale things up, but it did go fullscreen properly, so I think that something might be weird with your instance.
What do you have there? (Windows? Linux? If the latter, which distribution?) At me it is Debian Jessie, no extras which would interfere here, WindowMaker as window manager, DRI working properly as far as qlxinfo etc. reports.
Debian Wheezy, and Debian Jessie.
Jubatian wrote:
Artcfox wrote:As far as the mouse warping, maybe add a mouseEvent to your renderer that only gets called if the mouse is moved, and inside there you could check to see if the renderer is fullscreen and then warp it to the center.
It would be nice if it could be solved all right within the renderer, but right now it seems impossible this way since you can not sniff the event stream reliably. You could with PeepEvents without interfering with other components, but that wouldn't be too reliable (since other events might arrive after this call, before the input part consumes all of them, which events would be missed then). Maybe some mechanism on the interface is necessary to allow the renderer for shaping individual input events if necessary.
What's wrong with changing:

Code: Select all

if (fullscreen)
{
    SDL_WarpMouseInWindow(window,400,300);
    SDL_GetRelativeMouseState(&mouse_dx,&mouse_dy);
}
to:

Code: Select all

o_renderer->mouseEvent()
and then inside the renderer's mouseEvent it will know if it's fullscreen, and can warp the mouse to the middle.
Jubatian wrote:The screenshot feature doesn't need to be discarded. A proper capture interface would suit all right for a screenshot interface as well (after all it's just single frame versus multiple).
It seems a waste to assemble and buffer every scanline just in case someone is going to take a screenshot if you're writing directly to the texture as each scanline comes in?
Jubatian wrote:In GitHub, when sending pull requests, you can do it against any fork of the repo, so you can send one for me just as well as for Uze. However I will try to merge the mainline and fix these up in some manner before that, hopefully to tomorrow it will be done. Then you should pull the then-current state of my linebuffer fork, pimp up your part accordingly, and send the pull against my fork then which I will merge in. (For line endings I will try to not forget cleaning it up so already existing files would have consistent Windows line ends, new files Unix line ends, a sane editor should not have problem with that).
Hmm... I'll take a look tomorrow.
Jubatian wrote:Bunch of bugs fixed - good :) So hopefully less warnings, less UB (Undefined Behavior), more consistent compiles. I only skimmed it through, but did I see that right that the SD part was faulty for some UB?
Yes. Unfortunately the Clang that Debian packaged doesn't include any of the extras because the maintainer didn't compile it using CMake, he used the autotools version, which doesn't include everything. What's missing is the Undefined Behavior Sanitizer, but I compiled the newest Clang/LLVM from source code in the hopes that I'll be able to use them to find and fix all of the undefined behaviors.
User avatar
Jubatian
Posts: 1564
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: Smoother rendering of high res modes

Post by Jubatian »

Debian Jessie... Ouch... Then something is broken here. Maybe the fact that this mess survived two distro upgrades already (Squeeze -> Wheezy -> Jessie) with all the configuration cruft piled up and failed to be collected / cleaned up during the years (package manager state is clean, but you poke around in /etc once a while, and the little hacks only stack). Eh, video card by the way?

Adding a mouseEvent() interface function is very specific to this particular problem. I would (and will) rather add a generic event filter which should be called on the top of the event processor, so the renderer may react to events. Maybe this would also be a good starting point for cleaning up the mess in the event processing, so for example the renderer may then catch renderer-specific input events there (but that may also bite back from interface design perspective, since you can not inject settings easily in other manners then, such as by command-line options).

You don't need to buffer every scanline, you can extract those when they are needed (when the function requesting a rendered line is called). It is necessary for a better capture feature too, which may not be 60FPS (the current capture feature looks a bit silly in this regard, 60FPS, but sloppy visual quality), but rather something more common (like 30FPS for standard NTSC video). Anyway, no matter how the interface is designed, it is necessary to have a capability to extract the rendered result (from the video hardware) so it can be recorded, since for example considering some nice NTSC filtered output, if the user "presses" record (or asks it on command line), he would expect the captured video also being filtered like that.

It is no problem if video recording is slower (or it is notably slower with specific renderers), the primary goal is to have the emulator normally running all right.

With UB be careful though, I suspect for example such in the instruction decoder, which if present, currently rely on the compiler behaving according to 2's complement arithmetic. These for example are not necessarily trivial to fix so the result is identical. So in any case you experience integer related UB, consider what the code would do with 2's complement arithmetic, and see whether that result makes any sense. If it does, then the fix is likely reproducing the exact same behavior with unsigned math.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Smoother rendering of high res modes

Post by Artcfox »

On board Intel graphics.

The video capture as it stands (with my latest patch) grabs all 1440x224 pixels perfectly, at exactly 59.94 fps, which is what the NTSC standard is for progressive. The scaling (and optional filtering) is all done by ffmpeg, but it actually gets passed every pixel. The upscaling to 720p is what allows YouTube to play the videos back at 59.94 fps, exactly matching the frame rate of the real hardware.

As far as extracting the pixels from the rendering hardware after the fact, if rendering to a texture, the pointer to the texture is write-only and not guaranteed to read back the same, so unless we buffer scanlines ourselves separately (or first render to a surface, and then copy it to a texture), then we do not have a reliable way to grab an entire frame at once to save as a screenshot.

As far as recording videos with effects applied, you would probably want to have a way to intercept and change the scanlines before they are being passed to the ffmpeg pipe.

Unsigned overflow is not UB. Signed overflow is, and thus is compiler dependent. It looks like all the bit tricks are done with unsigned so all that looks good. It was things like buffer overflows, and functions saying they return a value and not returning a value that was causing problems.
User avatar
uze6666
Site Admin
Posts: 4801
Joined: Tue Aug 12, 2008 9:13 pm
Location: Montreal, Canada
Contact:

Re: Smoother rendering of high res modes

Post by uze6666 »

With the ability to record a video, is screenshotting really necessary?
Very important for me, I use it all the time there is a new game for the news, wiki, etc. In fact it should even be improved someday to export as png instead of bmp.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Smoother rendering of high res modes

Post by Artcfox »

uze6666 wrote:
With the ability to record a video, is screenshotting really necessary?
Very important for me, I use it all the time there is a new game for the news, wiki, etc. In fact it should even be improved someday to export as png instead of bmp.
Okay. I'm pretty familiar with libpng. That's what I used in the level compiler I wrote for Bugz. What size would you want the screenshots to be, because right now they would end up as 1440x224, since all the pixels are there and SDL is what's scaling it to fit the window. (On an aside, being able to play a game at the unscaled resolution of 1440x224 might not be a bad thing if you're trying to debug the visual output for a new video mode.)
User avatar
uze6666
Site Admin
Posts: 4801
Joined: Tue Aug 12, 2008 9:13 pm
Location: Montreal, Canada
Contact:

Re: Smoother rendering of high res modes

Post by uze6666 »

Same resolution as before, 640x448 and perhaps also shift-printscreen for a regular sized version (320x224) would be nice.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Smoother rendering of high res modes

Post by D3thAdd3r »

Sucks to lose the screenshot I guess. I feel like there is an efficient way to do it since certainly every first person shooter allows it, with things like live security camera screen written to texture. I think you have to lock the buffer and stall though.

For whatever reason I always hit Print Screen which copies the screen and I paste it, crop it, scale it, whatever, and save it as .png. Not sure why I do that since it's a bit of a pain, but works for every program :|
Artcfox wrote:(On an aside, being able to play a game at the unscaled resolution of 1440x224 might not be a bad thing if you're trying to debug the visual output for a new video mode.)
I tend to agree on this.
Post Reply