Page 1 of 2

Online Gconvert JS

Posted: Fri Jan 11, 2019 6:54 am
by nicksen782
I have put together an online version of Gconvert. It is not an Emscripten build of Gconvert with a web interface (although I am interested in doing that.) It is a web application where all conversion happens through JavaScript. You do not actually even need PHP either. Just host it on whatever since all the work happens in JavaScript.

The Online Gconvert JS tool does NOT support mega-tiles or code tiles. Sorry. It does however, support 6x8 tiles, 8x8, tiles, or whatever dimension as long as it is specified in the .xml. Perhaps I could work with those in the future if I gain some experience with them.

This software is intended for Uzebox, the Retro Minimalist Game Console.
Uzebox website: http://www.uzebox.org
Uzebox software, including Gconvert can be found here: https://github.com/Uzebox/uzebox
Gconvert, specifically, can be found here: https://github.com/Uzebox/uzebox/tree/m ... s/gconvert

OnlineGconvertJS is a complete rewrite of the Uzebox Gconvert.

New features in OnlineGconvertJS:
Tile de-duplication is optional.
Final tileset is shown as an image.
Original tileset image is shown but with duplicate and unused tiles grayed out.
Lots of downloads. Nearly all output (and input) can be downloaded from the program.
Has integration with the Uzebox Asset Manager which allows for additional features:
* Saving the output .inc files to the web server.
* Querying the UAM database for known games and their asset resources (.xml, .png, etc.)
* Requires UAM to be integrated and a valid login to UAM with assigned permissions.
Does not require UAM to be integrated. Will work without it but minus the UAM features.

OnlineGconvertJS FULL FEATURE LIST:
INPUT:
Load XML, load IMG, Validate XML.
Download XML, download IMG.
Includes some sample data files as templates.

MAP EDITOR:
Load XML and IMG into map editor.
Can create new tile maps.
Has hover preview options and an easy double-click to create map feature.
Tilemaps can have different output types. (PROGMEM, C2BIN, SKIPMAP, NOWHERE.)
Tile de-duplication is optional. On by default.
Can specify tileset output types (PROGMEM, C2BIN.)

OUTPUT:
Outputs .inc files into text boxes.
Final tileset is shown and can be downloaded.
Original tileset with used duplicate tiles is shown.
Any tile used in a map and seen more than once will be grayed out.
Any unused tile will also be grayed out. Can download this image also.
Final tileset has a hover preview that will indicate the tile id in the tileset.
Lots of download features available:
input xml, input img, maps img,
tileset img, marked dupes img, progmem.inc, c2bin.inc,
All input files in zip, all output files in zip, everything in a zip.

PERFORMANCE:
Each portion of the conversion portion of the application (OUTPUT) is timed.
You can see what takes the most time in the conversion.
Conversions are very fast!
On my computer, an overworld tileset consisting of 94856 tiles was reduced down to 88 tiles (it was a simple tileset) and the process took 1.94 seconds. This includes tile deduplication.
A more typical tileset consisting of 9270 tiles reduced down to 196 tile and took 0.23 seconds.

COMPATIBILITY:
Works in Firefox, Chrome, as well as Internet Explorer 11.

LINKS:
https://nicksen782.net/UAM/APP_gconvert
https://github.com/nicksen782/OnlineGconvertJS

Re: Online Gconvert JS

Posted: Fri Jan 11, 2019 8:49 am
by L4rry
Awesome! Looks really good. I've had a quick play around and everything seems to work pretty smoothly and intuitively. I really like the feature of graying out duplicate files. I've uploaded my Tank Fu tileset and can instantly see where there are unique tiles where there really shouldn't be in my boss tile set. So at a glance, it's easy to see how and where I can optimize. Great stuff!

Re: Online Gconvert JS

Posted: Sun Jan 13, 2019 9:05 am
by nicksen782
I'm glad that it helped! I've been working on a browser-based gconvert for a long time now and use it exclusively. But, it looked bad and I really wanted to polish it and get it out there.

What surprises me is how fast it can be done in a web browser. There is a TON of data crunching going on and it just cuts right through it. To convert the Zelda 1 overworld tileset and create maps in my initial version (a couple years ago) took upwards of 40 seconds and the browser would freeze while you wait! Now, an even larger image and tilemap set takes less than 2 seconds. Most conversions are nearly instant. I actually looked into using web workers for the data crunching (to get another thread) but it really is fast enough as it is. This application has really taught me quite a bit. So, another feather in my hat and another thank you to Uzebox in general for giving me a motivation to learn. Arduino brought me here and Uzebox brought me to web apps and a stronger understanding of C and hopefully sometime in the future, ASM.

... and I do have a nearly completed game in the works... finally! Mode 3 with standard Uzebox features (no SPIRAM or WIFI.) It's coming.

Re: Online Gconvert JS

Posted: Wed Jan 16, 2019 4:39 pm
by Jubatian
Huh, this should be something quite useful if only I knew what to do with it (never used GConvert). I have some suspicion that it might not be functional on my system (Firefox ESR 60.4.0esr (64-bit) on Debian Linux), clicking any of the buttons appear to do nothing, surfaces are black, I can only succesfully click text links (Download IMG sample gives me a Megatris sample) and adjust the numbers under the Tile Map Editor's Preview box. I dunno whether I am just missing doing something which would be normally obvious to someone who used GConvert, which would get all this interacting proper.

Anyway, should be something great with lots of work involved!

Re: Online Gconvert JS

Posted: Wed Jan 16, 2019 5:00 pm
by nicksen782
64.0.2 (64-bit) on Windows 10 works. I used the latest version offered by Linux Mint also. I actually had it working full in Internet Explorer 11 which I am assuming is older than Firefox 60.4.0. So, I'm not sure what is going on here. When was Firefox 60.4.0 released?

Firefox really insists on having buttons and inputs with generous spacing and it messes up layout a bit. I thought I had it fixed but I can see that the map records in the map preview take up two lines instead of just one. I'll have to work on that.

The surfaces (center top, and right side) of the INPUT section are for the source image and the xml that will drive the rest of the program. I support the main format of the official gconvert (and also support two others which I have not discussed yet.)

Gconvert takes an image and an xml file that defines regions of that image that will be tile maps. From the used tiles within the tilemaps a tile set is generated and then tiles are checked for duplicates. Tilemaps point to tile ids in the tileset. The deduplicating will find duplicates of a tile and use tile id of the original view of that tile. Then, only one copy of the tile is stored, so de-duplication. My program does the same thing and normally should give the same results as gconvert if only using features that gconvert has.

So, as a quick program test you can go to the test data box in the upper-left and choose a game. Megatris is a good choice. You'll see the source image appear and the XML as well. Then click "Load into the Map Editor" at the bottom-left.

The screen will scroll down to the next session. The data from the xml file has been broken down into rows. Think in tiles. L and T (that is left and top) indicate how many tiles from the left or top the first tile in a tilemap will be. So, 0 and 0 would give you the upper-left corner tile. W and H (width and height) indicate how many tiles wide that tilemap is from the start point. Similar with height. You can think of it like a rectangle covering a region of contiguous tiles. You can edit maps or you can continue by clicking the "PROCESS" button at the bottom-left.

Again, the screen will scroll to the next section and the map data will be converted. You'll see used unique tiles displayed (with unused and dupes being drawn grayed out). In the center you will see the result tileset as an image. Hover over any part of it to see what the tile id would be. Finally, you'll see on the right the gconvert text output. This is what you would put in your game for graphical resources. It starts with the tileset data followed by tilemaps that use that tileset data.

Does this help at all?

... I just downloaded 60.4.0 and I can confirm what you have found. I'll look into this.

*EDIT*
Okay, the bug was due to a polyfill that did not succeed. It was for the Full Screen effect. I disabled the polyfill. The polyfiller is designed to fail if a loaded polyfill does not actually work. In Firefox ESR 60.4 it fails. Please try again. The full screen feature will not work right with ESR but it won't throw an exception anymore either. Actually, online gconvert doesn't actually use that feature, but the online emulator does.

Re: Online Gconvert JS

Posted: Wed Jan 16, 2019 6:16 pm
by Jubatian
W-O-W... Sorry, wasn't at the machine, that was quick!

I just set this computer up about a month ago. I have a crappy traffic-limited internet solution, so it would be costly for me to keep stuff perpetually at the most recent version. They won't trust me for broadband being new in the country, all I could get is one of these lousy SIM based Wifi boxes, at least a little better than a USB dongle since those never work right with Linux.

Anyway, glad you found it, and of course now things are a lot more responsive here!

Re: Online Gconvert JS

Posted: Sun Jan 20, 2019 9:46 pm
by rocifier
Cool. What's GConvert?

Re: Online Gconvert JS

Posted: Mon Jan 21, 2019 1:41 pm
by L4rry
rocifier wrote: Sun Jan 20, 2019 9:46 pm Cool. What's GConvert?
gconvert is a command line tool, developed in c++, that is provided along with the main uzebox source code here: https://github.com/Uzebox/uzebox/tree/m ... s/gconvert

In summary, you provide it with an XML file describing your tile set along with a bitmap image. The tool will then generate c source files describing your tile set as PROGMEM arrays to include in your uzebox game source.

You can read more about it here

'Online Gconvert JS' is an effort to port this to a web based tool with additional features.

Re: Online Gconvert JS

Posted: Mon Jan 21, 2019 3:17 pm
by nicksen782
@L4rry: Yes, that is correct. The previous posts go into detail but the TLDR version is exactly what you just said.

I would like to work on Mega (Meta) Tile Compression as the next feature in the Online Gconvert JS. Iros used this with long (wide) scrolling levels. What about something else like an overhead RPG screen? Scrolling won't be required for this. The biggest part of that type of game is actually the tilemaps themselves, not the tiles. I get the idea that a mega-tile encapsulates tiles so that you get a 1x1 tile that actually has 4 tiles (2x2) inside or whatever you choose the dimensions to be.

So, you take the same map XML that you would use for the whole map but you wrap it with a mega-map that has the new "tile" dimensions. The result is that you now have a map that isn't using 8x8 tiles but now uses larger "tiles". So, mega-tile-width=5 and the width=255 gives 51 (simple division.) What does Gconvert do then? I'll assume the inner tiles are still part of the tileset.

What are the downsides? I mean, this sounds really good almost as if we all should be using mega-maps to save on flash (so, more tiles, more music, more game!)

Questions:
Gconvert only writes the mega-map to the .INC? The original tilemap doesn't exist?
How do you actually DrawMap2 the map with Uzebox?

*EDIT* Upon looking at the Iros source it appears that you have your own draw function for the mega-tile maps. You put mega-tiles on the gconvert wiki page, perhaps you could add something to it that would indicate how to draw these maps? That would likely increase adoption of this great idea.

Re: Online Gconvert JS

Posted: Mon Jan 21, 2019 4:31 pm
by L4rry
It works exactly as you describe :) There is still a deduplicated 8x8 tileset generated in the .inc file.

Yes, Good point. I can add a routine to the wiki describing how to extract a tile using mega maps. Wven better, I could optionally generate such a function as part of the actual convert. I'll try and find some time for that over the weekend. I also want to implement meta-meta-etc. tiles as well as change terminoligy from 'mega' to 'meta'

PS. This will work fine for scrolling RPG sand boxes too. I tested it on a 4k x 4k scrolling level for an abandoned RPG I was working on.