ESP8266 bootloader?

Topics related to the API, programming discussions & questions, coding tips, bugs, etc. should go here.
Post Reply
User avatar
Jubatian
Posts: 1560
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

ESP8266 bootloader?

Post by Jubatian »

I just got an idea, pondering about my own ARM console and Nicksen's visions of a networked game loader using the ESP8266.

Was it possible to make a bootloader which would be capable to interact with the ESP8622 to fetch games through Wifi? Now I don't have any knowledge in that regard, how communications work at low level there to accomplish it, what I could envision is most importantly a capability to send files to the ESP8266 from a host device the user has through Wifi (to upload whatever game he wants), or secondly to be able to fetch games off of the Internet (this is what I guess Nicksen is after).

However I have experience with the bootloader, and may have some ideas how one could be realized which is capable to work with the ESP8266, but also depends on it.

The situation is that the bootloader has to be crammed into 4K of Flash, while the 4K of RAM is actually not too small to do its job. If the ESP8266 could be programmed with some software designed to interact with the ATmega644 to realize a game loader together, then it may be possible to accomplish this without cycling the ATmega644 Flash.

The interaction could start with that for example the font for the bootloader was stored on the ESP8266, which frees up a nice chunk of Flash on the ATmega itself for adding code to work together with the ESP8266. A maybe could be some crude binary interpreter of some very simple instruction set to carry out some of the tasks, which could this way again be supplied by the ESP8266, possibly in some relation like if the ATmega was a "dumb terminal" for the ESP8266 (and thus the software on the ESP8266 could control how the GUI would look like within the limitations of the minimalistic bootloader graphic mode of course).

Of course then this would be a bootloader specialized for the ATmega644 + ESP8266 system, incapable to work without the ESP8266 (it could launch the game of course, and possibly even still autoprogram SD cards with a single game on them, but no Game Loader - so not entirely useless).

A key difficulty would be making it possible to put the games on the SD card (filesystem!), although maybe it could be possible to get the ESP8266 doing that, too, the bootloader on the ATmega644 only acting as a tunnel to read / write SD sectors.

I could envision this a path to make the user or developer experience smoother on an ESP8266 Uzebox.

What do you think? Feasible? Acceptable?
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: ESP8266 bootloader?

Post by nicksen782 »

I was actually thinking some similar. We have the ESP8266 and it has it's own flash, ram, and eeprom. We still of course want a retro-type game experience so the game itself would be within the ATMEGA. Calling the ESP8266 to perform additional tasks seems like a good use of the hardware.

What sort of tasks? Perhaps the bootloader could be programmed to detect the ESP8266 and then hand over control. Can the ESP send video? I know not directly but could it stream data to the ATMEGA as if it was just taking a video stream and reading the gamepads? This does not sound like a simple thing and I do not immediately see how the bootloader could do both (support for ESP and non-ESP systems.) The bootloader is likely feature-complete for a non-ESP system, right? What about a brand-new bootloader that runs on the ATMEGA but as more of a slave to the program in the ESP other than performing the actual writes to flash/sd and reading the gamepad?

I did not understand what you meant about cycling the ATMEGA's flash. Wouldn't the game still need to be written to flash? Or... are you talking about something where the ATMEGA bootloader just gives control to the ESP which would download the files and send them in chunks to the ATMEGA bootloader for file writing? The ESP and the SD are both SPI. Are they on the same bus? If so, control could be temp provided to the ESP to write the files to the SD. Would there be enough room on the ESP to do full file management AND still provide the AT command set? We would still have the SPIRAM which should be on the same bus. Maybe that would be helpful also?

ESP wiki: https://en.wikipedia.org/wiki/ESP8266 . It has 80k of user ram and 512k to 4096k of flash?

An important step will be updating the ESP firmware and creating some sort of setup program for wifi and perhaps other settings.
User avatar
Jubatian
Posts: 1560
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: ESP8266 bootloader?

Post by Jubatian »

nicksen782 wrote: Sat Feb 02, 2019 8:27 pmThe bootloader is likely feature-complete for a non-ESP system, right? What about a brand-new bootloader that runs on the ATMEGA but as more of a slave to the program in the ESP other than performing the actual writes to flash/sd and reading the gamepad?
Yes, I mean a mostly new bootloader designed for the Atmega644 + ESP8266 system. Since the bootloader interface is there and it has a valid important use-case for games (transferring the burden of understanding SDSC/SDHC and FAT16/FAT32 onto the bootloader), that will stay, and there is no gain in ripping so much off of the bootloader that it wouldn't be able to start the game on its own, or automatically program and start the game from the SD card if that's the only game on it. So if for some reason the ESP8266 became inaccessible, the Uzebox itself still wouldn't become a brick.

What would be removed is the User Interface (the Game Selector in which you browse games), and in the place of that would be the added functionality necessary to interact with the ESP8266 so the ESP8266 could operate the GUI and access the SD card through the SD sector read / write interface (the software running on the ESP8266 carrying an own, complete FAT filesystem implementation so it can properly write games or even defrag if needed). And, of course a command to ask the bootloader to actually program and launch a game.
nicksen782 wrote: Sat Feb 02, 2019 8:27 pmI did not understand what you meant about cycling the ATMEGA's flash.
That without such a bootloader, whenever you wanted to access the Net for games, you would have to program the networked game loader (a .uze file).

The Uzebox would still be an Uzebox, of course, games wouldn't be impacted in any manner, this approach would be intended to add a capability of networking into the bootloader (so you would no longer need to select and program the networked game loader when you wanted to get something through Wifi). Of course nothing stops you from writing such an ESP8266 application (burned into the ESP8266) which realizes some games by this supposed new bootloader's features, but such games would be very crude in comparison to normal Uzebox games (as the bootloader's video mode is extremely limited, audio capabilities stop at a simple click sound). However they wouldn't cycle the Flash :)

I just had this idea seeing you would like to go on with something networked, and I guessed if this seems possible, lets throw the idea in. It would make the feature a lot more accessible (not only it wouldn't cycle the Flash, but you wouldn't have to select the networked game loader's uze in the bootloader's game selector out of more than a hundred games every time you wanted to update your games or download something).
User avatar
nicksen782
Posts: 714
Joined: Wed Feb 01, 2012 8:23 pm
Location: Detroit, United States
Contact:

Re: ESP8266 bootloader?

Post by nicksen782 »

Yes, it would absolutely require the flash for my game downloader and then of course for every game ever played.

The bootloader's video display is limited. Understandable. Cool that you could get the ESP to drive the bootloader into changing the display though. Not so good for a game but it would be great for utilities such as we are discussing. The ESP utilities and the bootloader will not need to be written often outside of development, right?

What about two bootloaders? The existing one is complete, yes? The new bootloader could be used for ESP systems instead. You would not need to consider checking if the ESP exists since the user wrote the new bootloader and would have chosen to do so hopefully knowing that the ESP would be required. I'm thinking strongly about the Uzebox portable which officially always had the ESP. The Uzebox console has the ESP/SPIRAM as an available add-on which is optional and was not always available. If I have an ESP then I'm going to choose to flash the ESP-aware bootloader. The current bootloader would work on the portable but not with the ESP functions. I would update my bootloader on my portable and consoles for the ESP version since they all have ESP. User choice.

Looks like my game loader idea is turning into an ESP program. I'll need to build a programmer. Something like this: https://www.iot-experiments.com/flashing-esp8266-esp01/ . I have seen that you can reprogram the ESP onboard without a programmer in newer versions. However, the firmware is old in my Uzenet modules (and with varying firmware) so I'll need to build a programmer circuit anyway. Something that the Uzenet module can plug into.
User avatar
D3thAdd3r
Posts: 3175
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: ESP8266 bootloader?

Post by D3thAdd3r »

Really interesting idea. I would have to believe it is possible, as the MCU on the ESP8266(which drives the separate WiFi chip) has been used as a C64 emulator, a standalone game console bitbanging, I think composite color via artifacts timing or some such, etc. A totally custom firmware is possible.

A virtual machine could be great for this. Perhaps a stripped down 6502. There are probably some ultraRISC instruction sets which would trade ram density for code space of the interpreter itself. Just speculation from me.
User avatar
Jubatian
Posts: 1560
Joined: Thu Oct 01, 2015 9:44 pm
Location: Hungary
Contact:

Re: ESP8266 bootloader?

Post by Jubatian »

Two bootloaders, yes.

The current one has its place on Uzeboxes without the ESP8266 or if you simply didn't quite liked the idea of the networked bootloader. The new one would be used on Uzeboxes with the ESP8266, for that there is no other option as a fundamental part of it would actually reside on the ESP8266.

It isn't quite possible to exploit the ESP8266 in any significant manner in games beyond it's intended purpose (networked games). The Uzebox can do 115Kbaud on the UART at most, you can not push that higher since the video modes wouldn't permit that, the maximal byte rate would be the same as the audio output's rate, for a corresponding UART baud rate that would be something around 160Kbaud.

Fractals maybe (as something calculation intensive), but you would have no way to display them since the ATmega wouldn't be able to hold the data.
nicksen782 wrote: Sat Feb 02, 2019 10:18 pmThe bootloader's video display is limited. Understandable. Cool that you could get the ESP to drive the bootloader into changing the display though. Not so good for a game but it would be great for utilities such as we are discussing. The ESP utilities and the bootloader will not need to be written often outside of development, right?
For now it is only a thought experiment. The ESP8266 supports running user software, the ATmega644 and the ESP8266 communicates by UART, so it should be just the matter of an appropriate protocol to pass around for example Video RAM contents to get the ATmega644 displaying what the software on the ESP8266 would like to display. A lots of things are doable, but for example downloading Tornado 2000 would be a really painful experience (even at the most optimal conditions assuming only the time taken for the UART transmission, it would take over 11 hours. In the reality once I transferred a 80Mb game to an old computer by 115200 baud UART connection, took 3 hours, at least not as painful than doing the same with floppies).
User avatar
D3thAdd3r
Posts: 3175
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: ESP8266 bootloader?

Post by D3thAdd3r »

Like Jubatian points out, the bandwidth between the ESP8266 and the Uzebox puts some hard upper limits on anything needing large amounts of data in guaranteed short time frames. I would think the approach would be good for some RTS game where there is neither ram nor cycles to do anything approaching even the oldest entries in the genre IMO. AI offloading for some games. But to use it for something like that necessitates a constantly changing ESP8266 software per game, and mingled with updates to support networked games. The bootloader seems a good direction since it could probably reach some minimum state to where it would not need to be updated again. For things that could benefit from offloading to a higher resource machine, a remote PC can do that though with added internet lag. I really need to get back into this Uzenet stuf, it has reached itself into my brain often for the last week!

It would be pretty awesome if the bootloader would implement only the bare minimum to display something and program games, with all possible menu/graphics/logic/etc downloaded so that updates are automatic and reflashes unnecessary. I do believe the ESP8266 is fast enough to run interpreted code and still run UART and WiFi fine for networked games. Only because the machine already would have a requirement for an ESP8266 as well as Internet access to operate correctly, I like the idea of maximizing benefit from what has already "been paid for" in that case.
Post Reply