Uzenet

Topics regarding the Uzebox hardware/AVCore/BaseBoard (i.e: PCB, resistors, connectors, part list, schematics, hardware issues, etc.) should go here.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Uzenet

Post by D3thAdd3r »

Hmm there could certainly be more than one version of the Ai Thinker. It seems like every guy and his brother has made some non-official version of the AT parser which doesn't help. QE..I think they basically got it out there as fast as possible and made it work incrementally later. They probably benefited from some open source contributions along the way. I haven't done extensive UDP testing with this version, were you unable to send them or you being a UDP "server"(ie. you accept UDP packets) didn't work? I know for sure at this point there is *some* version that totally works judging by the more extensive UDP programs people do than I personally have, yet. Strangely my wifi happens to have a 10 character password and I have'n't tried it on a different router. I will definitely need to change it and make sure it works with other numbers. That is a pretty bad bug to have in firmware!

The only hard part of it is fumbling with the UART and string parsing at this point. The interface is a little too high level for my liking, as I'd prefer to have 1:1 socket level control so you could actually do things like ICMP or if they would just add a "AT+PING=.." with a binary stream. There is a work around for anything like that but it's grosser than making custom firmware. Basically send a connectionless UDP packet to 255.255.255.255 broadcast, then you have to have a PC running a program that will read the payload and interpret it as a command, do it, and send back the result or something.

If you were willing to accept having a PC in the mix, which I am not personally against, you can do damn near anything. Everyone has an Arduino or something laying around that works as a nice 5v USB<->Serial bridge; just connect Tx<->Rx,Rx<->Tx,Gnd<->Gnd and it works with the Uzebox. ESP8266 emulator on PC, done. At that point any device that gets a signal to the Arduino works if it speaks 8266 transparently which leaves options for people with parts laying around doing nothing.
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Uzenet

Post by Artcfox »

D3thAdd3r wrote:Hmm there could certainly be more than one version of the Ai Thinker. It seems like every guy and his brother has made some non-official version of the AT parser which doesn't help. QE..I think they basically got it out there as fast as possible and made it work incrementally later. They probably benefited from some open source contributions along the way. I haven't done extensive UDP testing with this version, were you unable to send them or you being a UDP "server"(ie. you accept UDP packets) didn't work? I know for sure at this point there is *some* version that totally works judging by the more extensive UDP programs people do than I personally have, yet. Strangely my wifi happens to have a 10 character password and I have'n't tried it on a different router. I will definitely need to change it and make sure it works with other numbers. That is a pretty bad bug to have in firmware!

The only hard part of it is fumbling with the UART and string parsing at this point. The interface is a little too high level for my liking, as I'd prefer to have 1:1 socket level control so you could actually do things like ICMP or if they would just add a "AT+PING=.." with a binary stream. There is a work around for anything like that but it's grosser than making custom firmware. Basically send a connectionless UDP packet to 255.255.255.255 broadcast, then you have to have a PC running a program that will read the payload and interpret it as a command, do it, and send back the result or something.

If you were willing to accept having a PC in the mix, which I am not personally against, you can do damn near anything. Everyone has an Arduino or something laying around that works as a nice 5v USB<->Serial bridge; just connect Tx<->Rx,Rx<->Tx,Gnd<->Gnd and it works with the Uzebox. ESP8266 emulator on PC, done. At that point any device that gets a signal to the Arduino works if it speaks 8266 transparently which leaves options for people with parts laying around doing nothing.
I was able to send and receive UDP packets fine, and the entire reason why I wanted to use UDP was that it was connectionless, and I wouldn't have to deal with re-establishing connections after frequent power failures. The main problem I had was is one of the ESP8266 boards got power cycled, then it would refuse to join the other ESP8266's AP again. I could list all APs, see the one I wanted to connect to, but it would time out and eventually give me an error if I tried to rejoin it, until I power-cycled the other unit. I had a pair that I was planning on using in a place where there was no Internet connection or WiFi router to read sensor data in one location, and broadcast it to the another device where it would then be displayed.

After that, and the fact that the only working UDP firmware at the time would only connect to APs that had exactly 10 characters in the name, I just got fed up with the whole idea of using an ESP8266 for anything. I have a bunch of WIZNET Serial-to-Ethernet gateway boards that I've used in the past, and those are much more reliable in my experience.

If you're interested, I still have my USART library where I implemented the Knuth-Morris-Pratt string searching algorithm to parse the responses from an ESP8266 within a given timeout. The cool thing about the KMP algorithm is it only needs to make a single pass through the incoming stream, so it doesn't need to buffer anything more than a single character at a time (the search strings are fixed and can be in PROGMEM), and you can search for multiple different strings within the same stream in parallel, which is useful for checking for both the OK and ERROR response at the same time (as opposed to just looking for an OK and having to wait for it to timeout if you get an ERROR). The timer stuff is designed for an ATmega328P, but that part could be easily changed to use an existing timebase.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Uzenet

Post by D3thAdd3r »

I did manage to get this version to crash as TCP(server) when you connect, disconnect, reconnect repeatedly, I will have to make sure the latest official version has that fixed. Old versions did have problem with ARP which is supposedly fixed, basically they just stopped responding to ARP requests after a while which screws up the ARP table in general and any communication with other devices. Past that, some people are saying to give this thing a 750ma-1A power supply, back when I had some shady setups I wasn't sure if it was power related or just normal horrible FW problems heh. Whatever random version this AI Thinker is, it works with 8 character passwords and UDP. And of course there is firmware for the AT parser and SDK version for the lower level stuff...which can be updated independently it seems..I honestly cannot keep track of it all, what is or was broken, it's so much more rage inducing than it is interesting but I think a lot better than a year ago.

Your KMP code I am very interested in, as I have still not implemented much in the way of string handling that is graceful or might also be slow. If it's really flexible I suppose all the games could use that as a standard technique. I never implemented such a thing, but it sounds a clever concept that is hard to beat.

Also curious on your experiences with WIZNET, and if you think there is any realistic way both devices could be supported by the same game? Besides convenience for some items, I will always say "a wire/fiber is better", the latency is lower and more consistent, etc...until quantum entanglement takes over. If I see a "Wave Function Collapse<->UART module" in the $4 range I will change my mind :lol:
User avatar
Artcfox
Posts: 1382
Joined: Thu Jun 04, 2015 5:35 pm
Contact:

Re: Uzenet

Post by Artcfox »

I PM'd you my entire project that used the ESP8266.

That ARP bug sounds terrible.

I used a 1 Amp switching mode power supply to power my ESP8266 boards.

I believe the Wiznet Serial-to-Ethernet gateway is very similar to the "transparent bridge" firmware out there for the ESP8266. Anything sent to the serial port will go out as a packet, and anything received will be sent over the serial port. If they are both pre-configured to bridge to the same IP:port combo, then I think they should act the same, with the exception of the ESP8266 transparent bridge firmware, which IIRC has a special +++ command that puts it into configuration mode, so as long as you don't send a packet with just "+++" as the payload, they would probably act the same.

I dont even want to think about what havoc firmware bugs in a $4 quantum <-> UART chip would cause. Half the molecules in my cat randomly being shifted two feet to the left? I'll pass! ;)
User avatar
uze6666
Site Admin
Posts: 4801
Joined: Tue Aug 12, 2008 9:13 pm
Location: Montreal, Canada
Contact:

Re: Uzenet

Post by uze6666 »

Say Lee, which version of the firwmare are you using? From the espressif sitethey are now at esp_iot_sdk_v1.3.0 (far from the 0.9.2!). AT is 0.4. I think its time to upgrade since there's a lot of fixes and new AT commands. Only thing is it appears to be a bit bigger in memory requirements so hopefully our modules are big enough.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Uzenet

Post by D3thAdd3r »

Code: Select all

AT version:0.21.0.0
SDK version:0.9.5
wow this IS very old
Oh yeah the improvements sounds promising too, and indeed it seems the 512k flash limits are already showing up. From my reading, the real issue with the 512k on most modules is that you must compile a non-FirmwareOverTheAir version, since normally it reserves significant flash space to download the firmware to? So I think we can still use the most recent version but I'll research and make sure. We don't need OTA upgrades with the right setup tool anyway, but those bug fixes and performance increases are critical.

I am focusing on old firmware versions people are likely to have on a module they get and make sure all quirks are handled. Autobaud is small and fast(<1 second) and should work with any module/version/firmware in any state you plug in, and successfully get it to a bit rate/baud of 57600 for fast transfers(but just realized some really old ones have to reboot to change bit rate, and some might be 115200 at startup...so have to handle that too..by autobauding immediately with interrupts off and polling I guess). It already parses a CWLAP to a scrollable list of all visible wifi APs that shows signal strength(letters as placeholder for graphics ATM), SSID name, and security type 100% reliable. You can scroll with D-pad to select an entry and it brings you to the password menu then joins with a AT+CWJAP and reports if successful or the error number. Now we can send a simple HTML 1.0 request to grab uzebox.net/firmware.txt. If the current one doesn't match the listed one, ask the user if he'd like to upgrade. I'm working on that part right now.
espsetup.jpg
espsetup.jpg (107.51 KiB) Viewed 5249 times
I probably said this before but the hardest part left is to make sure the firmware upgrade protocol is correct. It also requires writing to the SD card(which I've never done, is this currently supported in PFF on hardware at least?) since the SPI ram is too small to store that file. The user will have to touch a wire from GPI0 to GND when the instruction/countdown timer on screen tells him to, the module then resets in upgrade mode(I recall you can let go of the wire at this point), and the Uzebox leisurely dumps downloaded SD data to the chip while the user enjoys a couple shots of whiskey. I already screwed up a flash on an ESP01 and it came back to life with another firmware upgrade, eh, if the code kill someone's module I will gladly mail them an updated one with a heartfelt apology :lol:
Artcfox wrote:I dont even want to think about what havoc firmware bugs in a $4 quantum <-> UART chip would cause.
Good point, I don't think I would want to be an early adopter on that either :lol:
User avatar
uze6666
Site Admin
Posts: 4801
Joined: Tue Aug 12, 2008 9:13 pm
Location: Montreal, Canada
Contact:

Re: Uzenet

Post by uze6666 »

I am focusing on old firmware versions people are likely to have on a module they get and make sure all quirks are handled. Autobaud is small and fast(<1 second) and should work with any module/version/firmware in any state you plug in, and successfully get it to a bit rate/baud of 57600 for fast transfers(but just realized some really old ones have to reboot to change bit rate, and some might be 115200 at startup...so have to handle that too..by autobauding immediately with interrupts off and polling I guess). It already parses a CWLAP to a scrollable list of all visible wifi APs that shows signal strength(letters as placeholder for graphics ATM), SSID name, and security type 100% reliable. You can scroll with D-pad to select an entry and it brings you to the password menu then joins with a AT+CWJAP and reports if successful or the error number. Now we can send a simple HTML 1.0 request to grab uzebox.net/firmware.txt. If the current one doesn't match the listed one, ask the user if he'd like to upgrade. I'm working on that part right now.
In the light of the latest releases of the firmware, I got to think more about it. Until an official release of uzenet is made, I'd say the latest and greatest firmware and ESP modules should always be the baseline. I ordered some recent esp-12e modules which are nice and black. I think the new Uzebox PCBs will then have be black to match. :P

You should really have a look at the latest AT spec as it contains many news features I think should become the baseline and will greatly simplify code. Among them is separate functions to set baud rates and such in a permanent/default and non-permanent way. So the Uzenet modules should ship with the default set at say 9600 to leave the maximum resources to the main program by default. If any game wants more, they must set it with the non-permanent function. Like this, upon reset the esp-module is always in a know state. There's also a way to finally get the rssi in an easy way.
Attachments
4A-ESP8266__AT Instruction Set__EN_v1.3.pdf
(869.3 KiB) Downloaded 313 times
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Uzenet

Post by D3thAdd3r »

Totally agree, messing with the old stuff is just to get these things to a usable state in the easiest way for the most possible users. Also a means to upgrade versions should we prefer some future one. Especially for UART like you say, it has to start at 9600 and present a known state so any game can reliably use it from reset. The new version allows temporary baud rate changes where this one stays across reset.

Of the couple ATs I hadn't seen SENDX is the most exciting. It starts transmitting a packet as soon as it encounters '/0', but will continue waiting for more bytes. I bet this should get us around the repeated sends causing error due to backed up TCP packets!

The 12e would be great and it's not an off blue. Since that would require the Uzenet addon PCB to be changed, it might at the same time be worth considering a jumper/shunt on board so the user can more easily put it into upgrade mode. Also, with stock firmware I think pulsing the GPIOs, even GPIO0 after it's started, has no effects. It might be possible to multiplex some expansion pin without issued to enter upgrade mode through software alone.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Uzenet

Post by D3thAdd3r »

BTW I find that converting Python to C is more cumbersome than I though. There are lots of elements of the basic libpy that I don't understand, and have to look up, and it doesn't inspire much confidence this will work in the end after the time investment. I never saw it before, but it was there the whole time: esptool-ck and I am experimenting with it. When I see it in C I actually understand how the flashing works as opposed to Phython I am barely familiar with. I don't understand it enough yet to think otherwise, so from what I can tell right now it requires a 4k buffer and so SPI ram is required. I suspect it could be done in chunks with SD->local ram if the 8266 isn't too picky on timing but I have no proof yet. Pretty sure this is the right codebase to work from for our purposes though.

Edit-
I also find that the Uzebox can seemingly detect a module that starts at 115200 every time and then change it to the preferred 57600. There are old modules that would only do 115200 and you couldn't change them so upgrading their firmware wouldn't be as friendly. I did not do any extensive testing, but it seems like 115200 is working OK for the short responses required to get status responses. Did you find issues with that baud rate since you finalized the UART code in HSYNC Alec? My only thought about it is it would be most likely to "hit" the highest baud rates possible if the calls were as evenly spaced as possible. Do you think pushing Rx to the very last thing, doing wait cycles before it of the left hsync_pulse and then having a right hsync_pulse where the Rx happens before anything else might help?

Basically <hsync>mix sound, tx, rx</hsync>--------------------<render scanline>-----------------------<hsync><rx, mix sound, tx>

I just have this feeling 115200 could work reliably.
User avatar
D3thAdd3r
Posts: 3222
Joined: Wed Apr 29, 2009 10:00 am
Location: Minneapolis, United States

Re: Uzenet

Post by D3thAdd3r »

Some stuff I am doing on Megatris right now. So far it is assuming the module starts at 9600 then configured it to TCP @ 57600 and I have no desire to go to UDP or 115200. This code is intended to be be a working drop in for any similar puzzle games and so is designed to be as small as possible, and ultimate performance as a secondary priority. The consoles are linking and, most of the time, agree that they are both running the same version of the same game. Then they transfer the names of the players to each other. This part I would definitely like some opinions on from other game developers since all of this is useless unless developers agree and build things that cooperate with each other.

I am thinking a maximum name length of 13 might be good as it allows for "CUNNINGFELLOW", the longest name of anyone active I noticed. Else even 9 might be good(dubiously for "D3THADD3R" ;) ) or people could do things like "CUNNING", or even "DTHADDR" if shorter seems better. I think 8 would be a bare minimum length though. Also I am considering not every game has the luxury of supporting Alpha and Numerals for player names so certain leet names like "L4rry" or "D3thAdd3r" would need to be "LARRY" or "DETHADDER". Unless someone can explain otherwise, I think requiring all names to be CAPS LOCK is best. I don't even remember a game that uses lower case letters or that would have the space to do so. Ram tile fonts on a dedicated VS screen...but not every game is mode 3. A couple ways games could do this also. I like this, mode 1 games might not mind 13 character names and could actually display them in the playfield like this:
uzenet_namelen.jpg
uzenet_namelen.jpg (31.98 KiB) Viewed 5056 times
Other games using <= 30 VRAM_TILES_H where 2 long names would basically be across the entire screen, or there might not be a place to logically do this, could simply have an Uzenet "CONNECTING..." and "MATCH UP" screen where it displays the names, then proceeds gameplay like normal without having player names displayed for the gameplay screens. The nice thing about a VS screen in a game already using ram tile video modes and SD, is that they could have much more elaborate VS screens. A mockup:
uzenet_vs.jpg
uzenet_vs.jpg (23.71 KiB) Viewed 5060 times
Just say a number with a vague description why it would be best and let's average it to build a consensus of those interested. Seems a small thing, but with so much not set in stone it's basically impossible to do anything on any part. Also, perhaps before uzebox.net lets you advertise yourself as "NICKSEN782" and affect his reputation you have to send a password. Problem is, it is a big code expense to retrieve that from EEPROM if your games doesn't already use it(like Megatris). So then another swing in advantage to the separate match making program it seems. One solution I see is for an initial setup program to also change the ESP8266 SoftAP password like:"NICKSEN782:SOMEPASSWORD123". That is retrievable instantly sending only one UART string. Then even across resets, any game could log in and know our local name without resorting to EEPROM.

The other question I'm pondering is would it be interesting to keep track of rankings, and if so, what would be the most universal way. The best I can think of is to use ELO rating system like they do in chess. This system automatically makes the scores skill relative to all other existing players. Then should it be per game or just overall Uzenet matches across any games played? I am not interested in just for the sake of it competition really, but if it would be another "cool factor" to draw people in then I would be interested in how this could be made easy to implement for any game in a small manner. I would really cherish Uzenet rantings so we can start to string them together into standards :ugeek:
Post Reply