Core 2.4.1 vs 2.4.2 What are the related issues?

[TD-er]

The last 2 weeks, the nightly builds are based again on:

• Core 2.4.1
• LWIP 1.4

The builds before that were based on:

• Core 2.4.2
• LWIP 2.0 low memory

Major changes from 2.4.1 => 2.4.2:

• Arduino cont stack overlaps system stack by 4 kB, freeing 4 kB of user memory. ( Move continuation stack from .bss onto sys stack esp8266/Arduino#4622 )
• Move lots of strings to flash, saving RAM.
• I2C: improve bus recovery procedure
• numerous WiFi bug fixes.
• Improve hardware serial
• New waveform generation code supporting multiple analogWrite, Servo, Tone at the same time (Support multiple tone(), analogWrite(), and Servo esp8266/Arduino#4640, Compatibility and IRQ fixed for waveform/tone/pwm esp8266/Arduino#4872)
• Add ESP.deepSleepMax() function to get maximum possible deep sleep time ( Mention ESP.deepSleepMax() in the list of ESP-specific functions esp8266/Arduino#4627 )

Possible breaking changes:
Overlap of stacks between system and Arduino cont:

• We define a number of rather large objects on the system stack.
• It is not entirely sure this behavior will introduce other stability issues.
• Large objects could be moved to the heap when in doubt.

New waveform generation code:

• (PWM flicker is back again with new core 2.4.2 #1653) changes stability of PWM signal.

Issues fixed in 2.4.2 and broken in 2.4.1:

• (since 20181004 NeoPixel SK6812/WS2812B pixel errors (2.4.1 vs 2.4.2) #1882) since 20181004 NeoPixel SK6812/WS2812B pixel errors (2.4.1 vs 2.4.2)
• (Restarts due to notification LED [mega-20180809] #1642) Restarts due to notification LED [mega-20180809]
• More available free memory makes the unit faster and perhaps less likely to crash due to allocation errors.

Are there any other issues which may help in deciding what version of the core library to use?
It is also possible to make different choices for different builds.
For example, for typical Sonoff builds (1M flash) it may be useful to have the extra memory. Although 2.4.2 builds may be bigger in size making it harder to perform OTA.

[v-a-d-e-r]

For me the best combination is core 2.4.2 and LwIP 1.4.0-RC2. Running stable on 20 different nodes. Current core 2.4.2+ from Git seems also running good, but takes 3-4% more CPU load with the same config as with core 2.4.2. :-/ All with current ESPeasy from Git. LwIP 2.0 and even 2.1 seem to respond slower on HTTP/UDP as 1.4.

[s0170071]

Thank you @TD-er to sum up the pros and cons of each core version. I think this is a more fundamental question which poses itself again whenever there is a new core. Migrating to a new version is a big step and of course means coping with new bugs. But on the other hand, new versions often have old bugs sorted out.

For those who don't know: LWIP is the LightWeight IP stack, a library which handles all TCP/IP wifi related stuff. It is compiles along with the core. Arduino IDE users can change it using the drop down menu. Core is the Arduino hardware abstraction layer for the ESP8266. You change that using the boards manager.

In my opinion there are three ways to go about it:

A: stick to a known version, i.e. 2.4.1 and LWIP 1.4

Both versions are obsolete. The advantage is that the bugs are known and often you can deal with them (using workarounds). The down side is that you don't benefit from any improvements. (LWIP 2 is way faster, core 2.4.2 brings more free memory (=heap)
Using this option makes only sense if we plan on skipping core 2.4.2 totally and maybe some more. Question is: how long can you avoid migrating to a current version?

B: using latest stable core (2.4.2 ) and latest stable LWIP (v2)

You already guessed it ;-) its faster and gives more memory. And other improvements. Of course they introduced new bugs too, like the PWM issue.
Unless there is no way around current core issues, things won't work, e.g. folks who depend on the PWM and can't figure out a fix have to use an older ESPEasy version. For now at least. But folks who want to use new features like e.g. endless serial buffers can get started...

C: mix versions every other day

Not to get people confused of course, but to distribute available bugs more evenly :-) And the devs can play a funny core version guessing game whenever there is a new issue. When core version 42 is available we finally use it for good. It should have everything sorted out- per definition.

[TD-er]

Do you know any other issue or use case which cannot run on either 2.4.1 or 2.4.2?
Just to help finding a good compromise in deciding what version to use (now) for what application.

For example, typical use can be plugin X or use case Y => use version Z.

[Grovkillen]

What I think could be a way forward is to stick with 2.4.1 and be able to release a proper stable version (finally! 😄) and straight after that release we could bump the core and start digging from there. I reckon keeping on bumping dependencies will make the release "forever" in front of us. Just my top of the mind feeling.

[ascillato]

Hi,

In my experience, core 2.4.2 is way more stable than 2.4.1.

[TD-er]

Hi,

In my experience, core 2.4.2 is way more stable than 2.4.1.

And what LWIP version?

[s0170071]

Shouldn't we re-evaluate the stability now that the stack seems fixed?

[TD-er]

@s0170071 Agreed, but testing it on a few nodes will not give the full picture.
And thus this is also a question whether we should make a few builds also using core 2.4.2?

I am running one node here on 2.4.2/LWIP1.4 and I already saw a HW watchdog reboot.

[ascillato]

With 2.4.2 lwip 2 we had some issues with a mqtt library but now we are using the new pubsubclient and lwip 2 hb is working great.

We found less issues with wifi repeaters with this lwip2.

Now, also the actual stage core is working fine with lwip2. They will release 2.5.0 in december. The great thing in this new core is that light_modem_sleep works fine. No issues.

[s0170071]

December? How about we release two versions in December? One 2.4.1/lwip1.4 and one 2.5.0/lwip2.0

[amigatommy]

I think too, we should not go backwards with core and lwip 2. Usually the plugin developer, in my case the "Neopixel plugin", should change something on his code to get it work. I think, he only has to change some timing relevant variables, maybe different values for each platform (ESP8266 / ESP32 ...) too.
Lets see, what happens with core2.5.0 ... ;o)

[micropet]

I am not a developer, but I like to try the different possibilities.

I've been using Core 2.4.2 LWIP 1.4.0-RC2 and 160 MHz for a couple of weeks, compiled with PlatformIO.

My units are now running for more than 14 days without reboot.
Before that, they boot every few hours.

Peter

[TD-er]

@ascillato What version of PubSubClient do you now use? You said: "but now we are using the new pubsubclient" Is it different from the one included in ESPeasy?

I had yesterday one of my nodes running core 2.4.2 + LWIP 1.4 and that's a big no-go combination.
Max uptime on that node was 140 minutes. All "Hardware Watchdog" reboots.
The same source, compiled with core 2.4.2 and LWIP 2.0 high bandwidth and still not a single reboot (9+ hours)