This is in every possible way extremely exciting news for me!
When trying to diagnose the nature of the crash, I was experimenting with adding a buffering scheme in HTTPGet and quickly discovered that buffering requests (and setting a limit on how often they are allowed to be sent) clearly helped reduce the risc of crashes. It was however when I modified the HTTPGet even further to make it use syncronized http requests (rather than the default async), that I noticed that this seemed to completely prevent the server from crashing. So the final clue was that something was disturbing the server between the http request being sent and the resulting (asyncronous) reply.
I now have the new development build (z-way-4.0.2-4-ge4433cb0-lws16_armhf.deb) installed and running the usual stress testing environment and must say that the results so far is very encouraging - it has been running smoothly for >27 hours by now! (I have of course reverted to the original HTTPGet module, and with the stressful test case I am running this would typically lead to a SIGSEGV crash within 2-6 hours)
I do want to keep the test running for a while before I dare stating the problem is 100% cured, but this is looking very promising indeed.
Huge thanks to PoltoS for always listening to feedback and never giving up!
Z-way-server occasionally goes down
Re: Z-way-server occasionally goes down
Same here.
My system is running for nearly 40 hours.
I have increased the pressure by adding another 7 HTTP devices to now have a total of 27.
I hope it will keep going.
My system is running for nearly 40 hours.
I have increased the pressure by adding another 7 HTTP devices to now have a total of 27.
I hope it will keep going.
-
- Posts: 16
- Joined: 15 Jan 2023 19:08
Re: Z-way-server occasionally goes down
Hi, I have had similar problems over time and I assumed it was the constant writing to the SD card. I always keep an img file of my latest setup but was sick of buying new cards.
Although it may not be ideal for everybody, I have seemed to have stopped the crashing by creating a CRON job for removing the log, tmp and message files form /var/log and then a clean reboot every few hours (when I know I may be asleep etc).
Although it may not be ideal for everybody, I have seemed to have stopped the crashing by creating a CRON job for removing the log, tmp and message files form /var/log and then a clean reboot every few hours (when I know I may be asleep etc).
Re: Z-way-server occasionally goes down
@harveybabes the problem you describe does not look to be related to this bug. Or it was just a coincidence that lead you to link it to clearing logs.
If you had a lot of async HTTP requests, it was certainly this bug.
If you had a lot of async HTTP requests, it was certainly this bug.
Re: Z-way-server occasionally goes down
@harveybabes: If you don't work a lot with the log file it may make sense to reduce the amount and depth of information captured in the log file.
This is defined in the file /opt/z-way-server/config.xml
There should be a line
<log-level>2</log-level>
The numeric value determines the amount of information. 0= log all, 9 = log almost nothing.
As much as I recall, 0 is the default.
You may want to reduce the log level as needed. 2 has been a good compromise for me.
On a Raspi you can edit the file with
This is defined in the file /opt/z-way-server/config.xml
There should be a line
<log-level>2</log-level>
The numeric value determines the amount of information. 0= log all, 9 = log almost nothing.
As much as I recall, 0 is the default.
You may want to reduce the log level as needed. 2 has been a good compromise for me.
On a Raspi you can edit the file with
Code: Select all
sudo nano /opt/z-way-server/config.xml
Re: Z-way-server occasionally goes down
And install logrotate (should be installed by default and from v4.0.2 it is installed via dependency). It will remove older logs and keep your folder reasonable
Re: Z-way-server occasionally goes down
I have implemented another idea: Put the whole z-way-server folder and the log onto an additional USB drive.
I think a good USB drive will last longer than a SD card. And if it fails, it can be exchanged much easier.
I think a good USB drive will last longer than a SD card. And if it fails, it can be exchanged much easier.
Raspberry Pi 3 Model B Rev 1.2
Raspbian GNU/Linux 10 (buster, 32bit)
RaZberry by Z-Wave.Me ZW0700 7.20.00 07.38/1766938484 1025/257
Z-Way version v4.1.2 from 2023-10-18 03:34:26 +0300
Raspbian GNU/Linux 10 (buster, 32bit)
RaZberry by Z-Wave.Me ZW0700 7.20.00 07.38/1766938484 1025/257
Z-Way version v4.1.2 from 2023-10-18 03:34:26 +0300
-
- Posts: 16
- Joined: 15 Jan 2023 19:08
Re: Z-way-server occasionally goes down
@JohannesF : thanks
Re: Z-way-server occasionally goes down
I had a power outage this morning, so my test session was unfortunately abruptly terminated after ~40 hours of uptime
Until that happened, I have not been able to detect any stability issues. I will restart things tonight and carry on testing.
Until that happened, I have not been able to detect any stability issues. I will restart things tonight and carry on testing.
Re: Z-way-server occasionally goes down
That's why I'm moving my systems to CM4 modules on a carrier board that supports PoE and has a UPS functionality built-in to the carrier board.
https://www.waveshare.com/wiki/CM4-POE-UPS-BASE