Server is constantly crashing

Discussions about RaZberry - Z-Wave board for Raspberry computer
pierre2302
Posts: 121
Joined: 15 Oct 2013 19:04

Re: Server is constantly crashing

Post by pierre2302 » 29 Oct 2017 23:25

I have no proof, put apart my experience!

An SD card is not made to collect as much reading and writing data. they end up becoming unstable.

https://raspberrypi.stackexchange.com/q ... partitions
https://raspberrypi.stackexchange.com/q ... durability
https://www.raspberrypi.org/forums/view ... &start=175
https://github.com/home-assistant/home- ... ssues/7400

That's why I recommend using an external hard drive.

After you do what you want. Me anyway I have more problem of crash since I installed an external hard drive.

it's worth a try for those with stability issues.
" Raspi 2 (RaZberry) / Raspi B (Razberry) " With OpenRemote Free 2.2.0_TTS-Email-Serial and Z-Way

kpanchen
Posts: 20
Joined: 08 Sep 2017 05:27

Re: Server is constantly crashing

Post by kpanchen » 30 Oct 2017 10:39

pierre2302 wrote:
29 Oct 2017 23:25

That's why I recommend using an external hard drive.

After you do what you want. Me anyway I have more problem of crash since I installed an external hard drive.

it's worth a try for those with stability issues.
So you do or you don't? What hard drive did you use?

pierre2302
Posts: 121
Joined: 15 Oct 2013 19:04

Re: Server is constantly crashing

Post by pierre2302 » 30 Oct 2017 13:08

I use a hard drive classic 2.5 "powered by raspberry.By cons it is thought to increase the power of USB port.

edit config.txt file
Add the following line "max_usb_current = 1" or change its value to 1 if it already exists.
save then reboot
" Raspi 2 (RaZberry) / Raspi B (Razberry) " With OpenRemote Free 2.2.0_TTS-Email-Serial and Z-Way

kpanchen
Posts: 20
Joined: 08 Sep 2017 05:27

Re: Server is constantly crashing

Post by kpanchen » 03 Nov 2017 03:06

After a few day running fine I think I've finally got what we're looking for:

Code: Select all

[New Thread 0x6f6ff450 (LWP 27681)]
[New Thread 0x710ff450 (LWP 27682)]
[Thread 0x710ff450 (LWP 27682) exited]
[Thread 0x6f6ff450 (LWP 27681) exited]
[New Thread 0x6f6ff450 (LWP 27811)]
[New Thread 0x710ff450 (LWP 27812)]
[Thread 0x710ff450 (LWP 27812) exited]
[Thread 0x6f6ff450 (LWP 27811) exited]
*** Error in `/opt/z-way-server/z-way-server': free(): invalid next size (normal): 0x726b58c8 ***

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x72eff450 (LWP 6615)]
0x76391f70 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56      ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x76391f70 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x76393324 in __GI_abort () at abort.c:89
#2  0x763cd954 in __libc_message (do_abort=<optimized out>,
    fmt=0x764836e8 "*** Error in `%s': %s: 0x%s ***\n")
    at ../sysdeps/posix/libc_fatal.c:175
#3  0x763d3b80 in malloc_printerr (action=1,
    str=0x764838a4 "free(): invalid next size (normal)", ptr=<optimized out>)
    at malloc.c:4996
#4  0x763d4b24 in _int_free (av=av@entry=0x73000010, p=<optimized out>,
    p@entry=0x726b58c0, have_lock=have_lock@entry=1) at malloc.c:3840
#5  0x763d705c in _int_realloc (av=av@entry=0x73000010,
    oldp=oldp@entry=0x726b5278, oldsize=oldsize@entry=808, nb=nb@entry=1608)
    at malloc.c:4366
#6  0x763d86f4 in __GI___libc_realloc (oldmem=0x726b5280, bytes=1600)
    at malloc.c:3029
#7  0x76856dd8 in xmlXPathCompExprAdd () from ./libs/libxml2.so.2
#8  0x7686e120 in xmlXPathCompileExpr () from ./libs/libxml2.so.2
#9  0x768783c4 in xmlXPathEvalExpr__internal_alias () from ./libs/libxml2.so.2
#10 0x768788b4 in xmlXPathEvalExpression__internal_alias ()
   from ./libs/libxml2.so.2
#11 0x76fafb28 in ?? () from ./libs/libzcommons.so
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) info thr
  Id   Target Id         Frame
* 10   Thread 0x72eff450 (LWP 6615) "zway/core" 0x76391f70 in __GI_raise (
    sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
  9    Thread 0x73aff450 (LWP 6614) "zway/timers" 0x76403360 in nanosleep ()
    at ../sysdeps/unix/syscall-template.S:81
  8    Thread 0x742ff450 (LWP 6613) "zway/webserver" 0x76403360 in nanosleep ()
    at ../sysdeps/unix/syscall-template.S:81
  7    Thread 0x74e73450 (LWP 6610) "zway/core" 0x76403360 in nanosleep ()
    at ../sysdeps/unix/syscall-template.S:81
  6    Thread 0x74e83450 (LWP 6609) "v8:SweeperThrea" 0x76986a40 in do_futex_wait (isem=isem@entry=0x64784) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:48
  5    Thread 0x74e93450 (LWP 6608) "v8:SweeperThrea" 0x76986a40 in do_futex_wait (isem=isem@entry=0x6467c) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:48
  4    Thread 0x74ea3450 (LWP 6607) "v8:SweeperThrea" 0x76986a40 in do_futex_wait (isem=isem@entry=0x64574) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:48
  3    Thread 0x74eb3450 (LWP 6606) "v8:SweeperThrea" 0x76986a40 in do_futex_wait (isem=isem@entry=0x6446c) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:48
  2    Thread 0x756b3450 (LWP 6605) "OptimizingCompi" 0x76986a40 in do_futex_wait (isem=isem@entry=0x64324) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:48
  1    Thread 0x7634b000 (LWP 6602) "z-way-server" 0x76403360 in nanosleep ()
    at ../sysdeps/unix/syscall-template.S:81
(gdb)
Please let me know if I can do more / something else. For now I will restart server with Monit.

User avatar
PoltoS
Posts: 4215
Joined: 26 Jan 2011 19:36

Re: Server is constantly crashing

Post by PoltoS » 03 Nov 2017 04:06

Thanks! This will be a hard one - this is signature for a memory corruption error. This means it would be not that easy to find who corrupted it, becuase it was in some code before this particular point.

What was in the log just before? Looks like it was parsing something, had to look in the XML files and then touched memory corrupted by something else.

Do you have names in UTF8 non-ASCII chars?

Finally, is it v2.3.6?

kpanchen
Posts: 20
Joined: 08 Sep 2017 05:27

Re: Server is constantly crashing

Post by kpanchen » 03 Nov 2017 08:29

I've attached log extract, have a look. Crash happened at 11:50, seems like for whatever reason server cannot send commands to a z-wave stack and eventually crashed. I've recovered at about 20:08.

I shouldn't have any non-Latin characters and nothing should be in UTF8. And yes, it is the latest version. If you want I can keep running server in debug mode, can we get more info out of that?

Thanks,
K.
Attachments
z-way-server_extr.zip
Log extract
(353.54 KiB) Downloaded 71 times

User avatar
PoltoS
Posts: 4215
Joined: 26 Jan 2011 19:36

Re: Server is constantly crashing

Post by PoltoS » 04 Nov 2017 01:33

Ok, I see. The crash happens on XML reading during command parsing indeed. But the memory corrupted was somewhere else.

Could you please send me by mail or PM the backup of your Expert UI to let me try interpret the log and reproduce the problem?

jet11x
Posts: 46
Joined: 29 Dec 2014 21:15

Re: Server is constantly crashing

Post by jet11x » 22 Jan 2018 00:21

I attach the details of the crash that I've just caught on my system, with gdb running as suggested.

I'm using version v2.3.6, build 28-09-2017 15:29:18.

I've had a few crashes since moving to v2.3.6, for this one and the other crash where I've checked the log file, the last line was an http request.
I couldn't use these prior to v2.3.6 as earlier version couldn't handle chunked data.

Hope the supplied information helps isolate the problem.
Please let me know if there's more information I can provide.
Attachments
crash-21-Jan-18.tar.gz
(1.11 KiB) Downloaded 64 times

User avatar
PoltoS
Posts: 4215
Joined: 26 Jan 2011 19:36

Re: Server is constantly crashing

Post by PoltoS » 22 Jan 2018 01:38

Hm... very interesting one. How long was uptime of z-way before that happened? Do you often do that particular request with similiar XML response?

The server crashed on creating XML object and attaching it to the V8 JS data tree. Never seen that. Reproducible?

jet11x
Posts: 46
Joined: 29 Dec 2014 21:15

Re: Server is constantly crashing

Post by jet11x » 22 Jan 2018 09:24

If just crashed again, but last time it took a few days before crashing.
The request and response is running every minute - capturing the state of my alarm (home/away)

Post Reply