V4.1.2: steadily increasing cpu load

[seattleneil]

Now I have a question of understanding: do you think, that the http.request() command is buggy?

Yes. The code for http.request() is in /opt/z-way-server/modules/modhttp.so. Since it is compiled code and Z-Wave.Me has not made the source code available on github (see: https://github.com/Z-Wave-Me), users wlll have to either wait for a fix, live with high CPU load or adopt a workaround. So far, the workaround previously described in this thread of using a "Virtual Device (JavaScript)" has solved my CPU load problem.

[PoltoS]

Thank you all for this deep analysis. We have checked and there are only small changes in this app since 3.2.3: only one lock added. I don't think it can affect the performance that high. In addition, we can't see it on our side and as I got, not all users do experience this issue.

It might be related to the name resolution (direct IP is mentioned or DNS resolution is needed to resolve a name) and LAN v.s. Internet HTTP requests target.

Please help us to collect more info: users who do and don't experience it, what are your HTTP requests:
- I do / don't experience the issue
- IP or DNS
- HTTP / HTTPS / WS / WSS
- to a service at localhost, LAN, Internet

[seattleneil]

Please help us to collect more info: users who do and don't experience it, what are your HTTP requests:
- I do / don't experience the issue
- IP or DNS
- HTTP / HTTPS / WS / WSS
- to a service at localhost, LAN, Internet

I do experience the issue, URL uses DNS, request is HTTP GET with no auth, LAN.

Two observations: (1) an HTTP device that is a reliable localhost service does not appear to cause an increase in CPU usage. (2) Periodic polling of an HTTP device that is slow to respond (or doesn't respond) appears to cause an increase in CPU usage.

Happy hunting!

[piet66]

Hi PoltoS,
I would very much welcome it if the problem could be solved soon.

What do you think of the following idea:

I'm currently on 3.2.3 and have no increasing CPU load. Since there are only small changes in modhttp.so, can I copy the 4.1.2 version of that file into the 3.2.3 folder and restart the zway server? Would that work?

If I then have increasing CPU load, this would prove the suspicion, if not then the reason must lie elsewhere.

[aLiEnHeAd]

Please help us to collect more info: users who do and don't experience it, what are your HTTP requests:
- I do / don't experience the issue
- IP or DNS
- HTTP / HTTPS / WS / WSS
- to a service at localhost, LAN, Internet

- I do experience the issue
- IP within my LAN
- HTTP GET and parsing the response (JSON.parse($$)[0].value.state?'on':'off')

[PoltoS]

I'm currently on 3.2.3 and have no increasing CPU load. Since there are only small changes in modhttp.so, can I copy the 4.1.2 version of that file into the 3.2.3 folder and restart the zway server? Would that work?

Thank you, piet66, for this idea. Yes, it was tested that it will work, so just copy the modhttp.so and let us know the result.

[PoltoS]

Thanks to @aLiEnHeAd, let's add a few more:
- I do / don't experience the issue
- IP / localhost / DNS
- HTTP / HTTPS / WS / WSS
- to a service at localhost, LAN, Internet
- parsed in XML / JSON / as text / no parsing of the result
- sync / async / don't know (is there an async = true in the http.request ?)

[seattleneil]

Thanks to @aLiEnHeAd, let's add a few more:

The HTTP Device app index.js sets async=true. This is not configurable from the app's web UI.

Parser for on/off status: JSON.stringify($$).split('"')[3] === 'ON' ? 'on' : 'off'

[piet66]

I'm currently on 3.2.3 and have no increasing CPU load. Since there are only small changes in modhttp.so, can I copy the 4.1.2 version of that file into the 3.2.3 folder and restart the zway server? Would that work?

Thank you, piet66, for this idea. Yes, it was tested that it will work, so just copy the modhttp.so and let us know the result.

Done it. It seems to work. Will observe it.

Thanks to @aLiEnHeAd, let's add a few more:
- I do / don't experience the issue
- IP / localhost / DNS
- HTTP / HTTPS / WS / WSS
- to a service at localhost, LAN, Internet
- parsed in XML / JSON / as text / no parsing of the result
- sync / async / don't know (is there an async = true in the http.request ?)

- I do experience the issue in 4.1.2
- fixed IP address
- HTTP
- to a local service within in the same server
- parsed in JSON
- async

And here for the sake of completeness the code (from module MxChartDB):

Code: Select all

    var request = {
        url:     url,
        method:  'POST',
        auth: {
            "login":    self.constants.username,
            "password": self.constants.password
        },
        data:    JSON.stringify(data),
        async:   true,
        success: function(response) {
                    ...
                 },
        error:   function(response) {
                    ...
                 }
    };
    http.request(request);

Code: Select all

    var request = {
        url:     url,
        method:  'GET',
        auth: {
            "login":    self.constants.username,
            "password": self.constants.password
        },
        async:   true,
        success: function(response) {
                    ...
                 },
        error:   function(response) {
                    ...
                 }
    };
    http.request(request);

[piet66]

I'm currently on 3.2.3 and have no increasing CPU load. Since there are only small changes in modhttp.so, can I copy the 4.1.2 version of that file into the 3.2.3 folder and restart the zway server? Would that work?

Thank you, piet66, for this idea. Yes, it was tested that it will work, so just copy the modhttp.so and let us know the result.

Done it. It seems to work. Will observe it.

After some days I can report the result of my experiment:
it needs slightly more CPU, but I cannot recognize any growing of the load.

This would indicate that modhttp.so is not the culprit.

For a complete proof, it may actually make sense to go additionally the other way round: running the 3.2.3 version in 4.1.2. If it's possible.