IOTMP: how to set new value for a resource, by device (like a publish)

Client Libraries
1

Hello,

I am using an esp32 with esp-idf framework and thinger.io latest iotmp library taken from github.

My device is a sort of automatic gate which makes some move (open/close) 3-4 times a day. When a move is done, a bunch of statistic data are generated (ex. maximum current drawn, maximum temperature reached and things like that); those statistics are to be collected, but does not make sense to collect them “every 15 minutes” or similar.

My aim is to initiate a refresh of those resources (statistics) when the move is done. For example, I declared a resource like this:

thing["openclose"] = [](thinger::iotmp::output& out) {
    out["value"] = st_aprchi;
};

For what I undestand, “openclose” is the name of the resource, and the resource can contain a few fields - I used just once for now, a numeric value 2=closed, 1=opened, 0=unknown.

What I want is that, when the device completes a move (commanded locally, not remotely), the new status is published suddenly, but I don’t see a way to do it. I prepared a data bucket:

image

and set the refresh mode as “stream by device”. But how can the device start to stream this resource? And, “stream” is not the proper term for what I want to do: I want to update the value just few times a day, it’s more like an event to say “the status has changed” or a “publish” (event+value). I tried to force a refresh using:

thing.stream(“openclose”);

but it doesn’t seem to work. Also, I’ve seen a similar (old) message on this very forum (Unsuccessfully trying to make dashboard snappier), but didn’t understand much and, while the author addressed exactly my same problem, it seems a different conclusion came out.

Thank you in advance.

2

Hi

If you want to save the when the movement is done, I would recommend to use the “write_bucket” instruction and control when is active this routine and how often the data is sent.

The bucket’s datasource should be “From device write call”.

Btw how long you want to save the data? or you want to save just one dataset when the movement is done? you need to control this into the loop.

BR

3

Thank you for reply.

…when the movement is done, … use the “write_bucket” instruction …

Ok. This is not exactly what I meant, but is doable. The problem with this approach is logical: the device knows everything about its resources (resource=an internal value), but should not need to know anything declared outside (i.e. the name of a data bucket in the cloud).

I can think about this mechanism as an “event”, and it probably solves the problem of data buckets - I will try.

The bucket’s datasource should be “From device write call”.

Ok. Now I see that, doing so, “the cloud” (my customer) can no more decide what data to acquire because everything is decided by the device, which can send anything - included values that are not resources at all. I have to check with my customer, as he will not have the freedom to choose what to put in his bucket.

Btw how long you want to save the data? or you want to save just one dataset when the movement is done? you need to control this into the loop.

I don’t grasp it. The situation is this: my customer will install a bunch (100? 1000?) of systems, and collect data to understand how the systems work in the field (things like “how much the system heats during the manouver”? What are the power consumptions in summer and in winter? How much time takes a manouver to complete?) and so on. Each time a manouver is done, the device has fresh data to send to the cloud. There can be 1-2 manouvers a day, and a manouver takes 1-2 minutes to complete. This is the classic situation where a polling has no sense, instead a push from the device is the best thing.

So, the reply to your question is “I want to save a dataset every time a movement is done”. I must add that I have not “the loop” - I use esp-idf and not Arduino, so I don’t have a loop, I believe that the thinger IoTMP library creates its own task and I have no control over that, and the documentation is sparse.

My last question is about dashboard. A widget can show a value from device, and update itself either by polling or by “stream by device”. What does “stream by device” mean? What should I write in the device firmware to “stream” something? The docs say something about this, but they are for Arduino. I can’t make a widget to refresh in real time, what is the correct way to do that?

Thanks in advance, regards,
linuxfan

4

Hello,

I tried to create a data bucket like this:

image
image334×648 16 KB

My device, when a move terminates, invokes this code:

bool ok = thing.write_bucket(
	"lastmove",
	thinger::iotmp::json_t {
		{"state",     st_aprchi},
		{"stopcause", st_stopcause},
		{"duration",  st_cyc_man_duration / 4},
		{"minforce",  st_cyc_ampmin / 10.0},
		{"maxforce",  st_cyc_ampmax / 10.0},
		{"maxstep",   st_cyc_maxstep},
		{"maxtemp",   conv_temperature(st_max_temperat)}
	}
);

if (!ok)
	ESP_LOGE("ERROR", "write_bucket failed");

What happens is that after a pause of a few seconds, iotmp disconnects, the write fails, and then iotmp reconnects. This is the log from the esp32 device:

I (2754714) APP: *** Cover state now 1
at this point the write_bucket() is invoked
…after a few seconds…
I (2767994) iotmp: Disconnected
E (2767994) ERROR: write_bucket failed
I (2767994) APP: v0.81 ready SFCR9vbGYpt4pN, ble=NO, cloud=yes
I (2767994) iotmp: State: CONNECTING
I (2767994) iotmp: Connecting to iot.thinger.io:25204
…
I (2768334) iotmp: State: AUTHENTICATED
…

What is wrong? TY.

5

Hi,

Oh I understand, I havent tested the bucket subscribed to a streaming resource, I can try and share my experience.

Note that the dataset must be consistent, I mean, sending some random datasets may cause unwanted behaviour, the recommendation is to send the same variables.

Would love to give better advice regarding espidf coding but im not experienced in that environment, what I 've seen is that they declare a while(1){} or a for(;;){} to force a infinite loop, but you need to give some free time to avoid the watchdog to restar the uC.

Maybe it is related with the way as you have your espidf code, I would recommend to apply the infinite loop emulator for testing purposes, and dont call the stream instruction more than 1time/sec, the uC may run a instruction thousands times/sec, but for sure it cannot be streamed this fast so for user better experience and uC estability 1stream/second is a good start.

I havent seen code like yours to define a resource into thinger environment, I do it by this way

      pson data;
      data["Val_1"] = value1;
      data["Val_2"] = value2;

      thing.write_bucket(bucketId, data);

I would recommend you to code by this way, although you can define the whole resource as function argument is easier to handle have it defined as a variable and then invoque into the “write_bucket” function

Btw note that the thing.write_bucket(... instruction must have a parameter to confirm the result of the operation, your example misses the 3rd argument to enable the confirmation of the operation.

And remember that the communication task does not run at the line of the command, the write_bucket (or any communication instruction) will set a flag that will run on the following thing.handle(); call, I havent worked with esp idf but I don’t know if this instruction is running in between the bool ok... declaration and the following if(!ok)

6

I havent seen code like yours… thing.write_bucket(
“lastmove”,
thinger::iotmp::json_t {
{“state”, st_aprchi},

The problem is not here. This style of coding is taken from readme.md which makes this example:

…
// Data buckets
thing.write_bucket(“sensor_data”, json_t{{“temp”, 23.5}});
…

Moreover, I tried anyway to construct a json outside the call, like you suggest: it does the same.

note that the thing.write_bucket(... instruction must have a parameter to confirm the result of the operation, your example misses the 3rd argument…

No, the esp-idf has not that 3rd argument.

Finally: I examined the library I am using (taken from here: GitHub - thinger-io/IOTMP-ESPIDF: IOTMP Client Library for ESP-IDF Framework · GitHub), and also compared to the Arduino one. They are quite similar, but different - they even declare similar identifiers with different values. I’ve even put some debug logging here and there in the library, all I’ve seen is that write_bucket() sends data to the server, and the server never replies.

I give up. The guy that wrote this library should take a look at it, I suppose, but I don’t know what to do about this. The library also contains some “test code”, but that test code does not tests buckets, it seems. So actually maybe that that function has never been tested.

Thank you for your effort, I’ve appreciated it.

Regards,
linuxfan

7

You’re right, I just checked the ESP-IDF and the library has some differences compared to the Arduino version.

However, I find it very strange that the server isn’t doing anything.

If you want to give it a try, I recommend checking the “Inspector” tab on the device page. This will give you information about what the server is listening from the device, not always the device end provide useful information.

BR

8

I tried the inspector page, it shows nothing. And this seems quite normal, if it is true that the data never arrives to the server. The write_bucket() function sends just an arbitrary json payload, and the server should simply store that single json, there is no filter, no other setting which could be wrong.

The json given to write_bucket() should be correct, because the other functions like multivalue resource also use json and do work. I think that write_bucket() generates a malformed packet the server refuses or ignores; a clear symptom of something going wrong is that after write_bucket() the connection breaks (the log shows that) and the library reconnects (and the log shows that too).

The library must contain a bug. Should I post another topic clearly stating I’ve [think I’ve] found a bug?