How to push data from a device?

andrew · February 2, 2016, 2:40pm

Hi,

I’m not very familiar with C++ lambda functions, would like some help with understanding the following.

In Thinger Dashboards UI there is an option for a metric to be ‘push from device’. This is also a great fit when I want to control the cycles and e.g. deep sleep the controller only to wake up and post to Thinger.

I don’t think I came across such an example in the documentation, though. Mind posting a quick way to do it?

P.S.: I have extensive software dev experience, just not in advanced c++

Andrew

alvarolb · February 2, 2016, 3:37pm

Hi Again Andrew! Good to know you are testing all thinger features!

There is simple example in the documentation, take a look to this code:

http://docs.thinger.io/arduino/#coding-streaming-resources

In this case the device is streaming the information when it considers that is necessary (i.e. a significative change). So it can be a similar approach in your case… send the information when it wakes from the deep sleep. This kind of push should be done inside the loop.

Hope that helps!

andrew · February 2, 2016, 5:28pm

I must be missing something basic.

// in setup()
{
thing[THINGER_RESOURCE_SOIL] >> [] (pson& out) {
    out = getSoilMoisture();
  };
}

void loop() {
  thing.handle();

  int soilMoisture = getSoilMoisture();
  bool done = thing.stream(thing[THINGER_RESOURCE_SOIL]);
  Serial.print("Push data ");
  Serial.println(done ? "ok" : "failed");

  Serial.println("Going to deep sleep");
  ESP.deepSleep(INTERVAL_MS * 1000);
}

I’m always seeing the stream failed:

[THINGER] Authenticated!
Reading: 33
Push data failed
Going to deep sleep

I did read that streams require ‘the other side’ to be listening to work, but not sure if I understood it correctly. There is a dashboard which had widgets reading (push by device) the metric. Isn’t this enough?

Maybe semantics of stream() are subtly different. Does it make sense to have a simple send()/push() in that case?

Thanks in advance!

andrew · February 2, 2016, 5:56pm

I noticed that my issues could be coming from a server assuming a client’s state and having a lag. If I don’t do deep sleep, it will eventually start working (although take like 10 attempts first).

If there’s a dependency like this, I’m not sure stream() is the right option. Is there a way to send a metric from a device to the server and disconnect?

alvarolb · February 2, 2016, 6:56pm

You are right, if the stream method is returning false, it means that there is no endpoint listening for this information. It may be caused by the deep sleep feature, that disconnects the device, the dashboards get disconnected also (a thumb down may be shown in the widget), and when you connect again the device, the dashboards will take some time to reconnect to the stream, but your device is going to sleep again… so it won’t work. All this stuff is more related to real-time communication where the connection is open.

So the current thinger version is not covering your use case, where you need to store a value periodically after a sleep period. This is a planned feature, where the user will be able to create data buckets and push data to them. In this case it will be easier to recover from a long sleep, get connected, push data, and sleep again.

What is your current sleep interval? Just to get an a idea of what is happening. To be sure there is no problem with the sleep method, we can try to set a ridiculous interval like 10 seconds, to see if the data is updated every 10 seconds.

EDIT: Now this feature is avaiable, check the posts below

andrew · February 2, 2016, 7:35pm

Thanks for confirming, Alvaro. While testing, the deep sleep interval is e.g. 10-30 seconds. The intent is for it to be in minutes later. Or, in case of a soil moisture, once an hour or so.

Let me know if I can help with any of the upcoming feature testing (am very curious about what you’re doing with Docker there )

alvarolb · February 2, 2016, 7:50pm

OK, will check what happens with the deep sleep with 10-30 seconds, as it should be working. Do you know if the RTC is running while the deep sleep is on? i.e. the millis() function is increasing properly?.

I will inform you as soon as I get some advance in the bucket feature. Docker is not being used at this moment, but it will be used for releasing easy to deploy server packages, as the thinger server is built completely in C++ and has some big dependencies like boost, openssl, CrytpCPP, MongoDB, etc., and compiling can be a mess. So docker will provide ready to run server packages.

andrew · February 2, 2016, 8:00pm

To the best of my knowledge, RTC continues to run in deep sleep for esp8266. Although it’s more of a simple timer, as it doesn’t know the day of week, etc. The chip shuts off most of the power and connections, and restarts with a boot (shortened, though), invoking setup() and loop(). For the sketch code it’s as if it were running from a cold boot, but there are boot flags which can be queried to understand context.

shean · January 17, 2017, 9:34pm

Hello -
Is the feature of being able to Push to the data bucket now available?

I would like to be able to locally store say 15 sets of one minute sensor readings with time/date stamp, then send these readings to a Thinger bucket at 15 minute intervals.

yahya_khaled · May 21, 2017, 2:16pm

Hi all, has there been any solution for this issue? I have a similar requirement. I’m trying to operate the ESP8266 on batteries so I have no choice but to use deep sleep feature. The use-case is temperature sensing every 5 minutes interval. Data going into the buckets are either server initiated or streamed from device. The first wont work with sleep mode and the second require RTC. Is there a solution?

shean · May 21, 2017, 3:09pm

I stream Temp & Humidity from the ESP8266 to Thinger using deep sleep.
I don’t use a RTC, just use the ESP.deepSleep(timesetting, WAKE_RF_DEFAULT) function to set the time interval for wakeup.
For me it’s accurate enough.
Thinger timestamps the data when it enters the Bucket.
If the interval timing is not critical, this works pretty well.

yahya_khaled · May 22, 2017, 3:04am

Hello Shean, can you kindly post a working sketch?
Most Thinger functions are call back such as in/out. How are you posting the data to the bucket if you are not streaming?

My current code:

void setup() {
  Serial.begin(115200);
  Wire.begin();
  delay(200);
  thing.add_wifi(SSID, SSID_PASSWORD);

 thing["temperature"] >> [](pson& out){     
    out = getTemperature();
  };
}

void loop() {
  thing.handle();
  thing.stream(thing["temperature"]);
  Serial.println("Going to deep sleep");
  ESP.deepSleep(1000000 * 30, WAKE_RF_DEFAULT); // 60 seconds
  delay(100);
}

I set my bucket to be updated from the device and configured to the same resource “temperature”:

My debug:

alvarolb · May 22, 2017, 7:48am

Take a look on this blog post: https://thinger.io/new-features-on-data-buckets/
Or this other doc: http://docs.thinger.io/hardware/climaStick/#quickstart-examples-data-recording-using-sleep

(We need to update the documentation in Arduino docs)

But it is basically like this:

void setup() {
  Serial.begin(115200);
  Wire.begin();
  delay(200);
  thing.add_wifi(SSID, SSID_PASSWORD);

 thing["temperature"] >> [](pson& out){     
    out = getTemperature();
  };
}

void loop() {
  thing.handle();
  thing.write_bucket("YourBucketId", "temperature");
  Serial.println("Going to deep sleep");
  ESP.deepSleep(1000000 * 30, WAKE_RF_DEFAULT); // 60 seconds
}

You need also to configure your bucket, and set Data Source as From Write Call to receive data in this way.

Notice that the stream does not work in this case, as for streaming the device requires a subscription on the resource, i.e., from the bucket, a dashboard, a websocket, etc. However, the subscription takes some loops to be completed, and we are sleeping the device before the subscription completes. The write_bucket function does not require a subscription, so it push the information directly to the bucket you specify in the call. Hope this explanation makes sense.

Best.

yahya_khaled · May 22, 2017, 8:09am

Thank you Alvarolb - you are always precise and helpful! I didn’t know about this feature.

shean · May 22, 2017, 8:09am

Hi,
Below is my code.
My sketch works as per Alvaro’s post above, using the from Write Call command for Bucket data source setting:

#include <SPI.h>
#include <ESP8266WiFi.h>
#include <ThingerESP8266.h>  // req'd by thinger

// THINGER ID LOGIN CREDENTIALS
#define USERNAME "name"
#define DEVICE_ID "ID"
#define DEVICE_CREDENTIAL "pw"
ThingerESP8266 thing(USERNAME, DEVICE_ID, DEVICE_CREDENTIAL);

// LAN WIFI CREDENTIALS
#define SSID "ssid"
#define SSID_PASSWORD "pw"

// SET A TIME PERIOD TO UPDATE TEMP AND HUMID READINGS
#define SLEEP_MS        60000 // 60,000 milliseconds FOR DEEPSLEEP MODE


// DHT11 START
#include "DHT.h"
float temperature;
float humidity;
#define DHTPIN D2     // ESP8266 pin number for DHT11 data pin
#define DHTTYPE DHT11   // DHT 11
DHT dht(DHTPIN, DHTTYPE);
// DHT11 END

// SETUP START
void setup() {

//// TEMP HUMIDITY DEVICE
  dht.begin();
//// END TEMP DEVICE
  
//// WIFI STUFF  
  Serial.begin(115200);
  Serial.println("Booting");
  WiFi.mode(WIFI_STA);
  WiFi.begin(SSID, SSID_PASSWORD);
  while (WiFi.waitForConnectResult() != WL_CONNECTED) {
    Serial.println("Connection Failed! Rebooting...");
    delay(5000);
    ESP.restart();
  }
//// WIFI STUFF END  

////  THINGER START
 thing.add_wifi(SSID, SSID_PASSWORD);
   thing["TempHum"] >> [](pson& out){ 
         out["temperature"] = dht.readTemperature();
         out["humidity"] = dht.readHumidity();
     };
//// THINGER END
  
}  
////  SETUP END

//// TEMP HUMIDITY SUBROUTINE (not req'd except for serial print)
void TempHum(){
     humidity = dht.readHumidity();
     temperature = dht.readTemperature();
  // Check if any reads failed and exit.
  if (isnan(humidity) || isnan(temperature)) {
    Serial.println("Failed to read from DHT sensor!");
    return;
  }
  Serial.print("Humidity: ");
  Serial.print(humidity);
  Serial.print(" %\t");
  Serial.print("Temperature: ");
  Serial.print(temperature);
  Serial.println(" *C ");
}
//// END TEMP HUMIDITY SUBROUTINE

void loop() {
  thing.handle();
  thing.write_bucket("Temp_Hum_Buck", "TempHum");
  ESP.deepSleep(SLEEP_MS*1000, WAKE_RF_DEFAULT); 
  
}

alvarolb · May 22, 2017, 9:32am

Nice you got it working @yahya_khaled

And @shean, you can remove the following code, as this is also handled by the thinger.io libraries.

yahya_khaled · May 22, 2017, 11:54am

Thank you!

yahya_khaled · May 22, 2017, 12:23pm

Frankly I had cases ESP8266 doesn’t reconnect after rebooting the router. I’m not sure if its L1, L2 or L3 problem. Next time it happens I will trouble shoot it.I’m using the library as is.

Handy_ID · March 6, 2020, 3:53pm

Thanks, but how if I want to create data bucket with multiple data from dashboard’s data every 15 minutes or other sampling time?

Handy_ID · March 6, 2020, 4:00pm

Refers to code below, how can I change the requires_recording to record data every 15 minutes? Give some resource code please. Thanks

void loop() {
thing.handle();
// use your own logic here to determine when to stream/record the resource.
if(requires_recording){
thing.stream(“TempHum”);
}
}