Value in the data bucket from esp8266 is too much

Mzgnwbsn · December 15, 2020, 12:24pm

My system is read a laboratorium room temperature and humidity with thinger io and save the data for a documentation, which is the thinger io does have all to do that.

I have some question about feature that involved in thinger.io and I need to be answered ASAP

When I uploaded the value of the temp and hum, the value on the data bucket is very lot like “27, 4000348282” How do I eliminate the numbers and make it just “27, 4” In data bucket view? Because I already change the script and it does nothing, still the same.
How can I export the bucket data with a date range, and form it to the time graph? Is there any way to do that?
Why the value in the data bucket having a string type of data? Why not integer or float or something with a numerical type of data? Because when I export it to a csv file and try to proccess the data in excel on own, the data is not changing. Because I think it’s a string or text not an integer or float.

I’m very glad if you could help me ASAP because I already search it in Google but I can’t find what I need for my problem.
Thank you and have a nice day

ega · December 15, 2020, 2:27pm

Hi I will try to answer you the same order you asked the questions:

The value’s decimals should be truncated at the microcontroller before it is reported to the cloud, the variable is a float and it will use all its memory asignated to store the number, you need to truncate the decimal positions, when you say that you changed the script, what do you mean?.
Export data and graphic are different situations, you may have a dashboard to show a graphic dislaying the evolution of certain variable, if you export the data to visualize in other program or platform is different.
The format is exported in .CSV, it is basically a text document, none of the fields and/or values has a data type established, when it is read it in an spreadsheets program (as MS Excel, for example) the program gives each field a particular format in the way it is configured to interpretate the file (for example to take the character to separate the fields as “,” or “;”, and for decimals separation if it is a “.” or a “,”).

Any aditional doubt, let us know.

Mzgnwbsn · December 16, 2020, 1:06am

thank you for the answer, but could u help me about the truncating the number?

float p = 3.1415926;
float previousTemp = -100.0;
float tempC = 0;
float previousHum = -100.0;
float Hum = 0;
float suhu;
float humi;
const char* ssid1 = “hotspott”;
const char* pass1 = “12345678”;
DHTesp dht;

void setup()
{
Serial.begin(115200);
tft.init(240, 240, SPI_MODE2);
dht.setup(16, DHTesp::DHT22);
Display();
connecttoWifi();
thing[“suhu”] >> [] (pson & out) {
out[“Suhu”] = suhu;
out[“Kelembapan”] = humi;
};
}

void loop()
{
thing.handle();

TempAndHumidity measurement = dht.getTempAndHumidity();
previousTemp = tempC;
tempC = measurement.temperature;
previousHum = Hum;
Hum = measurement.humidity;
suhu = roundf(tempC100)/100;
humi = roundf(Hum100)/100;
Serial.print("Temperature: ");
Serial.println(tempC);
Serial.print("Humidity: ");
Serial.println(measurement.humidity);
if(previousTemp!=tempC )
{
deletePreviousTemp();
Finaltemp();

if(previousHum!=Hum){
deletePreviousHum();
FinalHum();
}

i already tried to use round trunc etc but still the same, the output that written in databucket still a lot of decimal number

ega · December 16, 2020, 1:27am

Please paste the sketch as preformatted text, after pasting it, select all and press the “</>” button in the toolbar to give the code format, is easier to read it.

For truncate the decimals I use this function:

float decimalPositions( float in_value, int decimal_place )
{
  float multiplier = powf( 10.0f, decimal_place );
  in_value = roundf( in_value * multiplier ) / multiplier;
  return in_value;
}

You should give the number to truncate and the decimal places you want, it should return it in that way

Hope this helps.

Mzgnwbsn · December 16, 2020, 12:04pm

thanks for the advice about the code format, because im new at this community.

i already tried the function that you adviced up there, but the result in the data bucket is still same.
am i write the code wrong? or it is just not work?

void setup()
{
  Serial.begin(115200);
   tft.init(240, 240, SPI_MODE2); 
  dht.setup(16, DHTesp::DHT22);
  Display();
  connecttoWifi();
    thing["suhu"] >> [] (pson & out) {
    out["Suhu"] = temp;
    out["Kelembapan"] = humi;
  };
}

void loop()
{
  thing.handle();
 
  TempAndHumidity measurement = dht.getTempAndHumidity();
  previousTemp = tempC;
  tempC = measurement.temperature;
  previousHum = Hum;
  Hum = measurement.humidity;
  
  temp = roundss(tempC,2);
  humi = roundss(Hum,2);
  Serial.print("Temperature: ");
  Serial.println(tempC); 
  Serial.print("hasil round: " );
  Serial.println(suhu);
  Serial.print("Humidity: ");
  Serial.println(Hum);
if(previousTemp!=tempC  )
    {
       deletePreviousTemp();
       Finaltemp();

    }
if(previousHum!=Hum){
         deletePreviousHum();
       FinalHum();
}
  delay(2000);
}

float roundss( float in_value, int decimal_place )
{
  float multiplier = powf( 10.0f, decimal_place );
  in_value = roundf( in_value * multiplier ) / multiplier;
  return in_value;
}


void connecttoWifi() {
  WiFi.begin(ssid1, pass1);
  while ( WiFi.status() != WL_CONNECTED ) {
    delay ( 500 );
    Serial.print ( "." );

  }
  Serial.print("\nWifi Connected!\n");
  thing.add_wifi(ssid1, pass1);
}

did i write the code correctly?

for infromation
suhu stands for temperature
kelembapan stands for humidity

ega · December 16, 2020, 1:26pm

That’s weird… it should work, well try this function to see if it works properly (or as we are wanting lol)

float roundss( float in_value, int decimal_place )
{
  float multiplier = powf( 10.0f, decimal_place );
  int aux = in_value * multiplier;
  float out_value = (float) aux / multiplier;
  return out_value;
}

Let me know how it goes.

Mzgnwbsn · December 16, 2020, 4:53pm

hi mr ega, i already tried your script and its still the same.
i confused why?
is it possible that the problem is came from the library?
here is the result at the data bucket

for your info, im using a dfrobot dht22 sensors and i used a library from this site https://wiki.dfrobot.com/DHT22_Temperature_and_humidity_module_SKU_SEN0137

but, is it possible if the value has to be converted first to string and then parse it to what i need, and convert it back to float and after that send it to the data bucket?

if its possible i just dont know how to script it

ega · December 16, 2020, 10:19pm

Yes it is weird, it is not an issue with the library, it is with the calculation in the microcontroller.

The option you are mentioning could work too, I will think how to figure it out.

Mzgnwbsn · December 17, 2020, 4:05am

hi mr ega. the problem is solved thank you for helping me so far.

i convert the float value to the string form using

String Mystr(val, int decimal_place);

and its work absolutely well
here’s the result

Mzgnwbsn · December 17, 2020, 4:08am

but mr ega i have some question about the data bucket, can i asked here? or i have to make a new topics?

ega · December 17, 2020, 5:57am

Well that’s really cool you get the solution!! I surely will apply it too.

Hmmm… I don’t know how related or far is your question from this topic, if you think is close, do it here, if you don’t think so, open a new one

Mzgnwbsn · December 17, 2020, 6:25pm

its acctually the next step after putting the data, its exporting the data to the csv format.
in the another disscussion you answered that the unix timestamp is GMT-0 which cause different timezone when export the data.
and you said that it have to be converted to be readable.
what do you mean about that? and how to convert it?

rin67630 · December 18, 2020, 11:38am

I am not caring with the weird number of decimals.
When you export it e.g. to excel, you set in excel the number of relevant digits you want to see.

ega · December 18, 2020, 1:55pm

The exported timestamp is Unix time, you have to set it to your particular time zone to read it in local time, and of course make it a readable date, because the Unix timestamp is the count in seconds (or you could be milliseconds as thinger uses it) since January 1, 1970 at GMT-0.

Tom_Whiting · October 19, 2022, 10:48am

Hi @Mzgnwbsn ,

Could you post your piece of code where you used String Mystr(val, int decimal_place); to send to the data bucket with 1 decimal place? I’m wanting to do the same thing as you did but I’m not able to get it to work.

Thanks,
Tom

ega · October 19, 2022, 1:59pm

Hi,

String Mystr(val, int decimal_place); is a String type variable declaration, is not a function, it basically takes the ‘val’ and truncates according the ‘decimal places’ argument.

Other method to truncate could be something like

out["var"] = (int) floatVariable; this will truncate the float to its integer part.

if you need two decimals (for example), I would try something like:

out["var"] = (float)((int) 100 * floatVariable) / 100; this will multiply the float * 100, then truncate to its integer part and after that divide it into 100 and return it as a float.

Hope this helps.

alvarolb · October 19, 2022, 5:44pm

Please, do not store numbers as Strings!!

~~String Mystr(val, int decimal_place);~~

Why? For example, if you later want to transform data, i.e., doing mean, max, sum, etc., and other aggregation features available on the platform, they will not work!

Moreover, if you want to compare the value against thresholds or other numeric values… it must be converted before, which is a mess and less efficient.

And… of course, we can always add an option in the bucket view to limit the number of digits to be displayed. Suppose tomorrow you consider that 3 digits is better than 2 or 1… With a String you would require to reflash all your devices.

So… do not use String(val, decimal places) here! If it is a number, use a number representation, either integers of floats/doubles depending on the data type.

Tom_Whiting · October 20, 2022, 10:42am

Thanks @ega,

I tried your suggestion for getting 2dp - out["var"] = (float)((int) 100 * floatVariable) / 100 however when I send to the data bucket, it is still showing as a large full float number e.g. 0.08701171725988388

Tom_Whiting · October 20, 2022, 10:47am

Thanks for the advice @alvarolb. The conversion to string did seem to work to 2dp in the data bucket, however on my graph in the dashboard, the values show with a .2 after the value.

If it’s possible, your suggestion to be able to add an option to select how many decimal places are displayed in the bucket would be a really useful feature.

Tom

ega · October 24, 2022, 10:25pm

Hi,

What @alvarolb recommends is to avoid to send a number as string, because it is stored as string and it takes more resources to operate at the platform, this is a good practice, the other way will work for sure, but is not the more efficient way to store the values.

On the other hand, you may select at the dashboard’s widgets how may decimals positions you want to see, so it will store as much as it can at the bucket, but the dashboard will show just the position you establish.

Hope this helps.