ESP8266 + UNO with https://github.com/bportaluri/WiFiEsp


#1

Can someone please help!?
I think this should be a basic error!!
It gives an error here:
ThingerWifi thing(USERNAME, DEVICE_ID, DEVICE_CREDENTIAL);
Error:
C:\Users\admin\Documents\Arduino\libraries\thinger.io\src/ThingerWifi.h:29:26: error: ‘WiFiClient’ does not name a type template class Client = WiFiClient

Really thanks!!

Code:

#include 
#include 
#include 

SoftwareSerial Serial1(7, 8);

#define USERNAME "****"
#define DEVICE_ID "****"
#define DEVICE_CREDENTIAL "****"

#define SSID "****"
#define SSID_PASSWORD "****"
int status = WL_IDLE_STATUS;

ThingerWifi thing(USERNAME, DEVICE_ID, DEVICE_CREDENTIAL);

void setup() {
  
  // initialize serial for debugging
  Serial.begin(115200);
  // initialize serial for ESP module
  Serial1.begin(9600);
  // initialize ESP module
  WiFi.init(&Serial1);

  // check for the presence of the shield
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("WiFi shield not present");
    // don't continue
    while (true);
  }

  // attempt to connect to WiFi network
  while ( status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(SSID);
    // Connect to WPA/WPA2 network
    status = WiFi.begin(SSID, SSID_PASSWORD);
  }

  // configure wifi network
  thing.add_wifi(SSID, SSID_PASSWORD);

  pinMode(2, OUTPUT);

  // pin control example (i.e. turning on/off a light, a relay, etc)
  thing["led"] <> outputValue(millis());

  // more details at http://docs.thinger.io/arduino/
}

void loop() {
  thing.handle();
}

Arduino and ESP01
#2

Hi,
I think the problem is you are using header ThingerWifi.h that implies using Wifi library. You are using WiFiEsp instead, so you have to modify/create ThingerWifi.
You have to change the Client class template as a minimum change. I also changed the class name, so I am giving you the whole file ThingerWifiEsp.h:

// The MIT License (MIT)
//
// Copyright (c) 2016 THINK BIG LABS SL
// Author: alvarolb@gmail.com (Alvaro Luis Bustamante)
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.

// Modification for WifiEsp: jiri@bilek.info

#ifndef THINGER_WIFI_ESP_H
#define THINGER_WIFI_ESP_H

#include "ThingerClient.h"

template <class Client = WiFiEspClient>
class ThingerWifiEspClient : public ThingerClient {

public:
    ThingerWifiEspClient(const char* user, const char* device, const char* device_credential) :
            ThingerClient(client_, user, device, device_credential)
    {}

    ~ThingerWifiEspClient(){

    }

protected:

    virtual bool network_connected(){
        return WiFi.status() == WL_CONNECTED && !(WiFi.localIP() == INADDR_NONE);
    }

    virtual bool connect_network(){
        long wifi_timeout = millis();
        THINGER_DEBUG_VALUE("NETWORK", "Connecting to network ", wifi_ssid_);
        WiFi.begin((char*)wifi_ssid_, (char*) wifi_password_);
        while( WiFi.status() != WL_CONNECTED) {
            if(millis() - wifi_timeout > 30000) return false;
        }
        THINGER_DEBUG("NETWORK", "Connected to WiFi!");
        wifi_timeout = millis();
        THINGER_DEBUG("NETWORK", "Getting IP Address...");
        while (WiFi.localIP() == INADDR_NONE) {
            if(millis() - wifi_timeout > 30000) return false;
        }
        THINGER_DEBUG_VALUE("NETWORK", "Got IP Address: ", WiFi.localIP());
        return true;
    }

public:

    void add_wifi(const char* ssid, const char* password)
    {
        wifi_ssid_ = ssid;
        wifi_password_ = password;
    }

protected:
    Client client_;
    const char* wifi_ssid_;
    const char* wifi_password_;
};

#define ThingerWifiEsp ThingerWifiEspClient<>

#endif

Nevertheless during my testing on thinger.io I realized there is a problem in ThingerClient.h, because of the fact that both parties (Arduino and ESP) could initiate a conversation and there was a big chance the incoming data from ESP was lost.
As a first try I added a following patch to a ThingerClient.h file. It is a sort of quick and dirty fix, I am still thinking of a better way not requiring to edit the ThingerClient file.

The function handle_connection in ThingerClient.h (line 226):

bool handle_connection()
{
    // following two lines are added:
    if (client_.available() > 0)  // if there are incoming data, treat the connection as OK
        return true;

  bool network = network_connected();

I hope this will help you.
Jiri


#3

Hi, I have updated the Arduino library in Github for supporting the WifiESP library (it should be available for direct download within some hours). However I have not tested it, as I have no an ESP8266 connected to an Arduino Uno. Let me know if it works!

Bests.


#4

Thanks a lot Jiri.

I was testing with the information/modification you give me. And for the first time here in thinger.io, very interesting platform.

To control the Led On/Off I need to put a delay about 500ms, without a delay it is very instable.

I can´t read any value in thinger.io, from my sensor/thing, only few times.

The code I test is:

#include <WiFiEsp.h>
#include <ThingerWifiEsp.h>
#include <SoftwareSerial.h>

SoftwareSerial Serial1(7, 8);

#define PhotoR_Pin A1  //define a pin for Photo resistor
#define USERNAME "*****"
#define DEVICE_ID "*****"
#define DEVICE_CREDENTIAL "*****"

#define SSID "*****"
#define SSID_PASSWORD "*****"
int status = WL_IDLE_STATUS;
int light;

ThingerWifiEsp thing(USERNAME, DEVICE_ID, DEVICE_CREDENTIAL);

void setup() {
  
  // initialize serial for debugging
  Serial.begin(115200);
  // initialize serial for ESP module
  Serial1.begin(9600);
  // initialize ESP module
  WiFi.init(&Serial1);



  // check for the presence of the shield
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("WiFi shield not present");
    // don't continue
    while (true);
  }

  // attempt to connect to WiFi network
  while ( status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(SSID);
    // Connect to WPA/WPA2 network
    status = WiFi.begin(SSID, SSID_PASSWORD);
  }


  pinMode(2, OUTPUT);

  // pin control example (i.e. turning on/off a light, a relay, etc)
  thing["led"] << digitalPin(2);

  // resource output example (i.e. reading a sensor value, a variable, etc)
  thing["Photoresistor"] >> outputValue(light);

  // more details at http://docs.thinger.io/arduino/
}

void loop() {
  light=analogRead(PhotoR_Pin);
  thing.handle();
  Serial.println(light);
  delay(500);
}

I already see that Alvarolb update the Wifi library in Github. I will test it!


First Attempt - I got an error in ThingerWifi.h
#5

Hi Alvaro,
thanks for your reply.

There is still a big problem in communication between thinger libraries and WifiEsp library on Arduino. I spent some time investigating and it seems that thinger server sends one subscribing packet with more than one resource identifier and the client then parses it one resource at a time and sends some acknowledge data on each of the resource identifiers as they are parsed from the message.
Unfortunately this behaviour is breaing the communication, as the WifiEsp library discards input buffer on sending data (I think this has to do with the fact of the resource-limited arduino, I mean its very small amount of RAM).

I found this sequence:

  1. SRV->CLIENT message subscribing resource AAA and BBB
  2. CLIENT reads a part of the message, decodes information about subscription of AAA
  3. CLIENT->SRV sends a message (acknowledge?)
    3a. CLIENT discards the input queue
    3b. CLIENT sends the messsage
  4. CLIENT would like to continue parsing but there is no data in the input queue.

I am tracing the problem, any information about the communication between server and client would be very helpful.

Thanks,
Jiri


#6

hi @JiriBilek, the sequence you have identified is right. The server can send two or more packets at once, i.e., subscribing to two different resources, requesting resources api, etc. The client will decode one input message for each .handle() call and will send a response. It is expected that the input buffer is not cleared until it is consumed. This approach is not greedy, so the loop can get a smoother behavior independently on the incoming messages. This is why it is so important to avoid delays inside the loop, to avoid delays while answering server requests.

Are you sure that the WifiEsp library discard input bytes while sending data? If this is true, we have a problem! We can modify the client libraries to parse all incoming messages in one .handle() call, but each message response should be done as the message is decoded. Otherwise, we will require to buffer all the incoming messages, to later provide a response. This involves dynamic memory, and would increase the latency, as it is required to parse all incoming messages. Anyway, it is still prone to other errors like receiving message A, and a half of message B, as the B bytes will be discarded.

Not sure why the WifiEsp is behaving like this, maybe it only has one I/O buffer that is shared for both read/write operations, but this is a very bad design! I will take a look on it!


#7

Hi Alvaro and Jiri!

After the modification from Jiri I can connect my device, I only can insert 1 widget for 1 Dashboard, if I add more they work bad or doesn´t work.

With the new library from Alvaro for some reason I can´t connect the device.

Thanks a lot!

AS


#8

The current example in Github have an error. Wifi.init should be done over Serial1, not over Serial.

I have an ESP8266 connected to an Arduino Nano. I had many problems at first while using the stock 115200 baudrate. SoftwareSerial is not able to handle this speed properly with the AVR. So I have changed the ESP8266 baudrate to 9600 and everything is quite smooth now and it connects fine to the Wifi and the platform, but seems to have some problems with the SSL at this moment.

Now I have the setup where the device connects, but there are problems while reading data. Will take a look on it soon. Here is my code for reference (working with the latest Thinger library).

// Can be installed from Library Manager or https://github.com/bportaluri/WiFiEsp
#define _DEBUG_
#define _DISABLE_TLS_
#include <WiFiEsp.h>
#include <ThingerESP8266AT.h>

#define USERNAME "*****"
#define DEVICE_ID "*****"
#define DEVICE_CREDENTIAL "******"

#define SSID "******"
#define SSID_PASSWORD "*****"

#define ARDUINO_LED 13

// Emulate Serial1 on pins 6/7 if not present
#ifndef HAVE_HWSERIAL1
#include "SoftwareSerial.h"
SoftwareSerial Serial1(6, 7); // RX, TX
#endif

ThingerESP8266AT thing(USERNAME, DEVICE_ID, DEVICE_CREDENTIAL);

void setup() {
  pinMode(ARDUINO_LED, OUTPUT);

  // serial for debug
  Serial.begin(9600);

  // initialize serial for ESP8266
  Serial1.begin(9600);

  // used to change the ESP8266 to 9600 baudrate (set with a 115200 connection)
  //Serial1.write("AT+UART_DEF=9600,8,1,0,0\r\n");

  // init ESP8266 in the additional serial port
  WiFi.init(&Serial1);
  if(WiFi.status() == WL_NO_SHIELD){
      // don't continue
      while (true);
  }

  thing.add_wifi(SSID, SSID_PASSWORD);

  // digital pin control example (i.e. turning on/off a light, a relay, configuring a parameter, etc)
  thing["led"] << digitalPin(ARDUINO_LED);

  // resource output example (i.e. reading a sensor value)
  thing["millis"] >> outputValue(millis());

  // more details at http://docs.thinger.io/arduino/
}

void loop() {
  thing.handle();
}

My debug output… where it connects but the connection is not stable enough.

[WiFiEsp] Initializing ESP module
[WiFiEsp] Initilization successful - 1.5.0
[NETWORK] Starting connection...
[NETWORK] Connecting to network ****
[WiFiEsp] Connected to ****
[NETWORK] Connected to WiFi!
[NETWORK] Getting IP Address...
[NETWORK] Got IP Address: 192.168.1.104
[NETWORK] Connected!
[_SOCKET] Connecting to iot.thinger.io:25200...
[_SOCKET] Using secure TLS/SSL connection: no
[WiFiEsp] Connecting to iot.thinger.io
[_SOCKET] Connected!
[THINGER] Authenticating. User: **** Device: ****
[THINGER] Writing bytes: 45 [OK]
[THINGER] Authenticated
[THINGER] Writing bytes: 2 [OK]
[THINGER] Available bytes: 14
[THINGER] Writing bytes: 44 [OK]
[THINGER] Writing bytes: 15 [OK]
[_SOCKET] Timeout!
[WiFiEsp] Disconnecting  3
[_SOCKET] Is now closed!

#9

Thanks Alvaro!

I get exactly the same results, it connects, but the connection is not stable enough.


#10

Yep! I will take a deeper look at the WiFiESP library to see how it is implemented. Maybe it can be improved a bit to behave exactly as the stock WifiClient. Right now seems to have problems while reading data from socket. There also some references in GitHub that may be related to our problems:


#11

Hi,
I confirm the WifiEsp library discards the incoming data on next transmit of AT command. There is generally no input buffer and the library reads the data directly from the SoftwareSerial buffer (on Arduino Uno). The fuction that clears the data is espEmptyBuf(), file EspDrv.cpp. You can see it is called frequently.

I am thinking of creating separate input buffer for data messages, at least for the shorter ones. The processor in Arduino doesn’t have sufficient RAM to create large buffers.


#12

I saw exactly the same things @JiriBilek, espEmptyBuf() is called every time any command is send (i.e checking the socket state or wifi state), dropping all incoming data from server. I enabled also the DEBUG output of the library, and saw that the server sends correctly the bytes, but the library still says that there is no data available. This is not the expected behavior. Not sure what is the best approach to solve this, but the library needs a rework. I suppose that the ESP8266 maintain different input buffers, as it needs to handle different TCP connections simultaneously, so i suppose you can query per socket different states, like checking if there is pending data, read from one socket, and so on…


#13

Alvaro,
I added an input buffer to the WifiEsp code so now the connection is quite stable but I am observing strange behaviour on the connection.
Could you please explain the thinger.io server functionality?

My setup:
a) three outputs and one output
b) two dashboards (one with all three outputs, the second one with the first output only), no buckets

My questions for approval or explanation:

  1. After the device connects to the server, the server requests (subscribes) only the resources that are contained in visible dashboards? So that when no dashboards are open, no requests are made, only keepalive messages are travelling.
  2. On every switching dashboards the server requests (subscribes) all the resources from the new dashboard?
  3. On timeout and following automatic reconnecting the server should act as in question 1?

I am experiencing strange subscription requests. Mostly the server requests only one resource, sometimes all three, but I cannot find deterministic behaviour in it. It is very common when I switch to a dashboard with more resources that only the first one is subscribed.

I am attaching the experimental version (needs more work definitely) of a changed file in WifiEsp with 128 byte input buffer. And my testing sketch.

Thanks,
Jiri

EspDrv.cpp (25.6 KB)
WifiEsp.ino.cpp (1.6 KB) - please rename to WifiEsp.ino


#14

Hi @JiriBilek! Nice you have something that works!! Great Job! I will try to test your code soon.

I will try to provide information about how the dasbhoards and resources works answering your questions:

  1. Yes, the resources defined in the device are not used for anything at a first instance. However, when you create and open a dashboard, the server subscribes to all the resources required to provide data to this dashboard. If you open a dashboard, the server request the resources to the device. If you close the dashboard, then the device will stop streaming the associated resources. The same happens when there are 2 or more dashboards. The subscription is only done once if both dashboards shares the same data. The same idea applies to buckets. No visible dashboards or buckets: the device should only send keepalives.
  2. Yes, if you close one dashboard, and the open the second one, the server will stop the resources for the first one, and then will request the resources for the second one.
  3. Yes, if the device is disconnected/reconnected while there is a dashboard open or data bucket associated, the server will automatically request all the resources to provide data to the dashboards or bucets.

So, if you have dash X with resources A, B, and a second one Y with A, B, and C, opening X will request A and B. Opening Y simultaneously, will request only C.

There are some other underlying optimizations, like, what happens when X request A at 2 seconds, and Y request it at 5 seconds? The server will request the resource at 1 second interval (greatest common divisor), and will provide data to each channel/dash at required interval. This use case can trigger additional subscriptions that modify the streaming interval when opening/closing the involved streams.

Not sure if it is more clear now. Let me know! :slight_smile:


#15

Thanks, @alvarolb.
Now I can better debug the communication.
From the wifi client debugging it seems to me that the server doesn’t act as you described. I will do more testing (dumping of incoming messages) and let you know.
Jiri


#16

Really really thanks to both! In particular to @JiriBilek for these recent great news!
I will test it a lot for sure.
I like o lot this platform, I´m very happy with this improvments!

The file “WifiEsp.ino” is for upload to the ATMEGA and file “EspDrv.cpp” is to replace the other with the same name, obvious.
For now I don´t have any question!


#17

@Andrei_Sousa: Please don’t forget the EspDrv is in very very early stage of modification. I will refine it once I get it working with thinger without problems.


#18

Really thanks @JiriBilek!

This is a great open source solution in my opinion (this platform with ESP8266 + ATMEGA328P).
I will test this solution a lot!

I already make some tests, I understand that this is in a very early stage, I think for now is not necessary explain in detail the problems I found.
The three outputs are working very good, only when reconect it fail sometimes.
The Led is more instable, sometimes only blink turn on and off, you already see it for sure.


#19

Having the same problems :slight_smile:


#20

@JiriBilek,

I installed the App in the Android Thinger.io.
When I make refresh the device sometimes appear the led or one of the outputs.
But the led switch appear almost always and is working very stable.

conclusion:
In the App is the contrary, the Led Works better than the outputs.


Arduino and ESP01