So I know the ESP8266 is old but I still like tinkering with them and just had to spend a two weeks (between work and life) starting anew project and trying to get the OTA (Over The Air) updating working as a starting point, rather than adding it as the last feature to my latest device.
I had this working before, but lost the original source code during the move to Sweden, so had to start from absolute scratch.
I figured I would document what actually needs to be done for this to work for my future self (and maybe someone out there also finds this interesting ;-).
Introduction
OTA stands for Over The Air. The idea here is that I do not need to plug my WiFi enabled device into the USB port, USB TTL Programmer etc to update the firmware. This is usefull for when you want to update the firmware of that device that is sitting by my front gate, to open, close and monitor the gate, while it's raining. I also don't have to climb into the ceiling of my house to go plug something into the controller that's managing some lighting, etc.
The server component here is a Windows IOT server running on a Raspberry Pi 3B that I also host css and js files on for some of the devices that have a web front-end. (A web site that manages my house is also running here so that every device does not need to be connected to some service out on the internet which I do not have control over.)
To get started you can have a look at: OTA Updates — ESP8266 Arduino Core 3.0.2 documentation (arduino-esp8266.readthedocs.io) Here the author show quite a few options of doing OTA which are much simpler. I wanted to use the advanced option for which he has examples in PHP. I wanted to run this on .net in C# though.
The Firmware Side
So here is my basic Arduino starter code for a simple ESP8266 project:
#include <WiFiManager.h>
#include "ESP8266httpUpdate.h"
const char* deviceName = "SomeDescriptiveName";// Device family name
const char* versionString = "0.1"; // Firmware version
const char* hwVersionString = "0.1"; // Hardware version
String eepBridgeURI = "192.168.1.103"; // Windows IOT server IP Address or URL
uint16_t eepBridgePort = 8084; // Windows IOT Server Port
void setup() {
Serial.begin(115200); // Get Serial going
delay(10);
Serial.println("");
Serial.println("");
Serial.println(deviceName);
Serial.print("- Hardware Version: ");
Serial.println(hwVersionString);
Serial.print("- Software Version: ");
Serial.println(versionString);
Serial.println("");
Serial.println("Booting");
WiFiManager wifiManager; // Connect to WiFi
wifiManager.setConfigPortalTimeout(180);
wifiManager.autoConnect("DeviceName","secureme");
while (WiFi.waitForConnectResult() != WL_CONNECTED) {
Serial.println("Connection Failed! Rebooting...");
delay(5000);
ESP.restart();
}
Serial.print("Connected to WiFi: ");
Serial.println(WiFi.localIP());
Serial.println();
Serial.print("Checking for updates at ");
String vString = deviceName; // Here I build a string including
vString += ".v"; // device name, firmware version and
vString += versionString; // hardware version.
vString += "h";
vString += hwVersionString;
Serial.print("http://");
Serial.print(eepBridgeURI);
Serial.print(":");
Serial.print(eepBridgePort);
Serial.println("/ota");
Serial.print("with version string: ");
Serial.println(vString);
// Here I execute the ESP8622 Http Update passing in the version string
t_httpUpdate_return ret = ESPhttpUpdate.update(eepBridgeURI, eepBridgePort, "/ota", vString);
switch(ret)
{
case HTTP_UPDATE_FAILED:
Serial.println("- Update Failed.");
Serial.print(" - Error (");
Serial.print(ESPhttpUpdate.getLastError());
Serial.print(") : ");
Serial.println(ESPhttpUpdate.getLastErrorString());
break;
case HTTP_UPDATE_NO_UPDATES:
Serial.println("- Up to date.");
break;
case HTTP_UPDATE_OK:
Serial.println("- Update ok."); //may never be called
break;
}
Serial.println();
Serial.println("Configuring pins.");
// Do other setup steps...
Serial.println("Setup complete");
}
void loop() {
// Do stuff...
}
Quick notes on the code above
I build quite a few different devices which all make use of the same OTA API.
- Device Name: This tells the server side components which device is requesting an update.
- Hardware Version: This allows me to deal with the same device family with different versions of hardware.
- Firmware Version: This is the number I search on, on the server, and then return you the highest number available (If it's higher than the current version).
When you run the above code and monitor what is send to your server the following set of headers are delivered:
User-Agent:ESP8266-http-Update
Content-Length:0
x-ESP8266-Chip-ID:55547542
x-ESP8266-STA-MAC:18:FE:34:D7:DF:D6
x-ESP8266-AP-MAC:1A:FE:34:D7:DF:D6
x-ESP8266-free-space:598016
x-ESP8266-sketch-size:362656
x-ESP8266-sketch-md5:685ff5d9eeda8a0873c94b0f56d350e4
x-ESP8266-chip-size:4194304
x-ESP8266-sdk-version:2.2.2-dev(38a443e)
x-ESP8266-mode:sketch
x-ESP8266-version:Gineer.Home.Rainguage.v0.5h0.1
As you can see, the version string I pass through arives on the server in the x-ESP8266-version header.
Server Side
The code below is the OTA API controller for my ASP.Net Core Web App which is running on the Windows IOT server.
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Net.Http.Headers;
using System.Net;
using System.Net.Http;
using System.Text;
namespace Gineer.Home.Web.Controllers
{
[Route("[controller]")]
[ApiController]
public class OTAController : ControllerBase
{
IHeaderDictionary headers;
[HttpGet]
public async Task<IActionResult> OTA()
{
headers = Request.Headers;
// Here we do some very basic checks to see that it is an ESP8266 update request
if (!checkHeader("User-Agent", "ESP8266-http-Update"))
{
return StatusCode((int)HttpStatusCode.Forbidden);
}
// Let's make sure the version string was actually sent through
if (!checkHeader("x-ESP8266-version"))
{
return StatusCode((int)HttpStatusCode.Forbidden);
}
// Let's extract the full version string
String DeviceVersionString = headers["x-ESP8266-version"];
// Extract the device name and the current hardware and software versions
// Example: DeviceName.v1.25h1.0
Int32 VersionStringIndex = DeviceVersionString.IndexOf(".v");
String DeviceName = DeviceVersionString.Substring(0, VersionStringIndex);
String DeviceVersionRaw = DeviceVersionString.Substring(VersionStringIndex + 1);
String[] VersionArray = DeviceVersionRaw.Substring(1).Split("h");
Double SoftwareVersion = Double.Parse(VersionArray[0]);
Double HardwareVersion = Double.Parse(VersionArray[1]);
// Here we build a search pattern to find any available updates for this device and hardware version
String[] files = new string[] { };
if (HardwareVersion.ToString() == ((Int32)HardwareVersion).ToString())
{
files = Directory.GetFiles("c:\\firmware", DeviceName + ".*.*h" + HardwareVersion.ToString("0.0") + ".bin");
}
else
{
files = Directory.GetFiles("c:\\firmware", DeviceName + ".*.*h" + HardwareVersion.ToString() + ".bin");
}
// Here We get the latest version available
String returnfilename = getHighestAvailableVersion(files, SoftwareVersion);
if (returnfilename == String.Empty)
{
// If the list comes back empty, the device is already on the latest version
return StatusCode((int)HttpStatusCode.NotModified);
}
// Read and return the newest firmware for this device
var stream = System.IO.File.OpenRead(returnfilename);
return new FileStreamResult(stream, "application/octet-stream");
}
// Get the highest version of available firmware files for this device
private string getHighestAvailableVersion(string[] files, double softwareVersion)
{
double highestSoftwareVersion = 0;
String filename = String.Empty;
foreach (var file in files)
{
String tempfilename = file.Substring(file.LastIndexOf("\\")+1, file.Length - 5 - file.LastIndexOf("\\"));
Int32 VersionStringIndex = tempfilename.IndexOf(".v");
String DeviceVersionRaw = tempfilename.Substring(VersionStringIndex + 1);
String[] VersionArray = DeviceVersionRaw.Substring(1).Split("h");
Double SoftwareVersion = Double.Parse(VersionArray[0]);
if (SoftwareVersion > highestSoftwareVersion)
{
highestSoftwareVersion = SoftwareVersion;
filename = file;
}
}
if (highestSoftwareVersion > softwareVersion)
{
return filename;
}
else
{
return String.Empty;
}
}
// Check the headers provided
private Boolean checkHeader(String name, String value = "")
{
// Does the required header exist?
if (!Request.Headers.Keys.Any(k => k.ToLower() == name.ToLower()))
{
return false;
}
if (value.Length > 0)
{
// Does the header contain the required value?
if (headers[name].ToString().ToLower() != value.ToLower())
{
return false;
}
}
return true;
}
}
}
Quick notes on the code above
On the Windows IOT server, I have a firmware folder called into which I simply copy the compiled bin files every time I create a new version.
- I simply open a powershell instance to the server
- Start the FTP Process
- Copy the new bin file to the firmware folder
- Stop the ftp process.
Now the new firmware is ready to go to any device that checks for an update.
Conclusion
Security
The above example does not contain any security, which you should always include in your implementation. There are a few easy examples like update password and MD5 hash checking, but I left those out of the example above for brevity. Make sure your server and your devices are secure.
Other options
In this example the device checks for an update every time it is turned on. I have other devices that contain a web front-end for which there is a button in the settings page that initiates the check for updates.
It's just the ESPhttpUpdate.update() method that must be executed to do the update.
ESP8266 Flash Size
Remember that the biggest firmware file you can send in this way has to be smaller than half your ESP's available flash storage.
By default, the arduino IDE set my ESP's Flash Size, to 1MB (FS:64KB OTA:~470KB), which aused my device to through an exception every time it tried to get a new firmware from the server.
Using the firmware at: Arduino/CheckFlashConfig.ino at master · esp8266/Arduino · GitHub to check, I confirmed that my device has 4MB of flash, so I set the Flash Size to 4MB (FS:1MB OTA:~1019KB).
After a hardwired upload, the OTA updates work perfectly.
That's it
I have now made sure this code (both firmware and serverside code) is nicely backed up (and even blogged here) for posterity.
I can now use the firmware above as a nice started template for anything ESP8266 which should require me to only fidle with my TTL RS232 cable (O, and let's not forget having to ground IO0 to initialise it into firmware upload mode) once for every new device.
