JSON and XML are the most common data serialization formats. The Internet is a hub for billions of different devices, applications and services. These devices and applications are built in different programming languages and on different platforms. In order for all of these wildly different devices and applications to communicate with each other effectively, data is serialized and deserialized across endpoints in a shareable format. JSON and XML are the two most important markup languages in the Internet world that allow data to be exchanged between different platforms, devices and applications in a standardized way. Devices and applications serialize data in JSON or XML format before transmitting it to the Internet. The receiving device or application at the other end of the Internet deserializes the JSON/XML data and extracts the necessary information according to your user code.
All IoT devices are also part of the same Internet universe. Most IoT devices use the JSON format to communicate data with a server, gateway, or other devices (on the same network). JSON is preferred over XML because it is simpler and less verbose. For IoT devices, even a small overhead can be a big burden for devices running by microcontrollers. This is why most IoT devices use JSON to exchange data over the Internet. Most IoT platforms and services also essentially use JSON, although they can also use XML format.
In this article, we will discuss how Arduino and Arduino-compatible microcontrollers serialize and deserialize JSON data for standard universal communication in IoT domains. Arduino and its compatible boards rely on the ArduinoJSON library to encode and decode JSON data for IoT and the Internet.
What is JSON?
JSON stands for JavaScript object notation. It is a lightweight text-based data exchange format. JSON is widely used in web applications and APIs to exchange data due to its simplicity and compatibility with various programming languages. It is used to serialize and transmit structured data over a network connection, such as data exchanged between a server and a client. JSON is human readable as well as easy for machines to parse and generate.
JSON Syntax
JSON syntax is derived from JavaScript object notation, but is language independent. JSON represents data as name/value pairs (similar to JavaScript object properties), arrays, or a combination of both. An object in JSON is enclosed in braces { }. Within the object, data is represented as a collection of key/value pairs. A key/value pair consists of a field name (enclosed in double quotes), followed by a colon (:), followed by a value. Commas separate key/value pairs (,). Values in JSON can be strings (enclosed in double quotes), numbers (integer or floating point), Boolean values (true or false), arrays, objects, or nulls. An array in JSON is an ordered list of values and is enclosed in square brackets. Commas separate the values in the array (,). Values in an array can be objects, numbers, strings, booleans or even other arrays. Data represented in JSON looks like this:
{“key1″:”value1”, “key2″:”value2”, “key3″:”value3”,….}
The following is a valid example of JSON data:
{“temperature”:25.21, “humidity”:59.07, “pressure”:1008.23}
Most of the time, the values returned by IoT servers or applications in JSON are other JSON objects or arrays.
Arduino, IoT and JSON
JSON is a common data exchange format on the Internet. Any IoT device powered by Arduino or a similar microcontroller can receive data from a web server, API or web service in JSON format only. Because JSON is simple and straightforward, even tiny machines like microcontrollers are capable of parsing and generating it. Markup language allows the transmission of useful data without any extra overhead. This is why most web applications that power IoT devices exchange data with devices via JSON. The same happens when devices need to send data to a web server, API or web service. Because microcontrollers can easily encode data into JSON, it is the de facto standard for embedded devices and applications to serialize data (such as sensor data) for transmission over the Internet. JSON allows IoT devices and IoT platforms and services to exchange data in minimal space.
What is ArduinoJSON?
Arduino and other microcontrollers need to be connected to the Internet through Ethernet or WiFi to exchange data over the Internet as an IoT device. For Arduino, the board needs an Ethernet shield, or connects to the Internet via a WiFi breakout board, or must have built-in WiFi or Ethernet functionality. On the programming side, in addition to Ethernet or WiFi libraries, Arduino and its compatible boards need the ArduinoJSON library to encode or decode JSON for effective communication in the IoT scenario.
ArduinoJSON is designed for constrained environments and optimized for minimal memory usage, making it suitable for microcontrollers with limited RAM. It also supports static and dynamic memory allocation, providing developers with flexibility based on their memory management strategies. The library will also filter the JSON data and transform it into different structures according to the application's needs; The library uses a DOM-like API, which means it can parse JSON data into a tree structure, making it easier to pass and extract values. Additionally, it supports static and dynamic memory allocation, giving developers flexibility in memory management strategies.
Installing the ArduinoJSON library
ArduinoJSON is not linked to Arduino libraries by default. Therefore, you need to install the library explicitly in the Arduino IDE. You can download the latest version of the library here as a ZIP file. Rename the ZIP folder to ArduinoJSON. Open the Arduino IDE and navigate to Sketch -> Add Library -> Add .ZIP Library. Select the ArduinoJSON ZIP folder and install it. Alternatively, the library can be found by navigating to Tools -> Manage Libraries and searching for ArduinoJSON in the search box. Select ArduinoJSON and click the install button. After installing the library, it can be imported into a sketch using the following instruction.
#include
ArduinoJSON library classes and functions
The ArduinoJSON library includes the following classes:
- Dynamic JSON Document
- Json static document
- JsonDocument
- Par Json
- JsonVariant
- JsonObject
- JSONArray
- Deserialization error
Dynamic JSON Document
This class represents a JSON document and uses dynamic memory allocation. It is suitable for situations where the size of the JSON document may vary. Document memory is allocated on the heap, making it more flexible but slightly slower than StaticJsonDocument. The class provides the following functions useful in manipulating JSON documents:
- clear : Clears the document and frees all memory.
- capacity: Returns the current capacity of the document.
- ShrinkToFit: Shrinks capacity to match current usage.
- to the
: Converts the document to a specific type (such as JsonObject or JsonArray). - I'm
: Creates a copy of the document as a specific type. - operator: Provides access to the elements of the JSON object or array.
- memoryUsage : Returns the amount of memory used by the document.
Json static document
This class is similar to DynamicJsonDocument, but uses static memory allocation. It is faster and more memory efficient for smaller, fixed-size JSON documents because the memory is allocated on the stack. This class is best suited for environments with limited memory resources. It provides the same functions as the DynamicJsonDocument class.
Json Document
This is a base class for DynamicJsonDocument and StaticJsonDocument. It provides the functionality common between these two classes, such as parsing JSON data from a string or stream and serializing the JSON document back to a string or stream. This class provides the following functions:
- deserializeJson: Parses a JSON string or stream in the document.
- serializeJson : Serializes the document into a JSON string or stream.
- I'm
: converts the document to JsonObject. - I'm
: converts the document to JsonArray.
JsonPair
This class allows you to manipulate a key-value pair in a JsonObject. This class is useful when iterating over all elements in a JsonObject. This class provides the following functions:
- key : Returns the key of the key-value pair.
- value : Returns the value of the key-value pair.
JsonVariant
This class allows you to manipulate a JSON value, which can be any type of JSON data: a number, a string, a boolean, an array (JsonArray), an object (JsonObject) or null. It provides functions to convert between types and check the actual type of the stored value. This class provides the following functions:
- to the
: Converts the value to a specified type. - is
: Checks whether the value can be converted to a specified type. - I'm
: Creates a copy of the value as a specified type. - set : assigns a new value.
- clear : resets the value to null.
JsonObject
This class allows you to manipulate a JSON object, a collection of key-value pairs. Allows you to add, access and remove members. Keys are strings and values can be of various types, including nested JsonArray or JsonObject. This class provides the following functions:
- createNestedObject : Creates a nested JsonObject.
- createNestedArray : Creates a nested JsonArray.
- containsKey : Checks whether the object contains a specified key.
- remove : Removes the element with the specified key.
- clear : Removes all key-value pairs from the object.
- size : Returns the number of key-value pairs.
- start/end: Provides iterators to traverse the object.
- operator: accesses or modifies a value associated with a key.
JSONArray
This class allows you to manipulate a JSON array. It provides functions to add, remove and access array elements. You can iterate over the elements in a JsonArray. This class provides the following functions.
- add : Adds a new element to the end of the array.
- remove : Removes the element at the specified index.
- clear : Removes all elements from the array.
- size : Returns the number of elements in the array.
- start/end: Provides iterators to traverse the array.
- operator: accesses an element at a specified index.
Deserialization error
When parsing JSON data, this class is used to represent the result. Indicates whether deserialization was successful or an error occurred. It provides methods to check the error type and retrieve an error message. This class provides the following functions.
- code : Returns the error code.
- c_str: Returns a C-style string representation of the error message.
Parsing JSON data with Arduino
Parsing and deserializing JSON data from a web server, API, or web service involves the following steps:
- Include the ArduinoJSON library: First, include the ArduinoJSON library in your Arduino sketch. The library must be installed in the Arduino IDE. The library is included in a sketch using the following statement:
#includeArduinoJson.h>
- Allocate a JSON document: Next, you need to create a JsonDocument object. You can choose between DynamicJsonDocument and StaticJsonDocument according to your memory management strategy. The following statement creates a dynamic JSON document that is 1,024 bytes in size:
Document DynamicJsonDocument(1024);
- Parse JSON data: Fetch JSON data into a string and deserialize it into the JSON document. The following is a valid example of parsing a JSON string.
const char* jsonString = “{\”sensor\”:\”gps\”,\”time\”:1351824120,\”data\”:(48.756080,2.302038)}”;
Error DeserializationError = deserializeJson(doc, jsonString);
if (error) {
Serial.print(“deserializeJson failed: “);
Serial.println(error.c_str );
turn back;
}
- Access specific data: Once the JSON is deserialized, you can access the data using JsonDocument. The following is a valid example of accessing specific values from a deserialized JSON document.
const char* sensor = doc(“sensor”); // “GPS”
long time = doc(“time”); //1351824120
double latitude = doc(“data”)(0); //48.756080
double longitude = doc(“data”)(1); //2.302038
Nested objects and arrays are also accessed in a similar way to regular key-value pairs. Remember that you should always check the size of your JSON data and adjust the JsonDocument size accordingly. The required size of the JSON document can be calculated using the ArduinoJson Assistant, which is available on the ArduinoJSON website.
Generating JSON data with Arduino
Generating and serializing JSON data using the ArduinoJSON library involves creating a JsonDocument, filling it with data, and serializing that data into a JSON string. It involves the following steps:
- Include the ArduinoJSON library: First, include the ArduinoJSON library in your Arduino sketch. The library must be installed in the Arduino IDE. The library is included in a sketch using the following statement:
#includeArduinoJson.h>
- Create a JSON document: Next, you need to create a JsonDocument object. You can choose between DynamicJsonDocument and StaticJsonDocument according to your memory management strategy. The following statement creates a dynamic JSON document that is 1,024 bytes in size.
Document DynamicJsonDocument(1024);
- Populate the JSON document: Add data to the JsonDocument. The document can consist of values, arrays, or nested objects. The following is a valid example of filling out a JSON document:
doc(“sensor”) = “gps”;
doc(“time”) = 1351824120;
JsonArray data = doc.createNestedArray(“data”);
data.add(48.756080);
data.add(2.302038) ;
- Serialize the JSON document: Finally, convert the JsonDocument to a JSON string using serializeJson. You can directly serialize it to a String, a character array, or any output stream (like Serial). Following is a valid example of JSON document serialization.
String output;
serializeJson(doc,output);
To serialize the document to the Arduino serial port, use the following instruction.
serializeJson(doc,Serial);
The JSON document must be large enough to contain the entire JSON object. You can use ArduinoJson Assistant to calculate the required size. Remember that each addition to the document consumes memory. So be careful with memory constraints, especially on smaller microcontrollers. Serialized JSON is in compact format. If you need a human-readable format, you can use the serializeJsonPretty function instead of serializeJson .
Conclusion
Whenever an IoT device communicates data with a web server, API, or web service, the data is received in JSON format or needs to be serialized in JSON format. Markup language is a standard for exchanging data over the internet across devices, applications and platforms. On Arduino and compatible boards, encoding and decoding JSON documents is possible with the help of the ArduinoJSON library. The library helps decode JSON documents received from a server/API/web service via an HTTP request and extracts the required values from the document. The library is also useful for serializing data into JSON for transmission to a web server or API.