Azure SDK for Embedded C
Azure SDK for Embedded C

Build Status

The Azure SDK for Embedded C is designed to allow small embedded (IoT) devices to communicate with Azure services. Since we expect our client library code to run on microcontrollers, which have very limited amounts of flash and RAM, and have slower CPUs, our C SDK does things very differently than the SDKs we offer for other languages.

With this in mind, there are many tenets or principles that we follow in order to properly address this target audience:

  • Customers of our SDK compile our source code along with their own.
  • We target the C99 programming language and test with gcc, clang, & MS Visual C compilers.
  • We offer very few abstractions making our code easy to understand and debug.
  • Our SDK is non allocating. That is, customers must allocate our data structures where they desire (global memory, heap, stack, etc.) and then pass the address of the allocated structure into our functions to initialize them and in order to perform various operations.
  • Unlike our other language SDKs, many things (such as composing an HTTP pipeline of policies) are done in source code as opposed to runtime. This reduces code size, improves execution speed and locks-in behavior, reducing the chance of bugs at runtime.
  • We support microcontrollers with no operating system, microcontrollers with a real-time operating system (like Azure RTOS), Linux, and Windows. Customers can implement custom platform layers to use our SDK on custom devices. We provide some platform layers, and encourage the community to submit platform layers to increase the out-of-the-box supported platforms.

Table of Contents

Documentation

We use doxygen to generate documentation for source code. You can find the generated, versioned documentation here.

The GitHub Repository

To get help with the SDK:

Services

The Azure SDK for Embedded C repo has been structured around the service libraries it provides:

  1. IoT - Library to connect Embedded Devices to Azure IoT services
  2. Storage - Library to send blob files to Azure IoT services

Structure

This repo is structured with two priorities:

  1. Separation of services/features to make it easier to find relevant information and resources.
  2. Simplified source file structuring to easily integrate features into a user's project.

/sdk - folder containing docs, sources, samples, tests for all SDK packages
    /docs - documentation for each service (iot, storage, etc)
    /inc - include directory - can be singularly included in your project to resolve all headers
    /samples - samples for each service
    /src - source files for each service
    /tests - tests for each service

For instructions on how to consume the libraries via CMake, please see here. For instructions on how consume the source code in an IDE, command line, or other build systems, please see here.

Master Branch

The master branch has the most recent code with new features and bug fixes. It does not represent the latest General Availability (GA) release of the SDK.

Release Branches and Release Tagging

When we make an official release, we will create a unique git tag containing the name and version to mark the commit. We'll use this tag for servicing via hotfix branches as well as debugging the code for a particular preview or stable release version. A release tag looks like this:

<package-name>_<package-version>

The latest release can be found in the release section of this repo.

For more information, please see this branching strategy document.

Getting Started Using the SDK

The SDK can be conveniently consumed either via CMake or other non-CMake methods (IDE workspaces, command line, and others).

CMake

  1. Install the required prerequisites:
    • CMake version 3.10 or later
    • C compiler: MSVC, gcc or clang are recommended
    • git to clone our Azure SDK repository with the desired tag

  2. Clone our Azure SDK repository, optionally using the desired version tag. 

     git clone https://github.com/Azure/azure-sdk-for-c

 git checkout <tag_name>

    For information about using a specific client library, see the README file located in the client library's folder which is a subdirectory under the /sdk/docs folder.

  3. Ensure the SDK builds correctly.

    • Create an output directory for your build artifacts (in this example, we named it build, but you can pick any name). 
       mkdir build
    • Navigate to that newly created directory. 
       cd build
    • Run cmake pointing to the sources at the root of the repo to generate the builds files. 
       cmake ..
    • Launch the underlying build system to compile the libraries. 
       cmake --build .

    This results in building each library as a static library file, placed in the output directory you created (for example build\sdk\core\az_core\Debug). At a minimum, you must have an Azure Core library, a Platform library, and an HTTP library. Then, you can build any additional Azure service client library you intend to use from within your application (for example build\sdk\storage\blobs\Debug). To use our client libraries in your application, just #include our public header files and then link your application's object files with our library files.

  4. Provide platform-specific implementations for functionality required by Azure Core. For more information, see the Azure Core Porting Guide.

CMake Options

By default, when building the project with no options, the following static libraries are generated:

  • Libraries:
    • az_core
    • az_iot
      • iot_provisioning, iot_hub, etc.
    • az_storage_blobs
      • Storage SDK blobs client.
    • az_noplatform
      • A platform abstraction which will compile but returns 0 or does nothing for all platform calls. This ensures the project can be compiled without the need to provide any specific platform implementation. This is useful if you want to use az_core without platform specific functions like time or sleep.
    • az_nohttp
      • Library that provides a no-op HTTP stack, returning AZ_ERROR_DEPENDENCY_NOT_PROVIDED. Similar to az_noplatform, this library ensures the project can be compiled without requiring any HTTP stack implementation. This is useful if you want to use az_core without az_http functionality.

The following CMake options are available for adding/removing project features.

Option Description Default Value
UNIT_TESTING Generates Unit Test for compilation. When turning this option ON, cmocka is a required dependency for compilation.
After Compiling, use ctest to run Unit Test. 		OFF
UNIT_TESTING_MOCKS This option works only with GCC. It uses -ld option from linker to mock functions during unit test. This is used to test platform or HTTP functions by mocking the return values. OFF
PRECONDITIONS Turning this option OFF would remove all method contracts. This is typically for shipping libraries for production to make it as optimized as possible. ON
TRANSPORT_CURL This option requires Libcurl dependency to be available. It generates an HTTP stack with libcurl for az_http to be able to send requests thru the wire. This library would replace the no_http. OFF
TRANSPORT_PAHO This option requires paho-mqtt dependency to be available. Provides Paho MQTT support for IoT. OFF
AZ_PLATFORM_IMPL This option can be set to any of the next values:
- No_value: default value is used and no_platform library is used.
- "POSIX": Provides implementation for Linux and Mac systems.
- "WIN32": Provides platform implementation for Windows based system
- "USER": Tells cmake to use an specific implementation provided by user. When setting this option, user must provide an implementation library and set option AZ_USER_PLATFORM_IMPL_NAME with the name of the library (i.e. -DAZ_PLATFORM_IMPL=USER -DAZ_USER_PLATFORM_IMPL_NAME=user_platform_lib). cmake will look for this library to link az_core 		No_value
  • Samples: Whenever UNIT_TESTING is ON, samples are built using the default PAL (see running samples section). This means that running samples would throw errors like: 
    ./keys_client_example
Running sample with no_op HTTP implementation.
Recompile az_core with an HTTP client implementation like CURL to see sample sending network requests.

i.e. cmake -DTRANSPORT_CURL=ON ..

VSCode

For convenience, you can quickly get started using VSCode and the CMake Extension by Microsoft. Included in the repo is a settings.json file here which the extension will use to configure a CMake project. To use it, copy the settings.json file from .vscode-config to your own .vscode directory. With this, you can run and debug samples and tests. Modify the variables in the file to your liking or as instructed by sample documentation and then select the following button in the extension:

VSCode CMake Config

From there you can select targets to build and debug.

NOTE: Especially on Windows, make sure you select a compiler platform version that matches the dependencies installed via VCPKG (i.e. x64 or x86). Additionally, the triplet to use should be specified in the VCPKG_DEFAULT_TRIPLET field in settings.json.

Source Files (IDE, command line, etc)

We have set up the repo for easy integration into other projects which don't use CMake. Two main features make this possible:

  • To resolve all header file relative paths, you only need to include sdk/inc in your project. All header files are included in the sdk with relative paths to clearly demarcate the services they belong to. A couple examples being:
#include <azure/core/az_span.h>
#include <azure/iot/az_iot_hub_client.h>
  • All source files are placed in a directory structure similar to the headers: sdk/src. Each service has its own subdirectory to separate files which you may be singularly interested in.

To use a specific service/feature, you may include the header file with the function declaration and compile the according .c containing the function implementation with your project.

The specific dependencies of each service may vary, but a couple rules of thumb should resolve the most typical of issues.

  1. All services depend on core (source files here). You may compile these files with your project to resolve core dependencies.
  2. Most services will require a platform file to be compiled with your project (see here for porting instructions). We have provided several implementations already here for windows, posix, and a no_platform for no-op stubs. Please compile one of these, for your respective platform, with your project.

The following compilation, preprocessor options will add or remove functionality in the SDK.

Option Description
AZ_NO_PRECONDITION_CHECKING Turns off precondition checks to maximize performance with removal of function precondition checking.
AZ_NO_LOGGING Removes all logging code and artifacts from the SDK (helps reduce code size).

Running Samples

See compiler options section to learn about how to build samples with HTTP implementation in order to be runnable.

After building samples with HTTP stack, set the environment variables for credentials. The samples read these environment values to authenticate to Azure services. See client secret here for additional details on Azure authentication.

# On linux, set env var like this. For Windows, do it from advanced settings/ env variables
# STORAGE Sample (only 1 env var required)
# URL must contain a valid container, blob and SaS token
# e.g "https://storageAccount.blob.core.windows.net/container/blob?sv=xxx&ss=xx&srt=xx&sp=xx&se=xx&st=xxx&spr=https,http&sig=xxx"
export AZURE_STORAGE_URL="https://??????????????"

Libcurl Global Init and Global Clean Up

When you select to build the libcurl http stack implementation, you have to make sure to call curl_global_init before using SDK client like Storage to send HTTP request to Azure.

You need to also call curl_global_cleanup once you no longer need to perform SDk client API calls.

Take a look to Storage Blob SDK client sample. Note how you can use function atexit() to set libcurl global clean up.

The reason for this is the fact of this functions are not thread-safe, and a customer can use libcurl not only for Azure SDK library but for some other purpose. More info here.

This is libcurl specific only.

Development Environment

Project contains files to work on Windows, Mac or Linux based OS.

Note For any environment variables set to use with CMake, the environment variables must be set BEFORE the first cmake generation command (cmake ..). The environment variables will NOT be picked up if you have already generated the build files, set environment variables, and then regenerate. In that case, you must either delete the CMakeCache.txt file or delete the folder in which you are generating build files and start again.

Windows

vcpkg is the easiest way to have dependencies installed. It downloads packages sources, headers and build libraries for whatever TRIPLET is set up (platform/arq). VCPKG maintains any installed package inside its own folder, allowing to have multiple vcpkg folder with different dependencies installed on each. This is also great because you don't have to install dependencies globally on your system.

Follow next steps to install VCPKG and have it linked to cmake. The vcpkg repository is checked out at the ref in vcpkg.yml. Azure SDK code in this version is known to work at that vcpkg ref.

# Clone vcpkg:
git clone https://github.com/Microsoft/vcpkg.git
# (consider this path as PATH_TO_VCPKG)
cd vcpkg
# Checkout the vcpkg ref from the vcpkg.yml file (link above)
# git checkout <vcpkg ref>
# build vcpkg (remove .bat on Linux/Mac)
.\bootstrap-vcpkg.bat
# install dependencies (remove .exe in Linux/Mac) and update triplet
.\vcpkg.exe install --triplet x64-windows-static curl[winssl] cmocka paho-mqtt
# Add this environment variables to link this VCPKG folder with cmake:
# VCPKG_DEFAULT_TRIPLET=x64-windows-static
# VCPKG_ROOT=PATH_TO_VCPKG (replace PATH_TO_VCPKG for where vcpkg is installed)

If you previously installed VCPKG and dependencies, you may need to run .\vcpkg.exe upgrade --no-dry-run to upgrade to the latest packages.

Note: Setting up a development environment in windows without VCPKG is not supported. It requires installing all dev-dependencies globally and manually setting cmake files to link each of them.

Follow next steps to build project from command prompt:

# cd to project folder
cd azure-sdk-for-c
# create a new folder to generate cmake files for building (i.e. build)
mkdir build
cd build
# generate files
# cmake will automatically detect what C compiler is used by system by default and will generate files for it
cmake ..
# compile files. Cmake would call compiler and linker to generate libs
cmake --build .

Note: The steps above would compile and generate the default output for azure-sdk-for-c which includes static libraries only. See section Compiler Options

Visual Studio 2019

Open project folder with Visual Studio. If VCPKG has been previously installed and set up like mentioned above. Everything will be ready to build. Right after opening project, Visual Studio will read cmake files and generate cache files automatically.

Linux

VCPKG

VCPKG can be used to download packages sources, headers and build libraries for whatever TRIPLET is set up (platform/architecture). VCPKG maintains any installed package inside its own folder, allowing to have multiple vcpkg folder with different dependencies installed on each. This is also great because you don't have to install dependencies globally on your system.

Follow next steps to install VCPKG and have it linked to cmake. The vcpkg repository is checked out at the ref in vcpkg.yml. Azure SDK code in this version is known to work at that vcpkg ref.

# Clone vcpkg:
git clone https://github.com/Microsoft/vcpkg.git
# (consider this path as PATH_TO_VCPKG)
cd vcpkg
# Checkout the vcpkg ref from the vcpkg.yml file (link above)
# git checkout <vcpkg ref>
# build vcpkg
./bootstrap-vcpkg.sh
./vcpkg install --triplet x64-linux curl cmocka paho-mqtt
export VCPKG_DEFAULT_TRIPLET=x64-linux
export VCPKG_ROOT=PATH_TO_VCPKG #replace PATH_TO_VCPKG for where vcpkg is installed

If you previously installed VCPKG and dependencies, you may need to run ./vcpkg upgrade --no-dry-run to upgrade to the latest packages.

Debian

Alternatively, for Ubuntu 18.04 you can use:

sudo apt install build-essential cmake libcmocka-dev libcmocka0 gcovr lcov doxygen curl libcurl4-openssl-dev libssl-dev ca-certificates

Build

# cd to project folder
cd azure-sdk-for-c
# create a new folder to generate cmake files for building (i.e. build)
mkdir build
cd build
# generate files
# cmake will automatically detect what C compiler is used by system by default and will generate files for it
cmake ..
# compile files. Cmake would call compiler and linker to generate libs
make

Note: The steps above would compile and generate the default output for azure-sdk-for-c which includes static libraries only. See section Compiler Options

Mac

VCPKG

VCPKG can be used to download packages sources, headers and build libraries for whatever TRIPLET is set up (platform/architecture). VCPKG maintains any installed package inside its own folder, allowing to have multiple vcpkg folder with different dependencies installed on each. This is also great because you don't have to install dependencies globally on your system.

First, ensure that you have the latest gcc installed: 

brew update
brew upgrade
brew info gcc
brew install gcc
brew cleanup

Follow next steps to install VCPKG and have it linked to cmake. The vcpkg repository is checked out at the ref in vcpkg.yml. Azure SDK code in this version is known to work at that vcpkg ref.

# Clone vcpkg:
git clone https://github.com/Microsoft/vcpkg.git
# (consider this path as PATH_TO_VCPKG)
cd vcpkg
# Checkout the vcpkg ref from the vcpkg.yml file (link above)
# git checkout <vcpkg ref>
# build vcpkg
./bootstrap-vcpkg.sh
./vcpkg install --triplet x64-osx curl cmocka paho-mqtt
export VCPKG_DEFAULT_TRIPLET=x64-osx
export VCPKG_ROOT=PATH_TO_VCPKG #replace PATH_TO_VCPKG for where vcpkg is installed

If you previously installed VCPKG and dependencies, you may need to run ./vcpkg upgrade --no-dry-run to upgrade to the latest packages.

Build

# cd to project folder
cd azure-sdk-for-c
# create a new folder to generate cmake files for building (i.e. build)
mkdir build
cd build
# generate files
# cmake will automatically detect what C compiler is used by system by default and will generate files for it
cmake ..
# compile files. Cmake would call compiler and linker to generate libs
make

Note: The steps above would compile and generate the default output for azure-sdk-for-c which includes static libraries only. See section Compiler Options

Using your own HTTP stack implementation

You can create and use your own HTTP stack and adapter. This is to avoid the libcurl implementation from Azure SDK.

The first step is to understand the two components that are required. The first one is an HTTP stack implementation that is capable of sending bits through the wire. Some examples of these are libcurl, win32, etc.

The second component is an HTTP transport adapter. This is the implementation code which takes an http request from Azure SDK Core and uses it to send it using the specific HTTP stack implementation. Azure SDK Core provides the next contract that this component needs to implement:

AZ_NODISCARD az_result
az_http_client_send_request(az_http_request const* request, az_http_response* ref_response);

For example, Azure SDK provides a cmake target az_curl (find it here) with the implementation code for the contract function mentioned before. It uses an az_http_request reference to create an specific libcurl request and send it though the wire. Then it uses libcurl response to fill the az_http_response reference structure.

Link your application with your own HTTP stack

Create your own http adapter for an Http stack and then use the following cmake command to have it linked to your application

target_link_libraries(your_application_target PRIVATE lib_adapter http_stack_lib)
# For instance, this is how we link libcurl and its adapter
target_link_libraries(blobs_client_example PRIVATE az_curl CURL::libcurl)

See the complete cmake file and how to link your own library here

SDK Architecture

At the heart of our SDK is, what we refer to as, Azure Core. This code defines several data types and functions for use by the client libraries that build on top of us such as an Azure Storage Blob client library and Azure IoT client libraries. Here are some of the features that customers use directly:

  • Spans: A span represents a byte buffer and is used for string manipulations, HTTP requests/responses, reading/writing JSON payloads. It allows us to return a substring within a larger string without any memory allocations. See the Working With Spans section of the Azure Core README for more information.
  • Logging: As our SDK performs operations, it can send log messages to a customer-defined callback. Customers can enable this to assist with debugging and diagnosing issues when leveraging our SDK code. See the Logging SDK Operations section of the Azure Core README for more information.
  • Contexts: Contexts offer an I/O cancellation mechanism. Multiple contexts can be composed together in your application's call tree. When a context is canceled, its children are also canceled. See the Canceling an Operation section of the Azure Core README for more information.
  • JSON: Non-allocating JSON reading and JSON writing data structures and operations.
  • HTTP: Non-allocating HTTP request and HTTP response data structures and operations.
  • Argument Validation: The SDK validates function arguments and invokes a callback when validation fails. By default, this callback suspends the calling thread forever. However, you can override this behavior and, in fact, you can disable all argument validation to get smaller and faster code. See the SDK Function Argument Validation section of the Azure Core README for more information.

In addition to the above features, Azure Core provides features available to client libraries written to access other Azure services. Customers use these features indirectly by way of interacting with a client library. By providing these features in Azure Core, the client libraries built on top of us will share a common implementation and many features will behave identically across client libraries. For example, Azure Core offers a standard set of credential types and an HTTP pipeline with logging, retry, and telemetry policies.

Contributing

For details on contributing to this repository, see the contributing guide.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Additional Helpful Links for Contributors

Many people all over the world have helped make this project better. You'll want to check out:

Community

  • Chat with other community members Join the chat at https://gitter.im/azure/azure-sdk-for-c

Reporting Security Issues and Security Bugs

Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (MSRC) secur.nosp@m.e@mi.nosp@m.croso.nosp@m.ft.c.nosp@m.om. You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Further information, including the MSRC PGP key, can be found in the Security TechCenter.

License

Azure SDK for Embedded C is licensed under the MIT license.

az_http_client_send_request
AZ_NODISCARD az_result az_http_client_send_request(az_http_request const *request, az_http_response *ref_response)
Sends an HTTP request through the wire and write the response into ref_response.
az_result
az_result
The type represents the various success and error conditions.
Definition: az_result.h:53
az_http_request
Structure used to represent an HTTP request. It contains an HTTP method, URL, headers and body....
Definition: az_http_transport.h:79
az_span.h
An az_span represents a contiguous byte buffer and is used for string manipulations,...
az_iot_hub_client.h
Definition for the Azure IoT Hub client.
az_http_response
Allows you to parse an HTTP response's status line, headers, and body.
Definition: az_http.h:157
Generated by   doxygen 1.8.20