How Notion Baked Intelligence Into the Server to Simplify the IoT

Keeping the Architecture Simple and Flexible

The most significant elements of the Notion's architecture have been the implementation of a tiers that includes programmable sensor nodes; end-to-end encryption; simple wireless gateways; and a server tier for aggregating data, doing analytics, sending alerts, and for allowing integration with other IoT services.

We removed technology from the buying decision for the user.

Each device is an all-in-one sensor package for many things someone would need to track around the home. For example, one sensor can tell if the doorbell knocks, or if a door is opened. There are all sorts of programmable thresholds for determining when an event has occurred, and if it has, information about this event is encrypted and then passed through the gateway to the cloud application tier.

The sensors collect data and do some fairly complex math to convert the raw sensor data into events. These are encrypted with AES 256 encryption and sent to the server which convert these events into actionable insights. "We are not just showing you accelerometer data, temperature data and sound data, we are turning it into something that is meaningful for people," Margoles explained.

For example, a user might not want to track if the front door opens during the day but they might be interested in receiving an alert if it opens at night. The sensor captures that information and the backend software can alert accordingly.

The sensors support over the air update capabilities. The goal is to get the basic devices out to the masses and for people to find use cases that are unique and that they are excited about. This makes it possible to optimize the firmware on the sensor and onboard the bridge after the fact.

The bridge is more or less streaming sensor data to the cloud. The sensors are optimized to only send important information in order to optimize battery power. This allows each device to get about two years of battery life depending on the use case. The use of encryption means that the sensor data and controls can be sent via the closest gateway, even if it happens to be at a neighbors house. The data rate sent by a collection of five sensors is less than 100,000 kilobytes per day.

This approach provides high flexibility and could also make it easier to scale up the infrastructure across a larger area such as a business complex. The current architecture can support nodes of up to 1200 sensors. In the long run. Notion is also looking at supporting businesses and B2B and B2C applications for insurance and properties businesses.

Using Microservices to Separate out Development and Deployment

The application infrastructure itself is built across a series of microservices which are each responsible for different parts of the data lifecycle using Docker as a container for each microservice. For example, one application gets event data from the sensors and send it to another for determining that the smoke alarm went off. Similar microservices are used for determining if motion data needs to be analyzed. This can tell you if the door opened or closed, or if someone knocks on the door.

The team started with all Ruby on rails, which was built as one massive application. The team still uses Ruby and Ruby on rails for many of the services. R is used for statistical modeling. They are also using Postgres for the user account database.

Docker allows them to leverage the tooling that exists around maintaining, monitoring, and coordinating containers. "One of the benefits of using Docker is that we have been able to remove the platform dependencies from our development, so we can push new versions of software out with more confidence," said Jordan Stone,  chief software architect at Notion.

Taking advantage of IoT back-end services

The Notion development team made extensive use of cloud integrations so they could focus most of their development efforts on adding value rather than reinventing the wheel. "Integrations are the way to be part of the connected landscape for a long time," said Stone. "The first thing to make sure of is that you are working with reputable companies, that you know where they have been, and where they are going."

A third-party sensor database called TempoIQ was chosen because it is optimized for storing sensor data at scale. Leveraging this database allows notion to build the product much faster. Stone said, "We are not a database company. Aside from the technical challenges. There is a lot of cost in building a database application for IoT applications."

An embedded device crash reporting service called Crashlytics gathers data about problems with the sensor nodes. Crowsnest helps diagnose these problems. The backend was deployed on the Digital Ocean cloud owing to its low-cost and high-performance. Stone said, "this lets us performance scale for less than other cloud providers."

Back-end integrations have also been developed with the Nest, Home Kit, and the IFTT platforms to enable the development of hybrid applications built on these ecosystems.  Stone said, "We integrate platforms like Nest in which the feature development is done by us. There is also supporting integration with other clouds through our own API that is OAuth protected. If someone is compromised and that integration is from them to us, we can revoke their access to our clouds."

Accelerating Development with cloud services Agile Lifecycle

The whole software development  delivery process is automated with delivery controls. The development team uses a series of environments to give developers a local development environment and user acceptance testing environment that runs at small scale. Jira provides much of the software development functionality to support agile, sprints, backlogs, and new features. Trello  is used to support new feature requests.

Once new code has been completed, it is pushed to GitHub, and if the unit tests pass,  it is pushed out to Quay.io, a service for managing private Docker repositories. Then all of the servers running that Docker image get notified to pull down the image with the updated code using Codeship, a continuous integration and delivery service. New code can be pushed out to the largest app in about four minutes, and to the other  microservices in about 60 seconds. They're currently pushing out anywhere from 1 to 5 new updates of delivered code to the production or UAT environment per day.

Getting Feedback from Users

A key element of Notion's  vision is to leverage users and developers extensively to add support for new applications and services. Making sense of raw data can be cumbersome and overtaxing to a user. For example a user might want an alert if the door opens when they are away, but not while at home. The presence of the phone in the home can help to weave redundant smart capabilities behind the application. Likewise, if it is raining and the window is open, the user might want to get an alert.

A huge focus of the team for mobile apps have been user tests to ensure that the applications provide the appropriate functionality. Margoles said, "All of us have been working on this for too long to know everything that should be built in. It would be self-centered of us to imagine we fully understood the landscape and what people really want."

Notion has already hosted two hackathons. Notion learned the developers wanted to add tasks to the sensors. One person displayed how many times the door was opened for getting beer. Others built an integration with Sonos.

Margoles said, "Our goal is to be platform agnostic. Notion wants the platform to expand and allow integrations with other devices and services that may exist in the home. Our stance is that the landscape is crowded. We are not trying to be the platform that controls everything. We want to provide the sensors the provide you the information you need."

How are you delivering infrastructure to support IoT devices? Let us know.