"To the optimist, the glass is half full.
To the pessimist, the glass is half empty.
To the engineer, the glass is twice as big as it needs to be.”
Edge AI and TinyML are having a moment. The tech world has woken up to the fact that it is possible to put machine learning models on small, inexpensive microcontrollers, and GitHub is now full of examples of TinyML models for all sorts of things.
These are interesting projects. They effectively illustrate the concept and demonstrate that it’s now feasible to do really big things on really small processors. We applaud them.
But at Reality AI we work with engineers building products. And building a product is very different from completing a project.
What do we mean by that? There’s just a lot more to product development.
“A product has to work everywhere our customers will use it and in all kinds of unpredictable conditions. Products are built by larger teams that include marketing, management, and manufacturing, as well as engineering. And everyone involved is worried about cost. Deeply worried about cost.”
Doing Projects vs. Building Products
A project is something that I create, here on my lab bench. Maybe I build it myself, or maybe I build it with a small team. It works for me (or us) in a limited range of conditions defined by our project scope, and it works on our particular apparatus.
A product, on the other hand, has to work everywhere our customers will use it and in all kinds of unpredictable conditions. It has to work on all versions and in all configurations that we sell. Products are built by larger teams that include marketing, management, and manufacturing, as well as engineering. And everyone involved is worried about cost. Deeply worried about cost.
So in part, the difference between a Project and a Product can be described as differences in Certainty and Scale. To build a product successfully and get it to release, we must achieve certainty about how it will function, certainty about the consistency of that function, and certainty about the cost -- the cost of components (the Bill of Materials, or BoM), the cost to manufacture, and the cost to support. And this certainty must be achieved at a scale that matches the scale of our customer base. Products need to work predictably and well in every context and in every way that customers expect to use them.
|Doing Projects||Building Products|
|Built by me, or by a small project team.||Built by the organization, including marketing, |
manufacturing, and management.
|Testing limited by project scope.||Testing as broad as our customer base |
and their use of the product.
|Proving feasibility, or aspects of functionality||Making something customers will buy, use, |
|Thinking about cost.||Making decisions about cost.|
When we’re talking about Edge AI / TinyML in product development, these differences have important ramifications:
|Doing Projects:||Building Products:|
|Sensor & Processing |
|Might use dev board or prototyping kit||Need production-ready components |
that can be sourced from the supply
chain at an acceptable cost.
Need to analyze and resolve cost/benefit
tradeoffs between different components,
sensor locations, and processor choices.
|ML Generality||Limited by project scope. Predicts |
accurately on project
hardware, but generality is untested.
|Must work on all versions of the device, |
regardless of small variations in manufacturing
and implementation. Must maintain accurate
predictions for the expected life of the product.
|Data Collection||Limited by project scope||Must provide some degree of coverage of a full |
range of variation in customer use and
|ML Tools||Dev kit and basic AI modeling software.||Broad set of engineering tools to understand |
issues presented by different hardware choices,
sources of variation, analysis of metadata,
explainability, and statistical confidence.
Building Products Requires Different Tools
In our experience, the generation of the TinyML model is the least challenging part of product development -- at least, it is if you’re working with a good toolset like Reality AI. Far more time consuming are the aspects of hardware development and data collection. That’s why we’ve focused on creating functionality that helps with these super-important product development problems: BoM optimization, AI-driven component selection and specification setting, AI-driven tolerance, and sensitivity testing, and features for managing the collection of data at scale by automatically checking data consistency, quality, and coverage.
YouTube demo videos and GitHub project repositories can make TinyML / Edge AI projects look easy. And with AutoML tools for Edge AI like Reality AI, they can be easier than ever. But the hard work of product development requires more than ML projects -- and we’re working to make all that easier too.
“If you expect that an ML model built for a feasibility PoC will translate directly into deployable product functionality, you are in for some disappointment.”
Products Start with Projects
Don’t get me wrong. Every product development effort our customers have launched began with a limited scope project. There is great value in a properly constructed Proof-of-Concept (PoC) at the beginning of an effort to prove feasibility and resolve important technical questions before spending time and resources on product development.
The danger in these early-stage projects is in expecting too much from them -- they rarely translate directly into a deployable or manufacturable product without further development of hardware, processing, and the machine learning models themselves. Only through extensive testing in a wide variety of circumstances do you discover all the corner cases that trip up models and require tweaks or retraining. If you expect that an ML model built for a feasibility PoC will translate directly into deployable product functionality, you are in for some disappointment.
To get there, you need to place the machine learning development into a proper product development process, including model development, component selection, integration, field testing, and end-to-end certification. While you’re at it, consider using some machine learning software that’s properly built to support that process.