Platform-Based Embedded Systems Development Eliminates Redundancies and Adds Efficiency

Table of Contents

• Executive Overview
• The Current Situation
• Complications
• Implications
• Recommendations

 

Executive Overview

The IoT marketplace of embedded devices is growing rapidly, but it has growing pains. With the proliferation of so many digital devices, the flow of new data has become a torrent, but a highly chaotic torrent.

Devices from different manufacturers are not interoperable. For example, a new smart meter with time of day pricing cannot connect to your hot water heater, because each system was developed separately and legacy systems can remain operational for decades. To make legacy devices work as a system requires an expensive custom interface.

The trend for digital devices is towards increasing interconnection, both wired and wireless.  It is increasingly possible and cost effective to connect embedded systems devices. This interconnection capability certainly expands the number of useful applications that can be developed as it allows multiple machines to work in unison. An example is the army of micro drones coordinated to work as one system like we saw in the 2017 super bowl. The automation of a factory controlled by coordinated robots is another. Moving from demonstrations of what is possible for industrial applications is a bigger step than it may appear. The low-cost connections of many new IoT devices are not suitable for many of the avionics, military, and industry applications that must be highly secure and reliable. The difficult part is to develop such communications that meet both the functional requirements of the application and comply to a complex array of different standards.

Devices from different manufacturers and technologies do not have a common or compatible approach to performance, communications, or security. There is no one platform that can meet all needs. Even within the commercial and industrial sector there is a wide range of categories for IoT devices and the functions they need to perform. Embedded systems software platforms need to be classified and the actual functions of communications, interoperability, and security need to be appropriate for the environment and the job they do. In this report, we will focus on looking under the hood and encourage suppliers to understand the industrial marketplace for embedded systems software platforms and encourage end users to demand the functions they need in their industry. It is easy to overlook the IoDT (Internet of Dangerous Things) and it is hard enough for IoT systems to collect accurate data but harder still with hackers looking to intercept your data stream.

Considering the wide range of hardware platforms, drivers, operating systems, middleware, wired and wireless communications protocols, and Human/Machine interface displays/devices, the IoT market is tremendously fragmented, and it is very difficult to achieve uniform interoperability and security when a collection of embedded systems are connected to work together as a system.

A platform-based approach for embedded systems development eliminates redundancies and adds efficiency which saves costs and speeds development. The basic idea is to develop and share key components that can be either proprietary or open source. The advantage of open source is sharing occurs over a wide audience, and security flaws are more evident. Proprietary software components can also add functions and speed development while reducing testing, but they must be compatible to the specific project and must offer known connectivity and security standards. Using the right embedded systems software platform, new technologies can be introduced quickly in as many products as possible. The automobile industry uses the concept of product platforms. This saves in development costs and new functions can boost sales and market share. By creating software modules that can connect, and efficiently share processor resources, embedded systems applications can be shared across a product family. This provides better code reuse and increases standardizations across products.

Of course, the quest for a new and better embedded systems depends on both the software platform and the hardware platform. The hardware platforms place many constraints on the software.  The hardware may come naked (bare metal), or they can have a Board support package (BSP), or the processor can be bundled with a software platform (Operating System with tools) that makes it easier to convert into a working product.

• Key finding #1 – New IoT sensors and IoT Edge gateways are streaming into the marketplace, but there are problems with interconnection, functionality, and security that limit industrial applications.
• Key finding #2 – The software costs for an embedded system can range from 1% of the price for simple high volume consumer embedded system devices to over 95% of the cost for low volume aviation or industrial embedded systems. The lack of IoT embedded systems standards contributes to many developers writing custom one-of-a-kind software that performs much the same function as software written by other developers.
• Key finding #3 – Security for many of the new IoT sensors is missing and this lack of security limits how such sensors should be used for different kinds of industrial processes. The author proposes methods to broaden the categorization for IoT embedded systems to differentiate how they might be used.
• Key finding #4 – IoT devices and the digital world of communications can collect streaming time series data and combine this data with many other sources of data like weather, video, snapshot digital photos, and much more information about the location and conditions of assets and people.  There is a lot of bad data that is collected along with good data, and this can reduce the usability of the data for various applications and when data is used for direct process control the control system usually must know the age of the data and if the data is valid.
• Key finding #5 – The developments in AI, Machine learning, and Data Analytics are heating up competition for new embedded system processors. Regression techniques like neural networks that act on large data sets are evolving and they require significant processing power. Some neural networks are used to create new neural networks. NVIDIA graphics processors have dominated early AI, but now competitors are making new chips that will likely broaden the available suppliers.
• Key finding #6 – In a recent presentation by Michael Barr of Barr associates at the ESC conference in Boston, 19% of embedded developers claimed the did not use any standards to build the software, and 18.5% claimed they had standards, but they were not followed.  The reason is the constant drive to quickly create dirt cheap devices for the consumer market. New embedded devices may look great at first look, but problems with software could be hidden.

The Current Situation

Several market researchers put the total market for embedded systems at about $170 Billion for 2016 with a CAGR of 6%. Some of these researchers estimate the software component of embedded systems at about 6% of that figure or $10 Billion with a CAGR of about 8%. It is likely this software is only the software evident to the end user. This means that hardware costs for embedded systems are very important, but the real story is a bit more complicated as software costs vary over an incredibly wide range. The cost of simple consumer embedded devices can be 99% hardware 1% software, but the cost of low volume embedded systems used for aircraft or highly reliable industrial controls can be over 95% software costs.

The key driver is the production volume of the device, although other factors are also involved. Software development for embedded systems that work in a machine that can present a danger to equipment, people, or the environment must use a much more formal software development method that complies to applicable standards. Additionally, such embedded systems require more extensive testing and documentation.  Additional testing by outside agencies and certification costs add to the software costs.

Security Issues

Software development for embedded consumer products and commercial embedded systems and IoT devices is a cut-throat business where cost and schedule often determine the winner.  In many cases security threats are not a central concern of the developer especially if the system is not interconnected. There is a large difference in security considerations between consumer products and industrial products.  Security considerations can include the following:

• Boot loader verification – Uses Hash or checksum fingerprint to confirm the proper program is loading, rejecting malicious code.
• Embedded CPU serial numbers – This can insure that only software intended for a specific processor can load and execute. Intel has a technique called CPUID that uses this security method.
• Control of source code – Software development teams typically use SCCS (Source Code Control Systems) or version control to regulate development and testing. It can be important to know the exact source code that is implemented in any embedded device.
• Proprietary vs. Open systems – It is more difficult to hide malicious code in an open system.  Tracking what code does is much easier with source code exposed.  Some real-time operating systems are proprietary, but many new devices are using open systems. Proprietary computer operating systems like Windows, Android, or MacOS tend to have more vulnerabilities that can be effectively found only by the supplier. Open systems like most versions of Linux tend to have less security issues. Government mandated backdoors intended for government surveillance have routinely been exploited by hackers. Once a hacker has access to your computer, exploits like the recent “WannaCry” ransomware attack are enabled. Proposed mandatory backdoors are typically about modifying the encryption products used by everyone to eavesdrop on the bad guys. The FBI wants Apple to provide decryption tools to ensure that everyone’s iPhone can be decrypted on demand by the FBI.
• Industrial measurements in hazardous or regulated processes require that sensors are calibrated to known accuracy. Sample frequency is fast enough for the application, each measurement is timestamped, and each measurement includes validity status bits. In this way, every digital measurement value includes not only the value but the timestamp and validity status. Many IoT systems cannot tolerate the cost of such an overhead, but not all IoT devices are employed on hazardous or regulated processes, so there are many home automation, manufacturing, or industrial situations where less secure measurements can be used. Security is especially important if the IoT measurement is directly involved in a robotic action, starts a motor, or moves a valve.

Market Drivers for embedded systems:

• Embedded system cost- important in every market, but price of consumer products with embedded systems is often crucial to business success.
• Time to market – The next new thing needs to hit the shelves fast, especially if you are playing catch up with competitors.
• Support costs – In many products there are no software upgrades. The embedded product is shipped with firmware that should work for the life of the product. Of course, end users are the best testing environment so it could be that replacement parts have new improved software. Major software defects that lead to recalls can be very expensive to a company’s reputation.
• Functional performance – Creating the new functions the market will accept is a tricky business.  Who wants a camera that has a phone in it? But then who needs a phone that comes with a camera?

Basic themes of product development:

• Moore law is coming to an end. Intel and AMD are slowing the rate at which they introduce new chips with increased transistor density
• Cost to compute is lower in the cloud or in servers, compared to computations in embedded systems. This is especially true for applications like Machine learning, Artificial Intelligence, and data analytics used on bigger databases as these applications can take very powerful processors. Neural networks are behind much of this capability, and they are known to take significant CPU power, especially when operating on large data sets.
• Cost of data transport is high. While fiber optic broadband has very high bandwidth there are many IoT devices that use wireless, or relatively slow wired connections. Many industrial applications require fast sampling rates with guaranteed time response and secure connections.
• Cost for data storage is less in the cloud. The time series history for years of data collected by a sensor is unlikely to be stored at the sensor, yet in many industrial applications sensor data is retained for decades in a server or in the cloud.

Because the cost to move data tends to be high, there is an incentive for edge devices to do more data storage and computing locally. Latency and network reliability can be improved, or at least mitigated, by doing more in the edge devices.

Digital sensors are increasingly being connected to networks via edge gateways, with connection to the internet and the internet connects to various local/hybrid/cloud servers, and there are various ways this collected data can provide value. It is important to understand that a great deal of useful data is created by analog sensors and even cameras and bar code readers must convert light to digital signals. To understand the value of the collected data it is a good idea to start at the sensors and understand how they work.  The saying “garbage in equals garbage out” still applies in the age of IoT.

Control systems that directly manipulate a robotic output, a motor, or a control valves are good examples of the importance of understanding the measurement. The basic control scheme is INPUTS -> ALGORITHMS -> OUTPUTS. When this is executed by machines alone without humans, you will run into problems if the INPUT sensor is not valid (accurate, timely, secure).

Open vs. proprietary software platforms

What software platforms exist for embedded systems that enable IoT sensors, gateways and data collection and sharing on cloud servers?

There are firms that create and sell proprietary operating systems and firms that create open source operating systems.  There is some IoT market movement towards open source. Which is better and how can a company create a business model?  On the one hand, there are developers of embedded systems that create valuable software like real time operating systems and one may think it would be wise to have patent protection and sell that technology. So why would a company ever donate technology like source code to an open source organization? This is a complex question, and the answer depends on the business model companies use.  If you consider yourself a software company and your revenue is based on the licenses you sell you may keep your intellectual property and source code in house.  Those who purchase embedded systems can choose between systems that include proprietary software licenses or systems that have open software. Software developers that work cooperatively on projects can also have a revenue stream based on services. Non-profits like universities, government agencies, certain trade organizations and social media groups tend to share technology and knowhow and these organization can be on the leading edge of emerging technologies. There is no one right business model, and there is a wide spectrum of embedded devices and there is great incentive to keep costs down and roll out new products quickly. The operating system for embedded systems is a key software platform choice. Once this choice is made, limits are placed on where that embedded system might live.

Embedded systems software platform organizations.

There are many organizations and software standards competing to standardize the market for IoT embedded systems. Considering the size and complexity of so many IoT markets, and their wide variety of requirements it is not surprising we are still quite far away from standard, reliable, plug and play systems that are immune from cyber-attack.

https://www.linuxfoundation.org/

Edge-X foundary – This in a Linux Foundation project that has a goal to “Unify the IoT Marketplace and Accelerate Enterprise IoT Deployments”. On April 24, 2017, the Linux Foundation announced the launch of the EdgeX Foundry, an open source project to build a common open framework for Internet of Things (IoT) edge computing and an ecosystem of interoperable components that unifies the marketplace and accelerates enterprise and Industrial IoT. The initiative is aligned around the simplification and standardization of Industrial IoT edge computing, while still allowing the ecosystem to add significant value.

Founding members of the EdgeX Foundary ( https://www.linuxfoundation.org/announcements/new-edgex-foundry-unifies-iot-marketplace-to-accelerate-enterprise-iot-deployments ) include:

Advanced Micro Devices (AMD), Alleantia, Analog Devices, Bayshore Networks, Beechwoods Software, Canonical, ClearBlade, CloudPlugs, Cloud of Things, Cumulocity, Davra Networks, Dell, Device Authority, Eigen Innovations, EpiSensor, FogHorn Systems, ForgeRock, Great Bay Software, IMS Evolve, IOTech, IoTium, KMC Controls, Kodaro, Linaro, MachineShop, Mobiliya, Mocana, Modius, NetFoundry, Neustar, Opto 22, relayr, RevTwo, RFMicron, Sight Machine, SoloInsight, Striim, Switch Automation, Two Bulls, V5 Systems, Vantiq, VMware and ZingBox. Industry affiliate members include: Cloud Foundry Foundation, EnOcean Alliance, Mainflux, Object Management Group, Project Haystack and ULE Alliance.

Dell provided EdgeX Foundry with its FUSE source code base under Apache 2.0. The code consists of more than a dozen microservices and over 125,000 lines of code and will be renamed to EdgeX. The code is intended to facilitate interoperability between existing connectivity standards and commercial applications such as edge analytics, security, system management and services. IoTX project code was recently merged into the EdgeX effort.  This code was previously supported by Two Bulls and Beechwoods Software who are members of EdgeX. The code is intended to be independent of operating systems.

It remains to be seen how this open source effort could evolve as there are development path options to support small cores or large cores, and it could be that some applications will require very fast network performance so the core could evolve toward specific network protocols.

ARM (Advanced RISC Machine)

This is a unique company that does not actually build microprocessors but they license the design for many others that do build them.  ARM controls the architecture for a large array of microprocessors and microcomputers especially in the area of portable devices. They claim to be the #1 embedded system Ecosystem. ARM processors are made by TI, Philips, and Intel among others.

General rules for ARM (with exceptions):

1. Designed for efficiency (vs. ease of programming)
2. Huge variety of processors (resulting from 1.)
3. Harvard architecture
4. Heterogeneous register sets
5. Limited instruction-level parallelism or VLIW ISA
6. Different operation modes (saturating arithmetic, fixed point)
7. Specialized microcontroller & DSP instructions (bit-field addressing, multiply/accumulate, bit-reversal, modulo addressing)
8. Multiple memory banks
9. No “fat”(MMU, caches, memory protection, target buffers, complex pipeline logic, ...)

 

Object Management Group http://www.omg.org/

The Object Management Group (OMG) is an international, open membership, not-for-profit technology standards consortium, founded in 1989. OMG standards are driven by vendors, end-users, academic institutions and government agencies. OMG Task Forces develop enterprise integration standards for a wide range of technologies and an even wider range of industries. OMG’s modeling standards, including the Unified Modeling Language® (UML®) and Model Driven Architecture (MDA), enable powerful visual design, execution and maintenance of software and other processes. OMG also hosts organizations such as the user-driven information-sharing Cloud Standards Customer Council (CSCC) and the IT industry software quality standardization group, the Consortium for IT Software Quality (CISQ). OMG also managed the Industrial Internet Consortium, the public-private partnership that was formed in 2014 with AT&T, Cisco, GE, IBM, and Intel to forward the development, adoption, and innovation of the Industrial IoT.

“Thread”, https://threadgroup.org/

“Thread” was designed with one goal in mind: To create the best way to connect and control products in the home.

Thread is an IPv6-based, closed-documentation, royalty-free networking protocol for Internet of Things (IoT) "smart" home automation devices to communicate on a local wireless mesh network. In July 2014, the "Thread Group" alliance was announced, which today is a working group with the companies Nest Labs (a subsidiary of Alphabet/Google), Samsung, ARM Holdings, Qualcomm, NXP Semiconductors/Freescale, Silicon Labs, Big Ass Solutions, Somfy, OSRAM, Tyco International, and the lock company Yale in an attempt to have Thread become the industry standard by providing Thread certification for products.

Thread uses 6LoWPAN, which in turn uses the IEEE 802.15.4 wireless protocol with mesh communication, as does ZigBee and other systems. Thread however is IP-addressable, with cloud access and AES encryption. It currently supports up to 250 devices in one local network mesh.

An "AS IS" BSD licensed free and open-source implementation of Thread (called "OpenThread") has also been released by Nest.

Thread Group has three tiers of membership: Affiliate, Contributor, & Sponsor. Thread is focused on making an impact in the market, rather than serving as a standards body that tries to solve all issues for all industries

AllJoyn

“AllJoyn” https://allseenalliance.org/

AllJoyn is a collaborative open source software framework that allows devices to communicate with other devices around them. AllJoyn framework is flexible, promotes proximal network and cloud connection is optional. A simple example would be a motion sensor letting a light bulb know no one is in the room it is lighting, so it can shut itself off.

In October 2016, they announced their merger into Iotivity. Also during the merging announcement, it was stated that current devices running either AllJoyn or Iotivity will be interoperable and backward compatible. Within the merging process, the project's license is changed to Apache 2.0 License which makes it easier to other open source projects to include AllJoyn in their projects.

Though the protocol started at Qualcomm, they have signed over AllJoyn, including the source code and trademark to the Linux Foundation with the creation of the AllSeen Alliance. The AllSeen Alliance has been created to promote some type of interoperability for the internet of things, and a number of consumer brands have signed on including LG, Sharp, Haier, Panasonic, Sony, Electrolux, Sears and Arçelik. Other members include Silicon Image, Cisco, TP-Link, Canary, Changhong, Two Bulls, Affinegy, doubleTwist, Fon, Harman, HTC, LIFX, Liteon, Muzzley, Onbiron, Sproutling, Microsoft and Wilocity.

October 10th of 2016 AllSeen Alliance merged with the Open Connectivity Foundation (OCF) under the OCF name and bylaws. OCF will now sponsor both the IoTivity and AllJoyn open source projects at The Linux Foundation. Both projects will collaborate to support future versions of the OCF specification in a single IoTivity implementation that combines the best of both technologies into a unified solution.

Open Interconnect Consortium https://openconnectivity.org/

The Open Connectivity Foundation (OCF) is an industry group whose stated mission is to develop specification standards, promote a set of interoperability guidelines, and provide a certification program for devices involved in the Internet of Things (IoT). It has become one of the biggest industrial connectivity standards organizations for IoT, including in its membership Samsung Electronics, Intel, Microsoft, Qualcomm and Electrolux. Currently, there are more than 300 member companies.

The OCF tries to realize the Internet of Things also called Network of Everything. The IoT requires easy discovery, and trusted and reliable connectivity between things. The OCF delivers a framework that enables these requirements via a specification, a reference implementation and a certification program. IoTivity, the open source reference implementation of the specifications, is actively developed by different members of the OCF.

A recent press release announcing the strategic alliance agreement between the Open Interconnect Consortium (OIC) and the Industrial Internet Consortium (IIC) serves to underscore the idea that more open collaboration is necessary to develop the kind of interoperability necessary to ensure the success of the Internet of Things.

The IoTivity is common to both OCF and AllJoyn. It is an open source project with broad industrial support.  The programs are written in C and C++ and run on Linux, Tizen, Android, and Arduino.

Protocols

The IoTivity system is designed to work over the top of existing protocols such as:

• Constrained Application Protocol (CoAP)
• Wi-Fi Direct
• Bluetooth low energy
• Bluetooth
• ANT+
• Zigbee
• Z-Wave

Features

• Discovery - the ability for a given device to find other devices around it as well as offering its own services to them
• Data Transmission - the ability to pass messages between devices in a standardized way
• Device management
• Data management

What industry standards exist for PLC, DCS, Safety systems

Field Sensor Group

• HART
• FF

Industrial Ethernet (compliance organizations)

• PI (PROFINET et al)
• ODVA (CIP for Ethernet, DeviceNet, ControlNet)
• EtherCAT Technology Group (EtherCAT)
• EPSG (Ethernet PowerLink)
• CLPA (CC-Link IE)
• SERCOS (SERCOS III)

Software Standards for embedded systems

What standards guide the development of software for embedded systems? It is unlikely that one could develop an embedded system without regard to standards and then attempt to use it in an industry that requires compliance to standards.  Retroactive compliance to many of these standards is not possible, so development must consider the end use right from the beginning. Suppliers should be aware of these standards and end users should insist on embedded systems that meet standards that match the end use application or regulatory requirements.

Standard	Type of standard
ARINC 653–compliant	a software specification for space and time partitioning in safety-critical avionics real-time operating systems (RTOS)
DO-178B/C,	Avionics Safety - Software Considerations in Airborne Systems and Equipment Certification is a guideline dealing with the safety of safety-critical software used in certain airborne systems. Although technically a guideline, it is (or was) a de facto standard for developing avionics software systems.
DO-254,	Avionics Safety - Design Assurance Guidance for Airborne Electronic Hardware is a document providing guidance for the development of airborne electronic hardware, published by RTCA, Incorporated.
DO-297.	Avionics Safety - “Integrated Modular Avionics (IMA) Development Guidance and Certification Considerations”
IMA	Integrated Modular Electronics, This is not a standard but a marketplace
V2V, V2X	Automotive security and communications (vehicle to vehicle)
ISO/IEC 12207	Software Development Life Cycle - Systems and software engineering – Software life cycle processes is an international standard for software lifecycle processes. It aims to be the standard that defines all the tasks required for developing and maintaining software.
MISRA C:2012	coding guidelines - is a set of software development guidelines for the C programming language developed by MISRA (Motor Industry Software Reliability Association). Its aims are to facilitate code safety, security, portability and reliability in the context of embedded systems, specifically those systems programmed in ISO C / C90 / C99.
ISO 26262	ISO 26262 is an adaptation of IEC 61508 for Automotive Electric/Electronic Systems. It is being widely adopted by the major car manufacturers.
ASIL D, C, B, A	Automotive Safety Integrity Level  ASIL D dictates the highest integrity requirements on the product and ASIL A the lowest
IEC 61508	Functional Safety of Electrical/Electronic/Programmable Electronic Safety-related Systems (E/E/PE, or E/E/PES).
IEC 61508-3 (SIL 3)	Safety Integrity Level 3 (SIL 3)-TUV
IEC 61499	The international standard IEC 61499, addressing the topic of function blocks for industrial process measurement and control systems, was initially published in 2005. The specification of IEC 61499 defines a generic model for distributed control systems and is based on the IEC 61131 standard.
IEC 61131	IEC 61131 is an IEC standard for programmable controllers. The standard describes, ladder logic, function blocks, and structured text.
SIL (seen table below)	Safety Integrity Level  -SIL 3 is the highest possible safety rating a software only component can achieve., certified to IEC 61508-3 SIL 3 by TÜV SÜD
OSEK/VDX compliant	"Open Systems and their Interfaces for the Electronics in Motor Vehicles") is a standards body that has produced specifications for an embedded operating system, a communications stack, and a network management protocol for automotive embedded systems.  VDX (Vehicle Distributed eXecutive), joined the consortium. Therefore, the official name is OSEK/VDX
NuttX	is a real-time operating system (RTOS) with an emphasis on standards compliance and small footprint. Scalable from 8-bit to 32-bit microcontroller environments, the primary governing standards in NuttX are Posix and ANSI standards
Posix	The Portable Operating System Interface (POSIX) is a family of standards specified by the IEEE Computer Society for maintaining compatibility between operating systems. POSIX defines the application programming interface (API), along with command line shells and utility interfaces, for software compatibility with variants of Unix and other operating systems.
ANSI	American National Standards Institute - is a private non-profit organization that oversees the development of voluntary consensus standards for products, services, processes, systems, and personnel in the United States
IEEE 830	This is a general standard about the format for the “Functional Requirement Specification” and is common to all forms of software development
DDS management group	Data Distribution Service - security specification for OMG Object
IEC 61850	Communication Protocol - The abstract data models defined in IEC 61850 can be mapped to a number of protocols. Current mappings in the standard are to MMS (Manufacturing Message Specification), GOOSE (Generic Object Oriented Substation Event), SMV (Sampled Measured Values), and to Web Services. These protocols can run over TCP/IP networks or substation LANs using high speed switched Ethernet to obtain the necessary response times below four milliseconds for protective relaying.
ModBUS	Serial Communication Protocol - developed by Modicon in 1979 for use with its programmable logic controllers (PLCs). In simple terms, it is a method used for transmitting information over serial lines between electronic devices
IEC 60870-6 (ICCP/TASE.2)	IEC 60870 part 6 is one of the IEC 60870 set of standards which define systems used for telecontrol (supervisory control and data acquisition) in electrical engineering and power system automation applications.
ISO 10303	Automation systems and integration — Product data representation and exchange. It is known informally as "STEP", which stands for "Standard for the Exchange of Product model data". ISO 10303 can represent 3D objects in Computer-aided design (CAD) and related information.
IPC	circuit board standards IPC is an international industry association dedicated to furthering the competitive excellence and financial success of more than 3,700 member companies
OPC	OPC is a series of standards and specifications for industrial telecommunication. An industrial automation industry task force developed the original standard in 1996 under the name OLE for Process Control (Object Linking and Embedding for process control). OPC specifies the communication of real-time plant data between control devices from different manufacturers.
OAGi, MTConnect [29], ProSTEP iViP [81], OPC Foundation [31], DMSC [32], and MESA [33].	Standards and best practices from these organizations are sometimes proposed in ISO and IEC to facilitate broader dissemination and adoption.
IEC 61511	technical standard which sets out practices in the engineering of systems that ensure the safety of an industrial process through the use of instrumentation
EAL	The Evaluation Assurance Level (EAL1 through EAL7) of an IT product or system is a numerical grade assigned following the completion of a Common Criteria security evaluation, an international standard in effect since 1999

 

 

Complications

IoT communications security

The world of digital communications has opened many security vulnerabilities. The developers of embedded systems often completely miss vulnerabilities. Digital communications are vulnerable to a knowledgeable adversarial that can look for and often find weak links in a security defense. Cryptographers have highly secure algorithms, yet this security can be breached by poor passwords, PIN’s, stolen encryption keys, or by insiders or misplaced human trust.

IoT embedded systems offer the promise of large numbers of dirt cheap sensors, the edge gateways to connect them to servers and to applications that can make an impact on manufacturing effectiveness. Some sensors can be wireless and use zero power. Tire pressure sensors often use RFID sensors remotely powered by inductive coupling.  NFC (near field communications) work in a similar way. I’ve personally received a flat tire warning while towing a trailer. I thought it was a false alarm at first but pulled over to check. Sure enough, I had a screw puncture with air hissing out. Now most of us would not think of this system as a security hazard, but without special protections it is possible to mimic the transmission of a low tire pressure with a remote device so the motorist gets that flat tire warning and pulls over only to be robbed. The point is that security must be considered end-to-end and it starts at the sensors and must consider all avenues of security breaches.

Industrial communications security

PLC, DCS, and Safety control systems have evolved with three basic subsystems, I/O processing, the controller (control algorithm execution), and Human Interface. These subsystems are tied together with communication links.  The I/O processing is an embedded system that typically supports a “Fieldbus” connection to multiple I/O subsystems.  The connection of individual sensors to the I/O subsystem can support HART, ProfiBus, or Foundation Fieldbus standards, and many field devices support these standards, although some support only basic connectivity like 4-20ma for analog, and 24VDC, 48VAC for contacts. The I/O systems connect to the controller, usually hardwired with cable or fiber and often with redundant networks using Ethernet. 

Control systems, have the capability to connect to a wide range of third party devices and other systems.  For example, connection to motor control centers, VFD drives, robotics, complex analyzer systems, vibration monitoring, or connection to a different vendor’s controls used for various units. Typically, a large industrial facility will standardize on a single vendor’s equipment.  Normally the “main instrument” vendor or the systems integrator must insure that all such required communication links are functional with the proper data flows, response time, and failure mode protections.  These links can be ModBus, RS232, RS480, OPC-UA or use other methods, but issues like time synchronization, data validity, speed of response, and power fail restart behavior need to be considered for industrial level signal integrity. Such systems tend to be point to point direct between control system vendors which is favorable for security. Where the wire paths are within a facility then physical access is normally restricted. While these methods seem archaic they do have security advantages.

Application portability

IoT embedded systems like sensors or edge gateways can connect to controllers which are also an embedded system. While PLC’s and DCS system are known to execute ladder logic, Boolean algebra, function blocks (like PID, characterizers, LEAD/LAG or general calculations), and support structured text based programming, there is virtually no portability between platforms. Although industry has standards like IEC 61131 that generally specify PLC’s have ladder, function blocks, and structured text, you cannot take a program from one PLC, Safety, or DCS manufacturer and load it into a 2nd PLC, Safety, or DCS system. All these systems for industrial controllers come with a programming environment or software configuration platform and many have pre-configured templates.  The development of control algorithms is necessarily customized to match specific processes with unique sensors, process behavior, and output devices. In the current industrial controller marketplace, each supplier provides their own control system configuration environment. A PID function block from one vendor does not exactly match the PID block from another. When processing plants migrate from an old control system to a new control system there is a significant code porting process involved. Although it is extra work this is not always as catastrophic as it may seem.  The nature of aging process control systems tends to result in a stream of process changes where control schemes can be discarded, and automatic control systems degrade to manual operation. Field sensor and actuators also degrade with time, so a control system migration project often needs to take a fresh look at field instruments and redesigning the control systems. RE-engineering becomes evident when the engineer asks if you want us to port this piece of dead code in the old system to a memory location in the new system where it will also be dead code.

Will the real IIoT please stand up?

Is there an opening for new IoT sensors, edge gateways, and controller architectures to challenge PLC, Safety, and DCS systems? In principal, most all the pieces seem to be there.  So how can IoT architectures compete with for process automation?  It is not hard to connect an IoT sensor to an I/O embedded subsystem or an Edge Gateway, so this sensor could be used for closed loop controls. Typically, this means the calibrated sensor would be periodically digitized (A/D converter), the signal conditioned (for RTD, Thermocouple mV curve, flow orifice plate etc.) applied to engineering units with range and scale. The sensor values would be timestamped and any available status diagnostic bits like “high out of range” would be computed.  The sensor value along with the timestamp and status bits would stay together as it arrives at the controller, because the controller acts differently if the measurement is bad or out of range. Few IoT sensors have this capability today, but it is possible.

It is certainly possible to create a new control system software configuration environment via IEEE 61131, and run algorithms with a guaranteed response time in an appropriate real-time operating system, although you are now created yet another unique configuration environment and your underlying hardware platform with OS, drivers, and communication links would need to be built to certain standards depending on the process or machine and how it is used for that process or machine.

There is of course the “open automation platform initiative” being pushed by Exxon-Mobile and Lockheed Martin, that is trying to standardize on a control system architecture that would move the PLC and DCS systems to a more open and portable system. Should a large number suppliers build products to such a standard, this would effectively change the status quo and open the process automation market to new suppliers. Should there become a truly portable, standard collection of ladder logic, Function Blocks, and structured text programming.it would then be possible to export the control schemes from one supplier and import them into another suppliers control system

IoT classification standards

 In the categories below we are concerned with ranking the failure risk for IoT systems. In general, the failure consequences of using a collection of IoT devices for an application needs to be evaluated. The IoT system design should be appropriate for the application to reduce the risk to people, property and the environment.

While few IoT embedded systems are used for safety protection, there is a great need to classify IoT sensors, devices, edge gateways and systems in a manner that matches the IoT system risk of failure to the consequences for failure. The SIL rating is not used to rate a product. Individual products are only suitable for use in a SIL environment. A SIL level applies to a Safety Instrumented System, however many embedded systems do include devices that prevent the larger machine or process from presenting a hazard. These embedded systems need to be reliable for the functions they perform relative to the machine or process they are connected to. Individual devices can comply to IEC 61508 parts 1 to 3 for safety related applications.

The key tasks that should be considered for classification is some form of Process Hazard Analysis (PHA), Layers of protection analysis (LOPA), and an “attack tree” to assess security vulnerabilities.

SIL (Safety Integrity Level). 

The meaning of the SIL varies depending on whether the functional component will be exposed to ”high” or ”low” demand:

• For systems that operate continuously (continuous mode) or systems that operate more than once per year (high demand mode), SIL specifies an allowable frequency of dangerous failure.
• For systems that operate intermittently and at most once a year (low demand mode), SIL specifies an allowable probability that the system will fail to respond on demand.

In the IoT rankings below we attempt to classify IoT system reliability and comment on potential uses.

IoT – Consumer (Dirt Cheap), Devices like watches, calculators, voltmeters, rangefinders, GPS, cameras, and many other devices.   Connectivity and Security are not a high priority. Compliance to standards tends towards power supplies, battery chargers, environmental hazards. Devices in this category should not be depended on for use in any dangerous operation. No software or hardware standards apply to this category, although many embedded systems may comply to standards, conventions, or re-use standard subsystems.

IoT – Commercial. Devices like microwaves, stoves, washers/dryers, water heaters, air conditioners, home security cameras, Intercoms, home hubs/routers, USB/firewire/Ethernet devices, cash registers, bank ATM, credit card readers, bar code or Q-code readers, sewing machines, machine tools, WiFi, Z-wave, ZigBee, and other wireless devices.  The commercial class of IoT has basic interconnection and security considerations.  Machines used for financial transactions would of course need to use established security methods, but security like private/public key encryption and digital signatures are applications above the systems itself. Failure of an ATM machine is inconvenient at it does not function but stealing an encryption key or PIN is the true risk that is not related to the embedded system reliability. Devices in IoT commercial class would be expected to be reliable, and if applicable, support security. They would not need to be built to IEC 61508 standards, but devices in this category should be developed based on predetermined written functional requirements.

IoT – Industrial – IoT and embedded systems used in: drones, trucks, railroads, or cars, like collision avoidance, ABS braking systems, or emissions control, building management systems, diagnostic systems for rotating machinery, smart meters, devices used to monitor regulated manufacturing processes like food or pharmaceuticals, collection of manufacturing data in time series historians, and edge gateways

Embedded systems or IoT devices in this category should be developed to meet appropriate industry standards consistent the end use. Equipment should be very reliable and secure and may include redundant or fault tolerant systems. Systems in this class that manipulate physical devices in the process could potentially pass signals through the internet, however care must be taken in proportion to the hazard of external monitoring or signal interception. An example application would read a barcode on a package with a portable wireless device, pass this signal to an edge gateway or even to a cloud applications and finally manipulate a conveyor belt diverter to route a package to its destination. The consequence of failure is more logistical than a safety hazard depending on what is in the box. In general, it would be desirable to meet IEC 61508 parts 1 and 2 or to be suitable for a SIL 2 environment.

IoT – for hazardous processes – Examples in this class are aircraft flight control or navigation systems, medical devices containing embedded systems like pacemakers, DCS/PLC systems, and emergency shutdown systems for refineries, chemical facilities or Nuclear power plants. These systems must be highly reliable and inherently secure.  Systems in this class that manipulate physical devices in the process may need to use isolated redundant networks without any ability for unauthorized monitoring or signal interruption. The manipulation of nuclear power plant shutdown rods would require compliance to specific safety standards, where the manipulation of a flashing warning light would have different requirements. IoT hazardous processes, should have devices suitable for use in a SIL 2 or SIL3 environment and it is desirable to meet IEC 61508 parts 1,2, and 3.  The processing functions would likely be fault tolerant and may include triple redundancy.

Implications

IoT sensors are creeping into industry through the back door

Low cost and highly capable or smart IoT sensors are creeping into industrial equipment, but it is unlikely they will manipulate any pump or valve on a hazardous process if the signal path is through a corporate network. Such a smart IoT sensor or edge gateway might directly connect to the main industrial control system or it could operate as an independent isolated system. Compressor or turbine vibration monitoring is an example of an isolated system that can act to trip the machine and protect it from catastrophic failure. Vibration monitoring systems can have their own control panels and human interface, and can include advanced analytics collected by vibration signatures obtained each time the machine starts or stops.

The new wave of IoT sensors can produce measurements and data that is combined with time series data from control systems in a local or cloud outside the control system firewall. A common design concept for control systems is the operator should have a single HMI window and that HMI window is normally a part of the control system. It has however been common for control rooms to have a second tier of operator interface screens where they can connect to the corporate network, view plant information displays, view management dashboards, obtain schedule information, check E-Mail, and have an internet connection. This second-tier operator interface is a back door where IoT related applications can impact the factory floor. In this way IoT applications can advise the operator about the state of assets or steer him/her to a more efficient operation.

New embedded systems software platforms will be a game changer

Traditionally the PLC, DCS, and Safety systems have been purpose built in a modular framework in a manner that put the control function in a separate module from the human interface. Typically, this meant the controller had a real-time operating system and could be arranged in a fault tolerant hardware configuration with redundant power supplies and redundant network connections. Safety system are often triple redundant and rated for a SIL 3 environment. The human interface can be arranged with various custom display and keyboards, but larger systems tend to use a Microsoft Windows platform, with a custom graphics package that interacts with the real-time control system.  Redundancy of the HMI is achieved by simply using multiple separate window PC’s (Thick Clients) or for larger systems multiple servers that can support many remote desktops (Thin Clients). As Windows machines are vulnerable to attack from network connections they tend to operate on an isolated network with a firewall that only allow data to flow from the process control network towards the more vulnerable corporate network.

Working with Lockheed Martin, ExxonMobil’s goal is to design a new architecture that will control and optimize refining and chemical manufacturing facilities while enabling future equipment and information services such as preventative maintenance and fleet optimization. The design and implementation is based on architecture standards that will ensure modularity, interoperability, extensibility, reuse, portability, and scalability of the new system. Exxon-Mobile have filed multiple patents which presumably would be eventually made open source.

Meanwhile, Intel’s Wind River Titanium Server platform is another new potentially game changing architecture. Highly reliable servers can run multiple VM's - I.e. a reliable RTOS running in one VM with multiple Windows VM's. Failure of a VM can spawn a new replacement in 1 ms. Intel Titanium platform advertised six nines reliability -  99.9999% which compares well to many PLC and DCS systems.

Hypervisor can manage multiple OS’s in a reliable server. A hypervisor or virtual machine monitor (VMM) is computer software, firmware, or hardware, that creates and runs virtual machines. The computer on which a hypervisor runs one or more virtual machines is called a host machine, and each virtual machine is called a guest machine. A real-time OS can run in one virtual machine, while a second OS like Windows 10 can handle data history, and GUI/HMI, a third OS like Windows Server can handle MES applications like real time historian, planning scheduling and control system configuration.

This computer architecture has potential to be a game changer with great cost reductions. With redundant communications links and redundant power supplies the architecture can have high reliability. It is unlikely that such an architecture could be classified as SIL 3, but it has a chance to replace certain PLC and DCS functions.

Another computer architecture is used by Amazon to accomplish bumpless transfer for critical web servers. This looks a lot like fault tolerant controller switching. The idea is a new processor can take over from the old processor and even upgrade the software bumplessly.  You might think of this as continuing to edit a Microsoft word document while switching from Windows 7 to Windows 10.

Blue Green deployment is a technique for releasing applications by shifting traffic between two identical environments running different versions of the application.

The point is there is an active battle for the next generation of embedded process control system.

• Existing PLC DS vendors are developing innovative new products and finding ways to incorporate IoT into their offerings.
• New automation suppliers like Bedrock Automation are introducing fundamentally new products to address reliability and security.
• End Users like Exxon-Mobile are leading the challenge for a new open control system architecture to reduce the support costs and keep their refineries and manufacturing plants running with a high level of reliable and secure automation.
• Chip makers like Intel are developing new computing platforms, like Titanium Server, that may prove suitable to host control systems as virtual machines
• Server technology like Amazon Route 53 designed for bumpless transfer may prove useful for the next controls system architecture.
• The relentless pace of IoT sensors, gateways, and software are developing solutions that can work for many industrial processes. These products are knocking at the back door in industrial control systems.

Recommendations

Recommended action #1

Whether you are building an embedded device, building a machine that contains an embedded device or whether you are the end users, establish requirements that meet your needs. Of all the documents that exist in any engineering activity the FDS (Functional Design Specification) is the most important.  Forgetting to include critical functions can have huge negative consequences. The drive for low cost IoT devices has often missed the requirements for interoperability and security needed for industrial processes.

Recommended action #2

IoT sensors and edge gateways often are not purpose built for industrial applications. Like the control engineers that trace signals from the output back to the measurements, it would be wise to trace the output of applications that uses such devices back to the source to understand the whole ecosystem.  If for example you have a big data set from multiple data sources and the application is machine learning and the output is a prediction of a machine failure or product quality issue, it would be a good idea to investigate data timestamping, data validity, data accuracy and data security. Whether the machine learning applications is running in the cloud, a server, or in an edge gateway, the quality of the result is influenced by the data used. One should not lose sight of the proven method of using humans with domain expertise to make data driven conclusions by collecting data from careful selected and placed relevant measuring sensors with the required speed, accuracy, and integrity.

When engineers design the control system, they take great care to specify the measurements they need to safely regulate a process. New statistical techniques, especially neural networks, can extract valuable conclusions from ever larger data sets. Simply giving a large data set to a data scientist without domain expertise is much less likely to be successful than, a data scientist that has a solid fundamental understanding of the process and understands the source of the data he/she must work with.

Recommended action #3

There are thousands of IoT connected devices that are constructed with a constantly changing network of embedded systems. Suppliers compete fiercely for low cost and more user functions with a constant stream of new embedded devices. All end users need to understand the architectural foundation of their network.  There are many technologies and micro standards for interconnection of devices with variable security, latency, and bandwidth. There are many standards, and compliance to standards is important, it only takes one link in the chain to break security, latency, and bandwidth. There is no simple IoT or embedded systems standard to build an industrial IoT network. The recommendation is to look at the whole network end to end with all the edges known and to design to those standards that apply to the intended use.

Recommended action #4

With the digitalization of most anything electrical, there is a flood of new IoT data and much of that is making its way to capable processors with many new software applications that can provide value. While this data mostly does not have the pedigree of industrial control system data, there is still enormous potential value to use this data for useful applications like: machine learning, artificial intelligence, regression, data analytics, and various monitoring, alerting, and appropriate automation tasks. With the flood of such data, both suppliers and end users need to sift, sort, filter, and organize the data, connect it to data processing engines or algorithms, and present the output to the appropriate people or machines.

Recommended action #5

Keep watching the competition for newer and better IoT embedded devices and the standards they are built to. There are many efforts to put standards to IoT devices and the underlying software and hardware platforms that support them. There are completely new software and hardware architectures that are potentially useful for industrial monitoring and control. History shows that standards for software platforms like operating systems communications protocols, and human machine interfaces tend to be short lived, and the next new thing can be hard to predict.

 

If you would like to buy this report or obtain information about how to become a client, please Contact Us