Week Number Roll Over for GPS Event Could Cause Control System Problems on April 6, 2019

Author photo: Rick Rys
ByRick Rys
Category:
Technology Trends

Like the Y2K millennial bug, there is some concern the Week Number Roll Over for GPS receivers has the potential to cause some disruption of electrical and process control systems on the rapidly approaching date of April 6, 2019.

This won't be the first time that a GPS related Week Number Roll Over (WNRO) event has occurred, as it happened once already on August 21, 1999 without major issues reported.  ARC is not expecting major disruptions on the April 6, 2019 WNRO event, but suggests collecting an inventory of your GPS receiving devices and checking WNRO compliance with your suppliers.  GPS time started on Jan 6, 1980 and the system design keeps time by incrementing the week numbers and seconds, from this starting time. This WNRO event is based on the concern that GPS receivers have been programmed with a 10-bit counter for the week number and on typical GPS receivers that counter reaches the limit of 1024 weeks (about 19.7 years) on April 6, 2019.  Typical week number counters roll over and restart at zero, a situation that may have been overlooked by programmers and software testing.

Critical Infrastructure

GPS receivers support a wide variety of critical grid and industrial control system functions that allow digital devices to stay synchronized in time.  The GPS signal contains a running timestamp that identifies the current week and current second within that week with an accuracy of up to 40ns.  GPS systems need super accurate timekeeping, which is useful for the most demanding controls system time applications.  The GPS time signal is converted to a UTC time format in the attached control system or measuring instrument.  In this way all digital devices with a clock in a network can be time synchronized.  GPS synchronization is useful for remote sensing devices, because it does not depend on a cellular or internet connection and is reliable and accurate.  Data collected from multiple geographically dispersed digital instruments, RTUs, IEDs, controllers, embedded systems, and PLCs that may each have their own GPS receiver and timekeeping.  These remote devices can be connected to a central monitoring and control system or transmission and distribution SCADA system.  This disparate data from multiple remote systems can be ordered in time to accurately show the sequence of events for trips, alerts, alarms, human intervention, or control or safety system actions. Local controllers in a DCS or SCADA system can synchronize a designated networked computer with a GPS timekeeping system, like IRIG B for example.  Some control systems have a master timekeeper that periodically updates devices in the DCS network from the designated timekeeping computer, avoiding the need for each device to have its own GPS connected antenna.  Depending on system design the timekeeping accuracy for industrial systems might range below a microsecond to tens of seconds.

Timestamping for fast sampling devices, like synchrophasors or rotating machinery vibration sensors, can collect data at up to 50,000Hz (20 microseconds) and every data sample can be timestamped and have quality data (bad, out of service) attached.  Such data may be buffered locally and filtered before appropriate sets of data are transmitted to central control or logging systems.  Control systems and data historians that receive data from multiple instruments may support control applications, alarm monitoring, and sequence of events applications that depend on these timestamps.

Controllers or safety systems may not have a battery backed clock, so they may boot up and set the time to an arbitrary value, like the beginning of Unix time (Jan 1, 1970), until a timekeeper function updates the controller time.  Once the controller time is synchronized, the loss of the timekeeper function means that time is kept by the processor clock.  A few years ago, I recall testing that showed ordinary PC clocks tend to lose several seconds in a day assuming they are isolated from an external timekeeping function.  Unix time is kept by counting the number of seconds from Jan 1, 1970 and a 32-bit counter will roll over on January 19, 2038 (the so-called Year 2038 problem).

Possible behaviors.

Several things could happen.

  • The GPS receiver was programmed correctly, and time smoothly marches on.  After all any programmer that creates a week number counter with 10 bits should know it would roll over in about 19.7 years from now and simply count the number of rollovers to compensate.
  • The GPS receiver rolled over and the time is reset to January 6, 1980, or the GPS receiver did not roll over, possibly halting the time.  This could have some bad consequences for displaying the chronological list of alarms, for trending with time on the x-axis, for evaluating the sequence of events, or for storing real time streaming data into a historian of any other software application that might use the timestamp information.  Some programs could crash or halt.

We do have some experience with time rollover events, and the Y2K event, or millennium Week Number Roll Over for GPS GPS%20satellite.JPGbug, showed the concern to be real, but an overabundance of caution meant there were only minor consequences.  Prior testing of industrial control systems did find problems and some programs simply crashed and would crash again if restarted.  To insure important programs remained operational, identified bugs were fixed, tested, and were dispatched to affected sites to replace defective programs.  Teams of support engineers were on call on the night of December 31, 1999, but the number of phone calls for Y2K bugs was relatively few.  IoT devices and low-cost embedded systems sold as commodities may not have the same level of support and tracking as industrial grade control and safety systems that were developed and tested to more exacting standards.  Many industrial control and safety systems include support contracts from the software suppliers and bug tracking from the user base.

Recommendations

If you own or operate a critical infrastructure your system support engineers should check WNRO compliance of all GPS receivers used for critical control systems.  Electrical and process control systems are the focus here, but, airlines, drones, GPS navigation systems, banks, cell phones, and other applications should also be checking their systems for proper behavior.

The U.S. Department of Homeland Security (DHS) has released a memorandum about a GPS rollover event coming in April 2019.

Check this partial list of GPW receivers to see if you have such equipment on your systems: https://gpsworldbuyersguide.com/

An overabundance of caution on this WNRO event is a relatively low expense compared to the failure of critical infrastructure.

Engage with ARC Advisory Group

Representative End User Clients
Representative Automation Clients
Representative Software Clients