Monitoring geographically distributed energy sites from a single centre

Starting point

The sites were tens of kilometres apart, each running on its own local control panel. There was no consolidated view of them at the centre.

In practice that meant this: when a site faulted, the news arrived via the phone of the technician who happened to drive there. Energy meter readings were taken by hand once a month and typed into a spreadsheet, so the consumption difference between two sites only became visible at month end, and only roughly. There was data logging on site, but it was local and closed — looking at the trend leading up to an event required a site visit.

Some sites also carried the remains of an earlier remote monitoring attempt that had been abandoned. The reason was instructive: when the link dropped, the data for that interval was lost permanently, so the reports produced series full of holes that nobody trusted. A system that is not trusted does not get used.

The system built

The architecture has three layers: site, transport, centre.

Site layer. A local data collector was installed at each site (industrial router plus an edge unit). It talks to the local PLC and meters over Modbus TCP/RTU, stores values on its own disk with timestamps, and forwards them to the centre. The critical decision was this: a site must remain able to run on its own. When the centre is unreachable, local control logic and local interlocks are unaffected; the centre monitors and reports, it does not decide.

Transport layer. The link is cellular (LTE) and intermittent by nature. So transfer was built as store and forward rather than publish-and-hope: data is made durable locally first and only leaves the local queue once the centre has acknowledged it. When the link returns after an outage, the accumulated records are sent with their own timestamps, not stamped “now”. That single detail removes the reason the earlier attempt failed.

On protocols, MQTT carries events and measurements, Modbus reaches the devices inside a site, and OPC UA exposes the centre to external systems. Remote access runs over a VPN tunnel established from site to centre rather than a port open to the internet; the site accepts no inbound connections.

Centre layer. SCADA provides a site list and map view, per-site detail screens, trend charts and an alarm list. The screen hierarchy follows the physical structure of a site.

Alarm rationalisation. In the inherited alarm set, a single event produced dozens of alarms. Alarms were reduced to root causes, derivative ones suppressed, and three things made mandatory for every alarm: a priority, the action the operator is expected to take, and — if no action is required — that it should not be an alarm at all. The approach in ISA-18.2 and ISA-101 guides this work; the underlying test is simple: an alarm list the operator does not read is a safety gap.

On-call notification. Alarms above a given priority are sent as notifications according to the on-call rota. The notification chain repeats and requires acknowledgement: if it is not acknowledged it escalates to the next person. A notification that requires no acknowledgement is equivalent to a phone on silent at 3 a.m.

Energy reporting. Per-site and total consumption, peak/day/night tariff periods, site-to-site comparison and period summaries. The reports are built on raw meter records; calculated values are kept separately, so when a calculation rule changes, history can be regenerated.

The hard part

The expected difficulty was connection loss, and that one was solved. The unexpected one was time synchronisation. The clocks on the site units drifted apart, and even a few seconds of skew can invert the order of events on a chart that puts two sites side by side. The drift came partly from NTP access disappearing along with the cellular link, and partly from the local clock of some edge units drifting on its own.

The fix: each record is produced against the site unit’s monotonic counter and clock corrections are applied retroactively; records are also carried in UTC and only converted to local time for display. As a side effect of the same decision, a daylight saving transition no longer produces a duplicated or missing hour in the reports.

The second difficulty was data volume. Sending every tag once per second consumed the cellular data allowance quickly. For measurements we combined a deadband with a periodic mandatory record: a value is sent immediately when it changes meaningfully, and once per fixed interval even when it does not. Without the mandatory record, a steady value looks like “no data” for hours and counts as a gap in the report.

Measurement and acceptance

At the centre of acceptance was a data completeness measurement: the number of records expected over a period compared with the number that reached the centre. That store and forward actually works can only be proven by deliberate outage testing — the site link was cut for defined durations, restored, and the records checked for losses.

On the alarm side, the measures were alarms per operator per day and the number of alarm floods, taken over equal-length periods before and after rationalisation. For the notification chain, the time from trigger to receipt and the escalation of an unacknowledged notification were tested separately.

FAT ran at the centre with the site units on a bench and simulated signals; SAT ran on site, with a separately signed protocol per site. For energy report acceptance, the report values were compared against the meter display in the field — when the arithmetic is right but the tag mapping is wrong, a report looks internally consistent and still has nothing to do with reality. The communication side is described on the industrial communication systems page and the reporting side on the data acquisition and analytics page.