We’ve recently published a white paper on the energy savings achieved through various initiatives undertaken by OVO relating to smart meters and their data streams. Since the official national smart meter rollout began in 2016, there has been very little information made publicly available on how well they are actually doing as far as reducing energy waste is concerned.
The motivation behind this work is therefore twofold: firstly, to improve transparency in the industry with respect to the benefits of smart meters; and, secondly, to set up a process for benchmarking initiatives led by OVO to help our members save energy and money.
Since we are the first UK supplier to share this type of information, we want to explain the methodology used to detect (typically) small changes in energy usage as a result of modified user behaviour. Hopefully, this will be useful for other data teams seeking to do the same.
Recent OVO Energy-Saving Initiatives
As a starting point, recent energy-saving initiatives led by OVO include:
The Smart meter rollout itself, as required by the government…
For background information on the rollout, the white paper is a good place to start. The requirement applies to both electric and gas meters, with the majority of UK domestic customers having both. Most members have real-time insight into their aggregated energy consumption via an In Home Display (IHD). Subject to the appropriate permissions, meter readings are also batch submitted to our servers via a secure network connection at a time interval specified by the member.
Energy disaggregation by appliance category, available via our online portal…
For reference, aggregated consumption is the total amount used within the home across all appliances, split out only by fuel type, whilst the disaggregated view splits consumption by appliance or appliance category such as laundry, cooking and refrigeration. Eligible OVO members can access both their aggregated and disaggregated views online using a personal login, with a lag of at least one day depending on the interval selected for meter reading communications. The disaggregated view is for electricity consumption only.
The Consumer Access Device (CAD) trial, which ran for 9 months in 2017...
In 2017, a subset of members volunteered to have a CAD installed alongside their regular IHD. A CAD is similar to an IHD in many ways, the main difference being that a CAD is a cloud-connected device that accesses real-time data from smart meters and sends it to a designated cloud service. Due to the opt-in nature of the trial, this cohort of members is considered to be our pool of ‘energy enthusiasts’.
These are all covered in more detail in the white paper.
Removing Noise From Meter Read Data
Having access to smart meter data is a massive win for data teams. Where previously we relied on members to submit infrequent and irregular readings at best, or estimates at worst, now we have access to an unprecedented amount of information relating to household energy usage.
Still, measuring true energy savings from members making a real effort to curtail their usage is not straightforward, mainly because the percentage change in the underlying is often small compared to seasonal variation. In addition, systematic shifts in user behaviour and anomalous readings also have the potential to distort the measured savings. Clearly, we need to remove as much noise as possible beforehand.
To account for seasonal variation, annualised meter readings were used to compare usage pre- and post- implementation. For electricity, we used Estimated Annual Consumption (EAC) indexed by Meter Point Number (MPN) from ELEXON, the settlement service provider. For gas, Annual Quantity (AQ) from Xoserve serves the same purpose. Both EACs and AQs are calculated by annualising the meter advance for a given meter point using the relevant daily load profile. Load profiles effectively allocate a fraction of the total annual usage to each day of the year under seasonal normal conditions, and are different for electricity and gas. Xoserve also applies a weather adjustment to arrive at an AQ that is weather-desensitised, allowing us to compare usage at the same meter point over different time periods.
External factors such as more stringent regulations for new appliance efficiency, improved insulation for newbuilds and the systematic shift towards electric vehicles can cause energy consumption to trend one way or another. We checked for systematic changes in electricity and gas consumption for traditional meter users: both have trended downwards over the past 2 years. All savings reported in the white paper are net of this shift.
Finally, anomalies were removed by normalising the annualised quantities and using three standard deviations as an inclusion threshold either side of the mean.
It is important to note that EAC is a weighted average of the most recent meter advance and previous EACs at a given meter point, except where readings are more than 6 months apart. Likewise, the AQ calculation relies on meter readings being taken at least 6 months apart. What this means in practice is that changes in consumption from one month to the next are smoothed out, making it difficult to identify step changes in user behaviour, or indeed gradual changes that occur over a period of less than 6 months.
Observed Energy Savings
This methodology is perfectly acceptable for retrospectively quantifying the effectiveness of initiatives intended to deliver long-term benefits to members. For example, we now know that OVO members who have had their traditional electricity meters replaced with smart ones can expect to save 4% on average over a period of 2 years (see Figure 1). The measured effects of smart meter installation on gas usage are statistically insignificant.
Figure 1: Percentage decrease in energy consumption following smart meter rollout
We also know that, despite largely positive feedback from members, there is no measurable benefit from having disaggregated data as far as energy usage is concerned (see Figure 2).
Figure 2: Percentage decrease in electricity usage following disaggregated consumption dissemination via the online customer portal
Finally, Figure 3 shows the energy savings achieved following CAD installation over a 6 month period for members who opted in to the 2017 CAD trial. Access to real-time data on multiple devices has no impact on energy usage. However, if we limit the smart meter rollout analysis to our cohort of energy enthusiasts, their energy savings 6 months after access to smart meter data far exceeds those of the general population. These gains exist for both electricity and gas customers.
Figure 3: Percentage decrease in energy consumption following CAD installation - this is compared to the reduction achieved over the same time period following smart meter installation for (i) CAD trial volunteers (our energy enthusiasts) and (ii) all customers included in the smart meter rollout study
This is a really important result that tells us that a combination of aggregated energy usage feedback and member engagement is key to driving down consumption. Potential sources of bias are discussed at length in the white paper.
We've gone through our approach to measuring the impact of smart meter installation and subsequent energy-saving enterprises on energy consumption. This has proven to be effective for retrospective analysis.
Where the methodology falls down is when we need to track the efficacy of initiatives more closely from the outset. A lag of 6 months is simply unacceptable if we want to iterate at speed. As such, we have calculated our own annualised advances for both electricity and gas using the same methodology as the service settlement providers, but without the dependency on historic annualised quantities. This will be the subject of our next Energy Savings blog post.