Background & Summary

Countries around the world have committed to the Paris Agreement goal of keeping the global average temperature rise to well below 2 °C above pre-industrial levels, whilst aiming to keep the rise to no more than 1.5 °C1. An essential component of achieving this goal is a rapid and deep decarbonisation of energy systems, with most scenarios involving substantial electrification of demand and increased end-use energy efficiency, among other things2. Empirical data on energy use, as well as on the factors which shape and predict it and the outcomes for end users, can play an important role in understanding the nature of this challenge and how to address it effectively and efficiently.

Within this context, we describe here the newly released IDEAL household energy dataset. The dataset consists of high-resolution sensor data on electricity and gas usage, linked to a wide range of predictor and outcome variables collected via additional sensors and surveys. Data was collected from a sample of 255 homes from Edinburgh and the nearby regions of the Lothians and south Fife, in Scotland, UK, in the 23 months up to 30 June 2018. Sensor data comprises electricity apparent power (1 Hz) and gas usage (per fixed-volume pulse) recorded at the meter, 12-second temperature, humidity and light data for each room, and 12-second boiler pipe temperatures for central heating and hot water pipes. Linked survey data includes occupant demographics, values, attitudes, and self-reported energy awareness, household income and energy tariffs, as well as a range of building, room and appliance attributes. Also included are variables describing local weather and each locality’s level of urbanisation, from secondary data sources.

In addition, 39 homes formed an ‘enhanced group’, having additional sensors to measure real power electricity use for the whole home, selected sub-circuits, and a selection of high power and user-controllable electrical appliances, and temperature sensors to indirectly indicate usage of gas-using or boiler-using appliances like hot water outlets, individual radiators and cooker hobs.

Data was collected from participating homes for between 55 and 673 days, with a mean of 286 days, median 267 days. Participants were going about their normal daily lives, although their involvement in the research project is likely to have prompted some behaviour change. In all homes, heating and hot water were provided primarily by a gas-fired combi-boiler system. The homes included a variety of building types and sizes and family structures, including single-occupancy, multiple adults and families with children.

Data was collected as part of two EPSRC-funded projects, IDEAL and BIGSMALL. The projects had several aims, including:

  • To develop a long-life, battery-powered, wireless sensor system providing high frequency measurements;
  • To investigate residential energy demand patterns, drivers and outcomes;
  • To advance Machine Learning methods to infer: (a) appliance use in homes based on whole-home electricity data (Non-Intrusive Load Monitoring, NILM); (b) usage of gas-using appliances, boilers, and individual radiators and hot water outlets based on whole-home gas data and ambient room temperature and humidity data;
  • To co-design with participants the ‘IDEAL app’: a selection of digital energy feedback and advice features available to participants via an app on a project-provided tablet, as well as via web browsers;
  • To evaluate the energy and other effects of making different sets of features available through the IDEAL app, in a Randomised Controlled Trial experimental study.

Figure 1 provides an overview of the data collected and IDEAL app features available to different groups of participants in the study.

Fig. 1
figure 1

Summary of available data in the IDEAL household energy dataset, and IDEAL app features received, by type of participant. home.install_type and home.study_class refer to fields in the home table in the dataset, and indicate how to identify and select each group of participants by type of installation or study group. Those interested in NILM research should utilise the enhancedgroup of homes. For details of the different study groups, see the Research design section. For a summary of the data available, see Tables 3 and 4; for details, see the Data acquisition section. For a summary of the IDEAL app features received by each group, see the project website, http://www.energyoracle.org/energy-feedback.html.

The data is likely to be of value for a wide range of research purposes, including:

  • Evaluating Machine Learning methods for inferring patterns of appliance usage using whole-home energy data;
  • Studying daily and seasonal patterns of energy use; the factors which ‘drive’ those patterns, including occupant, building, appliance and weather factors; and the outcomes, including levels of expenditure, indoor temperature and humidity, and occupant satisfaction.
  • Evaluating assumptions about occupant behaviours that are incorporated into building performance models, such as the UK’s Standard Assessment Procedure (SAP).

Related datasets, suitable for research into one or more of these subjects, are listed in Table 1. The table focuses on UK household energy datasets for highest comparability, although it also includes non-UK-based datasets that are commonly used in NILM research. In many cases, the IDEAL household energy dataset includes a wider range of sensor and survey data, and/or a larger sample size and duration of participation, which may provide greater opportunities for researching these topics. The research team are in the process of disseminating analyses and results from the projects using this dataset. A list of these works is available on the lead team’s website (https://wp.inf.ed.ac.uk/sustainlab/), which will be updated as new work is released.

References

  1. United Nations. Paris Agreement. (United Nations, 2015).

  2. IPCC. Global Warming of 1.5°C: An IPCC Special Report on the Impacts of Global Warming of 1.5 °C Above Pre-industrial Levels and Related Global Greenhouse Gas Emission Pathways, in the Context of Strengthening the Global Response to the Threat of Climate Change, Sustainable Development, and Efforts to Eradicate Poverty. (Intergovernmental Panel on Climate Change, World Meteorological Organization, 2018).

  3. DECC. Smart Metering Implementation Programme: Smart Metering Equipment Technical Specifications Version 1.57. (Department for Energy and Climate Change, 2014).

  4. Hagenaars, A. J. M., de Vos, K. & Zaidi, M. A. Poverty Statistics in the Late 1980s: Research Based on Micro-Data (Office for Official Publications of the European Communities, 1994).

  5. HMRC. National Statistics: Percentile points from 1 to 99 for total income before and after tax. (HM Revenue & Customs, 2020).

  6. Scottish Government Geographic Information Science & Analysis Team, Rural and Environment Science and Analytical Services Division Scottish Government Urban Rural Classification 2016. (Scottish Government, 2018).

  7. Shove, E., Pantzar, M. & Watson, M. The Dynamics of Social Practice: Everyday Life and How It Changes (Sage, 2012).

  8. Akrich, M. In Shaping Technology/Building Society: Studies in Sociotechnical Change(eds. Bijker, W. E. & Law, J.) 205–224 (MIT Press, 1992).

  9. Steg, L. Promoting household energy conservation. Energy Policy 36, 4449–4453 (2008).

    Article Google Scholar

  10. Lindenberg, S. & Steg, L. Normative, gain and hedonic goal frames guiding environmental behavior. Journal of Social Issues 63, 117–137 (2007).

    Article Google Scholar

  11. Becker, G. A Treatise on the Family (Harvard University Press, 1981).

  12. Ferrer-i-Carbonell, A. & van den Bergh, J. A micro-econometric analysis of determinants of unsustainable consumption in the Netherlands. Environmental & Resource Economics 27, 367–389 (2004).

    Article Google Scholar

  13. Hazas, M., Friday, A. & Scott, J. Look back before leaping forward: four decades of domestic energy inquiry. IEEE Pervasive Computing 10, 13–19 (2011).

    Article Google Scholar

  14. Froehlich, J., Findlater, L. & Landay, J. The design of eco-feedback technology. CHI ’10: Proceedings of the 28th International Conference on Human Factors in Computing Systems 1999–2008 (2010).

  15. Pullinger, M., Lovell, H. & Webb, J. Influencing household energy practices: a critical review of UK smart metering standards and commercial feedback devices. Technology Analysis & Strategic Management 26, 1144–1162 (2014).

    Article Google Scholar

  16. Zimmermann, J. P. et al. Household Electricity Survey, a study of domestic electrical product usage. Final report issue 4. Report No. Intertek Report R66141 (Intertek Testing & Certification Ltd, 2012).

  17. Weather data Weather Underground https://www.wunderground.com (2018).

  18. Goddard, N. et al. The IDEAL Household Energy Dataset. Edinburgh DataSharehttps://doi.org/10.7488/ds/2836 (2021).

  19. Ofgem. Typical Domestic Consumption Values Ofgemhttps://www.ofgem.gov.uk/gas/retail-market/monitoring-data-and-statistics/typical-domestic-consumption-values (2020).

  20. Browne, A. L., Anderson, B. & Medd, W. Lancaster SPRG/ARCC Water Surveyhttps://eprints.soton.ac.uk/439637 (2011).

  21. Axsen, J., TyreeHageman, J. & Lentz, A. Lifestyle practices and pro-environmental technology. Ecological Economics 82, 64–74 (2012).

    Article Google Scholar

  22. Ministry of Housing, Communities & Local Government. English Housing Survey 2013 to 2014: Questionnaire and physical survey formhttps://www.gov.uk/government/publications/english-housing-survey-2013-to-2014-questionnaire-and-physical-survey-form (HM Government, 2015).

  23. ONS Social Surveys. Living Costs and Food Survey Volume B: The Household Questionnaire User Guide, January – December 2011, Great Britain and Northern Ireland. Study No. 7047 http://doc.ukdataservice.ac.uk/doc/7047/mrdoc/pdf/2011/7047volume_b_household_questionnaire_2011.pdf (UK Data Archive, 2012).

  24. Schulenberg, S. E. & Melton, A. M. A. The Computer Aversion, Attitudes, and Familiarity Index (CAAFI): a validity study. Computers in Human Behavior 24, 2620–2638 (2008).

    Article Google Scholar

  25. Eurofound. Third European Quality of Life Survey: Questionnairehttps://www.eurofound.europa.eu/publications/other/2012/quality-of-life/third-european-quality-of-life-survey-questionnaire (Foundation for the Improvement of Living and Working Conditions, 2012).

  26. University of Essex, Institute for Social and Economic Research. Understanding Society: Waves 1-10, 2009-2019 and Harmonised BHPS: Waves 1-18, 1991-2009. 13th Edition. Study No. 6614 https://doi.org/10.5255/UKDA-SN-6614-14 (UK Data Archive, 2019).

  27. Inglehart, R. et al. World Values Survey Wave 6 (2010-2014), Official Questionnaire v.4, June 2012 http://www.worldvaluessurvey.org/WVSDocumentationWV6.jsp (JD Systems Institute, 2014).

  28. Webb, J., Hawkey, D., McCrone, D. & Tingey, M. Bringing Warmth to Wyndford: Household Survey of New District Heating. Heat and the City Project https://heatandthecity.org.uk/project/heat-and-the-city/ (2014).

  29. Webb, J., Hawkey, D., Mccrone, D. & Tingey, M. House, home and transforming energy in a cold climate. Families, Relationships and Societies 5, 411–429 (2016).

    Article Google Scholar

  30. Haines, V. et al. The DEFACTO field trial: Methodology and data sets. Loughborough University https://doi.org/10.17028/rd.lboro.7837940.v1 (2019).

  31. Buswell, R. et al. LEEDR Project Home Energy Dataset. Loughborough Universityhttps://doi.org/10.17028/rd.lboro.6176450.v1 (2018).

  32. Richardson, I. & Thomson, M. One-Minute Resolution Domestic Electricity Use Data, 2008–2009. UK Data Service https://doi.org/10.5255/UKDA-SN-6583-1 (2010).

  33. Kolter, J. Z. & Johnson, M. J. REDD: A public data set for energy disaggregation research. Proceedings of the SustKDD Workshop on Data Mining Applications in Sustainability (2011).

  34. Murray, D., Stankovic, L. & Stankovic, V. An electrical load measurements dataset of United Kingdom households from a two-year longitudinal study. Sci. Data 4, 1–12 (2017).

    Article CAS Google Scholar

  35. Webborn, E., Elam, S., McKenna, E. & Oreszczyn, T. Utilising smart meter data for research and innovation in the UK. ECEEE Summer Study Proceedings, 1387–1396 (2019).

  36. Kelly, J. & Knottenbelt, W. The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes. Scientific Data 2 (2015).