Published work

A list of all the Big Data Teams published work. Code repository links where available via Recently added publications are marked with a

Please contact us at ons.big.data.project@ons.gov.uk if you would like more information about any of the work we have done or are doing!

Methodology working papers

• Špakulová I., Gask K., Hopper, N.A. and James, M. (2019) Using data science for the address matching service
• Šakulová I., Dove I., Bates, A. and Turner, A. (2019) Synthetic data pilot
• Špakulová I. (2018) Caravan park recognition in aerial imagery
• Snow M. (2018) Unsupervised Document Clustering with Cluster Topic Identification
• Williams S., Sozzi A. (2017) Comparing densities of mobile cell towers with population estimates
• Gask K. (2017) Identifying caravan homes in Zoopla data
• Williams S. (2017) Statistical uses for mobile phone data - literature review
• Sozzi A. (2016) Comparing counts of electricity meters and addresses by postcode in England and Wales
• Gask K., Williams S. (2015) Analysing low electricity consumption using DECC data
• Gask K., Williams S. (2015) Comparing travel flows between 2011 Census and Oyster card data
• Abbott O. (2014) ONS Innovation Laboratories

Government Statistical Service Methodology series

• Chatzoglou C., Manassis T., Gammon S., Swier N. (2016) Use of Graph Databases to Improve the Management and Quality of Linked Data
• Williams S., Gask K. (2015) Modelling sample data from smart type electricity meters to assess potential within official statistics
• Swier N., Komarniczky B., Clapperton B. (2015) Using geolocated Twitter traces to infer residence and mobility

Office for National Statistics papers

• Greenaway, M. (2018) ONS Web-scraping policy
• Williams S., Weakley S. (2017) Research Outputs: Using mobile phone data to estimate commuting flows
• Bhardwaj H., Flower T., Lee P., Mayhew M. (August 2017) Research indices using web scraped price data: August 2017 update
• Metcalfe E., Flower T., Lewis T., Mayhew M., Rowland E. (2016) Research indices using web scraped price data: clustering large datasets into price indices (CLIP)
• Breton R., Flower T., Mayhew M., Metcalfe E., Milliken N., Payne C., Smith T., Winton J., Woods A. (2016) Research indices using web-scraped data
• Beeson J., (2015) Web scraped data - extreme price changes
• Anderson B., Newing A. (2015) Using energy metering data to support official statistics: A feasibility study

Big Data European Statistical System Network (ESSnet Big Data)

Wikis

• ESSnet WP1 Web-scraping job vacancies wiki
• ESSnet WP2 Web-scraping enterprise statistics wiki
• ESSnet WP5 Mobile phone data wiki
• ESSnet WP7 Multiple Domains wiki

  Specific Papers

• ESSnet WP5 Final Reports
• ESSNet pilot report on web scraping for job vacancy statistics Nigel Swier, Frantisek Hajnovic (ONS, UK) Ingegerd Jansson, Dan Wu (SCB, Sweden) Boro Nikic (SURS, Slovenia) Christina Pierrakou (ELSTAT, Greece) Martina Rengers (DESTATIS, Germany), August 2017
• ESSNet WP7 pilot report: Social media sentiment on events and links with well-being Sozzi, A., Morris, C., Brett, K., Gask K. and Swier, N.
• ESSNet Google Trends as a source for measuring sentiment and personal well-being Nigel Swier, September 2016

Other published papers and work

• Hajnovic F., Sozzi A. (NTTS 2019 Conference paper) Outlier Detection Methods for mixed-type and large-scale data like Census
• Gask K. (RSS 2018 Conference paper) Statistics on jobs, businesses and people – where data science is adding value
• Rowland E., Lawrence J., Davis N., Fitzroy A., Vince B., Elliot D. (RSS 2018 Conference paper) Traffic flow as an early indicator for GDP growth
• Sozzi A., Greaney S. (GSS Symposium 2018 Conference paper) Using Machine Learning and NLP to automate Crime Survey for England and Wales (CSEW) offence coding
• Hajnovic F. (2018 European Conference on Quality in Official Statistics paper) Measuring the quality of commercial and big data sources for official statistics
• Swier N. (ISI 2017 Conference paper) How should web scraping be organised for official statistics?
• Lewis E. (2017, MSc Dissertation, University of Cardiff) Using open source data to measure national wellbeing
• Abbott O., Lee P., Upson M., Gregory M., Duhaney D. (Statistical Data Science conference 2017 Book Chapter) Blending Data Science and statistics across government. Statistical Data Science Chapter 10.
• Sozzi A. (NTTS 2017 Conference paper) Measuring Sustainability Reporting using Web Scraping and Natural Language Processing
• Naylor J., Swier N., Williams S. (IAOS 2016 Conference paper) Estimating population mobility using big data sources - the benefits and the challenges Slides
• Swier, N. (Eurostat Social Statistics Conference 2016) Webscraping for Job Vacancy Statistics
• Naylor J., Swier N., Williams S. (NTTS 2015 Conference paper) The ONS Big Data Project
• Chatzoglou C., Manassis T., Gammon S., Swier N. (2016 GSS Methodology Symposium paper) Use of graph databases to improve the management and quality of linked data

Videos of us!

• Data Science in official statistics: The Story so far - RSS Conference 2018
• Owen Abbott keynote ‘Beyond the traditional - Data Science in Official Statistics’ LIDA Seminar Series
• Owen Abbott interviewed at Central Government, Business and Technology conference, 2016
• Jane Naylor presenting at the RSS Conference, 2015
• Jane Naylor interviewed at Data for Policy, 2015
• Nigel Swier interviewed at Data for Policy, 2015
• Tom Hunter-Smith interviewed at Data for Policy, 2015