Skip to content

Global Data Barometer Handbook (Draft)

You are viewing the draft Global Data Barometer Research Handbook (2021).

Introduction

Data is a source of power. It can be exploited for private gain and used to limit freedom, or it can be deployed as a public good—a resource for tackling social challenges, enabling collaboration, driving innovation, and improving accountability.

The Global Data Barometer aims to fill critical knowledge gaps on how the use of data is evolving across different sectors, regions, and countries around the world. The Barometer's design thus investigates how the policies and practices of governing, releasing, and using data for the public good are unfolding across the globe.

Building on the Open Data Barometer, which has been used to drive policymaking, advocacy, and academic research around the world, the Global Data Barometer asks the question: To what extent are countries managing data for the public good?

To answer this question, the GDB will incorporate both quantitative and qualitative assessments, drawing on primary data collected through an expert survey to be implemented by a network of global partners and researchers, as well as data to be drawn from existing secondary sources.

Primary data collection is structured around a selection of country-wide indicators which seek to measure progress in terms of data governance, capabilities, availability, and use and impact. In addition, specific thematic indicators will focus on the state of data for public good in relation to specific sectors or areas of public policy. This combined breadth and depth approach is designed to:

  • Review and assess the overall environment for data for the public good in each country studied;
  • Explore whether assessments of the overall environment reflect existing data practices in particular thematic sectors;
  • Ensure that the collection of indicator data supports reuse in ‘companion’ products developed by project partners and maximises the potential reuse value of the data collected within the broader research and policy community.

This handbook contains the majority of the modules and indicators that will be included in our upcoming expert survey. The complete list of modules and indicators, including proposed secondary data sources, will be released later in 2021. The handbook is a living document to be updated as the pilot edition moves forward in its study of data for the public good.

2021 Pilot Edition

This 2021 edition of the Global Data Barometer is being approached as a pilot in order to explore several research innovations:

  • Expanding beyond the limited open data focus of past indicators to examine data more broadly for the public good;
  • New indicator designs and research guidance;
  • New partnership models for developing and using thematic indicator data;
  • New approaches to field work through a network of regional hubs in order to benefit from localised expertise.

Lessons learned as a result of this pilot edition will be applied directly to future editions of the Barometer.

Themes & Modules

The overall structure of the Barometer was developed following a participatory design workshop, which identified four core components, a range of potential themes, and a number of cross-cutting issues to be addressed in the structure of the study.

On the Themes & Modules page, you will find a full list of the topics to be covered within the pilot edition for which researchers will be asked to provide primary data. These include core modules, thematic modules, and snapshot modules. These modules serve to structure the survey for presentation to researchers, analysts, and potential users of GDB data. Although modules may vary in scope and size, their influence on the overall Barometer score will be balanced through weights applied during index calculations.

Core Modules

Governance and capability are core components of the Barometer that have been developed as standalone modules, containing a series of primary indicators designed to provide a country-wide assessment of two of the most critical issue areas related to the development of an effective data ecosystem:

These modules are complemented by additional indicators on governance and capability within the thematic and snapshot modules. The other two core components of the Barometer, availability and use, are assessed primarily through the thematic and snapshot modules.

Thematic Modules

The overall selection of thematic modules for this pilot edition of the Barometer has been based on the desire to cover a range of government functions and a range of sustainable development goals, as well as to benefit from the expertise of partners who are contributing to the in-depth thematic modules. Currently, a range of GDB partners are contributing to the development of the following thematic modules:

Snapshot Modules

In addition to the comprehensive thematic modules, the GDB contains snapshots assessments for several more themes in order to serve the broader data research network and provide continuity with the past editions of the Open Data Barometer. Indicators for these snapshot modules are generally limited to assessing data availability. Snapshots include:

Indicator Development

The GDB’s approach to indicator development seeks to:

  • Generate indicator scores through the use of discrete elements. This is a change from the Open Data Barometer scoring method that asked experts to provide a 0–10 score, and responds both to a desire from partners for more structured, granular data, and to past feedback that the previous scoring method led to unexplained variation between assessments. In the GDB Handbook, you will see the sub-questions under each indicator used to generate indicator scores. The exact calculation of scores is not included, but will be shared later in the year following further piloting and consultation.
  • Anchor indicators in established agreements and practices. For the majority of indicators, a review of the literature, international standards, and agreements has been completed to ensure that indicators assess countries against reasonable benchmarks that are rooted in applicable international agreements or commitments. In particular, we draw on the Sustainable Development Goals as the basis for international agreement on public goods.
  • Identify bright spots and recognise different systems of government. In particular, we have sought to increase the sensitivity of indicators to federal systems and to cases where good practices may exist at an agency or sub-national level, even if the practice is not yet widespread. We have also sought to avoid cultural or high-income country biases in the design of indicators by utilising examples and evidence from a wider range of settings. However, during this consultation phase, specific feedback is sought to ensure indicators provide a fair assessment across all country contexts.
  • Generate actionable data. For a number of themes, we have worked with partners to understand how they might use the primary data generated by the Global Data Barometer, as well as how that data can support improved government practice.

Within the Handbook, all of the indicators proposed for the pilot edition of the Barometer can be reviewed via the individual modules or by accessing the master list of indicators that is accessible from all module pages. Certain indicators and selected indicator sub-questions have been included in order to build upon past editions of the Open Data Barometer and provide continuity with former study metrics, including assessing the presence of open data frameworks, initiatives, and datasets. In some cases, new secondary sources have been identified to replace previous Open Data Barometer questions; therefore, not all Open Data Barometer data categories are necessarily reflected in the GDB handbook.

Cross-cutting Issues

The design of indicators for the pilot edition of the Barometer has also taken into account the need to provide insight into a number of cross-cutting issues, including the evolution of artificial intelligence, data as a critical tool to advance development and meet SDGs, and issues of inclusion, diversity, and gender. These cross-cutting concerns are reflected in the choice of indicators and in the sub-questions they contain (e.g. assessing any evidence that AI application has been considered in data governance or addressing inclusion in the collection and use of data).

Share Your Comments and Annotations

This version of the handbook has been published in advance of our expert survey to invite feedback on the proposed indicators for the GDB pilot edition. The public review process is designed to improve the assessment methodology, as well as the overall approach of the GDB.

Please review any or all aspects of the handbook and indicate any improvement that could be made to provide clearer definitions, improve indicators, respond to different global contexts, or address important issues that may have been missed. Key objectives of the review include:

  • Support the validation of the primary indicators for the pilot edition of the Barometer;
  • Bring in perspectives, experiences, and expertise to support a ‘co-creation’ process around the Global Data Barometer methodology;
  • Help identify missing elements or refine aspects which are not clear in the handbook;
  • Provide any additional insight into the expected level of effort required by national researchers to complete the data collection phase.

We welcome both positive and critical feedback. You can share feedback as annotations, or using the feedback form on each indicator.

To read more about the Global Data Barometer project, please see the public website.