COMMISSION IMPLEMENTING REGULATION (EU) 2019/2180
of 16 December 2019
specifying the detailed arrangements and content for the quality reports pursuant to Regulation (EU) 2019/1700 of the European Parliament and of the Council
(Text with EEA relevance)
THE EUROPEAN COMMISSION,
Having regard to the Treaty on the Functioning of the European Union,
Having regard to Regulation (EU) 2019/1700 of the European Parliament and of the Council of 10 October 2019 establishing a common framework for European statistics relating to persons and households, based on data at individual level collected from samples, amending Regulations (EC) No 808/2004, (EC) No 452/2008 and (EC) No 1338/2008 of the European Parliament and of the Council, and repealing Regulation (EC) No 1177/2003 of the European Parliament and of the Council and Council Regulation (EC) No 577/98 (1), and in particular Article 13(6) thereof,
Whereas:
(1) Detailed arrangements and content of the quality reports, including indications of the method for assessing compliance with precision requirements should be specified by the Commission in relation to the data to be transmitted by the Member States to Eurostat.
(2) The measures set out in this Regulation are in accordance with the opinion of the European Statistical System Committee,
HAS ADOPTED THIS REGULATION:
Article 1
Subject matter
This Regulation lays down the detailed arrangements for the quality reports and their required content, including an outline of the method for assessing compliance with precision requirements, on the data that Member States are required to transmit to the Commission (Eurostat) under Regulation (EU) 2019/1700.
Article 2
Definitions
For the purposes of this Regulation, the following definitions shall apply:
(1) ‘proxy interview’ means an interview with someone other than the person from whom information is being sought following specific rules set by each survey specifying in which cases proxy interviews can be accepted;
(2) ‘non-response’ means the failure of a survey to collect data for all data items in the survey questionnaire or from all of the population units designated for data collection, or both, specifically:
(a) ‘unit non-response’ is a type of non-response occurring when no data are collected about a population unit designated for data collection;
(b) ‘item non-response’ is a type of non-response occurring when no data are collected about a survey variable in relation to a designated population unit;
(3) ‘sampling error’ means part of the difference between a population value and an estimate of that value derived from a random sample, arising from the fact that only a subset of the population is enumerated;
(4) ‘non-sampling error’ means error in survey estimates that is not attributable to sampling fluctuations;
(5) ‘substitution’ with regards to respondents means the replacement of a unit that was originally sampled by another unit, including an inter- or intra-household replacement;
(6) ‘eligible units’ means the set of population units selected from the sample frame that are part of the target population;
(7) ‘ineligible units’ means units in the sample that are not part of the target population;
(8) ‘net sample’, also referred to as ‘achieved sample’, means the set of population units (including substitution units) selected from the sampling frame, from which enough information has been obtained to include the unit in the survey estimations;
(9) ‘gross sample’, also referred to as ‘initial sample’, means the set of population units initially selected from the sampling frame. The gross sample comprises the eligible units (the net sample and the non-response units) as well as the ineligible units;
(10) ‘imputation’ means a procedure for entering a value for a specific data item where no response is available.
Article 3
Quality reports
Quality reports shall contain quality-related data and metadata in accordance with the quality criteria and statistical concepts set out in the Annex. These reports shall also refer to any instances in which the appropriate quality criteria were not complied with or statistical concepts have not been correctly applied, or both.
Article 4
Outline of methods for assessing compliance with precision requirements
The Commission (Eurostat) shall assess the extent to which the data transmitted by the Member States pursuant to Annex II to the Regulation (EU) 2019/1700 meet precision requirements. Where non-compliance with the precision requirements is detected by the Commission (Eurostat), it shall be assessed according to:
— its magnitude and frequency and the impact it has on the quality of the key indicators, especially their comparability;
— whether it can be promptly rectified and whether the Member States make the necessary corrections in an effective manner;
— whether non-compliance can be indirectly mitigated, particularly through estimation techniques, and whether the Member States are taking appropriate mitigating measures;
— the extent to which the Member States exercise control over non-compliance, which can arise for reasons beyond their control;
— the extent to which non-compliance persists during successive rounds of data collection;
— whether there is a corrective action plan approved by the Commission (Eurostat) and whether it is effectively implemented; assessment of such a plan will take account of the length of time needed to rectify instances of non-compliance, in particular in the case of panel data collections.
Article 5
Technical standards for transmitting quality reports
1. To support quality management and process documentation, quality reports shall be transmitted in accordance with the technical standards established by the Commission (Eurostat).
2. To allow electronic retrieval of data, quality reports shall be sent to the Commission (Eurostat) through the single entry point.
Article 6
This Regulation shall enter into force on the twentieth day following that of its publication in the
Official Journal of the European Union
.
This Regulation shall be binding in its entirety and directly applicable in all Member States.
Done at Brussels, 16 December 2019.
For the Commission
The President
Ursula VON DER LEYEN
(1)
OJ L 261 I, 14.10.2019, p. 1
.
ANNEX
Quality criteria and statistical concepts
The quality report should contain quality-related data and metadata in line with the following quality criteria and statistical concepts.
If a particular statistical concept is not relevant to a statistical operation, that concept should remain in the quality report, accompanied by the words ‘Not applicable’.
1. CONTACTS
Individual or organisational contact points for data or metadata, including contact details.
2. STATISTICAL PRESENTATION
Description of the data disseminated, which can be displayed to users as tables, graphs or maps.
2.1.
Data description
Description of the main characteristics of the dataset.
2.2.
Classification systems
Where applicable, the list of classifications and breakdowns used in data, and any deviations from European statistical standards or international standards.
2.3.
Sector coverage
Description of the main themes covered by the dataset.
2.4.
Statistical concepts and definitions, including the reference period
The list of all the variables that deviate from the standard definition, mentioning the national concepts used and any differences between national concepts and the respective data collections.
2.5.
Statistical units
Description of the observation units.
2.6.
Statistical population
Description of the target statistical population or populations to which the dataset refers, i.e. the population about which information is to be collected.
2.6.1.
Population(s) not covered
Information about any sub-populations not covered by the data collection (e.g. homeless people or people living in institutions), including description of any such population and its best quantitative estimate.
2.7.
Reference area
Description of the geographical area to which the statistical phenomenon measured relates: the geographical area covered and a list of any regions that are excluded.
2.8.
Time coverage
The periods or points in time to which the observation refers.
3. STATISTICAL PROCESSING
Operations performed on data to derive new information in accordance with a given set of rules.
3.1.
Source data
Description of the source of the raw statistical data (e.g. interviews, administrative data, any other sources). If administrative registers are used, they should be clearly described (source, primary purpose, possible shortcomings, etc.)
3.1.1.
Sampling frame
Description of the methods used to obtain or create the sampling frame.
3.1.2.
Sample design
Description of the following aspects:
— Type of sampling design (stratified, multi-stage, clustered, one stage, two stages)
— Stratification and sub-stratification criteria
— Sample size
3.2.
Frequency of data collection
Information on frequency at which a dataset is collected.
3.3.
Data collection
Description of the methods used to gather data (CAPI, CAWI, CATI, etc.). The national questionnaire used for data collection should be attached, along with its translation into English.
3.4.
Data validation
Description of procedures used for checking and validating the source and output data, including explanation of how the results of these validations are monitored and used.
3.5.
Data compilation
Description of the data compilation process (e.g. data editing, imputation, weighting, adjustment for non-response, calibration, model used, etc.) Each step of weighting should be described separately: calculation of design weights; non-response adjustment (how the design weight is corrected, taking account of differences in response rates); calibration (the level and variables used in the adjustment, method applied); calculation of final weights.
4. QUALITY MANAGEMENT
Systems and frameworks in place within an organisation to manage the quality of statistical products and processes.
4.1.
Quality assurance
Description of the quality assurance framework and/or quality management system (e.g. EFQM, ISO 9000) used in the organisation.
4.2.
Quality assessment
Description of the overall quality of statistical outputs, summarising the main strengths and any quality deficiencies in the standard quality criteria: relevance, accuracy, reliability, timeliness, punctuality, comparability and coherence. Any trade-offs between quality aspects and any quality improvements foreseen can be mentioned.
5. RELEVANCE
5.1.
User needs
Information (if available) on (any new) needs of the users in relation to data collected.
5.2.
User satisfaction
Information (if available) on level of satisfaction of data users in relation to data collected and made available.
5.3.
Completeness
Description of any non-compliance in terms of variables that are not transmitted.
6. ACCURACY AND RELIABILITY
6.1.
Overall accuracy
Summary of the various components of an assessment of accuracy, linked to a given dataset or domain:
— Description of the main sources of random and systematic errors in the statistical outputs, with a summary assessment of all errors, focusing particularly on the impact on key estimates.
— If relevant, data revision aspects.
6.2.
Sampling error
— Description of the methodology for calculating precision estimates.
— Precision measures of estimates in accordance with the technical specifications of the individual datasets.
— The standard errors at national and, where required, regional level (NUTS 2) for the main indicators as mentioned in Annex II to Regulation (EU) 2019/1700.
6.3.
Non-sampling error
6.3.1.
Coverage error
Description of divergence between the frame population and the target population.
— Frequency and timing of frame updates.
— Errors due to discrepancies between the sampling frame and the target population and sub-populations (over-coverage, under-coverage, misclassifications).
6.3.2.
Measurement error
Description of errors that occur during data collection, causing recorded values of variables to differ from true values.
— Description of efforts made in questionnaire design and testing (including addressing errors arising from multi-mode and/or multi-source data collection).
— Description of interviewer training.
— Proxy interview rates.
6.3.3.
Non-response error
Description of:
— Available characteristics of non-respondents.
— Unit and item non-response rates.
— Substitution rates.
— Gross sample size (initial sample size), number of eligible units and net sample size, including substitution units (achieved sample size)
6.3.4.
Processing error
Description of any error in processing and its impact on final data collection results, arising from faulty implementation of correctly planned implementation methods.
— Description of quality checks and the data editing process.
— Description of imputation procedures.
— Imputation rates.
6.3.5.
Model assumption error
Where applicable: description of error arising from domain-specific models needed to define the target of estimation.
6.4.
Seasonal adjustment (where applicable)
Description of statistical techniques used to remove seasonal effects that influence a data series.
6.5.
Data revision – policy
Description of policy aimed at ensuring the transparency of disseminated data, with preliminary data being revised after compilation.
6.6.
Data revision – practice
Information on data revision practice.
7. TIMELINESS AND PUNCTUALITY
Information on:
— Date of the dissemination of national results.
— Number of days between the end of fieldwork and the first fully validated delivery of data to the Commission (Eurostat).
— Date of the first full delivery of data to the Commission (Eurostat). If the delivery of data does not meet the deadline laid down in Regulation (EU) 2019/1700, a reason for the delay should be given.
8. COHERENCE AND COMPARABILITY
Description of how the requirements established in the specific domain have been fulfilled, including, where relevant, the impact of any deviations from the questionnaire and definitions.
8.1.
Comparability – geographical
Description of any problems of comparability between different regions of the country.
8.2.
Comparability – over time
Information on the length of comparable time series, including the years when any series breaks occurred, and the reasons for them.
8.3.
Coherence – cross domain
Comparison with external sources for all relevant variables, where the Member States concerned consider such external data to be sufficiently reliable.
8.4.
Coherence – sub-annual and annual statistics
If applicable.
8.5.
Coherence – National accounts
If applicable.
8.6.
Coherence – internal
Information on any lack of coherence in the output of the statistical process.
9. ACCESSIBILITY AND CLARITY
Information on:
— Dissemination formats.
— Documentation on methodology and quality.
10. COST AND BURDEN
Burden on respondents and, if available, cost associated with the collection and production of the statistical product. The average duration of household interviews should be stated. If possible and relevant, the duration of household interview should be reported by mode of data collection.
11. CONFIDENTIALITY
Information on ownership of data, indicating the extent to which their unauthorised disclosure could be prejudicial or harmful to the interests of the source or other relevant parties.
— Confidentiality policy - description of any provisions in addition to European legislation that are relevant to the statistical confidentiality applied to the data.
— Confidentiality – data treatment: general description of the rules applied to treating microdata and macrodata (including tabular data) with regard to statistical confidentiality.
12. COMMENT
Supplementary descriptive text that can be included in the quality report.
Feedback