Introduction
The European Agency for Safety and Health at Work (EU-OSHA) is developing the EU OSH Info System that will address different aspects of occupational safety and health (OSH) in all EU Member States as well as Iceland, Norway, and Switzerland. This is a strategic, long-term activity for EU-OSHA, based on collaboration and consensus among key stakeholders: the Directorate-General (DG) for Employment, Social Affairs and Inclusion, National Contact Points, data providers and data-holding EU institutions, as well as data providers and data-holding institutions at national level.
The EU OSH Barometer, which is one of the two components of the EU OSH INFO System – the other being the periodic ‘OSH in Europe – state and trends’ report – is available online, and currently addresses 14 major themes related to OSH. Reinforcing the section on social dialogue would contribute towards the overall improvement of the barometer. At the moment, it comprises four quantitative indicators based on the European Survey of Enterprises on New and Emerging Risks (ESENER), which show the presence (in %) of OSH institutional forms of employee representation in a company: joint consultative employment forum or similar; trade union representation; health and safety representative; and health and safety committee. In addition, the barometer features country profiles containing qualitative information on social dialogue and a data visualisation tool (DVT) providing a qualitative description of the EU Member States’ social dialogue systems.
To improve the EU OSH Barometer on social dialogue, EU-OSHA commissioned the applied social research centre Notus to develop a conceptual approach and identify an initial set of indicators for building a composite indicator of social dialogue and OSH[1].
This technical note presents the results of the application of this conceptual approach and a proposal for a composite indicator of industrial relations in the field of OSH: the OSH-IR index.
The first section describes the methodology used to compute this index. The second section presents the main results with a focus on two aspects: first, the structure of the OSH-IR index (dimensions and indicators); second, the scores obtained and their visualisation. The last section provides some concluding remarks for further analysis.
Methodology
A composite indicator (or synthetic index) measures a multidimensional concept that cannot be captured by a single indicator. It is formed by compiling individual indicators into a single index, on the basis of an underlying model of the multidimensional concept being measured[2].
The methodology used for building the OSH-IR index draws on the Handbook on Constructing Composite Indicators[3], developed by the OECD and the European Commission’s Joint Research Centre (JRC). It can be summarised in the following steps:
- adopt a theoretical framework;
- select sources and indicators;
- process data;
- establish the measurement framework;
- weigh and aggregate indicators;
- calculate and assess the index.
Adopting a theoretical framework
The OSH-IR index adopts the conceptual framework developed by EU-OSHA[1] to measure the quality of industrial relations in the field of OSH at company level.
The starting point was to focus on the purpose of industrial relations in this field[4]. In this context, EU-OSHA[1] concluded that the key objective of industrial relations in the field of OSH at company level is to ensure the participation of employers and employees (via trade unions, works councils, shop stewards, or other forms of employee representation) in the governance of OSH, including the regulation, implementation, and enforcement of OSH standards, on the basis of a shared understanding of OSH aims and a mutual commitment to fostering safety and health at work in the broadest sense.
In line with this approach, four key dimensions were considered for the assessment of the quality of industrial relations in the field of OSH.
- Representation: the right of employees to seek a union, working committee, or delegate to represent them for the purpose of regulating, implementing, or enforcing OSH standards. Employee representation is rooted in EU Member States’ labour laws on trade unions and the representation of workers in the workplace. It is associated with various forms of worker representation such as trade union workplace level sections, work councils, and shop stewards.
- Participation: this refers to employee involvement in regulating, implementing, or enforcing OSH standards at company level through indirect forms of representation (broadly speaking, the involvement of employee representatives in decision-making processes). Participation at company level can be mapped along a continuum with ‘no participation’ and ‘co-determination’ at opposite ends. Intermediate levels include participation practices in which, in line with Directive 2002/14/EC, employees receive information, or, in a further step, are consulted.
- Influence: influence is linked to the relative bargaining power and ability of employee representatives to exert influence over the governance of OSH, that is, the actual impact of employee representatives in its regulation, implementation, and enforcement.
- Involvement in OSH governance: this refers to the extent of involvement of management and employees in the design and implementation of OSH at company level, including the relevance for management and the means provided by management for the effective involvement of employees and their representatives in the governance of OSH.
The main limitations of this conceptual approach have already been highlighted by EU-OSHA[1]. Firstly, it only considers company level. Although this choice is attributable mainly to data availability (no comparative data at sectoral or national level is available for the EU-27 countries), it is also worth noting that, in the literature, company level is considered the most important with regard to social dialogue and OSH. Secondly, it does not take into account the content of social dialogue and collective bargaining in the field of OSH – also due to a lack of information. Nevertheless, the conceptual approach does include a key aspect of OSH governance: it considers how employees and their representatives are involved in the identification and prevention of OSH risks.
Selecting sources and indicators
The next step consists of selecting the most adequate sources and indicators for measuring the four dimensions. For this purpose, the indicators initially selected by EU-OSHA[1] and other potential indicators were further assessed on account of the quality criteria presented in Table 1.
These criteria are based on the quality assessment and assurance framework of the European Statistical System (ESS)(the quality assessment and assurance framework of the European Statistical System (ESS)[5],[6] evaluates the quality of already produced statistical outputs, based on principles 11 through 15 of the European Statistics Code of Practice (CoP))[7], and the literature on selecting and processing indicators[8],[9].
Table 1. Quality criteria for assessing indicators
| Relevance | Indicators should have a clear conceptual link with industrial relations in the field of OSH at company level. |
| Accuracy and reliability | Indicators should be accurate and reliably measure the phenomenon they intend to measure and not be confounded by other factors. Indicators should be sensitive to changes, and changes in their values should have a clear and unambiguous meaning. |
| Intelligibility and easy interpretation | Indicators should be sufficiently simple to be intuitive and unambiguously interpreted in practice. Indicators should have a clear meaning with respect to the phenomenon analysed, either ‘positive’, meaning that higher values are considered positively, or ‘negative’. |
| Timeliness and punctuality | Indicators should be released in accordance with an agreed schedule and soon after the period to which they refer. The time gap between the collection and reporting of data should be minimal, to ensure that indicators are reporting current rather than historical information. |
| Sustainability | It indicates the updating frequency of indicators. If an indicator aims to monitor progress, special one-off surveys should not be included. |
| Coherence and comparability | It shows whether concepts, definitions, methodologies, and actual data are consistent internally and across space and time. |
| Accessibility and clarity | It indicates if data are available and accompanied with adequate explanatory information (metadata). |
Source: Eurofound[8]
The result was a dashboard on the quality of industrial relations in the field of OSH at company level, with a set of indicators for the EU-27 countries . The dashboard is presented in Annex 1 and includes the definition of and the source for each indicator, as well as other details.
The indicators come from the following sources.
- The European Survey of Enterprises on New and Emerging Risks (ESENER) carried out by EU-OSHA.
- The European Working Conditions Survey (EWCS) carried out by Eurofound.
- The European Company Survey (ECS) carried out by Eurofound. It includes the survey of management (ECS-MGT) and the survey of employees’ representatives (ECS-ER).
- The Organisation for Economic Co-operation and Development (OECD)/Amsterdam Institute for Labour Studies (AIAS) 2021 database on Institutional Characteristics of Trade Unions, Wage Setting, State Intervention, and Social Pacts (OECD-AIAS-ICTWSS, 2021).
It should be noted that the analysis of the three surveys over the course of time revealed comparability problems in almost all potential indicators. For this reason, only indicators from the latest round of surveys were assessed (2019 ESENER, 2021 EWCS and 2019 ECS).
In total, 37 indicators meeting the ‘relevance’ criterion were assessed, out of which 28 were considered normative indicators. Normative indicators are those that meet the ‘intelligibility and easy interpretation’ criterion: that is, they can be clearly understood as positive or negative for the quality of industrial relations. By definition, only normative indicators can be used to build a composite indicator. The rest of the indicators (classified as contextual) are potentially useful for other types of statistical analysis.
The assessment also revealed that the three normative indicators based on the 2019 ECS-ER survey had problems with accuracy and reliability due to the low number of responses, with five countries having supplied fewer than 15 responses (Czechia, Greece, Ireland, Latvia and Malta). Therefore, the data from these five countries were regarded as missing.
The indicators are both cardinal and ordinal (that is, based on a scale). Both types of indicators were treated with statistical methods for quantitative variables. According to the literature, this approach can be used when the objective is to maintain comparability among the indicators, to allow for easy interpretation and to achieve consistent results that are not distorted by the assignment of the ordinal categories[10],[11],[12].
Taking the dashboard as a point of departure, the following sections explain the process of selecting and retaining the normative indicators to calculate the OSH-IR index.
Processing data
All indicators included in the dashboard were processed by following the steps below.
- Orientation
All the indicators of a synthetic index must be oriented in the same direction, meaning that higher values indicate either better or worse performance. In our case, this condition was already met by all indicators, with higher values indicating better performance. Therefore, it was not necessary to reverse any indicator.
- Detection and treatment of outliers
The presence of outliers may result in polarised and biased overall results. For this purpose, values outside the 1.5 interquartile range were checked. Indicators containing outliers were defined as those having distributions with an absolute skewness value greater than 2 and a simultaneous kurtosis greater than 3.5.The analysis showed that the dashboard contains no outliers.
- Imputation of missing data
The construction of a composite indicator requires a complete dataset. The dashboard contains two types of missing data.
- Indicators from the OECD/AIAS ICTWSS database that are not available for all countries for the same year. In this case, the most recent value was used.
- Indicators from the 2019 ECS-ER survey that are not available for five countries, due to the aforementioned accuracy problems (low response rate). In this case, the average of the available data within each indicator was used.
- Normalisation
Normalisation refers to the process of harmonising indicators with different measurement units and ranges of variation. This is needed to ensure the comparability of the indicators to be included in a synthetic index. Three types of normalisation methods were applied:
- Standardisation: for every indicator, the value of each country is subtracted from the average across countries and then divided by the standard deviation across countries. The distribution of the new indicators has a mean of 0 and a standard deviation of 1.
- Min-max normalisation: for every indicator, the value of each country is subtracted from the minimum value registered, then divided by its observed range and finally multiplied by 100. All normalised indicators have the same range [0, 100]
- Min-max normalisation based on the theoretical ranges: for every indicator, the value of each country is subtracted from the minimum value that the indicator can register theoretically, then divided by its range and finally multiplied by 100. All normalised indicators have the same range [0, 100]
The final normalisation method will be the one that provides the most robust index, as explained below.
Establishing the measurement framework
The measurement framework is defined as the final set of indicators used to compute the index and their structure in dimensions. It is derived from the dashboard and the following multivariate analysis:
- Correlation analysis: this consists of computing the bivariate correlations of all indicators based on the Pearson coefficient. The aim is to explore the interrelations between indicators and maintain those with significant and positive correlations.
- Principal component analysis (PCA): this seeks to reduce dimensions and generate new variables called ‘components’. Components are based on the linear combinations of the set of variables and preserve the maximum possible total variance of the original dataset. PCA is used to explore the underlying structure of the data, particularly how different variables change in relation to each other and how they are associated.
- Cronbach Coefficient Alpha: this is a measure of reliability based on the correlation between indicators. A high coefficient denotes that the indicators measure the underlaying multidimensional phenomenon effectively.
This multivariate analysis permits computation of the index as result of a set of indicators that work well based on their internal statistical relationship and in concordance with the conceptual framework.
As a result of this analysis, seven indicators were selected and structured in two dimensions. The final correlation matrix shows strong and significant correlations, and the Cronbach Alpha Coefficient value is good (0.79). Annex 2 presents details of the multivariate analysis. The conceptual discussion of the measurement framework is addressed in section 3.
Weighting and aggregating indicators
The computation of a composite indicator requires us to weight and aggregate indicators. Weighting reflects the relative importance of each indicator and dimension. Thus, any weighting method reflects a value judgment3. Two weighting methods were considered adequate and were therefore tested.
- Equal weights of dimensions: the two dimensions have the same weight (0.50). The weight of each indicator is obtained by dividing the weight of the dimension by the number of indicators of the corresponding dimension.
- Weights based on the PCA: the weight of the indicators is based on the outcomes of the PCA (contribution of each indicator to the total variance explained by the principal components). The weight of each dimension is calculated by adding the weight of all the indicators included in the dimension.
Concerning aggregation, indicators were combined using the arithmetic mean to calculate each dimension. Two options were then tested to combine the dimensions and calculate the index: the geometric mean and the harmonic mean. The choice to test either the geometrical or the harmonic mean for aggregating dimensions is conceptual. The use of geometric or harmonic aggregation means (instead of the arithmetic mean) is intended to reduce the compensability effect between dimensions. A country with a low score for one dimension will need a much higher score on the other to improve the overall score.
The final weighting and aggregation method will be the one that provides the most robust index according to the analysis explained the next section.
Calculating and assessing the index
This final step follows the multi-modelling approach applied in the Gender Equality Index[13]and Eurofound’s composite indicator of industrial relations[8]. This approach can be summarised as follows.
- Calculating all potential versions of the index based on the combination of the different methods explained above. Therefore, 12 versions of the index were calculated, combining 3 normalisation methods, 2 weighting methods, and 2 aggregation methods.
- Selecting the most robust index. For this purpose, the median of the 12 versions was calculated for each country. Next, we computed the difference between each version and the median by country. Finally, the version that minimises these differences and lies closest to the median was selected as the most robust one.
Figure 1 presents the methods used to compute the OSH-IR index. Indicators are normalised through the min-max method based on the theoretical ranges; weighted based on the results of the PCA; and aggregated using the arithmetic mean to calculate the dimensions, and then the harmonic mean to obtain the overall index.
Figure 1: OSH-IR index - Methods used for normalisation, weighting and aggregation
Source: Authors’ elaboration.
Conducting this multi-modelling approach reduces the subjectivity associated with the selection of a single option for normalisation, weighting, or aggregation; and contributes to making the composite indicator robust and transparent.
The quality assessment of the OSH-IR index attests to its robustness. The distribution of the differences between the country rank provided by the selected index and the country ranks of the other 11 versions was calculated. The distribution shows a clear peak around 0, which is a good indication of robustness. More concretely, 45% of the cases do not change their position, 73% shift in rank by one position, and 85% of cases shift by two positions at most (see Annex 4 for further detail).
Composite indicator results
Dimensions and indicators
Table 2 presents the measurement framework of the Social Dialogue and OSH Index by dimensions and indicators.
Table 2: Industrial relations in the field of OSH Index – Dimensions and indicators
| Dimension | Indicator | Definition | Source |
| Industrial democracy and OSH employees’ involvement | Rights of works councils | 0 = Works council or similar (union or non-union) bodies of employee representation confronting management do not exist or are exceptional.
1 = Information and consultation rights (without judicial redress).
2 = Economic and social rights, consultation (advice, with possibility of judicial redress).
3 = Economic and social rights, including co-determination on some issues (e.g., mergers, take-overs, restructuring, etc.). | OECD-AIAS-ICTWSS 2021 |
| Employee representation in the workplace (coverage employees), as reported by employees | Percentage of employees in workplaces with any form of official employee representation body (as reported by employees). | Eurofound, EWCS 2021 |
| Collective bargaining coverage | Adjusted bargaining coverage rate: proportion of all wage earners with a right to bargaining, WCB*100/(WSEE-WSTAT), (0-100). WCB: employees covered by collective bargaining. WSEE: employed wage and salary workers WSTAT: employees excluded from collective bargaining. | OECD-AIAS-ICTWSS 2021 |
| Establishments where employees are usually involved in the design and implementation of risk assessment | Percentage of establishments where employees are usually involved in the design and implementation or risk assessment (as reported by those ‘who know best about health and safety in the establishments’). | EU-OSHA, ESENER 2019 |
| Establishments where employees have a role in the design and set-up of measures to address psychosocial risks | Percentage of establishments where employees have a role in the design and set-up of measures to address psychosocial risks (as reported by ‘those who know best about health and safety in the establishments’). | EU-OSHA, ESENER 2019 |
| OSH representation and influence | Employee OSH representation in the workplace (coverage establishments) | Percentage of establishments with an OSH delegate elected by employees or OSH committee (as reported by those ‘who know best about health and safety in the establishments’). | EU-OSHA, ESENER 2019 |
| Importance of meeting the expectations of employees and their representatives for addressing OSH in the establishment | Mean assessment: 0 = Not a reason
1 = Minor reason
2 = Major reason (as reported by those ‘who know best about health and safety in the establishments’). | EU-OSHA, ESENER 2019 |
Source: Authors’ elaboration.
The OSH-IR index is made up of two dimensions:
- Industrial democracy and OSH employees’ involvement
This dimension is composed of five indicators that measure the quality of industrial democracy and the extent of the involvement of employees in the design and implementation of OSH at company level.
The quality of industrial democracy is measured by three indicators in relation to the scope of employees’ representation and participation rights at company level, the coverage of employee representation in the workplace, and its incidence/influence on the governance of employment relationships through collective bargaining. The extent of OSH employees’ involvement is measured through two indicators that address the design and implementation of risk assessment, as well as of measures to address psychosocial risks.
- OSH representation and influence
This dimension is made up of two indicators. The first indicator enables the measuring of employee OSH representation at company level. The second indicator allows the measuring of the influence of employees and their representatives in the field of OSH.
The final dimensions of the index diverge from the initial theoretical framework in terms of four key aspects: representation, participation, influence, and involvement[1]. These also reflect interesting patterns about industrial democracy and OSH governance that should be further explored. On the one hand, attention should be drawn to the lack of correlation between indicators measuring industrial democracy and indicators measuring OSH representation and influence. This could be explained by the specificities of the legislation regulating the selection of OSH workers’ representatives and its lack of embeddedness within industrial relations institutions at company level (i.e., in some countries OSH workers’ representation is not exercised through workers’ representative bodies elected by the employees).
On the other hand, results show a high correlation between industrial democracy indicators and indicators measuring individual workers’ involvement in OSH governance. This could indicate that the direct participation of workers in OSH governance does not necessarily come at the expense of collective forms of workers participation, as has been suggested[14]. Rather, the data indicate that strong industrial democracy at company level can indeed favour or, at least, be compatible with workers’ direct participation in the field of OSH.
Scores and visualisation
The OSH-IR index and its two dimensions may be visualised in a number of ways. The first example, reported in Figure 2, shows the scores of the OSH-IR Index and their two dimensions for the EU-27. Country scores are presented in the left-side table. Each column presents a colour scale (ranging from green = ‘high’ to red = ‘low’) in order to indicate the relative performance in each dimension and the index. The chart on the right shows the overall index scores.
Figure 2: OSH-IR Index. Main results
Source: Authors’ elaboration.
Another example of visual presentation of the indicator is displayed in figures 3 and 4, which show the radar charts for the two dimensions across countries. Radar charts are useful to visualise normalised scores and gain insight into existing differences. Radar charts can also be used for other purposes, such as comparing two countries or differences between indicators within a country.
Figure 3: Industrial democracy and OSH employees’ involvement (Dimension 1) by country (EU-27)
Source: Authors’ elaboration.
Figure 4: OSH representation and involvement (Dimension 2) by country (EU-27)
Source: Authors’ elaboration.
Figure 5 provides an alternative visualisation of cross-country differences, contrasting the scores of dimension 1 on the x axis and dimension 2 on the y axis.
Figure 5: OSH-IR scores by dimension
Finally, maps complement other visualisation examples by providing an indication of similarities and differences by geographical area. Figure 6 shows a map indicating the overall scores of the OSH-IR index through a colour scale.
Figure 6: OSH-IR scores
Concluding remarks
The objective of this study was to build a composite indicator of social dialogue and OSH, based on the conceptual framework developed in EU-OSHA[1]. This technical note provides a detailed description of the methodology used to calculate the OSH-IR index, a conceptual discussion of its structure and, finally, suggestions for visualising the results so as to foster the usefulness of this index for comparative purposes.
As indicated by Nardo et al.[3], the reliability and potential usefulness of a composite indicator depends on two main aspects: 1) having a robust conceptual framework, and 2) having a comprehensive set of high-quality indicators measuring the different dimensions covered by this framework.
As already indicated in EU-OSHA[1], the lack of quality information is one of the main limitations to building this index. The analysis carried out also shows unexpected results with regard to the conceptual framework as well as interesting patterns about industrial democracy and OSH governance, which should be explored further. This indicates that additional research in this field is needed, as well as increased efforts to collect more comprehensive information in a systematic way.
