Skip to main content

Introduction

The European Agency for Safety and Health at Work (EU-OSHA) is developing the EU OSH Info System that will address different aspects of occupational safety and health (OSH) in all EU Member States as well as Iceland, Norway, and Switzerland. This is a strategic, long-term activity for EU-OSHA, based on collaboration and consensus among key stakeholders: the Directorate-General (DG) for Employment, Social Affairs and Inclusion, National Contact Points, data providers and data-holding EU institutions, as well as data providers and data-holding institutions at national level.

The EU OSH Barometer, which is one of the two components of the EU OSH INFO System – the other being the periodic ‘OSH in Europe – state and trends’ report – is available online, and currently addresses 14 major themes related to OSH. Reinforcing the section on social dialogue would contribute towards the overall improvement of the barometer. At the moment, it comprises four quantitative indicators based on the European Survey of Enterprises on New and Emerging Risks (ESENER), which show the presence (in %) of OSH institutional forms of employee representation in a company: joint consultative employment forum or similar; trade union representation; health and safety representative; and health and safety committee. In addition, the barometer features country profiles containing qualitative information on social dialogue and a data visualisation tool (DVT) providing a qualitative description of the EU Member States’ social dialogue systems.

To improve the EU OSH Barometer on social dialogue, EU-OSHA commissioned the applied social research centre Notus to develop a conceptual approach and identify an initial set of indicators for building a composite indicator of social dialogue and OSH[1].

This technical note presents the results of the application of this conceptual approach and a proposal for a composite indicator of industrial relations in the field of OSH: the OSH-IR index.

The first section describes the methodology used to compute this index. The second section presents the main results with a focus on two aspects: first, the structure of the OSH-IR index (dimensions and indicators); second, the scores obtained and their visualisation. The last section provides some concluding remarks for further analysis.

Methodology

A composite indicator (or synthetic index) measures a multidimensional concept that cannot be captured by a single indicator. It is formed by compiling individual indicators into a single index, on the basis of an underlying model of the multidimensional concept being measured[2].

The methodology used for building the OSH-IR index draws on the Handbook on Constructing Composite Indicators[3], developed by the OECD and the European Commission’s Joint Research Centre (JRC). It can be summarised in the following steps:

  1. adopt a theoretical framework;
  2. select sources and indicators;
  3. process data;
  4. establish the measurement framework; 
  5. weigh and aggregate indicators;
  6. calculate and assess the index.

Adopting a theoretical framework

The OSH-IR index adopts the conceptual framework developed by EU-OSHA[1] to measure the quality of industrial relations in the field of OSH at company level.

The starting point was to focus on the purpose of industrial relations in this field[4]. In this context, EU-OSHA[1] concluded that the key objective of industrial relations in the field of OSH at company level is to ensure the participation of employers and employees (via trade unions, works councils, shop stewards, or other forms of employee representation) in the governance of OSH, including the regulation, implementation, and enforcement of OSH standards, on the basis of a shared understanding of OSH aims and a mutual commitment to fostering safety and health at work in the broadest sense.

In line with this approach, four key dimensions were considered for the assessment of the quality of industrial relations in the field of OSH.

  • Representation: the right of employees to seek a union, working committee, or delegate to represent them for the purpose of regulating, implementing, or enforcing OSH standards. Employee representation is rooted in EU Member States’ labour laws on trade unions and the representation of workers in the workplace. It is associated with various forms of worker representation such as trade union workplace level sections, work councils, and shop stewards.
  • Participation: this refers to employee involvement in regulating, implementing, or enforcing OSH standards at company level through indirect forms of representation (broadly speaking, the involvement of employee representatives in decision-making processes). Participation at company level can be mapped along a continuum with ‘no participation’ and ‘co-determination’ at opposite ends. Intermediate levels include participation practices in which, in line with Directive 2002/14/EC, employees receive information, or, in a further step, are consulted.
  • Influence: influence is linked to the relative bargaining power and ability of employee representatives to exert influence over the governance of OSH, that is, the actual impact of employee representatives in its regulation, implementation, and enforcement.
  • Involvement in OSH governance: this refers to the extent of involvement of management and employees in the design and implementation of OSH at company level, including the relevance for management and the means provided by management for the effective involvement of employees and their representatives in the governance of OSH.

The main limitations of this conceptual approach have already been highlighted by EU-OSHA[1]. Firstly, it only considers company level. Although this choice is attributable mainly to data availability (no comparative data at sectoral or national level is available for the EU-27 countries), it is also worth noting that, in the literature, company level is considered the most important with regard to social dialogue and OSH. Secondly, it does not take into account the content of social dialogue and collective bargaining in the field of OSH – also due to a lack of information. Nevertheless, the conceptual approach does include a key aspect of OSH governance: it considers how employees and their representatives are involved in the identification and prevention of OSH risks.

Selecting sources and indicators

The next step consists of selecting the most adequate sources and indicators for measuring the four dimensions. For this purpose, the indicators initially selected by EU-OSHA[1] and other  potential indicators were further assessed on account of the quality criteria presented in Table 1.

These criteria are based on the quality assessment and assurance framework of the European Statistical System (ESS)(the quality assessment and assurance framework of the European Statistical System (ESS)[5],[6] evaluates the quality of already produced statistical outputs, based on principles 11 through 15 of the European Statistics Code of Practice (CoP))[7], and the literature on selecting and processing indicators[8],[9].

Table 1. Quality criteria for assessing indicators

Relevance

Indicators should have a clear conceptual link with industrial relations in the field of OSH at company level.

Accuracy and reliability

Indicators should be accurate and reliably measure the phenomenon they intend to measure and not be confounded by other factors. Indicators should be sensitive to changes, and changes in their values should have a clear and unambiguous meaning.

Intelligibility and easy interpretation

Indicators should be sufficiently simple to be intuitive and unambiguously interpreted in practice. Indicators should have a clear meaning with respect to the phenomenon analysed, either ‘positive’, meaning that higher values are considered positively, or ‘negative’.

Timeliness and punctuality

Indicators should be released in accordance with an agreed schedule and soon after the period to which they refer. The time gap between the collection and reporting of data should be minimal, to ensure that indicators are reporting current rather than historical information.

Sustainability

It indicates the updating frequency of indicators. If an indicator aims to monitor progress, special one-off surveys should not be included.

Coherence and comparability

It shows whether concepts, definitions, methodologies, and actual data are consistent internally and across space and time.

Accessibility and clarity

It indicates if data are available and accompanied with adequate explanatory information (metadata).

Source: Eurofound[8]

The result was a dashboard on the quality of industrial relations in the field of OSH at company level, with a set of indicators for the EU-27 countries . The dashboard is presented in Annex 1 and includes the definition of and the source for each indicator, as well as other details.

The indicators come from the following sources.

  • The European Survey of Enterprises on New and Emerging Risks (ESENER) carried out by EU-OSHA.
  • The European Working Conditions Survey (EWCS) carried out by Eurofound.
  • The European Company Survey (ECS) carried out by Eurofound. It includes the survey of management (ECS-MGT) and the survey of employees’ representatives (ECS-ER).
  • The Organisation for Economic Co-operation and Development (OECD)/Amsterdam Institute for Labour Studies (AIAS) 2021 database on Institutional Characteristics of Trade Unions, Wage Setting, State Intervention, and Social Pacts (OECD-AIAS-ICTWSS, 2021).

It should be noted that the analysis of the three surveys over the course of time revealed comparability problems in almost all potential indicators. For this reason, only indicators from the latest round of surveys were assessed (2019 ESENER, 2021 EWCS and 2019 ECS).

In total, 37 indicators meeting the ‘relevance’ criterion were assessed, out of which 28 were considered normative indicators. Normative indicators are those that meet the ‘intelligibility and easy interpretation’ criterion: that is, they can be clearly understood as positive or negative for the quality of industrial relations. By definition, only normative indicators can be used to build a composite indicator. The rest of the indicators (classified as contextual) are potentially useful for other types of statistical analysis.

The assessment also revealed that the three normative indicators based on the 2019 ECS-ER survey had problems with accuracy and reliability due to the low number of responses, with five countries having supplied fewer than 15 responses (Czechia, Greece, Ireland, Latvia and Malta). Therefore, the data from these five countries were regarded as missing.

The indicators are both cardinal and ordinal (that is, based on a scale). Both types of indicators were treated with statistical methods for quantitative variables. According to the literature, this approach can be used when the objective is to maintain comparability among the indicators, to allow for easy interpretation and to achieve consistent results that are not distorted by the assignment of the ordinal categories[10],[11],[12].

Taking the dashboard as a point of departure, the following sections explain the process of selecting and retaining the normative indicators to calculate the OSH-IR index.

Processing data

All indicators included in the dashboard were processed by following the steps below.

  1. Orientation

All the indicators of a synthetic index must be oriented in the same direction, meaning that higher values indicate either better or worse performance. In our case, this condition was already met by all indicators, with higher values indicating better performance. Therefore, it was not necessary to reverse any indicator.

  1. Detection and treatment of outliers

The presence of outliers may result in polarised and biased overall results. For this purpose, values outside the 1.5 interquartile range were checked. Indicators containing outliers were defined as those having distributions with an absolute skewness value greater than 2 and a simultaneous kurtosis greater than 3.5.The analysis showed that the dashboard contains no outliers.

  1. Imputation of missing data

The construction of a composite indicator requires a complete dataset. The dashboard contains two types of missing data.

  • Indicators from the OECD/AIAS ICTWSS database that are not available for all countries for the same year. In this case, the most recent value was used.
  • Indicators from the 2019 ECS-ER survey that are not available for five countries, due to the aforementioned accuracy problems (low response rate). In this case, the average of the available data within each indicator was used.
  1. Normalisation

Normalisation refers to the process of harmonising indicators with different measurement units and ranges of variation. This is needed to ensure the comparability of the indicators to be included in a synthetic index. Three types of normalisation methods were applied:

  • Standardisation: for every indicator, the value of each country is subtracted from the average across countries and then divided by the standard deviation across countries. The distribution of the new indicators has a mean of 0 and a standard deviation of 1.
  • Min-max normalisation: for every indicator, the value of each country is subtracted from the minimum value registered, then divided by its observed range and finally multiplied by 100. All normalised indicators have the same range [0, 100]
  • Min-max normalisation based on the theoretical ranges: for every indicator, the value of each country is subtracted from the minimum value that the indicator can register theoretically, then divided by its range and finally multiplied by 100. All normalised indicators have the same range [0, 100]

The final normalisation method will be the one that provides the most robust index, as explained below.

Establishing the measurement framework

The measurement framework is defined as the final set of indicators used to compute the index and their structure in dimensions. It is derived from the dashboard and the following multivariate analysis:

  • Correlation analysis: this consists of computing the bivariate correlations of all indicators based on the Pearson coefficient. The aim is to explore the interrelations between indicators and maintain those with significant and positive correlations.
  • Principal component analysis (PCA): this seeks to reduce dimensions and generate new variables called ‘components’. Components are based on the linear combinations of the set of variables and preserve the maximum possible total variance of the original dataset. PCA is used to explore the underlying structure of the data, particularly how different variables change in relation to each other and how they are associated.
  • Cronbach Coefficient Alpha: this is a measure of reliability based on the correlation between indicators. A high coefficient denotes that the indicators measure the underlaying multidimensional phenomenon effectively.

This multivariate analysis permits computation of the index as result of a set of indicators that work well based on their internal statistical relationship and in concordance with the conceptual framework.

As a result of this analysis, seven indicators were selected and structured in two dimensions. The final correlation matrix shows strong and significant correlations, and the Cronbach Alpha Coefficient value is good (0.79). Annex 2 presents details of the multivariate analysis. The conceptual discussion of the measurement framework is addressed in section 3.

Weighting and aggregating indicators

The computation of a composite indicator requires us to weight and aggregate indicators. Weighting reflects the relative importance of each indicator and dimension. Thus, any weighting method reflects a value judgment3. Two weighting methods were considered adequate and were therefore tested.

  • Equal weights of dimensions: the two dimensions have the same weight (0.50). The weight of each indicator is obtained by dividing the weight of the dimension by the number of indicators of the corresponding dimension.
  • Weights based on the PCA: the weight of the indicators is based on the outcomes of the PCA (contribution of each indicator to the total variance explained by the principal components). The weight of each dimension is calculated by adding the weight of all the indicators included in the dimension.

Concerning aggregation, indicators were combined using the arithmetic mean to calculate each dimension. Two options were then tested to combine the dimensions and calculate the index: the geometric mean and the harmonic mean. The choice to test either the geometrical or the harmonic mean for aggregating dimensions is conceptual. The use of geometric or harmonic aggregation means (instead of the arithmetic mean) is intended to reduce the compensability effect between dimensions. A country with a low score for one dimension will need a much higher score on the other to improve the overall score.

The final weighting and aggregation method will be the one that provides the most robust index according to the analysis explained the next section.

Calculating and assessing the index

This final step follows the multi-modelling approach applied in the Gender Equality Index[13] and Eurofound’s composite indicator of industrial relations[8]. This approach can be summarised as follows.

  • Calculating all potential versions of the index based on the combination of the different methods explained above. Therefore, 12 versions of the index were calculated, combining 3 normalisation methods, 2 weighting methods, and 2 aggregation methods.
  • Selecting the most robust index. For this purpose, the median of the 12 versions was calculated for each country. Next, we computed the difference between each version and the median by country. Finally, the version that minimises these differences and lies closest to the median was selected as the most robust one.

Figure 1 presents the methods used to compute the OSH-IR index. Indicators are normalised through the min-max method based on the theoretical ranges; weighted based on the results of the PCA; and aggregated using the arithmetic mean to calculate the dimensions, and then the harmonic mean to obtain the overall index.

Figure 1: OSH-IR index - Methods used for normalisation, weighting and aggregation

Imagen
OSH-IR index - Methods used for normalisation, weighting and aggregation

Source: Authors’ elaboration.

 

Conducting this multi-modelling approach reduces the subjectivity associated with the selection of a single option for normalisation, weighting, or aggregation; and contributes to making the composite indicator robust and transparent.

The quality assessment of the OSH-IR index attests to its robustness. The distribution of the differences between the country rank provided by the selected index and the country ranks of the other 11 versions was calculated. The distribution shows a clear peak around 0, which is a good indication of robustness. More concretely, 45% of the cases do not change their position, 73% shift in rank by one position, and 85% of cases shift by two positions at most (see Annex 4 for further detail).

Composite indicator results

Table 2 presents the measurement framework of the Social Dialogue and OSH Index by dimensions and indicators.

Table 2: Industrial relations in the field of OSH Index – Dimensions and indicators

Imagen
Industrial relations

Source: Authors’ elaboration.

The OSH-IR index is made up of two dimensions:

  • Industrial democracy and OSH employees’ involvement

This dimension is composed of five indicators that measure the quality of industrial democracy and the extent of the involvement of employees in the design and implementation of OSH at company level.

The quality of industrial democracy is measured by three indicators in relation to the scope of employees’ representation and participation rights at company level, the coverage of employee representation in the workplace, and its incidence/influence on the governance of employment relationships through collective bargaining. The extent of OSH employees’ involvement is measured through two indicators that address the design and implementation of risk assessment, as well as of measures to address psychosocial risks.

  •  OSH representation and influence

This dimension is made up of two indicators. The first indicator enables the measuring of employee OSH representation at company level. The second indicator allows the measuring of the influence of employees and their representatives in the field of OSH.

The final dimensions of the index diverge from the initial theoretical framework in terms of four key aspects: representation, participation, influence, and involvement[1]. These also reflect interesting patterns about industrial democracy and OSH governance that should be further explored. On the one hand, attention should be drawn to the lack of correlation between indicators measuring industrial democracy and indicators measuring OSH representation and influence. This could be explained by the specificities of the legislation regulating the selection of OSH workers’ representatives and its lack of embeddedness within industrial relations institutions at company level (i.e., in some countries OSH workers’ representation is not exercised through workers’ representative bodies elected by the employees).

On the other hand, results show a high correlation between industrial democracy indicators and indicators measuring individual workers’ involvement in OSH governance. This could indicate that the direct participation of workers in OSH governance does not necessarily come at the expense of collective forms of workers participation, as has been suggested[14]. Rather, the data indicate that strong industrial democracy at company level can indeed favour or, at least, be compatible with workers’ direct participation in the field of OSH.

Scores and visualisation

The OSH-IR index and its two dimensions may be visualised in a number of ways. The first example, reported in Figure 2, shows the scores of the OSH-IR Index and their two dimensions for the EU-27. Country scores are presented in the left-side table. Each column presents a colour scale (ranging from green = ‘high’ to red = ‘low’) in order to indicate the relative performance in each dimension and the index. The chart on the right shows the overall index scores.

Figure 2: OSH-IR Index. Main results

Imagen
OSH-IR Indez. Main results

Source: Authors’ elaboration.

Another example of visual presentation of the indicator is displayed in figures 3 and 4, which show the radar charts for the two dimensions across countries. Radar charts are useful to visualise normalised scores and gain insight into existing differences. Radar charts can also be used for other purposes, such as comparing two countries or differences between indicators within a country.

Figure 3: Industrial democracy and OSH employees’ involvement (Dimension 1) by country (EU-27)

Imagen
 Industrial democracy and OSH employees involvement by country EU-27

Source: Authors’ elaboration.

 

Figure 4: OSH representation and involvement (Dimension 2) by country (EU-27)

Imagen
OSH representation and involvement by country EU-27

Source: Authors’ elaboration.

 

Figure 5 provides an alternative visualisation of cross-country differences, contrasting the scores of dimension 1 on the x axis and dimension 2 on the y axis.

Figure 5: OSH-IR scores by dimension

Imagen
OSH-IR scores by dimension

Finally, maps complement other visualisation examples by providing an indication of similarities and differences by geographical area. Figure 6 shows a map indicating the overall scores of the OSH-IR index through a colour scale.

Figure 6: OSH-IR scores

Imagen
OSH-IR scores

Concluding remarks

The objective of this study was to build a composite indicator of social dialogue and OSH, based on the conceptual framework developed in EU-OSHA[1]. This technical note provides a detailed description of the methodology used to calculate the OSH-IR index, a conceptual discussion of its structure and, finally, suggestions for visualising the results so as to foster the usefulness of this index for comparative purposes.

As indicated by Nardo et al.[3], the reliability and potential usefulness of a composite indicator depends on two main aspects: 1) having a robust conceptual framework, and 2) having a comprehensive set of high-quality indicators measuring the different dimensions covered by this framework.

As already indicated in EU-OSHA[1], the lack of quality information is one of the main limitations to building this index. The analysis carried out also shows unexpected results with regard to the conceptual framework as well as interesting patterns about industrial democracy and OSH governance, which should be explored further. This indicates that additional research in this field is needed, as well as increased efforts to collect more comprehensive information in a systematic way.

 

 

References

[1]  EU-OSHA – European Agency for Safety and Health at Work (2022). Measuring the quality of social dialogue and collective bargaining in the field of occupational safety and health. Available at: https://osha.europa.eu/en/publications/measuring-quality-social-dialogue-and-collective-bargaining-field-occupational-safety-and-health

[2] OECD (2013), Glossary of statistical terms, web page, accessed 31 October 2018.

[3]  Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffman, A., and Giovannini, E. (2005). Handbook on Constructing Composite Indicators: Methodology and User Guide, OECD Publishing, Paris. Available at: https://www.oecd.org/sdd/42495745.pdf

[4]  Budd, J. W. (2004). Employment with a Human Face: Balancing Efficiency, Equity, and Voice, Cornell University Press, Ithaca, New York, US.

[5] Eurostat (2014). ESS handbook for quality reports, Publications Office of the European Union, Luxembourg. https://doi.org/10.2785/983454

[6] Eurostat (2015). Quality Assurance Framework of the European Statistical System, version 1.2. Available at: https://ec.europa.eu/eurostat/documents/64157/4392716/ESS-QAF-V1-2final.pdf/bbf5970c-1adf-46c8-afc3-58ce177a0646

[7] Eurostat (2011). European statistics code of practice, Publications Office of the European Union, Luxembourg. https://doi.org/10.2785/18474

[8]  Eurofound (2018). Measuring varieties of industrial relations in Europe: A quantitative analysis, Publications Office of the European Union, Luxembourg. Available at: https://www.eurofound.europa.eu/publications/report/2018/measuring-varieties-of-industrial-relations-in-europe-a-quantitative-analysis

[9] Sanz de Miguel, P., Welz, C., Caprile, M. & Rodríguez Contreras, R. (2020). Industrial democracy in Europe: a quantitative approach. Labour & Industry: a journal of the social and economic relations of work 30(2),101-132.

[10]  Kim, J-O. and Mueller, C. W. (1978). Factor Analysis: Statistical Methods and Practical Issues, Sage Publications, Beverly Hills, California.

[11]  Ferrer-i-Carbonell, A. and Frijters, P. (2004). ‘How important is methodology for the estimates of the determinants of happiness?’, The Economic Journal, Vol. 114, No. 497, pp. 641–659. https://doi.org/10.1111/j.1468-0297.2004.00235.x

[12] Blanchflower, D. G. (2008). International Evidence on Well-being, IZA Discussion Paper No. 3354, Institute for the Study of Labour, Bonn. Available at: https://docs.iza.org/dp3354.pdf

[13] EIGE (European Institute for Gender Equality) (2017), Gender Equality Index 2017: Methodological report, Publications Office of the European Union, Luxembourg.

[14] Walters D. & Wadsworth E. (2019) Participation in safety and health in European workplaces: Framing the capture of representation. European Journal of Industrial Relations, 26(1):75-90. Available at: https://journals.sagepub.com/doi/full/10.1177/0959680119835670

Select theme