Safety series: performance-based regulations have changed oversight responsibilities

It has almost become a cliché to highlight the aviation industry’s extraordinary safety record over recent years, certainly compared to a generation ago. Although accidents involving commercial airliners of course do still unfortunately happen, the overall safety picture, particularly within Europe, has shown one of constant incremental improvement. The question that we need to ask though is how can this record be sustained?

Conventionally, safety regulation has been underpinned by a prescriptive system of rules, processes and standards imposed on airports, airlines and other organisations that provide aviation products and services. As this prescriptive system has matured, however, its potential to deliver further safety improvements has become limited. With the number of commercial flights forecast to rise significantly in the next decade, the demands on safety oversight systems will only increase. The sheer pace of change across the aviation industry, largely resulting from advances in technology, present further challenges for the conventional approach to safety regulation. In a nutshell, oversight must evolve to remain relevant and effective in delivering safety outcomes.

The UK Government’s Better Regulation Principles also require regulators to reduce the burden associated with existing rules and to regulate only when necessary in a way that is targeted, proportionate, and informed by a comprehensive assessment of risk. As with many regulated sectors, the evolution of standards and rules often lags behind developments in technology, business models and consumer trends, making them less and less fit for purpose.

All of which brings us to the concept of performance-based regulation (PBR). The UK Civil Aviation Authority embraced the PBR model in 2014 and it has radically changed the way we carry out our oversight responsibilities and collect, analyse and use safety information.

The PBR approach strengthens and standardise many of our existing regulatory processes and functions. But it doesn’t just benefit us. PBR leads to much closer cooperation between us, the regulator, and the accountable manager of regulated organisations, ensuring both parties recognise the same risks.

All about data

If we look behind aviation’s positive safety statistics, we uncover incidents and occurrences that could be described as ‘near misses’, or situations that had the potential to develop into full blown accidents. Reacting only in response to these near misses is clearly not the best way to prevent them happening again. A more sensible approach is to closely examine the causal factors of incidents to understand what went wrong and how to prevent any re-occurrence.  

Consistently gathering and analysing operational safety data provides the foundation for developing an organisation’s overall risk picture. Data can be gathered from a range of sources – audit findings, mandatory occurrence reports (MORs), and even the expert knowledge of staff. Once analysed by a dedicated team of technical specialists to identify specific safety risks the data is then assessed and prioritised using a similar methodology to that used in many industry safety management systems. As always with statistical analysis the quality of what comes out is only as good as what goes in. Data is a key enabler of PBR and operators are strongly encouraged to have sound data management policies in place.  

Strong performers

Assessing the performance of an organisation, and how it manages its own safety risk is at the core of a the PBR regime. CAA operations inspector and surveyors will work closely with each airport or airline accountable manager to agree the actions needed to uphold standards and further enhance safety. Of course, this one-to-one engagement formed a major part of our traditional safety oversight, but the difference now is PBR joins-up a range of processes under the control of a single management system.

We have developed bespoke software to allow our inspectors and surveyors the ability to modify an organisation’s risk picture, and we share the updates immediately. The software, creates standard reports and management information and provides a means for inspectors/surveyors to capture their confidence levels about specific parts of an entity’s operations, measuring trends over time. Internal review meetings then discuss each operator’s key risks, safety performance and oversight priorities. These meetings provide opportunities for inspectors/surveyors specialising in different areas to combine intelligence and link their findings.

Identifying an organisation’s safety risks through analysing relevant data and then assessing its overall performance we call Performance Based Oversight. This allows us to focus direct oversight on those areas where it is needed and utilise our time in a much more efficient way. For example, between January 2016 and May 2018 the CAA re-prioritised some excess oversight time away from seven aerodromes and 10 air navigation service providers (ANSPs) to three specific aerodromes and seven other ANSPs where increased oversight was deemed to be beneficial. So, operations which may be managing a substantial change, or experiencing an increase in risk will have oversight proportionately assigned to ensure safety standards and regulations are met.

The bigger picture

Feeding all this oversight activity into a single system allows us to really understand risks to specific sectors. By grouping safety risk information derived from individual operators into relevant sectors – such as large airports – we not only identify their top-level risks, but we also learn how to employ best practice approaches to managing those risks.

Enhancing our knowledge base of aviation safety clearly presents an opportunity to tackle risk factors proactively – before an incident or series of incidents occur. The processes for subsequently monitoring the impact of our safety decisions is also strengthened by a clear line of sight to our safety priorities and the creation of an expanded suite of safety performance indicators.

Making more informed decisions with industry to manage the top safety risks in each sector also makes complying with international requirements much easier. EASA, the FAA and ICAO are all fully aligned with the principles of PBR. 

The development of the PBR Safety Risk Framework has enabled the CAA to start analysing sector performance which has improved our ability to provide targeted and proportionate levels of oversight while redirecting valuable resource to the areas where it is required the most.

Entity average confidence level against the sector average (aerodrome certificate) 

PBR assigning future oversight against performance

Direct action

So, PBR allows us to direct our resources proportionately, focusing our oversight activity and safety improvement projects on the organisations and sectors where standards are not being upheld – or where there is the significant potential to enhance safety.

We knew from the beginning that PBR could only be implemented with full industry co-operation. This new way of working was clearly not something that the CAA could impose on organisations who had not fully bought into the concept. A CAA-industry working group was therefore established to identify and tackle any issues that might stand in the way of implementation. During its first 12 months the group focused on three main areas:

1. Tracking the benefits and costs of adopting a PBR approach.
2. Identifying opportunities to improve how safety data is gathered and used.
3. Considering how people’s skills and safety culture should evolve to support PBR.

Their work proved pivotal in shaping the CAA’s approach to PBR. The use of case studies for example, illustrated how safety data could be better applied to accrue benefits in a PBR environment. The size and nature of the capability gap across industry and the CAA to embed the PBR approach was also a major focus of attention.

Counting the benefits

Ultimately, PBR encourages a shared approach to identifying where safety improvements are needed. It offers the potential to nudge industry collectively towards making improvements, incentivised by the prospect of a slightly less onerous inspection regime for higher performers. It also plays an important role in ensuring that the regulator’s processes and decision making are focused, proportionate and consistent. Inconsistencies make it difficult for the well intentioned to act well and it allows the ill-intentioned to capitalise on loopholes to circumvent unfavourable aspects of the regulatory regime.

Recent engagement between the CAA PBR team and accountable managers throughout the industry identified that the conversations between the two sides had vastly improved. Risk-rich conversations are now more commonplace between the CAA and industry, with positive risk management and opportunities to improve performance and safety standards; highlighted as key benefits. 

To regulate effectively the CAA must have real expert knowledge of the sector and understand how organisations operate. We must strike a delicate balance between being an understanding partner of the industry and an independent, objective regulator. It is especially important that PBR is not seen to take any safety responsibilities away from the industry. The accountability for managing risks and delivering safety is owned by the industry. The regulator’s role is to understand the risks (particularly at the sector-wide and total system levels) and ensure that they are being managed effectively. The use of a tick box audit method to achieve this is no longer an option.

Trust is vital to the success of the PBR approach: both trust from consumers that their safety is properly protected; and trust from the industry that the regulator is capable and understands enough about the sector’s risks to carry out its role

Aviation’s safety record can indeed be sustained, and even improved, if the industry and its regulators fully embrace a new way of working together. We believe PBR fits the bill.