Rising unpredictability requires greater awareness of potential model deterioration

In addition, fintechs will often place greater confidence in models to perform key tasks, such as approving lending decisions, where the result on operational effectiveness alone is clear with some fintechs having automated decisions rates in excess of 90%, even for complex products such as secured lending.

This level of automation results in an increased reliance being placed on the models with regards to the profitability of the subsequent portfolio and fulfilling expectations around compliance and treating customers fairly. Furthermore, once on the balance sheet, firms are required to continually assess the risk and quality of their loans for use in accounting and capital calculations. This in turn leads to the creation of significant model risk within these organisations.

Model risk relates to the risk incurred from a reliance on models in the operation of the organisation. This risk can exist due to a number of sources including, but not limited to, the model not performing as expected, not being used in the correct manner, not being developed correctly, not being appropriately maintained, inaccuracies in its implementation, or corruption of the data feeding the model. Should one or more of these issues occur, the realisation of the risks can manifest themselves financially, reputationally or through compliance issues.

The mitigation of these risks is through strong internal governance, ongoing monitoring and good understanding and direction from the top of the organisation, i.e. the board. Within the UK the Prudential Regulation Authority is beginning to provide more direction with the publication of consultation Paper 6/22 – Model Risk Management Principles for Banks. This consultation paper outlines the expectations regarding model risk frameworks, including the role of senior personnel.

The burden on senior firm members to hold the required technical knowledge is high, and is critical when models are possibly the most significant tool in the running of the organisation. There is a danger that non-technical specialists can rely on models being objective and infallible and therefore outside their area of consideration and direction. However, there is much subjectivity and limitations within models that, if not understood at the top level, can lead to what in effect becomes a high level of delegation and the subsequent creation of undetected model risk.

A likely manifestation of model risk is financial risk due to poor model-driven decisions as a result of models not being reflective of the true likely outcomes. This can occur for a number of reasons but is most commonly due to deterioration in the model performance from point of development.

Robust monitoring 

Model deterioration is common and is often mitigated through robust model monitoring and ongoing maintenance. However, model monitoring only gives an indication of how well the model has predicted reality in the recent past. In times of large change and uncertainty, this is not sufficient to protect against poor decision-making. In recent times the impact of Covid on model performance has been monumental, with both characteristics and outcomes being highly distorted. The current cost of living crisis introduces similar considerations, where it is known that certain pockets of the population will suffer greater than others and will perform worse in terms of their ability to service debt. Some models may not accurately reflect this, reducing their effectiveness.

Therefore, there is a need within the model risk framework to include an element of qualitative assessment of more forward-looking measures. This obviously takes time, effort and resources, and may not be fully aligned to an aspirational lean organisational approach, reliant upon data and models, but it is a necessity to guard against large negative impacts on business performance.

This oversight and human intervention comes back to the need to have strong senior involvement with a good risk culture up to board level (i.e., a culture that is free of the hubris that can occur in some organisations), and also a strong second line risk function that holds to account the side of the business mandated to drive growth.

Ultimately, a well evolved risk function integrated into the running of the organisation should be seen as an enabler that allows confident expansion into new areas. For organisations with a heavy reliance on models, this confidence comes from the function focusing on a strong model risk framework ensuring that existing model risk issues are identified through quantitative analysis and technical validation, upcoming risk issues are anticipated through human intervention and oversight, and that all of these are communicated to, and understood by, senior management, including the board.