Models have been used pervasively in all industries for a long time but their governance and the emphasis on robust development and independent validation was lax and not well defined until 2011. This changed with the publication of SR 11-7 by the Fed in 2011 specially for the financial service firms in US; the SR 11-7 publication for the first time laid out regulatory expectations for model development, model validation and model risk governance for all financial institutions. It has been more than a decade since the publication of the guidance and it has had a monumental impact on use of models within financial services in the US. Model risk which was largely thought to just include independent validation prior to SR 11-7 has moved from the fringes of risk management to occupy a predominant part of risk management and now covers the entire model lifecycle.
The next decade will see increasing use of models in non-traditional areas and use of big data and machine learning and model risk will have to evolve and adapt. Along those lines, I have been thinking and was curious about model risk practices and approaches in industries other than banks and in particular thought of the actuarial industry which is also heavily reliant on use of models to a substantial extent. My search led me to a model risk project that was established by the ‘Institute and Faculty of Actuaries Risk Management Research and Thought Leadership Committee’ in October 2013. This project was set-up to focus on foundational areas of model risk management and culminated in a report published in 2015. This 2015 paper on model risk within actuarial industry generated lot of positive feedback and won the Peter Clark prize in 2015 which is given annually for the best paper written by members of the actuarial profession. While a lot of the views on model risk governance in the paper leverages and in some cases builds on the concepts outlined in SR 11-7, there were some unique perspectives on model risk presented in that paper which I have summarized below;
Model Risk created by the Cultural Attitude towards Model Risk: One of the most distinct points made in this paper and unique from SR 11-7 and all other other international regulatory guidance is to highlight the various psychological and cultural attitudes towards model risk amongst model users/owners/developers and the type of (non) model risk it creates. The paper divides up the population of model owners/developers into (1) Confident Model Users (2) Conscientious Modellers (3) Uncertainty Avoiders and (4) Intuitive Decision Makers based on their confidence on the model outputs (Y-axis in figure below) and their perceived legitimacy of the modeling approach (X-axis in figure below). The paper makes the argument that each of these 4 attitudes is a partial but necessary response to inherent model uncertainties and recommends that model risk management must require feedback and provide responses to each of these attitudes. Some practical suggestions to address these 4 distinct cultural attitudes within a model risk framework recommended in the paper are;
- Embedding call kinds of challenge within decision making; irrespective of whether decisions are based on model output or on intuition
- Identifying individuals having each of the four attitudes noted above and ensuring that, to the extent possible, over-representation of one group to the exclusion of others is avoided in model governance
- Careful composition of governance committees: in such committees, a range of departments and rationalities should be represented, reflecting risk-specialist, technical, operational, and commercial perspectives
As practitioners, we have to acknowledge that different attitudes to model risk exist and they actually foster effective challenge and the paper recommends ways to incorporate these attitudes within model risk governance.