I recently came across a situation that left me wondering, are model governance requests from banks and other industries really this demanding? I work for a company that provides inference as a service, and we’ve noticed that banks, in particular, ask for an overwhelming amount of information through model governance.
The requests are so detailed that it ends up being a huge burn on time and resources, with a ridiculous back-and-forth process. I’m talking about questions like:
* What specific features are used in the model?
* What specific data sources do we use?
* Can we provide a detailed explanation of how we arrived at our modeling methodology, including what other models we considered, the results of those other models, and the rationale for our decision with a comparative analysis?
* Can we provide a list of all metrics used to evaluate model performance, and why we chose those metrics?
* What’s the time frame for our train/test/val sets, down to the day?
I’m curious, is this the new normal? Are banks and other industries just being overly cautious, or is my organization not doing enough to address these concerns?
I’d love to hear from others who have experience with model governance requests from banks or other industries. Are there any resources out there that show what’s considered industry standard? How do your organizations handle these requests?
Sharing your experiences or insights would be super helpful in understanding what’s normal and what we can do to improve our reporting to customers who are very concerned about these kinds of things.