Guides

Published

March 6, 2024

Our guides offer step-by-step instructions for frequent tasks you perform on the ValidMind platform, organized by category:

Onboarding
Model inventory
Model documentation
Model validation

Key ValidMind concepts

Model documentation: A structured and detailed record pertaining to a model, encompassing key components such as its underlying assumptions, methodologies, data sources, inputs, performance metrics, evaluations, limitations, and intended uses. Within the realm of model risk management, this documentation serves to ensure transparency, adherence to regulatory requirements, and a clear understanding of potential risks associated with the model’s application.
Validation report: A formal document prepared as part of the model validation process, outlining the findings, assessments, and recommendations related to a specific model’s performance, appropriateness, and limitations. Provides a comprehensive review of the model’s conceptual framework, data sources and integrity, calibration methods, and performance outcomes. Within model risk management, the validation report is crucial for ensuring transparency, demonstrating regulatory compliance, and offering actionable insights for model refinement or adjustments.
Documentation Template: Functions as a test suite and lays out the structure of model documentation, segmented into various sections and sub-sections. ValidMind templates come with pre-defined sections, similar to test placeholders, including boilerplates and spaces designated for documentation and test results. You can edit these templates to suit your needs.

Model documentation is populated by the ValidMind Developer Framework when the tests contained in a template run successfully, completing the test suite. This process ensures that the model meets all the specified requirements. At the same time, template placeholders get populated with content that documents the model.

Tests: A function contained in the ValidMind Developer Framework, which is designed to run a specific quantitative test on the dataset or model. Test results are sent to the ValidMind Platform to generate the model documentation according to the relevant templates.
Test suites: A collection of tests which are meant to run together to automate generate model documentation end-to-end for specific use-cases.

Example: the classifier_full_suite test suite runs the tests from the tabular_dataset and classifier test suites to fully document the data and model sections for binary classification model use-cases.

Onboarding

To onboard your organization, teams or business units, and users onto the ValidMind platform:

Onboarding users

Onboarding users involves controlling and organizing who has access to the ValidMind platform. Tasks include inviting your initial users, creating groups, and assigning roles and permissions.

Configure AWS PrivateLink

To keep your network traffic private and minimizes its attack surface, configure AWS PrivateLink to establish a private connection between ValidMind and your company network.

Model inventory

To configure the model inventory to your organization’s requirements and to register new models:

Edit model inventory fields

Edit individual model detailed fields in the model inventory to customize them to your needs or to ensure that model details are accurate and up to date.

Register models in the inventory

To be able to document models using ValidMind’s model documentation and validation features, you need to register them in the model inventory first.

Model documentation

To document and test your models in your own model development environment:

Get started with the ValidMind Developer Framework

As a model developer, you use the ValidMind Developer Framework to automate the generation of model documentation by uploading documentation artifacts and test results to the ValidMind platform.

Code samples

Our code samples, based on Jupyter notebooks, showcase the capabilities and features of the ValidMind Developer Framework, while also providing you with useful examples that you can build on and adapt for your own use cases.

To work with documentation, documentation templates, and model documentation in the Platform UI, and to collaborate with model validators:

Model validation

To set up approvals, prepare validation reports, collaborate with model developers, link evidence to your reports, or work with model documentation findings:

Preparing validation reports

As part of preparing validation reports, you can view the guidelines, collaborate with model developers, add or edit content to your draft validation report, link to evidence, or work with project findings.

Work with model findings

Model findings are detailed observations identified during the model validation process, highlighting any major or minor issues, deficiencies, model limitations, stability and robustness concerns, or needed adjustments.

View reports

Reports provide quick insights into your model validation efforts, detailing critical findings, risk exposure, and compliance status to ensure effective oversight and management of model-related risks.

Export documentation

If you need access to your model model documentation or validation reports outside of the ValidMind platform, you can export your model documentation and validation report as Word or PDF files.

Onboarding

Onboarding users

Configure AWS PrivateLink

Model inventory

Edit model inventory fields

Register models in the inventory

Model documentation

Get started with the ValidMind Developer Framework

Code samples

Working with documentation templates

Working with model documentation

Export documentation

Model validation

Preparing validation reports

Work with model findings

View reports

Export documentation