Get started

Welcome to ValidMind!

ValidMind is a platform designed to streamline the management of risk for AI models, including those used in machine learning (ML), natural language processing (NLP), and large language models (LLMs). The platform offers tools that cater to both model developers and validators, simplifying key aspects of model risk management.

What do I use the ValidMind platform for?

Model developers and validators play important roles in managing model risk, including risk that stems from generative AI and machine learning models. From complying with regulations to ensuring that institutional standards are followed, your team members are tasked with the careful documentation, testing, and independent validation of models.

The purpose of these efforts is to ensure that good risk management principles are followed throughout the model lifecycle. To assist you with these processes of documenting and validating models, ValidMind provides a number of tools that you can employ regardless of the technology used to build your models.

The ValidMind AI risk platform provides two main products components:

1. The developer framework is a library of tools and methods designed to automate generating model documentation and running validation tests. The framework is designed to be platform agnostic and integrates with your existing development environment.

   For Python developers, a single installation command provides access to all the functions:

   pip install validmind

2. The ValidMind Platform UI is an easy-to-use web-based UI that enables you to track the model lifecycle:

• Customize workflows to manage the model documentation and validation process.
• Review and edit the documentation and test metrics generated by the developer framework.
• Collaborate with and capture feedback from model developers and model validators.
• Generate validation reports and approvals.

For more information about the benefits that ValidMind can offer, check out the ValidMind overview.

Key ValidMind concepts

Model documentation

A structured and detailed record pertaining to a model, encompassing key components such as its underlying assumptions, methodologies, data sources, inputs, performance metrics, evaluations, limitations, and intended uses. Within the realm of model risk management, this documentation serves to ensure transparency, adherence to regulatory requirements, and a clear understanding of potential risks associated with the model’s application.

Validation report

A formal document prepared as part of the model validation process, outlining the findings, assessments, and recommendations related to a specific model’s performance, appropriateness, and limitations. Provides a comprehensive review of the model’s conceptual framework, data sources and integrity, calibration methods, and performance outcomes. Within model risk management, the validation report is crucial for ensuring transparency, demonstrating regulatory compliance, and offering actionable insights for model refinement or adjustments.

Documentation Template

Functions as a test suite and lays out the structure of model documentation, segmented into various sections and sub-sections. ValidMind templates come with pre-defined sections, similar to test placeholders, including boilerplates and spaces designated for documentation and test results. You can edit these templates to suit your needs.

Model documentation is populated by the ValidMind Developer Framework when the tests contained in a template run successfully, completing the test suite. This process ensures that the model meets all the specified requirements. At the same time, template placeholders get populated with content that documents the model.

Tests

A function contained in the ValidMind Developer Framework, which is designed to run a specific quantitative test on the dataset or model. Test results are sent to the ValidMind Platform to generate the model documentation according to the relevant templates.

Test suites

A collection of tests which are meant to run together to automate generate model documentation end-to-end for specific use-cases.

Example: the classifier_full_suite test suite runs the tests from the tabular_dataset and classifier test suites to fully document the data and model sections for binary classification model use-cases.

How do I get started?

On the ValidMind platform, everything starts with the model inventory: you first register a new model and then manage the model lifecycle through the different activities that are part of your existing model risk management processes.

Approval workflow

A typical high-level model approval workflow looks like this:

graph LR
    A[Model<br>registration] --> B[Initial<br>validation]
    B --> C[Validation<br>approval]
    C --> D[In production]
    D --> E[Periodic review<br>and revalidation]
    E --> B

New model registration

Select a documentation template when registering a new inventory model to start your model documentation. You then use the model inventory to manage the metadata associated with the model, including all compliance and regulatory attributes.

Initial validation

Triggers a new documentation workflow to yield a model that will be ready for production deployment after its documentation and validation reports have been approved.

Validation approval

Perform validation of the model to ensure that it meets the needs for which it was designed. You can also connect to third-party systems to send events when a model has been approved for production.

In production

Use the model in production while ensuring its ongoing reliability, accuracy, and compliance with regulations by monitoring the model’s performance.

Periodic review and revalidation

As part of regular performance monitoring or change management, you follow a process similar to that seen in the Initial validation step.

Next steps

The fastest way to explore what ValidMind can offer is with our QuickStart:

• Try out the ValidMind Developer Framework
• Explore the ValidMind Platform UI

If you have already tried the QuickStart, more how-to instructions and links to our FAQs can be found under Next steps.

Need an account for ValidMind?

Signing up is FREE — Create your account.