Exploring Tests in the Developer Framework:

A Comprehensive Guide to Finding and Viewing Tests

Welcome to this comprehensive guide to the ValidMind Developer Framework tests module! In this notebook, we’ll dive deep into the utilities available for viewing and understanding the various tests that ValidMind provides. Whether you’re just getting started or looking for advanced tips, you’ll find clear examples and explanations to assist you every step of the way.

Before we delve into the details, let’s import the describe_test and list_tests functions from the validmind.tests module. These are the two functions that can be used to easily filter through tests and view details for individual tests.

from validmind.tests import describe_test, list_tests, list_task_types, list_tags, list_tasks_and_tags
/Users/anilsorathiya/Library/Caches/pypoetry/virtualenvs/validmind-1QuffXMV-py3.9/lib/python3.9/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
  warnings.warn(

Listing All Tests

The list_tests function provides a convenient way to retrieve all available tests in the validmind.tests module. When invoked without any parameters, it returns a pandas DataFrame containing detailed information about each test.

list_tests()
Test Type Name Description ID
ThresholdTest Bias Evaluates bias in a Large Language Model based on the order and distribution of exemplars in a prompt.... validmind.prompt_validation.Bias
ThresholdTest Clarity Evaluates and scores the clarity of prompts in a Large Language Model based on specified guidelines.... validmind.prompt_validation.Clarity
ThresholdTest Specificity Evaluates and scores the specificity of prompts provided to a Large Language Model (LLM), based on clarity,... validmind.prompt_validation.Specificity
ThresholdTest Robustness Assesses the robustness of prompts provided to a Large Language Model under varying conditions and contexts.... validmind.prompt_validation.Robustness
ThresholdTest Negative Instruction Evaluates and grades the use of affirmative, proactive language over negative instructions in LLM prompts.... validmind.prompt_validation.NegativeInstruction
ThresholdTest Conciseness Analyzes and grades the conciseness of prompts provided to a Large Language Model.... validmind.prompt_validation.Conciseness
ThresholdTest Delimitation Evaluates the proper use of delimiters in prompts provided to Large Language Models.... validmind.prompt_validation.Delimitation
Metric Bert Score Evaluates text generation models' performance by calculating precision, recall, and F1 score based on BERT... validmind.model_validation.BertScore
Metric Regard Score **Purpose:**... validmind.model_validation.RegardScore
Metric Bleu Score Assesses translation quality by comparing machine-translated sentences with human-translated ones using BLEU score.... validmind.model_validation.BleuScore
Metric Contextual Recall Evaluates a Natural Language Generation model's ability to generate contextually relevant and factually correct... validmind.model_validation.ContextualRecall
Metric Regard Histogram **Purpose:**... validmind.model_validation.RegardHistogram
Metric Toxicity Histogram **Purpose:**... validmind.model_validation.ToxicityHistogram
Metric Rouge Metrics Evaluates the quality of machine-generated text using various ROUGE metrics, and visualizes the results.... validmind.model_validation.RougeMetrics
Metric Model Metadata Extracts and summarizes critical metadata from a machine learning model instance for comprehensive analysis.... validmind.model_validation.ModelMetadata
Metric Bert Score Aggregate Evaluates the aggregate performance of text generation models by computing the average precision, recall,... validmind.model_validation.BertScoreAggregate
Metric Cluster Size Distribution Compares and visualizes the distribution of cluster sizes in model predictions and actual data for assessing... validmind.model_validation.ClusterSizeDistribution
Metric Token Disparity Assess and visualize token count disparity between model's predicted and actual dataset.... validmind.model_validation.TokenDisparity
Metric Toxicity Score **Purpose:**... validmind.model_validation.ToxicityScore
Metric Rouge Metrics Aggregate Evaluates the average quality of machine-generated text using various ROUGE metrics and visualizes the aggregated results.... validmind.model_validation.RougeMetricsAggregate
Metric Embeddings Visualization D Visualizes 2D representation of text embeddings generated by a model using t-SNE technique.... validmind.model_validation.embeddings.EmbeddingsVisualization2D
ThresholdTest Stability Analysis Random Noise Evaluate robustness of embeddings models to random noise introduced by using... validmind.model_validation.embeddings.StabilityAnalysisRandomNoise
Metric Cosine Similarity Distribution Assesses the similarity between predicted text embeddings from a model using a Cosine Similarity distribution... validmind.model_validation.embeddings.CosineSimilarityDistribution
ThresholdTest Stability Analysis Translation Evaluate robustness of embeddings models to noise introduced by translating... validmind.model_validation.embeddings.StabilityAnalysisTranslation
Metric Cluster Distribution Assesses the distribution of text embeddings across clusters produced by a model using KMeans clustering.... validmind.model_validation.embeddings.ClusterDistribution
ThresholdTest Stability Analysis Base class for embeddings stability analysis tests validmind.model_validation.embeddings.StabilityAnalysis
ThresholdTest Stability Analysis Keyword Evaluate robustness of embeddings models to keyword swaps on the test dataset... validmind.model_validation.embeddings.StabilityAnalysisKeyword
ThresholdTest Stability Analysis Synonyms Evaluates the stability of text embeddings models when words in test data are replaced by their synonyms randomly.... validmind.model_validation.embeddings.StabilityAnalysisSynonyms
Metric Descriptive Analytics Evaluates statistical properties of text embeddings in an ML model via mean, median, and standard deviation... validmind.model_validation.embeddings.DescriptiveAnalytics
Metric Regression Models Performance Comparison Compares and evaluates the performance of multiple regression models using five different metrics: MAE, MSE, RMSE,... validmind.model_validation.sklearn.RegressionModelsPerformanceComparison
Metric Adjusted Mutual Information Evaluates clustering model performance by measuring mutual information between true and predicted labels, adjusting... validmind.model_validation.sklearn.AdjustedMutualInformation
Metric Silhouette Plot Calculates and visualizes Silhouette Score, assessing degree of data point suitability to its cluster in ML models.... validmind.model_validation.sklearn.SilhouettePlot
ThresholdTest Robustness Diagnosis Evaluates the robustness of a machine learning model by injecting Gaussian noise to input data and measuring... validmind.model_validation.sklearn.RobustnessDiagnosis
Metric Adjusted Rand Index Measures the similarity between two data clusters using the Adjusted Rand Index (ARI) metric in clustering machine... validmind.model_validation.sklearn.AdjustedRandIndex
Metric SHAP Global Importance Evaluates and visualizes global feature importance using SHAP values for model explanation and risk identification.... validmind.model_validation.sklearn.SHAPGlobalImportance
Metric Confusion Matrix Evaluates and visually represents the classification ML model's predictive performance using a Confusion Matrix... validmind.model_validation.sklearn.ConfusionMatrix
Metric Homogeneity Score Assesses clustering homogeneity by comparing true and predicted labels, scoring from 0 (heterogeneous) to 1... validmind.model_validation.sklearn.HomogeneityScore
Metric Completeness Score Evaluates a clustering model's capacity to categorize instances from a single class into the same cluster.... validmind.model_validation.sklearn.CompletenessScore
ThresholdTest Overfit Diagnosis Detects and visualizes overfit regions in an ML model by comparing performance on training and test datasets.... validmind.model_validation.sklearn.OverfitDiagnosis
Metric Cluster Performance Metrics Evaluates the performance of clustering machine learning models using multiple established metrics.... validmind.model_validation.sklearn.ClusterPerformanceMetrics
Metric Permutation Feature Importance Assesses the significance of each feature in a model by evaluating the impact on model performance when feature... validmind.model_validation.sklearn.PermutationFeatureImportance
Metric Fowlkes Mallows Score Evaluates the similarity between predicted and actual cluster assignments in a model using the Fowlkes-Mallows... validmind.model_validation.sklearn.FowlkesMallowsScore
ThresholdTest Minimum ROCAUC Score Validates model by checking if the ROC AUC score meets or surpasses a specified threshold.... validmind.model_validation.sklearn.MinimumROCAUCScore
Metric Cluster Cosine Similarity Measures the intra-cluster similarity of a clustering model using cosine similarity.... validmind.model_validation.sklearn.ClusterCosineSimilarity
Metric Precision Recall Curve Evaluates the precision-recall trade-off for binary classification models and visualizes the Precision-Recall curve.... validmind.model_validation.sklearn.PrecisionRecallCurve
Metric Classifier Performance Evaluates performance of binary or multiclass classification models using precision, recall, F1-Score, accuracy,... validmind.model_validation.sklearn.ClassifierPerformance
Metric V Measure Evaluates homogeneity and completeness of a clustering model using the V Measure Score.... validmind.model_validation.sklearn.VMeasure
ThresholdTest Minimum Score Evaluates if the model's F1 score on the validation set meets a predefined minimum threshold.... validmind.model_validation.sklearn.MinimumF1Score
Metric ROC Curve Evaluates binary classification model performance by generating and plotting the Receiver Operating Characteristic... validmind.model_validation.sklearn.ROCCurve
Metric Regression Square **Purpose**: The purpose of the RegressionR2Square Metric test is to measure the overall goodness-of-fit of a... validmind.model_validation.sklearn.RegressionR2Square
Metric Regression Errors **Purpose**: This metric is used to measure the performance of a regression model. It gauges the model's accuracy... validmind.model_validation.sklearn.RegressionErrors
Metric Cluster Performance Evaluates and compares a clustering model's performance on training and testing datasets using multiple defined... validmind.model_validation.sklearn.ClusterPerformance
ThresholdTest Training Test Degradation Tests if model performance degradation between training and test datasets exceeds a predefined threshold.... validmind.model_validation.sklearn.TrainingTestDegradation
Metric Hyper Parameters Tuning Exerts exhaustive grid search to identify optimal hyperparameters for the model, improving performance.... validmind.model_validation.sklearn.HyperParametersTuning
Metric K Means Clusters Optimization Optimizes the number of clusters in K-means models using Elbow and Silhouette methods.... validmind.model_validation.sklearn.KMeansClustersOptimization
Metric Models Performance Comparison Evaluates and compares the performance of multiple Machine Learning models using various metrics like accuracy,... validmind.model_validation.sklearn.ModelsPerformanceComparison
ThresholdTest Weakspots Diagnosis Identifies and visualizes weak spots in a machine learning model's performance across various sections of the... validmind.model_validation.sklearn.WeakspotsDiagnosis
Metric Population Stability Index Evaluates the Population Stability Index (PSI) to quantify the stability of an ML model's predictions across... validmind.model_validation.sklearn.PopulationStabilityIndex
ThresholdTest Minimum Accuracy Checks if the model's prediction accuracy meets or surpasses a specified threshold.... validmind.model_validation.sklearn.MinimumAccuracy
Metric Regression Models Coeffs Compares feature importance by evaluating and contrasting coefficients of different regression models.... validmind.model_validation.statsmodels.RegressionModelsCoeffs
Metric Box Pierce Detects autocorrelation in time-series data through the Box-Pierce test to validate model performance.... validmind.model_validation.statsmodels.BoxPierce
Metric Regression Coeffs Plot Visualizes regression coefficients with 95% confidence intervals to assess predictor variables' impact on response... validmind.model_validation.statsmodels.RegressionCoeffsPlot
Metric Regression Model Sensitivity Plot Tests the sensitivity of a regression model to variations in independent variables by applying shocks and... validmind.model_validation.statsmodels.RegressionModelSensitivityPlot
Metric Regression Models Performance Evaluates and compares regression models' performance using R-squared, Adjusted R-squared, and MSE metrics.... validmind.model_validation.statsmodels.RegressionModelsPerformance
Metric Zivot Andrews Arch Evaluates the order of integration and stationarity of time series data using Zivot-Andrews unit root test.... validmind.model_validation.statsmodels.ZivotAndrewsArch
Metric Regression Model Outsample Comparison Computes MSE and RMSE for multiple regression models using out-of-sample test to assess model's prediction accuracy... validmind.model_validation.statsmodels.RegressionModelOutsampleComparison
Metric Regression Model Forecast Plot Levels Compares and visualizes forecasted and actual values of regression models on both raw and transformed datasets.... validmind.model_validation.statsmodels.RegressionModelForecastPlotLevels
Metric Log Regression Confusion Matrix Generates a confusion matrix for logistic regression model performance, utilizing thresholded probabilities for... validmind.model_validation.statsmodels.LogRegressionConfusionMatrix
Metric PD Rating Class Plot Assesses and visualizes credit risk distribution across different rating classes within a dataset via default... validmind.model_validation.statsmodels.PDRatingClassPlot
Metric Scorecard Histogram Creates histograms of credit scores, from both default and non-default instances, generated by a credit-risk model.... validmind.model_validation.statsmodels.ScorecardHistogram
Metric Feature Importance And Significance Evaluates and visualizes the statistical significance and feature importance using regression and decision tree... validmind.model_validation.statsmodels.FeatureImportanceAndSignificance
Metric L Jung Box Assesses autocorrelations in dataset features by performing a Ljung-Box test on each feature.... validmind.model_validation.statsmodels.LJungBox
Metric Logistic Reg Prediction Histogram Generates and visualizes histograms of the Probability of Default predictions for both positive and negative... validmind.model_validation.statsmodels.LogisticRegPredictionHistogram
Metric Jarque Bera Assesses normality of dataset features in an ML model using the Jarque-Bera test.... validmind.model_validation.statsmodels.JarqueBera
Metric Phillips Perron Arch Executes Phillips-Perron test to assess the stationarity of time series data in each ML model feature.... validmind.model_validation.statsmodels.PhillipsPerronArch
Metric Kolmogorov Smirnov Executes a feature-wise Kolmogorov-Smirnov test to evaluate alignment with normal distribution in datasets.... validmind.model_validation.statsmodels.KolmogorovSmirnov
Metric Residuals Visual Inspection Provides a comprehensive visual analysis of residuals for regression models utilizing various plot types.... validmind.model_validation.statsmodels.ResidualsVisualInspection
Metric Shapiro Wilk Evaluates feature-wise normality of training data using the Shapiro-Wilk test.... validmind.model_validation.statsmodels.ShapiroWilk
Metric Scorecard Bucket Histogram Evaluates and visualizes distribution of risk categories in a classification model's scores, useful in credit risk... validmind.model_validation.statsmodels.ScorecardBucketHistogram
Metric Regression Model In Sample Comparison Evaluates and compares in-sample performance of multiple regression models using R-Squared, Adjusted R-Squared,... validmind.model_validation.statsmodels.RegressionModelInSampleComparison
Metric Regression Feature Significance Assesses and visualizes the statistical significance of features in a set of regression models.... validmind.model_validation.statsmodels.RegressionFeatureSignificance
Metric Regression Model Summary Evaluates regression model performance using metrics including R-Squared, Adjusted R-Squared, MSE, and RMSE.... validmind.model_validation.statsmodels.RegressionModelSummary
Metric KPSS Executes KPSS unit root test to validate stationarity of time-series data in machine learning model.... validmind.model_validation.statsmodels.KPSS
Metric Lilliefors Assesses the normality of feature distributions in an ML model's training dataset using the Lilliefors test.... validmind.model_validation.statsmodels.Lilliefors
Metric Logistic Reg Cumulative Prob Visualizes cumulative probabilities of positive and negative classes for both training and testing in logistic... validmind.model_validation.statsmodels.LogisticRegCumulativeProb
Metric Runs Test Executes Runs Test on ML model to detect non-random patterns in output data sequence.... validmind.model_validation.statsmodels.RunsTest
Metric Scorecard Probabilities Histogram Evaluates risk classification of a model by visualizing the distribution of default probability across score... validmind.model_validation.statsmodels.ScorecardProbabilitiesHistogram
Metric DFGLS Arch Executes Dickey-Fuller GLS metric to determine order of integration and check stationarity in time series data.... validmind.model_validation.statsmodels.DFGLSArch
Metric Auto ARIMA Evaluates ARIMA models for time-series forecasting, ranking them using Bayesian and Akaike Information Criteria.... validmind.model_validation.statsmodels.AutoARIMA
ThresholdTest ADF Test Assesses the stationarity of time series data using the Augmented Dickey-Fuller (ADF) test.... validmind.model_validation.statsmodels.ADFTest
Metric GINI Table Evaluates classification model performance using AUC, GINI, and KS metrics for training and test datasets.... validmind.model_validation.statsmodels.GINITable
Metric Regression Model Forecast Plot Generates plots to visually compare the forecasted outcomes of one or more regression models against actual... validmind.model_validation.statsmodels.RegressionModelForecastPlot
Metric ADF Assesses the stationarity of a time series dataset using the Augmented Dickey-Fuller (ADF) test.... validmind.model_validation.statsmodels.ADF
Metric Durbin Watson Test Assesses autocorrelation in time series data features using the Durbin-Watson statistic.... validmind.model_validation.statsmodels.DurbinWatsonTest
Metric Missing Values Risk Assesses and quantifies the risk related to missing values in a dataset used for training an ML model.... validmind.data_validation.MissingValuesRisk
Metric IQR Outliers Table Determines and summarizes outliers in numerical features using Interquartile Range method.... validmind.data_validation.IQROutliersTable
Metric Bivariate Features Bar Plots Generates visual bar plots to analyze the relationship between paired features within categorical data in the model.... validmind.data_validation.BivariateFeaturesBarPlots
ThresholdTest Skewness Evaluates the skewness of numerical data in a machine learning model and checks if it falls below a set maximum... validmind.data_validation.Skewness
ThresholdTest Duplicates Tests dataset for duplicate entries, ensuring model reliability via data quality verification.... validmind.data_validation.Duplicates
Metric Missing Values Bar Plot Creates a bar plot showcasing the percentage of missing values in each column of the dataset with risk... validmind.data_validation.MissingValuesBarPlot
Metric Dataset Description Provides comprehensive analysis and statistical summaries of each field in a machine learning model's dataset.... validmind.data_validation.DatasetDescription
Metric Scatter Plot Creates a scatter plot matrix to visually analyze feature relationships, patterns, and outliers in a dataset.... validmind.data_validation.ScatterPlot
ThresholdTest Time Series Outliers Identifies and visualizes outliers in time-series data using z-score method.... validmind.data_validation.TimeSeriesOutliers
Metric Tabular Categorical Bar Plots Generates and visualizes bar plots for each category in categorical features to evaluate dataset's composition.... validmind.data_validation.TabularCategoricalBarPlots
Metric Auto Stationarity Automates Augmented Dickey-Fuller test to assess stationarity across multiple time series in a DataFrame.... validmind.data_validation.AutoStationarity
Metric Descriptive Statistics Performs a detailed descriptive statistical analysis of both numerical and categorical data within a model's... validmind.data_validation.DescriptiveStatistics
Metric ANOVA One Way Table Applies one-way ANOVA (Analysis of Variance) to identify statistically significant numerical features in the... validmind.data_validation.ANOVAOneWayTable
Metric Target Rate Bar Plots Generates bar plots visualizing the default rates of categorical features for a classification machine learning... validmind.data_validation.TargetRateBarPlots
Metric Pearson Correlation Matrix Evaluates linear dependency between numerical variables in a dataset via a Pearson Correlation coefficient heat map.... validmind.data_validation.PearsonCorrelationMatrix
Metric Feature Target Correlation Plot Visualizes the correlation between input features and model's target output in a color-coded horizontal bar plot.... validmind.data_validation.FeatureTargetCorrelationPlot
Metric Tabular Numerical Histograms Generates histograms for each numerical feature in a dataset to provide visual insights into data distribution and... validmind.data_validation.TabularNumericalHistograms
Metric Isolation Forest Outliers Detects outliers in a dataset using the Isolation Forest algorithm and visualizes results through scatter plots.... validmind.data_validation.IsolationForestOutliers
Metric Chi Squared Features Table Executes Chi-Squared test for each categorical feature against a target column to assess significant association.... validmind.data_validation.ChiSquaredFeaturesTable
ThresholdTest High Cardinality Assesses the number of unique values in categorical columns to detect high cardinality and potential overfitting.... validmind.data_validation.HighCardinality
ThresholdTest Missing Values Evaluates dataset quality by ensuring missing value ratio across all features does not exceed a set threshold.... validmind.data_validation.MissingValues
Metric Default Ratesby Risk Band Plot Generates a bar plot showcasing the distribution of default rates across different risk bands in a dataset.... validmind.data_validation.DefaultRatesbyRiskBandPlot
Metric Rolling Stats Plot This test evaluates the stationarity of time series data by plotting its rolling mean and standard deviation.... validmind.data_validation.RollingStatsPlot
Metric Tabular Description Tables Summarizes key descriptive statistics for numerical, categorical, and datetime variables in a dataset.... validmind.data_validation.TabularDescriptionTables
Metric Auto MA Automatically selects the optimal Moving Average (MA) order for each variable in a time series dataset based on... validmind.data_validation.AutoMA
ThresholdTest Unique Rows Verifies the diversity of the dataset by ensuring that the count of unique rows exceeds a prescribed threshold.... validmind.data_validation.UniqueRows
ThresholdTest Too Many Zero Values Identifies numerical columns in a dataset that contain an excessive number of zero values, defined by a threshold... validmind.data_validation.TooManyZeroValues
ThresholdTest High Pearson Correlation Identifies highly correlated feature pairs in a dataset suggesting feature redundancy or multicollinearity.... validmind.data_validation.HighPearsonCorrelation
Metric AC Fand PACF Plot Analyzes time series data using Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots to... validmind.data_validation.ACFandPACFPlot
Metric Bivariate Histograms Generates bivariate histograms for paired features, aiding in visual inspection of categorical variables'... validmind.data_validation.BivariateHistograms
Metric WOE Bin Table Calculates and assesses the Weight of Evidence (WoE) and Information Value (IV) of each feature in a ML model.... validmind.data_validation.WOEBinTable
Metric Heatmap Feature Correlations Creates a heatmap to visually represent correlation patterns between pairs of numerical features in a dataset.... validmind.data_validation.HeatmapFeatureCorrelations
ThresholdTest Time Series Frequency Evaluates consistency of time series data frequency and generates a frequency plot.... validmind.data_validation.TimeSeriesFrequency
Metric Dataset Split Evaluates and visualizes the distribution proportions among training, testing, and validation datasets of an ML... validmind.data_validation.DatasetSplit
Metric Spread Plot Visualizes the spread relationship between pairs of time-series variables in a dataset, thereby aiding in... validmind.data_validation.SpreadPlot
Metric Time Series Line Plot Generates and analyses time-series data through line plots revealing trends, patterns, anomalies over time.... validmind.data_validation.TimeSeriesLinePlot
Metric Pi T Credit Scores Histogram Generates a histogram visualization for observed and predicted credit default scores.... validmind.data_validation.PiTCreditScoresHistogram
Metric Auto Seasonality Automatically identifies and quantifies optimal seasonality in time series data to improve forecasting model... validmind.data_validation.AutoSeasonality
Metric Bivariate Scatter Plots Generates bivariate scatterplots to visually inspect relationships between pairs of predictor variables in machine... validmind.data_validation.BivariateScatterPlots
Metric Engle Granger Coint Validates co-integration in pairs of time series data using the Engle-Granger test and classifies them as... validmind.data_validation.EngleGrangerCoint
ThresholdTest Time Series Missing Values Validates time-series data quality by confirming the count of missing values is below a certain threshold.... validmind.data_validation.TimeSeriesMissingValues
Metric Time Series Histogram Visualizes distribution of time-series data using histograms and Kernel Density Estimation (KDE) lines.... validmind.data_validation.TimeSeriesHistogram
Metric Lagged Correlation Heatmap Assesses and visualizes correlation between target variable and lagged independent variables in a time-series... validmind.data_validation.LaggedCorrelationHeatmap
Metric Seasonal Decompose Decomposes dataset features into observed, trend, seasonal, and residual components to identify patterns and... validmind.data_validation.SeasonalDecompose
Metric WOE Bin Plots Generates visualizations of Weight of Evidence (WoE) and Information Value (IV) for understanding predictive power... validmind.data_validation.WOEBinPlots
ThresholdTest Class Imbalance Evaluates and quantifies class distribution imbalance in a dataset used by a machine learning model.... validmind.data_validation.ClassImbalance
Metric IQR Outliers Bar Plot Visualizes outlier distribution across percentiles in numerical data using Interquartile Range (IQR) method.... validmind.data_validation.IQROutliersBarPlot
Metric Pi TPD Histogram Assesses credit risk prediction accuracy of a model by comparing actual and predicted defaults at a chosen point in... validmind.data_validation.PiTPDHistogram
Metric Auto AR Automatically identifies the optimal Autoregressive (AR) order for a time series using BIC and AIC criteria.... validmind.data_validation.AutoAR
Metric Tabular Date Time Histograms Generates histograms to provide graphical insight into the distribution of time intervals in model's datetime data.... validmind.data_validation.TabularDateTimeHistograms
Metric Punctuations Analyzes and visualizes the frequency distribution of punctuation usage in a given text dataset.... validmind.data_validation.nlp.Punctuations
Metric Common Words Identifies and visualizes the 40 most frequent non-stopwords in a specified text column within a dataset.... validmind.data_validation.nlp.CommonWords
ThresholdTest Hashtags Assesses hashtag frequency in a text column, highlighting usage trends and potential dataset bias or spam.... validmind.data_validation.nlp.Hashtags
ThresholdTest Mentions Calculates and visualizes frequencies of '@' prefixed mentions in a text-based dataset for NLP model analysis.... validmind.data_validation.nlp.Mentions
Metric Text Description Performs comprehensive textual analysis on a dataset using NLTK, evaluating various parameters and generating... validmind.data_validation.nlp.TextDescription
ThresholdTest Stop Words Evaluates and visualizes the frequency of English stop words in a text dataset against a defined threshold.... validmind.data_validation.nlp.StopWords

Understanding Tags and Task Types

Effectively using ValidMind’s tests involves a deep understanding of its ‘tags’ and ‘task types’. Here’s a breakdown:

  • Task Types: Represent the kind of modeling task associated with a test. For instance:
    • classification: Works with Classification Models and Datasets
    • regression: Works with Regression Models and Datasets
    • text classification: Works with Text Classification Models and Datasets
    • text summarization: Works with Text Summarization Models and Datasets
  • Tags: Free-form descriptors providing more details about the test, what data and models the test is compatible with and what category the test falls into etc. Some examples include:
    • llm: Tests that work with Large Language Models
    • nlp: Tests relevant for natural language processing.
    • binary_classification: Tests for binary classification tasks.
    • forecasting: Tests for forecasting and time-series analysis.
    • tabular_data: Tests for tabular data like CSVs and Excel spreadsheets.

You can use the functions list_task_types() and list_tags() to view all the task_types and tags used for classifying all the tests available in the developer framework:

list_task_types()
['feature_extraction',
 'regression',
 'classification',
 'clustering',
 'text_summarization',
 'text_classification']
list_tags()
['regard_histogram',
 'llm',
 'feature_importance',
 'anomaly_detection',
 'kmeans',
 'credit_risk',
 'time_series_data',
 'model_comparison',
 'tabular_data',
 'numerical_data',
 'model_interpretation',
 'visualization',
 'multiclass_classification',
 'statsmodels',
 'correlation',
 'statistical_test',
 'model_diagnosis',
 'categorical_data',
 'forecasting',
 'unit_root_test',
 'frequency_analysis',
 'binary_classification',
 'model_metadata',
 'senstivity_analysis',
 'toxicity_histogram',
 'few_shot',
 'data_quality',
 'data_distribution',
 'logistic_regression',
 'text_embeddings',
 'text_data',
 'nlp',
 'zero_shot',
 'risk_analysis',
 'regard_score',
 'model_selection',
 'model_performance',
 'seasonality',
 'sklearn',
 'stationarity',
 'toxicity_line_plot']

If you want to see which tags correspond to which task type, you can use the function list_tasks_and_tags():

list_tasks_and_tags()
Task Type Tags
text_classification regard_histogram, llm, feature_importance, time_series_data, model_comparison, tabular_data, multiclass_classification, visualization, model_diagnosis, frequency_analysis, binary_classification, model_metadata, toxicity_histogram, few_shot, text_data, nlp, zero_shot, regard_score, model_performance, sklearn, toxicity_line_plot
text_summarization nlp, zero_shot, llm, regard_histogram, regard_score, model_metadata, time_series_data, tabular_data, toxicity_histogram, few_shot, visualization, frequency_analysis, toxicity_line_plot, text_data
classification feature_importance, anomaly_detection, credit_risk, time_series_data, model_comparison, tabular_data, numerical_data, multiclass_classification, visualization, statsmodels, correlation, statistical_test, model_diagnosis, categorical_data, binary_classification, model_metadata, data_quality, data_distribution, logistic_regression, text_data, risk_analysis, model_performance, sklearn
regression feature_importance, time_series_data, model_comparison, tabular_data, numerical_data, model_interpretation, visualization, statsmodels, correlation, statistical_test, categorical_data, forecasting, unit_root_test, model_metadata, senstivity_analysis, data_quality, data_distribution, text_data, risk_analysis, model_selection, model_performance, seasonality, sklearn, stationarity
clustering kmeans, model_performance, sklearn
feature_extraction llm, visualization, text_embeddings, text_data

Searching for Specific Tests using tags and task_types

While listing all tests is valuable, there are times when you need to narrow down your search. The list_tests function offers filter, task, and tags parameters to assist in this.

If you’re targeting a specific test or tests that match a particular task type, the filter parameter comes in handy. For example, to list tests that are compatible with ‘sklearn’ models:

list_tests(filter="sklearn")
Test Type Name Description ID
ThresholdTest Minimum Score Evaluates if the model's F1 score on the validation set meets a predefined minimum threshold.... validmind.model_validation.sklearn.MinimumF1Score
Metric SHAP Global Importance Evaluates and visualizes global feature importance using SHAP values for model explanation and risk identification.... validmind.model_validation.sklearn.SHAPGlobalImportance
ThresholdTest Overfit Diagnosis Detects and visualizes overfit regions in an ML model by comparing performance on training and test datasets.... validmind.model_validation.sklearn.OverfitDiagnosis
Metric Confusion Matrix Evaluates and visually represents the classification ML model's predictive performance using a Confusion Matrix... validmind.model_validation.sklearn.ConfusionMatrix
Metric Classifier Performance Evaluates performance of binary or multiclass classification models using precision, recall, F1-Score, accuracy,... validmind.model_validation.sklearn.ClassifierPerformance
ThresholdTest Minimum ROCAUC Score Validates model by checking if the ROC AUC score meets or surpasses a specified threshold.... validmind.model_validation.sklearn.MinimumROCAUCScore
Metric Adjusted Rand Index Measures the similarity between two data clusters using the Adjusted Rand Index (ARI) metric in clustering machine... validmind.model_validation.sklearn.AdjustedRandIndex
Metric K Means Clusters Optimization Optimizes the number of clusters in K-means models using Elbow and Silhouette methods.... validmind.model_validation.sklearn.KMeansClustersOptimization
Metric Homogeneity Score Assesses clustering homogeneity by comparing true and predicted labels, scoring from 0 (heterogeneous) to 1... validmind.model_validation.sklearn.HomogeneityScore
Metric ROC Curve Evaluates binary classification model performance by generating and plotting the Receiver Operating Characteristic... validmind.model_validation.sklearn.ROCCurve
Metric Permutation Feature Importance Assesses the significance of each feature in a model by evaluating the impact on model performance when feature... validmind.model_validation.sklearn.PermutationFeatureImportance
Metric Cluster Performance Evaluates and compares a clustering model's performance on training and testing datasets using multiple defined... validmind.model_validation.sklearn.ClusterPerformance
ThresholdTest Weakspots Diagnosis Identifies and visualizes weak spots in a machine learning model's performance across various sections of the... validmind.model_validation.sklearn.WeakspotsDiagnosis
Metric Silhouette Plot Calculates and visualizes Silhouette Score, assessing degree of data point suitability to its cluster in ML models.... validmind.model_validation.sklearn.SilhouettePlot
Metric Regression Errors **Purpose**: This metric is used to measure the performance of a regression model. It gauges the model's accuracy... validmind.model_validation.sklearn.RegressionErrors
Metric Hyper Parameters Tuning Exerts exhaustive grid search to identify optimal hyperparameters for the model, improving performance.... validmind.model_validation.sklearn.HyperParametersTuning
Metric Adjusted Mutual Information Evaluates clustering model performance by measuring mutual information between true and predicted labels, adjusting... validmind.model_validation.sklearn.AdjustedMutualInformation
Metric Completeness Score Evaluates a clustering model's capacity to categorize instances from a single class into the same cluster.... validmind.model_validation.sklearn.CompletenessScore
ThresholdTest Minimum Accuracy Checks if the model's prediction accuracy meets or surpasses a specified threshold.... validmind.model_validation.sklearn.MinimumAccuracy
Metric Regression Square **Purpose**: The purpose of the RegressionR2Square Metric test is to measure the overall goodness-of-fit of a... validmind.model_validation.sklearn.RegressionR2Square
Metric V Measure Evaluates homogeneity and completeness of a clustering model using the V Measure Score.... validmind.model_validation.sklearn.VMeasure
Metric Regression Models Performance Comparison Compares and evaluates the performance of multiple regression models using five different metrics: MAE, MSE, RMSE,... validmind.model_validation.sklearn.RegressionModelsPerformanceComparison
Metric Precision Recall Curve Evaluates the precision-recall trade-off for binary classification models and visualizes the Precision-Recall curve.... validmind.model_validation.sklearn.PrecisionRecallCurve
Metric Cluster Cosine Similarity Measures the intra-cluster similarity of a clustering model using cosine similarity.... validmind.model_validation.sklearn.ClusterCosineSimilarity
Metric Cluster Size Distribution Compares and visualizes the distribution of cluster sizes in model predictions and actual data for assessing... validmind.model_validation.ClusterSizeDistribution
Metric Fowlkes Mallows Score Evaluates the similarity between predicted and actual cluster assignments in a model using the Fowlkes-Mallows... validmind.model_validation.sklearn.FowlkesMallowsScore
ThresholdTest Training Test Degradation Tests if model performance degradation between training and test datasets exceeds a predefined threshold.... validmind.model_validation.sklearn.TrainingTestDegradation
Metric Models Performance Comparison Evaluates and compares the performance of multiple Machine Learning models using various metrics like accuracy,... validmind.model_validation.sklearn.ModelsPerformanceComparison
Metric Population Stability Index Evaluates the Population Stability Index (PSI) to quantify the stability of an ML model's predictions across... validmind.model_validation.sklearn.PopulationStabilityIndex
Metric Cluster Performance Metrics Evaluates the performance of clustering machine learning models using multiple established metrics.... validmind.model_validation.sklearn.ClusterPerformanceMetrics
ThresholdTest Robustness Diagnosis Evaluates the robustness of a machine learning model by injecting Gaussian noise to input data and measuring... validmind.model_validation.sklearn.RobustnessDiagnosis

The task parameter is designed for pinpointing tests that align with a specific task type. For instance, to find tests tailored for ‘classification’ tasks:

list_tests(task="classification")
Test Type Name Description ID
Metric Model Metadata Extracts and summarizes critical metadata from a machine learning model instance for comprehensive analysis.... validmind.model_validation.ModelMetadata
ThresholdTest Robustness Diagnosis Evaluates the robustness of a machine learning model by injecting Gaussian noise to input data and measuring... validmind.model_validation.sklearn.RobustnessDiagnosis
Metric SHAP Global Importance Evaluates and visualizes global feature importance using SHAP values for model explanation and risk identification.... validmind.model_validation.sklearn.SHAPGlobalImportance
Metric Confusion Matrix Evaluates and visually represents the classification ML model's predictive performance using a Confusion Matrix... validmind.model_validation.sklearn.ConfusionMatrix
ThresholdTest Overfit Diagnosis Detects and visualizes overfit regions in an ML model by comparing performance on training and test datasets.... validmind.model_validation.sklearn.OverfitDiagnosis
Metric Permutation Feature Importance Assesses the significance of each feature in a model by evaluating the impact on model performance when feature... validmind.model_validation.sklearn.PermutationFeatureImportance
ThresholdTest Minimum ROCAUC Score Validates model by checking if the ROC AUC score meets or surpasses a specified threshold.... validmind.model_validation.sklearn.MinimumROCAUCScore
Metric Precision Recall Curve Evaluates the precision-recall trade-off for binary classification models and visualizes the Precision-Recall curve.... validmind.model_validation.sklearn.PrecisionRecallCurve
Metric Classifier Performance Evaluates performance of binary or multiclass classification models using precision, recall, F1-Score, accuracy,... validmind.model_validation.sklearn.ClassifierPerformance
ThresholdTest Minimum Score Evaluates if the model's F1 score on the validation set meets a predefined minimum threshold.... validmind.model_validation.sklearn.MinimumF1Score
Metric ROC Curve Evaluates binary classification model performance by generating and plotting the Receiver Operating Characteristic... validmind.model_validation.sklearn.ROCCurve
ThresholdTest Training Test Degradation Tests if model performance degradation between training and test datasets exceeds a predefined threshold.... validmind.model_validation.sklearn.TrainingTestDegradation
Metric Hyper Parameters Tuning Exerts exhaustive grid search to identify optimal hyperparameters for the model, improving performance.... validmind.model_validation.sklearn.HyperParametersTuning
Metric Models Performance Comparison Evaluates and compares the performance of multiple Machine Learning models using various metrics like accuracy,... validmind.model_validation.sklearn.ModelsPerformanceComparison
ThresholdTest Weakspots Diagnosis Identifies and visualizes weak spots in a machine learning model's performance across various sections of the... validmind.model_validation.sklearn.WeakspotsDiagnosis
Metric Population Stability Index Evaluates the Population Stability Index (PSI) to quantify the stability of an ML model's predictions across... validmind.model_validation.sklearn.PopulationStabilityIndex
ThresholdTest Minimum Accuracy Checks if the model's prediction accuracy meets or surpasses a specified threshold.... validmind.model_validation.sklearn.MinimumAccuracy
Metric Log Regression Confusion Matrix Generates a confusion matrix for logistic regression model performance, utilizing thresholded probabilities for... validmind.model_validation.statsmodels.LogRegressionConfusionMatrix
Metric PD Rating Class Plot Assesses and visualizes credit risk distribution across different rating classes within a dataset via default... validmind.model_validation.statsmodels.PDRatingClassPlot
Metric Scorecard Histogram Creates histograms of credit scores, from both default and non-default instances, generated by a credit-risk model.... validmind.model_validation.statsmodels.ScorecardHistogram
Metric Logistic Reg Prediction Histogram Generates and visualizes histograms of the Probability of Default predictions for both positive and negative... validmind.model_validation.statsmodels.LogisticRegPredictionHistogram
Metric Jarque Bera Assesses normality of dataset features in an ML model using the Jarque-Bera test.... validmind.model_validation.statsmodels.JarqueBera
Metric Kolmogorov Smirnov Executes a feature-wise Kolmogorov-Smirnov test to evaluate alignment with normal distribution in datasets.... validmind.model_validation.statsmodels.KolmogorovSmirnov
Metric Shapiro Wilk Evaluates feature-wise normality of training data using the Shapiro-Wilk test.... validmind.model_validation.statsmodels.ShapiroWilk
Metric Scorecard Bucket Histogram Evaluates and visualizes distribution of risk categories in a classification model's scores, useful in credit risk... validmind.model_validation.statsmodels.ScorecardBucketHistogram
Metric Lilliefors Assesses the normality of feature distributions in an ML model's training dataset using the Lilliefors test.... validmind.model_validation.statsmodels.Lilliefors
Metric Logistic Reg Cumulative Prob Visualizes cumulative probabilities of positive and negative classes for both training and testing in logistic... validmind.model_validation.statsmodels.LogisticRegCumulativeProb
Metric Runs Test Executes Runs Test on ML model to detect non-random patterns in output data sequence.... validmind.model_validation.statsmodels.RunsTest
Metric Scorecard Probabilities Histogram Evaluates risk classification of a model by visualizing the distribution of default probability across score... validmind.model_validation.statsmodels.ScorecardProbabilitiesHistogram
Metric GINI Table Evaluates classification model performance using AUC, GINI, and KS metrics for training and test datasets.... validmind.model_validation.statsmodels.GINITable
Metric Missing Values Risk Assesses and quantifies the risk related to missing values in a dataset used for training an ML model.... validmind.data_validation.MissingValuesRisk
Metric IQR Outliers Table Determines and summarizes outliers in numerical features using Interquartile Range method.... validmind.data_validation.IQROutliersTable
Metric Bivariate Features Bar Plots Generates visual bar plots to analyze the relationship between paired features within categorical data in the model.... validmind.data_validation.BivariateFeaturesBarPlots
ThresholdTest Skewness Evaluates the skewness of numerical data in a machine learning model and checks if it falls below a set maximum... validmind.data_validation.Skewness
ThresholdTest Duplicates Tests dataset for duplicate entries, ensuring model reliability via data quality verification.... validmind.data_validation.Duplicates
Metric Missing Values Bar Plot Creates a bar plot showcasing the percentage of missing values in each column of the dataset with risk... validmind.data_validation.MissingValuesBarPlot
Metric Dataset Description Provides comprehensive analysis and statistical summaries of each field in a machine learning model's dataset.... validmind.data_validation.DatasetDescription
Metric Scatter Plot Creates a scatter plot matrix to visually analyze feature relationships, patterns, and outliers in a dataset.... validmind.data_validation.ScatterPlot
Metric Tabular Categorical Bar Plots Generates and visualizes bar plots for each category in categorical features to evaluate dataset's composition.... validmind.data_validation.TabularCategoricalBarPlots
Metric Descriptive Statistics Performs a detailed descriptive statistical analysis of both numerical and categorical data within a model's... validmind.data_validation.DescriptiveStatistics
Metric ANOVA One Way Table Applies one-way ANOVA (Analysis of Variance) to identify statistically significant numerical features in the... validmind.data_validation.ANOVAOneWayTable
Metric Target Rate Bar Plots Generates bar plots visualizing the default rates of categorical features for a classification machine learning... validmind.data_validation.TargetRateBarPlots
Metric Pearson Correlation Matrix Evaluates linear dependency between numerical variables in a dataset via a Pearson Correlation coefficient heat map.... validmind.data_validation.PearsonCorrelationMatrix
Metric Feature Target Correlation Plot Visualizes the correlation between input features and model's target output in a color-coded horizontal bar plot.... validmind.data_validation.FeatureTargetCorrelationPlot
Metric Tabular Numerical Histograms Generates histograms for each numerical feature in a dataset to provide visual insights into data distribution and... validmind.data_validation.TabularNumericalHistograms
Metric Isolation Forest Outliers Detects outliers in a dataset using the Isolation Forest algorithm and visualizes results through scatter plots.... validmind.data_validation.IsolationForestOutliers
Metric Chi Squared Features Table Executes Chi-Squared test for each categorical feature against a target column to assess significant association.... validmind.data_validation.ChiSquaredFeaturesTable
ThresholdTest High Cardinality Assesses the number of unique values in categorical columns to detect high cardinality and potential overfitting.... validmind.data_validation.HighCardinality
ThresholdTest Missing Values Evaluates dataset quality by ensuring missing value ratio across all features does not exceed a set threshold.... validmind.data_validation.MissingValues
Metric Default Ratesby Risk Band Plot Generates a bar plot showcasing the distribution of default rates across different risk bands in a dataset.... validmind.data_validation.DefaultRatesbyRiskBandPlot
Metric Tabular Description Tables Summarizes key descriptive statistics for numerical, categorical, and datetime variables in a dataset.... validmind.data_validation.TabularDescriptionTables
ThresholdTest Unique Rows Verifies the diversity of the dataset by ensuring that the count of unique rows exceeds a prescribed threshold.... validmind.data_validation.UniqueRows
ThresholdTest Too Many Zero Values Identifies numerical columns in a dataset that contain an excessive number of zero values, defined by a threshold... validmind.data_validation.TooManyZeroValues
ThresholdTest High Pearson Correlation Identifies highly correlated feature pairs in a dataset suggesting feature redundancy or multicollinearity.... validmind.data_validation.HighPearsonCorrelation
Metric Bivariate Histograms Generates bivariate histograms for paired features, aiding in visual inspection of categorical variables'... validmind.data_validation.BivariateHistograms
Metric WOE Bin Table Calculates and assesses the Weight of Evidence (WoE) and Information Value (IV) of each feature in a ML model.... validmind.data_validation.WOEBinTable
Metric Heatmap Feature Correlations Creates a heatmap to visually represent correlation patterns between pairs of numerical features in a dataset.... validmind.data_validation.HeatmapFeatureCorrelations
Metric Dataset Split Evaluates and visualizes the distribution proportions among training, testing, and validation datasets of an ML... validmind.data_validation.DatasetSplit
Metric Pi T Credit Scores Histogram Generates a histogram visualization for observed and predicted credit default scores.... validmind.data_validation.PiTCreditScoresHistogram
Metric Bivariate Scatter Plots Generates bivariate scatterplots to visually inspect relationships between pairs of predictor variables in machine... validmind.data_validation.BivariateScatterPlots
Metric WOE Bin Plots Generates visualizations of Weight of Evidence (WoE) and Information Value (IV) for understanding predictive power... validmind.data_validation.WOEBinPlots
ThresholdTest Class Imbalance Evaluates and quantifies class distribution imbalance in a dataset used by a machine learning model.... validmind.data_validation.ClassImbalance
Metric IQR Outliers Bar Plot Visualizes outlier distribution across percentiles in numerical data using Interquartile Range (IQR) method.... validmind.data_validation.IQROutliersBarPlot
Metric Pi TPD Histogram Assesses credit risk prediction accuracy of a model by comparing actual and predicted defaults at a chosen point in... validmind.data_validation.PiTPDHistogram
Metric Tabular Date Time Histograms Generates histograms to provide graphical insight into the distribution of time intervals in model's datetime data.... validmind.data_validation.TabularDateTimeHistograms

The tags parameter facilitates searching tests by their tags. For instance, if you’re interested in only tests associated designed for model_performance that produce a plot (denoted by the visualization tag)

list_tests(tags=["model_performance", "visualization"])
Test Type Name Description ID
Metric Confusion Matrix Evaluates and visually represents the classification ML model's predictive performance using a Confusion Matrix... validmind.model_validation.sklearn.ConfusionMatrix
Metric Precision Recall Curve Evaluates the precision-recall trade-off for binary classification models and visualizes the Precision-Recall curve.... validmind.model_validation.sklearn.PrecisionRecallCurve
Metric ROC Curve Evaluates binary classification model performance by generating and plotting the Receiver Operating Characteristic... validmind.model_validation.sklearn.ROCCurve
ThresholdTest Training Test Degradation Tests if model performance degradation between training and test datasets exceeds a predefined threshold.... validmind.model_validation.sklearn.TrainingTestDegradation
Metric Log Regression Confusion Matrix Generates a confusion matrix for logistic regression model performance, utilizing thresholded probabilities for... validmind.model_validation.statsmodels.LogRegressionConfusionMatrix
Metric GINI Table Evaluates classification model performance using AUC, GINI, and KS metrics for training and test datasets.... validmind.model_validation.statsmodels.GINITable

The above parameters can be combined to create complex queries. For instance, to find tests that are compatible with ‘sklearn’ models, designed for ‘classification’ tasks, and produce a plot:

list_tests(tags=["model_performance", "visualization", "sklearn"], task="classification")
Test Type Name Description ID
Metric Confusion Matrix Evaluates and visually represents the classification ML model's predictive performance using a Confusion Matrix... validmind.model_validation.sklearn.ConfusionMatrix
Metric Precision Recall Curve Evaluates the precision-recall trade-off for binary classification models and visualizes the Precision-Recall curve.... validmind.model_validation.sklearn.PrecisionRecallCurve
Metric ROC Curve Evaluates binary classification model performance by generating and plotting the Receiver Operating Characteristic... validmind.model_validation.sklearn.ROCCurve
ThresholdTest Training Test Degradation Tests if model performance degradation between training and test datasets exceeds a predefined threshold.... validmind.model_validation.sklearn.TrainingTestDegradation

Programmatic Use

To work with a specific set of tests programmatically, you can store the results in a variable. For instance, let’s list all tests that are designed for Text Summarization tests and store them in text_summarization_tests for further use.

text_summarization_tests = list_tests(task="text_summarization", pretty=False)
text_summarization_tests
['validmind.prompt_validation.Bias',
 'validmind.prompt_validation.Clarity',
 'validmind.prompt_validation.Specificity',
 'validmind.prompt_validation.Robustness',
 'validmind.prompt_validation.NegativeInstruction',
 'validmind.prompt_validation.Conciseness',
 'validmind.prompt_validation.Delimitation',
 'validmind.model_validation.RegardScore',
 'validmind.model_validation.RegardHistogram',
 'validmind.model_validation.ToxicityHistogram',
 'validmind.model_validation.ModelMetadata',
 'validmind.model_validation.ToxicityScore',
 'validmind.data_validation.DatasetDescription',
 'validmind.data_validation.DatasetSplit',
 'validmind.data_validation.nlp.Punctuations',
 'validmind.data_validation.nlp.CommonWords',
 'validmind.data_validation.nlp.Hashtags',
 'validmind.data_validation.nlp.Mentions',
 'validmind.data_validation.nlp.StopWords']

Delving into Test Details with describe_test

After identifying a set of potential tests, you might want to explore the specifics of an individual test. The describe_test function provides a deep dive into the details of a test. It reveals the test name, description, ID, test type, and required inputs. Below, we showcase how to describe a test using its ID:

describe_test("validmind.model_validation.sklearn.OverfitDiagnosis")

Conclusion and Next Steps

By harnessing the functionalities presented in this guide, you should be able to easily list and filter through all of ValidMind’s available tests and find those you are interested in running against your model and/or dataset. The next step is to take the IDs of the tests you’d like to run and either create a Test Suite for reuse or just run them directly to try them out. See the other notebooks for a tutorial on how to do both.