Data handling
Does ValidMind handle large datasets?
The developer framework executes test suites and functions locally in your environment and is not limited by dataset size.
What about the confidentiality of data sent to ValidMind?
ValidMind does not send datasets outside of your environment. Additionally, ValidMind adheres to a strict data confidentiality and retention policy, compliant with the SOC 2 security standard.
For more information, refer to our data privacy policy.
Can the tool automatically document other non-standard ETL steps or performance metrics from notebooks?
Support for more complex data processing pipelines is on our roadmap. We are implementing connector interfaces that will allow us to extract relationships between raw data sources and final post-processed datasets for Spark and Snowflake.
How does the tool manage model changes?
ValidMind allows model developers to re-run documentation functions with the developer framework to capture changes in the model, such as changes in the number of features or hyperparameters.
After a model developer makes a change in their development environment, such as in a Jupyter notebook, they can execute the relevant ValidMind documentation function to update the corresponding documentation section. ValidMind automatically recreates the relevant figures and tables, updating them in the online documentation.
ValidMind is currently working on a version history function, which will allow users to see the history of changes made to the documentation.
Can you accommodate Spark DataFrames?
Our developer framework can extract dataset quality metrics on Pandas DataFrame, NumPy arrays, or Spark DataFrame instances using standard metrics provided by popular open-source frameworks such as scikit-learn, statsmodels, and more.
Each test defines a mapping to the different supported dataset and/or model interfaces: when passing a Spark DataFrame, our framework will directly call native evaluation metrics provided by the SparkML API or custom ones built by the developer (such as via UDFs).
How does ValidMind handle end-user computing and spreadsheet models?
Customers will be able to register spreadsheet models in the model inventory and centralize tracking of the associated documentation files with the inventory metadata (roadmap item). However, ValidMind cannot automate documentation generation for spreadsheet models.