Release Notes

For notes on upgrades and deprecations to the Robust Intelligence REST API, refer to the API changelog.

Unless noted otherwise, you must upgrade all Robust Intelligence components to take advantage of the changes listed here.

2.1

Category field deprecated

The string category field is deprecated in favor of the test_category enum field. The string category field will be removed in v2.3. category is currently a sub-field of TestBatchResult, TestCase, and CategorySummaryMetric.

Python 3.7 no longer supported

Robust Intelligence now supports Python 3.8 through Python 3.11. As of Robust Intelligence 2.1, projects using Python 3.7 are no longer supported. Projects based on Python 3.7 should be migrated to Python 3.8 or later.

delta_lake connection_info renamed

The delta_lake connection_info has been renamed to databricks. This loads a Databricks Delta Lake Table.

Before:

{
    "connection_info": {
        "delta_lake": {
            "table_name": "Table",
        }
    },
    "data_params": ...,
}

After:

{
    "connection_info": {
        "databricks": {
            "table_name": "Table",
        }
    },
    "data_params": ...,
}

2.0 GA

global_exclude_columns parameter has moved

The location in the Test Suite Configuration to specify the global_exclude_columns parameter has changed.

Before:

stress_test_config = {
    # other config params
    "test_suite_config": {
        "individual_tests_config": {
            "global_exclude_columns": ["foo", "bar"]
        }
    }
}

After:

stress_test_config = {
    # other config params
    "test_suite_config": {
        "global_exclude_columns": ["foo", "bar"]
    }
}

Any existing projects with this parameter specified in the project’s Test Suite Configuration may need to be updated with this new version of the configuration.

The specification for the test categories that Robust Intelligence runs has changed, both at a project level and for individual Stress Tests.

Before:

# Update the project's Stress Testing, Continuous Testing categories
project.update_test_suite_config({
	"categories": [
	  {
	      "name": "Model Performance",
	      "run_st": True,
	      "run_ct": True
	  },
	  {
	      "name": "Subset Performance",
	      "run_st": True,
	      "run_ct": False
	  },
	  {
	      "name": "Drift",
	      "run_st": False,
	      "run_ct": True
	  }
	],
	# ...
})

# When running Stress Testing, user can override the project's Stress Testing categories:
stress_test_config = {
    # ... other params in stress test config
    "test_suite_config": {
        "categories": [
            {
                "name": "Bias and Fairness",
                "run_st": True,
                "run_ct": False
            }
        ],
      # ... other params in test suite config
    }
}

After:

# Update the project's Stress Testing, Continuous Testing categories
project.update_stress_testing_categories(
    ["TEST_CATEGORY_TYPE_MODEL_PERFORMANCE", "TEST_CATEGORY_TYPE_SUBSET_PERFORMANCE"]
)
project.update_continuous_testing_categories(
    ["TEST_CATEGORY_TYPE_MODEL_PERFORMANCE", "TEST_CATEGORY_TYPE_DRIFT"]
)

# When running Stress Testing, user can override the project's Stress Testing categories:
stress_test_config = {
    # ... other params in stress test config
    "categories": ["TEST_CATEGORY_TYPE_BIAS_AND_FAIRNESS"]
    "test_suite_config": {
      # ... other params in test suite config (if any)
    }
}