# RIME v18 Release Note

## Important Notes
- Please ensure all outstanding jobs are completed before upgrading to the v18 version.
- Install the latest version of the SDK.
- SDK breaking changes
  - The SDK method run_firewall_incremental_data() is now called start_continuous_test().
  - The SDK method run_firewall_from_reference has been deprecated.
  - The SDK method upload_dataset_file() is now called upload_file() and its allows uploading
  of any file types.
  - The SDK method upload_model_directory() is now called upload_directory().


## Changelog

### Stress Testing
- **Summary Tests** - This new top-level set of meta-tests aggregate the results of individual tests and enable users to quickly get answers to common questions like, for example, "Is my model robust to data drift?" or "Is my model sensitive to adversarial attacks?" Each summary test can be conceptually imagined to be a different category of tests, and now replaces categories in the RI Platform nomenclature. Each summary test contains various relevant summary metrics and an overall pass/fail result as well.
- **Improved Custom Tests** - Users can now easily specify a type of abnormal input to create an Abnormal Input custom test.

### Firewall and Continuous Testing
- **New Auto-configured Summary Metrics** - There are several new summary metrics that Continuous Testing tracks over time. All of these metrics have auto-configured thresholds to indicate various levels of production model failure.
- **Metric Thresholding** - All summary metric thresholds can now be edited from both the SDK and the UI.
- **Alerts and Notifications** - Users can now set up monitoring notifications for various summary metrics crossing thresholds. These notifications can be sent via emails or webhooks that can be configured from the UI.
- **Online Configurability** - Continuous tests can now have their configuration edited after production deployment so that users can modify both the CLI configs (which allow for updating data paths and the like) and the test configs (which allow for the updating of test thresholds and arguments).
- **Querying Results** - Besides viewing RIME results in the web application, users can use the Python SDK to retrieve continuous testing results.

### Compliance Management
- **Bias and Fairness test category** - These tests now work for all tabular modalities - binary and multi classification, regressions, and ranking.
- **Monitoring Compliance Requirements** - Bias and Fairness test category now works in Continuous Testing.
- **Auto-generated Model Cards** -  A comprehensive downloadable report that is auto-populated with validation information from test runs and allows for entering of custom information. This can be used to meet internal team or external regulator reporting requirements.

### RIME Platform Changes
- **Cloud Deployment (Beta)** - The Robust Intelligence Platform is now composed of a **control plane** and a **data plane**. This modularization enables a new deployment option, **Cloud (Beta)**, in which customers need only install the **data plane** and can license the **control plane** from Robust Intelligence. The **data plane** contains an agent that executes the model testing suite, while the **control plane** handles core application services like serving the web client. This way your data and models stay in your environment, and Robust Intelligence hosts and manages testing-related metadata for you. [More information here.](/for_admins/installation_guide.rst)
- **User Authentication** - Admins can now manage OIDC users from the members page. [More information here.](/for_admins/how_to_guides/users.md)
- **Audit Logs** - Admins can access a records of events and activity on the RI platform.
- **Managed Images**
  - RI Platform now provides 4 base images (Tabular, NLP, Images, ALL) that can be used to create managed images. By default, the image used is "ALL".
  - Ability to add apt system requirements to Managed Images.