# AI Stress Testing

This tutorial will get you started using the RIME CV CLI to stress test an image
classification model using the <a target="_blank" rel="noopener" href="https://cvml.ist.ac.at/AwA2/">
Animals with Attributes 2 Dataset</a>. If you are interested in learning more
about RIME for CV, reach out to your RI trial representatives.

### Setup
{{ cv_setup_extra_note }}

### Running Stress Testing on the Image Classification Example

In this tutorial, we will cover pre-production testing of an animal classification model using
the RIME CV automated test suite.

Then, to kick off a CV Test run, execute the following command in your local terminal:

```bash
rime-engine run-images --config-path images_examples/classification/awa2/stress_test_config.json
```

If you explore the test config in `images_examples/classification/cv/stress_tests_config.json` you'll see that we've configured a few parameters to specify the data, model, and other task-specific information.
{{ tabular_ui_redirect }}

{{ cv_config_note }}

{{ cli_note }}

### Running Stress Testing on the Object Detection Example

In this tutorial, we will cover pre-production testing of an object detection model trained on Pascal VOC using the RIME CV automated test suite.

Then, to kick off a CV Test run, execute the following command in your local terminal:

```bash
rime-engine run-images --config-path images_examples/detection/voc/stress_test_config.json
```

If you explore the test config in `images_examples/detection/voc/stress_test_config.json` you'll see that we've configured a few parameters to specify the data, model, and other task-specific information.
{{ tabular_ui_redirect }}

{{ cv_config_note }}

{{ cli_note }}

### Running Stress Testing on your own Model and Datasets

To run RIME using your own data and model, please consult the
[CV Data Guide](/configuration/cv/task_data_format.md)
guide for the expected data format and [How to Create a CV Model File](specify_model_cv.md) for step-by-step instructions on how to connect your model to the testing framework.

Model inference is usually the most time-consuming part of the testing framework, which is why we recommend specifying cached prediction logs. This can be done by providing predictions within the datafile itself (by adding the "probabilities" key to each data sample) or by [specifying them in a separate file](/configuration/cv/prediction_info). In the classification example above, we provide the prediction within the datafile itself (see `images_examples/classification/awa2/data/train_inputs_trial.json`). For an example of providing predictions within the datafile itself for object detection, see `images_examples/detection/voc/stress_test_config.json`.

However, if you do not wish to create a prediction log beforehand, RIME can call your model during a test run and infer its performance using a subsample of the provided datasets.

### Conclusion

Congratulations! You've successfully used RIME to test out the various CV models. Assuming the RIME backend was launched successfully, you can navigate to the [RIME web interface](http://localhost:4000/projects) and view the test results.

Once again, we strongly recommended that you run RIME using precomputed predictions. This will greatly improve both the RIME runtime and the test suite result quality. Model inference tends to be the most computationally expensive part of each RIME run. While access to the model is still required for some tests due to design constraints (e.g., the use of randomness, iterative attacks, etc.), providing predictions in advance can help RIME avoid redundant computation so each run is fast and focused.

### Troubleshooting
{{ troubleshooting_note }}