Uploading Data to RIME

You can use the SDK to upload data files and model directories to a s3 bucket that RIME has permission to use.

Uploading Data Files

To begin, initialize the Client and point it to the location of your RIME backend.

from rime_sdk import Client

rime_client = Client("rime.<YOUR_ORG_NAME>.rbst.io", "<YOUR_API_TOKEN>")

Afterwards, use the upload_file(file_path, upload_path) method to upload data.

When the upload_path is not specified, the path contains a random string as directory name.

ref_path = rime_client.upload_file(file_path="./titanic/datasets/train.csv")
print("ref_path:", ref_path)
"ref_path: s3://rime-blob-c78976e9367daf-environment/files/84f82c37-0d8f-43af-8d05-76jknwefnjkm/data/train.csv"

When the upload_path is specified, the path contains the upload_path string in place of the random string.

ref_path = client.upload_file(file_path="./titanic/datasets/train.csv", upload_path="rime_tests/titanic_tests")
print("ref_path:", ref_path)
"ref_path: s3://rime-blob-c78976e9367daf-environment/files/rime_tests/titanic_tests/data/train.csv"

Uploading Images

RIME expects image dataset files as a list of JSON dictionaries with an images_features parameter that references an image by an absolute or relative path. Each image in the dataset file is uploaded to the RIME cluster’s storage.

Use upload_local_image_dataset_file to upload an image dataset file. When your image paths already reference external blob storage, use upload_file.

When the upload_path is not specified, the path contains a random string as directory name.

img_dataset_dir = upload_local_image_dataset_file(file_path="./titanic/images.tgz", image_features=["Image Path"])
print("ref_path:", ref_path)
"ref_path: s3://rime-blob-c78976e9367daf-environment/files/84f82c37-0d8f-43af-8d05-76jknwefnjkm/data/images.tgz"

When the upload_path is specified, the path contains the upload_path string in place of the random string.

img_dataset_dir = upload_local_image_dataset_file(file_path="./titanic/images.tgz", image_features=["Image Path"], upload_path="rime_tests/titanic_tests")
print("ref_path:", ref_path)
"ref_path: s3://rime-blob-c78976e9367daf-environment/files/rime_tests/titanic_tests/data/images.tgz"

Uploading Model Directories

The model directory is a directory containing the model.py file and any extras, such as pre-processing code or the pickle file, that the model.py file depends on.

Model directories can easily be uploaded using the upload_directory(file_path, upload_hidden, upload_path) method.

When the upload_path is not specified, the path contains a random string as directory name.

model_dir = rime_client.upload_directory(dir_path="./titanic/model")
print("model_dir:", model_dir)
"model_dir: s3://rime-blob-c78976e9367daf-environment/files/c83aa17d-709b-48b5-89ed-a9e4bad42f0b/model"

When the upload_path is specified, the path contains the upload_path string in place of the random string.

model_dir = client.upload_directory(dir_path="./titanic/model", upload_path="rime_tests/titanic_tests")
print("ref_path:", ref_path)
"model_dir: s3://rime-blob-c78976e9367daf-environment/files/rime_tests/titanic_tests/model"