Endpoints

Creating a RAG Endpoint

To create a RAG Endpoint,

1. Click on New RAG Endpoint button in the top right corner of the FloTorch Console.
2. Provide a Name, Description, Vector Storage (optional) and Generating Model for the RAG Endpoint. RAG Endpoint name should be unique.
3. Once Generating Model is selected, we will automatically fetch the list of Generating Model Versions from the provider.
4. Select the Generating Model Version you want to add to the RAG Endpoint. If you don’t see the Generating Model Version you want to add, you can manually type the Generating Model Version name.
5. You can optionally add System Prompt.
6. Click on Submit button

Once a RAG Endpoint is created, you can see the RAG Endpoint in the RAG Endpoints page. You can access this RAG Endpoint using the name of the RAG Endpoint.

You should make a POST request to the following endpoint to generate a response using RAG Technique.

https://<your-flotorch-gateway-instance-url>/api/v1/rag/<your-rag-endpoint-name>

The request body should be a JSON object with the following fields:

{
    "query": "What is the capital of France?"
}

RAG endpoints can also be created directly from FloTorch Experiment Studio., When you choose to deploy an experiment, it create a RAG Endpoint and you can access it using the name of the RAG Endpoint.

Editing a RAG Endpoint

1. In the endpoints table, open the Actions (three dots) menu for the endpoint and select Edit.
2. In the edit modal, update the fields you need.
3. Click Update to save changes.

Archiving and Unarchiving a RAG Endpoint

Archiving hides a RAG endpoint from the active list but keeps it available for dependencies. You can restore it later.

Archive a RAG endpoint

1. In the left navigation, go to RAG Endpoints.
2. In the endpoints table, open the Actions (three dots) menu for the endpoint and select Archive.
3. In the confirmation dialog, review the message: “Are you sure you want to archive this RAG endpoint? You can restore it later if needed.”
4. Click Archive. The endpoint is archived and removed from the active list. Existing dependencies continue to work.
5. To view archived endpoints, open the top filters dropdown and select Archived.

Unarchive a RAG endpoint

1. In the endpoints table, use the top filters dropdown and select Archived.
2. Open the Actions (three dots) menu for the endpoint and select Unarchive.

Deleting a RAG Endpoint

Deleting a RAG endpoint permanently removes the endpoint and all related dependencies. This action cannot be undone.

1. In the left navigation, go to RAG Endpoints.
2. In the endpoints table, open the Actions (three dots) menu for the endpoint and select Delete.
3. Review the dependency list in the deletion modal.
4. Click Continue. A confirmation modal opens and asks you to type the endpoint name.
5. Enter the endpoint name and click Permanently Delete.

After confirmation, the endpoint and all listed dependencies are deleted permanently and cannot be recovered.