Guardrail Repository

FloTorch Guardrails Repository is a centralized collection of all your guardrails that you can reuse across different models and applications in your workspace.

Types of Guardrails

Custom Guardrails

Custom guardrails are created and managed entirely within FloTorch. These include:

• Keyword-based filters: Define specific words or phrases to block or filter
• Regex-based filters: Use regular expressions to detect patterns like personal information, financial data, etc.

Custom guardrails support all actions (Block, Redact, Replace, Log) and all severity levels (Low, Medium, High, Critical).

Provider Guardrails

Provider guardrails integrate with external AI service providers’ native guardrail capabilities:

• AWS Bedrock Guardrails: Leverage Amazon Bedrock’s built-in content safety and filtering features
• Future provider integrations: Support for additional providers coming soon

Provider guardrails may have limited action and severity options depending on the provider’s capabilities. For example, AWS Bedrock guardrails support Block, Redact, and Log actions, and Low, Medium, and High severity levels.

Creating a Guardrail

Creating Custom Guardrails

To create a custom guardrail:

1. Navigate to the Guardrails section in the FloTorch Console
2. Click the New Guardrail button in the top right corner
3. Choose from the available options:
   • Create Custom Guardrail: Build your own from scratch
   • Use Template: Start with a pre-built template (SSN Detection, Phone Number Detection)
4. Provide the required information:
   • Name: Unique identifier for your guardrail (alphanumeric with dashes only)
   • Description: Explain what this guardrail does (optional)
   • Type: Select Keyword or Regex
   • Action: Choose Block, Redact, Replace, or Log
   • Severity: Set Low, Medium, High, or Critical
   • Status: Set Active or Inactive (optional, defaults to Active)
   • Configuration: Provide type-specific settings:
     • For Keyword: Enter one or more keywords to match
     • For Regex: Enter one or more regex patterns to match
     • For Replace action: Provide the replacement text
5. Click Create to save your guardrail

Creating Provider Guardrails

To create a provider guardrail:

1. Navigate to the Guardrails section in the FloTorch Console
2. Click the New Guardrail button
3. Select Add Provider Guardrail
4. Choose your provider (e.g., AWS Bedrock) from the dropdown
5. Select the specific guardrail from the provider’s available options
6. Choose the version of the guardrail
7. Configure the action and severity for your use case (options will be limited based on provider capabilities)
8. Provide a name (alphanumeric with dashes only) and description
9. Optionally set the status to Active or Inactive
10. Click Create to save

Note: Before creating provider guardrails, you must first configure a Guardrails Provider in the Providers section. See the Guardrails Providers documentation for more information.

Configuring Guardrails for Models

Once created, guardrails can be applied to models at different hooks:

1. Navigate to your model in the Model Registry
2. Go to the model version you want to configure
3. Select the Guardrails tab
4. Choose the appropriate hook:
   • Input: Filter incoming requests
   • Output: Filter model responses
   • Input-Output: Filter both requests and responses
5. Click Add Guardrail and select from your repository
6. Configure the order of execution (drag and drop to reorder)
7. Optionally override the action and severity for this specific model

Guardrail Execution Order

When multiple guardrails are applied to the same hook, they execute in the order you specify. You can drag and drop guardrails to change their execution order. This is important because:

• Earlier guardrails can prevent later ones from running
• The order affects which action is taken if multiple guardrails match
• You can prioritize critical guardrails by placing them first

Updating a Guardrail

To update a guardrail:

1. Navigate to the Guardrails section
2. Find the guardrail you want to update
3. Click the Actions dropdown button (three vertical dots)
4. Select Edit option
5. Update the fields you want to modify:
   • Custom Guardrails: You can update name, description, type-specific configuration (keywords/regex), action, severity, and status
   • Provider Guardrails: You can update name, description, guardrail version, action, severity, and status
6. Click Update to save changes

Important Notes:

• You cannot change the type of a guardrail (e.g., from Keyword to Regex) after creation
• For provider guardrails, you cannot change the underlying provider or guardrail type
• Changes to guardrails will automatically apply to all models using that guardrail, unless the model has overridden the specific action or severity
• For Replace action, you must provide replacement text in the configuration

Deleting a Guardrail

Deleting guardrails is currently not supported to prevent accidental removal of guardrails that are actively being used by models. If you need to disable a guardrail:

1. Edit the guardrail and set its status to Inactive
2. Or remove it from individual models where it’s not needed

Best Practices

• Start with templates: Use pre-built templates for common use cases like SSN or phone number detection
• Use descriptive names: Use alphanumeric characters and dashes to create clear, descriptive names
• Test thoroughly: Always test your guardrails in a development environment before deploying to production
• Monitor logs: Use the Log action to understand what content is being filtered before enabling blocking
• Order matters: Place critical guardrails (like blocking malicious content) before less critical ones
• Document your rules: Provide clear descriptions for your guardrails to help team members understand their purpose
• Regular review: Periodically review and update your guardrails based on new threats and requirements
• Use status wisely: Set guardrails to Inactive instead of deleting them to preserve configuration history