Guardrails Providers

Guardrails Providers are external content safety services that enable you to leverage provider-native guardrail capabilities within FloTorch. These providers offer advanced AI-powered content filtering and moderation features.

Supported Guardrails Providers

Currently, FloTorch supports the following Guardrails Providers:

• AWS Bedrock - AWS Bedrock provides native guardrail capabilities with comprehensive content safety features, including detection of harmful content, PII, and custom categories with real-time processing and multi-language support.

Other Guardrails Providers are planned to be supported in the future.

Creating a Guardrails Provider

To create a Guardrails Provider:

1. Navigate to the Providers section in the FloTorch Console
2. Click on Add Provider button in the top right corner
3. Select Guardrail as the provider type
4. Choose from the available providers (e.g., Amazon Bedrock)
5. Provide a name for the provider (must be unique within your workspace)
6. Optionally provide a description
7. Configure provider-specific settings (see below)
8. Click Create to save the provider

AWS Bedrock Configuration

When creating an AWS Bedrock Guardrails Provider, you need to configure:

• Name: A descriptive name for your provider (e.g., “Production Bedrock Guardrails”)
• Description: Optional description of the provider’s purpose
• Region: AWS region where your Bedrock service is available (e.g., us-east-1, us-west-2)
• Access Key ID: Your AWS access key ID
• Secret Access Key: Your AWS secret access key

Note: Ensure that the AWS credentials have appropriate permissions to access Bedrock Guardrails in the specified region.

Once a provider is created, you can see it in the Providers section filtered by Guardrail type.

Creating Provider Guardrails

After setting up a Guardrails Provider, you can create guardrails that use the provider’s capabilities:

1. Navigate to the Guardrails section in the FloTorch Console
2. Click on New Guardrail button in the top right corner
3. Select Add Provider Guardrail from the options
4. Choose your configured Guardrails Provider from the dropdown
5. Select the specific guardrail from the provider’s available options
6. Choose the version of the guardrail (if multiple versions are available)
7. Configure the action for your use case:
   • AWS Bedrock: Supports Block, Redact, and Log (Replace is not supported)
8. Configure the severity level:
   • AWS Bedrock: Supports Low, Medium, and High (Critical is not supported)
9. Provide a name (alphanumeric with dashes only) for the guardrail
10. Optionally provide a description
11. Optionally set the status to Active or Inactive
12. Click Create to add the provider guardrail to your repository

Important: The available actions and severity levels depend on the provider’s capabilities. The UI will only show options that are supported by the selected provider.

Updating a Guardrails Provider

Guardrails Providers can be updated to modify their configuration:

1. Navigate to the Providers section in the FloTorch Console
2. Filter by type Guardrail to see your guardrail providers
3. Find the provider you want to update
4. Click on Actions dropdown (three vertical dots) and select Edit
5. Update the provider configuration (e.g., credentials, region)
6. Click Update to save changes

Note: Updating a provider’s configuration will affect all guardrails using that provider.

Updating Provider Guardrails

Provider guardrails can be updated to modify their behavior:

1. Navigate to the Guardrails section
2. Find the provider guardrail you want to update
3. Click on Actions dropdown button (three vertical dots) and select Edit
4. Update the fields you want to modify:
   • Name: Change the guardrail name
   • Description: Update the description
   • Version: Select a different version if available
   • Action: Change the action (limited to provider-supported options)
   • Severity: Change the severity level (limited to provider-supported options)
   • Status: Set to Active or Inactive
5. Click Update to save changes

Note: You cannot change the underlying guardrail type or provider once created. If you need a different guardrail or provider, create a new one.

Deleting a Guardrails Provider

Deleting a Guardrails Provider is not supported at the moment to prevent accidental removal of providers that are actively being used by guardrails.

If you need to stop using a provider:

1. First remove or disable all guardrails using that provider
2. Contact support for assistance with provider removal if needed

Deleting Provider Guardrails

Deleting provider guardrails is not supported at the moment to prevent accidental removal of guardrails that are actively being used by models.

If you need to disable a provider guardrail:

1. Edit the guardrail and set its status to Inactive
2. Or remove it from individual models where it’s not needed

Key Features of Guardrails Providers

AWS Bedrock Guardrails

Content Safety Categories:

• Hate speech and harassment detection
• Violence and harmful content filtering
• Sexual content and nudity protection
• Personally identifiable information (PII) detection
• Custom categories and rules

Supported Actions:

• Block: Prevent content from being processed or returned
• Redact: Remove or mask sensitive content while allowing the rest to proceed
• Log: Record incidents for monitoring and auditing

Supported Severity Levels:

• Low: Minor content concerns that should be monitored
• Medium: Moderate content issues that require attention
• High: Serious content violations that need immediate action

Additional Features:

• Multi-language content filtering
• Real-time processing capabilities
• Version management for testing and deployment
• Comprehensive logging and monitoring
• Integration with AWS CloudWatch for metrics and alerts

Integration with Models

Provider guardrails integrate seamlessly with FloTorch models:

1. Navigate to your model in the Model Registry
2. Go to the model version you want to configure
3. Select the Guardrails tab
4. Choose the appropriate hook (Input, Output, or Input-Output)
5. Click Add Guardrail and select your provider guardrail
6. Configure the execution order by dragging and dropping guardrails
7. Optionally override action and severity settings for this specific model (within provider-supported options)

Best Practices

• Provider Selection: Choose providers based on your specific content safety requirements and geographic needs
• Credential Security: Use AWS IAM roles with least-privilege access for Bedrock providers
• Start Conservative: Begin with higher severity levels and blocking actions, then adjust based on monitoring
• Monitor and Adjust: Review logs regularly and adjust settings as needed
• Version Control: Keep track of guardrail versions and test new versions before deploying to production
• Order Matters: Place critical guardrails first in the execution order
• Test Thoroughly: Validate detection accuracy in a development environment before production use
• Documentation: Document your guardrail configurations, purposes, and any specific requirements
• Regional Compliance: Choose AWS regions that comply with your data residency requirements
• Cost Management: Monitor usage and costs associated with provider guardrails