Capability 1. Providing developers and data scientists with secure access to generative AI FMs (model inference)
The following architecture diagram illustrates the AWS services recommended for the Generative AI account for this capability. The scope of this capability is to give users access to foundation models (FMs) for chat and image generation.
The Generative AI account is dedicated to securing generative AI functionality through the use of Amazon Bedrock. We will build out this account (and the architecture diagram) with functionality throughout this guide. The account includes services for storing conversations for users and maintaining a prompt store. The account also includes security services to implement security guardrails and centralized security governance. Users can gain federated access by using an identity provider (IdP) to securely access a virtual private cloud (VPC) in the Generative AI account. AWS PrivateLink supports private connectivity from your VPC to Amazon Bedrock endpoint services. You should create an Amazon S3 gateway endpoint for the model invocation logs and prompt store bucket in Amazon S3 that the VPC environment is configured to access. You should also create an Amazon CloudWatch Logs gateway endpoint for the CloudWatch logs that the VPC environment is configured to access.
Rationale
Granting users access to generative AI FMs enables them to use advanced models
for tasks such as natural language processing, image generation, and enhancing
efficiency and decision making. This access fosters innovation within an organization
because employees can experiment with new applications and develop cutting-edge
solutions, which ultimately improves productivity and provides competitive advantages.
This use case corresponds to Scope 3 of the Generative AI Security Scoping Matrix
When you give users access to the generative AI FMs in Amazon Bedrock, you should address these key security considerations:
-
Secure access to the model invocation, conversation history, and prompt store
-
Encryption of conversations and the prompt store
-
Monitoring for potential security risks such as prompt injection or sensitive information disclosure
The next section discusses these security considerations and generative AI functionality.
Security considerations
Generative AI workloads face unique risks, including prompt injection attacks during
model inference. Threat actors could craft malicious queries that force continuous
output, leading to excessive resource consumption, or craft prompts that result in
inappropriate model responses. Additionally, end users might inadvertently misuse these
systems by inputting sensitive information in prompts. Amazon Bedrock offers robust
security controls for data protection, access control, network security, logging and
monitoring and input/output validation that can help mitigate these risks. These are
discussed in the following sections. For more information about the risks associated
with generative AI workloads, see OWASP Top 10 for Large Language Model Applications
Remediations
Identity and access management
Do not use IAM users because they have long-term credentials such as usernames and passwords. Instead, use temporary credentials when accessing AWS. You can use an identity provider (IdP) for your human users to provide federated access to AWS accounts by assuming IAM roles, which provide temporary credentials.
For centralized access management, use AWS IAM Identity Center. To learn more about IAM Identity Center and various architectural patterns, see the IAM deep dive section of this guide.
To access Amazon Bedrock, you must have a minimum set of permissions. Access to Amazon Bedrock FMs isn't granted by default. To gain access to an FM, an IAM identity with sufficient permissions has to request access through the Amazon Bedrock console. For information about how you can add, remove, and control model access permissions, see Model access in the Amazon Bedrock documentation.
To securely provide access to Amazon Bedrock, customize the Amazon Bedrock policy examples according to your requirements to ensure that only the required permissions are allowed.
Network security
AWS PrivateLink enables you to connect to some AWS services, services hosted by other AWS accounts (referred to as endpoint services), and supported AWS Marketplace partner services, by using private IP addresses in your VPC. The interface endpoints are created directly inside your VPC by using elastic network interfaces and IP addresses in your VPC's subnets. This approach uses Amazon VPC security groups to manage access to the endpoints. Use AWS PrivateLink to establish private connectivity from your VPC to Amazon Bedrock endpoint services without exposing your traffic to the internet. PrivateLink gives you private connectivity to the API endpoint in the Amazon Bedrock service account, so instances in your VPC don't need public IP addresses to access Amazon Bedrock.
Logging and monitoring
Enable model invocation logging. Use model invocation logging to collect invocation logs, model input data, and model output data for all Amazon Bedrock model invocations in your AWS account. By default, logging is disabled. You can enable invocation logging to collect the full request data, response data, IAM invocation role, and metadata associated with all calls that are performed in your account.
Important
You maintain full ownership and control over your invocation logging data and can use IAM policies and encryption to ensure that only authorized personnel can access it. Neither AWS nor the model providers have visibility or access to your data.
Configure logging to provide the destination resources where the log data will be published. Amazon Bedrock provides native support for destinations such as Amazon CloudWatch Logs and Amazon Simple Storage Service (Amazon S3). We recommend that you configure both sources to store model invocation logs.
Implement automated abuse detection mechanisms to help prevent potential misuse, including prompt injection or sensitive information disclosure. Configure alerts to notify administrators when potential misuse has been detected. This can be achieved through custom CloudWatch metrics and alarms based on CloudWatch metrics.
Monitor Amazon Bedrock API activities by using AWS CloudTrail. Consider saving and managing commonly used prompts in a prompt store for your end users. We recommend that you use Amazon S3 for the prompt store.
Design consideration
You must evaluate this approach against your compliance and privacy requirements. Model invocation logs may collect sensitive data as part of model input and model putput, which might not be appropriate for your use case, and, in some cases, might not meet the risk compliance objectives you have.
Input and output validation
If you want to implement Guardrails for Amazon Bedrock for your users who interact with Amazon Bedrock models, you will need to deploy your guardrail to production and invoke the version of the guardrail in your application. This would require creating and securing a workload that interfaces with the Amazon Bedrock API.
Recommended AWS services
Note
The AWS services discussed in this section and for other capabilities are specific to the use cases that are discussed in these sections. In addition, you should have a set of common security services such as AWS Security Hub, Amazon GuardDuty, AWS Config, IAM Access Analyzer, and AWS CloudTrail organization trail in all AWS accounts to enable consistent guardrails and provide centralized monitoring, management, and governance across your organization. See the section Deploying common security services within all AWS accounts earlier in this guide to understand the functionality and architectural best practices for these services.
Amazon S3
Amazon S3 is an object storage service that offers scalability, data availability, security, and performance. For recommended security best practices, see the Amazon S3 documentation, online tech talks, and deeper dives in blog posts.
Host your model invocation logs and commonly used prompts as
a prompt store
Use versioning for backups and apply object-level immutability with Amazon S3 Object Lock
Amazon CloudWatch
Amazon CloudWatch monitors applications, responds to performance changes, optimizes resource use, and provides insights into operational health. By collecting data across AWS resources, CloudWatch gives you visibility into system-wide performance and lets you set alarms, automatically react to changes, and gain a unified view of operational health.
Use CloudWatch to monitor and generate alarms on system events that describe changes in Amazon Bedrock and Amazon S3. Configure alerts to notify administrators when prompts might indicate prompt injection or sensitive information disclosure. This can be achieved through custom CloudWatch metrics and alarms based on log patterns. Encrypt log data in CloudWatch Logs with a customer managed key that you create, own, and manage. For additional network security hardening, you can create a gateway endpoint for CloudWatch Logs that the VPC environment is configured to access. You can centralize monitoring by using Amazon CloudWatch Observability Access Manager in the Security OU Security Tooling account. Manage access permissions to your CloudWatch Logs resources by using the principle of least privilege.
AWS CloudTrail
AWS CloudTrail supports the governance, compliance, and auditing of activity in your AWS account. With CloudTrail, you can log, continuously monitor, and retain account activity related to actions across your AWS infrastructure.
Use CloudTrail to log and monitor all create, read, update, and delete (CRUD) actions to Amazon Bedrock and Amazon S3. For more information, see Log Amazon Bedrock API calls using AWS CloudTrail in the Amazon Bedrock documentation and Logging Amazon S3 API calls using AWS CloudTrail in the Amazon S3 documentation.
CloudTrail logs from Amazon Bedrock do not include prompt and completion information. We recommend that you use an organization trail that logs all events for all accounts in your organization. Forward all the CloudTrail logs from the Generative AI account to the Security OU Log Archive account. With centralized logs in place, you can monitor, audit, and generate alerts on Amazon S3 object access, unauthorized activity by identities, IAM policy changes, and other critical activities performed on sensitive resources. For more information, see security best practices in AWS CloudTrail.
Amazon Macie
Amazon Macie is a fully managed data security and data privacy service that uses machine learning and pattern matching to discover and help protect your sensitive data in AWS. You need to identify the type and classification of data your workload is processing to ensure that appropriate controls are enforced. Macie can help identify sensitive data in your prompt store and model invocation logs stored in S3 buckets. You can use Macie to automate discovery, logging, and reporting of sensitive data in Amazon S3. You can do this in two ways: by configuring Macie to perform automated sensitive data discovery, and by creating and running sensitive data discovery jobs. For more information, see Discovering sensitive data with Amazon Macie in the Macie documentation.