Deploy from AWS ECR

Learn how to deploy Serverless workers from Amazon Elastic Container Registry (ECR) with automated credential management.

This guide shows how to deploy Serverless workers from Amazon Elastic Container Registry (ECR). ECR integration requires additional setup compared to other container registries like Docker Hub or GitHub, because ECR credentials expire every 12 hours by default. The means you’ll need to set up automated credential refreshing to ensure your Serverless endpoints can continuously access your private ECR repositories.

Because of the additional setup required, we recommend deploying workers with Docker Hub or GitHub Container Registry if ECR-specific features aren’t required.

Requirements

Before integrating ECR with Runpod, make sure that you have:

• An AWS account with ECR access.
• The AWS CLI configured on your local machine with appropriate permissions.
• A Runpod account with API access.
• A local folder containing all the necessary components to build a worker image: a Dockerfile, handler function, and requirements.txt file.

If you want to test this workflow but haven’t yet created the necessary files, you can download this basic worker template.

Step 1: Create an ECR repository and push your container image

First, create an ECR repository and push your container image.

# Create ECR repository
aws ecr create-repository --repository-name my-serverless-worker --region us-east-1

# Get login token and authenticate Docker
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com

# Build and tag your image; run these commands in the same directory as your Dockerfile
docker build --platform linux/amd64 -t my-serverless-worker .
docker tag my-serverless-worker:latest AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/my-serverless-worker:latest

# Push to ECR
docker push AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/my-serverless-worker:latest

Replace AWS_ACCOUNT_ID with your actual AWS account ID number and adjust the region as needed.

Step 2: Create initial container registry credentials

Generate ECR credentials and add them to Runpod:

# Get ECR authorization token
aws ecr get-authorization-token --region us-east-1 --query 'authorizationData[0].authorizationToken' --output text | base64 -d

This returns credentials in the format AWS:password. Use these to create container registry authentication in Runpod:

REST API

curl -X POST "https://rest.runpod.io/v1/containerregistryauth" \
  -H "Authorization: Bearer YOUR_RUNPOD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "ECR Credentials",
    "username": "AWS",
    "password": "YOUR_ECR_TOKEN_PASSWORD"
  }'

Save the returned id value - you’ll need it for the automation script and endpoint configuration.

Step 3: Set up automated credential refresh

Create an AWS Lambda function to automatically refresh ECR credentials in Runpod:

Lambda function code

import json
import boto3
from botocore.vendored import requests
import base64
import os

def lambda_handler(event, context):
    # Initialize AWS clients
    ecr_client = boto3.client('ecr')
    
    # Runpod configuration
    runpod_api_key = os.environ['RUNPOD_API_KEY']
    registry_auth_id = os.environ['REGISTRY_AUTH_ID']
    
    try:
        # Get new ECR authorization token
        response = ecr_client.get_authorization_token()
        auth_data = response['authorizationData'][0]
        
        # Decode the authorization token
        token = base64.b64decode(auth_data['authorizationToken']).decode('utf-8')
        username, password = token.split(':', 1)
        
        # Update Runpod container registry credentials
        update_response = requests.patch(
            f"https://rest.runpod.io/v1/containerregistryauth/{registry_auth_id}",
            headers={
                "Authorization": f"Bearer {runpod_api_key}",
                "Content-Type": "application/json"
            },
            json={
                "username": username,
                "password": password
            }
        )
        
        if update_response.status_code == 200:
            print("Successfully updated Runpod ECR credentials")
            return {
                'statusCode': 200,
                'body': json.dumps('ECR credentials updated successfully')
            }
        else:
            print(f"Failed to update credentials: {update_response.text}")
            return {
                'statusCode': 500,
                'body': json.dumps('Failed to update credentials')
            }
            
    except Exception as e:
        print(f"Error: {str(e)}")
        return {
            'statusCode': 500,
            'body': json.dumps(f'Error: {str(e)}')
        }

Lambda configuration

1. Create a new Lambda function in the AWS Console
2. Set the runtime to Python 3.9 or later.
3. Add the following environment variables:
   • RUNPOD_API_KEY: Your Runpod API key.
   • REGISTRY_AUTH_ID: The container registry auth ID from Step 2.
4. Attach an IAM role with the AmazonEC2ContainerRegistryReadOnly policy.
5. Set up an EventBridge (CloudWatch Events) rule to trigger the function every 6 hours:

{
  "Rules": [
    {
      "Name": "ECRCredentialRefresh",
      "ScheduleExpression": "rate(6 hours)",
      "State": "ENABLED",
      "Targets": [
        {
          "Id": "1",
          "Arn": "arn:aws:lambda:us-east-1:AWS_ACCOUNT_ID:function:refresh-ecr-credentials"
        }
      ]
    }
  ]
}

Step 4: Deploy Serverless endpoint with ECR image

Once credential automation is set up, you can create your Serverless endpoint using the Runpod console or the REST API.

Runpod Console

Follow these steps to create your Serverless endpoint in the Runpod console:

1. Navigate to the Serverless section.
2. Click New Endpoint.
3. Select Docker Image as your source.
4. Enter your ECR image URL: 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-serverless-worker:latest.
5. In the Advanced section, select your ECR credentials from the Container Registry Auth dropdown.
6. Configure other endpoint settings as needed.
7. Click Create Endpoint.

Step 5: Monitor credential refresh

Check your Lambda function logs in CloudWatch to ensure credentials are being refreshed successfully:

aws logs filter-log-events \
  --log-group-name /aws/lambda/refresh-ecr-credentials \
  --start-time $(date -d '1 hour ago' +%s)000

Common issues and solutions

Image pull failures: If your endpoint fails to pull the ECR image:

1. Verify the ECR image URL is correct
2. Check that the Lambda function is running successfully
3. Ensure the container registry auth ID is properly configured
4. Confirm your ECR repository permissions allow the necessary access

Credential expiration: If you see authentication errors:

1. Manually trigger the Lambda function to refresh credentials immediately
2. Check that the CloudWatch Events rule is properly configured
3. Verify the Lambda function has the correct IAM permissions

Performance considerations: ECR image pulls may be slower than Docker Hub:

1. Consider using smaller base images to reduce pull times
2. Enable FlashBoot in your endpoint configuration for faster cold starts
3. Use network volumes to cache frequently accessed data

Best practices

• Monitor Lambda execution: Set up CloudWatch alarms for Lambda function failures.
• Use specific image tags: Avoid using :latest tags in production deployments.
• Implement retry logic: Add error handling and retry mechanisms to your Lambda function.
• Regional considerations: Deploy Lambda functions in the same region as your ECR repositories.
• Security: Use least-privilege IAM policies and rotate Runpod API keys regularly.

Alternative approaches

If automated credential refresh adds complexity to your workflow, consider these alternatives:

• Docker Hub: Simpler authentication with longer-lived credentials.
• GitHub Container Registry: Integrated with GitHub workflows and repositories.
• Public ECR: Use Amazon’s public ECR for open-source projects without authentication.

Next steps

Now that you’ve integrated ECR with Runpod, you can:

• Learn how to configure your endpoint.
• Set up monitoring and logging for your endpoints.