Integrate AWS Transfer for SFTP With A Custom Identity Provider

Get a step-by-step guide on integrating AWS Transfer for SFTP with custom identity providers, improving data security and access control for business needs.

  • date icon

    Sunday, Feb 24, 2019

Integrate AWS Transfer for SFTP With A Custom Identity Provider

AWS Transfers for SFTP is a fully managed service that allows to easily upload/download data to/from AWS S3 using the SFTP protocol. In the previous blog post, we created a managed SFTP endpoint using the public key authentication. Sometimes, a username/password authentication may be required, e.g. if the access should be provided to existing users, e.g. from the AWS Cognito User Pool. In this blog post we are giving step-by-step instructions on how to implement a custom authentication for AWS Transfers for SFTP.

Check out our new blog post about AWS SFTP custom identity provider for Active Directory. It also contains a very handy CloudFormation template that can be fully customized for your needs!

Deployment

We’ll be using the Serverless framework to create corresponding infrastructure. We assume the AWS Cognito Userpool already exists to simulate a real-world scenario. The resulting CloudFormation stack contains:

  • API Gateway
  • AWS Lambda function which validates username/password supplied to the SFTP endpoint
  • A custom resource for the AWS Transfers for SFTP since at the moment the blog post was written, it was not present in the CloudFormation.

Here is a diagram with the resulting infrastructure:

Custom resource definition is provided below:

SftpServer: Type: “Custom::SFTPServer” Version: “1.0” DependsOn: - CustomSftpServerManagerLambdaFunction - TransferIdentityProviderRole - TransferLoggingRole - ApiGatewayRestApi - SftpBucket Properties: ServiceToken: Fn::GetAtt: [ “CustomSftpServerManagerLambdaFunction”, “Arn” ] InvocationRole: Fn::GetAtt: [ “TransferIdentityProviderRole”, “Arn” ] LoggingRole: Fn::GetAtt: [ “TransferLoggingRole”, “Arn” ] AuthorizeUrl: Fn::Join: - "" - - https:// - Ref: ApiGatewayRestApi - .execute-api. - Ref: ‘AWS::Region’ - .amazonaws.com/dev/

IAM Roles and Permissions

The Lambda execution role is rather standard:

LambdaExecutionRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - lambda.amazonaws.com Action: “sts:AssumeRole” ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

For our SFTP endpoint we need to create two roles: TransferIdentityProviderRole(for API Gateway invocation) and TransferLoggingRole(for logging):

TransferIdentityProviderRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - transfer.amazonaws.com Action: sts:AssumeRole Policies: - PolicyName: TransferCanInvokeAuthorizeApi PolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Action: - execute-api:Invoke Resource: Fn::Join: - ”:” - - “arn” - Ref: ‘AWS::Partition’ - execute-api - Ref: ‘AWS::Region’ - Ref: ‘AWS::AccountId’ - Fn::Join: - "" - - Ref: ApiGatewayRestApi - /${self:provider.stage}/GET/* - PolicyName: TransferCanReadAuthorizeApi PolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Action: - apigateway:GET Resource: ”*” TransferLoggingRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - transfer.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSTransferLoggingAccess

Another required role is the common SFTP user role. ARN of this role in the response from Lambda will indicate a successful authorization:

SftpUserRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - transfer.amazonaws.com Action: “sts:AssumeRole” Policies: - PolicyName: SftpBucketAccessPolicy PolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Action: - s3:* Resource: Fn::Join: - ”:” - - “arn” - Ref: ‘AWS::Partition’ - “s3::” - ${env:TRANSFER_BUCKET_NAME}*

Additionally, if you would like to use custom “SFTPServer” resource, you should define a role that allows managing the AWS Transfer for SFTP and also write logs to CloudWatch:

LambdaSftpManageRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - lambda.amazonaws.com Action: “sts:AssumeRole” Policies: - PolicyName: LambdaSftpManageRole PolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Action: - transfer:CreateServer - transfer:UpdateServer - transfer:DeleteServer - logs:CreateLogGroup - logs:CreateLogStream - logs:PutLogEvents - iam:PassRole - apigateway:GET Resource: ”*”

IMPORTANT: iam:PassRole and apigateway:GET permissions are required, otherwise you will get an error if you try to create an SFTP Server.

Lambdas declaration

First, lets take a look at “SftpAuthorizer” lambda configuration:

SftpAuthorizer: handler: src/handler.authorize role: LambdaExecutionRole environment: COGNITO_USER_POOL_ID: ${env:COGNITO_USER_POOL_ID} COGNITO_CLIENT_ID: ${env:COGNITO_CLIENT_ID} SERVER_ID: { “Fn::GetAtt”: [“SftpServer”, “ServerId” ] } BUCKET_ARN: { “Fn::GetAtt”: [“SftpBucket”, “Arn” ] } ROLE_ARN: { “Fn::GetAtt”: [“SftpUserRole”, “Arn” ] } events: - http: path: /servers/{serverId}/users/{user}/config method: GET authorizer: aws_iam

This lambda is being triggered by a request to AWS API Gateway and is used for authorization event handling. You should provide the following environment variables:

  • COGNITOUSER_POOL_ID and COGNITO_CLIENT_ID – AWS Cognito IDs
  • ROLE_ARN – an ARN of a common role for your SFTP users
  • SERVER_ID – a server id of our SFTP server (we will get it from the custom resource)
  • BUCKET_ARN – an ARN of an S3 bucket which you would like to serve to your users

IMPORTANT NOTE: Please note that you must specify /servers/{serverId}/users/{user}/config URL because it’s a path which AWS Transfers for SFTP uses by convention.

Also, you need to define the “SFTPServer” resource management lambda:

CustomSftpServerManager: handler: src/handler.manageSftpServer role: LambdaSftpManageRole timeout: 180

Lambdas implementation

Now when we are done with the configuration, the last step is creating handlers for configured lambdas. First, let’s create the “authorize” function. This function will authorize users in the AWS Cognito using amazon-cognito-identity-js library. If the user was successfully authorized it should return the user’s role ARN. Optionally, you can also provide Scope-Down Policy which allows restricting access for users based on HomeBucket and HomeDirectory variables, e.g. to define directories that will be available to the SFTP user. For this blog post, we created a policy which allows access only for bucket prefix that equals a username. If te user is unauthorized, the function should return a response with an empty body. An example of an authorized response code:

const response = { headers: { “Access-Control-Allow-Origin”: "", “Content-Type”: “application/json” }, body: JSON.stringify({ Role: process.env.ROLE_ARN, Policy: { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowListingOfUserFolder", "Action": [ "s3:ListBucket" ], "Effect": "Allow", "Resource": [ + ‘“arn:aws:s3:::${transfer:HomeBucket}”’ + ], "Condition": { "StringLike": { "s3:prefix": [ + ’”${transfer:UserName}/”,’ + ’”${transfer:UserName}”’ + ] } } }, { "Sid": "AWSTransferRequirements", "Effect": "Allow", "Action": [ "s3:ListAllMyBuckets", "s3:GetBucketLocation" ], "Resource": "*" }, { "Sid": "HomeDirObjectAccess", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObjectVersion", "s3:DeleteObject", "s3:GetObjectVersion" ], + ‘“Resource”: “arn:aws:s3:::${transfer:HomeDirectory}*”’ + } ] },
HomeDirectory: /${process.env.BUCKET_ARN.substring("arn:aws:s3:::".length)}/${username}/, HomeBucket: process.env.BUCKET_ARN.substring(“arn:aws:s3:::“.length) }), statusCode: 200 };

IMPORTANT NOTE: Policy parameter must be a string, it doesn’t work with an object type.

The second “manageSftpServer” handler is used for custom SFTPServer resource management. It handles create, update and delete events. There is also a utility “sendResponse” function which sends a response from the custom resource to the CloudFormation.

Summary

As you may noticed, AWS Transfer for SFTP with a custom identity provider configuration is not just “plug-and-play”, but it’s very flexible and can work with any identity provider.

Additionally, you can specify more conditions for the authorization, for example, grant access only for particular groups or for users, that have certain attribute values.

If you don’t want to do all of this configuration by yourself, you can download our project from the GitHub using the link below, provide some environment variables and simply deploy your SFTP server with the “sls deploy” command.

Useful Links

Blog

Read More Posts

Your Trusted Partner in Custom Software Development.

The Top Barriers to IoT Success and How to Address Them
date icon

Wednesday, Dec 11, 2024

The Top Barriers to IoT Success and How to Address Them

IoT (Internet of Things) has disrupted industries with its advanced connectivity and data-driven insights. However, wit...

Read More
Serverless computing and EDoS attacks
date icon

Tuesday, Nov 19, 2024

Serverless computing and EDoS attacks

Hello everyone. I'm pretty sure while reading the title you had only one question in mind "What's the heck EDoS attack?...

Read More
Finding the Perfect Software Solution without an In-House Development Team: Partnering with AgileVision
date icon

Sunday, Nov 10, 2024

Finding the Perfect Software Solution without an In-House Development Team: Partnering with AgileVision

Creating Custom Software for Businesses For many businesses custom software can help in many areas such as streamli...

Read More
cta-image

Unsure of How We Can Help?

Before committing, we can complete a detailed audit to assess your current systems and identify where we can add value. Whether it’s custom software, cloud migration, or API integration, our audit provides clarity on how we can best support your business and ensure our solutions align with your goals

Book a discovery call