AWS Transfers for SFTP is a fully managed service that allows to easily upload/download data to/from AWS S3 using the SFTP protocol. In the previous blog post, we created a managed SFTP endpoint using the public key authentication. Sometimes, a username/password authentication may be required, e.g. if the access should be provided to existing users, e.g. from the AWS Cognito User Pool. In this blog post we are giving step-by-step instructions on how to implement a custom authentication for AWS Transfers for SFTP.
Check out our new blog post about AWS SFTP custom identity provider for Active Directory. It also contains a very handy CloudFormation template that can be fully customized for your needs!
Deployment
We’ll be using the Serverless framework to create corresponding infrastructure. We assume the AWS Cognito Userpool already exists to simulate a real-world scenario. The resulting CloudFormation stack contains:
- API Gateway
- AWS Lambda function which validates username/password supplied to the SFTP endpoint
- A custom resource for the AWS Transfers for SFTP since at the moment the blog post was written, it was not present in the CloudFormation.
Here is a diagram with the resulting infrastructure:
Custom resource definition is provided below:
SftpServer: Type: “Custom::SFTPServer” Version: “1.0” DependsOn: - CustomSftpServerManagerLambdaFunction - TransferIdentityProviderRole - TransferLoggingRole - ApiGatewayRestApi - SftpBucket Properties: ServiceToken: Fn::GetAtt: [ “CustomSftpServerManagerLambdaFunction”, “Arn” ] InvocationRole: Fn::GetAtt: [ “TransferIdentityProviderRole”, “Arn” ] LoggingRole: Fn::GetAtt: [ “TransferLoggingRole”, “Arn” ] AuthorizeUrl: Fn::Join: - "" - - https:// - Ref: ApiGatewayRestApi - .execute-api. - Ref: ‘AWS::Region’ - .amazonaws.com/dev/
IAM Roles and Permissions
The Lambda execution role is rather standard:
LambdaExecutionRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - lambda.amazonaws.com Action: “sts:AssumeRole” ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
For our SFTP endpoint we need to create two roles: TransferIdentityProviderRole(for API Gateway invocation) and TransferLoggingRole(for logging):
TransferIdentityProviderRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - transfer.amazonaws.com Action: sts:AssumeRole Policies: - PolicyName: TransferCanInvokeAuthorizeApi PolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Action: - execute-api:Invoke Resource: Fn::Join: - ”:” - - “arn” - Ref: ‘AWS::Partition’ - execute-api - Ref: ‘AWS::Region’ - Ref: ‘AWS::AccountId’ - Fn::Join: - "" - - Ref: ApiGatewayRestApi - /${self:provider.stage}/GET/* - PolicyName: TransferCanReadAuthorizeApi PolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Action: - apigateway:GET Resource: ”*” TransferLoggingRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - transfer.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSTransferLoggingAccess
Another required role is the common SFTP user role. ARN of this role in the response from Lambda will indicate a successful authorization:
SftpUserRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - transfer.amazonaws.com Action: “sts:AssumeRole” Policies: - PolicyName: SftpBucketAccessPolicy PolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Action: - s3:* Resource: Fn::Join: - ”:” - - “arn” - Ref: ‘AWS::Partition’ - “s3::” - ${env:TRANSFER_BUCKET_NAME}*
Additionally, if you would like to use custom “SFTPServer” resource, you should define a role that allows managing the AWS Transfer for SFTP and also write logs to CloudWatch:
LambdaSftpManageRole: Type: AWS::IAM::Role Properties: Path: /sftp/ AssumeRolePolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Principal: Service: - lambda.amazonaws.com Action: “sts:AssumeRole” Policies: - PolicyName: LambdaSftpManageRole PolicyDocument: Version: ‘2012-10-17’ Statement: - Effect: Allow Action: - transfer:CreateServer - transfer:UpdateServer - transfer:DeleteServer - logs:CreateLogGroup - logs:CreateLogStream - logs:PutLogEvents - iam:PassRole - apigateway:GET Resource: ”*”
IMPORTANT: iam:PassRole and apigateway:GET permissions are required, otherwise you will get an error if you try to create an SFTP Server.
Lambdas declaration
First, lets take a look at “SftpAuthorizer” lambda configuration:
SftpAuthorizer: handler: src/handler.authorize role: LambdaExecutionRole environment: COGNITO_USER_POOL_ID: ${env:COGNITO_USER_POOL_ID} COGNITO_CLIENT_ID: ${env:COGNITO_CLIENT_ID} SERVER_ID: { “Fn::GetAtt”: [“SftpServer”, “ServerId” ] } BUCKET_ARN: { “Fn::GetAtt”: [“SftpBucket”, “Arn” ] } ROLE_ARN: { “Fn::GetAtt”: [“SftpUserRole”, “Arn” ] } events: - http: path: /servers/{serverId}/users/{user}/config method: GET authorizer: aws_iam
This lambda is being triggered by a request to AWS API Gateway and is used for authorization event handling. You should provide the following environment variables:
- COGNITOUSER_POOL_ID and COGNITO_CLIENT_ID – AWS Cognito IDs
- ROLE_ARN – an ARN of a common role for your SFTP users
- SERVER_ID – a server id of our SFTP server (we will get it from the custom resource)
- BUCKET_ARN – an ARN of an S3 bucket which you would like to serve to your users
IMPORTANT NOTE: Please note that you must specify /servers/{serverId}/users/{user}/config URL because it’s a path which AWS Transfers for SFTP uses by convention.
Also, you need to define the “SFTPServer” resource management lambda:
CustomSftpServerManager: handler: src/handler.manageSftpServer role: LambdaSftpManageRole timeout: 180
Lambdas implementation
Now when we are done with the configuration, the last step is creating handlers for configured lambdas. First, let’s create the “authorize” function. This function will authorize users in the AWS Cognito using amazon-cognito-identity-js library. If the user was successfully authorized it should return the user’s role ARN. Optionally, you can also provide Scope-Down Policy which allows restricting access for users based on HomeBucket and HomeDirectory variables, e.g. to define directories that will be available to the SFTP user. For this blog post, we created a policy which allows access only for bucket prefix that equals a username. If te user is unauthorized, the function should return a response with an empty body. An example of an authorized response code:
const response = { headers: { “Access-Control-Allow-Origin”: "", “Content-Type”: “application/json” }, body: JSON.stringify({ Role: process.env.ROLE_ARN, Policy:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowListingOfUserFolder", "Action": [ "s3:ListBucket" ], "Effect": "Allow", "Resource": [+ ‘“arn:aws:s3:::${transfer:HomeBucket}”’ +], "Condition": { "StringLike": { "s3:prefix": [+ ’”${transfer:UserName}/”,’ + ’”${transfer:UserName}”’ +] } } }, { "Sid": "AWSTransferRequirements", "Effect": "Allow", "Action": [ "s3:ListAllMyBuckets", "s3:GetBucketLocation" ], "Resource": "*" }, { "Sid": "HomeDirObjectAccess", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObjectVersion", "s3:DeleteObject", "s3:GetObjectVersion" ],+ ‘“Resource”: “arn:aws:s3:::${transfer:HomeDirectory}*”’ +} ] },
HomeDirectory:/${process.env.BUCKET_ARN.substring("arn:aws:s3:::".length)}/${username}/, HomeBucket: process.env.BUCKET_ARN.substring(“arn:aws:s3:::“.length) }), statusCode: 200 };
IMPORTANT NOTE: Policy parameter must be a string, it doesn’t work with an object type.
The second “manageSftpServer” handler is used for custom SFTPServer resource management. It handles create, update and delete events. There is also a utility “sendResponse” function which sends a response from the custom resource to the CloudFormation.
Summary
As you may noticed, AWS Transfer for SFTP with a custom identity provider configuration is not just “plug-and-play”, but it’s very flexible and can work with any identity provider.
Additionally, you can specify more conditions for the authorization, for example, grant access only for particular groups or for users, that have certain attribute values.
If you don’t want to do all of this configuration by yourself, you can download our project from the GitHub using the link below, provide some environment variables and simply deploy your SFTP server with the “sls deploy” command.
