How to Copy Multiple Files to AWS S3 using AWS CLI

This article shows how to upload files to the AWS S3 bucket using the AWS s3 CLI cp command, which is especially useful when you copy multiple files and folders to S3 in an automatic way. You can also set the task to be scheduled to run at a certain time of day.

Install AWS Cli Package if not done

Download the AWS CLI installer and install it on your PC.

The default location is C:\Program Files\Amazon\AWSCLI, and you can confirm installation with –version check.

Store Default Credential Profile for AWS CLI

You need at least a default credential stored on your PC when using CLI. At the command prompt, run “aws configure list” and set the default profile if nothing is retrieved by using AWS configure.

Upload to S3 bucket

Example 1: Copying one file to S3 bucket

aws s3 cp filename s3://s3bucket_with_targetfolder

Using AWS Explorer, I could verify the copy was done as expected.

Example 2: Copying a folder and subfolders to S3 bucket

aws s3 cp local_foldername s3://s3bucket_with_targetfolder --recursive

If the target folder does not exist on the S3 bucket, it will be created.

AWS Explorer in Visual Studio showed that uploading to the s3 bucket was completed as expected.


Example 3: Excluding certain folders when copying to S3 bucket

aws s3 cp local_foldername s3://s3bucket_with_targetfolder --recursive --exclude "folder-to-exclude/*"

I wanted to exclude /bin and /obj folders from uploaded to S3.

AWS Explorer in Visual Studio showed that the \bin\ and \obj\ folders were not copied.

 

Leave a Comment