Manage operations on S3 Bucket using aws-cli from the EC2 instance

We can create and manage the S3 bucket using the AWS console. Sometimes there can be a need where we want to create and perform operations on the S3 Bucket from the command line. If you are not aware of the S3 bucket and would like to learn about it click here where you can find an article about the S3 bucket written by me.

Before we proceed with this article I assume you are familiar with the S3 bucket.

In this article, we will see the steps and commands to create an S3 bucket from the command line and perform basic operations on it like copy files, sync files,  etc.


  1. AWS Account  (Create if you don’t have one).
  2. Basic understanding of S3 Bucket (Click here to learn more about S3 Buckets).
  3. Ubuntu 20.04 EC2 Instance(Click here to learn about the EC2 instance and the steps to create it from the AWS Console. You can choose an AMI of Ubuntu 20.04 instead of 18.04).
  4. IAM Role with S3FullAccess Policy attached to it (Click here to learn to create an IAM Role).

What will we do?

  1. Login to AWS Account.
  2. Install aws-cli.
  3. Attach an IAM Role to the EC2 instance.
  4. Create a Bucket using aws-cli.
  5. Perform the basic operation on S3 Bucket using aws-cli.

Login to AWS

Before we proceed with the installation of the aws-cli command on an EC2 instance, let's first login to our account.

Click here to go to AWS Login Page.

Upon hitting the above link you will see the login page as follows.

Login Page

Once you log in successfully into your found you will see the main AWS dashboard as below.

AWS dashboard

Install aws-cli

Connect to your AWS EC2 Instance. 

Let's update the repository first.

sudo apt-get update

Install aws-cli command using the following command on the Ubuntu server.

sudo apt install awscli

Take the version of the aws-cli command.

aws --version

Till this point, we just have an EC2 instance without any required IAM Role attached to it.

Now if we try to check the identity using the following command we will be asked to configure credentials to be used to perform any operation.

aws sts get-caller-identity

Check aws command and access to the AWS Account

Attach an IAM Role to the EC2 instance.

Go to the AM console in your AWS account and check if the rule has the required policy attached to it.

Here you can see the role has AmazonS3FullAccess policy attached to it.

IAM Role with at least full access to S3 bucket

Now, go to the EC2 to console and select the instance which you are using to perform operations on the S3 bucket. 

Here, click on Actions --> Security --> Modify IAM role to attach a rule to the instance.

Attach an IAM Role to EC2 instance

You will see your screen where you can select a rule to be attached from the drop-down list. Click on save to proceed further.

Select and attach an IAM Role to EC2 instance

Perform the basic operation on S3 Bucket using aws-cli

Let's go back to the EC2 terminal and execute the following command.

aws sts get-caller-identity

Verify your identity

Since we have attached the required rule to the instance, upon executing the above command to check the identity of the role being used to perform operations from the terminal, we can see the User ID, Account and its ARN in response. This means we have successfully authenticated our EC2 instance. Now we are ready to perform operations from the terminal using the role attached to the EC2 instance.

To check the existing S3  Buckets in the AWS Account, execute the following command

aws s3 ls

Let's create a new bucket named "rahul-new-bucket-dec-2020". Make sure the bucket name must be globally unique.

aws s3 mb s3://rahul-new-bucket-dec-2020

Now, again if we list the buckets we can see a new bucket has been successfully created and is available under the list of existing buckets.

aws s3 ls

Let's create a new file on our local machine

touch file-for-s3

We can copy files from our local machine to the S3 bucket

aws s3 cp file-for-s3  s3://rahul-new-bucket-dec-2020

We can even list the content of the S3 bucket

aws s3 ls s3://rahul-new-bucket-dec-2020

The way we copy files from the local machine to the S3 bucket, we can also sync a local directory with the bucket.

Sync does not copy the existing files and folders from the local machine to the bucket. It just copies the files which have been newly created or modified

Let's create a new file call our local machine

touch new-file

Time we will use sync.

aws s3 sync . s3://rahul-new-bucket-dec-2020

Now let's again create a new file

touch file-after-sync

Now you can see that if we sync again, only newly created files are getting copied. The files which have already been copied to the bucket are not getting copied.

aws s3 sync . s3://rahul-new-bucket-dec-2020

But this is not the case with the copy option. It copies all files irrespective of the files which are available on S3.

aws s3 cp . s3://rahul-new-bucket-dec-2020 --recursive

Refer to the following screenshot to understand the commands we just tried.

Create a bucket and perform copy and sync operation

Let's check what files are available in our S3 bucket.

aws s3 ls s3://rahul-new-bucket-dec-2020

We can delete a particular file from the S3 Bucket using the "rm" subcommand.

aws s3 rm s3://rahul-new-bucket-dec-2020/new-file

Check if the file has been deleted or not from the bucket

aws s3 ls s3://rahul-new-bucket-dec-2020

We can even delete all the objects from the bucket using the --recursive option

aws s3 rm s3://rahul-new-bucket-dec-2020 --recursive

Check if the objects have been deleted from the bucket or not

aws s3 ls s3://rahul-new-bucket-dec-2020

Refer to the following screenshot to understand the operations we just performed.

Remove objects from S3 bucket

The way we could delete objects from the bucket we can also delete the bucket itself

First, list all the buckets available in the account.

 aws s3 ls

Delete the required Bucket using the "rb" sub-command.

 aws s3 rb s3://rahul-new-bucket-dec-2020

We can see that the specified bucket has been deleted from the count now

aws s3 ls

Remove S3 Bucket

We can remove the rule that we had attached to the instance so that the instance can no more be able to authorize itself.

To remove the attached Role go to the EC2 instance, click on Actions --> Security --> Modify IAM role and remove the rule which was attached and save the configuration.

Detach the IAM role

Confirm your operation.

Confirm the action

This time if we execute "aws s3 ls" command, we won't be able to list buckets from the account. We will not be able to perform any of the operations from the EC2 Instance.

Check the access


The way we manage the AWS S3 bucket from the AWS console, we can also manage it from the CLI. We saw the steps to install the aws-cli command-line utility. We also saw the steps to attach and detach the I am Role from the EC2 instance. We performed a few basic operations on the S3 bucket using the aws-cli command-line utility.

Share this page:

0 Comment(s)