How to Use the “download_file()” Method in S3 Using Python Boto3?

Amazon Web Services has taken the world by storm due to its unmatchable features, cost-optimal services, extensive storage areas, and various data-related services. Simple Storage Service (S3) is one of the most popular services of AWS that acts as an underlying storage area for many other AWS services such as EC2, Lambda, RDS, Athena, etc. 

While the S3 buckets efficiently store confidential data, users can also implement various encryption formats on the data, limit the other users by specifying access permissions of the bucket, enable versioning for maintaining object history or enable object lock for retaining the data, etc. Equipped with all these exciting features, Simple Storage Service does not end here. It is fully manageable, highly scalable, and highly available in various AWS regions. 

Learn more: Getting Started With Buckets: Overview of Amazon Simple Storage Service

AWS services and resources can be accessed by using two different approaches i.e., via Console or CLI. Boto3 SDK supported by Python3 is one of the easiest ways to access, connect, and manage the AWS resources programmatically. This SDK is equipped with all the necessary libraries, dependencies, methods, and preconfigured tools required by the user for communicating with AWS. 

What is the Client Interface in Boto3?

Boto3 provides two abstractions to its users for accessing and managing the AWS resources i.e., via Client or Resource. The client abstraction of the Boto3 SDK offers a low-level interface that allows the users to access the AWS resources by using various methods and specifying certain parameters explicitly. 

The methods that are provided by the Client interface include some of the common operations such as creating the bucket, downloading files, deleting objects, or uploading files, etc. In this tutorial, we will be implementing the copy() method of the Boto3 SDK using the Client interface.

Prerequisite

Before getting started with the implementation of the download() method in Boto3, there are some prerequisites for this tutorial: 

• AWS CLI is installed on the local machines.
• Visual Studio Code for executing the code
• Python runtime environment for using Boto3 SDK.

What is the “download_file()” Method in Boto3? 

Data is stored in the form of objects within the S3 buckets. These objects can be downloaded either directly from the console or by using the download_file() function of Boto3 SDK. The download_file() method is a flexible and powerful way of downloading files to a directory by using the programmatic approach. There are two variants of this method provided i.e., download_file() and download_fileobj().

Initial Configurations

Boto3 SDK is a prerequisite for this tutorial. To install Boto3, create an empty directory and open it within the VS code:

Inside the folder, tap the “New file” option. Provide the name of the file with the “.py” extension. The “.py” extension symbolizes the Python file:

For installing the “Boto3”, open the terminal by clicking the “Terminal” option from the toolbar. Tap the “New terminal” option from the drop-down list:

After the terminal is opened, below given is the command to install the Boto3 SDK:

pip install boto3

The SDK has been successfully installed. To log into the AWS account, provide the following command in the terminal:

aws configure

Provide the Secret Access key, Access key, etc of the AWS account to the terminal:

How to Use the “download_file()” Method in S3 Using Boto3?

The download_file() method in the Boto3 allows the user to download the objects directly from the S3 bucket. This section of the article presents a demonstration of the download_file() method.

Syntax

The syntax of this function is given below:

S3.Client.download_file(Bucket, Key, Filename, ExtraArgs=None, Callback=None, Config=None)

Following is a brief description of the parameters of the download_file() syntax:

• Bucket: It accepts a string value and is used to specify the name of the source bucket for downloading the object.
• Key: it refers to the name of the Key to download from. It is of string data type.
• Filename: The path or the name of the file which is to be downloaded. It is defined in the string format.
• ExtraArgs: This field refers to the “Extra arguments” that are used for various purposes and can be specified by the user. The ExtraArgs are of dict type. 
• Callback: It is of function type and is defined as the number of bytes that are transferred periodically while a file is being downloaded. 
• Config: This field refers to the transfer configurations. The transfer configurations are used when performing the transfer. 

Example: Download file Using download_file() Method in Boto3

By using the above-mentioned syntax, the following code provides a simple demonstration of the download_file() method:

import boto3
s3=boto3.client('s3')

s3.download_file(Filename="downloaded.txt",Bucket="newbucket191008", Key="track.txt")
                         
                     

print("successful")

The brief description of the above code is given below:

• Filename: In this field provide the name by which the file is to be downloaded to the local machine. For this demo, we have provided the name “downloaded.txt” for the file to be downloaded. Replace it with the name of your preference.
• Bucket: Specify the bucket from which the file is to be downloaded. At the moment, we have provided the name of the bucket configured in our AWS account. Replace it with the bucket’s name in your account.
• Key: Specify the name of the object which is to be downloaded to the local machine. Currently, the file uploaded to the specified S3 bucket is “track.txt”. Replace it with the file name present in your S3 bucket. 

Output

Replace the “<FileName>” with the name of your python file and execute the code by using the following command:

python <Filename>

After the execution of the code file, the output window will display the following message:

From the left navigation pane of the Visual Studio Code, we can see that the file has been successfully downloaded to the directory:

That is all from this section of the article.

How to Download Files in Binary Mode in S3 Using Boto3?

The “download_fileobj()” method is used to download the file-like objects from the S3 buckets. This method performs the multipart download in multiple threads.

Syntax

The syntax for this function is given as follows:

S3.Client.download_fileobj(Bucket, Key, Fileobj, ExtraArgs=None, Callback=None, Config=None)

Below is the description of the parameters used for the download_fileobj() method:

• Bucket: This field is used to input the name of the file from which the object is to be downloaded.
• Key: It refers to the name of the key to download from.
• Fileobj: This field inputs the name of the file which is to be downloaded. Modes like read or write can be specified which defines the permission to either read the object or write the object. 
• ExtraArgs: This field refers to the “Extra arguments” that are used for various purposes and can be specified by the user. The ExtraArgs are of dict type. 
• Callback: It is of function type and is defined as the number of bytes that are transferred periodically while a file is being downloaded. 
• Config: It presents the transfer configurations. The transfer configurations are used when performing the transfer. 

Example: Download file Using download_fileobj() Method in Boto3

The download_fileobj() method only accepts the writable file-like object and therefore, the file must be opened first in the binary mode and not in the text mode. Code given below provides a demonstration of using the download_fileobj() method in Boto3:

import boto3
s3=boto3.client('s3')

with open("downloaded1.txt", "wb") as data:
    s3.download_fileobj("newbucket191008", "track.txt", data)
                         
                     

print("successful")

A brief description of the code is given below:

• With open(): This field refers to the opening of the file (downloaded1.txt) in the write mode. The downloaded1.txt name will be assigned to the file when it is downloaded and will be opened in the written format (wb) for manipulation by the user. 
• s3.download_fileobj(): it refers to the function that will be used to implement this functionality. It accepts three parameters i.e., the name of the bucket (newbucket191008), the file that is to be downloaded in the binary format(track.txt), and the file name that was specified while opening the file(data). The file will be downloaded and stored in the “data” variable in the binary format for later use by the user.

Output

Execute the Python file by using the following command:

python <FileName>

The output window displayed the “successful” message as specified in the print statement upon the completion of the code:

From the left navigation pane of the VS code, we can verify that the file has been downloaded to the directory successfully:

That is all from this section of the tutorial.

Conclusion

To download the files using the Boto3 SDK, specify the Bucket name, Key, and File name in the download_file() method and then execute the Python code file. Before executing the Python file, ensure that the Python runtime environment is configured on the local machine. To access the S3 service of the AWS, the user is required to sign in to the AWS account via CLI by using the “aws configure” command. The download_file() method will directly download the file to the directory while the download_fileobj() method will download the object to the directory by performing a multipart operation. This article provides a step-by-step demonstration of downloading the objects from an S3 bucket to a local machine.