Category: Question and Answers
Updated

This solution is summarized from an archived support forum post. This information may have changed. If you notice an error, please let us know in Discord.

Ability to head object in S3?

Issue

I'm wondering if it's possible to use an S3 datasource query to check if a file exists at a certain path without downloading it because it's big. However, I don't see the option to use HeadObject in the list of queries. Am I missing something or is this not supported?

Resolution

The solution to check if a file exists in an S3 bucket without downloading it is by sending a HEAD request to the object. This can be achieved through the AWS SDK for any programming language.

Using the AWS SDK for Python, the code would look like this:

import boto3

s3 = boto3.client('s3')
bucket_name = 'my-bucket'
key = 'path/to/my/object'

try:
    s3.head_object(Bucket=bucket_name, Key=key)
    print(f"Object {key} exists in {bucket_name}!")
except:
    print(f"Object {key} does not exist in {bucket_name}.")

This code sends a HEAD request to the specified object in the S3 bucket my-bucket. If the object exists, it will print a message saying so. If the object does not exist, it will print a message saying it does not exist.

Note that the boto3.client method requires AWS credentials to be set up either as environment variables or through an AWS configuration file in the user's home directory.