Object Storage - Disable GCS Content Listing

By default, a Google Cloud Storage (GCS) bucket, private or public,  will list its contents when navigating to the bucket's root directory. In most cases, it is prudent to restrict access to a file listing, as it can help prevent website scraping and undesired viewership. The following guide will show how to disable directory listings on GCS buckets through StrikeTracker.

Disable on the Bucket Level

To prevent the public listing of your files, you may choose to modulate on a bucket level. The bucket level option is best if you have numerous CDN Sites connected to a single bucket.

Bucket Settings

First, we will enable website settings on the desired GCS bucket with gsutil. Skip ahead to Step 3 if you already have the correct credentials to access your bucket with gsutil.

1. Retrieve your Object Storage Credentials
   1. Log in to https://striketracker.highwinds.com
   2. Navigate to Account > Object Storage > Service Accounts
   3. Generate a Service Account and a key
      
2. Authenticate with GCloud

            gcloud auth activate-service-account --key-file {your_key_file

3. Enable Website settings on your Bucket

            gsutil web set -m index.html gs://{bucket_name}

Site Settings

On the CDN side, we will need to update the Host Header sent on Origin Pull Requests. Sending a non-default host-header will allow your GCS bucket to behave like a website, not listing bucket contents.

1. Navigate to your Sttiketracker Origin for the GCS Bucket
   1. Log in to https://striketracker.highwinds.com
   2. Under Configuration in the left-hand sidebar, select Origins.
      
   3. Choose the desired GCS Origin from the origin list.
2. Modify the origin's Headers to add when pulling from this origin
   1. 1. After opening the Origin Settings, Select Advanced Settings.
         
      2. At the bottom of Advanced Settings, add a host header with your bucket name to be appended when pulling from the GCS origin.
         

Your Bucket is now in website mode. Public and Private buckets will no longer list bucket contents, 404'ing at the root directory if no index.html is present. Any future sites added with the same GCS origin will not need an additional update.

Disable on the Site Level

The site method requires no additional work with gsutil and is done entirely via the CDN. This method is preferred if you are planning to only connect the origin to a single site, or are struggling to implement the Bucket Level method.

Site Settings

To disable on the site level, a simple update to the Origin URL Filter is needed to prevent access to the root URL of your site.

1. Log in to https://striketracker.highwinds.com
2. Navigate to your Site > Origin Settings
3. Under Origin Connections > URL Filter, enter the Regex statement
   

         regex:!/$/

   

This statement will prevent any request where the path is blank from initiating an origin request from the CDN. Therefore, no users will be able to reach the bucket's file listing. You will need to perform this configuration update for each of your sites using a GCS bucket.