Skip to main content

Serving WordPress images from S3 without plugins

Note:This post uses NGINX but a similar thing could be done with Apache

I recently finished building a small online shop for a family member. They had been running a shop for several years using a popular E-Commerce marketplace. Now they wanted their own website with complete control.

To keep costs low, I offered to build them something using Free Software that I could host on my VPS.

I settled on using WordPress as the CMS and WooCommerce for the shop.

When developing the website, my goal was to only use WordPress plugins when necessary. This is due to the increase in attack surface area they tend to create.

Before launching the shop, I needed to decide where to store the media library. The shop has various products and each product requires many images.

After uploading an image, WordPress will create several versions at different resolutions. All these images add up and at some point the disk space would run out. A cheaper home was needed.

Solution

After deciding to use Amazon S3 as my storage, I looked to set up an automatic transfer. There are several popular plugins that can handle this for you. Yet, I was trying to avoid using plugins.

Another problem, some plugins required a paid version to move existing images.

An option I ruled out was editing WordPress or it's database. I wanted to handle it via the web server in a way that's invisible.

After a few minutes of thinking, The following solution emerged:

  1. Create a public S3 bucket
  2. Create the same path in the bucket as the upload folder.
  3. Use the AWS CLI tool to move the assets to the bucket.
  4. Write an NGINX rule to check if the local image exists
  5. If not then proxy it via S3
  6. Store the aws command in a script & schedule it to run daily with cron.

Command to copy existing files to bucket

After installing the AWS CLI, you can run this to move all the files in the uploads folder to your S3 bucket. You should add --dryrun to the end while you're testing.

aws s3 mv /srv/example.com/wp-content/uploads s3://[bucket-name]/wp-content/uploads/ --recursive

If you want to exclude certain files.

--exclude "some-directory/*" --exclude "another-directory/*"

Automating with CRON

Once your confident it works, save it to a text file and making it executable with chmod +x. You can then add it to the system crontab in /etc/crontab

This will run the script every day at 3:00 AM.

00 3 * * * [username] /home/[username]/bin/mv-uploads-s3.sh > /dev/null 2&1

NGINX config for checking locally then fallback

Place inside your websites server block

server {
        listen 443 ssl;
        ...
        root /srv/example.com;
        ...
        location ^~ /wp-content/uploads {
            try_files $uri @s3;
        }
        location @s3 {                                                             
            proxy_pass https://[bucket-name].s3-ap-southeast-1.amazonaws.com;
        }                                                                          
    }

For any requests to the /wp-content/uploads folder, NGINX will first try the local files. If that fails then it will proxy_pass it to the Amazon bucket.

If you tried to access

https://example.com/wp-content/uploads/2021/03/example.jpg

It would check

/srv/example.com/wp-content/uploads/2021/03/example.jpg

If this file doesn't exist, it will try

https://[bucket-name].s3-ap-southeast-1.amazonaws.com/wp-content/uploads/2021/03/example.jpg

The end user will see the file as if it came from the original URL.