Restore deleted S3 object versions

If you use S3 to host static websites, I recommend enabling versioning. It's saved my bacon more than once, most recently this weekend when I passed a --delete flag to a sync command that I should not have passed.

Of course, by the time I'd realized my mistake, I'd already deleted more items than intended — so many more that restoring them via the console would have taken a few hours.

Realizing this, I went in search of answers. Unfortunately, all of the results for my search turned up answers for S3 objects being stored using the AWS S3 Glacier storage class — the restore-object command, for example. What I needed was a way to restore items stored using S3's STANDARD Storage class.

After a few clicks, I stumbled across this StackOverflow question, Commandline tools to restore a versioned deleted file in S3?, which gave me a clue.

Thirty minutes later, and I had restored all of my unintentionally-deleted items.

Deleted objects and delete markers

Before we continue, I want to explain a little bit about how S3 versioning works.

Every time you upload an S3 object with the same name as an existing key¹, S3 adds a VersionID to the previous object and served the new object at the same URI.

When deleting, an object, however, that new object is a special type of object known as a delete marker. S3 still assigns an identifier to the previous object, but the URI will return a 404 Not Found error instead. What's more, DELETE operation are the only kind of operation you can use with a delete marker.

Listing object versions

The first step in restoring my objects to get a list of objects and their versions using the list-object-versions command of the s3api library (which is part of the AWS Command Line Interface).

aws s3api list-object-versions --bucket example.com

This command returns a JSON response, containing two properties: Versions and DeleteMarkers

{
    "Versions": […],
    "DeleteMarkers": [
        {
            "Owner": {
                "DisplayName": "thedisplayuser",
                "ID": "5e19567de39e074bc9e64451445f7eca9ee0734f6251d8b499c5aa6cb5497449"
            },
            "Key": "index.html",
            "VersionId": "tuVqN1XyZiZcAxLNIapmASmYmasScuJM",
            "IsLatest": true,
            "LastModified": "2019-01-18T03:55:57.000Z"
        }
    ]
}

Each object returned in DeleteMarkers contains a VersionId and the date the object was last modified. We can use one or both of these properties to find the items we want to restore.

I decided to cache the lisf of objects locally by redirecting the output to a file.

aws s3api list-object-versions --bucket example.com > versions.json

Then I imported the contents into my script.

Restoring a deleted S3 object

Restoring a deleted S3 object to a previous version is simple, but perhaps counterintuitive. To restore deleted all objects, you'll need to delete the delete marker, using the delete-object command.

aws s3api delete-object --bucket example.com --key /key-path.html --version-id tuVqN1XyZiZcAxLNIapmASmYmasScuJM

Now, the only thing left to do is to tie it all together in a script. What follows is the quick-and-dirty (emphasis on quick and dirty), but commented PHP script I wrote to do the job.

#!/usr/bin/php
<?php

// Get the file contents as a string
$json_str = file_get_contents('versions.json');

// Convert it to a JSON object
$versions = json_decode($json_str);

// Return items that were deleted on or after January 19, 2019, at 1am 
function exclude_unmodified_beforetoday($item) {
  return strtotime($item->LastModified) > strtotime("2019-01-19T01:00:40.000Z"); 
}

// Run this command for each object we want to restore
function restore_item($item) {
  $tpl = 'aws s3api delete-object --bucket example.com --key %1$s --version-id %2$s';
  $cmd = sprintf($tpl, $item->Key, $item->VersionId);
  exec($cmd);

  printf('Restored key: %1$s (version: %2$s)'.PHP_EOL, $item->Key, $item->VersionId);
}

$vs = array_filter((array)$vs, 'exclude_unmodified_beforetoday');

$results = array_walk($vs, 'restore_item');

Copy and paste this script into a file named restore-objects.php and run it using either php restore-objects.php, or ./restore-objects.php. If you use the latter, you may have to change the permissions on the script first (chmod 755 restore-objects.php).