0

We’ve been using Google Cloud Storage Transfer service and in our data source (AWS) we had a directory accidentally deleted, so we figured it would be in the data sink however upon taking a looking it wasn’t there despite versioning being on.

This leads us to believe in Storage Transfer the option deleteObjectsUniqueInSink hard deletes objects in the sink and removes them from the archive.

We'e been unable to confirm this in the documentation.

Is GCS Transfer Service's deleteObjectsUniqueInSink parameter in the TransferSpec mutually exclusive with GCS's object versioning soft-delete?

Improve this question

1 Answer 1

Reset to default
2

When the deleteObjectsUniqueInSink option is enabled, Google Cloud Storage Transfer will

  1. List only the live versions of objects in source and destination buckets.
  2. Copy any objects unique in the source to the destination bucket.
  3. Issue a versioned delete for any unique objects in the destination bucket.

If the unique object is still live at the time that Google Cloud Storage Transfer issues the deletion, it will be archived. If another process, such as Object Lifecycle Management, archived the object before the deletion occurs, the object could be permanently deleted at this point rather than archived.

Edit: Specifying the version in the delete results in a hard delete (Objects Delete Documentation), so transfer service is currently performing hard deletes for unique objects. We will update the service to instead perform soft deletions.

Edit: The behavior has been changed. From now on deletions in versioned buckets will be soft deletes rather than hard deletes.

Improve this answer
2
  • Is this documented anywhere?
    – barnabus
    Commented Oct 22, 2018 at 21:23
  • Not that I know of, the Object Versioning is the closest documentation that I'm aware of.
    – Joseph Cox
    Commented Oct 24, 2018 at 17:39

Not the answer you're looking for? Browse other questions tagged or ask your own question.