{"id":31092,"date":"2020-12-21T16:35:33","date_gmt":"2020-12-21T21:35:33","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=31092"},"modified":"2020-12-21T17:00:18","modified_gmt":"2020-12-21T22:00:18","slug":"s3-strong-consistency","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2020\/12\/21\/s3-strong-consistency\/","title":{"rendered":"S3 Strong Consistency"},"content":{"rendered":"

Amazon<\/a> (via Tim Bray<\/a>, Hacker News<\/a>):<\/p>\n

Amazon S3 now delivers strong read-after-write consistency automatically for all applications. Unlike other cloud providers, Amazon S3 delivers strong read-after-write consistency for any storage request, without changes to performance or availability, without sacrificing regional isolation for applications, and at no additional cost.<\/p><\/blockquote>\n\n

Jeff Barr<\/a>:<\/p>\n

One of the more interesting (and sometimes a bit confusing) aspects of S3 and other large-scale distributed systems is commonly known as eventual consistency. In a nutshell, after a call to an S3 API function such as PUT that stores or modifies data, there’s a small time window where the data has been accepted and durably stored, but not yet visible to all GET or LIST requests.<\/p>\n

[…]<\/p>\n

Effective immediately, all S3 GET<\/a>, PUT<\/a>, and LIST<\/a> operations, as well as operations that change object tags<\/a>, ACLs<\/a>, or metadata, are now strongly consistent. What you write is what you will read, and the results of a LIST will be an accurate reflection of what’s in the bucket. This applies to all existing and new S3 objects, works in all regions, and is available to you at no extra charge! There’s no impact on performance, you can update an object hundreds of times per second if you’d like, and there are no global dependencies.<\/p><\/blockquote>\n\n

They don’t say how this works, but presumably the tradeoff is reduced availability, with the expected latency and error rate low enough that you don’t mind.<\/p>\n\n

25519FUUU<\/a>:<\/p>\n

The most devious s3 consistency issue I encountered was that S3 bucket configs are actually stored in S3, and were not read-after-write consistent.<\/p>

It was bad. In order to “update” a bucket config programmatically you need to read the entire existing config, make an update, and then PUT it back (overwriting what’s there). The problem is when you went to read the config it’s possibly 15 minutes old or more, and when you put it back you overwrite any changes that may not be consistent.<\/p><\/blockquote>\n\n

Previously:<\/p>\n