Databases with V
May 16, 2025 at 04:51 AM
James Cowling: I designed Dropbox's storage system and modeled its durability. Durability numbers (11 9's etc) are meaningless because competent providers don't lose data because of disk failures, they lose data because of bugs and operator error. Yes S3 has lost data. No it wasn't because some disks failed. If you're building your own infrastructure you should heavily invest in release process and validation testing (link in reply). You're not going to do a better job than a major cloud provider though. The best thing you can do for your own durability is to choose a competent provider and then ensure you don't accidentally delete or corrupt own data on it: 1. Ideally never mutate an object in S3, add a new version instead. 2. Never live-delete any data. Mark it for deletion and then use a lifecycle policy to clean it up after a week. This way you have time to react to a bug in your own stack. https://dropbox.tech/infrastructure/pocket-watch
❤️ 👍 💡 🙏 🪢 12

Comments