![]() ![]() ![]() WANT: de-dupe data from different hosts.Attic had a corruption issue at scale:.SHOULD: Scale to >2TB single repos (releases+distfiles historical).SHOULD: support permanent/forever retention.Having multiple readers (of files to backup)/writers (of backup data) would massively improve performance. ![]() Some data sets for backup are very large, but change slowly, and we have lots of IO behind them.MUST NOT: require a complete local copy of the data to be retained.Maybe free S3-based storage from a sponsor (Dreamhost, DigitalOcean.).has a Borg/Attic option at $0.03USD/GB/month ( ).AWS S3-IA/Glacier is USD0.0125/GB/mo or less.This can be in the form of simply syncing the off-host storage to the cloud.MUST: Support cloud storage (AWS-S3, Ceph-S3, other).MUST: Support External/off-host storage.SHOULD: be optional, some repos are already over pre-compressed data (eg distfiles, releases).MUST NOT: disclose the passphrase or key data to generate new backups.SHOULD: encryption should be optional, as we have some public data.MUST: be open-source (including server-side if any).The loss of a backup server will require at least restoring metadata from the other server or cloud, but hopefully not full data from the cloud. We should get backups going again, and cover EVERYTHING in future, and keep a redundant copy of the backups in the cloud, since we lost multiple hosts before :-(.īackups will probably be sent to a backup server "nearby" (in the same continent), and archived from there to multiple cloud locations. Infra used to have good backups, but all past backups got lost with multiple sponsor losses (we had two copies of the data, one in Europe, one in USA, both got lost for different reasons, in a short time frame). ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |