feat(wpcarro/blog): Create short post about restic
Read the blog post for more information :) Change-Id: I46a2f8ec9dc0a6546062e645f34186aa9195c70f Reviewed-on: https://cl.tvl.fyi/c/depot/+/5907 Reviewed-by: wpcarro <wpcarro@gmail.com> Autosubmit: wpcarro <wpcarro@gmail.com> Tested-by: BuildkiteCI
This commit is contained in:
		
							parent
							
								
									2ec0d36119
								
							
						
					
					
						commit
						aaa0f92c45
					
				
					 2 changed files with 98 additions and 0 deletions
				
			
		|  | @ -36,4 +36,11 @@ | |||
|     content = ./posts/csharp-unused-variables.md; | ||||
|     draft = false; | ||||
|   } | ||||
|   { | ||||
|     key = "restic-field-guide"; | ||||
|     title = "Beginner's Field Guide to restic"; | ||||
|     date = 1656645093; | ||||
|     content = ./posts/restic.md; | ||||
|     draft = false; | ||||
|   } | ||||
| ] | ||||
|  |  | |||
							
								
								
									
										91
									
								
								users/wpcarro/website/blog/posts/restic.md
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										91
									
								
								users/wpcarro/website/blog/posts/restic.md
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,91 @@ | |||
| Continuing along the trend that [Profpatsch][2] recently inspired in me: writing | ||||
| short notes to myself instead of fully fledged blog posts aimed at some | ||||
| unknowable audience. Today we're looking at how I burned myself by only | ||||
| *partially* RTFD. | ||||
| 
 | ||||
| ## Background | ||||
| 
 | ||||
| I recently started using [restic][4] and NixOS thanks to the help of [TVL's | ||||
| `restic.nix` module][1]. I setup `1x/h` backups to [MinIO][3] (S3-compatible | ||||
| storage) for just a handful of `/var/lib` directories (`~9GiB` total), but after | ||||
| a few days MinIO reported that my bucket size was `O(100GiB)`! | ||||
| 
 | ||||
| > What's going on? | ||||
| > -- me | ||||
| 
 | ||||
| ```shell | ||||
| $ restic stats | ||||
| repository 763bfe37 opened successfully, password is correct | ||||
| scanning... | ||||
| Stats in restore-size mode: | ||||
| Snapshots processed:   175 | ||||
|    Total File Count:   31369384 | ||||
|          Total Size:   21.027 GiB | ||||
| ``` | ||||
| 
 | ||||
| > Wait: 20GiB... wat? | ||||
| > -- me (moments later) | ||||
| 
 | ||||
| Maybe we're snapshotting our MinIO buckets, and that's contributing to our | ||||
| bucket size. Checking the logs proved that `restic` was backing-up `1.5GiB/h`, | ||||
| which supported MinIO's reports. | ||||
| 
 | ||||
| > Ah maybe `restic stats` isn't reporting what I *think* it's reporting... | ||||
| > -- me (again) | ||||
| 
 | ||||
| Let's consult Le Docs: | ||||
| 
 | ||||
| ```shell | ||||
| $ restic stats -h | ||||
| 
 | ||||
| The "stats" command walks one or multiple snapshots in a repository | ||||
| and accumulates statistics about the data stored therein. It reports | ||||
| on the number of unique files and their sizes, according to one of | ||||
| the counting modes as given by the --mode flag. | ||||
| 
 | ||||
| It operates on all snapshots matching the selection criteria or all | ||||
| snapshots if nothing is specified. The special snapshot ID "latest" | ||||
| is also supported. Some modes make more sense over | ||||
| just a single snapshot, while others are useful across all snapshots, | ||||
| depending on what you are trying to calculate. | ||||
| 
 | ||||
| [to be continued] | ||||
| ``` | ||||
| 
 | ||||
| This is where I stopped reading (the first time). But then I returned a second | ||||
| time as I was running low on theories... | ||||
| 
 | ||||
| ```shell | ||||
| [continued] | ||||
| 
 | ||||
| The modes are: | ||||
| 
 | ||||
| * restore-size: (default) Counts the size of the restored files. | ||||
| * files-by-contents: Counts total size of files, where a file is | ||||
|    considered unique if it has unique contents. | ||||
| * raw-data: Counts the size of blobs in the repository, regardless of | ||||
|   how many files reference them. | ||||
| * blobs-per-file: A combination of files-by-contents and raw-data. | ||||
| ``` | ||||
| 
 | ||||
| Bingo: `--mode=raw-data` **not** `--mode=restore-size`. | ||||
| 
 | ||||
| ## Solution | ||||
| 
 | ||||
| ```shell | ||||
| $ restic stats --mode=raw-data | ||||
| repository 763bfe37 opened successfully, password is correct | ||||
| scanning... | ||||
| Stats in raw-data mode: | ||||
| Snapshots processed:   175 | ||||
|    Total Blob Count:   710988 | ||||
|          Total Size:   303.216 GiB | ||||
| ``` | ||||
| 
 | ||||
| > Ah... the world agrees again. | ||||
| > -- me | ||||
| 
 | ||||
| [1]: https://cs.tvl.fyi/depot@2ec0d3611960b163a7052e8554ba065f3c89a8cc/-/blob/ops/modules/restic.nix?L9 | ||||
| [2]: https://github.com/profpatsch | ||||
| [3]: https://min.io/ | ||||
| [4]: https://restic.net/ | ||||
		Loading…
	
	Add table
		Add a link
		
	
		Reference in a new issue