maurice/snix
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
snix/contrib/archivist
History
Florian Klink ae4d967288 chore(ops): move archivist machine to ops and contrib 
contrib/ gets the clickhouse patching, the bucket log parsing code and
the awscli setup and shell.

ops/ gets the machine config itself.

Change-Id: If8b8f8cce5ca9c2b4d19e17be9a8b895ac35e84a
Reviewed-on: https://cl.snix.dev/c/snix/+/30163
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-20 12:21:44 +00:00
..
default.nix chore(ops): move archivist machine to ops and contrib 2025-03-20 12:21:44 +00:00
OWNERS chore(ops): move archivist machine to ops and contrib 2025-03-20 12:21:44 +00:00
parse_bucket_logs.rs chore(ops): move archivist machine to ops and contrib 2025-03-20 12:21:44 +00:00
README.md chore(ops): move archivist machine to ops and contrib 2025-03-20 12:21:44 +00:00

README.md

archivist

This directory contains various scripts and helpers used for nix-archivist tasks.

It's used from the archivist EC2 instance, as well as standalone.

AWS Profile setup

There's 2 AWS Accounts, reachable via the nixos.awsapps.com SSO portal.

archeologist

This is assuming the archeologist AWS role in the main NixOS account.

archivist

This is a separate AWS Account, only for the archivist project. We can assume AWSAdministratorAccess in there.

archivist-ec2 Machine

The archivist-ec2 machine currently is deployed in the main NixOS account.

It regularly processes S3 bucket logs and dumps them in parquet format into another bucket. In the future, we want to move this machine to the dedicated archivist AWS account.