We want to be able to play the files directly from the web
browser (jukebox).
Luckily, transmission does not seem to change the filenames from the
ones given by the torrent file, so we can literally parse the torrent
file and construct a path to the media file, extraordinary.
Adjusts the caddy reverse proxy to serve the given transmission
directory (using my weird sshfs forwarding scheme in the shell.nix
preset lol), then redirect from a handler that maps from
torrentId/fileId to the actual file.
Change-Id: Iab5faf7cc06066f3253031af31e137c0e28f54e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13270
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Gets rid of the giant bootstrap dependency and replaces it with a
little CSS file that formats our table nicely.
Change-Id: I3e0bb552481503679e7b229c4feaa4e9805f5eae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13269
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Some experiments in speeding up search efficiency.
This is more of a “is this possible” thing, and it looks like it’s
possible, but does not really improve anything regarding the rate
limiting.
The idea is that we can start everything at the same time as async,
but use a semaphore to have only 5 things run at once. That also means
that as soon as something is done, we immediately start the next task.
The asyncs are guaranteed to be cleaned up by the `ResourceT`
wrapper (eventually).
It’s pretty cool how Conduit makes writing these side-effecting things
pretty possible.
Change-Id: Ibadead7db293373b415840960602fa71920fc653
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13246
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
This bunches up 5 search page requests to run at the same time.
We use a conduit now, so we could get smart about returning partial
results and such (if for example upstream puts us into the rate limit,
which they do after 10 requests.
Change-Id: Idbb174334fa499c16b3426a8d129deaf3a1d3b0b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13245
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
I missed this! It speeds up `/artist` by another 50%.
Change-Id: I138c38185ea7fb606a139b436b9c956f7b8e1a37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13244
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
move the label stuff into its own temporary module (until we figure
out what to put into pa-label).
Also rewrite a few things to use t2/t3.
Change-Id: I8cc8678ec01a56d6c738eb4833a3ba566a7a1e20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13242
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
This updates the expression to make use of "QMK userspaces".
We use the fork from bastardkb qmk to get support for v3, and also use
their userspace, but then copy our slightly modified keymap in.
Change-Id: I2078e5942560aa509c091df8f41d30894ee8ac37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13243
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
* Bump Emacs 29 to 30 to address CVEs in prior versions:
https://github.com/NixOS/nixpkgs/pull/386174
* //3p/overlays/tvl:
- Drop upstreamed fix for buildkite-agent
- Drop tpm2-pkcs11 patch for an issue that has been
addressed in 1.9.1.
- Drop Nix 2.3 patch for home-manager. An alternative
to it has been upstreamed in
<https://github.com/nix-community/home-manager/pull/5067>.
* //users/flokli/presentations: disable derivations that have
been failing since the latest chromium upgrade (presumably).
reveal-md … --print fails to export a PDF. Enabling debug
output reveals that a timeout in pupeteer is hit.
Change-Id: Id83eb5e5fe2db77e648817c5c737b2f95b43deeb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13217
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
We need `artist_ids` in most of the queries, so let’s make them a
generated column and put an index on them.
This reduces times for getArtistNameById from ~300ms to ~20ms and for
the main “latest” table from ~250ms to ~60ms.
The `artist_has_been_snatched` logic moves into the torrent filter, so
we don’t have to work on such large records further down.
Change-Id: I5a1929bd9c81ea0031e512d01897c0e460ded077
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13240
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Or rather make it less slow lol.
I should put the artists in a computed column now.
Change-Id: I28f6546353e2930156eaeb427ba5122559aea49b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13239
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
A simple favourite system that adds an artist to your favourites when
you go to their artist page for the first time.
Also, if you fetch something from an artist, they get automatically
added to the favourites.
This is pretty much just done by adding more crap to the main SQL
query, and has the disadvantage that the `artist_has_been_snatched`
CTE slows down the query a lot for the favourite filter (by 50% or
even sometimes 150% in a weird degenerate case). I think we can
optimize this somewhat, but that needs a small rewrite of the query
CTE stages.
Change-Id: I7aa6d99dc26e24eb22ef56ffd4d2db5c6978ad48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13238
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
For the next step, we need to put more things in WITH-Clauses, so
let’s prepare it now to get a nicer diff.
Change-Id: Ibd5b67c77c87ce3d6aa0ab04ca3d66f9b90dc856
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13237
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
For latest releases, I really don’t want to see bootlegs and stuff.
Change-Id: I3a98d7b81d2ca7962dcef619d4dcfcbd18e98f84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13226
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Let’s start improving the main page.
So far, it was just all release groups sorted by weight on a single
page, which was not super helpful (and got kinda large).
The first feature is to show the latest releases that are known.
This is done by torrent group ID. This ID does not always correspond
to the date, but can also be a very old album that gets uploaded,
or (seldomly) a group that gets merged for metadata.
We should think about restricting this to favourites, automatically
marking everything as favourite where we have an album downloaded or
clicked on the artist before, and then selectively allow to un-mark
it after the fact.
An even stronger “not interested” could be used to automatically
reclaim seedbox space once it becomes an issue.
Eventually (after implementing favourites), we should introduce a
job system that automatically updates these entries every few hours.
Maybe even have a “very interested” feature that automatically
downloads everything new for an artist?
And then a “veryvery interested” feature that also buys the thing from
bandcamp lol
Change-Id: I467c350722279ff37150f847f5014d7e0e67e626
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13225
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
The RED Gazelle API is so inconsistent, smh.
Sometimes they return the release type string, sometimes the integer
representing it. So let’s handle both!
Change-Id: Ie99f01342a13269df5c1e2036aaede54d7149516
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13223
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
For all big page reloads, we want the `<head>` of the page to start
being transmitted even while doing database requests.
So let’s use the `Wai.ResponseStream` to do exactly that. The handler
provides the contents of `<head>`, we start streaming that, meanwhile
it calculates the `<body>` and once that is ready transmits it.
This means we can load all our static resources before the page
even starts sending any body data, meaning the css and html is already
there when we reach `</html>`.
Sweet.
The `<title>` in `artistPage` was depending on the table data, so I
moved it into a separate SELECT.
We can do all of it in parallel as well. Sweet.
~~~
This also adds static file prefetching of source maps if provided.
Change-Id: Ib824430594733b4c8e86ee1096c8afba2df1a66d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13221
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Extracts the required version out of the telega.el Dockerfile (this seems to be
the authoritative source), and matches that against what nixpkgs has.
In a future commit I'll improve this to reduce the likelihood of blocking a
channel bump (by also pinning tdlib, and issuing warnings when nixpkgs has a
newer telega/tdlib).
Change-Id: I1129c1f6b38aa58eb8661f2ad9bc6fa19382d81c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13220
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Use the default emacs attribute over the versioned emacs28 attribute. On
current canon, this implies an upgrade to Emacs 29.
Emacs < 30 has some CVEs that have never been patched in those release
series. The next channel bump will upgrade pkgs.emacs to Emacs 30, so
just using the default alias seems to be the easiest solution.
See also <https://github.com/NixOS/nixpkgs/pull/386174>.
Change-Id: I1580bae138cc0801e0f3431b1a02c8a4585c2996
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13216
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Proxy the resources through our server instead of always requesting
them from the CDN, leaking lots of usage data.
This prefetches the resources at server startup (instead of e.g.
adding them to the build environment statically), which is a nice
compromise I think. It will also keep the integrity verification in
place, but that is still done in the client.
Adds `rel=preload` as well, so we start fetching asap.
No caching (yet). We could easily add a hash over the content though
so the client never has to re-request the resources.
Change-Id: I9aac80cfb1ded09e578ba2a70dcf982bf5322ff6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13215
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
The settings could actually be improved with an htmx snippet, so that
we don’t have to load the whole page every time.
Change-Id: I472940533881ff24c0111188e7265c59665bc29d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13213
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Throws out htmx for search in favor of a full page-reload for search.
This means we can reload the page and it will re-do the search.
It’s not POST anymore cause we don’t really change anything, just
do some caching when doing the search. Could probably also cache the
search result, but simple simple
Change-Id: I70b0cbac55f67c986aeecfedcf0b3968890b3b9c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13212
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
The `bencode` library is anything but production-grade, but it’s
enough to parse the torrent files generated by Gazelle lol.
This should help with … I haven’t really figured out yet what it helps
with I guess. But it was fun. I like the `Parse` abstraction very
much. It can parse XML and it can parse Bencode. Good.
Change-Id: If1331de423eab3e91ce9db6e2a7eb84da51b18a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13211
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Experiment: can we convert any Html-producing conduit into a
self-updating htmx snippet? Yes!
Add the resulting handler and initial snippet, and it will poll the
handler every second or so until the conduit stops emitting values.
Change-Id: I0bed17a5db8691a19df6c334ae1268dd6a3d3123
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13209
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
The unpacked lrclib dump is 40GB at this point, but compresses very
well (to about 10GB give or take). So I thought about how to store it
in a compressed on disk, but still allow sqlite to query
efficiently (i.e. use a compression format that has arbitrary seek
indices).
Squashfs! with zstd compression!
```
mksquashfs ~/.cache/lyric-bak/ \
~/tmp/squashtest \
-comp zstd \
-Xcompression-level 5
```
Still need to write a helper that does the download & compression for
me to update the database once in a blue moon.
Unfortunately, sqlite-utils does not allow URL connection strings yet,
so I had to do a horrible SQL interpolation bit.
Issue tracker: https://mastodon.xyz/@Profpatsch/114083481108799334
This also changes the query to filter out every track that has neither
plain nor synched lyrics (yeah those exist for some reason?).
Change-Id: Ief0519ae8d9e5f14b66f6df6e3bcecae911ea142
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13208
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
This adds code allowing to parse (and emit) the "internal-json" log
format that Nixcpp emits.
Move the Verbosity struct from worker_protocol.rs out into
src/log/mod.rs, it's not only used there.
Change-Id: Ia35e739c83e06080a677b576616bdd0d4a821cf4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13167
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Reviewed-by: edef <edef@edef.eu>
See 72dd5818b7/CHANGELOG.md
for the nom changelog.
Most notably, there's now a .parse() to be added:
`combinator(arg)(input)` -> `combinator(arg).parse(input)`
There also doesn't need to be a tuple combinator (it's implemented on
tuples directly).
This also refactors the string / byte field parsing parts, to make them
more concise.
Change-Id: I9e8a3cedd07d6705be391898eb6a486fb8164069
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13193
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: Brian Olsen <me@griff.name>
I am already using this outside of depot where it isn't under VCS. It
makes sense to canonicalize it, also to ensure that it stays somewhat
generic.
Change-Id: I5595d98ab4198794c395feb4d3c08df1e2d01a36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13184
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Remember //nix/escapeExecline? Well, this is worse (for some possible
meanings of the word). Instead of taking a list and escaping/rendering
it to an execline script, Execline reimplements part of execlineb(1):
It takes a (nested) list where any enclosed list signifies an execline
block and produces the corresponding execline argv form as described in
execline-block(7). This means the result of Execline can directly be
executed using •SH without the need for execlineb(1).
Consequently, execlineb(1)'s management of positional parameters and the
environment are not available. This is fine for the intended purpose of
Execline (glueing together shell commands efficiently without messing
around with the FFI in BQN for pipe(2) etc.).
Change-Id: Ief69b1bab919c16b6e39c3f5dc3db628766c5a8c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13180
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
I've managed to (several times!) lock myself out of the office network by
rebuilding while the configuration was unavailable. Might as well assert that.
Change-Id: Ib2685c0e8dab977a3cb576606e70f8960968b574
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13168
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Included changes:
* targeted primary NixOS channel back at `nixos-unstable`
* //3p/chicago95: disable new broken symlink check
* //tvix: omit unnecessary lifetime clippy recommends
* //users/sterni/blipqn: wait a bit for data to arrive in test so it
succeeds under load.
* //fun/paroxysm: force pkg-config flag of pq-sys by adding a bogus
dependency on it. Otherwise, pq-sys will try to use pg_config
which does not work correctly in pkgs.libpq at the moment.
* //users/flokli/keyboards/dilemma: disable temporarily
Change-Id: I6d53bd7bca6886f3457e1f41505e97314f4cd191
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13119
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: aspen <root@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
