Commit graph

158 commits

Author SHA1 Message Date
ccf1871604 further fixes (commit_id not in Lwt monad anymore) 2024-12-09 11:15:04 +01:00
df53182114 minor fix, the commit_id is not in Lwt.t anymore 2024-12-06 08:51:11 +01:00
bbd09258d0 Merge pull request 'update to git-kv 0.1.1 -- use val commit :)' (#31) from git-kv-011 into main
Reviewed-on: #31
Reviewed-by: Reynir Björnsson <reynir@reynir.dk>
2024-12-03 12:26:53 +00:00
16dd833477 update to git-kv 0.1.1 -- use val commit :) 2024-11-29 13:58:17 +01:00
ddb0f610e4 Merge pull request 'when creating the tarball, only include the packages folder, repo and version' (#30) from tar-include-only-packages-version-repo into main
Reviewed-on: #30
Reviewed-by: Reynir Björnsson <reynir@reynir.dk>
2024-11-28 09:12:34 +00:00
80f5511133 when creating the tarball, only include the packages folder, repo and version
fixes #29
2024-11-27 19:36:48 +01:00
6447339f64 Merge pull request 'add some Lwt.pause during Disk.check to allow the web server to process requests' (#26) from add-pause-during-check into main
Reviewed-on: #26
Reviewed-by: Reynir Björnsson <reynir@reynir.dk>
2024-11-21 11:31:04 +00:00
2c606dbeb4 Merge pull request 'specify sizes of partitions in MB, not in sectors' (#27) from specify-in-mb into main
Reviewed-on: #27
Reviewed-by: Reynir Björnsson <reynir@reynir.dk>
2024-11-21 11:27:40 +00:00
c5e091a294 specify sizes of partitions in MB, not in sectors 2024-11-21 12:11:48 +01:00
c57b070d87 add some Lwt.pause during Disk.check to allow the web server to process requests 2024-11-20 12:42:06 +01:00
2edc311a33 Merge pull request 'revise startup, address urls pointing to same sha256 and support mirrors (upstream and in opam file)' (#24) from startup into main
Reviewed-on: #24
2024-11-20 10:38:21 +00:00
a9c249ea86 Use Map.fold over List.fold_left ... (Map.bindings _) 2024-11-19 16:26:02 +01:00
ec45a6a77a Prioritize mirrors over upstream caches
Also expand on the semantics of --upstream-cache.
2024-11-19 16:04:08 +01:00
62d62420b7 remove some logs 2024-11-14 19:39:38 +01:00
da4533c2d3 uniquify urls in respect to sha256 checksums (#21)
this reduces 19139 urls down to 18563 urls
2024-11-14 19:25:14 +01:00
6a0ae2bcab take mirrors into account (#13) and allow upstream-caches (#5)
This is done by introducing a set of alternative download locations.
2024-11-14 17:40:22 +01:00
c376a4b70e checked: do on file-by-file basis, incrementally 2024-11-14 16:58:20 +01:00
4481923ade revise startup (as proposed in #18):
- recover git (from disk or download)
- make index.tar.gz
- start web service
- check disk (unless skip-verify-sha256)
- dump git state
- start downloads
- enable update job, and hook
2024-11-14 16:36:10 +01:00
7e09f08767 print git last modified as well as HTTP date 2024-11-14 16:35:56 +01:00
6dab71a9ac introduce more data on status page:
- remaining downloads
- identified urls
- last git fetch & fetch status

see #22
2024-11-14 15:50:26 +01:00
e51550aedc if there's a http error, return the error 2024-11-13 20:03:51 +01:00
982a35a5b1 status: sort time as newest first 2024-11-13 14:14:32 +01:00
97f68a85e9 cope with git-kv change in 0.0.5: digest returns the raw digest 2024-11-13 12:47:36 +01:00
921ee6b684 Merge pull request 'use git-kv 0.0.5+ API for to/of_octets' (#17) from git-kv-05 into main
Reviewed-on: #17
Reviewed-by: Reynir Björnsson <reynir@reynir.dk>
2024-11-08 14:41:12 +00:00
68d2ec8e98 use git-kv 0.0.5+ API for to/of_octets 2024-11-08 15:31:50 +01:00
ccc8e562b1 require http-mirage-client 2024-11-08 13:56:47 +01:00
96368097e9 Merge pull request 'Use swapfs' (#16) from swap into main
Reviewed-on: #16
2024-11-08 12:54:01 +00:00
1f9e3e6e23 log less, now that status is around 2024-11-08 13:51:33 +01:00
1e75be6900 remove logging from opam_file (now reported to status) 2024-11-08 13:46:05 +01:00
c3d5c74075 use K.remote, not the entire repo 2024-11-06 13:15:10 +01:00
be87d19797 reset errors, and sort failures 2024-11-06 13:08:28 +01:00
37008e81f3 organize failures into sections 2024-11-06 12:57:43 +01:00
8ba4cfae00 group download failures by error 2024-11-04 18:43:50 +01:00
9c50538877 record and preserve opam file parsing issues 2024-11-04 18:43:21 +01:00
a47193f147 tweaks 2024-11-04 17:33:50 +01:00
1e35bfefbd initialization: potentially rename bad data 2024-11-04 17:33:33 +01:00
a9b8f18192 stream git contents, also make the tarball and the find_urls in one go 2024-11-04 17:17:52 +01:00
2312092e42 first write to a temporary filename, and rename later 2024-11-04 17:17:37 +01:00
4bec3bfbd8 restore checksum failure error 2024-11-04 16:50:42 +01:00
f48cc19fc4 drop superfluous 'unknown' 2024-11-04 16:46:01 +01:00
7689397ac3 remove bad archive list 2024-11-04 16:14:09 +01:00
e59f02a16f always use swap, remove the pending / to_delete stuff 2024-11-04 16:13:36 +01:00
456340562d Use swapfs 2024-11-01 14:35:08 +01:00
f9620e9011 Merge pull request 'Add a download status page, recording:' (#15) from dl-status into main
Reviewed-on: #15
Reviewed-by: Reynir Björnsson <reynir@reynir.dk>
2024-11-01 11:01:51 +00:00
53af2665fa record free bytes in tar archive 2024-10-31 17:45:34 +01:00
f40083692a tweaks 2024-10-31 17:17:34 +01:00
eb95821b2e Add a download status page, recording:
- number of archives on disk
- current downloads
- failed downloads
2024-10-31 11:30:52 +01:00
9bb86b3507 gptar is released now, no need to pin 2024-10-28 19:35:16 +01:00
826cc85b71 Merge pull request 'extra-source' (#14) from extra-source into main
Reviewed-on: #14
Reviewed-by: Reynir Björnsson <reynir@reynir.dk>
2024-10-26 14:42:11 +00:00
9bfde48f43 also handle extra-source, fixes #8 2024-10-24 15:03:35 +02:00