Commit graph

120 commits

Author SHA1 Message Date
4bec3bfbd8 restore checksum failure error 2024-11-04 16:50:42 +01:00
f48cc19fc4 drop superfluous 'unknown' 2024-11-04 16:46:01 +01:00
7689397ac3 remove bad archive list 2024-11-04 16:14:09 +01:00
e59f02a16f always use swap, remove the pending / to_delete stuff 2024-11-04 16:13:36 +01:00
456340562d Use swapfs 2024-11-01 14:35:08 +01:00
f9620e9011 Merge pull request 'Add a download status page, recording:' (#15) from dl-status into main
Reviewed-on: #15
Reviewed-by: Reynir Björnsson <reynir@reynir.dk>
2024-11-01 11:01:51 +00:00
53af2665fa record free bytes in tar archive 2024-10-31 17:45:34 +01:00
f40083692a tweaks 2024-10-31 17:17:34 +01:00
eb95821b2e Add a download status page, recording:
- number of archives on disk
- current downloads
- failed downloads
2024-10-31 11:30:52 +01:00
9bb86b3507 gptar is released now, no need to pin 2024-10-28 19:35:16 +01:00
826cc85b71 Merge pull request 'extra-source' (#14) from extra-source into main
Reviewed-on: #14
Reviewed-by: Reynir Björnsson <reynir@reynir.dk>
2024-10-26 14:42:11 +00:00
9bfde48f43 also handle extra-source, fixes #8 2024-10-24 15:03:35 +02:00
9a2576c423 extract the opam parsing function to a separate module (opam_file.ml / Opam_file)
prepares for handling extra-source as well
2024-10-24 14:45:31 +02:00
8af9f04dd0 Hex-encode computed checksum in log message
Oops!
2024-10-18 10:43:09 +02:00
5accfcfc08 More informative log message 2024-10-18 10:26:43 +02:00
625081abff Merge pull request 'Improve logging of bad checksums' (#7) from log-bad-checksum into main
Reviewed-on: #7
2024-10-17 16:12:30 +00:00
1016c54ee7 Merge pull request 'Demote log level for unavailable packages' (#6) from demote-unavailable into main
Reviewed-on: #6
2024-10-17 16:12:09 +00:00
5938a60289 Improve logging of bad checksums
We now print the computed hash \o/
2024-10-17 17:26:00 +02:00
7a71e095da Demote log level for unavailable packages
Many packages are marked unavailable these days in preparation of
opam-repository cleanup \o/ so let's be a little less noisy.
2024-10-17 17:18:16 +02:00
1241753a8c Merge pull request 'Use GPTar table' (#3) from gptar' into main
Reviewed-on: #3
2024-10-16 09:10:32 +00:00
26643fbcde Add a message on successful --initialize-disk 2024-10-16 11:06:48 +02:00
30266f4e09 Fix sector alignment bug in formatting 2024-10-16 11:05:27 +02:00
068c640dac Reset the partitions when initializing the disk
THIS DESTROYS DATA
2024-10-16 11:05:27 +02:00
719b4ea45d Repo tarball: use level Ustar 2024-10-16 11:05:27 +02:00
ceb4674ec2 Add code for formatting a disk 2024-10-16 11:05:24 +02:00
d36a0714e7 Use GPTar table
We expect the disk to be formatted already.
2024-10-16 10:56:20 +02:00
9ada5c4a94 Merge pull request 'update to mirage 4.8' (#4) from mirage-48 into main
Reviewed-on: #4
2024-10-16 08:40:11 +00:00
58656926e3 update to mirage 4.8 2024-10-11 13:31:48 +02:00
534292ec3f Merge pull request 'Remove unnecessary complexity' (#2) from opam-checksums into main
Reviewed-on: #2
2024-10-03 22:10:08 +00:00
02f9c2f9c7 Remove unnecessary complexity
Opam 2 only supports md5, sha256 and sha512. We don't need the extra
complexity of other hash algorithms. Opam 2 will fail to parse such an
opam file anyway.
2024-10-03 18:10:21 +02:00
a99f7f65a3 Merge pull request 'Upgrade opam-mirror' (#1) from with-new-tar into main
Reviewed-on: #1
2024-10-03 13:42:12 +00:00
6490801ce1 Update dependencies, refactor
- Tar.3.0.0 was released! So we can drop the pin and depend on it
  directly.
- Use digestif in favor of the bygone mirage-crypto hash
- Move most of the hashing logic into archive_checksum.ml
- When checking the checksum we should check the checksum chosen, too. I
  believe this was a bug that was hidden due to opam packages always
  using one of md5, sha256 or sha512 for checksums.
2024-10-03 14:37:37 +02:00
1b1414c5ca Fix the compilation of the unikernel with the last version of ocaml-tar 2024-08-01 13:50:12 +02:00
8fc8f1c62d Update the unikernel with the new version of tar 2024-07-31 14:54:26 +02:00
c2ffbdb891 Update to mirage.4.6.0 2024-07-15 12:22:57 +02:00
e002bf8730 Merge pull request 'Set upper bound on mirage-block-partition' (#29) from partition-bounds into main
Reviewed-on: #29
2024-03-21 10:35:14 +00:00
61be0a3ff5 Set upper bound on mirage-block-partition
In preparation of a breaking change.
2024-03-21 11:24:47 +01:00
b3a74b0c1d Merge pull request 'mirage-kv 6.0.1' (#27) from mirage-kv-6 into main
Reviewed-on: https://git.robur.io/robur/opam-mirror/pulls/27
2023-05-28 14:31:10 +00:00
bbb3d2336a avoid gmp dependency in config.ml -- this is just a dune cache issue 2023-05-27 19:19:27 +02:00
1ebf370a4f compiles now 2023-05-02 16:13:43 +02:00
Robur
fd8ce3be03 more wip 2023-05-02 12:03:25 +00:00
ebeadf69d8 Skip unavailable packages 2023-02-08 16:30:56 +01:00
adf6564385 Print error and exit if we can't get commit id 2023-02-08 16:30:30 +01:00
8c7e71127f Try to remove files before marking for deletion
You know, it might just work...
2023-02-08 16:29:55 +01:00
1a6c6b8f9d Return error on bad checksum when finalizing write 2023-01-31 10:27:25 +01:00
a3128e1de5 Add new hash mismatches 2023-01-26 09:37:17 +01:00
8e326ecbc5 WIP partial writes
If possible, downloads are streamed to disk in the /pending/ directory
in the tar filesystem. If the download is successful and the checksums
match the file is renamed to its sha256 hash. Otherwise, it is moved
under /to-delete/ so it can be deleted by an operator.

Before downloading we check if it has been downloaded before, but we
need to check as well if it is being downloaded (in /pending/) or if we
unsuccessfully downloaded it before (e.g. failed checksum, stored in
/to-delete/).

It is not very elegant code, and it could do with a thorough review or
rewrite.
2023-01-25 11:34:31 +01:00
b76f2997f5 Migrate to mirage-kv 6 2023-01-23 09:13:42 +01:00
93c490bcb5 Update to mirage.4.3.1 and its alpn_client 2023-01-20 14:46:22 +01:00
c46f15baa3 Merge pull request 'Log before and after restoring/fetching git state' (#26) from log-before-git-operation into main
Reviewed-on: https://git.robur.io/robur/opam-mirror/pulls/26
2022-11-17 12:30:07 +00:00