diff --git a/Posts/ReproducibleOPAM b/Posts/ReproducibleOPAM index a9c3610..40680b2 100644 --- a/Posts/ReproducibleOPAM +++ b/Posts/ReproducibleOPAM @@ -62,7 +62,7 @@ As mentioned above, if locale is set, ocamlgraph needs to be patched to emit a ( The OCaml program [`crunch`](https://github.com/mirage/ocaml-crunch) embeds a subdirectory as OCaml code into a binary, which we use in MirageOS quite regularly for static assets, etc. This plays in several ways into reproducibility: on the one hand, it needs a timestamp for its `last_modified` functionality (and adheres since [June 2018](https://github.com/mirage/ocaml-crunch/pull/45) to the `SOURCE_DATE_EPOCH` spec, thanks to Xavier Clerc). On the other hand, it used before version 3.2.0 (released Dec 14th) hashtables for storing the file contents, where iteration is not deterministic (the insertion is not sorted), [fixed in PR #51](https://github.com/mirage/ocaml-crunch/pull/51) by using a Map instead. -In functoria, a tool used to configure MirageOS devices and their dependencies, can emit a list of opam packages which were required to build the unikernel. This uses `opam list --required-by --installed --rec `, which calls the cudf solver, that is during the rebuild dropping some packages. The [PR #189](https://github.com/mirage/functoria/pull/189) avoids by not using the `--rec` argument, but manually computing the fixpoint. +In functoria, a tool used to configure MirageOS devices and their dependencies, can emit a list of opam packages which were required to build the unikernel. This uses `opam list --required-by --installed --rec `, which uses the cudf graph ([thanks to Raja for explanation](https://github.com/mirage/functoria/pull/189#issuecomment-566696426)), that is during the rebuild dropping some packages. The [PR #189](https://github.com/mirage/functoria/pull/189) avoids by not using the `--rec` argument, but manually computing the fixpoint. Certainly, the choice of environment variables, and whether to vary them (as [debian does](https://tests.reproducible-builds.org/debian/index_variations.html)) or to not define them (or normalise) while building, is arguably. Since MirageOS does neither support time zone nor internationalisation, there is no need to prematurely solving this issue. On related note, even with different locale settings, MirageOS unikernels are reproducible apart from an [issue in ocamlgraph #90](https://github.com/backtracking/ocamlgraph/pull/90) embedding the output of [`date`](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/date.html), which is different depending on `LANG` and locale (`LC_*`) settings. @@ -74,7 +74,7 @@ I only tested a certain subset of opam packages and MirageOS unikernels, mainly As mentioned above, more environment, such as the CPU features, and external system packages, should be captured in the build environment. -When comparing OCaml libraries, some output files (cmt / cmti / cma / cmxa) are not deterministic, but contain minimal diverge where I was not able to spot the root cause. It would be great to fix this, likely in the OCaml compiler distribution. Since the final result, the binary I'm interested in, is not affected by non-identical intermediate build products, I hope someone (you?) is interested in improving on this side. OCaml bytecode output also seems to be non-deterministic. +When comparing OCaml libraries, some output files (cmt / cmti / cma / cmxa) are not deterministic, but contain minimal diverge where I was not able to spot the root cause. It would be great to fix this, likely in the OCaml compiler distribution. Since the final result, the binary I'm interested in, is not affected by non-identical intermediate build products, I hope someone (you?) is interested in improving on this side. OCaml bytecode output also seems to be non-deterministic. There is [a discussion on the coq issue tracker](https://github.com/coq/coq/issues/11229) which may be related. In contrast to initial plans, I did not used the [`BUILD_PATH_PREFIX_MAP`](https://reproducible-builds.org/specs/build-path-prefix-map/) environment variable, which is implemented in OCaml by [PR #1515](https://github.com/ocaml/ocaml/pull/1515) (and followups). The main reasons are that something in the OCaml toolchain (I suspect the bytecode interpreter) needed absolute paths to find libraries, thus I'd need a symlink from the left-hand side to the current build directory, which was tedious. Also, my installed assembler does not respect the build path prefix map, and BUILD_PATH_PREFIX_MAP is not widely supported. See e.g. the Debian [zarith](https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope-results/ocaml-zarith.html) package with different build paths and its effects on the binary.