317 lines
No EOL
18 KiB
Text
317 lines
No EOL
18 KiB
Text
<!DOCTYPE html>
|
|
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Conex, establish trust in community repositories</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Conex, establish trust in community repositories" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Conex, establish trust in community repositories</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/package signing" class="tag">package signing</a><a href="/tags/security" class="tag">security</a><a href="/tags/overview" class="tag">overview</a></div><span class="date">Published: 2017-02-16 (last updated: 2021-11-19)</span><article><p>Less than two years after the initial proposal, we're happy to present conex
|
|
0.9.2. Pleas note that this is still work in progress, to be deployed with opam
|
|
2.0 and the <a href="https://github.com/ocaml/opam-repository">opam repository</a>.</p>
|
|
<p><img src="/static/img/conex.png" alt="screenshot" /></p>
|
|
<p><a href="https://github.com/hannesm/conex">Conex</a> is a library to verify and attest release integrity and
|
|
authenticity of a community repository through the use of cryptographic signatures.</p>
|
|
<p>Packages are collected in a community repository to provide an index and
|
|
allowing cross-references. Authors submit their packages to the repository. which
|
|
is curated by a team of janitors. Information
|
|
about a package stored in a repository includes: license, author, releases,
|
|
their dependencies, build instructions, url, tarball checksum. When someone
|
|
publishes a new package, the janitors integrate it into the repository, if it
|
|
compiles and passes some validity checks. For example, its name must not be misleading,
|
|
nor may it be too general.</p>
|
|
<p>Janitors keep an eye on the repository and fix emergent failures. A new
|
|
compiler release, or a release of a package on which other packages depend, might break the compilation of
|
|
a package. Janitors usually fix these problems by adding a patch to the build script, or introducing
|
|
a version constraint in the repository.</p>
|
|
<p><em>Conex</em> ensures that every release of each package has been approved by its author or a quorum of janitors.
|
|
A conex-aware client initially verifies the repository using janitor key fingerprints as anchor.
|
|
Afterwards, the on-disk repository is trusted, and every update is verified (as a patch) individually.
|
|
This incremental verification is accomplished by ensuring all resources
|
|
that the patch modifies result in a valid repository with
|
|
sufficient approvals. Additionally, monotonicity is preserved by
|
|
embedding counters in each resource, and enforcing a counter
|
|
increment after modification.
|
|
This mechanism avoids rollback attacks, when an
|
|
attacker presents you an old version of the repository.</p>
|
|
<p>A timestamping service (NYI) will periodically approve a global view of the
|
|
verified repository, together with a timestamp. This is then used by the client
|
|
to prevent mix-and-match attacks, where an attacker mixes some old packages and
|
|
some new ones. Also, the client is able to detect freeze attacks, since at
|
|
least every day there should be a new signature done by the timestamping service.</p>
|
|
<p>The trust is rooted in digital signatures by package authors. The server which
|
|
hosts the repository does not need to be trusted. Neither does the host serving
|
|
release tarballs.</p>
|
|
<p>If a single janitor would be powerful enough to approve a key for any author,
|
|
compromising one janitor would be sufficient to enroll any new identities,
|
|
modify dependencies, build scripts, etc. In conex, a quorum of janitors (let's
|
|
say 3) have to approve such changes. This is different from current workflows,
|
|
where a single janitor with access to the repository can merge fixes.</p>
|
|
<p>Conex adds metadata, in form of resources, to the repository to ensure integrity and
|
|
authenticity. There are different kinds of resources:</p>
|
|
<ul>
|
|
<li><em>Authors</em>, consisting of a unique identifier, public key(s), accounts.
|
|
</li>
|
|
<li><em>Teams</em>, sharing the same namespace as authors, containing a set of members.
|
|
</li>
|
|
<li><em>Authorisation</em>, one for each package, describing which identities are authorised for the package.
|
|
</li>
|
|
<li><em>Package index</em>, for each package, listing all releases.
|
|
</li>
|
|
<li><em>Release</em>, for each release, listing checksums of all data files.
|
|
</li>
|
|
</ul>
|
|
<p>Modifications to identities and authorisations need to be approved by a quorum
|
|
of janitors, package index and release files can be modified either by an authorised
|
|
id or by a quorum of janitors.</p>
|
|
<h2>Documentation</h2>
|
|
<p><a href="https://hannesm.github.io/conex/doc/">API documentation</a> is
|
|
available online, also a <a href="https://hannesm.github.io/conex/coverage/">coverage
|
|
report</a>.</p>
|
|
<p>We presented an <a href="https://github.com/hannesm/conex-paper/raw/master/paper.pdf">abstract at OCaml
|
|
2016</a> about an
|
|
earlier design.</p>
|
|
<p>Another article on an <a href="http://opam.ocaml.org/blog/Signing-the-opam-repository/">earlier design (from
|
|
2015)</a> is also
|
|
available.</p>
|
|
<p>Conex is inspired by <a href="https://theupdateframework.github.io/">the update
|
|
framework</a>, especially on their <a href="https://isis.poly.edu/~jcappos/papers/samuel_tuf_ccs_2010.pdf">CCS 2010
|
|
paper</a>, and
|
|
adapted to the opam repository.</p>
|
|
<p>The <a href="https://github.com/theupdateframework/tuf/blob/develop/docs/tuf-spec.txt">TUF
|
|
spec</a>
|
|
has a good overview of attacks and threat model, both of which are shared by conex.</p>
|
|
<h2>What's missing</h2>
|
|
<ul>
|
|
<li>See <a href="https://github.com/hannesm/conex/issues/7">issue 7</a> for a laundry list
|
|
</li>
|
|
<li>Timestamping service
|
|
</li>
|
|
<li>Key revocation and rollover
|
|
</li>
|
|
<li>Tool to approve a PR (for janitors)
|
|
</li>
|
|
<li>Camelus like opam-repository check bot
|
|
</li>
|
|
<li>Integration into release management systems
|
|
</li>
|
|
</ul>
|
|
<h2>Getting started</h2>
|
|
<p>At the moment, our <a href="https://github.com/ocaml/opam-repository">opam repository</a>
|
|
does not include any metadata needed for signing. We're in a bootstrap phase:
|
|
we need you to generate a keypair, claim your packages, and approve your releases.</p>
|
|
<p>We cannot verify the main opam repository yet, but opam2 has support for a
|
|
<a href="http://opam.ocaml.org/doc/2.0/Manual.html#configfield-repository-validation-command"><code>repository validation command</code></a>,
|
|
builtin, which should then call out to <code>conex_verify</code> (there is a <code>--nostrict</code>
|
|
flag for the impatient). There is also an <a href="https://github.com/hannesm/testrepo">example repository</a> which uses the opam validation command.</p>
|
|
<p>To reduce the manual work, we analysed 7000 PRs of the opam repository within
|
|
the last 4.5 years (more details <a href="/Posts/Maintainers">here</a>.
|
|
This resulted in an educated guess who are the people
|
|
modifying each package, which we use as a basis whom to authorise for
|
|
which packages. Please check with <code>conex_author status</code> below whether your team
|
|
membership and authorised packages were inferred correctly.</p>
|
|
<p>Each individual author - you - need to generate their private key, submit
|
|
their public key and starts approving releases (and old ones after careful
|
|
checking that the build script, patches, and tarball checksum are valid).
|
|
Each resource can be approved in multiple versions at the same time.</p>
|
|
<h3>Installation</h3>
|
|
<p>TODO: remove clone once <a href="https://github.com/ocaml/opam-repository/pull/8494">PR 8494</a> is merged.</p>
|
|
<pre><code class="language-bash">$ git clone -b auth https://github.com/hannesm/opam-repository.git repo
|
|
$ opam install conex
|
|
$ cd repo
|
|
</code></pre>
|
|
<p>This will install conex, namely command line utilities, <code>conex_author</code> and
|
|
<code>conex_verify_nocrypto</code>/<code>conex_verify_openssl</code>. All files read and written by conex are in the usual
|
|
opam file format. This means can always manually modify them (but be careful,
|
|
modifications need to increment counters, add checksums, and be signed). Conex
|
|
does not deal with git, you have to manually <code>git add</code> files and open pull
|
|
requests.</p>
|
|
<h3>Author enrollment</h3>
|
|
<p>For the opam repository, we will use GitHub ids as conex ids. Thus, your conex
|
|
id and your GitHub id should match up.</p>
|
|
<pre><code class="language-bash">repo$ conex_author init --repo ~/repo --id hannesm
|
|
Created keypair hannesm. Join teams, claim your packages, sign your approved resources and open a PR :)
|
|
</code></pre>
|
|
<p>This attempts to parse <code>~/repo/id/hannesm</code>, errors if it is a team or an author
|
|
with a publickey. Otherwise it generates a keypair, writes the private part as
|
|
<code>home.hannes.repo.hannesm.private</code> (the absolute path separated by dots,
|
|
followed by your id, and <code>private</code> - if you move your repository, rename your
|
|
private key) into <code>~/.conex/</code>, the checksums of the public part and your
|
|
accounts into <code>~/repo/id/hannesm</code>. See <code>conex_author help init</code> for more
|
|
options (esp. additional verbosity <code>-v</code> can be helpful).</p>
|
|
<pre><code class="language-bash">repo$ git status -s
|
|
M id/hannesm
|
|
|
|
repo$ git diff //abbreviated output
|
|
- ["counter" 0x0]
|
|
+ ["counter" 0x1]
|
|
|
|
- ["resources" []]
|
|
+ [
|
|
+ "resources"
|
|
+ [
|
|
+ [
|
|
+ ["typ" "key"]
|
|
+ ["name" "hannesm"]
|
|
+ ["index" 0x1]
|
|
+ ["digest" ["SHA256" "ht9ztjjDwWwD/id6LSVi7nKqVyCHQuQu9ORpr8Zo2aY="]]
|
|
+ ]
|
|
+ [
|
|
+ ["typ" "account"]
|
|
+ ["name" "hannesm"]
|
|
+ ["index" 0x2]
|
|
+ ["digest" ["SHA256" "aCsktJ5M9PI6T+m1NIQtuIFYILFkqoHKwBxwvuzpuzg="]]
|
|
+ ]
|
|
+
|
|
+keys: [
|
|
+ [
|
|
+ [
|
|
+ "RSA"
|
|
+ """
|
|
+-----BEGIN PUBLIC KEY-----
|
|
+MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAyUhArwt4XcxLanARyH9S
|
|
...
|
|
+9KQdg6QnLsQh/j74QKLOZacCAwEAAQ==
|
|
+-----END PUBLIC KEY-----"""
|
|
+ 0x58A3419F
|
|
+ ]
|
|
+ [
|
|
+ 0x58A79A1D
|
|
+ "RSA-PSS-SHA256"
|
|
+ "HqqicsDx4hG9pFM5E7"
|
|
+ ]
|
|
+ ]
|
|
+]
|
|
</code></pre>
|
|
<h3>Status</h3>
|
|
<p>If you have a single identity and contribute to a single signed opam repository,
|
|
you don't need to specify <code>--id</code> or <code>--repo</code> from now on.</p>
|
|
<p>The <code>status</code> subcommand presents an author-specific view on the repository. It
|
|
lists the own public keys, team membership, queued resources, and authorised
|
|
packages.</p>
|
|
<p>The opam repository is in a transitionary state, we explicitly pass <code>--quorum 0</code>, which means that every checksum is valid (approved by a quorum of 0
|
|
janitors).</p>
|
|
<pre><code class="language-bash">repo$ conex_author status --quorum 0 arp
|
|
author hannesm #1 (created 0) verified 3 resources, 0 queued
|
|
4096 bit RSA key created 1487094175 approved, SHA256: ht9ztjjDwWwD/id6LSVi7nKqVyCHQuQu9ORpr8Zo2aY=
|
|
account GitHub hannesm approved
|
|
account email hannes@mehnert.org approved
|
|
package arp authorisation approved
|
|
conex_author: [ERROR] package index arp was not found in repository
|
|
</code></pre>
|
|
<p>This shows your key material and accounts, team membership and packages you are
|
|
authorised to modify (inferred as described
|
|
<a href="/Posts/Maintainers">here</a>.</p>
|
|
<p>The <code>--noteam</code> argument limits the package list to only these you are personally
|
|
authorised for. The <code>--id</code> argument presents you with a view of another author,
|
|
or from a team perspective. The positional argument is a prefix matching on
|
|
package names (leave empty for all).</p>
|
|
<h3>Resource approval</h3>
|
|
<p>Each resource needs to be approved individually. Each author has a local queue
|
|
for to-be-signed resources, which is extended with <code>authorisation</code>, <code>init</code>,
|
|
<code>key</code>, <code>release</code>, and <code>team</code> (all have a <code>--dry-run</code> flag). The queue can be
|
|
dropped using <code>conex_author reset</code>. Below shown is <code>conex_author sign</code>, which
|
|
let's you interactively approve queued resources and cryptopgraphically signs
|
|
your approved resources afterwards.</p>
|
|
<p>The output of <code>conex_author status</code> listed an authorisation for <code>conf-gsl</code>,
|
|
which I don't feel responsible for. Let's drop my privileges:</p>
|
|
<pre><code class="language-bash">repo$ conex_author authorisation conf-gsl --remove -m hannesm
|
|
modified authorisation and added resource to your queue.
|
|
</code></pre>
|
|
<p>I checked my arp release careful (checksums of tarballs are correct, opam files
|
|
do not execute arbitrary shell code, etc.), and approve this package and its
|
|
single release:</p>
|
|
<pre><code class="language-bash">repo$ conex_author release arp
|
|
conex_author.native: [WARNING] package index arp was not found in repository
|
|
conex_author.native: [WARNING] release arp.0.1.1 was not found in repository
|
|
wrote release and added resources to your queue.
|
|
</code></pre>
|
|
<p>Once finished with joining and leaving teams (using the <code>team</code> subcommand),
|
|
claiming packages (using the <code>authorisation</code> subcommand), and approve releases
|
|
(using the <code>release</code> subcommand), you have to cryprographically sign your queued
|
|
resource modifications:</p>
|
|
<pre><code class="language-bash">repo$ conex_author sign
|
|
release arp.0.1.1 #1 (created 1487269425)
|
|
[descr: SHA256: aCsNvcj3cBKO0GESWG4r3AzoUEnI0pHGSyEDYNPouoE=;
|
|
opam: SHA256: nqy6lD1UP+kXj3+oPXLt2VMUIENEuHMVlVaG2V4z3p0=;
|
|
url: SHA256: FaUPievda6cEMjNkWdi0kGVK7t6EpWGfQ4q2NTSTcy0=]
|
|
approved (yes/No)?
|
|
package arp #1 (created 1487269425) [arp.0.1.1]
|
|
approved (yes/No)?y
|
|
authorisation conf-gsl #1 (created 0) empty
|
|
approved (yes/No)?y
|
|
wrote hannesm to disk
|
|
|
|
repo$ conex_author status --quorum 0 arp
|
|
author hannesm #1 (created 0) verified 7 resources, 0 queued
|
|
4096 bit RSA key created 1487094175 approved, SHA256: ht9ztjjDwWwD/id6LSVi7nKqVyCHQuQu9ORpr8Zo2aY=
|
|
account GitHub hannesm approved
|
|
account email hannes@mehnert.org approved
|
|
package arp authorisation approved package index approved
|
|
release arp.0.1.1: approved
|
|
</code></pre>
|
|
<p>If you now modify anything in <code>packages/arp</code> (add subdirectories, modify opam,
|
|
etc.), this will not be automatically approved (see below for how to do this).</p>
|
|
<p>You manually need to <code>git add</code> some created files.</p>
|
|
<pre><code class="language-bash">repo$ git status -s
|
|
M id/hannesm
|
|
M packages/conf-gsl/authorisation
|
|
?? packages/arp/arp.0.1.1/release
|
|
?? packages/arp/package
|
|
|
|
repo$ git add packages/arp/arp.0.1.1/release packages/arp/package
|
|
repo$ git commit -m "hannesm key enrollment and some fixes" id packages
|
|
</code></pre>
|
|
<p>Now push this to your fork, and open a PR on opam-repository!</p>
|
|
<h3>Editing a package</h3>
|
|
<p>If you need to modify a released package, you modify the opam file (as before,
|
|
e.g. introducing a conflict with a dependency), and then approve the
|
|
modifications. After your local modifications, <code>conex_author status</code> will
|
|
complain:</p>
|
|
<pre><code class="language-bash">repo$ conex_author status arp --quorum 0
|
|
package arp authorisation approved package index approved
|
|
release arp.0.1.1: checksums for arp.0.1.1 differ, missing on disk: empty, missing in checksums file: empty, checksums differ: [have opam: SHA256: QSGUU9HdPOrwoRs6XJka4cZpd8h+8NN1Auu5IMN8ew4= want opam: SHA256: nqy6lD1UP+kXj3+oPXLt2VMUIENEuHMVlVaG2V4z3p0=]
|
|
|
|
repo$ conex_author release arp.0.1.1
|
|
released and added resources to your resource list.
|
|
|
|
repo$ conex_author sign
|
|
release arp.0.1.1 #1 (created 1487269943)
|
|
[descr: SHA256: aCsNvcj3cBKO0GESWG4r3AzoUEnI0pHGSyEDYNPouoE=;
|
|
opam: SHA256: QSGUU9HdPOrwoRs6XJka4cZpd8h+8NN1Auu5IMN8ew4=;
|
|
url: SHA256: FaUPievda6cEMjNkWdi0kGVK7t6EpWGfQ4q2NTSTcy0=]
|
|
approved (yes/No)? y
|
|
wrote hannesm to disk
|
|
</code></pre>
|
|
<p>The <code>release</code> subcommand recomputed the checksums, incremented the counter, and
|
|
added it to your queue. The <code>sign</code> command signed the approved resource.</p>
|
|
<pre><code class="language-bash">repo$ git status -s
|
|
M id/hannesm
|
|
M packages/arp/arp.0.1.1/opam
|
|
M packages/arp/arp.0.1.1/package
|
|
|
|
repo$ git commit -m "fixed broken arp package" id packages
|
|
</code></pre>
|
|
<h3>Janitor tools</h3>
|
|
<p>Janitors need to approve teams, keys, accounts, and authorisations.</p>
|
|
<p>To approve resources which are already in the repository on disk,
|
|
the <code>key</code> subcommand queues approval of keys and accounts of the provided author:</p>
|
|
<pre><code class="language-bash">repo$ conex_author key avsm
|
|
added keys and accounts to your resource list.
|
|
</code></pre>
|
|
<p>The <code>authorisation</code> subcommand, and <code>team</code> subcommand behave similarly for
|
|
authorisations and teams.</p>
|
|
<p>Bulk operations are supported as well:</p>
|
|
<pre><code class="language-bash">conex_author authorisation all
|
|
</code></pre>
|
|
<p>This will approve all authorisations of the repository which are not yet
|
|
approved by you. Similar for the <code>key</code> and <code>team</code> subcommands, which also
|
|
accept <code>all</code>.</p>
|
|
<p>Don't forget to <code>conex_author sign</code> afterwards (or <code>yes | conex_author sign</code>).</p>
|
|
<h3>Verification</h3>
|
|
<p>The two command line utlities, <code>conex_verify_openssl</code> and
|
|
<code>conex_verify_nocrypto</code> contain the same logic and same command line arguments.</p>
|
|
<p>For bootstrapping purposes (<code>nocrypto</code> is an opam package with dependencies),
|
|
<code>conex_verify_openssl</code> relies on the openssl command line tool (version 1.0.0
|
|
and above) for digest computation and verification of the RSA-PSS signature.</p>
|
|
<p>The goal is to use the opam2 provided hooks, but before we have signatures we
|
|
cannot enable them.</p>
|
|
<p>See the <a href="https://github.com/hannesm/testrepo">example repository</a> for initial
|
|
verification experiments, and opam2 integration.</p>
|
|
<p>I'm interested in feedback, please open an issue on the <a href="https://github.com/hannesm/conex">conex
|
|
repository</a>. This article itself is stored as
|
|
Markdown <a href="https://git.robur.io/hannes/hannes.robur.coop">in a different repository</a>.</p>
|
|
</article></div></div></main></body></html> |