robur/blog.robur.coop
4
0
Fork 1
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

Built from 33ef4926e2

Browse source
This commit is contained in:
Reynir Björnsson 2024-10-16 09:48:56 +00:00
parent 40ec2db552
commit 890055c942
15 changed files with 354 additions and 293 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

50
articles/2024-02-03-python-str-repr.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - Python&apos;s `str.__repr__()` Robur's blogPython&apos;s `str.__repr__()`
</title> </title>
<meta name="description" content="Reimplementing Python string escaping in OCaml"> <meta name="description" content="Reimplementing Python string escaping in OCaml">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,11 +20,11 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>Python&apos;s `str.__repr__()`</h1> <h1>Python&apos;s `str.__repr__()`</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/python.html">python</a></li><li><a href="/tags/unicode.html">unicode</a></li></ul><p>Sometimes software is written using whatever built-ins you find in your programming language of choice. <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Python">Python</a></li><li><a href="https://blog.robur.coop/tags.html#tag-unicode">unicode</a></li></ul><p>Sometimes software is written using whatever built-ins you find in your programming language of choice.
This is usually great! This is usually great!
However, it can happen that you depend on the precise semantics of those built-ins. However, it can happen that you depend on the precise semantics of those built-ins.
This can be a problem if those semantics become important to your software and you need to port it to another programming language. This can be a problem if those semantics become important to your software and you need to port it to another programming language.
@ -49,18 +48,18 @@ In that case OCaml would escape the double quote with a backslash (<code>\&quot;
So a regular expression substitution was added to replace the escape sequence with just a double quote. So a regular expression substitution was added to replace the escape sequence with just a double quote.
This pattern of finding small differences between Python and OCaml escaping had been repeated, This pattern of finding small differences between Python and OCaml escaping had been repeated,
and eventually I decided to take a more rigorous approach to it.</p> and eventually I decided to take a more rigorous approach to it.</p>
<h2>What is a string?</h2> <h2 id="what-is-a-string"><a class="anchor" aria-hidden="true" href="#what-is-a-string"></a>What is a string?</h2>
<p>First of all, what is a string? In Python? And in OCaml? <p>First of all, what is a string? In Python? And in OCaml?
In OCaml a string is just a sequence of bytes. In OCaml a string is just a sequence of bytes.
Any bytes, even <code>NUL</code> bytes. Any bytes, even <code>NUL</code> bytes.
There is no concept of unicode in OCaml strings.<br> There is no concept of unicode in OCaml strings.<br>
In Python there is the <code>str</code> type which is a sequence of Unicode code points<sup><a href="#fn-python-bytes" id="ref-1-fn-python-bytes" role="doc-noteref" class="fn-label">[1]</a></sup>. In Python there is the <code>str</code> type which is a sequence of Unicode code points[^python-bytes].
I can recommend reading Daniel Bünzli's <a href="https://ocaml.org/p/uucp/13.0.0/doc/unicode.html#minimal">minimal introduction to Unicode</a>. I can recommend reading Daniel Bünzli's <a href="https://ocaml.org/p/uucp/13.0.0/doc/unicode.html#minimal">minimal introduction to Unicode</a>.
Already here there is a significant gap in semantics between Python and OCaml. Already here there is a significant gap in semantics between Python and OCaml.
For many practical purposes we can get away with using the OCaml <code>string</code> type and treating it as a UTF-8 encoded Unicode string. For many practical purposes we can get away with using the OCaml <code>string</code> type and treating it as a UTF-8 encoded Unicode string.
This is what I will do as in both the Python code and the OCaml code the data being read is a UTF-8 (or often only the US ASCII subset) encoded string.</p> This is what I will do as in both the Python code and the OCaml code the data being read is a UTF-8 (or often only the US ASCII subset) encoded string.</p>
<h2>What does a string literal look like?</h2> <h2 id="what-does-a-string-literal-look-like"><a class="anchor" aria-hidden="true" href="#what-does-a-string-literal-look-like"></a>What does a string literal look like?</h2>
<h3>OCaml</h3> <h3 id="ocaml"><a class="anchor" aria-hidden="true" href="#ocaml"></a>OCaml</h3>
<p>I will not dive too deep into the details of OCaml string literals, and focus mostly on how they are escaped by the language built-ins (<code>String.escaped</code>, <code>Printf.printf &quot;%S&quot;</code>). <p>I will not dive too deep into the details of OCaml string literals, and focus mostly on how they are escaped by the language built-ins (<code>String.escaped</code>, <code>Printf.printf &quot;%S&quot;</code>).
Normal printable ASCII is printed as-is. Normal printable ASCII is printed as-is.
That is, letters, numbers and other symbols except for backslash and double quote. That is, letters, numbers and other symbols except for backslash and double quote.
@ -71,7 +70,7 @@ Finally I also want to mention the Unicode code point escape sequence <code>\u{3
While the escape functions do not use it, it will become handy later on. While the escape functions do not use it, it will become handy later on.
Illegal escape sequences (escape sequences that are not recognized) will emit a warning but otherwise result in the escape sequence as-is. Illegal escape sequences (escape sequences that are not recognized) will emit a warning but otherwise result in the escape sequence as-is.
It is common to compile OCaml programs with warnings-as-errors, however.</p> It is common to compile OCaml programs with warnings-as-errors, however.</p>
<h3>Python</h3> <h3 id="python"><a class="anchor" aria-hidden="true" href="#python"></a>Python</h3>
<p>Python has a number of different string literals and string-like literals. <p>Python has a number of different string literals and string-like literals.
They all use single quote or double quote to delimit the string (or string-like) literals. They all use single quote or double quote to delimit the string (or string-like) literals.
There is a preference towards single quotes in <code>str.__repr__()</code>. There is a preference towards single quotes in <code>str.__repr__()</code>.
@ -82,7 +81,7 @@ The string literal can optionally have a prefix character that modifies what typ
That means backslash escape sequences are not interpreted. That means backslash escape sequences are not interpreted.
In my experiments they seem to be quasi-interpreted, however! In my experiments they seem to be quasi-interpreted, however!
The string <code>r&quot;\&quot;</code> is considered unterminated! The string <code>r&quot;\&quot;</code> is considered unterminated!
But <code>r&quot;\&quot;&quot;</code> is fine as is interpreted as <code>'\\&quot;'</code><sup><a href="#fn-raw-escape-example" id="ref-1-fn-raw-escape-example" role="doc-noteref" class="fn-label">[2]</a></sup>. But <code>r&quot;\&quot;&quot;</code> is fine as is interpreted as <code>'\\&quot;'</code>[^raw-escape-example].
Why this is the case I have not found a good explanation for.</p> Why this is the case I have not found a good explanation for.</p>
<p>The <code>b</code>-prefixed strings are <code>bytes</code> literals. <p>The <code>b</code>-prefixed strings are <code>bytes</code> literals.
This is close to OCaml strings.</p> This is close to OCaml strings.</p>
@ -94,7 +93,7 @@ There is as far as I know <strong>no</strong> decimal notation.
The output of <code>str.__repr__()</code> uses the hexadecimal notation over the octal notation. The output of <code>str.__repr__()</code> uses the hexadecimal notation over the octal notation.
As Python strings are Unicode code point sequences we need more than two hexadecimal digits to be able to represent all valid &quot;characters&quot;. As Python strings are Unicode code point sequences we need more than two hexadecimal digits to be able to represent all valid &quot;characters&quot;.
Thus there are the longer <code>\u0032</code> and the longest <code>\U00000032</code>.</p> Thus there are the longer <code>\u0032</code> and the longest <code>\U00000032</code>.</p>
<h2>Intermezzo</h2> <h2 id="intermezzo"><a class="anchor" aria-hidden="true" href="#intermezzo"></a>Intermezzo</h2>
<p>While studying Python string literals I discovered several odd corners of the syntax and semantics besides the raw string quasi-escape sequence mentioned earlier. <p>While studying Python string literals I discovered several odd corners of the syntax and semantics besides the raw string quasi-escape sequence mentioned earlier.
One fact is that Python doesn't have a separate character or Unicode code point type. One fact is that Python doesn't have a separate character or Unicode code point type.
Instead, a character is a one element string. Instead, a character is a one element string.
@ -118,7 +117,7 @@ Well, bytes is a byte sequence and <code>b&quot;\u0032&quot;</code> is not inter
Writing <code>&quot;\xff&quot;.encode()</code> which encodes the string <code>&quot;\xff&quot;</code> to UTF-8 is <strong>not</strong> the same as <code>b&quot;\xff&quot;</code>. Writing <code>&quot;\xff&quot;.encode()</code> which encodes the string <code>&quot;\xff&quot;</code> to UTF-8 is <strong>not</strong> the same as <code>b&quot;\xff&quot;</code>.
The bytes <code>&quot;\xff&quot;</code> consist of a single byte with decimal value 255, The bytes <code>&quot;\xff&quot;</code> consist of a single byte with decimal value 255,
and the Unicode wizards reading will know that the Unicode code point 255 (or U+FF) is encoded in two bytes in UTF-8.</p> and the Unicode wizards reading will know that the Unicode code point 255 (or U+FF) is encoded in two bytes in UTF-8.</p>
<h2>Where is the Python code?</h2> <h2 id="where-is-the-python-code"><a class="anchor" aria-hidden="true" href="#where-is-the-python-code"></a>Where is the Python code?</h2>
<p>Finding the implementation of <code>str.__repr__()</code> turned out to not be so easy. <p>Finding the implementation of <code>str.__repr__()</code> turned out to not be so easy.
In the end I asked on the Internet and got a link to <a href="https://github.com/python/cpython/blob/963904335e579bfe39101adf3fd6a0cf705975ff/Objects/unicodeobject.c#L12245-L12405">cpython's <code>Objects/unicodeobject.c</code></a>. In the end I asked on the Internet and got a link to <a href="https://github.com/python/cpython/blob/963904335e579bfe39101adf3fd6a0cf705975ff/Objects/unicodeobject.c#L12245-L12405">cpython's <code>Objects/unicodeobject.c</code></a>.
And holy cow! And holy cow!
@ -138,14 +137,14 @@ The first 32 characters and the last US ASCII character (DEL or <code>\x7f</code
I then wrote some simple tests by hand. I then wrote some simple tests by hand.
Then I discovered the OCaml <a href="https://github.com/zshipko/ocaml-py">py</a> library which provides bindings to Python from OCaml. Then I discovered the OCaml <a href="https://github.com/zshipko/ocaml-py">py</a> library which provides bindings to Python from OCaml.
Great! This I can use to test my implementation against Python!</p> Great! This I can use to test my implementation against Python!</p>
<h2>How about Unicode?</h2> <h2 id="how-about-unicode"><a class="anchor" aria-hidden="true" href="#how-about-unicode"></a>How about Unicode?</h2>
<p>For the non-ascii characters (or code points rather) they are either considered <em>printable</em> or <em>non-printable</em>. <p>For the non-ascii characters (or code points rather) they are either considered <em>printable</em> or <em>non-printable</em>.
For now let's look at what that means for the output. For now let's look at what that means for the output.
A printable character is copied as-is. A printable character is copied as-is.
That is, there is no escaping done. That is, there is no escaping done.
Non-printable characters must be escaped, and python wil use <code>\xHH</code>, <code>\uHHHH</code> or <code>\UHHHHHHHH</code> depending on how many hexadecimal digits are necessary to represent the code point. Non-printable characters must be escaped, and python wil use <code>\xHH</code>, <code>\uHHHH</code> or <code>\UHHHHHHHH</code> depending on how many hexadecimal digits are necessary to represent the code point.
That is, the latin-1 subset of ASCII (<code>0x80</code>-<code>0xff</code>) can be represented using <code>\xHH</code> and neither <code>\u00HH</code> nor <code>\U000000HH</code> will be used etc.</p> That is, the latin-1 subset of ASCII (<code>0x80</code>-<code>0xff</code>) can be represented using <code>\xHH</code> and neither <code>\u00HH</code> nor <code>\U000000HH</code> will be used etc.</p>
<h3>What is a printable Unicode character?</h3> <h3 id="what-is-a-printable-unicode-character"><a class="anchor" aria-hidden="true" href="#what-is-a-printable-unicode-character"></a>What is a printable Unicode character?</h3>
<p>In the cpython <a href="https://github.com/python/cpython/blob/963904335e579bfe39101adf3fd6a0cf705975ff/Objects/unicodeobject.c#L12245-L12405">function</a> mentioned earlier they use the function <code>Py_UNICODE_ISPRINTABLE</code>. <p>In the cpython <a href="https://github.com/python/cpython/blob/963904335e579bfe39101adf3fd6a0cf705975ff/Objects/unicodeobject.c#L12245-L12405">function</a> mentioned earlier they use the function <code>Py_UNICODE_ISPRINTABLE</code>.
I had a local clone of the cpython git repository where I ran <code>git grep Py_UNICODE_ISPRINTABLE</code> to find information about it. I had a local clone of the cpython git repository where I ran <code>git grep Py_UNICODE_ISPRINTABLE</code> to find information about it.
In <a href="https://github.com/python/cpython/blob/963904335e579bfe39101adf3fd6a0cf705975ff/Doc/c-api/unicode.rst?plain=1#L257-L265">unicode.rst</a> I found a documentation string for the function that describes it to return false if the character is nonprintable with the definition of nonprintable as the code point being in the categories &quot;Other&quot; or &quot;Separator&quot; in the Unicode character database <strong>with the exception of ASCII space</strong> (U+20 or <code> </code>).</p> In <a href="https://github.com/python/cpython/blob/963904335e579bfe39101adf3fd6a0cf705975ff/Doc/c-api/unicode.rst?plain=1#L257-L265">unicode.rst</a> I found a documentation string for the function that describes it to return false if the character is nonprintable with the definition of nonprintable as the code point being in the categories &quot;Other&quot; or &quot;Separator&quot; in the Unicode character database <strong>with the exception of ASCII space</strong> (U+20 or <code> </code>).</p>
@ -234,7 +233,7 @@ Not only is the code that instigated this journey highly dependent on Python-spe
</code></pre> </code></pre>
<p>Great! I modify the test suite to first detect the unicode version python uses and then pass that version to the OCaml function. <p>Great! I modify the test suite to first detect the unicode version python uses and then pass that version to the OCaml function.
Now I can't find anymore failing test cases!</p> Now I can't find anymore failing test cases!</p>
<h2>Epilogue</h2> <h2 id="epilogue"><a class="anchor" aria-hidden="true" href="#epilogue"></a>Epilogue</h2>
<p>What can we learn from this? <p>What can we learn from this?
It is easy to say in hindsight that a different representation should have been chosen. It is easy to say in hindsight that a different representation should have been chosen.
However, arriving at this insight takes time. However, arriving at this insight takes time.
@ -248,7 +247,7 @@ Below is the output of <code>help(str.__repr__)</code>:</p>
</code></pre> </code></pre>
<p>Language and (standard) library designers could consider whether the slightly nicer looking strings are worth the added complexity users eventually are going to rely on - inadvertently or not. <p>Language and (standard) library designers could consider whether the slightly nicer looking strings are worth the added complexity users eventually are going to rely on - inadvertently or not.
I do think strings and bytes in Python are a bit too complex. I do think strings and bytes in Python are a bit too complex.
It is not easy to get a language lawyer<sup><a href="#fn-language-lawyer" id="ref-1-fn-language-lawyer" role="doc-noteref" class="fn-label">[3]</a></sup> level understanding. It is not easy to get a language lawyer[^language-lawyer] level understanding.
In my opinion it is a mistake to not at least print a warning if there are illegal escape sequences - especially considering there are escape sequences that are valid in one string literal but not another.</p> In my opinion it is a mistake to not at least print a warning if there are illegal escape sequences - especially considering there are escape sequences that are valid in one string literal but not another.</p>
<p>Unfortunately it is often the case that to get a precise specification it is necessary to look at the implementation. <p>Unfortunately it is often the case that to get a precise specification it is necessary to look at the implementation.
For testing your implementation hand-written tests are good. For testing your implementation hand-written tests are good.
@ -262,15 +261,10 @@ It may be the last time I need to understand Python's <code>str.__repr__()</code
I have documented the code to make it more approachable and maintainable by others. I have documented the code to make it more approachable and maintainable by others.
Hopefully it is not something that you need, but in case it is useful to you it is licensed under a permissive license.</p> Hopefully it is not something that you need, but in case it is useful to you it is licensed under a permissive license.</p>
<p>If you have a project in OCaml or want to port something to OCaml and would like help from me and my colleagues at <a href="https://robur.coop/">Robur</a> please <a href="https://robur.coop/Contact">get in touch</a> with us and we will figure something out.</p> <p>If you have a project in OCaml or want to port something to OCaml and would like help from me and my colleagues at <a href="https://robur.coop/">Robur</a> please <a href="https://robur.coop/Contact">get in touch</a> with us and we will figure something out.</p>
<section role="doc-endnotes"><ol> <p>[^python-bytes]: There is as well the <code>bytes</code> type which is a byte sequence like OCaml's <code>string</code>.
<li id="fn-python-bytes">
<p>There is as well the <code>bytes</code> type which is a byte sequence like OCaml's <code>string</code>.
The Python code in question is using <code>str</code> however.</p> The Python code in question is using <code>str</code> however.</p>
<span><a href="#ref-1-fn-python-bytes" role="doc-backlink" class="fn-label">↩︎︎</a></span></li><li id="fn-raw-escape-example"> <p>[^raw-escape-example]: Note I use single quotes for the output. This is what Python would do. It would be equivalent to <code>&quot;\\\&quot;&quot;</code>.</p>
<p>Note I use single quotes for the output. This is what Python would do. It would be equivalent to <code>&quot;\\\&quot;&quot;</code>.</p> <p>[^language-lawyer]: <a href="http://catb.org/jargon/html/L/language-lawyer.html">A person, usually an experienced or senior software engineer, who is intimately familiar with many or most of the numerous restrictions and features (both useful and esoteric) applicable to one or more computer programming languages. A language lawyer is distinguished by the ability to show you the five sentences scattered through a 200-plus-page manual that together imply the answer to your question “if only you had thought to look there”.</a></p>
<span><a href="#ref-1-fn-raw-escape-example" role="doc-backlink" class="fn-label">↩︎︎</a></span></li><li id="fn-language-lawyer">
<p><a href="http://catb.org/jargon/html/L/language-lawyer.html">A person, usually an experienced or senior software engineer, who is intimately familiar with many or most of the numerous restrictions and features (both useful and esoteric) applicable to one or more computer programming languages. A language lawyer is distinguished by the ability to show you the five sentences scattered through a 200-plus-page manual that together imply the answer to your question “if only you had thought to look there”.</a></p>
<span><a href="#ref-1-fn-language-lawyer" role="doc-backlink" class="fn-label">↩︎︎</a></span></li></ol></section>
</article> </article>

57
articles/2024-08-21-OpenVPN-and-MirageVPN.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - MirageVPN and OpenVPN Robur's blogMirageVPN and OpenVPN
</title> </title>
<meta name="description" content="Discoveries made implementing MirageVPN, a OpenVPN-compatible VPN library"> <meta name="description" content="Discoveries made implementing MirageVPN, a OpenVPN-compatible VPN library">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,17 +20,17 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>MirageVPN and OpenVPN</h1> <h1>MirageVPN and OpenVPN</h1>
<ul class="tags-list"><li><a href="/tags/miragevpn.html">miragevpn</a></li><li><a href="/tags/openvpn.html">openvpn</a></li><li><a href="/tags/security.html">security</a></li></ul><p>At <a href="https://robur.coop/">Robur</a> we have been busy at work implementing our OpenVPN™-compatible MirageVPN software. <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-MirageVPN">MirageVPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-OpenVPN">OpenVPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li></ul><p>At <a href="https://robur.coop/">Robur</a> we have been busy at work implementing our OpenVPN™-compatible MirageVPN software.
Recently we have implemented the <a href="https://blog.robur.coop/articles/miragevpn-server.html">server side</a>. Recently we have implemented the <a href="https://blog.robur.coop/articles/miragevpn-server.html">server side</a>.
In order to implement this side of the protocol I studied parts of the OpenVPN™ source code and performed experiments to understand what the implementation does at the protocol level. In order to implement this side of the protocol I studied parts of the OpenVPN™ source code and performed experiments to understand what the implementation does at the protocol level.
Studying the OpenVPN™ implementation has lead me to discover two security issues: CVE-2024-28882 and CVE-2024-5594. Studying the OpenVPN™ implementation has lead me to discover two security issues: CVE-2024-28882 and CVE-2024-5594.
In this article I will talk about the relevant parts of the protocol, and describe the security issues in detail.</p> In this article I will talk about the relevant parts of the protocol, and describe the security issues in detail.</p>
<p>A VPN establishes a secure tunnel in which (usually) IP packets are sent. <p>A VPN establishes a secure tunnel in which (usually) IP packets are sent.
The OpenVPN protocol establishes a TLS tunnel<sup><a href="#fn-openvpn-tls" id="ref-1-fn-openvpn-tls" role="doc-noteref" class="fn-label">[1]</a></sup> with which key material and configuration options are negotiated. The OpenVPN protocol establishes a TLS tunnel[^openvpn-tls] with which key material and configuration options are negotiated.
Once established the TLS tunnel is used to exchange so-called control channel messages. Once established the TLS tunnel is used to exchange so-called control channel messages.
They are NUL-terminated (well, more on that later) text messages sent in a single TLS record frame (mostly, more on that later).</p> They are NUL-terminated (well, more on that later) text messages sent in a single TLS record frame (mostly, more on that later).</p>
<p>I will describe two (groups) of control channel messages (and a bonus control channel message):</p> <p>I will describe two (groups) of control channel messages (and a bonus control channel message):</p>
@ -41,7 +40,7 @@ They are NUL-terminated (well, more on that later) text messages sent in a singl
<li>(<code>AUTH_FAILED</code>)</li> <li>(<code>AUTH_FAILED</code>)</li>
</ul> </ul>
<p>The <code>EXIT</code>, <code>RESTART</code>, and <code>HALT</code> messages share similarity. <p>The <code>EXIT</code>, <code>RESTART</code>, and <code>HALT</code> messages share similarity.
They are all three used to signal to the client that it should disconnect<sup><a href="#fn-disconnect" id="ref-1-fn-disconnect" role="doc-noteref" class="fn-label">[2]</a></sup> from the server. They are all three used to signal to the client that it should disconnect[^disconnect] from the server.
<code>HALT</code> tells the client to disconnect and suggests the client should terminate. <code>HALT</code> tells the client to disconnect and suggests the client should terminate.
<code>RESTART</code> also tells the client to disconnect and suggests the client can reconnect either to the same server or the next server if multiple are configured depending on flags in the message. <code>RESTART</code> also tells the client to disconnect and suggests the client can reconnect either to the same server or the next server if multiple are configured depending on flags in the message.
<code>EXIT</code> tells the <em>peer</em> that it is exiting and the <em>peer</em> should disconnect. <code>EXIT</code> tells the <em>peer</em> that it is exiting and the <em>peer</em> should disconnect.
@ -50,13 +49,13 @@ It informs the peer that the sender is exiting and will (soon) not be receiving
<p>Because the underlying transport can either be TCP or UDP the sender may have no guarantees that the message arrives. <p>Because the underlying transport can either be TCP or UDP the sender may have no guarantees that the message arrives.
OpenVPN's control channel implements a reliable layer with ACKs and retransmissions to work around that. OpenVPN's control channel implements a reliable layer with ACKs and retransmissions to work around that.
To accomodate this OpenVPN™ will wait five seconds before disconnecting to allow for retransmission of the exit message.</p> To accomodate this OpenVPN™ will wait five seconds before disconnecting to allow for retransmission of the exit message.</p>
<h3>The bug</h3> <h3 id="the-bug"><a class="anchor" aria-hidden="true" href="#the-bug"></a>The bug</h3>
<p>While I was working on implementing more control channel message types I modified a client application that connects to a server and sends pings over the tunnel - instead of ICMPv4 echo requests I modified it to send the <code>EXIT</code> control channel message once a second. <p>While I was working on implementing more control channel message types I modified a client application that connects to a server and sends pings over the tunnel - instead of ICMPv4 echo requests I modified it to send the <code>EXIT</code> control channel message once a second.
In the server logs I saw that the server successfully received the <code>EXIT</code> message! In the server logs I saw that the server successfully received the <code>EXIT</code> message!
But nothing else happened. But nothing else happened.
The server just kept receiving <code>EXIT</code> messages but for some reason it never disconnected the client.</p> The server just kept receiving <code>EXIT</code> messages but for some reason it never disconnected the client.</p>
<p>Curious about this behavior I dived into the OpenVPN™ source code and found that on each <code>EXIT</code> message it (re)schedules an exit (disconnect) timer! That is, every time the server receives an <code>EXIT</code> message it'll go &quot;OK! I'll shut down this connection in five seconds&quot; forgetting it promised to do so earlier, too.</p> <p>Curious about this behavior I dived into the OpenVPN™ source code and found that on each <code>EXIT</code> message it (re)schedules an exit (disconnect) timer! That is, every time the server receives an <code>EXIT</code> message it'll go &quot;OK! I'll shut down this connection in five seconds&quot; forgetting it promised to do so earlier, too.</p>
<h3>Implications</h3> <h3 id="implications"><a class="anchor" aria-hidden="true" href="#implications"></a>Implications</h3>
<p>At first this seemed like a relatively benign bug. <p>At first this seemed like a relatively benign bug.
What's the worst that could happen if a client says &quot;let's stop in five second! No, five seconds from now! No, five seconds from now!&quot; etc? What's the worst that could happen if a client says &quot;let's stop in five second! No, five seconds from now! No, five seconds from now!&quot; etc?
Well, it turns out the same timer is used when the server sends an exit message. Well, it turns out the same timer is used when the server sends an exit message.
@ -68,7 +67,7 @@ The management interface is a text protocol to communicate with the OpenVPN serv
One command is the <code>client-kill</code> command. One command is the <code>client-kill</code> command.
The documentation says to use this command to &quot;[i]mmediately kill a client instance[...]&quot;. The documentation says to use this command to &quot;[i]mmediately kill a client instance[...]&quot;.
In practice it sends an exit message to the client (either a custom one or the default <code>RESTART</code>). In practice it sends an exit message to the client (either a custom one or the default <code>RESTART</code>).
I learnt that it shares code paths with the exit control messages to schedule an exit (disconnect)<sup><a href="#fn-kill-immediately" id="ref-1-fn-kill-immediately" role="doc-noteref" class="fn-label">[3]</a></sup>. I learnt that it shares code paths with the exit control messages to schedule an exit (disconnect)[^kill-immediately].
That is, <code>client-kill</code> schedules the same five second timer.</p> That is, <code>client-kill</code> schedules the same five second timer.</p>
<p>Thus a malicious client can, instead of exiting on receiving an exit or <code>RESTART</code> message, send back repeatedly <code>EXIT</code> to the server to reset the five second timer. <p>Thus a malicious client can, instead of exiting on receiving an exit or <code>RESTART</code> message, send back repeatedly <code>EXIT</code> to the server to reset the five second timer.
This way the client can indefinitely delay the exit/disconnect assuming sufficiently stable and responsive network. This way the client can indefinitely delay the exit/disconnect assuming sufficiently stable and responsive network.
@ -82,12 +81,12 @@ I don't know.
If you do or are aware of software that enforces policies this way please do reach out to <a href="https://reyn.ir/contact.html">me</a>. If you do or are aware of software that enforces policies this way please do reach out to <a href="https://reyn.ir/contact.html">me</a>.
It would be interesting to hear and discuss. It would be interesting to hear and discuss.
The OpenVPN security@ mailing list took it seriously enough to assign it CVE-2024-28882.</p> The OpenVPN security@ mailing list took it seriously enough to assign it CVE-2024-28882.</p>
<h2>OpenVPN configuration language</h2> <h2 id="openvpn-configuration-language"><a class="anchor" aria-hidden="true" href="#openvpn-configuration-language"></a>OpenVPN configuration language</h2>
<p>Next up we have <code>PUSH_REQUEST</code> / <code>PUSH_REPLY</code>. <p>Next up we have <code>PUSH_REQUEST</code> / <code>PUSH_REPLY</code>.
As the names suggest it's a request/response protocol. As the names suggest it's a request/response protocol.
It is used to communicate configuration options from the server to the client. It is used to communicate configuration options from the server to the client.
These options include routes, ip address configuration, negotiated cryptographic algorithms. These options include routes, ip address configuration, negotiated cryptographic algorithms.
The client signals it would like to receive configuration options from the server by sending the <code>PUSH_REQUEST</code> control channel message<sup><a href="#fn-proto-push-request" id="ref-1-fn-proto-push-request" role="doc-noteref" class="fn-label">[4]</a></sup>. The client signals it would like to receive configuration options from the server by sending the <code>PUSH_REQUEST</code> control channel message[^proto-push-request].
The server then sends a <code>PUSH_REPLY</code> message.</p> The server then sends a <code>PUSH_REPLY</code> message.</p>
<p>The format of a <code>PUSH_REPLY</code> message is <code>PUSH_REPLY,</code> followed by a comma separated list of OpenVPN configuration directives terminated by a NUL byte as in other control channel messages. <p>The format of a <code>PUSH_REPLY</code> message is <code>PUSH_REPLY,</code> followed by a comma separated list of OpenVPN configuration directives terminated by a NUL byte as in other control channel messages.
Note that this means pushed configuration directives cannot contain commas.</p> Note that this means pushed configuration directives cannot contain commas.</p>
@ -95,8 +94,8 @@ Note that this means pushed configuration directives cannot contain commas.</p>
I learned some quirks of the configuration language which I find surprising and somewhat hard to implement. I learned some quirks of the configuration language which I find surprising and somewhat hard to implement.
I will not cover all corners of the configuration language.</p> I will not cover all corners of the configuration language.</p>
<p>In some sense you could say the configuration language of OpenVPN™ is line based. <p>In some sense you could say the configuration language of OpenVPN™ is line based.
At least, the first step to parsing configuration directives as OpenVPN 2.X does is to read one line at a time and parse it as one configuration directive<sup><a href="#fn-inline-files" id="ref-1-fn-inline-files" role="doc-noteref" class="fn-label">[5]</a></sup>. At least, the first step to parsing configuration directives as OpenVPN 2.X does is to read one line at a time and parse it as one configuration directive[^inline-files].
A line is whatever <code>fgets()</code> says it is - this includes the newline if not at the end of the file<sup><a href="#fn-configuration-newlines" id="ref-1-fn-configuration-newlines" role="doc-noteref" class="fn-label">[6]</a></sup>. A line is whatever <code>fgets()</code> says it is - this includes the newline if not at the end of the file[^configuration-newlines].
This is how it is for configuration files. This is how it is for configuration files.
However, if it is a <code>PUSH_REPLY</code> a <em>&quot;line&quot;</em> is the text string up to a comma or the end of file (or, importantly, a NUL byte). However, if it is a <code>PUSH_REPLY</code> a <em>&quot;line&quot;</em> is the text string up to a comma or the end of file (or, importantly, a NUL byte).
This &quot;line&quot; tokenization is done by repeatedly calling OpenVPN™'s <code>buf_parse(buf, ',', line, sizeof(line))</code> function.</p> This &quot;line&quot; tokenization is done by repeatedly calling OpenVPN™'s <code>buf_parse(buf, ',', line, sizeof(line))</code> function.</p>
@ -196,14 +195,14 @@ As a proof of concept I sent a <code>PUSH_REPLY</code> with an embedded BEL char
<p>The <code>^G</code> is how the BEL character is printed in my terminal. <p>The <code>^G</code> is how the BEL character is printed in my terminal.
I was also able to hear an audible bell.</p> I was also able to hear an audible bell.</p>
<p>A more thorough explanation on how terminal escape sequences can be exploited can be found on <a href="https://www.gresearch.com/news/g-research-the-terminal-escapes/">G-Reasearch's blog</a>.</p> <p>A more thorough explanation on how terminal escape sequences can be exploited can be found on <a href="https://www.gresearch.com/news/g-research-the-terminal-escapes/">G-Reasearch's blog</a>.</p>
<h3>The fix</h3> <h3 id="the-fix"><a class="anchor" aria-hidden="true" href="#the-fix"></a>The fix</h3>
<p>The fix also is also a first step towards decoupling the control channel messaging from the TLS record frames. <p>The fix also is also a first step towards decoupling the control channel messaging from the TLS record frames.
First, the data is split on NUL bytes in order to get the control channel message(s), and then messages are rejected if they contain illegal characters. First, the data is split on NUL bytes in order to get the control channel message(s), and then messages are rejected if they contain illegal characters.
This solves the vulnerability described previously.</p> This solves the vulnerability described previously.</p>
<p>Unfortunately, it turns out that especially for the <code>AUTH_FAILED</code> control channel message it is easy to create invalid messages: <p>Unfortunately, it turns out that especially for the <code>AUTH_FAILED</code> control channel message it is easy to create invalid messages:
If 2FA is implemented using the script mechanism sending custom messages they easily end with a newline asking the client to enter the verification code. If 2FA is implemented using the script mechanism sending custom messages they easily end with a newline asking the client to enter the verification code.
I believe in 2.6.12 the client tolerates trailing newline characters.</p> I believe in 2.6.12 the client tolerates trailing newline characters.</p>
<h2>Conclusion</h2> <h2 id="conclusion"><a class="anchor" aria-hidden="true" href="#conclusion"></a>Conclusion</h2>
<p>The first bug, the timer rescheduling bug, is at least 20 years old! <p>The first bug, the timer rescheduling bug, is at least 20 years old!
It hasn't always been exploitable, but the bug itself goes back as far as the git history does. It hasn't always been exploitable, but the bug itself goes back as far as the git history does.
I haven't attempted further software archeology to find the exact time of introduction. I haven't attempted further software archeology to find the exact time of introduction.
@ -211,20 +210,12 @@ Either way, it's old and gone unnoticed for quite a while.</p>
<p>I think this shows that diversity in implementations is a great way to exercise corner cases, push forward (protocol) documentation efforts and get thorough code review by motivated peers. <p>I think this shows that diversity in implementations is a great way to exercise corner cases, push forward (protocol) documentation efforts and get thorough code review by motivated peers.
This work was funded by <a href="https://nlnet.nl/project/MirageVPN/">the EU NGI Assure Fund through NLnet</a>. This work was funded by <a href="https://nlnet.nl/project/MirageVPN/">the EU NGI Assure Fund through NLnet</a>.
In my opinion, this shows that funding one open source project can have a positive impact on other open source projects, too.</p> In my opinion, this shows that funding one open source project can have a positive impact on other open source projects, too.</p>
<section role="doc-endnotes"><ol> <p>[^openvpn-tls]: This is not always the case. It is possible to use static shared secret keys, but it is mostly considered deprecated.
<li id="fn-openvpn-tls"> [^disconnect]: I say &quot;disconnect&quot; even when the underlying transport is the connection-less UDP.
<p>This is not always the case. It is possible to use static shared secret keys, but it is mostly considered deprecated.</p> [^kill-immediately]: As the alert reader might have realized this is inaccurate. It does not kill the client &quot;immediately&quot; as it will wait five seconds after the exit message is sent before exiting. At best this will kill a cooperating client once it's received the kill message.
<span><a href="#ref-1-fn-openvpn-tls" role="doc-backlink" class="fn-label">↩︎︎</a></span></li><li id="fn-disconnect"> [^proto-push-request]: There is another mechanism to request a <code>PUSH_REPLY</code> earlier with less roundtrips, but let's ignore that for now. The exact message is <code>PUSH_REQUEST&lt;NUL-BYTE&gt;</code> as messages need to be NUL-terminated.
<p>I say &quot;disconnect&quot; even when the underlying transport is the connection-less UDP.</p> [^inline-files]: An exception being inline files which can span multiple lines. They vaguely resemble XML tags with an open <code>&lt;tag&gt;</code> and close <code>&lt;/tag&gt;</code> each on their own line with the data in between. I doubt these are sent in <code>PUSH_REPLY</code>s, but I can't rule out without diving into the source code that it isn't possible to send inline files.
<span><a href="#ref-1-fn-disconnect" role="doc-backlink" class="fn-label">↩︎︎</a></span></li><li id="fn-kill-immediately"> [^configuration-newlines]: This results in the quirk that it is possible to sort-of escape a newline in a configuration directive. But since the line splitting is done <em>first</em> it's not possible to continue the directive on the next line! I believe this is mostly useless, but it is a way to inject line feeds in configuration options without modifying the OpenVPN source code.</p>
<p>As the alert reader might have realized this is inaccurate. It does not kill the client &quot;immediately&quot; as it will wait five seconds after the exit message is sent before exiting. At best this will kill a cooperating client once it's received the kill message.</p>
<span><a href="#ref-1-fn-kill-immediately" role="doc-backlink" class="fn-label">↩︎︎</a></span></li><li id="fn-proto-push-request">
<p>There is another mechanism to request a <code>PUSH_REPLY</code> earlier with less roundtrips, but let's ignore that for now. The exact message is <code>PUSH_REQUEST&lt;NUL-BYTE&gt;</code> as messages need to be NUL-terminated.</p>
<span><a href="#ref-1-fn-proto-push-request" role="doc-backlink" class="fn-label">↩︎︎</a></span></li><li id="fn-inline-files">
<p>An exception being inline files which can span multiple lines. They vaguely resemble XML tags with an open <code>&lt;tag&gt;</code> and close <code>&lt;/tag&gt;</code> each on their own line with the data in between. I doubt these are sent in <code>PUSH_REPLY</code>s, but I can't rule out without diving into the source code that it isn't possible to send inline files.</p>
<span><a href="#ref-1-fn-inline-files" role="doc-backlink" class="fn-label">↩︎︎</a></span></li><li id="fn-configuration-newlines">
<p>This results in the quirk that it is possible to sort-of escape a newline in a configuration directive. But since the line splitting is done <em>first</em> it's not possible to continue the directive on the next line! I believe this is mostly useless, but it is a way to inject line feeds in configuration options without modifying the OpenVPN source code.</p>
<span><a href="#ref-1-fn-configuration-newlines" role="doc-backlink" class="fn-label">↩︎︎</a></span></li></ol></section>
</article> </article>

30
articles/gptar.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - GPTar Robur's blogGPTar
</title> </title>
<meta name="description" content="Hybrid GUID partition table and tar archive"> <meta name="description" content="Hybrid GUID partition table and tar archive">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,17 +20,17 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>GPTar</h1> <h1>GPTar</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/gpt.html">gpt</a></li><li><a href="/tags/tar.html">tar</a></li><li><a href="/tags/mbr.html">mbr</a></li><li><a href="/tags/persistent storage.html">persistent storage</a></li></ul><p>At <a href="https://robur.coop/">Robur</a> we developed a piece of software for mirroring or exposing an <a href="https://opam.ocaml.org/">opam</a> repository. <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-gpt">gpt</a></li><li><a href="https://blog.robur.coop/tags.html#tag-tar">tar</a></li><li><a href="https://blog.robur.coop/tags.html#tag-mbr">mbr</a></li><li><a href="https://blog.robur.coop/tags.html#tag-persistent storage">persistent storage</a></li></ul><p>At <a href="https://robur.coop/">Robur</a> we developed a piece of software for mirroring or exposing an <a href="https://opam.ocaml.org/">opam</a> repository.
We have it deployed at <a href="https://opam.robur.coop/">opam.robur.coop</a>, and you can use it as an alternative to opam.ocaml.org. We have it deployed at <a href="https://opam.robur.coop/">opam.robur.coop</a>, and you can use it as an alternative to opam.ocaml.org.
It is usually more up-to-date with the git <a href="https://github.com/ocaml/opam-repository">opam-repository</a> than opam.ocaml.org although in the past it suffered from <a href="https://blog.osau.re/articles/lwt_pause.html">occasional availability issues</a>. It is usually more up-to-date with the git <a href="https://github.com/ocaml/opam-repository">opam-repository</a> than opam.ocaml.org although in the past it suffered from <a href="https://blog.osau.re/articles/lwt_pause.html">occasional availability issues</a>.
I can recommend reading Hannes' post about <a href="https://hannes.robur.coop/Posts/OpamMirror">opam-mirror</a>. I can recommend reading Hannes' post about <a href="https://hannes.robur.coop/Posts/OpamMirror">opam-mirror</a>.
This article is about adding a partition table to the disk <a href="https://hannes.robur.coop/Posts/OpamMirror#code-development-and-improvements">as used by opam-mirror</a>. This article is about adding a partition table to the disk <a href="https://hannes.robur.coop/Posts/OpamMirror#code-development-and-improvements">as used by opam-mirror</a>.
For background I can recommend reading the previously linked subsection of the opam-mirror article.</p> For background I can recommend reading the previously linked subsection of the opam-mirror article.</p>
<h2>The opam-mirror persistent storage scheme</h2> <h2 id="the-opam-mirror-persistent-storage-scheme"><a class="anchor" aria-hidden="true" href="#the-opam-mirror-persistent-storage-scheme"></a>The opam-mirror persistent storage scheme</h2>
<p>Opam-mirror uses a single block device for its persistent storage. <p>Opam-mirror uses a single block device for its persistent storage.
On the block device it stores cached source code archives from the opam repository. On the block device it stores cached source code archives from the opam repository.
These are stored in a <a href="https://en.wikipedia.org/wiki/Tar_(computing)">tar archive </a> consisting of files whose file name is the sha256 checksum of the file contents. These are stored in a <a href="https://en.wikipedia.org/wiki/Tar_(computing)">tar archive </a> consisting of files whose file name is the sha256 checksum of the file contents.
@ -49,7 +48,7 @@ Therefore I have for a long time been wanting to use an on-disk partition table.
The problem is both <a href="https://en.wikipedia.org/wiki/MBR_partition_table">MBR</a> and <a href="https://en.m.wikipedia.org/wiki/GUID_Partition_Table">GPT</a> (GUID Partition Table) store the table at the beginning of the disk. The problem is both <a href="https://en.wikipedia.org/wiki/MBR_partition_table">MBR</a> and <a href="https://en.m.wikipedia.org/wiki/GUID_Partition_Table">GPT</a> (GUID Partition Table) store the table at the beginning of the disk.
If we write a partition table at the beginning it is suddenly not a valid tar archive anymore. If we write a partition table at the beginning it is suddenly not a valid tar archive anymore.
Of course, in Mirage we can just write and read the table at the end if we please, but then we lose the ability to inspect the partition table in the host system.</p> Of course, in Mirage we can just write and read the table at the end if we please, but then we lose the ability to inspect the partition table in the host system.</p>
<h2>GPT header as tar file name</h2> <h2 id="gpt-header-as-tar-file-name"><a class="anchor" aria-hidden="true" href="#gpt-header-as-tar-file-name"></a>GPT header as tar file name</h2>
<p>My first approach, which turned out to be a dead end, was when I realized that a GPT header consists of 92 bytes at the beginning followed by reserved space for the remainder of the LBA. <p>My first approach, which turned out to be a dead end, was when I realized that a GPT header consists of 92 bytes at the beginning followed by reserved space for the remainder of the LBA.
The reserved space should be all zeroes, but it seems no one bothers to enforce this. The reserved space should be all zeroes, but it seems no one bothers to enforce this.
What's more is that a tar header starts with the file name in the first 100 bytes. What's more is that a tar header starts with the file name in the first 100 bytes.
@ -71,7 +70,7 @@ The changes made are viewable on <a href="https://github.com/reynir/gptar/compar
I also had to work around a <a href="https://github.com/mirage/ocaml-tar/issues/144">bug in ocaml-tar</a>. I also had to work around a <a href="https://github.com/mirage/ocaml-tar/issues/144">bug in ocaml-tar</a>.
GNU tar was successfully able to list the archive. GNU tar was successfully able to list the archive.
A quirk is that the archive will start with a dummy file <code>GPTAR</code> which consists of any remaining space in the first LBA if the sector size is greater than 512 bytes followed by the partition table.</p> A quirk is that the archive will start with a dummy file <code>GPTAR</code> which consists of any remaining space in the first LBA if the sector size is greater than 512 bytes followed by the partition table.</p>
<h2>Protective MBR</h2> <h2 id="protective-mbr"><a class="anchor" aria-hidden="true" href="#protective-mbr"></a>Protective MBR</h2>
<p>Unfortunately, neither fdisk nor parted recognized the GPT partition table. <p>Unfortunately, neither fdisk nor parted recognized the GPT partition table.
I was able to successfully read the partition table using <a href="https://github.com/mirage/ocaml-gpt">ocaml-gpt</a> however. I was able to successfully read the partition table using <a href="https://github.com/mirage/ocaml-gpt">ocaml-gpt</a> however.
This puzzled me. This puzzled me.
@ -79,7 +78,7 @@ Then I got a hunch: I had read about <a href="https://en.m.wikipedia.org/wiki/GU
I had always thought it was optional and not needed in a new system such as Mirage that doesn't have to care too much about legacy code and operating systems.</p> I had always thought it was optional and not needed in a new system such as Mirage that doesn't have to care too much about legacy code and operating systems.</p>
<p>So I started comparing the layout of MBR and tar. <p>So I started comparing the layout of MBR and tar.
The V7 tar format only uses the first 257 bytes of the 512 byte block. The V7 tar format only uses the first 257 bytes of the 512 byte block.
The V7 format is differentiated by the UStar, POSIX/pax and old GNU tar formats by not having the string <code>ustar</code> at byte offset 257<sup><a href="#fn-tar-ustar" id="ref-1-fn-tar-ustar" role="doc-noteref" class="fn-label">[1]</a></sup>. The V7 format is differentiated by the UStar, POSIX/pax and old GNU tar formats by not having the string <code>ustar</code> at byte offset 257[^tar-ustar].
The master boot record format starts with the bootstrap code area. The master boot record format starts with the bootstrap code area.
In the classic format it is the first 446 bytes. In the classic format it is the first 446 bytes.
In the modern standard MBR format the first 446 bytes are mostly bootstrap code too with the exception of a handful bytes at offset 218 or so which are used for a timestamp or so. In the modern standard MBR format the first 446 bytes are mostly bootstrap code too with the exception of a handful bytes at offset 218 or so which are used for a timestamp or so.
@ -113,7 +112,7 @@ Number Start End Size File system Name Flags
-r-------- 0/0 14 1970-01-01 01:00 test.txt -r-------- 0/0 14 1970-01-01 01:00 test.txt
</code></pre> </code></pre>
<p>The <a href="https://github.com/reynir/gptar">code</a> is freely available on GitHub.</p> <p>The <a href="https://github.com/reynir/gptar">code</a> is freely available on GitHub.</p>
<h2>Future work</h2> <h2 id="future-work"><a class="anchor" aria-hidden="true" href="#future-work"></a>Future work</h2>
<p>One thing that bothers me a bit is the dummy file <code>GPTAR</code>. <p>One thing that bothers me a bit is the dummy file <code>GPTAR</code>.
By using the <code>G</code> link indicator GNU tar will print a warning about the unknown file type <code>G</code>, By using the <code>G</code> link indicator GNU tar will print a warning about the unknown file type <code>G</code>,
but it will still extract the dummy file when extracting the archive. but it will still extract the dummy file when extracting the archive.
@ -132,10 +131,7 @@ If the sector size is greater than 512 we can use the remaining space in LBA 0 t
I may try this for a sector size of 4096, but I'm not happy that it doesn't work with sector size 512 which solo5 will default to.</li> I may try this for a sector size of 4096, but I'm not happy that it doesn't work with sector size 512 which solo5 will default to.</li>
</ul> </ul>
<p>If you have other ideas what I can do please reach out!</p> <p>If you have other ideas what I can do please reach out!</p>
<section role="doc-endnotes"><ol> <p>[^tar-ustar]: This is somewhat simplified. There are some more nuances between the different formats, but for this purpose they don't matter much.</p>
<li id="fn-tar-ustar">
<p>This is somewhat simplified. There are some more nuances between the different formats, but for this purpose they don't matter much.</p>
<span><a href="#ref-1-fn-tar-ustar" role="doc-backlink" class="fn-label">↩︎︎</a></span></li></ol></section>
</article> </article>

33
articles/lwt_pause.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - Cooperation and Lwt.pause Robur's blogCooperation and Lwt.pause
</title> </title>
<meta name="description" content="A disgression about Lwt and Miou"> <meta name="description" content="A disgression about Lwt and Miou">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,11 +20,11 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>Cooperation and Lwt.pause</h1> <h1>Cooperation and Lwt.pause</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/scheduler.html">scheduler</a></li><li><a href="/tags/community.html">community</a></li><li><a href="/tags/unikernel.html">unikernel</a></li><li><a href="/tags/git.html">git</a></li></ul><p>Here's a concrete example of the notion of availability and the scheduler used <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Scheduler">Scheduler</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Community">Community</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Unikernel">Unikernel</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Git">Git</a></li></ul><p>Here's a concrete example of the notion of availability and the scheduler used
(in this case Lwt). As you may know, at Robur we have developed a unikernel: (in this case Lwt). As you may know, at Robur we have developed a unikernel:
<a href="https://git.robur.coop/robur/opam-mirror">opam-mirror</a>. It launches an HTTP service that can be used as an <a href="https://git.robur.coop/robur/opam-mirror">opam-mirror</a>. It launches an HTTP service that can be used as an
OPAM overlay available from a Git repository (with <code>opam repository add &lt;name&gt; &lt;url&gt;</code>).</p> OPAM overlay available from a Git repository (with <code>opam repository add &lt;name&gt; &lt;url&gt;</code>).</p>
@ -47,7 +46,7 @@ can be interesting to have such a unikernel available).</p>
about it at the Mirleft retreat, when we tried to get the repository from Git, about it at the Mirleft retreat, when we tried to get the repository from Git,
we had a (fairly long) unavailability of our HTTP server. Basically, we had to we had a (fairly long) unavailability of our HTTP server. Basically, we had to
wait ~10 min before the service offered by the unikernel was available.</p> wait ~10 min before the service offered by the unikernel was available.</p>
<h2>Availability</h2> <h2 id="availability"><a class="anchor" aria-hidden="true" href="#availability"></a>Availability</h2>
<p>If you follow my <a href="https://blog.osau.re/tags/scheduler.html">articles</a>, as far as Miou is concerned, from <p>If you follow my <a href="https://blog.osau.re/tags/scheduler.html">articles</a>, as far as Miou is concerned, from
the outset I talk of the notion of availability if we were to make yet another the outset I talk of the notion of availability if we were to make yet another
new scheduler for OCaml 5. We emphasised this notion because we had quite a few new scheduler for OCaml 5. We emphasised this notion because we had quite a few
@ -71,7 +70,7 @@ occurred. Lwt always tries to go as far down your chain as possible:</p>
<p>Lwt is rather sparse in adding cooperation points besides <code>Lwt.pause</code> and <p>Lwt is rather sparse in adding cooperation points besides <code>Lwt.pause</code> and
read/write operations, in contrast with Async where the bind operator is a read/write operations, in contrast with Async where the bind operator is a
cooperation point.</p> cooperation point.</p>
<h3>If there is no I/O, do not wrap in Lwt</h3> <h3 id="if-there-is-no-io-do-not-wrap-in-lwt"><a class="anchor" aria-hidden="true" href="#if-there-is-no-io-do-not-wrap-in-lwt"></a>If there is no I/O, do not wrap in Lwt</h3>
<p>It was (bad<sup><a href="#fn1">1</a></sup>) advice I was given. If a function doesn't do <p>It was (bad<sup><a href="#fn1">1</a></sup>) advice I was given. If a function doesn't do
I/O, there's no point in putting it in Lwt. At first glance, however, the idea I/O, there's no point in putting it in Lwt. At first glance, however, the idea
may be a good one. If you have a function that doesn't do I/O, whether it's in may be a good one. If you have a function that doesn't do I/O, whether it's in
@ -135,7 +134,7 @@ sort1: [|0; 6|]
<p><strong><tag id="fn1">1</tag></strong>: However, if you are not interested in availability <p><strong><tag id="fn1">1</tag></strong>: However, if you are not interested in availability
and would like the scheduler to try to resolve your promises as quickly as and would like the scheduler to try to resolve your promises as quickly as
possible, this advice is clearly valid.</p> possible, this advice is clearly valid.</p>
<h4>Performances</h4> <h4 id="performances"><a class="anchor" aria-hidden="true" href="#performances"></a>Performances</h4>
<p>It should be noted, however, that Lwt has an impact. Even if the behaviour is <p>It should be noted, however, that Lwt has an impact. Even if the behaviour is
the same, the Lwt layer is not free. A quick benchmark shows that there is an the same, the Lwt layer is not free. A quick benchmark shows that there is an
overhead:</p> overhead:</p>
@ -162,7 +161,7 @@ sort1 0.000676s
<p>This is the fairly obvious argument for not using Lwt when there's no I/O. Then, <p>This is the fairly obvious argument for not using Lwt when there's no I/O. Then,
if the Lwt monad is really needed, a simple <code>Lwt.return</code> at the very last if the Lwt monad is really needed, a simple <code>Lwt.return</code> at the very last
instance is sufficient (or, better, the use of <code>Lwt.map</code> / <code>&gt;|=</code>).</p> instance is sufficient (or, better, the use of <code>Lwt.map</code> / <code>&gt;|=</code>).</p>
<h4>Cooperation and concrete example</h4> <h4 id="cooperation-and-concrete-example"><a class="anchor" aria-hidden="true" href="#cooperation-and-concrete-example"></a>Cooperation and concrete example</h4>
<p>So <code>Lwt.both</code> is the one to use when we want to run two processes <p>So <code>Lwt.both</code> is the one to use when we want to run two processes
&quot;at the same time&quot;. For the example, <a href="https://github.com/mirage/ocaml-git">ocaml-git</a> attempts <em>both</em> to &quot;at the same time&quot;. For the example, <a href="https://github.com/mirage/ocaml-git">ocaml-git</a> attempts <em>both</em> to
retrieve a repository and also to analyse it. This can be seen in this snippet retrieve a repository and also to analyse it. This can be seen in this snippet
@ -172,7 +171,7 @@ case, both the left and right side do I/O (the left side binds into a socket
while the right side saves Git objects in your file system). So, in our tests while the right side saves Git objects in your file system). So, in our tests
with <code>Git_unix</code>, we were able to see that the analysis (right-hand side) was with <code>Git_unix</code>, we were able to see that the analysis (right-hand side) was
well executed and 'interleaved' with the reception of objects from the network.</p> well executed and 'interleaved' with the reception of objects from the network.</p>
<h3>Composability</h3> <h3 id="composability"><a class="anchor" aria-hidden="true" href="#composability"></a>Composability</h3>
<p>However, if we go back to our initial problem, we were talking about our <p>However, if we go back to our initial problem, we were talking about our
opam-mirror unikernel. As you might expect, there is no standalone MirageOS file opam-mirror unikernel. As you might expect, there is no standalone MirageOS file
system (and many of our unikernels don't need one). So, in the case of system (and many of our unikernels don't need one). So, in the case of
@ -208,7 +207,7 @@ will always offer a certain availability to other services (such as an HTTP
service) while the other will offer a Lwt function which will try to go as far service) while the other will offer a Lwt function which will try to go as far
as possible quite to make other services unavailable.</p> as possible quite to make other services unavailable.</p>
<p>Composing with one or the other therefore does not produce the same behavior.</p> <p>Composing with one or the other therefore does not produce the same behavior.</p>
<h4>Where to put <code>Lwt.pause</code>?</h4> <h4 id="where-to-put-lwtpause"><a class="anchor" aria-hidden="true" href="#where-to-put-lwtpause"></a>Where to put <code>Lwt.pause</code>?</h4>
<p>In this case, our <code>analyse_pack</code> does read/write on the Git store. As far as <p>In this case, our <code>analyse_pack</code> does read/write on the Git store. As far as
<code>Git_mem</code> is concerned, we said that these read/write accesses were just <code>Git_mem</code> is concerned, we said that these read/write accesses were just
accesses to a <code>Hashtbl</code>.</p> accesses to a <code>Hashtbl</code>.</p>
@ -225,7 +224,7 @@ to put, by hand, <code>Lwt.pause</code>.</li>
<li>In the end, Lwt has no mechanisms for ensuring the availability of a service <li>In the end, Lwt has no mechanisms for ensuring the availability of a service
(this is something that must be taken into account by the implementer).</li> (this is something that must be taken into account by the implementer).</li>
</ol> </ol>
<h3>In-depth knowledge of Lwt</h3> <h3 id="in-depth-knowledge-of-lwt"><a class="anchor" aria-hidden="true" href="#in-depth-knowledge-of-lwt"></a>In-depth knowledge of Lwt</h3>
<p>I haven't mentioned another problem we encountered with <a href="https://cambium.inria.fr/~agueneau/">Armael</a> when <p>I haven't mentioned another problem we encountered with <a href="https://cambium.inria.fr/~agueneau/">Armael</a> when
implementing <a href="https://discuss.ocaml.org/t/ann-release-of-multipart-form-0-2-0/7704#memory-bound-implementation">multipart_form</a> where the use of stream meant that implementing <a href="https://discuss.ocaml.org/t/ann-release-of-multipart-form-0-2-0/7704#memory-bound-implementation">multipart_form</a> where the use of stream meant that
Lwt didn't interleave the two processes and the use of a <em>bounded stream</em> was Lwt didn't interleave the two processes and the use of a <em>bounded stream</em> was
@ -234,7 +233,7 @@ possible in one of two branches of a <code>Lwt.both</code>.</p>
<p>This allows us to conclude that beyond the monad, Lwt has subtleties in its <p>This allows us to conclude that beyond the monad, Lwt has subtleties in its
behaviour which may be different from another scheduler such as Async (hence the behaviour which may be different from another scheduler such as Async (hence the
incompatibility between the two, which is not just of the <code>'a t</code> type).</p> incompatibility between the two, which is not just of the <code>'a t</code> type).</p>
<h3>Digression on Miou</h3> <h3 id="digression-on-miou"><a class="anchor" aria-hidden="true" href="#digression-on-miou"></a>Digression on Miou</h3>
<p>That's why we put so much emphasis on the notion of availability when it comes <p>That's why we put so much emphasis on the notion of availability when it comes
to Miou: to avoid repeating the mistakes of the past. The choices that can be to Miou: to avoid repeating the mistakes of the past. The choices that can be
made with regard to this notion in particular have a major impact, and can be made with regard to this notion in particular have a major impact, and can be
@ -245,7 +244,7 @@ through the use of <code>Effect.Shallow</code> which requires us to always re-at
handler (our scheduler) as soon as an effect is produced, unlike <code>Effect.Deep</code> handler (our scheduler) as soon as an effect is produced, unlike <code>Effect.Deep</code>
which can re-use the same handler for several effects. In other words, and as which can re-use the same handler for several effects. In other words, and as
we've described here, <strong>an effect yields</strong>!</p> we've described here, <strong>an effect yields</strong>!</p>
<h2>Conclusion</h2> <h2 id="conclusion"><a class="anchor" aria-hidden="true" href="#conclusion"></a>Conclusion</h2>
<p>As far as opam-mirror is concerned, we now have an unikernel that is available <p>As far as opam-mirror is concerned, we now have an unikernel that is available
even if it attempts to clone a Git repository and save Git objects in memory. At even if it attempts to clone a Git repository and save Git objects in memory. At
least, an HTTP service can co-exist with ocaml-git!</p> least, an HTTP service can co-exist with ocaml-git!</p>

25
articles/miragevpn-ncp.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - MirageVPN updated (AEAD, NCP) Robur's blogMirageVPN updated (AEAD, NCP)
</title> </title>
<meta name="description" content="How we resurrected MirageVPN from its bitrot state"> <meta name="description" content="How we resurrected MirageVPN from its bitrot state">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,13 +20,13 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>MirageVPN updated (AEAD, NCP)</h1> <h1>MirageVPN updated (AEAD, NCP)</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/vpn.html">vpn</a></li><li><a href="/tags/security.html">security</a></li></ul><h2>Updating MirageVPN</h2> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-VPN">VPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li></ul><h2 id="updating-miragevpn"><a class="anchor" aria-hidden="true" href="#updating-miragevpn"></a>Updating MirageVPN</h2>
<p>As announced <a href="https://blog.robur.coop/articles/miragevpn.html">earlier this month</a>, we've been working hard over the last months on MirageVPN (initially developed in 2019, targeting OpenVPN™ 2.4.7, now 2.6.6). We managed to receive funding from <a href="https://www.assure.ngi.eu/">NGI Assure</a> call (via <a href="https://nlnet.nl">NLnet</a>). We've made over 250 commits with more than 10k lines added, and 18k lines removed. We closed nearly all old issues, and opened 100 fresh ones, of which we already closed more than half of them. :D</p> <p>As announced <a href="https://blog.robur.coop/articles/miragevpn.html">earlier this month</a>, we've been working hard over the last months on MirageVPN (initially developed in 2019, targeting OpenVPN™ 2.4.7, now 2.6.6). We managed to receive funding from <a href="https://www.assure.ngi.eu/">NGI Assure</a> call (via <a href="https://nlnet.nl">NLnet</a>). We've made over 250 commits with more than 10k lines added, and 18k lines removed. We closed nearly all old issues, and opened 100 fresh ones, of which we already closed more than half of them. :D</p>
<h3>Actual bugs fixed (that were leading to non-working MirageVPN applications)</h3> <h3 id="actual-bugs-fixed-that-were-leading-to-non-working-miragevpn-applications"><a class="anchor" aria-hidden="true" href="#actual-bugs-fixed-that-were-leading-to-non-working-miragevpn-applications"></a>Actual bugs fixed (that were leading to non-working MirageVPN applications)</h3>
<p>In more detail, we had a specific configuration running over all the years, namely UDP mode with static keys (no TLS handshake, etc.). There were several issues (bitrot) that we encountered and solved along the path, amongst others:</p> <p>In more detail, we had a specific configuration running over all the years, namely UDP mode with static keys (no TLS handshake, etc.). There were several issues (bitrot) that we encountered and solved along the path, amongst others:</p>
<ul> <ul>
<li>related to the <a href="https://github.com/robur-coop/miragevpn/pull/111">static-key mode and TCP/IP</a>,</li> <li>related to the <a href="https://github.com/robur-coop/miragevpn/pull/111">static-key mode and TCP/IP</a>,</li>
@ -35,15 +34,15 @@
<li><a href="https://github.com/robur-coop/miragevpn/pull/110">outgoing TLS packets</a>.</li> <li><a href="https://github.com/robur-coop/miragevpn/pull/110">outgoing TLS packets</a>.</li>
</ul> </ul>
<p>To avoid any future breakage while revising the code (cleaning it up, extending it), we are now building several unikernels as part of our CI system. We also have setup OpenVPN™ servers with various configurations that we periodically test with our new code (we'll also work on further automation thereof).</p> <p>To avoid any future breakage while revising the code (cleaning it up, extending it), we are now building several unikernels as part of our CI system. We also have setup OpenVPN™ servers with various configurations that we periodically test with our new code (we'll also work on further automation thereof).</p>
<h3>New features: AEAD ciphers, supporting more configuration primitives</h3> <h3 id="new-features-aead-ciphers-supporting-more-configuration-primitives"><a class="anchor" aria-hidden="true" href="#new-features-aead-ciphers-supporting-more-configuration-primitives"></a>New features: AEAD ciphers, supporting more configuration primitives</h3>
<p>We added various configuration primitives, amongst them configuratble tls ciphersuites, minimal and maximal tls version to use, <a href="https://blog.robur.coop/articles/miragevpn.html">tls-crypt-v2</a>, verify-x509-name, cipher, remote-random, ...</p> <p>We added various configuration primitives, amongst them configuratble tls ciphersuites, minimal and maximal tls version to use, <a href="https://blog.robur.coop/articles/miragevpn.html">tls-crypt-v2</a>, verify-x509-name, cipher, remote-random, ...</p>
<p>From a cryptographic point of view, we are now supporting more <a href="https://github.com/robur-coop/miragevpn/pull/108">authentication hashes</a> via the configuration directive <code>auth</code>, namely the SHA2 family - previously, only SHA1 was supported, <a href="https://github.com/robur-coop/miragevpn/pull/125">AEAD ciphers</a> (AES-128-GCM, AES-256-GCM, CHACHA20-POLY1305) - previously only AES-256-CBC was supported.</p> <p>From a cryptographic point of view, we are now supporting more <a href="https://github.com/robur-coop/miragevpn/pull/108">authentication hashes</a> via the configuration directive <code>auth</code>, namely the SHA2 family - previously, only SHA1 was supported, <a href="https://github.com/robur-coop/miragevpn/pull/125">AEAD ciphers</a> (AES-128-GCM, AES-256-GCM, CHACHA20-POLY1305) - previously only AES-256-CBC was supported.</p>
<h3>NCP - Negotiation of cryptographic parameters</h3> <h3 id="ncp---negotiation-of-cryptographic-parameters"><a class="anchor" aria-hidden="true" href="#ncp---negotiation-of-cryptographic-parameters"></a>NCP - Negotiation of cryptographic parameters</h3>
<p>OpenVPN™ has a way to negotiate cryptographic parameters, instead of hardcoding them in the configuration. The client can propose its supported ciphers, and other features (MTU, directly request a push message for IP configuration, use TLS exporter secret instead of the hand-crafted (TLS 1.0 based PRF), ...) once the TLS handshake has been completed.</p> <p>OpenVPN™ has a way to negotiate cryptographic parameters, instead of hardcoding them in the configuration. The client can propose its supported ciphers, and other features (MTU, directly request a push message for IP configuration, use TLS exporter secret instead of the hand-crafted (TLS 1.0 based PRF), ...) once the TLS handshake has been completed.</p>
<p>We are now supporting this negotiation protocol, and have been working on the different extensions that are useful to us. Namely, transmitting the <a href="https://github.com/robur-coop/miragevpn/pull/121">supported ciphers</a>, <a href="https://github.com/robur-coop/miragevpn/pull/129">request push</a> (which deletes an entire round-trip), <a href="https://github.com/robur-coop/miragevpn/pull/163">TLS-exporter</a>. This will also be part of the <a href="https://git.robur.coop/robur/openvpn-spec">protocol specification</a> that we're working on while finishing our implementation.</p> <p>We are now supporting this negotiation protocol, and have been working on the different extensions that are useful to us. Namely, transmitting the <a href="https://github.com/robur-coop/miragevpn/pull/121">supported ciphers</a>, <a href="https://github.com/robur-coop/miragevpn/pull/129">request push</a> (which deletes an entire round-trip), <a href="https://github.com/robur-coop/miragevpn/pull/163">TLS-exporter</a>. This will also be part of the <a href="https://git.robur.coop/robur/openvpn-spec">protocol specification</a> that we're working on while finishing our implementation.</p>
<h3>Cleanups and refactorings</h3> <h3 id="cleanups-and-refactorings"><a class="anchor" aria-hidden="true" href="#cleanups-and-refactorings"></a>Cleanups and refactorings</h3>
<p>We also took some time to cleanup our code base, removing <code>Lwt.fail</code> (which doesn't produce proper backtraces), using lzo from the decompress package (since that code has been upstreamed a while ago), remove unneeded dependencies (rresult, astring), avoiding <code>assert false</code> in pattern matches by improving types, improve the log output (include a timestamp, show log source, use colors).</p> <p>We also took some time to cleanup our code base, removing <code>Lwt.fail</code> (which doesn't produce proper backtraces), using lzo from the decompress package (since that code has been upstreamed a while ago), remove unneeded dependencies (rresult, astring), avoiding <code>assert false</code> in pattern matches by improving types, improve the log output (include a timestamp, show log source, use colors).</p>
<h3>Future</h3> <h3 id="future"><a class="anchor" aria-hidden="true" href="#future"></a>Future</h3>
<p>There is still some work that we want to do, namely a QubesOS client implementation, an operators manual, extending our specification, resurrecting and adapting the server implementation, supporting more NCP features (if appropriate), etc. So stay tuned, we'll also provide reproducible binaries once we're ready.</p> <p>There is still some work that we want to do, namely a QubesOS client implementation, an operators manual, extending our specification, resurrecting and adapting the server implementation, supporting more NCP features (if appropriate), etc. So stay tuned, we'll also provide reproducible binaries once we're ready.</p>
<p>Don't hesitate to reach out to us on <a href="https://github.com/robur-coop/miragevpn/issues">GitHub</a>, <a href="https://robur.coop/Contact">by mail</a> or me personally <a href="https://mastodon.social/@hannesm">on Mastodon</a> if you're stuck.</p> <p>Don't hesitate to reach out to us on <a href="https://github.com/robur-coop/miragevpn/issues">GitHub</a>, <a href="https://robur.coop/Contact">by mail</a> or me personally <a href="https://mastodon.social/@hannesm">on Mastodon</a> if you're stuck.</p>

28
articles/miragevpn-performance.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - Speeding up MirageVPN and use it in the wild Robur's blogSpeeding up MirageVPN and use it in the wild
</title> </title>
<meta name="description" content="Performance engineering of MirageVPN, speeding it up by a factor of 25."> <meta name="description" content="Performance engineering of MirageVPN, speeding it up by a factor of 25.">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,33 +20,30 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>Speeding up MirageVPN and use it in the wild</h1> <h1>Speeding up MirageVPN and use it in the wild</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/cryptography.html">cryptography</a></li><li><a href="/tags/security.html">security</a></li><li><a href="/tags/vpn.html">vpn</a></li><li><a href="/tags/performance.html">performance</a></li></ul><p>As we were busy continuing to work on <a href="https://github.com/robur-coop/miragevpn">MirageVPN</a>, we got in touch with <a href="https://eduvpn.org">eduVPN</a>, who are interested about deploying MirageVPN. We got example configuration from their side, and <a href="https://github.com/robur-coop/miragevpn/pull/201">fixed</a> <a href="https://github.com/robur-coop/miragevpn/pull/168">some</a> <a href="https://github.com/robur-coop/miragevpn/pull/202">issues</a>, and also implemented <a href="https://github.com/robur-coop/miragevpn/pull/169">tls-crypt</a> - which was straightforward since we earlier spend time to implement <a href="https://blog.robur.coop/articles/miragevpn.html">tls-crypt-v2</a>.</p> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-cryptography">cryptography</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li><li><a href="https://blog.robur.coop/tags.html#tag-VPN">VPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-performance">performance</a></li></ul><p>As we were busy continuing to work on <a href="https://github.com/robur-coop/miragevpn">MirageVPN</a>, we got in touch with <a href="https://eduvpn.org">eduVPN</a>, who are interested about deploying MirageVPN. We got example configuration from their side, and <a href="https://github.com/robur-coop/miragevpn/pull/201">fixed</a> <a href="https://github.com/robur-coop/miragevpn/pull/168">some</a> <a href="https://github.com/robur-coop/miragevpn/pull/202">issues</a>, and also implemented <a href="https://github.com/robur-coop/miragevpn/pull/169">tls-crypt</a> - which was straightforward since we earlier spend time to implement <a href="https://blog.robur.coop/articles/miragevpn.html">tls-crypt-v2</a>.</p>
<p>In January, they gave MirageVPN another try, and <a href="https://github.com/robur-coop/miragevpn/issues/206">measured the performance</a> -- which was very poor -- MirageVPN (run as a Unix binary) provided a bandwith of 9.3Mb/s, while OpenVPN provided a bandwidth of 360Mb/s (using a VPN tunnel over TCP).</p> <p>In January, they gave MirageVPN another try, and <a href="https://github.com/robur-coop/miragevpn/issues/206">measured the performance</a> -- which was very poor -- MirageVPN (run as a Unix binary) provided a bandwith of 9.3Mb/s, while OpenVPN provided a bandwidth of 360Mb/s (using a VPN tunnel over TCP).</p>
<p>We aim at spending less resources for computing, thus the result was not satisfying for us. We re-read a lot of code, refactored a lot, and are now at ~250Mb/s.</p> <p>We aim at spending less resources for computing, thus the result was not satisfying for us. We re-read a lot of code, refactored a lot, and are now at ~250Mb/s.</p>
<h2>Tooling for performance engineering of OCaml</h2> <h2 id="tooling-for-performance-engineering-of-ocaml"><a class="anchor" aria-hidden="true" href="#tooling-for-performance-engineering-of-ocaml"></a>Tooling for performance engineering of OCaml</h2>
<p>As a first approach we connected with the MirageVPN unix client &amp; OpenVPN client to a eduVPN server and ran speed tests using <a href="https://fast.com">fast.com</a>. This was sufficient to show the initial huge gap in download speeds between MirageVPN and OpenVPN. There is <em>a lot</em> of noise in this approach as there are many computers and routers involved in this setup, and it is hard to reproduce.</p> <p>As a first approach we connected with the MirageVPN unix client &amp; OpenVPN client to a eduVPN server and ran speed tests using <a href="https://fast.com">fast.com</a>. This was sufficient to show the initial huge gap in download speeds between MirageVPN and OpenVPN. There is <em>a lot</em> of noise in this approach as there are many computers and routers involved in this setup, and it is hard to reproduce.</p>
<p>To get more reproducible results we set up a local VM with openvpn and iperf3 installed. On the host machine we can then connect to the VM's OpenVPN server and run iperf3 against the <em>VPN</em> ip address. This worked more reliably. However, it was still noisy and not suitable to measure single digit percentage changes in performance. <p>To get more reproducible results we set up a local VM with openvpn and iperf3 installed. On the host machine we can then connect to the VM's OpenVPN server and run iperf3 against the <em>VPN</em> ip address. This worked more reliably. However, it was still noisy and not suitable to measure single digit percentage changes in performance.
To better guide the performance engineering, we also developed <a href="https://github.com/robur-coop/miragevpn/pull/230">a microbenchmark</a> using OCaml tooling. This will setup a client and server without any input and output, and transfer data in memory.</p> To better guide the performance engineering, we also developed <a href="https://github.com/robur-coop/miragevpn/pull/230">a microbenchmark</a> using OCaml tooling. This will setup a client and server without any input and output, and transfer data in memory.</p>
<p>We also re-read our code and used the Linux utility <a href="https://perf.wiki.kernel.org/index.php/Main_Page"><code>perf</code></a> together with <a href="https://github.com/brendangregg/FlameGraph">Flamegraph</a> to graph its output. This works nicely with OCaml programs (we're using the 4.14.1 compiler and runtime system). We did the performance engineering on Unix binaries, i.e. not on MirageOS unikernels - but the MirageVPN protocol is used in both scenarios - thus the performance improvements described here are also in the MirageVPN unikernels.</p> <p>We also re-read our code and used the Linux utility <a href="https://perf.wiki.kernel.org/index.php/Main_Page"><code>perf</code></a> together with <a href="https://github.com/brendangregg/FlameGraph">Flamegraph</a> to graph its output. This works nicely with OCaml programs (we're using the 4.14.1 compiler and runtime system). We did the performance engineering on Unix binaries, i.e. not on MirageOS unikernels - but the MirageVPN protocol is used in both scenarios - thus the performance improvements described here are also in the MirageVPN unikernels.</p>
<h2>Takeaway of performance engineering</h2> <h2 id="takeaway-of-performance-engineering"><a class="anchor" aria-hidden="true" href="#takeaway-of-performance-engineering"></a>Takeaway of performance engineering</h2>
<p>The learnings of our performance engineering are in three areas:</p> <p>The learnings of our performance engineering are in three areas:</p>
<ul> <ul>
<li>Formatting strings is computational expensive -- thus if in an error case a hexdump is produced of a packet, its construction must be delayed for when the error case is executed (we have <a href="https://github.com/robur-coop/miragevpn/pull/220">this PR</a> and <a href="https://github.com/robur-coop/miragevpn/pull/209">that PR</a>). Alain Frisch wrote a nice <a href="https://www.lexifi.com/blog/ocaml/note-about-performance-printf-and-format/#">blog post</a> at LexiFi about performance of <code>Printf</code> and <code>Format</code><sup><a href="#fn-lexifi-date" id="ref-1-fn-lexifi-date" role="doc-noteref" class="fn-label">[1]</a></sup>.</li> <li>Formatting strings is computational expensive -- thus if in an error case a hexdump is produced of a packet, its construction must be delayed for when the error case is executed (we have <a href="https://github.com/robur-coop/miragevpn/pull/220">this PR</a> and <a href="https://github.com/robur-coop/miragevpn/pull/209">that PR</a>). Alain Frisch wrote a nice <a href="https://www.lexifi.com/blog/ocaml/note-about-performance-printf-and-format/#">blog post</a> at LexiFi about performance of <code>Printf</code> and <code>Format</code>[^lexifi-date].</li>
<li>Rethink allocations: fundamentally, only a single big buffer (to be send out) for each incoming packet should be allocated, not a series of buffers that are concatenated (see <a href="https://github.com/robur-coop/miragevpn/pull/217">this PR</a> and <a href="https://github.com/robur-coop/miragevpn/pull/219">that PR</a>). Additionally, not zeroing out the just allocated buffer (if it is filled with data anyways) removes some further instructions (see <a href="https://github.com/robur-coop/miragevpn/pull/218">this PR</a>). And we figured that appending to an empty buffer nevertheless allocated and copied in OCaml, so we worked on <a href="https://github.com/robur-coop/miragevpn/pull/214">this PR</a>.</li> <li>Rethink allocations: fundamentally, only a single big buffer (to be send out) for each incoming packet should be allocated, not a series of buffers that are concatenated (see <a href="https://github.com/robur-coop/miragevpn/pull/217">this PR</a> and <a href="https://github.com/robur-coop/miragevpn/pull/219">that PR</a>). Additionally, not zeroing out the just allocated buffer (if it is filled with data anyways) removes some further instructions (see <a href="https://github.com/robur-coop/miragevpn/pull/218">this PR</a>). And we figured that appending to an empty buffer nevertheless allocated and copied in OCaml, so we worked on <a href="https://github.com/robur-coop/miragevpn/pull/214">this PR</a>.</li>
<li>Still an open topic is: we are in the memory-safe language OCaml, and we sometimes extract data out of a buffer (or set data in a buffer). Now, each operation lead to bounds checks (that we do not touch memory that is not allocated or not ours). However, if we just checked for the buffer being long enough (either by checking the length, or by allocating a specific amount of data), these bounds checks are superfluous. So far, we don't have an automated solution for this issue, but we are <a href="https://discuss.ocaml.org/t/bounds-checks-for-string-and-bytes-when-retrieving-or-setting-subparts-thereof/">discussing it in the OCaml community</a>, and are eager to find a solution to avoid unneeded computations.</li> <li>Still an open topic is: we are in the memory-safe language OCaml, and we sometimes extract data out of a buffer (or set data in a buffer). Now, each operation lead to bounds checks (that we do not touch memory that is not allocated or not ours). However, if we just checked for the buffer being long enough (either by checking the length, or by allocating a specific amount of data), these bounds checks are superfluous. So far, we don't have an automated solution for this issue, but we are <a href="https://discuss.ocaml.org/t/bounds-checks-for-string-and-bytes-when-retrieving-or-setting-subparts-thereof/">discussing it in the OCaml community</a>, and are eager to find a solution to avoid unneeded computations.</li>
</ul> </ul>
<h2>Conclusion</h2> <h2 id="conclusion"><a class="anchor" aria-hidden="true" href="#conclusion"></a>Conclusion</h2>
<p>To conclude: we already achieved a factor of 25 in performance by adapting the code in various ways. We have ideas to improve the performance even more in the future - we also work on using OCaml string and bytes, instead of off-the-OCaml-heap-allocated bigarrays (see <a href="https://blog.robur.coop/articles/speeding-ec-string.html">our previous article</a>, which provided some speedups).</p> <p>To conclude: we already achieved a factor of 25 in performance by adapting the code in various ways. We have ideas to improve the performance even more in the future - we also work on using OCaml string and bytes, instead of off-the-OCaml-heap-allocated bigarrays (see <a href="https://blog.robur.coop/articles/speeding-ec-string.html">our previous article</a>, which provided some speedups).</p>
<p>Don't hesitate to reach out to us on <a href="https://github.com/robur-coop/miragevpn/issues">GitHub</a>, or <a href="https://robur.coop/Contact">by mail</a> if you're stuck.</p> <p>Don't hesitate to reach out to us on <a href="https://github.com/robur-coop/miragevpn/issues">GitHub</a>, or <a href="https://robur.coop/Contact">by mail</a> if you're stuck.</p>
<p>We want to thank <a href="https://nlnet.nl">NLnet</a> for their funding (via <a href="https://www.assure.ngi.eu/">NGI assure</a>), and <a href="https://eduvpn.org">eduVPN</a> for their interest.</p> <p>We want to thank <a href="https://nlnet.nl">NLnet</a> for their funding (via <a href="https://www.assure.ngi.eu/">NGI assure</a>), and <a href="https://eduvpn.org">eduVPN</a> for their interest.</p>
<section role="doc-endnotes"><ol> <p>[^lexifi-date]: It has come to our attention that the blog post is rather old (2012) and that the implementation has been completely rewritten since then.</p>
<li id="fn-lexifi-date">
<p>It has come to our attention that the blog post is rather old (2012) and that the implementation has been completely rewritten since then.</p>
<span><a href="#ref-1-fn-lexifi-date" role="doc-backlink" class="fn-label">↩︎︎</a></span></li></ol></section>
</article> </article>

17
articles/miragevpn-server.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - MirageVPN server Robur's blogMirageVPN server
</title> </title>
<meta name="description" content="Announcement of our MirageVPN server."> <meta name="description" content="Announcement of our MirageVPN server.">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,13 +20,13 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>MirageVPN server</h1> <h1>MirageVPN server</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/cryptography.html">cryptography</a></li><li><a href="/tags/security.html">security</a></li><li><a href="/tags/vpn.html">vpn</a></li></ul><p>It is a great pleasure to finally announce that we have finished a server implementation for MirageVPN (OpenVPN™-compatible). This allows to setup a very robust VPN network on both the client and the server side.</p> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-cryptography">cryptography</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li><li><a href="https://blog.robur.coop/tags.html#tag-VPN">VPN</a></li></ul><p>It is a great pleasure to finally announce that we have finished a server implementation for MirageVPN (OpenVPN™-compatible). This allows to setup a very robust VPN network on both the client and the server side.</p>
<p>As announced last year, <a href="https://blog.robur.coop/articles/miragevpn.html">MirageVPN</a> is a reimplemtation of OpenVPN™ in OCaml, with <a href="https://mirage.io">MirageOS</a> unikernels.</p> <p>As announced last year, <a href="https://blog.robur.coop/articles/miragevpn.html">MirageVPN</a> is a reimplemtation of OpenVPN™ in OCaml, with <a href="https://mirage.io">MirageOS</a> unikernels.</p>
<h2>Why a MirageVPN server?</h2> <h2 id="why-a-miragevpn-server"><a class="anchor" aria-hidden="true" href="#why-a-miragevpn-server"></a>Why a MirageVPN server?</h2>
<p>Providing Internet services with programming languages that have not much safety requires a lot of discipline by the developers to avoid issues which may lead to exploitable services that are attacked (and thus will circumvent any security goals). Especially services that are critical for security and privacy, it is crucial to avoid common memory safety pitfalls.</p> <p>Providing Internet services with programming languages that have not much safety requires a lot of discipline by the developers to avoid issues which may lead to exploitable services that are attacked (and thus will circumvent any security goals). Especially services that are critical for security and privacy, it is crucial to avoid common memory safety pitfalls.</p>
<p>Some years back, when we worked on the client implementation, we also drafted a server implementation. The reasoning was that a lot of the code was already there, and just a few things needed to be developed to allow clients to connect there.</p> <p>Some years back, when we worked on the client implementation, we also drafted a server implementation. The reasoning was that a lot of the code was already there, and just a few things needed to be developed to allow clients to connect there.</p>
<p>Now, we spend several months to push our server implementation into a state where it is usable and we are happy for everyone who wants to test it. We also adapted the modern ciphers we recently implemented for the client, and also tls-crypt and tls-crypt-v2 for the server implementation.</p> <p>Now, we spend several months to push our server implementation into a state where it is usable and we are happy for everyone who wants to test it. We also adapted the modern ciphers we recently implemented for the client, and also tls-crypt and tls-crypt-v2 for the server implementation.</p>

37
articles/miragevpn.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - MirageVPN &amp; tls-crypt-v2 Robur's blogMirageVPN &amp; tls-crypt-v2
</title> </title>
<meta name="description" content="How we implementated tls-crypt-v2 for miragevpn"> <meta name="description" content="How we implementated tls-crypt-v2 for miragevpn">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,15 +20,15 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>MirageVPN &amp; tls-crypt-v2</h1> <h1>MirageVPN &amp; tls-crypt-v2</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/vpn.html">vpn</a></li><li><a href="/tags/security.html">security</a></li></ul><p>In 2019 <a href="https://robur.coop/">Robur</a> started working on a <a href="https://github.com/robur-coop/miragevpn/">OpenVPN™-compatible implementation in OCaml</a>. <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-VPN">VPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li></ul><p>In 2019 <a href="https://robur.coop/">Robur</a> started working on a <a href="https://github.com/robur-coop/miragevpn/">OpenVPN™-compatible implementation in OCaml</a>.
The project was funded for 6 months in 2019 by <a href="https://prototypefund.de">prototypefund</a>. The project was funded for 6 months in 2019 by <a href="https://prototypefund.de">prototypefund</a>.
In late 2022 we applied again for funding this time to the <a href="https://www.assure.ngi.eu/">NGI Assure</a> open call, and our application was eventually accepted. In late 2022 we applied again for funding this time to the <a href="https://www.assure.ngi.eu/">NGI Assure</a> open call, and our application was eventually accepted.
In this blog post I will explain why reimplementing the OpenVPN™ protocol in OCaml is a worthwhile effort, and describe the Miragevpn implementation and in particular the <code>tls-crypt-v2</code> mechanism.</p> In this blog post I will explain why reimplementing the OpenVPN™ protocol in OCaml is a worthwhile effort, and describe the Miragevpn implementation and in particular the <code>tls-crypt-v2</code> mechanism.</p>
<h2>What even is OpenVPN™?</h2> <h2 id="what-even-is-openvpn™"><a class="anchor" aria-hidden="true" href="#what-even-is-openvpn™"></a>What even is OpenVPN™?</h2>
<p><a href="https://openvpn.net/">OpenVPN™</a> is a protocol and software implementation to provide <a href="https://en.wikipedia.org/wiki/Virtual_private_network">virtual private networks</a>: computer networks that do not exist in hardware and are encrypted and tunnelled through existing networks. <p><a href="https://openvpn.net/">OpenVPN™</a> is a protocol and software implementation to provide <a href="https://en.wikipedia.org/wiki/Virtual_private_network">virtual private networks</a>: computer networks that do not exist in hardware and are encrypted and tunnelled through existing networks.
Common use cases for this is to provide access to internal networks for remote workers, and for routing internet traffic through another machine for various reasons e.g. when using untrusted wifi, privacy from a snooping ISP, circumventing geoblock etc.</p> Common use cases for this is to provide access to internal networks for remote workers, and for routing internet traffic through another machine for various reasons e.g. when using untrusted wifi, privacy from a snooping ISP, circumventing geoblock etc.</p>
<p>It is a protocol that has been worked on and evolved over the decades. <p>It is a protocol that has been worked on and evolved over the decades.
@ -38,16 +37,16 @@ The modes can be categorized into two main categories: static mode and TLS mode.
The former mode uses static symmetric keys, and will be removed in the upcoming OpenVPN™ 2.7 (community edition). The former mode uses static symmetric keys, and will be removed in the upcoming OpenVPN™ 2.7 (community edition).
I will not focus on static mode in this post. I will not focus on static mode in this post.
The latter uses separate data &amp; control channels where the control channel uses TLS - more on that later.</p> The latter uses separate data &amp; control channels where the control channel uses TLS - more on that later.</p>
<h3>Why reimplement it? And why in OCaml?</h3> <h3 id="why-reimplement-it-and-why-in-ocaml"><a class="anchor" aria-hidden="true" href="#why-reimplement-it-and-why-in-ocaml"></a>Why reimplement it? And why in OCaml?</h3>
<p>Before diving into TLS mode and eventually tls-crypt-v2 it's worth to briefly discuss why we spend time reimplementing the OpenVPN™ protocol. <p>Before diving into TLS mode and eventually tls-crypt-v2 it's worth to briefly discuss why we spend time reimplementing the OpenVPN™ protocol.
You may ask yourself: why not just use the existing tried and tested implementation?</p> You may ask yourself: why not just use the existing tried and tested implementation?</p>
<p>OpenVPN™ community edition is implemented in the C programming language. <p>OpenVPN™ community edition is implemented in the C programming language.
It heavily uses the OpenSSL library<sup><a href="#fn-mbedtls" id="ref-1-fn-mbedtls" role="doc-noteref" class="fn-label">[1]</a></sup> which is as well written in C and has in the past had some notable security vulnerabilities. It heavily uses the OpenSSL library[^mbedtls] which is as well written in C and has in the past had some notable security vulnerabilities.
Many vulnerabilities and bugs in C can be easily avoided in other languages due to bounds checking and stricter and more expressive type systems. Many vulnerabilities and bugs in C can be easily avoided in other languages due to bounds checking and stricter and more expressive type systems.
The state machine of the protocol can be more easily be expressed in OCaml, and some properties of the protocol can be encoded in the type system.</p> The state machine of the protocol can be more easily be expressed in OCaml, and some properties of the protocol can be encoded in the type system.</p>
<p>Another reason is <a href="https://mirage.io/">Mirage OS</a>, a library operating system implemented in OCaml. <p>Another reason is <a href="https://mirage.io/">Mirage OS</a>, a library operating system implemented in OCaml.
We work on the Mirage project and write applications (unikernels) using Mirage. We work on the Mirage project and write applications (unikernels) using Mirage.
In many cases it would be desirable to be able to connect to an existing VPN network<sup><a href="#fn-vpn-network" id="ref-1-fn-vpn-network" role="doc-noteref" class="fn-label">[2]</a></sup>, In many cases it would be desirable to be able to connect to an existing VPN network[^vpn-network],
or be able to offer a VPN network to clients using OpenVPN™.</p> or be able to offer a VPN network to clients using OpenVPN™.</p>
<p>Consider a VPN provider: <p>Consider a VPN provider:
The VPN provider runs many machines that run an operating system in order to run the user-space OpenVPN™ service. The VPN provider runs many machines that run an operating system in order to run the user-space OpenVPN™ service.
@ -61,7 +60,7 @@ The networking provided to a application (virtual machine) can be restricted to
It is possible to use OpenVPN™ for such a setup, but that requires running OpenVPN™ in a full Linux virtual machine. It is possible to use OpenVPN™ for such a setup, but that requires running OpenVPN™ in a full Linux virtual machine.
With Mirage OS the resource footprint is typically much smaller than an equivalent application running in a Linux virtual machine; often the memory footprint is smaller by an order.</p> With Mirage OS the resource footprint is typically much smaller than an equivalent application running in a Linux virtual machine; often the memory footprint is smaller by an order.</p>
<p>Finally, while it's not an explicit goal of ours, reimplementing a protocol without an explicit specification can help uncover bugs and things that need better documentation in the original implementation.</p> <p>Finally, while it's not an explicit goal of ours, reimplementing a protocol without an explicit specification can help uncover bugs and things that need better documentation in the original implementation.</p>
<h3>TLS mode</h3> <h3 id="tls-mode"><a class="anchor" aria-hidden="true" href="#tls-mode"></a>TLS mode</h3>
<p>There are different variants of TLS mode, but what they share is separate &quot;control&quot; channel and &quot;data&quot; channel. <p>There are different variants of TLS mode, but what they share is separate &quot;control&quot; channel and &quot;data&quot; channel.
The control channel is used to do a TLS handshake, and with the established TLS session data channel encryption keys, username/password authentication, etc. is negotiated. The control channel is used to do a TLS handshake, and with the established TLS session data channel encryption keys, username/password authentication, etc. is negotiated.
Once this dance has been performed and data channel encryption keys have been negotiated the peers can exchange IP packets over the data channel.</p> Once this dance has been performed and data channel encryption keys have been negotiated the peers can exchange IP packets over the data channel.</p>
@ -72,7 +71,7 @@ This makes it possible for an OpenVPN™ server to reject early clients that don
In <code>tls-crypt</code> the control channel is encrypted as well as hmac authenticated using a pre-shared key. In <code>tls-crypt</code> the control channel is encrypted as well as hmac authenticated using a pre-shared key.
Common to both is that the pre-shared key is shared between the server and all clients. Common to both is that the pre-shared key is shared between the server and all clients.
For large deployments this significantly reduces the usefulness - the key is more likely to be leaked the greate the number of clients who share this key.</p> For large deployments this significantly reduces the usefulness - the key is more likely to be leaked the greate the number of clients who share this key.</p>
<h3>tls-crypt-v2</h3> <h3 id="tls-crypt-v2"><a class="anchor" aria-hidden="true" href="#tls-crypt-v2"></a>tls-crypt-v2</h3>
<p>To improve on <code>tls-crypt</code>, <code>tls-crypt-v2</code> uses one pre-shared key per client. <p>To improve on <code>tls-crypt</code>, <code>tls-crypt-v2</code> uses one pre-shared key per client.
This could be a lot of keys for the server to keep track of, so instead of storing all the client keys on the server the server has a special tls-crypt-v2 server key that is used to <em><a href="https://en.wikipedia.org/wiki/Key_wrap">wrap</a></em> the client keys. This could be a lot of keys for the server to keep track of, so instead of storing all the client keys on the server the server has a special tls-crypt-v2 server key that is used to <em><a href="https://en.wikipedia.org/wiki/Key_wrap">wrap</a></em> the client keys.
That is, each client has their own client key as well as the client key wrapped using the server key. That is, each client has their own client key as well as the client key wrapped using the server key.
@ -98,7 +97,7 @@ The server responds in a similar manner with a sequence number of <code>0x0f0000
At the moment only one tag and one value is defined which signifies the server supports HMAC cookies - this seems unnecessarily complex, but is done to allow future extensibility. At the moment only one tag and one value is defined which signifies the server supports HMAC cookies - this seems unnecessarily complex, but is done to allow future extensibility.
Finally, if the server supports HMAC cookies, the client sends a packet where the wrapped key is appended in cleartext. Finally, if the server supports HMAC cookies, the client sends a packet where the wrapped key is appended in cleartext.
The server is now able to decrypt the third packet without having to keep the key from the first packet around and can verify the session id.</p> The server is now able to decrypt the third packet without having to keep the key from the first packet around and can verify the session id.</p>
<h2>Cool! Let's deploy it!</h2> <h2 id="cool-lets-deploy-it"><a class="anchor" aria-hidden="true" href="#cool-lets-deploy-it"></a>Cool! Let's deploy it!</h2>
<p>Great! <p>Great!
We build on a daily basis unikernels in our <a href="https://builds.robur.coop/">reproducible builds setup</a>. We build on a daily basis unikernels in our <a href="https://builds.robur.coop/">reproducible builds setup</a>.
At the time of writing we have published a <a href="https://builds.robur.coop/job/miragevpn-router">Miragevpn router unikernel</a> acting as a client. At the time of writing we have published a <a href="https://builds.robur.coop/job/miragevpn-router">Miragevpn router unikernel</a> acting as a client.
@ -106,12 +105,8 @@ For general instructions on running Mirage unikernels see our <a href="https://r
The unikernel will need a block device containing the OpenVPN™ configuration and a network device. The unikernel will need a block device containing the OpenVPN™ configuration and a network device.
More detailed instructions Will Follow Soon™! More detailed instructions Will Follow Soon™!
Don't hesitate to reach out to us on <a href="https://github.com/robur-coop/miragevpn/issues">GitHub</a>, <a href="https://robur.coop/Contact">by mail</a> or me personally <a href="https://bsd.network/@reynir">on Mastodon</a> if you're stuck.</p> Don't hesitate to reach out to us on <a href="https://github.com/robur-coop/miragevpn/issues">GitHub</a>, <a href="https://robur.coop/Contact">by mail</a> or me personally <a href="https://bsd.network/@reynir">on Mastodon</a> if you're stuck.</p>
<section role="doc-endnotes"><ol> <p>[^mbedtls]: It is possible to compile OpenVPN™ community edition with Mbed TLS instead of OpenSSL which is written in C as well.</p>
<li id="fn-mbedtls"> <p>[^vpn-network]: I use the term &quot;VPN network&quot; to mean the virtual private network itself. It is a bit odd because the 'N' in 'VPN' is 'Network', but without disambiguation 'VPN' could refer to the network itself, the software or the service.</p>
<p>It is possible to compile OpenVPN™ community edition with Mbed TLS instead of OpenSSL which is written in C as well.</p>
<span><a href="#ref-1-fn-mbedtls" role="doc-backlink" class="fn-label">↩︎︎</a></span></li><li id="fn-vpn-network">
<p>I use the term &quot;VPN network&quot; to mean the virtual private network itself. It is a bit odd because the 'N' in 'VPN' is 'Network', but without disambiguation 'VPN' could refer to the network itself, the software or the service.</p>
<span><a href="#ref-1-fn-vpn-network" role="doc-backlink" class="fn-label">↩︎︎</a></span></li></ol></section>
</article> </article>

19
articles/qubes-miragevpn.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - qubes-miragevpn, a MirageVPN client for QubesOS Robur's blogqubes-miragevpn, a MirageVPN client for QubesOS
</title> </title>
<meta name="description" content="A new OpenVPN client for QubesOS"> <meta name="description" content="A new OpenVPN client for QubesOS">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,11 +20,11 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>qubes-miragevpn, a MirageVPN client for QubesOS</h1> <h1>qubes-miragevpn, a MirageVPN client for QubesOS</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/vpn.html">vpn</a></li><li><a href="/tags/unikernel.html">unikernel</a></li><li><a href="/tags/qubesos.html">qubesos</a></li></ul><p>We are pleased to announce the arrival of a new unikernel: <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-vpn">vpn</a></li><li><a href="https://blog.robur.coop/tags.html#tag-unikernel">unikernel</a></li><li><a href="https://blog.robur.coop/tags.html#tag-QubesOS">QubesOS</a></li></ul><p>We are pleased to announce the arrival of a new unikernel:
<a href="https://github.com/robur-coop/qubes-miragevpn">qubes-miragevpn</a>. The latter is the result of work begun <a href="https://github.com/robur-coop/qubes-miragevpn">qubes-miragevpn</a>. The latter is the result of work begun
several months ago on <a href="https://github.com/robur-coop/miragevpn">miragevpn</a>.</p> several months ago on <a href="https://github.com/robur-coop/miragevpn">miragevpn</a>.</p>
<p>Indeed, with the ambition of completing our unikernel suite and the success of <p>Indeed, with the ambition of completing our unikernel suite and the success of
@ -34,7 +33,7 @@ QubesOS - we thought it would be a good idea to offer this community a unikernel
capable of acting as an OpenVPN client, from which other virtual machines (app capable of acting as an OpenVPN client, from which other virtual machines (app
qubes) can connect so that all their connections pass through the OpenVPN qubes) can connect so that all their connections pass through the OpenVPN
tunnel.</p> tunnel.</p>
<h2>QubesOS &amp; MirageOS</h2> <h2 id="qubesos--mirageos"><a class="anchor" aria-hidden="true" href="#qubesos--mirageos"></a>QubesOS &amp; MirageOS</h2>
<p>Unikernels and QubesOS have always been a tempting idea for users in the sense <p>Unikernels and QubesOS have always been a tempting idea for users in the sense
that a network application (such as a firewall or VPN client) could be smaller that a network application (such as a firewall or VPN client) could be smaller
than a Linux kernel: no keyboard, mouse, wifi management, etc. Just network than a Linux kernel: no keyboard, mouse, wifi management, etc. Just network
@ -60,7 +59,7 @@ fine example of what can actually be done with MirageOS, and of real utility.</p
<a href="https://github.com/mato">mato</a> were (and still are) heavily involved in the work between QubesOS <a href="https://github.com/mato">mato</a> were (and still are) heavily involved in the work between QubesOS
and MirageOS. We'd also like to thank them, because if we're able to continue and MirageOS. We'd also like to thank them, because if we're able to continue
this adventure, it's also thanks to them.</p> this adventure, it's also thanks to them.</p>
<h2>QubesOS &amp; MirageVPN</h2> <h2 id="qubesos--miragevpn"><a class="anchor" aria-hidden="true" href="#qubesos--miragevpn"></a>QubesOS &amp; MirageVPN</h2>
<p>So, after a lengthy development phase for MirageVPN, we set about developing a <p>So, after a lengthy development phase for MirageVPN, we set about developing a
unikernel for QubesOS to offer an OpenVPN client as an operating system. We'd unikernel for QubesOS to offer an OpenVPN client as an operating system. We'd
like to give special thanks to <a href="https://github.com/palainp">Pierre Alain</a>, who helped us to better like to give special thanks to <a href="https://github.com/palainp">Pierre Alain</a>, who helped us to better

107
articles/speeding-ec-string.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - Speeding elliptic curve cryptography Robur's blogSpeeding elliptic curve cryptography
</title> </title>
<meta name="description" content="How we improved the performance of elliptic curves by only modifying the underlying byte array"> <meta name="description" content="How we improved the performance of elliptic curves by only modifying the underlying byte array">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,24 +20,24 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>Speeding elliptic curve cryptography</h1> <h1>Speeding elliptic curve cryptography</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/cryptography.html">cryptography</a></li><li><a href="/tags/security.html">security</a></li></ul><p>TL;DR: replacing cstruct with string, we gain a factor of 2.5 in performance.</p> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-cryptography">cryptography</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li></ul><p>TL;DR: replacing cstruct with string, we gain a factor of 2.5 in performance.</p>
<h2>Mirage-crypto-ec</h2> <h2 id="mirage-crypto-ec"><a class="anchor" aria-hidden="true" href="#mirage-crypto-ec"></a>Mirage-crypto-ec</h2>
<p>In April 2021 We published our implementation of <a href="https://hannes.robur.coop/Posts/EC">elliptic curve cryptography</a> (as <code>mirage-crypto-ec</code> opam package) - this is DSA and DH for NIST curves P224, P256, P384, and P521, and also Ed25519 (EdDSA) and X25519 (ECDH). We use <a href="https://github.com/mit-plv/fiat-crypto/">fiat-crypto</a> for the cryptographic primitives, which emits C code that by construction is correct (note: earlier we stated &quot;free of timing side-channels&quot;, but this is a huge challenge, and as <a href="https://discuss.systems/@edwintorok/111925959867297453">reported by Edwin Török</a> likely impossible on current x86 hardware). More C code (such as <code>point_add</code>, <code>point_double</code>, and further 25519 computations including tables) have been taken from the BoringSSL code base. A lot of OCaml code originates from our TLS 1.3 work in 2018, where Etienne Millon, Nathan Rebours, and Clément Pascutto interfaced <a href="https://github.com/mirage/fiat/">elliptic curves for OCaml</a> (with the goal of being usable with MirageOS).</p> <p>In April 2021 We published our implementation of <a href="https://hannes.robur.coop/Posts/EC">elliptic curve cryptography</a> (as <code>mirage-crypto-ec</code> opam package) - this is DSA and DH for NIST curves P224, P256, P384, and P521, and also Ed25519 (EdDSA) and X25519 (ECDH). We use <a href="https://github.com/mit-plv/fiat-crypto/">fiat-crypto</a> for the cryptographic primitives, which emits C code that by construction is correct (note: earlier we stated &quot;free of timing side-channels&quot;, but this is a huge challenge, and as <a href="https://discuss.systems/@edwintorok/111925959867297453">reported by Edwin Török</a> likely impossible on current x86 hardware). More C code (such as <code>point_add</code>, <code>point_double</code>, and further 25519 computations including tables) have been taken from the BoringSSL code base. A lot of OCaml code originates from our TLS 1.3 work in 2018, where Etienne Millon, Nathan Rebours, and Clément Pascutto interfaced <a href="https://github.com/mirage/fiat/">elliptic curves for OCaml</a> (with the goal of being usable with MirageOS).</p>
<p>The goal of mirage-crypto-ec was: develop elliptic curve support for OCaml &amp; MirageOS quickly - which didn't leave much time to focus on performance. As time goes by, our mileage varies, and we're keen to use fewer resources - and thus fewer CPU time and a smaller memory footprint is preferable.</p> <p>The goal of mirage-crypto-ec was: develop elliptic curve support for OCaml &amp; MirageOS quickly - which didn't leave much time to focus on performance. As time goes by, our mileage varies, and we're keen to use fewer resources - and thus fewer CPU time and a smaller memory footprint is preferable.</p>
<h2>Memory allocation and calls to C</h2> <h2 id="memory-allocation-and-calls-to-c"><a class="anchor" aria-hidden="true" href="#memory-allocation-and-calls-to-c"></a>Memory allocation and calls to C</h2>
<p>OCaml uses managed memory with a generational copying collection. To safely call a C function at any point in time when the arguments are OCaml values (memory allocated on the OCaml heap), it is crucial that while the C function is executed, the arguments should stay at the same memory location, and not being moved by the GC. Otherwise the C code may be upset retrieving wrong data or accessing unmapped memory.</p> <p>OCaml uses managed memory with a generational copying collection. To safely call a C function at any point in time when the arguments are OCaml values (memory allocated on the OCaml heap), it is crucial that while the C function is executed, the arguments should stay at the same memory location, and not being moved by the GC. Otherwise the C code may be upset retrieving wrong data or accessing unmapped memory.</p>
<p>There are several strategies to achieve this, ranging from &quot;let's use another memory area where the GC doesn't mess around with&quot;, &quot;do not run any GC while executing the C code&quot; (read further in the OCaml <a href="https://v2.ocaml.org/releases/4.14/htmlman/intfc.html#ss:c-direct-call">cheaper C calls</a> manual), &quot;deeply copy the arguments to a non-moving memory area before executing C code&quot;, and likely others.</p> <p>There are several strategies to achieve this, ranging from &quot;let's use another memory area where the GC doesn't mess around with&quot;, &quot;do not run any GC while executing the C code&quot; (read further in the OCaml <a href="https://v2.ocaml.org/releases/4.14/htmlman/intfc.html#ss:c-direct-call">cheaper C calls</a> manual), &quot;deeply copy the arguments to a non-moving memory area before executing C code&quot;, and likely others.</p>
<p>For our elliptic curve operations, the C code is pretty simple - there are no memory allocations happening in C, neither are exceptions raised. Also, the execution time of the code is constant and pretty small.</p> <p>For our elliptic curve operations, the C code is pretty simple - there are no memory allocations happening in C, neither are exceptions raised. Also, the execution time of the code is constant and pretty small.</p>
<h2>ocaml-cstruct</h2> <h2 id="ocaml-cstruct"><a class="anchor" aria-hidden="true" href="#ocaml-cstruct"></a>ocaml-cstruct</h2>
<p>In the <a href="https://mirage.io">MirageOS</a> ecosystem, a core library is <a href="https://github.com/mirage/ocaml-cstruct">cstruct</a> - which purpose is manifold: provide ppx rewriters to define C structure layouts in OCaml (getter/setter functions are generated), as well as enums; also a fundamental idea is to use OCaml bigarray which is non-moving memory not allocated on the OCaml heap but directly by calling <code>malloc</code>. The memory can even be page-aligned, as required by some C software, such as Xen. Convenient functionality, such as &quot;retrieve a big-endian unsigned 32 bit integer from offset X in this buffer&quot; are provided as well.</p> <p>In the <a href="https://mirage.io">MirageOS</a> ecosystem, a core library is <a href="https://github.com/mirage/ocaml-cstruct">cstruct</a> - which purpose is manifold: provide ppx rewriters to define C structure layouts in OCaml (getter/setter functions are generated), as well as enums; also a fundamental idea is to use OCaml bigarray which is non-moving memory not allocated on the OCaml heap but directly by calling <code>malloc</code>. The memory can even be page-aligned, as required by some C software, such as Xen. Convenient functionality, such as &quot;retrieve a big-endian unsigned 32 bit integer from offset X in this buffer&quot; are provided as well.</p>
<p>But there's a downside to it - as time moves along, Xen is no longer the only target for MirageOS, and other virtualization mechanisms (such as KVM / virtio) do not require page-aligned memory ranges that are retained at a given memory address. It also turns out that cstruct spends a lot of time in bounds checks. Another huge downside is that OCaml tooling (such as statmemprof) was for a long time (maybe still is not?) unaware of out-of-OCaml-GC allocated memory (cstruct uses bigarray as underlying buffer). Freeing up the memory requires finalizers to be executed - after all pretty tedious (expensive) and against the OCaml runtime philosophy.</p> <p>But there's a downside to it - as time moves along, Xen is no longer the only target for MirageOS, and other virtualization mechanisms (such as KVM / virtio) do not require page-aligned memory ranges that are retained at a given memory address. It also turns out that cstruct spends a lot of time in bounds checks. Another huge downside is that OCaml tooling (such as statmemprof) was for a long time (maybe still is not?) unaware of out-of-OCaml-GC allocated memory (cstruct uses bigarray as underlying buffer). Freeing up the memory requires finalizers to be executed - after all pretty tedious (expensive) and against the OCaml runtime philosophy.</p>
<p>As time moves forward, also the OCaml standard library got support for (a) strings are immutable byte vectors now (since 4.06 - released in 2017 -- there's as well an interface for mutable/immutable cstruct, but that is not used as far as I can tell), (b) retrieve a certain amount of octets in a string or byte as (unsigned) integer number (since 4.08 - released in 2019, while some additional functionality is only available in 4.13).</p> <p>As time moves forward, also the OCaml standard library got support for (a) strings are immutable byte vectors now (since 4.06 - released in 2017 -- there's as well an interface for mutable/immutable cstruct, but that is not used as far as I can tell), (b) retrieve a certain amount of octets in a string or byte as (unsigned) integer number (since 4.08 - released in 2019, while some additional functionality is only available in 4.13).</p>
<p>Still, bigarrays are necessary in certain situations - if you need to have a non-moving (shared) area of memory, as in the Xen interface, but also e.g. when you compute in parallel in different processes, or when you need mmap()ed files.</p> <p>Still, bigarrays are necessary in certain situations - if you need to have a non-moving (shared) area of memory, as in the Xen interface, but also e.g. when you compute in parallel in different processes, or when you need mmap()ed files.</p>
<h2>Putting it together</h2> <h2 id="putting-it-together"><a class="anchor" aria-hidden="true" href="#putting-it-together"></a>Putting it together</h2>
<p>Already in October 2021, Romain <a href="https://github.com/mirage/mirage-crypto/pull/146">proposed</a> to not use cstruct, but bytes for mirage-crypto-ec. The PR was sitting around since there were benchmarks missing, and developer time was small. But recently, Virgile Robles <a href="https://github.com/mirage/mirage-crypto/pull/191">proposed</a> another line of work to use pre-computed tables for NIST curves to speed up the elliptic curve cryptography. Conducting performance evaluation resulted that the &quot;use bytes instead of cstruct&quot; combined with pre-computed tables made a huge difference (factor of 6) compared to the latest release.</p> <p>Already in October 2021, Romain <a href="https://github.com/mirage/mirage-crypto/pull/146">proposed</a> to not use cstruct, but bytes for mirage-crypto-ec. The PR was sitting around since there were benchmarks missing, and developer time was small. But recently, Virgile Robles <a href="https://github.com/mirage/mirage-crypto/pull/191">proposed</a> another line of work to use pre-computed tables for NIST curves to speed up the elliptic curve cryptography. Conducting performance evaluation resulted that the &quot;use bytes instead of cstruct&quot; combined with pre-computed tables made a huge difference (factor of 6) compared to the latest release.</p>
<p>To ease reviewing changes, we decided to focus on landing the &quot;use bytes instead of cstruct&quot; first, and gladly Pierre Alain had already rebased the existing patch onto the latest release of mirage-crypto-ec. We also went further and use string where applicable instead of bytes. For safety reasons we also introduced an API layer which (a) allocates a byte vector for the result (b) calls the primitive, and (c) transforms the byte vector into an immutable string. This API is more in line with functional programming (immutable values), and since allocations and deallocations of values are cheap, there's no measurable performance decrease.</p> <p>To ease reviewing changes, we decided to focus on landing the &quot;use bytes instead of cstruct&quot; first, and gladly Pierre Alain had already rebased the existing patch onto the latest release of mirage-crypto-ec. We also went further and use string where applicable instead of bytes. For safety reasons we also introduced an API layer which (a) allocates a byte vector for the result (b) calls the primitive, and (c) transforms the byte vector into an immutable string. This API is more in line with functional programming (immutable values), and since allocations and deallocations of values are cheap, there's no measurable performance decrease.</p>
<p>All the changes are internal, there's no external API that needs to be adjusted - still there's at the API boundary one conversion of cstruct to string (and back for the return value) done.</p> <p>All the changes are internal, there's no external API that needs to be adjusted - still there's at the API boundary one conversion of cstruct to string (and back for the return value) done.</p>
@ -48,81 +47,25 @@
<p>With PR#146, the flame graph looks different:</p> <p>With PR#146, the flame graph looks different:</p>
<p><img src="../images/trace-string-770.svg" alt="Flamegraph of ECDSA sign with string" ></p> <p><img src="../images/trace-string-770.svg" alt="Flamegraph of ECDSA sign with string" ></p>
<p>Now, the allocation towers do not exist anymore. The time of a sign operation is spend in <code>inv</code>, <code>scalar_mult</code>, and <code>x_of_finite_point_mod_n</code>. There's still room for improvements in these operations.</p> <p>Now, the allocation towers do not exist anymore. The time of a sign operation is spend in <code>inv</code>, <code>scalar_mult</code>, and <code>x_of_finite_point_mod_n</code>. There's still room for improvements in these operations.</p>
<h2>Performance numbers</h2> <h2 id="performance-numbers"><a class="anchor" aria-hidden="true" href="#performance-numbers"></a>Performance numbers</h2>
<p>All numbers were gathered on a Lenovo X250 laptop with a Intel i7-5600U CPU @ 2.60GHz. We used OCaml 4.14.1 as compiler. The baseline is OpenSSL 3.0.12. All numbers are in operations per second.</p> <p>All numbers were gathered on a Lenovo X250 laptop with a Intel i7-5600U CPU @ 2.60GHz. We used OCaml 4.14.1 as compiler. The baseline is OpenSSL 3.0.12. All numbers are in operations per second.</p>
<p>NIST P-256</p> <p>NIST P-256</p>
<div role="region"><table> <p>| op | 0.11.2 | PR#146 | speedup | OpenSSL | speedup |
<tr> | - | - | - | - | - | - |
<th>op</th> | sign | 748 | 1806 | 2.41x | 34392 | 19.04x |
<th>0.11.2</th> | verify | 285 | 655 | 2.30x | 12999 | 19.85x |
<th>PR#146</th> | ecdh | 858 | 1785 | 2.08x | 16514 | 9.25x |</p>
<th>speedup</th> <p>Curve 25519</p>
<th>OpenSSL</th> <p>| op | 0.11.2 | PR#146 | speedup | OpenSSL | speedup |
<th>speedup</th> | - | - | - | - | - | - |
</tr> | sign | 10713 | 11560 | 1.08x | 21943 | 1.90x |
<tr> | verify | 7600 | 8314 | 1.09x | 7081 | 0.85x |
<td>sign</td> | ecdh | 12144 | 13457 | 1.11x | 26201 | 1.95x |</p>
<td>748</td> <p>Note: to re-create the performance numbers, you can run <code>openssl speed ecdsap256 ecdhp256 ed25519 ecdhx25519</code> - for the OCaml site, use <code>dune bu bench/speed.exe --rel</code> and <code>_build/default/bench/speed.exe ecdsa-sign ecdsa-verify ecdh-share</code>.</p>
<td>1806</td>
<td>2.41x</td>
<td>34392</td>
<td>19.04x</td>
</tr>
<tr>
<td>verify</td>
<td>285</td>
<td>655</td>
<td>2.30x</td>
<td>12999</td>
<td>19.85x</td>
</tr>
<tr>
<td>ecdh</td>
<td>858</td>
<td>1785</td>
<td>2.08x</td>
<td>16514</td>
<td>9.25x</td>
</tr>
</table></div><p>Curve 25519</p>
<div role="region"><table>
<tr>
<th>op</th>
<th>0.11.2</th>
<th>PR#146</th>
<th>speedup</th>
<th>OpenSSL</th>
<th>speedup</th>
</tr>
<tr>
<td>sign</td>
<td>10713</td>
<td>11560</td>
<td>1.08x</td>
<td>21943</td>
<td>1.90x</td>
</tr>
<tr>
<td>verify</td>
<td>7600</td>
<td>8314</td>
<td>1.09x</td>
<td>7081</td>
<td>0.85x</td>
</tr>
<tr>
<td>ecdh</td>
<td>12144</td>
<td>13457</td>
<td>1.11x</td>
<td>26201</td>
<td>1.95x</td>
</tr>
</table></div><p>Note: to re-create the performance numbers, you can run <code>openssl speed ecdsap256 ecdhp256 ed25519 ecdhx25519</code> - for the OCaml site, use <code>dune bu bench/speed.exe --rel</code> and <code>_build/default/bench/speed.exe ecdsa-sign ecdsa-verify ecdh-share</code>.</p>
<p>The performance improvements are up to 2.5 times compared to the latest mirage-crypto-ec release (look at the 4th column). In comparison to OpenSSL, we still lack a factor of 20 for the NIST curves, and up to a factor of 2 for 25519 computations (look at the last column).</p> <p>The performance improvements are up to 2.5 times compared to the latest mirage-crypto-ec release (look at the 4th column). In comparison to OpenSSL, we still lack a factor of 20 for the NIST curves, and up to a factor of 2 for 25519 computations (look at the last column).</p>
<p>If you have ideas for improvements, let us know via an issue, eMail, or a pull request :) We started to <a href="https://github.com/mirage/mirage-crypto/issues/193">gather some</a> for 25519 by comparing our code with changes in BoringSSL over the last years.</p> <p>If you have ideas for improvements, let us know via an issue, eMail, or a pull request :) We started to <a href="https://github.com/mirage/mirage-crypto/issues/193">gather some</a> for 25519 by comparing our code with changes in BoringSSL over the last years.</p>
<p>As a spoiler, for P-256 sign there's another improvement of around 4.5 with <a href="https://github.com/mirage/mirage-crypto/pull/191">Virgile's PR</a> using pre-computed tables also for NIST curves.</p> <p>As a spoiler, for P-256 sign there's another improvement of around 4.5 with <a href="https://github.com/mirage/mirage-crypto/pull/191">Virgile's PR</a> using pre-computed tables also for NIST curves.</p>
<h2>The road ahead for 2024</h2> <h2 id="the-road-ahead-for-2024"><a class="anchor" aria-hidden="true" href="#the-road-ahead-for-2024"></a>The road ahead for 2024</h2>
<p>Remove all cstruct, everywhere, apart from in mirage-block-xen and mirage-net-xen ;). It was a fine decision in the early MirageOS days, but from a performance point of view, and for making our packages more broadly usable without many dependencies, it is time to remove cstruct. Earlier this year we already <a href="https://github.com/mirage/ocaml-tar/pull/137">removed cstruct from ocaml-tar</a> for similar reasons.</p> <p>Remove all cstruct, everywhere, apart from in mirage-block-xen and mirage-net-xen ;). It was a fine decision in the early MirageOS days, but from a performance point of view, and for making our packages more broadly usable without many dependencies, it is time to remove cstruct. Earlier this year we already <a href="https://github.com/mirage/ocaml-tar/pull/137">removed cstruct from ocaml-tar</a> for similar reasons.</p>
<p>Our MirageOS work is only partially funded, we cross-fund our work by commercial contracts and public (EU) funding. We are part of a non-profit company, you can make a (tax-deducable - at least in the EU) <a href="https://aenderwerk.de/donate/">donation</a> (select &quot;DONATION robur&quot; in the dropdown menu).</p> <p>Our MirageOS work is only partially funded, we cross-fund our work by commercial contracts and public (EU) funding. We are part of a non-profit company, you can make a (tax-deducable - at least in the EU) <a href="https://aenderwerk.de/donate/">donation</a> (select &quot;DONATION robur&quot; in the dropdown menu).</p>
<p>We're keen to get MirageOS deployed in production - if you would like to do that, don't hesitate to reach out to us via eMail team at robur.coop</p> <p>We're keen to get MirageOS deployed in production - if you would like to do that, don't hesitate to reach out to us via eMail team at robur.coop</p>

31
articles/tar-release.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - The new Tar release, a retrospective Robur's blogThe new Tar release, a retrospective
</title> </title>
<meta name="description" content="A little retrospective to the new Tar release and changes"> <meta name="description" content="A little retrospective to the new Tar release and changes">
<link type="text/css" rel="stylesheet" href="../css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="../css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="../js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="../feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,11 +20,11 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a href="/index.html">Back to index</a> <main><a href="https://blog.robur.coop/index.html">Back to index</a>
<article> <article>
<h1>The new Tar release, a retrospective</h1> <h1>The new Tar release, a retrospective</h1>
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/cstruct.html">cstruct</a></li><li><a href="/tags/functors.html">functors</a></li></ul><p>We are delighted to announce the new release of <code>ocaml-tar</code>. A small library for <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Cstruct">Cstruct</a></li><li><a href="https://blog.robur.coop/tags.html#tag-functors">functors</a></li></ul><p>We are delighted to announce the new release of <code>ocaml-tar</code>. A small library for
reading and writing tar archives in OCaml. Since this is a major release, we'll reading and writing tar archives in OCaml. Since this is a major release, we'll
take the time in this article to explain the work that's been done by the take the time in this article to explain the work that's been done by the
cooperative on this project.</p> cooperative on this project.</p>
@ -37,7 +36,7 @@ and a project that's been around since... 2012 (over 10 years!).</p>
<p>But we intend to maintain and improve it, since we're using it for the <p>But we intend to maintain and improve it, since we're using it for the
<a href="https://hannes.robur.coop/Posts/OpamMirror">opam-mirror</a> project among other things - this unikernel is to <a href="https://hannes.robur.coop/Posts/OpamMirror">opam-mirror</a> project among other things - this unikernel is to
provide an opam-repository &quot;tarball&quot; for opam when you do <code>opam update</code>.</p> provide an opam-repository &quot;tarball&quot; for opam when you do <code>opam update</code>.</p>
<h2><code>Cstruct.t</code> &amp; bytes</h2> <h2 id="cstructt--bytes"><a class="anchor" aria-hidden="true" href="#cstructt--bytes"></a><code>Cstruct.t</code> &amp; bytes</h2>
<p>As some of you may have noticed, over the last few months we've begun a fairly <p>As some of you may have noticed, over the last few months we've begun a fairly
substantial change to the Mirage ecosystem, replacing the use of <code>Cstruct.t</code> in substantial change to the Mirage ecosystem, replacing the use of <code>Cstruct.t</code> in
key places with bytes/string.</p> key places with bytes/string.</p>
@ -54,7 +53,7 @@ buffer, invalid access). There's also a small benchmark to support our initial
intuition<sup><a href="#fn1">1</a></sup>.</p> intuition<sup><a href="#fn1">1</a></sup>.</p>
<p>But this PR can also be an opportunity to understand the existence of <p>But this PR can also be an opportunity to understand the existence of
<code>Cstruct.t</code> in the Mirage ecosystem and the reasons for this historic choice.</p> <code>Cstruct.t</code> in the Mirage ecosystem and the reasons for this historic choice.</p>
<h3><code>Cstruct.t</code> as a non-moveable data</h3> <h3 id="cstructt-as-a-non-moveable-data"><a class="anchor" aria-hidden="true" href="#cstructt-as-a-non-moveable-data"></a><code>Cstruct.t</code> as a non-moveable data</h3>
<p>I've already <a href="https://discuss.ocaml.org/t/buffered-io-bytes-vs-bigstring/8978/3">made</a> a list of pros/cons when it comes to <p>I've already <a href="https://discuss.ocaml.org/t/buffered-io-bytes-vs-bigstring/8978/3">made</a> a list of pros/cons when it comes to
bigarrays. Indeed, <code>Cstruct.t</code> is based on a bigarray:</p> bigarrays. Indeed, <code>Cstruct.t</code> is based on a bigarray:</p>
<pre><code class="language-ocaml">type buffer = (char, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t <pre><code class="language-ocaml">type buffer = (char, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t
@ -111,7 +110,7 @@ knowing when to <code>free()</code> the zone as soon as the value is no longer i
Reference-counting is used to then allocate &quot;small&quot; values in the OCaml heap Reference-counting is used to then allocate &quot;small&quot; values in the OCaml heap
and use them to manipulate <em>indirectly</em> the bigarray.</li> and use them to manipulate <em>indirectly</em> the bigarray.</li>
</ul> </ul>
<h4>Ownership, proxy and GC</h4> <h4 id="ownership-proxy-and-gc"><a class="anchor" aria-hidden="true" href="#ownership-proxy-and-gc"></a>Ownership, proxy and GC</h4>
<p>This last point deserves a little clarification, particularly with regard to the <p>This last point deserves a little clarification, particularly with regard to the
<code>Bigarray.sub</code> function. This function will not create a new, smaller bigarray <code>Bigarray.sub</code> function. This function will not create a new, smaller bigarray
and copy what was in the old one to the new one (as <code>Bytes.sub</code>/<code>String.sub</code> and copy what was in the old one to the new one (as <code>Bytes.sub</code>/<code>String.sub</code>
@ -155,7 +154,7 @@ ownership of a <code>Cstruct.t</code> in the type to prevent unauthorized access
place on this subject<sup><a href="#fn2">2</a></sup>.</p> place on this subject<sup><a href="#fn2">2</a></sup>.</p>
<p>It should be noted that, with regard to the third point, the problem also <p>It should be noted that, with regard to the third point, the problem also
applies to bytes and the use of <code>Bytes.unsafe_to_string</code>!</p> applies to bytes and the use of <code>Bytes.unsafe_to_string</code>!</p>
<h3>Conclusion about Cstruct</h3> <h3 id="conclusion-about-cstruct"><a class="anchor" aria-hidden="true" href="#conclusion-about-cstruct"></a>Conclusion about Cstruct</h3>
<p>We hope we've been thorough enough in our experience with Cstruct. If we go back <p>We hope we've been thorough enough in our experience with Cstruct. If we go back
to the initial definition of our <code>Cstruct.t</code> shown above and take all the to the initial definition of our <code>Cstruct.t</code> shown above and take all the
history into account, it becomes increasingly difficult to argue for a history into account, it becomes increasingly difficult to argue for a
@ -181,7 +180,7 @@ seems that the change is worthwhile.</p>
capabilities and using the type system to enforce certain characteristics. To capabilities and using the type system to enforce certain characteristics. To
date, <code>Cstruct_cap</code> has not been used anywhere, which raises a real question date, <code>Cstruct_cap</code> has not been used anywhere, which raises a real question
about the advantages/disadvantages in everyday use.</p> about the advantages/disadvantages in everyday use.</p>
<h2>Functors</h2> <h2 id="functors"><a class="anchor" aria-hidden="true" href="#functors"></a>Functors</h2>
<p>This is perhaps the other point of the Mirage ecosystem that is also the subject <p>This is perhaps the other point of the Mirage ecosystem that is also the subject
of debate. Functors! Before we talk about functors, we need to understand their of debate. Functors! Before we talk about functors, we need to understand their
relevance in the context of Mirage.</p> relevance in the context of Mirage.</p>
@ -272,7 +271,7 @@ do, then let the user implement the <code>run</code> function according to the s
<p>In short, based on this list and the various experiments we've carried out on a <p>In short, based on this list and the various experiments we've carried out on a
number of projects, we've decided to remove the functors from <code>ocaml-tar</code>! The number of projects, we've decided to remove the functors from <code>ocaml-tar</code>! The
crucial question now is: which method to choose?</p> crucial question now is: which method to choose?</p>
<h3>The best answers</h3> <h3 id="the-best-answers"><a class="anchor" aria-hidden="true" href="#the-best-answers"></a>The best answers</h3>
<p>There's no real answer to that, and in truth it depends on what level of <p>There's no real answer to that, and in truth it depends on what level of
abstraction you're at. In fact, you'd like to have a fairly simple method of abstraction you're at. In fact, you'd like to have a fairly simple method of
abstraction from the system at the start and at the lowest level, to end up abstraction from the system at the start and at the lowest level, to end up
@ -316,7 +315,7 @@ type ('a, 'err) t =
</code></pre> </code></pre>
<p>However, and this is where we come back to OCaml's limitations and where <p>However, and this is where we come back to OCaml's limitations and where
functors could help us: higher kinded polymorphism!</p> functors could help us: higher kinded polymorphism!</p>
<h3>Higher kinded Polymorphism</h3> <h3 id="higher-kinded-polymorphism"><a class="anchor" aria-hidden="true" href="#higher-kinded-polymorphism"></a>Higher kinded Polymorphism</h3>
<p>If we return to our functor example above, there's one element that may be of <p>If we return to our functor example above, there's one element that may be of
interest: <code>T with type t = T.t succ</code></p> interest: <code>T with type t = T.t succ</code></p>
<p>In other words, add a constraint to a signature type. A constraint often seen <p>In other words, add a constraint to a signature type. A constraint often seen
@ -388,7 +387,7 @@ val miou : 'a -&gt; ('a, 'err, miou) t
</code></pre> </code></pre>
<p>But this way, Tar can always be derived from another system, and the process for <p>But this way, Tar can always be derived from another system, and the process for
extracting entries from a Tar file is the same for <strong>all</strong> systems!</p> extracting entries from a Tar file is the same for <strong>all</strong> systems!</p>
<h2>Conclusion</h2> <h2 id="conclusion"><a class="anchor" aria-hidden="true" href="#conclusion"></a>Conclusion</h2>
<p>This Tar release isn't as impressive as this article, but it does sum up all the <p>This Tar release isn't as impressive as this article, but it does sum up all the
work we've been able to do over the last few months and years. We hope that our work we've been able to do over the last few months and years. We hope that our
work is appreciated and that this article, which sets out all the thoughts we've work is appreciated and that this article, which sets out all the thoughts we've

BIN
css/.style.css.swp Normal file
View file

Binary file not shown.

4
css/style.css
View file

@ -197,6 +197,10 @@ article code {
color: #fff; color: #fff;
} }
.tag-box:target > h3 > span {
background-color: #c2410c;
}
.tag-box > h3 > span::before { .tag-box > h3 > span::before {
content: "#"; content: "#";
} }

61
index.html
View file

@ -1,4 +1,3 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
@ -6,13 +5,13 @@
<meta http-equiv="x-ua-compatible" content="ie=edge"> <meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title> <title>
Robur's blog - Index Robur's blogIndex
</title> </title>
<meta name="description" content="The famous root of the website"> <meta name="description" content="The famous root of the website">
<link type="text/css" rel="stylesheet" href="./css/hl.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/hl.css">
<link type="text/css" rel="stylesheet" href="./css/style.css"> <link type="text/css" rel="stylesheet" href="https://blog.robur.coop/css/style.css">
<script src="./js/hl.js"></script> <script src="https://blog.robur.coop/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="./feed.xml" title="blog.robur.coop"> <link rel="alternate" type="application/rss+xml" href="https://blog.robur.coop/feed.xml" title="blog.robur.coop">
</head> </head>
<body> <body>
<header> <header>
@ -21,7 +20,7 @@
The <strong>Robur</strong> cooperative blog. The <strong>Robur</strong> cooperative blog.
</blockquote> </blockquote>
</header> </header>
<main><a class="small-button rss" href="./feed.xml">RSS</a><p>The Robur blog.</p> <main><a class="small-button rss" href="https://blog.robur.coop/feed.xml">RSS</a><p>The Robur blog.</p>
<h3>Essays and ramblings</h3> <h3>Essays and ramblings</h3>
@ -32,10 +31,10 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2024-08-21</span> <span class="date">2024-08-21</span>
<a href="articles/2024-08-21-OpenVPN-and-MirageVPN.html">MirageVPN and OpenVPN</a><br /> <a href="https://blog.robur.coop/articles/2024-08-21-OpenVPN-and-MirageVPN.html">MirageVPN and OpenVPN</a><br />
<p>Discoveries made implementing MirageVPN, a OpenVPN-compatible VPN library</p> <p>Discoveries made implementing MirageVPN, a OpenVPN-compatible VPN library</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/miragevpn.html">miragevpn</a></li><li><a href="/tags/openvpn.html">openvpn</a></li><li><a href="/tags/security.html">security</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-MirageVPN">MirageVPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-OpenVPN">OpenVPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
@ -45,10 +44,10 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2024-08-15</span> <span class="date">2024-08-15</span>
<a href="articles/tar-release.html">The new Tar release, a retrospective</a><br /> <a href="https://blog.robur.coop/articles/tar-release.html">The new Tar release, a retrospective</a><br />
<p>A little retrospective to the new Tar release and changes</p> <p>A little retrospective to the new Tar release and changes</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/cstruct.html">cstruct</a></li><li><a href="/tags/functors.html">functors</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Cstruct">Cstruct</a></li><li><a href="https://blog.robur.coop/tags.html#tag-functors">functors</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
@ -58,10 +57,10 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2024-06-24</span> <span class="date">2024-06-24</span>
<a href="articles/qubes-miragevpn.html">qubes-miragevpn, a MirageVPN client for QubesOS</a><br /> <a href="https://blog.robur.coop/articles/qubes-miragevpn.html">qubes-miragevpn, a MirageVPN client for QubesOS</a><br />
<p>A new OpenVPN client for QubesOS</p> <p>A new OpenVPN client for QubesOS</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/vpn.html">vpn</a></li><li><a href="/tags/unikernel.html">unikernel</a></li><li><a href="/tags/qubesos.html">qubesos</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-vpn">vpn</a></li><li><a href="https://blog.robur.coop/tags.html#tag-unikernel">unikernel</a></li><li><a href="https://blog.robur.coop/tags.html#tag-QubesOS">QubesOS</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
@ -71,10 +70,10 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2024-06-17</span> <span class="date">2024-06-17</span>
<a href="articles/miragevpn-server.html">MirageVPN server</a><br /> <a href="https://blog.robur.coop/articles/miragevpn-server.html">MirageVPN server</a><br />
<p>Announcement of our MirageVPN server.</p> <p>Announcement of our MirageVPN server.</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/cryptography.html">cryptography</a></li><li><a href="/tags/security.html">security</a></li><li><a href="/tags/vpn.html">vpn</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-cryptography">cryptography</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li><li><a href="https://blog.robur.coop/tags.html#tag-VPN">VPN</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
@ -84,10 +83,10 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2024-04-16</span> <span class="date">2024-04-16</span>
<a href="articles/miragevpn-performance.html">Speeding up MirageVPN and use it in the wild</a><br /> <a href="https://blog.robur.coop/articles/miragevpn-performance.html">Speeding up MirageVPN and use it in the wild</a><br />
<p>Performance engineering of MirageVPN, speeding it up by a factor of 25.</p> <p>Performance engineering of MirageVPN, speeding it up by a factor of 25.</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/cryptography.html">cryptography</a></li><li><a href="/tags/security.html">security</a></li><li><a href="/tags/vpn.html">vpn</a></li><li><a href="/tags/performance.html">performance</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-cryptography">cryptography</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li><li><a href="https://blog.robur.coop/tags.html#tag-VPN">VPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-performance">performance</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
@ -97,10 +96,10 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2024-02-21</span> <span class="date">2024-02-21</span>
<a href="articles/gptar.html">GPTar</a><br /> <a href="https://blog.robur.coop/articles/gptar.html">GPTar</a><br />
<p>Hybrid GUID partition table and tar archive</p> <p>Hybrid GUID partition table and tar archive</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/gpt.html">gpt</a></li><li><a href="/tags/tar.html">tar</a></li><li><a href="/tags/mbr.html">mbr</a></li><li><a href="/tags/persistent storage.html">persistent storage</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-gpt">gpt</a></li><li><a href="https://blog.robur.coop/tags.html#tag-tar">tar</a></li><li><a href="https://blog.robur.coop/tags.html#tag-mbr">mbr</a></li><li><a href="https://blog.robur.coop/tags.html#tag-persistent storage">persistent storage</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
@ -110,23 +109,23 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2024-02-13</span> <span class="date">2024-02-13</span>
<a href="articles/speeding-ec-string.html">Speeding elliptic curve cryptography</a><br /> <a href="https://blog.robur.coop/articles/speeding-ec-string.html">Speeding elliptic curve cryptography</a><br />
<p>How we improved the performance of elliptic curves by only modifying the underlying byte array</p> <p>How we improved the performance of elliptic curves by only modifying the underlying byte array</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/cryptography.html">cryptography</a></li><li><a href="/tags/security.html">security</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-cryptography">cryptography</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
<div class="side"> <div class="side">
<a href="https://blog.robur.coop/"> <a href="https://blog.osau.re/">
<img src="https://www.gravatar.com/avatar/12de8f3dc8e39098964964c759c981f1"> <img src="https://www.gravatar.com/avatar/e243d18f97471424ca390e85820797ac">
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2024-02-11</span> <span class="date">2024-02-11</span>
<a href="articles/lwt_pause.html">Cooperation and Lwt.pause</a><br /> <a href="https://blog.robur.coop/articles/lwt_pause.html">Cooperation and Lwt.pause</a><br />
<p>A disgression about Lwt and Miou</p> <p>A disgression about Lwt and Miou</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/scheduler.html">scheduler</a></li><li><a href="/tags/community.html">community</a></li><li><a href="/tags/unikernel.html">unikernel</a></li><li><a href="/tags/git.html">git</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Scheduler">Scheduler</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Community">Community</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Unikernel">Unikernel</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Git">Git</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
@ -136,10 +135,10 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2024-02-03</span> <span class="date">2024-02-03</span>
<a href="articles/2024-02-03-python-str-repr.html">Python&apos;s `str.__repr__()`</a><br /> <a href="https://blog.robur.coop/articles/2024-02-03-python-str-repr.html">Python&apos;s `str.__repr__()`</a><br />
<p>Reimplementing Python string escaping in OCaml</p> <p>Reimplementing Python string escaping in OCaml</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/python.html">python</a></li><li><a href="/tags/unicode.html">unicode</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-Python">Python</a></li><li><a href="https://blog.robur.coop/tags.html#tag-unicode">unicode</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
@ -149,10 +148,10 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2023-11-20</span> <span class="date">2023-11-20</span>
<a href="articles/miragevpn-ncp.html">MirageVPN updated (AEAD, NCP)</a><br /> <a href="https://blog.robur.coop/articles/miragevpn-ncp.html">MirageVPN updated (AEAD, NCP)</a><br />
<p>How we resurrected MirageVPN from its bitrot state</p> <p>How we resurrected MirageVPN from its bitrot state</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/vpn.html">vpn</a></li><li><a href="/tags/security.html">security</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-VPN">VPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li></ul>
</div> </div>
</div> </div>
</li><li> </li><li>
@ -162,10 +161,10 @@
</a></div> </a></div>
<div class="content"> <div class="content">
<span class="date">2023-11-14</span> <span class="date">2023-11-14</span>
<a href="articles/miragevpn.html">MirageVPN &amp; tls-crypt-v2</a><br /> <a href="https://blog.robur.coop/articles/miragevpn.html">MirageVPN &amp; tls-crypt-v2</a><br />
<p>How we implementated tls-crypt-v2 for miragevpn</p> <p>How we implementated tls-crypt-v2 for miragevpn</p>
<div class="bottom"> <div class="bottom">
<ul class="tags-list"><li><a href="/tags/ocaml.html">ocaml</a></li><li><a href="/tags/mirageos.html">mirageos</a></li><li><a href="/tags/vpn.html">vpn</a></li><li><a href="/tags/security.html">security</a></li></ul> <ul class="tags-list"><li><a href="https://blog.robur.coop/tags.html#tag-OCaml">OCaml</a></li><li><a href="https://blog.robur.coop/tags.html#tag-MirageOS">MirageOS</a></li><li><a href="https://blog.robur.coop/tags.html#tag-VPN">VPN</a></li><li><a href="https://blog.robur.coop/tags.html#tag-security">security</a></li></ul>
</div> </div>
</div> </div>
</li></ol> </li></ol>

148
tags.html Normal file
View file

@ -0,0 +1,148 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>
Robur's blog
</title>
<meta name="description" content="">
<link type="text/css" rel="stylesheet" href="/css/hl.css">
<link type="text/css" rel="stylesheet" href="/css/style.css">
<script src="/js/hl.js"></script>
<link rel="alternate" type="application/rss+xml" href="/feed.xml" title="blog.robur.coop">
</head>
<body>
<header>
<h1>blog.robur.coop</h1>
<blockquote>
The <strong>Robur</strong> cooperative blog.
</blockquote>
</header>
<main><a href="/index.html">Back to index</a>
<ul class="tags-list aeration"><li><a href="#tag-Community">Community</a></li><li><a href="#tag-Cstruct">Cstruct</a></li><li><a href="#tag-Git">Git</a></li><li><a href="#tag-MirageOS">MirageOS</a></li><li><a href="#tag-MirageVPN">MirageVPN</a></li><li><a href="#tag-OCaml">OCaml</a></li><li><a href="#tag-OpenVPN">OpenVPN</a></li><li><a href="#tag-Python">Python</a></li><li><a href="#tag-QubesOS">QubesOS</a></li><li><a href="#tag-Scheduler">Scheduler</a></li><li><a href="#tag-Unikernel">Unikernel</a></li><li><a href="#tag-VPN">VPN</a></li><li><a href="#tag-cryptography">cryptography</a></li><li><a href="#tag-functors">functors</a></li><li><a href="#tag-gpt">gpt</a></li><li><a href="#tag-mbr">mbr</a></li><li><a href="#tag-performance">performance</a></li><li><a href="#tag-persistent storage">persistent storage</a></li><li><a href="#tag-security">security</a></li><li><a href="#tag-tar">tar</a></li><li><a href="#tag-unicode">unicode</a></li><li><a href="#tag-unikernel">unikernel</a></li><li><a href="#tag-vpn">vpn</a></li></ul><div class="tag-box" id="tag-Community">
<h3>
<span>Community</span>
</h3>
<ul><li><a href="/articles/lwt_pause.html">Cooperation and Lwt.pause</a></li></ul>
</div><div class="tag-box" id="tag-Cstruct">
<h3>
<span>Cstruct</span>
</h3>
<ul><li><a href="/articles/tar-release.html">The new Tar release, a retrospective</a></li></ul>
</div><div class="tag-box" id="tag-Git">
<h3>
<span>Git</span>
</h3>
<ul><li><a href="/articles/lwt_pause.html">Cooperation and Lwt.pause</a></li></ul>
</div><div class="tag-box" id="tag-MirageOS">
<h3>
<span>MirageOS</span>
</h3>
<ul><li><a href="/articles/miragevpn.html">MirageVPN &amp; tls-crypt-v2</a></li><li><a href="/articles/miragevpn-ncp.html">MirageVPN updated (AEAD, NCP)</a></li><li><a href="/articles/speeding-ec-string.html">Speeding elliptic curve cryptography</a></li><li><a href="/articles/miragevpn-performance.html">Speeding up MirageVPN and use it in the wild</a></li><li><a href="/articles/miragevpn-server.html">MirageVPN server</a></li></ul>
</div><div class="tag-box" id="tag-MirageVPN">
<h3>
<span>MirageVPN</span>
</h3>
<ul><li><a href="/articles/2024-08-21-OpenVPN-and-MirageVPN.html">MirageVPN and OpenVPN</a></li></ul>
</div><div class="tag-box" id="tag-OCaml">
<h3>
<span>OCaml</span>
</h3>
<ul><li><a href="/articles/miragevpn.html">MirageVPN &amp; tls-crypt-v2</a></li><li><a href="/articles/miragevpn-ncp.html">MirageVPN updated (AEAD, NCP)</a></li><li><a href="/articles/2024-02-03-python-str-repr.html">Python&apos;s `str.__repr__()`</a></li><li><a href="/articles/lwt_pause.html">Cooperation and Lwt.pause</a></li><li><a href="/articles/speeding-ec-string.html">Speeding elliptic curve cryptography</a></li><li><a href="/articles/gptar.html">GPTar</a></li><li><a href="/articles/miragevpn-performance.html">Speeding up MirageVPN and use it in the wild</a></li><li><a href="/articles/miragevpn-server.html">MirageVPN server</a></li><li><a href="/articles/qubes-miragevpn.html">qubes-miragevpn, a MirageVPN client for QubesOS</a></li><li><a href="/articles/tar-release.html">The new Tar release, a retrospective</a></li></ul>
</div><div class="tag-box" id="tag-OpenVPN">
<h3>
<span>OpenVPN</span>
</h3>
<ul><li><a href="/articles/2024-08-21-OpenVPN-and-MirageVPN.html">MirageVPN and OpenVPN</a></li></ul>
</div><div class="tag-box" id="tag-Python">
<h3>
<span>Python</span>
</h3>
<ul><li><a href="/articles/2024-02-03-python-str-repr.html">Python&apos;s `str.__repr__()`</a></li></ul>
</div><div class="tag-box" id="tag-QubesOS">
<h3>
<span>QubesOS</span>
</h3>
<ul><li><a href="/articles/qubes-miragevpn.html">qubes-miragevpn, a MirageVPN client for QubesOS</a></li></ul>
</div><div class="tag-box" id="tag-Scheduler">
<h3>
<span>Scheduler</span>
</h3>
<ul><li><a href="/articles/lwt_pause.html">Cooperation and Lwt.pause</a></li></ul>
</div><div class="tag-box" id="tag-Unikernel">
<h3>
<span>Unikernel</span>
</h3>
<ul><li><a href="/articles/lwt_pause.html">Cooperation and Lwt.pause</a></li></ul>
</div><div class="tag-box" id="tag-VPN">
<h3>
<span>VPN</span>
</h3>
<ul><li><a href="/articles/miragevpn.html">MirageVPN &amp; tls-crypt-v2</a></li><li><a href="/articles/miragevpn-ncp.html">MirageVPN updated (AEAD, NCP)</a></li><li><a href="/articles/miragevpn-performance.html">Speeding up MirageVPN and use it in the wild</a></li><li><a href="/articles/miragevpn-server.html">MirageVPN server</a></li></ul>
</div><div class="tag-box" id="tag-cryptography">
<h3>
<span>cryptography</span>
</h3>
<ul><li><a href="/articles/speeding-ec-string.html">Speeding elliptic curve cryptography</a></li><li><a href="/articles/miragevpn-performance.html">Speeding up MirageVPN and use it in the wild</a></li><li><a href="/articles/miragevpn-server.html">MirageVPN server</a></li></ul>
</div><div class="tag-box" id="tag-functors">
<h3>
<span>functors</span>
</h3>
<ul><li><a href="/articles/tar-release.html">The new Tar release, a retrospective</a></li></ul>
</div><div class="tag-box" id="tag-gpt">
<h3>
<span>gpt</span>
</h3>
<ul><li><a href="/articles/gptar.html">GPTar</a></li></ul>
</div><div class="tag-box" id="tag-mbr">
<h3>
<span>mbr</span>
</h3>
<ul><li><a href="/articles/gptar.html">GPTar</a></li></ul>
</div><div class="tag-box" id="tag-performance">
<h3>
<span>performance</span>
</h3>
<ul><li><a href="/articles/miragevpn-performance.html">Speeding up MirageVPN and use it in the wild</a></li></ul>
</div><div class="tag-box" id="tag-persistent storage">
<h3>
<span>persistent storage</span>
</h3>
<ul><li><a href="/articles/gptar.html">GPTar</a></li></ul>
</div><div class="tag-box" id="tag-security">
<h3>
<span>security</span>
</h3>
<ul><li><a href="/articles/miragevpn.html">MirageVPN &amp; tls-crypt-v2</a></li><li><a href="/articles/miragevpn-ncp.html">MirageVPN updated (AEAD, NCP)</a></li><li><a href="/articles/speeding-ec-string.html">Speeding elliptic curve cryptography</a></li><li><a href="/articles/miragevpn-performance.html">Speeding up MirageVPN and use it in the wild</a></li><li><a href="/articles/miragevpn-server.html">MirageVPN server</a></li><li><a href="/articles/2024-08-21-OpenVPN-and-MirageVPN.html">MirageVPN and OpenVPN</a></li></ul>
</div><div class="tag-box" id="tag-tar">
<h3>
<span>tar</span>
</h3>
<ul><li><a href="/articles/gptar.html">GPTar</a></li></ul>
</div><div class="tag-box" id="tag-unicode">
<h3>
<span>unicode</span>
</h3>
<ul><li><a href="/articles/2024-02-03-python-str-repr.html">Python&apos;s `str.__repr__()`</a></li></ul>
</div><div class="tag-box" id="tag-unikernel">
<h3>
<span>unikernel</span>
</h3>
<ul><li><a href="/articles/qubes-miragevpn.html">qubes-miragevpn, a MirageVPN client for QubesOS</a></li></ul>
</div><div class="tag-box" id="tag-vpn">
<h3>
<span>vpn</span>
</h3>
<ul><li><a href="/articles/qubes-miragevpn.html">qubes-miragevpn, a MirageVPN client for QubesOS</a></li></ul>
</div>
</main>
<footer>
<a href="https://github.com/xhtmlboi/yocaml">Powered by <strong>YOCaml</strong></a>
<br />
</footer>
<script>hljs.highlightAll();</script>
</body>
</html>