hannes/hannes.robur.coop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

updated from main (commit 1ecc61c2d0)

Browse source
This commit is contained in:
Canopy bot 2024-05-16 14:19:19 +00:00
parent 65bc7e6c40
commit 6b114f48ad
2 changed files with 4 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
Posts/Retreat2024
View file

@ -10,7 +10,7 @@
<p>Also related to the deployment discussion, I re-started work from last year's retreat on a Web UI for <a href="/Posts/Albatross">Albatross</a>. Welcome <a href="https://github.com/robur-coop/mollymawk">mollymawk</a>. The idea is to have both a UI but also a HTTP API for deploying MirageOS unikernels via e.g. GitHub actions by a simple HTTP POST request. Observation and management of unikernels is also part of mollymawk. <a href="https://github.com/PizieDust">Pixie</a> worked a lot on the JavaScript and web side of it, while I put some effort into the MirageOS and Albatross interaction. Within days we got an initial prototype up and running and could show it to others. This already is able to gather information of running unikernels, the console output, and create and destroy unikernels. We will continue on mollymawk in the future.</p>
<p>From the design point, mollymawk communicates via TLS to albatross. It contains the TLS certificate and private key to access albatross (and generates temporary certificates for all the actions required). This also means that mollymawk needs to protect its secrets, otherwise anyone can mess around with albatross. Mollymawk itself is a unikernel that runs on top of albatross. What could possibly go wrong? We plan to add authentication soon to mollymawk, and can then use it for our own purposes. This will greatly improve the state of our running unikernels, since updating our unikernels from our reproducible build infrastructure will then be a click on a button, with a revert button if there are issues.</p>
<h2 id="defunctorising-mirageos">Defunctorising MirageOS</h2>
<p>Other discussions were about porting existing OCaml code to MirageOS, and what are the hurdles. The excessive use of functors are usually cumbersome for existing projects that need to revise their internal code structure. I have some issues with the use of functors as well, especially at places where they don't seem to be necessary. A rule of thumb I learned is that a functor is great if you want to have something in the same application instantiated with different modules -- i.e. a set of string and a set of integers. Now, in MirageOS we use even for time (providing the function <code>sleep</code>), clocks (monotonic and posix time), random (random data), network interface functors. The reason is that in earlier days (before mirage 4.0.0) our compilation strategy used installed opam packages. Since mirage 4, a monorepo is used to compile the unikernel.</p>
<p>Other discussions were about porting existing OCaml code to MirageOS, and what are the hurdles. The excessive use of functors are usually cumbersome for existing projects that need to revise their internal code structure. I have some issues with the use of functors as well, especially at places where they don't seem to be necessary. A rule of thumb I learned is that a functor is great if you want to have something in the same application instantiated with different modules -- i.e. a set of string and a set of integers. Now, in MirageOS we use even for time (providing the function <code>sleep</code>), clocks (monotonic and posix time), random (random data), network interface functors. There's some history about it: mirage before 4.0 used opam packages for &quot;cross-compilation&quot;. Since mirage 4, a monorepo is used to compile the unikernel. This allows to reconsider the options we have.</p>
<p>Talking about time - I don't think anyone wants to have two different sleep functions in their application, similar for monotonic clock and posix time. Also the existing open source unikernels only use a single time and clock. The random interface has as well some history connected, since our cryptographic library used to bundle everything in a single opam package, and required arbitrary precision integers (via zarith using gmp), and we didn't want to impose a dependency on gmp for every unikernel. Nowadays, the CSPRNG code is separated from gmp, and we can just use it. The network interface: indeed if you're using solo5 (hvt or virtio or whatnot), you will never need a unix (tap-based) network implementation, neither xen netfront/netback. Thus, the functorisation is not necessary.</p>
<p>At the retreat we discussed and thought about replacing the functors in MirageOS with other mechanisms - conditional compilation (depending on the target) or the linking trick (dune variants) or a ppx are candidates, apart from other things (maybe first-class modules fit the requirement as well).</p>
<p>Another benefit of conditional compilation and dune variants is editor support -- jumping to the definition of time (clock, etc.) now simply works because the compiler knows which code is used at runtime (in contrast to when a functor is used).</p>

6
atom
View file

@ -1,4 +1,4 @@
<feed xmlns="http://www.w3.org/2005/Atom"><link href="https://hannes.robur.coop/atom" rel="self"/><id>urn:uuid:981361ca-e71d-4997-a52c-baeee78e4156</id><title type="text">full stack engineer</title><updated>2024-05-16T09:46:39-00:00</updated><entry><summary type="html">&lt;p&gt;My involvement and experience of the MirageOS retreat 2024&lt;/p&gt;
<feed xmlns="http://www.w3.org/2005/Atom"><link href="https://hannes.robur.coop/atom" rel="self"/><id>urn:uuid:981361ca-e71d-4997-a52c-baeee78e4156</id><title type="text">full stack engineer</title><updated>2024-05-16T14:19:12-00:00</updated><entry><summary type="html">&lt;p&gt;My involvement and experience of the MirageOS retreat 2024&lt;/p&gt;
</summary><published>2024-05-15T14:38:51-00:00</published><link href="/Posts/Retreat2024" rel="alternate"/><content type="html">&lt;p&gt;End of April I spent a week in Marrakech to work on MirageOS at our annual &lt;a href=&quot;https://retreat.mirage.io&quot;&gt;retreat&lt;/a&gt;. This was the 12th time when it actually took place, and it was amazing. We were a total of 17 people attending, of which 5 people used ground transportation. I have respect for them, to take the 3 or more day trip from Berlin or Glasgow or Paris or whereever to Marrakech - with a ferry in between (which availability depends on the wind etc.). This time, I didn't take ground transportation since I had appointments just before and just after the retreat nearby Berlin where I had to be in person. The food an weather was really nice and motivating.&lt;/p&gt;
&lt;p&gt;Of the 17 people there were lots of new faces. It is really nice to see an active community which is welcoming to new people, and also that there's change: a lot of fresh ideas are brought by these people and we rethink what we are doing and why.&lt;/p&gt;
&lt;p&gt;Our daily routine included breakfast, circle, lunch, dinner, (sometimes) presentations. There wasn't more formal structure, and there was no need for it since people kept themselves busy working on various projects. We had a bigger discussion about &amp;quot;deploying MirageOS&amp;quot;, which is a reoccurring theme. Even running a MirageOS unikernel on a laptop is for people who are not very familiar with networking pretty hard. In addition, our &lt;a href=&quot;https://mirage.io&quot;&gt;website&lt;/a&gt; only briefly described network setup, and included DHCP very early (which is overwhelming to setup and use on your laptop).&lt;/p&gt;
@ -10,7 +10,7 @@
&lt;p&gt;Also related to the deployment discussion, I re-started work from last year's retreat on a Web UI for &lt;a href=&quot;/Posts/Albatross&quot;&gt;Albatross&lt;/a&gt;. Welcome &lt;a href=&quot;https://github.com/robur-coop/mollymawk&quot;&gt;mollymawk&lt;/a&gt;. The idea is to have both a UI but also a HTTP API for deploying MirageOS unikernels via e.g. GitHub actions by a simple HTTP POST request. Observation and management of unikernels is also part of mollymawk. &lt;a href=&quot;https://github.com/PizieDust&quot;&gt;Pixie&lt;/a&gt; worked a lot on the JavaScript and web side of it, while I put some effort into the MirageOS and Albatross interaction. Within days we got an initial prototype up and running and could show it to others. This already is able to gather information of running unikernels, the console output, and create and destroy unikernels. We will continue on mollymawk in the future.&lt;/p&gt;
&lt;p&gt;From the design point, mollymawk communicates via TLS to albatross. It contains the TLS certificate and private key to access albatross (and generates temporary certificates for all the actions required). This also means that mollymawk needs to protect its secrets, otherwise anyone can mess around with albatross. Mollymawk itself is a unikernel that runs on top of albatross. What could possibly go wrong? We plan to add authentication soon to mollymawk, and can then use it for our own purposes. This will greatly improve the state of our running unikernels, since updating our unikernels from our reproducible build infrastructure will then be a click on a button, with a revert button if there are issues.&lt;/p&gt;
&lt;h2 id=&quot;defunctorising-mirageos&quot;&gt;Defunctorising MirageOS&lt;/h2&gt;
&lt;p&gt;Other discussions were about porting existing OCaml code to MirageOS, and what are the hurdles. The excessive use of functors are usually cumbersome for existing projects that need to revise their internal code structure. I have some issues with the use of functors as well, especially at places where they don't seem to be necessary. A rule of thumb I learned is that a functor is great if you want to have something in the same application instantiated with different modules -- i.e. a set of string and a set of integers. Now, in MirageOS we use even for time (providing the function &lt;code&gt;sleep&lt;/code&gt;), clocks (monotonic and posix time), random (random data), network interface functors. The reason is that in earlier days (before mirage 4.0.0) our compilation strategy used installed opam packages. Since mirage 4, a monorepo is used to compile the unikernel.&lt;/p&gt;
&lt;p&gt;Other discussions were about porting existing OCaml code to MirageOS, and what are the hurdles. The excessive use of functors are usually cumbersome for existing projects that need to revise their internal code structure. I have some issues with the use of functors as well, especially at places where they don't seem to be necessary. A rule of thumb I learned is that a functor is great if you want to have something in the same application instantiated with different modules -- i.e. a set of string and a set of integers. Now, in MirageOS we use even for time (providing the function &lt;code&gt;sleep&lt;/code&gt;), clocks (monotonic and posix time), random (random data), network interface functors. There's some history about it: mirage before 4.0 used opam packages for &amp;quot;cross-compilation&amp;quot;. Since mirage 4, a monorepo is used to compile the unikernel. This allows to reconsider the options we have.&lt;/p&gt;
&lt;p&gt;Talking about time - I don't think anyone wants to have two different sleep functions in their application, similar for monotonic clock and posix time. Also the existing open source unikernels only use a single time and clock. The random interface has as well some history connected, since our cryptographic library used to bundle everything in a single opam package, and required arbitrary precision integers (via zarith using gmp), and we didn't want to impose a dependency on gmp for every unikernel. Nowadays, the CSPRNG code is separated from gmp, and we can just use it. The network interface: indeed if you're using solo5 (hvt or virtio or whatnot), you will never need a unix (tap-based) network implementation, neither xen netfront/netback. Thus, the functorisation is not necessary.&lt;/p&gt;
&lt;p&gt;At the retreat we discussed and thought about replacing the functors in MirageOS with other mechanisms - conditional compilation (depending on the target) or the linking trick (dune variants) or a ppx are candidates, apart from other things (maybe first-class modules fit the requirement as well).&lt;/p&gt;
&lt;p&gt;Another benefit of conditional compilation and dune variants is editor support -- jumping to the definition of time (clock, etc.) now simply works because the compiler knows which code is used at runtime (in contrast to when a functor is used).&lt;/p&gt;
@ -21,7 +21,7 @@
&lt;p&gt;I'm looking forward to other writeups and future retreats, or other events where I can say hi to attendees.&lt;/p&gt;
&lt;p&gt;We at &lt;a href=&quot;https://robur.coop&quot;&gt;Robur&lt;/a&gt; are working as a collective since 2018 on public funding, commercial contracts, and donations. Our mission is to get sustainable, robust, and secure MirageOS unikernels developed and deployed. Running your own digital communication infrastructure should be easy, including trustworthy binaries and smooth upgrades. You can help us continue our work by &lt;a href=&quot;https://aenderwerk.de/donate/&quot;&gt;donating&lt;/a&gt; (select Robur from the drop-down or put &amp;quot;donation Robur&amp;quot; in the purpose of the bank transfer).&lt;/p&gt;
&lt;p&gt;If you have any questions, reach us best via eMail to team AT robur DOT coop.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/community" term="community"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:2c9e2634-78c5-5957-b6f2-d0706f80d7eb</id><title type="text">The MirageOS retreat 2024</title><updated>2024-05-16T09:46:39-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;Core Internet protocols require operational experiments, even if formally specified&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/community" term="community"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:2c9e2634-78c5-5957-b6f2-d0706f80d7eb</id><title type="text">The MirageOS retreat 2024</title><updated>2024-05-16T14:19:12-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;Core Internet protocols require operational experiments, even if formally specified&lt;/p&gt;
</summary><published>2023-11-28T21:17:01-00:00</published><link href="/Posts/TCP-ns" rel="alternate"/><content type="html">&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Transmission_Control_Protocol&quot;&gt;Transmission Control Protocol (TCP)&lt;/a&gt; is one of the main Internet protocols. Usually spoken on top of the Internet Protocol (legacy version 4 or version 6), it provides a reliable, ordered, and error-checked stream of octets. When an application uses TCP, they get these properties for free (in contrast to UDP).&lt;/p&gt;
&lt;p&gt;As common for Internet protocols, TCP is specified in a series of so-called requests for comments (RFC). The latest revised version from August 2022 is &lt;a href=&quot;https://datatracker.ietf.org/doc/html/rfc9293&quot;&gt;RFC 9293&lt;/a&gt;; the initial one was &lt;a href=&quot;https://datatracker.ietf.org/doc/html/rfc793&quot;&gt;RFC 793&lt;/a&gt; from September 1981.&lt;/p&gt;
&lt;h1 id=&quot;my-brief-personal-tcp-story&quot;&gt;My brief personal TCP story&lt;/h1&gt;