robur/blog.robur.coop
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add text about different approaches to measuring

Browse source
This commit is contained in:
Reynir Björnsson 2024-04-16 14:10:10 +02:00
parent 933644ce23
commit 8b9aa6b18f
1 changed files with 3 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
articles/miragevpn-performance.md
View file

@ -36,7 +36,9 @@ The learnings of our performance engineering are in three areas:
- Rethink allocations: fundamentally, only a single big buffer (to be send out) for each incoming packet should be allocated, not a series of buffers that are concatenated (see [this PR](https://github.com/robur-coop/miragevpn/pull/217) and [that PR](https://github.com/robur-coop/miragevpn/pull/219)). Additionally, not zeroing out the just allocated buffer (if it is filled with data anyways) removes some further instructions (see [this PR](https://github.com/robur-coop/miragevpn/pull/218)). And we figured that appending to an empty buffer nevertheless allocated and copied in OCaml, so we worked on [this PR](https://github.com/robur-coop/miragevpn/pull/214). - Rethink allocations: fundamentally, only a single big buffer (to be send out) for each incoming packet should be allocated, not a series of buffers that are concatenated (see [this PR](https://github.com/robur-coop/miragevpn/pull/217) and [that PR](https://github.com/robur-coop/miragevpn/pull/219)). Additionally, not zeroing out the just allocated buffer (if it is filled with data anyways) removes some further instructions (see [this PR](https://github.com/robur-coop/miragevpn/pull/218)). And we figured that appending to an empty buffer nevertheless allocated and copied in OCaml, so we worked on [this PR](https://github.com/robur-coop/miragevpn/pull/214).
- Still an open topic is: we are in the memory-safe language OCaml, and we sometimes extract data out of a buffer (or set data in a buffer). Now, each operation lead to bounds checks (that we do not touch memory that is not allocated or not ours). However, if we just checked for the buffer being long enough (either by checking the length, or by allocating a specific amount of data), these bounds checks are superfluous. So far, we don't have an automated solution for this issue, but we are [discussing it in the OCaml community](https://discuss.ocaml.org/t/bounds-checks-for-string-and-bytes-when-retrieving-or-setting-subparts-thereof/), and are eager to find a solution to avoid unneeded computations. - Still an open topic is: we are in the memory-safe language OCaml, and we sometimes extract data out of a buffer (or set data in a buffer). Now, each operation lead to bounds checks (that we do not touch memory that is not allocated or not ours). However, if we just checked for the buffer being long enough (either by checking the length, or by allocating a specific amount of data), these bounds checks are superfluous. So far, we don't have an automated solution for this issue, but we are [discussing it in the OCaml community](https://discuss.ocaml.org/t/bounds-checks-for-string-and-bytes-when-retrieving-or-setting-subparts-thereof/), and are eager to find a solution to avoid unneeded computations.
To guide the performance engineering, we also developed [a microbenchmark](https://github.com/robur-coop/miragevpn/pull/230) using OCaml tooling. This will setup a client and server without any input and output, and transfer data. As a first approach we connected with the MirageVPN unix client & OpenVPN client to a eduVPN server and ran speed tests using fast.com. This was sufficient to show the initial huge gap in download speeds between MirageVPN and OpenVPN. There is *a lot* of noise in this approach as there are many computers and routers involved in this setup, and it is hard to reproduce.
To get more reproducible results we set up a local VM with openvpn and iperf3 installed. On the host machine we can then connect to the VM's OpenVPN server and run iperf3 against the *VPN* ip address. This worked more reliably. However, it was still noisy and not suitable to measure single digit percentage changes in performance.
To better guide the performance engineering, we also developed [a microbenchmark](https://github.com/robur-coop/miragevpn/pull/230) using OCaml tooling. This will setup a client and server without any input and output, and transfer data in memory.
To conclude: we already achieved a factor of 25 in performance by adapting the code in various ways. We have ideas to improve the performance even more in the future - we also work on using OCaml string and bytes, instead of off-the-OCaml-heap-allocated bigarrays (see [our previous article](https://blog.robur.coop/articles/speeding-ec-string.html), which provided some speedups). To conclude: we already achieved a factor of 25 in performance by adapting the code in various ways. We have ideas to improve the performance even more in the future - we also work on using OCaml string and bytes, instead of off-the-OCaml-heap-allocated bigarrays (see [our previous article](https://blog.robur.coop/articles/speeding-ec-string.html), which provided some speedups).