robur/builder-web
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
537 commits 8 branches 2 tags 6.6 MiB
Commit graph

263 commits

Author SHA1 Message Date
rand00
126fe38465 Views: Configured .ocp-indent.conf to sensible defaults, and indented everything with this. Settings are: 
normal
base = 2
type = 2
in = 0
with = 0
match_clause = 2
ppx_stritem_ext = 2
max_indent = 2
strict_with = never
strict_else = always
strict_comments = false
align_ops = true
align_params = auto
 2022-02-03 14:00:24 +01:00
rand00
87442c4a09 Views: Builds: Separated html-generation out into named functions for ease of reading 2022-02-03 13:50:37 +01:00
rand00
b631b05de2 Views: Syntax 2022-02-03 13:28:16 +01:00
rand00
db3f87934b Views++: Made long parameter-lists into named parameters 2022-02-03 13:27:22 +01:00
rand00
3680336b22 Views: Removed outcommented + fixed some 80-col problems 2022-02-03 12:57:21 +01:00
rand00
f8b17e6b17 Views: Removed global open of Tyxml.Html and switched to prefix - 
.. there are too many common names in this modules scope, and code gets easier to understand
 .. also I don't think global open is a good idea in general for this kind of module, so better
    to fix this style now than be sorry later
 2022-02-03 12:54:02 +01:00
rand00
ae5c5cb67d Views: Using ocp-indent to indent everything consistently + fixed code aesthetics (80 char rule etc.) 2022-02-02 23:27:25 +01:00
rand00
c17802d84f Views++: Factored Job.Build to Job_build + Renamed Builder to Builds to avoid collision with lib 2022-02-02 23:03:16 +01:00
rand00
7358567e55 Utils: Moved omd-helpers into Omd submodule here 2022-02-02 22:59:32 +01:00
rand
f3aa2a2c90 Safer Lwt_process usage for calling builder-viz cmd (#71) 
Co-authored-by: Reynir Björnsson <reynir@reynir.dk>
Co-authored-by: rand00 <oth.rand@gmail.com>
Reviewed-on: https://git.robur.io/robur/builder-web/pulls/71
Co-authored-by: rand <rand@r7p5.earth>
Co-committed-by: rand <rand@r7p5.earth>
 2022-02-02 14:50:44 +00:00
Reynir Björnsson
6a645b7358 Merge branch 'tar' 2022-02-01 12:21:56 +00:00
Reynir Björnsson
3fe07c7b34 Clean up trailing spaces 2022-02-01 11:43:07 +00:00
rand00
d6c04e861e Views: Removed usage of containers 2022-02-01 11:30:44 +00:00
rand00
161fec77af Views: Using Option.of_list 2022-02-01 11:30:44 +00:00
rand00
b564191b81 Re-enabled warnings as errors in dune file + fixed unused 'id' in builder_web.ml 2022-02-01 11:30:44 +00:00
rand00
8897f525fe Builder_web: visualization_cmd: Changed bin to be 'builder-viz' + let it be searched in PATH 2022-02-01 11:30:44 +00:00
rand00
5a9c1237a1 Builder_web: Added timeout to visualization_cmd Lwt_process.pread 2022-02-01 11:30:44 +00:00
rand00
2a1b75ba2b Removals of uncommented code and unneccesary parens 2022-02-01 11:30:44 +00:00
rand00
6f95f7e965 Views: Switch viz order + fixed iframe size 2022-02-01 11:30:44 +00:00
rand00
86b5bf870d Builder_web.treemap_visualization_cmd: More informative param-name 2022-02-01 11:30:44 +00:00
rand00
1bba2d7855 Views: Adjusted height of deps viz iframe again 2022-02-01 11:30:44 +00:00
rand00
c80ee590bd New centered flex-layout for builds page with 2 columns + some refactorings around Views 2022-02-01 11:30:44 +00:00
rand00
915468bbf1 Views: Adjusted height of deps viz iframe 2022-02-01 11:30:44 +00:00
rand00
48ba9b7bfe Views: Also rendering dependencies for non-binary packages 2022-02-01 11:30:44 +00:00
rand00
64cf4fc350 Added dependencies visualization to builds pages 2022-02-01 11:30:44 +00:00
rand00
dde9d5b2da Builder_web.job_build_treemap: Calculating treemap visualization via CLI call to new binary 
.. this doesn't block server on requests, as Lwt IO is run in parallel
 2022-02-01 11:30:44 +00:00
rand00
afbf9357b0 lib/dune: Added opam-graph 2022-02-01 11:30:44 +00:00
rand00
6658244a18 Views: Only rendering treemap for unikernels with '.debug' artifact 2022-02-01 11:30:44 +00:00
rand00
ade1ea3a38 Builder_web.job_build_treemap: Changed title of excluded treemap chunks 2022-02-01 11:30:44 +00:00
rand00
c6ff42d391 Views: Fix fize of treemap 2022-02-01 11:30:44 +00:00
rand00
a0254b3e70 Builder_web.job_build_treemap: Changed title of new excluded treemap chunks 2022-02-01 11:30:44 +00:00
rand00
82c5614440 Builder_web.job_build_treemap: Calculating extra chunks inserted in treemap-scale 2022-02-01 11:30:44 +00:00
rand00
fdd00a17ab Builder_web.job_build_treemap: Changed minimal size of treemap node based on example treemap 2022-02-01 11:30:44 +00:00
rand00
b8c40861f3 Builder_web.job_build_treemap: Implementing partitioning of Info tree 2022-02-01 11:30:44 +00:00
rand00
79c40473b4 Views: Changed background color 2022-02-01 11:30:44 +00:00
rand00
6a70220dee Removed debug-printing + Adjusted size of treemap iframe 2022-02-01 11:30:44 +00:00
Robur
07b5daff9f Fix to the code extracting binary size for treemap visualization 2022-02-01 11:30:44 +00:00
rand00
d247846e35 WIP: debugging 2022-02-01 11:30:44 +00:00
rand00
462bbf5942 Treemap scale: Passing on binary-size to new Treemap renderer 2022-02-01 11:30:44 +00:00
rand00
c9ab07832e Views: Removed old code 2022-02-01 11:30:44 +00:00
rand00
4e2d069b26 Views: Chosen a static width relative to textsize for treemap 2022-02-01 11:30:44 +00:00
rand00
b52e3bc0b0 Views: Rewrote static css to be one big string instead 2022-02-01 11:30:44 +00:00
rand00
5548c04a3e Several changes related to treemap rendering: 
* Rendering treemap in build-page as iframe, as the svg needs static CSS classes to work; and static CSS doesn't compose
  * Added 'include_static_css' param to Views.layout (useful for later)
  * Added Builder_web.dream_svg helper if we want to serve a svg directly
 2022-02-01 11:30:44 +00:00
Robur
7fa8402eee use modulectomy to render svg of the unikernel binaries 2022-02-01 11:30:44 +00:00
Reynir Björnsson
92a43fbfdd Add .../all.tar endpoint with artifacts 
One stop shop for all the build artifacts!
 2022-01-25 20:01:08 +00:00
Robur
8489d1ff36 Remove README from build page 2021-12-02 14:49:45 +00:00
Robur
19633e84ea Reuse DB connection for consequtive queries 2021-12-02 13:58:44 +00:00
Robur
0afec1617b markdown sanitization, addresses issue #46 2021-12-02 11:27:00 +00:00
Reynir Björnsson
68237ef382 Rewrite queries to take optional platform 
A number of queries were the same except one would take another string
argument and add `AND platform = $N` in its WHERE clause. This commit
merges them and does `AND ($N IS NULL OR platform = $N)` and the client
code in Model doesn't have to check the string option.
 2021-11-18 11:56:17 +01:00
Robur
088b55acc3 remove failed builds from job page 
add a job page with failed builds, link to it
 2021-11-17 16:39:49 +00:00
Robur
2e82778e87 Only show successful builds on the front page 
Failed builds can be viewed on the linked /failed-builds/ page.
 2021-11-17 16:00:58 +00:00
Robur
e1d950ad5b link to failed-builds, paginate failed-builds 2021-11-17 15:54:13 +00:00
Robur
0910a05bbd builds: improve comparisons (text and query) to earlier and later builds 
only take into consideration the builds that have a different output
 2021-11-17 15:28:15 +00:00
Robur
e5168e1b4f Display timestamps without time zone offset, use Z instead to indicate time zone offset 0 2021-11-17 14:44:41 +00:00
Robur
740e76b73e remove comment about missing failed-builds page 2021-11-17 14:37:25 +00:00
Robur
c6128ca24b /failed-builds/: fix query and improve page 2021-11-17 14:33:05 +00:00
Robur
8ee69d7211 Work around caqti assertion failure 2021-11-17 14:06:18 +00:00
Robur
c566cd0215 Work towards failed-builds page 2021-11-17 14:02:04 +00:00
Robur
e15bd00fe5 improve comparison page and always link with current build as head (right_build) 2021-11-12 13:01:20 +00:00
Reynir Björnsson
1dd1fe54ba Add breadcrumb navigation 
Fixes #59.
 2021-11-12 12:37:25 +00:00
Robur
16748b8995 Show builds by platform 2021-11-08 15:03:19 +00:00
Robur
594c6d5917 remove unused queries 2021-11-05 12:49:16 +00:00
Robur
c76cead3f7 view: show platform on build page 2021-11-05 11:02:30 +00:00
Robur
e69d1beb9f Add platform to build table 2021-11-05 10:45:26 +00:00
Reynir Björnsson
a9ff2dd033 Remove rresult 2021-11-05 10:10:44 +00:00
Robur
045dbcf23d adapt to builder 0.2.0 changes 2021-11-03 14:40:04 +00:00
Reynir Björnsson
9dc3fe7abe upload_binary: add binary_name query parameter 
This allows for binary upload with a binary name other than <job>.bin,
making bootstrapping easier.
 2021-10-26 13:24:17 +02:00
Robur
87a6b95e8c Model.read_file: avoid file descriptor leak 2021-10-18 13:45:53 +00:00
Alain Armand
fa1cf92702 avoid deprecated functions from Fmt (strf is now str) 2021-10-18 13:22:07 +00:00
Reynir Björnsson
71a016fdae Fix up mix up 
When builds were uploaded the script and console was mixed up. This
commit fixes that and includes a fixup migration script.
 2021-09-10 13:00:04 +02:00
Reynir Björnsson
f24a9297d0 Re-add viewing build script and build console 
Instead of displaying the script and console output in the build page
the build page links to new endpoints for viewing the script and the
console output.
 2021-09-09 18:06:08 +02:00
Robur
edcbf73386 Add /job/<job>/build/<build>/main-binary redirect endpoint 
Fixes #52
 2021-09-08 09:10:30 +00:00
Robur
8279bc1c26 store relative path for console and script 2021-09-07 11:53:51 +00:00
Robur
f7823da321 fix console output order 2021-09-07 10:04:11 +00:00
Robur
7c04469825 Prepare staging dir before writing 
The preparation of the staging dir when saving a build was handled in
Model.save_all, but we expect it to be created in Model.save_console_and_script.

This commit refactors the staging dir preparation into a function for better
clarity.
 2021-09-07 09:35:26 +00:00
Robur
e7daf0366b Rename local variable meta(s) -> build(s) 2021-09-07 09:35:26 +00:00
Robur
17420c389b WIP 2021-09-07 09:35:26 +00:00
Hannes Mehnert
4c42865ca7 fix compilation 2021-08-03 15:12:53 +02:00
Hannes Mehnert
cd633087d5 Infer section from job name rather than extension 2021-08-03 12:28:18 +00:00
Robur
392286dd98 remove README.md and build-hashes from build_artifact table and file system 
no longer store full file on disk
 2021-07-12 14:10:54 +00:00
Robur
aa4db9b6a8 in /job/:job/build/:build output links to: 
- builds that reproduced the binary with the same inputs
- builds that reproduced the binary with different inputs (only one for each input_id)
- builds with same input that produced a different output

/job/:job group by hash
 2021-07-12 13:31:35 +00:00
Robur
6ec40365ab Add input_id to build table 
The input_id column is a checksum of the files relevant for reproducibility.
 2021-07-06 13:54:25 +00:00
Robur
be26e56fd4 Output links to comparisons of other builds with the identical main binary 
in /job/:job/build/:build
 2021-07-06 10:23:29 +00:00
hannes
9c326679ba investigate differences in build, install, and uri when opam file differed (#48) 
compare environment and system packages as well

investigate differences in build, install, and uri when opam file differed

Co-authored-by: Robur <team@robur.coop>
Reviewed-on: https://git.robur.io/robur/builder-web/pulls/48
Co-Authored-By: hannes <hannes@mehnert.org>
Co-Committed-By: hannes <hannes@mehnert.org>
 2021-07-06 08:34:17 +00:00
Reynir Björnsson
7c7282894b Typed database IDs (#47) 
Typed database IDs

Reviewed-on: https://git.robur.io/robur/builder-web/pulls/47
Co-Authored-By: Reynir Björnsson <reynir@reynir.dk>
Co-Committed-By: Reynir Björnsson <reynir@reynir.dk>
 2021-07-05 12:45:08 +00:00
Robur
1e3fcf984f Allow README.md being present anywhere 
The one in root takes precedence, but opam-installer doesn't really
want to install things into %{prefix}%.
 2021-06-30 14:10:02 +00:00
Robur
009fa49e9e render README from latest build (preserved as a tag value in job_tag table) 2021-06-30 12:47:30 +00:00
Robur
5285872865 minor changes to main site 2021-06-30 11:38:02 +00:00
Robur
b4996939af Job_tag: try to retrieve the tag value - insert if none, update if some 2021-06-30 11:09:01 +00:00
Robur
eb786088e7 Builder_db.Job_tag.add: use INSERT OR REPLACE to avoid constraint violation 2021-06-30 09:56:47 +00:00
Robur
9a271add7b add tag to jobs: description 2021-06-29 15:15:16 +00:00
Robur
e45497e97c add tags to jobs: section and synopsis, inferred from the latest successful build 2021-06-29 14:59:08 +00:00
Robur
88377adb7c adapt to builder changes (job -> script_job) 2021-06-25 16:43:47 +00:00
Robur
b279eb521b Adapt to build input files removal 
* Do not synthesize empty input files list
* Drop table build_file
 2021-06-25 10:26:03 +00:00
Robur
5555c87afd Add the /job/:job/upload endpoint which receives a raw binary. 
Fixes #38
 2021-06-09 14:26:05 +00:00
Robur
901c24ab27 if_error: if not found, present a not found message 2021-06-09 14:25:00 +00:00
Robur
31a501e0c4 Add user column to build 
We want to track who uploaded what builds.
A `nologin` user is created in the migration that owns all preexisting builds.
 2021-06-09 11:54:24 +00:00
Robur
02bfbc956f Improve HTTP status codes (4xx on user data failure) 
Fixes #43
 2021-06-09 09:48:51 +00:00
Robur
eaf8a609c9 Add an access_list to grant per-job user upload permissions 
Fixes #36
 2021-06-08 14:54:23 +00:00
Reynir Björnsson
8f4a45bf76 Revise error handling 
- also validate UUID to be 36 characters
- also error with not_found for unknown jobs
 2021-06-08 10:26:57 +00:00
Reynir Björnsson
e4407902f5 Do not read file if not modified 2021-06-05 08:18:39 +02:00
Reynir Björnsson
847e0cffe4 Check if-not-modified 
We are still reading the whole file from disk even when it's not needed.
 2021-06-04 13:01:28 +02:00
Robur
a45a584831 Generate opam compare links only for successful builds 2021-06-02 13:05:10 +00:00
Robur
fb2515e713 Add /build/latest/ redirect 2021-06-02 12:23:40 +00:00
Robur
4fec5ed38c Builder_db: use foreign key for build.main_binary 2021-06-02 10:29:08 +00:00
Robur
49003ca21f Store relative paths for build artifacts in database 2021-06-01 15:43:55 +00:00
Robur
10351c65bd Migrate to dream 2021-06-01 14:06:36 +00:00
Reynir Björnsson
bc7a2006e3 More precise error type information 2021-06-01 10:33:35 +02:00
Reynir Björnsson
4f17b8b8a6 Link to opam diff 
Compare with latest build as well as previous build.
 2021-04-23 15:00:12 +02:00
Reynir Björnsson
9974510787 Display builds with start time 2021-04-23 09:46:23 +02:00
Reynir Björnsson
5f1f106c91 Link back to builds when comparing 2021-04-23 09:46:23 +02:00
Reynir Björnsson
c51399823c Packages w/ same version different opam 2021-04-23 09:45:35 +02:00
Reynir Björnsson
ac43298620 Fix anchor fragment links 2021-04-23 09:45:35 +02:00
Reynir Björnsson
031a2042b0 opam-switch diff 2021-04-23 09:45:35 +02:00
Reynir Björnsson
265e00a718 Do DB rollback on failed build insert 2021-03-25 10:42:04 +01:00
Reynir Björnsson
7b5119189d Filesystem transactions and cleanup 2021-03-24 14:49:23 +01:00
Reynir Björnsson
a7be8f8f47 Be consistent with order of sha256 and size 2021-02-26 10:05:53 +01:00
Reynir Björnsson
71cc34bac3 Use int instead of int64 for file sizes 
We are not going to use the extra bit, and String.length returns the
size as an int anyway.
 2021-02-25 15:27:45 +01:00
Reynir Björnsson
3046994f4c Add filesize to view 2021-02-25 15:18:43 +01:00
Reynir Björnsson
535d2ac0b9 Add file sizes 2021-02-24 11:31:31 +01:00
reynir
7b81d78554 Use scrypt (#32) 
Switch to using scrypt for password hashing

Co-authored-by: Reynir Björnsson <reynir@reynir.dk>
Reviewed-on: https://git.robur.io/robur/builder-web/pulls/32
Co-Authored-By: reynir <reynir@reynir.dk>
Co-Committed-By: reynir <reynir@reynir.dk>
 2021-02-23 15:20:18 +00:00
Reynir Björnsson
8d211dc831 Check database version 
Exit if the application id and user version are not as expected.
 2021-02-17 14:15:57 +01:00
Reynir Björnsson
1c5de383ea /hash takes query parameters, add hash search form 2021-02-02 17:23:57 +01:00
Reynir Björnsson
6b96cae318 Add /hash/:hash endpoint 
It looks up the given hash and finds the latest build producing an
artifact with this hash.
 2021-02-02 09:56:17 +01:00
Reynir Björnsson
3a106342f5 Show latest build info in jobs view 
This does N+1 queries where N is the number of jobs.
 2021-01-29 14:34:56 +01:00
Reynir Björnsson
53addf4825 Fix bug in /upload 
The localpath was used instead of the filepath which broke the
reference.
 2021-01-29 10:17:03 +01:00
Reynir Björnsson
96a3da36cd Show build main binary and hash in job view 2021-01-29 10:17:03 +01:00
Reynir Björnsson
56737ec71b Migration code, model aware of main binary 
Sqlite3 application_id and user_version are now set to identify the
database is a builder-web database, and the user_version represents the
schema version.

The 'build' table is extended with a 'main_binary' column. This
represents the main binary artifact from the build. This is decided by
there being exactly one file in bin/.

A migration tool is written that does both migrations and rollbacks, and
migration and rollback is implemented for the above mentioned change.
 2021-01-29 10:15:31 +01:00
Reynir Björnsson
a936915b74 Also add the new function to the interface... 2021-01-28 16:05:32 +00:00
Reynir Björnsson
041faadb54 Return 409 conflic if build uuid already exists 2021-01-28 16:05:32 +00:00
Reynir Björnsson
7a8f853663 Remove /refresh endpoint 
It depends on builder-db add and was a hack anywya.
 2021-01-27 20:10:27 +00:00
Reynir Björnsson
484658d23a Fix bugs 
Parent directories for input file and output files must be created.
 2021-01-21 18:12:26 +01:00
Reynir Björnsson
0f98541feb Store files on disk 2021-01-21 17:51:58 +01:00
Reynir Björnsson
22ef6395b8 Whitespace, log output 2021-01-21 12:02:07 +01:00
Reynir Björnsson
a6fe9ada45 Add password_iter option 2021-01-21 12:01:47 +01:00
Reynir Björnsson
01babd0d0d User authentication and upload endpoint 2021-01-21 10:11:37 +01:00
Reynir Björnsson
035d471844 Add refresh endpoint hack 2021-01-19 10:44:32 +01:00
Reynir Björnsson
5298108d1a Implement sqlite database backed builder-web 2021-01-19 10:44:32 +01:00
Reynir Björnsson
0309737ef7 Add orb mime knowledge 2021-01-06 14:28:10 +01:00
Reynir Björnsson
37a916517d Display SHA256 digest only, and in hex 2021-01-06 11:48:24 +01:00
Reynir Björnsson
dd55491138 Document assumption 2020-12-22 13:45:54 +01:00
Reynir Björnsson
f2ab0d797f Simplify job_run_file and return 404 2020-12-22 13:36:14 +01:00
Reynir Björnsson
bf1777b7cb Toggle hide elements, artifact download links, etc 
Add job script, links to artifacts and a way to hide job script, build
log etc
 2020-12-21 11:22:35 +01:00
Reynir Björnsson
be1ae9cdab Implement downloading build artifacts 2020-12-21 11:21:10 +01:00
Reynir Björnsson
5f8325b60b Add SHA256 & SHA512 digests of build artifacts 2020-12-14 08:18:13 +01:00
Reynir Björnsson
bac7f8019d More css 2020-12-09 13:45:09 +01:00
Reynir Björnsson
0eadf387e6 Add anchor to timestamps 2020-12-09 13:44:44 +01:00
Reynir Björnsson
774074be63 timestamp html and css improvements 
Each time stamp has an ID that can be linked to. Currently, there's no
mechanism for getting the fragment. A solution could be to wrap the
timestamp in an anchor.
 2020-12-09 11:39:39 +01:00
Reynir Björnsson
eb039308ec Simplify RunMap compare 2020-12-08 11:49:42 +01:00
Reynir Björnsson
465ede64d6 Lwt-ify model 2020-12-08 11:49:26 +01:00