Skip to content

Parallelised build, improved Github Actions (7 minutes build -> 3:20) - #1275

Closed
Guthib-of-Dan wants to merge 4 commits into
uNetworking:masterfrom
Guthib-of-Dan:master
Closed

Parallelised build, improved Github Actions (7 minutes build -> 3:20)#1275
Guthib-of-Dan wants to merge 4 commits into
uNetworking:masterfrom
Guthib-of-Dan:master

Conversation

@Guthib-of-Dan

Copy link
Copy Markdown

Main changes are

  • build.c was split into build-conf.h and build.c. This lets you clearly separate concerns and improve orientation
  • Makefile now compiles build.c ONLY to build.exe, which runs equally on Linux, MacOS and Windows + less headache
  • overall refactor of build.c to ensure readability of scripts + removed buffering of output for printf to see logs immediately and debug much more efficiently
  • added get_cpu_count to maximise thoughput of build boringssl and lsquic, parallelised build of uWebSockets.js binaries on Linux and MacOS (windows GitHub Actions usually give 1 -> 2 threads, not large performance gain). But if needed it can be done for Windows as well
  • silenced curl (!) requests to just work and log nothing
  • added caching of Git submodules to GitHub Actions, based on their update rates. So if several versions in a row don't bump uWebSockets -> Actions time drops SIGNIFICANTLY from 7 minutes each to 3:20 minutes MAX (compilation of uWebSockets.js with reused deps takes 10 seconds on my pc). As soon as .gitmodules changes (uWebSockets gets updated) -> cache changes and works the same way for next builds
  • Made GitHub Actions commit all binaries with 1 commit instead of 4 (for each os) + combined ASAN build after main build in Linux and MacOS to reduce overhead and reuse available folder structure

Guthib-of-Dan added 2 commits April 16, 2026 20:00
@Guthib-of-Dan

Guthib-of-Dan commented Apr 18, 2026 •

Copy link
Copy Markdown
Author

And NO package managers, cache is just a compiled directory of git submodules.

- name: Setup Make fo Windows
if: matrix.os == 'windows-latest'
uses: ilammy/setup-nasm@v1.2.1
run: choco install make

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a supply chain attack surface.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a static library though. But I will replace it then

@uNetworkingAB

Copy link
Copy Markdown
Contributor

This PR has many interesting parts. The speedup is mainly the cache of submodules?

Problem is, and to be very blunt and economical. The gain (GitHub Actions build time) vs. the complexity (review time) is probably not worth it. Particularly since this change moves in a different direction from my own "vision". You add complexity to lower GitHub Actions build time, I strive to reduce complexity and move all platforms to the same compilation path (the recent move to use Clang++ on all platforms was such a massive improvement). I have an abandoned branch that uses even more common Clang++ flags and removes the build_windows for one common path and while I never finished it, that's the kind of direction the build files are moving towards.

Also, this proejct lived for at least 5 years where build times were 50 minutes. And that was never a problem. So going from 50 -> 5 minutes while reducing complexity (that's right, the move was due to removing stuff), this change of yours cuts 3 minutes by introducing more complexity and more spinning wheels to care about.

Further on, I have lived long enough to know from experience that build caches ALWAYS blow up in your face sooner or later. I cannot recollect how many times I have solved problems by clearing such caches.

TLDR; it's certainly good and all, but this is not where the money is. Can this PR be merged? Certainly, it absolutely can. But I don't have the time or interest to review it. We can save it and return to it later (there are some raisins worth picking like the setbuf fix).

@Guthib-of-Dan

Copy link
Copy Markdown
Author

I see the point, but my desire to make this library better in any possible way still persists. If you can’t review large modifications and those changes that add complexity, I will just split this into parts that are manageable and self-explanatory.

@Guthib-of-Dan

Copy link
Copy Markdown
Author

So I am cancelling this pull request. I will PR its functionality step by step. I just understood that I had to make PRs only from separate branches and update "master" in fork only when PR gets approved. So now I have to reload the fork. I am still learning, ok?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants