JS String performance - #1262
Conversation
…f8 string data direct access
|
It needs a little work, |
|
NativeString needs something like |
|
It can be even simpler than that, let me think |
|
Even simpler is having 2 objects: NativeStringContext and NativeString When NativeStringContext destroys, it rewinds the offset of a preallocated buffer to 0. This buffer can be 16mb. When NativeString needs to allocate it just takes from that 16mb NativeStringContext's buffer and increases the offset |
|
I will split it into NativeStringContext and NativeString in a seaprate PR |
|
Latest commit runs 20% faster in following for me: ./http_load_test 40 localhost 3000 16 ./http_load_test 40 localhost 3000 16 |
I mean, the new char[length] was a rare slow path, so idk if it was really worth creating a pool for it? The hot path is the utf8 path, using String::ValueView for direct access, without any allocation. |
|
I’d like to argue about 99% of strings being utf8, particularly in uWebSockets case. We are talking about the backend capable of real-time communication, which, obviously, is more about users’ freedom of messaging rather than English-only APIs |
|
It is true, I was referring to the http side, missing the websocket side, where utf16 may be more used.
|
|
From the latest commit I got: |
For utf16, you still need to copy it once via the code conversion. WriteUtf8V2 here does not perform any allocation (it will just hit the pool allocator) |
|
Actually you can remove Utf8LengthV2 (this is linear in length) and just rely on:
In hindsight. So you only call WriteUtf8V2 using all available capacity of the pool, then in hindsight adjust the pool offset based on how much was actually written. That's by far the most elegant way to handle Utf16 |
|
So the pool needs this interface: std::string_view capacity() |
|
It's not that complex to convert UTF16 into UTF8: |
|
Can also just use simdutf like so: |
* Revert "Fix NativeStringContext: fixed 8MB pool, malloc fallback, and proper free (#1264)" This reverts commit 71cac73. Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com> * Revert "Replace NativeString heap allocations with thread-local pool allocator (#1263)" This reverts commit 3e2ea85. Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com> * Revert "Update NativeString to use better copy mechanism and ValueView for utf8 string data direct access (#1262)" This reverts commit 6b4968f. Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com> * Revert "Add DataView support, update RecognizedString type (#1258)" This reverts commit 0b0c8f9. Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com>
As discussed regarding the performance of writeHeader, sending JS strings to c++ costs a lot (about 0.5% per string).
This is because the c++ wrapper actually performs a copy of the string locally.
Luckily, with Node.js 24, v8 introduced the String::ValueView, which allows direct access to the string data as we do for binary.
v8 also added
String::WriteUtf8V2(), a better way of string copying than creating a newString::Utf8Valueobject.I have implemented these and tested locally on a real app, it seems to work well so far.
One thing though, is that
String::ValueViewis a non-owning ephemeral view, it is only valid for current GC window.We have to consume the data synchronously or the pointer may become invalid, but I think it is the same thing for binary data?
If we need the data to be persistent on some place, we could add a
bool needStableto the NativeString constructor.Benchmark code:
Benchmark results for 7 writeHeader(13B, 13B):
binary: 162k Req/Sec
Old copy with Utf8Value:
utf16: 149k Req/Sec
utf-8: 155k Req/Sec
New copy with WriteUtf8V2:
utf16: 153k Req/Sec +2.7%
utf-8: 157k Req/Sec +1.3%
Direct access with ValueView:
utf-8: 163k Req/Sec +5.2%