Skip to content

JS String performance - #1262

Merged
uNetworkingAB merged 1 commit into
uNetworking:masterfrom
ValoChet:stringView
Mar 29, 2026
Merged

JS String performance#1262
uNetworkingAB merged 1 commit into
uNetworking:masterfrom
ValoChet:stringView

Conversation

@ValoChet

@ValoChet ValoChet commented Mar 29, 2026 •

Copy link
Copy Markdown
Contributor

As discussed regarding the performance of writeHeader, sending JS strings to c++ costs a lot (about 0.5% per string).
This is because the c++ wrapper actually performs a copy of the string locally.

Luckily, with Node.js 24, v8 introduced the String::ValueView, which allows direct access to the string data as we do for binary.
v8 also added String::WriteUtf8V2(), a better way of string copying than creating a new String::Utf8Value object.

I have implemented these and tested locally on a real app, it seems to work well so far.

One thing though, is that String::ValueView is a non-owning ephemeral view, it is only valid for current GC window.
We have to consume the data synchronously or the pointer may become invalid, but I think it is the same thing for binary data?
If we need the data to be persistent on some place, we could add a bool needStable to the NativeString constructor.

Benchmark code:

import uWS from "./uwsmaster/dist/uws.js";

const port = 9001;

const nbCall = 7;
const utf8String = "Abcdefghijklm";
const binaryString = new TextEncoder().encode(utf8String).buffer;
const utf16String = Buffer.from(binaryString).toString("utf16le");

const app = uWS./*SSL*/App({
    key_file_name: 'misc/key.pem',
    cert_file_name: 'misc/cert.pem',
    passphrase: '1234'
}).get('/test', (res, req) => {
    res.write(Date.now().toString());
    res.write(binaryString);
    res.write(utf16String);
    res.write(utf8String);
    res.end();
}).get('/binary', (res, req) => {
    for (let i = 0; i < nbCall; i ++) res.writeHeader(binaryString, binaryString);
    res.end();
}).get('/utf16', (res, req) => {
    for (let i = 0; i < nbCall; i ++) res.writeHeader(utf16String, utf16String);
    res.end();
}).get('/utf8', (res, req) => {
    for (let i = 0; i < nbCall; i ++) res.writeHeader(utf8String, utf8String);
    res.end();
}).listen(port, (token) => {
    if (token) {
        console.log('Listening to port ' + port);
    } else {
        console.log('Failed to listen to port ' + port);
    }
});

Benchmark results for 7 writeHeader(13B, 13B):

binary: 162k Req/Sec

Old copy with Utf8Value:
utf16: 149k Req/Sec
utf-8: 155k Req/Sec

New copy with WriteUtf8V2:
utf16: 153k Req/Sec +2.7%
utf-8: 157k Req/Sec +1.3%

Direct access with ValueView:
utf-8: 163k Req/Sec +5.2%

@ValoChet
ValoChet marked this pull request as ready for review March 29, 2026 17:34
@ValoChet
ValoChet marked this pull request as draft March 29, 2026 17:34
@uNetworkingAB

Copy link
Copy Markdown
Contributor

It needs a little work, data = new char[length]; is not something you want in a hot path if it can be avoided. A NativeString needs to pick a preallocated buffer from a pool.

@uNetworkingAB

Copy link
Copy Markdown
Contributor

NativeString needs something like static std::string preallocated_pool[8]; int preallocated_pool_top = 0; where any first allocation takes from the pool and increases the size of that buffer to fit, and leaves it like that unless size is more than some threshold like 1MB.

@uNetworkingAB

Copy link
Copy Markdown
Contributor

It can be even simpler than that, let me think

@uNetworkingAB

Copy link
Copy Markdown
Contributor

Even simpler is having 2 objects:

NativeStringContext and NativeString

When NativeStringContext destroys, it rewinds the offset of a preallocated buffer to 0. This buffer can be 16mb.

When NativeString needs to allocate it just takes from that 16mb NativeStringContext's buffer and increases the offset

@uNetworkingAB
uNetworkingAB marked this pull request as ready for review March 29, 2026 18:38
@uNetworkingAB
uNetworkingAB merged commit 6b4968f into uNetworking:master Mar 29, 2026
1 check failed
@uNetworkingAB

Copy link
Copy Markdown
Contributor

I will split it into NativeStringContext and NativeString in a seaprate PR

@uNetworkingAB

Copy link
Copy Markdown
Contributor

Latest commit runs 20% faster in following for me:

}).get('/', (res, req) => {
  res.writeHeader("Hello", "There");
  res.writeHeader("Hi", "On you");
  res.end('Hello World!');

./http_load_test 40 localhost 3000 16
Using pipeline factor of 16
request size 1248
Running benchmark now...
Req/sec: 354896.000000
Req/sec: 354584.000000

./http_load_test 40 localhost 3000 16
Using pipeline factor of 16
request size 1248
Running benchmark now...
Req/sec: 413100.000000
Req/sec: 419204.000000
Req/sec: 422752.000000

@ValoChet

Copy link
Copy Markdown
Contributor Author

It needs a little work, data = new char[length]; is not something you want in a hot path if it can be avoided. A NativeString needs to pick a preallocated buffer from a pool.

I mean, the new char[length] was a rare slow path, so idk if it was really worth creating a pool for it?
Node.js stores strings as utf8 if they contain only utf8 chars, which are 99% of strings.

The hot path is the utf8 path, using String::ValueView for direct access, without any allocation.
It only copies and converts if it is a utf16 string, or node.js version < 24.

@Guthib-of-Dan

Copy link
Copy Markdown

I’d like to argue about 99% of strings being utf8, particularly in uWebSockets case. We are talking about the backend capable of real-time communication, which, obviously, is more about users’ freedom of messaging rather than English-only APIs

@ValoChet

Copy link
Copy Markdown
Contributor Author

It is true, I was referring to the http side, missing the websocket side, where utf16 may be more used.

String::ValueView also provides direct access to utf16 string buffer as a uint16_t *, maybe we can find a way to convert it without allocation tho?

@ValoChet

Copy link
Copy Markdown
Contributor Author

From the latest commit I got:
binary: 161k Req/Sec
utf16copy: 158k Req/Sec
utf-8copy: 160k Req/Sec
utf-8view: 162k Req/Sec

@uNetworkingAB

Copy link
Copy Markdown
Contributor

String::ValueView also provides direct access to utf16 string buffer as a uint16_t *, maybe we can find a way to convert it without allocation tho?

                    // utf16: copy and convert to utf8
                    length = string->Utf8LengthV2(isolate);
                    data = ctx.alloc(length);
                    allocated = true;
                    string->WriteUtf8V2(isolate, data, length);

For utf16, you still need to copy it once via the code conversion. WriteUtf8V2 here does not perform any allocation (it will just hit the pool allocator)

@uNetworkingAB

Copy link
Copy Markdown
Contributor

Actually you can remove Utf8LengthV2 (this is linear in length) and just rely on:

size_t * processed_characters_return = nullptr

In hindsight. So you only call WriteUtf8V2 using all available capacity of the pool, then in hindsight adjust the pool offset based on how much was actually written.

That's by far the most elegant way to handle Utf16

@uNetworkingAB

Copy link
Copy Markdown
Contributor

So the pool needs this interface:

std::string_view capacity()
consume(int)

@uNetworkingAB

Copy link
Copy Markdown
Contributor

It's not that complex to convert UTF16 into UTF8:

/**
 * Converts a UTF-16 string to UTF-8.
 * @param out      Pointer to the destination buffer.
 * @param in       Pointer to the source UTF-16 buffer.
 * @param in_len   Number of UTF-16 elements to process.
 * @return         The number of bytes written to the 'out' buffer.
 */
int utf16_to_utf8(unsigned char* out, const unsigned short* in, int in_len) {
    int out_idx = 0;

    for (int i = 0; i < in_len; i++) {
        unsigned int cp = in[i];

        // Handle Surrogate Pairs
        if (cp >= 0xD800 && cp <= 0xDBFF) {
            if (i + 1 < in_len) {
                unsigned int next = in[++i];
                if (next >= 0xDC00 && next <= 0xDFFF) {
                    // Combine high and low surrogates into a single code point
                    cp = 0x10000 + ((cp & 0x3FF) << 10) + (next & 0x3FF);
                }
            }
        }

        // Encode Code Point into UTF-8 bytes
        if (cp <= 0x7F) {
            out[out_idx++] = (unsigned char)cp;
        } 
        else if (cp <= 0x7FF) {
            out[out_idx++] = (unsigned char)((cp >> 6) | 0xC0);
            out[out_idx++] = (unsigned char)((cp & 0x3F) | 0x80);
        } 
        else if (cp <= 0xFFFF) {
            out[out_idx++] = (unsigned char)((cp >> 12) | 0xE0);
            out[out_idx++] = (unsigned char)(((cp >> 6) & 0x3F) | 0x80);
            out[out_idx++] = (unsigned char)((cp & 0x3F) | 0x80);
        } 
        else if (cp <= 0x10FFFF) {
            out[out_idx++] = (unsigned char)((cp >> 18) | 0xF0);
            out[out_idx++] = (unsigned char)(((cp >> 12) & 0x3F) | 0x80);
            out[out_idx++] = (unsigned char)(((cp >> 6) & 0x3F) | 0x80);
            out[out_idx++] = (unsigned char)((cp & 0x3F) | 0x80);
        }
    }

    return out_idx;
}

@uNetworkingAB

Copy link
Copy Markdown
Contributor

Can also just use simdutf like so:

size_t actual_len = simdutf::convert_utf16_to_utf8(
        utf16_input.data(), 
        utf16_input.length(), 
        utf8_output.data()
    );

Copilot AI added a commit that referenced this pull request Apr 2, 2026
…w for utf8 string data direct access (#1262)"

This reverts commit 6b4968f.

Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com>
uNetworkingAB added a commit that referenced this pull request Apr 2, 2026
* Revert "Fix NativeStringContext: fixed 8MB pool, malloc fallback, and proper free (#1264)"

This reverts commit 71cac73.

Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com>

* Revert "Replace NativeString heap allocations with thread-local pool allocator (#1263)"

This reverts commit 3e2ea85.

Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com>

* Revert "Update NativeString to use better copy mechanism and ValueView for utf8 string data direct access (#1262)"

This reverts commit 6b4968f.

Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com>

* Revert "Add DataView support, update RecognizedString type (#1258)"

This reverts commit 0b0c8f9.

Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: uNetworkingAB <110806833+uNetworkingAB@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants