node.js – What do encodings represent in node:stream?

I’m creating a Transform stream to alter the output from a Readable stream before sending it to a Writable stream: childProcess.stdout.pipe(transform).pipe(process.stdout).

However, I’m having difficulty understanding how encodings are meant to be used with streams, to the point that much of it seems directly opposite what I would expect. In particular, when my transform() method receives a chunk which is a Buffer, the specified encoding is always "buffer", while string chunks can have any of a number of encodings.

But… isn’t that backwards? Strings are meant to represent sequences of characters (stored in memory as UCS-2/UTF-16/ish, sure, but conceptually just code points) whereas buffers are sequences of bytes. Characters can be encoded as bytes, so a string needs encoded to become a buffer and a buffer needs decoded to become a string. Therefore, shouldn’t buffers have encodings (if they’re meant to represent character data at all) while strings should not?

Yet when receiving a buffer from a stream, I don’t get told what encoding was used to create it. My transform stream doesn’t even have access to the source stream (after all, not all sources are streams), so I can’t check to see if an encoding has been set on it. Conversely, when I receive a string, I get an encoding I don’t know what to do with.

Unfortunately, neither digging through the documentation nor searching the web have been very enlightening. What am I missing?

Read more here: Source link