Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v8: serialize BigInt64Array and BigUint64Array #43571

Merged
merged 1 commit into from
Jul 15, 2022
Merged

v8: serialize BigInt64Array and BigUint64Array #43571

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 46 additions & 34 deletions lib/v8.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,8 @@

const {
Array,
ArrayBuffer,
ArrayPrototypeForEach,
ArrayPrototypePush,
BigInt64Array,
BigUint64Array,
DataView,
Error,
Float32Array,
Expand All @@ -27,7 +26,6 @@ const {
Int32Array,
Int8Array,
ObjectPrototypeToString,
SafeMap,
Uint16Array,
Uint32Array,
Uint8Array,
Expand Down Expand Up @@ -247,29 +245,40 @@ Deserializer.prototype.readRawBytes = function readRawBytes(length) {
length);
};

/* Keep track of how to handle different ArrayBufferViews.
* The default Serializer for Node does not use the V8 methods for serializing
* those objects because Node's `Buffer` objects use pooled allocation in many
* cases, and their underlying `ArrayBuffer`s would show up in the
* serialization. Because a) those may contain sensitive data and the user
* may not be aware of that and b) they are often much larger than the `Buffer`
* itself, custom serialization is applied. */
const arrayBufferViewTypes = [Int8Array, Uint8Array, Uint8ClampedArray,
Int16Array, Uint16Array, Int32Array, Uint32Array,
Float32Array, Float64Array, DataView];

const arrayBufferViewTypeToIndex = new SafeMap();

{
const dummy = new ArrayBuffer();
ArrayPrototypeForEach(arrayBufferViewTypes, (ctor, i) => {
const tag = ObjectPrototypeToString(new ctor(dummy));
arrayBufferViewTypeToIndex.set(tag, i);
});
function arrayBufferViewTypeToIndex(abView) {
const type = ObjectPrototypeToString(abView);
BridgeAR marked this conversation as resolved.
Show resolved Hide resolved
if (type === '[object Int8Array]') return 0;
if (type === '[object Uint8Array]') return 1;
if (type === '[object Uint8ClampedArray]') return 2;
if (type === '[object Int16Array]') return 3;
if (type === '[object Uint16Array]') return 4;
if (type === '[object Int32Array]') return 5;
if (type === '[object Uint32Array]') return 6;
if (type === '[object Float32Array]') return 7;
if (type === '[object Float64Array]') return 8;
if (type === '[object DataView]') return 9;
// Index 10 is FastBuffer.
if (type === '[object BigInt64Array]') return 11;
if (type === '[object BigUint64Array]') return 12;
return -1;
}

const bufferConstructorIndex =
ArrayPrototypePush(arrayBufferViewTypes, FastBuffer) - 1;
Comment on lines -271 to -272
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line made it kind of ugly and complex to simply extend the array literal on line 259.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following would work and be simpler (even though a bit hacky):

// Add "Array" as entry to arrayBufferViewTypes[10] above. Other constructors also work.
arrayBufferViewTypeToIndex.delete('[object Array]');
arrayBufferViewTypes[10] = FastBuffer;
const bufferConstructorIndex = 10;

Copy link
Member Author

@bnoordhuis bnoordhuis Jun 26, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially did something like that to keep the diff small but I made the judgment call the open-coded approach is going to be easier to understand and update.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is actually a even simpler solution:

--- a/lib/v8.js
+++ b/lib/v8.js
@@ -256,20 +256,23 @@ Deserializer.prototype.readRawBytes = function readRawBytes(length) {
  * itself, custom serialization is applied. */
 const arrayBufferViewTypes = [Int8Array, Uint8Array, Uint8ClampedArray,
                               Int16Array, Uint16Array, Int32Array, Uint32Array,
-                              Float32Array, Float64Array, DataView];
+                              Float32Array, Float64Array, DataView,
+                              FastBuffer, BigInt64Array, BigUint64Array];
 
 const arrayBufferViewTypeToIndex = new SafeMap();
 
 {
   const dummy = new ArrayBuffer();
   ArrayPrototypeForEach(arrayBufferViewTypes, (ctor, i) => {
+    if (ctor === FastBuffer) {
+      return;
+    }
     const tag = ObjectPrototypeToString(new ctor(dummy));
     arrayBufferViewTypeToIndex.set(tag, i);
   });
 }
 
-const bufferConstructorIndex =
-  ArrayPrototypePush(arrayBufferViewTypes, FastBuffer) - 1;
+const bufferConstructorIndex = 10;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I never replied. That would work but... I don't get warm fuzzies from do-this-except-when-it's-that logic. It's shorter but IMO the virtue of the open-coded approach is it's glaringly obvious what it does because it's so stupidly simple.

function arrayBufferViewIndexToType(index) {
if (index === 0) return Int8Array;
if (index === 1) return Uint8Array;
if (index === 2) return Uint8ClampedArray;
if (index === 3) return Int16Array;
if (index === 4) return Uint16Array;
if (index === 5) return Int32Array;
if (index === 6) return Uint32Array;
if (index === 7) return Float32Array;
if (index === 8) return Float64Array;
if (index === 9) return DataView;
if (index === 10) return FastBuffer;
if (index === 11) return BigInt64Array;
if (index === 12) return BigUint64Array;
return undefined;
}

class DefaultSerializer extends Serializer {
constructor() {
Expand All @@ -285,14 +294,17 @@ class DefaultSerializer extends Serializer {
* @returns {void}
*/
_writeHostObject(abView) {
let i = 0;
if (abView.constructor === Buffer) {
i = bufferConstructorIndex;
} else {
const tag = ObjectPrototypeToString(abView);
i = arrayBufferViewTypeToIndex.get(tag);

if (i === undefined) {
// Keep track of how to handle different ArrayBufferViews. The default
// Serializer for Node does not use the V8 methods for serializing those
// objects because Node's `Buffer` objects use pooled allocation in many
// cases, and their underlying `ArrayBuffer`s would show up in the
// serialization. Because a) those may contain sensitive data and the user
// may not be aware of that and b) they are often much larger than the
// `Buffer` itself, custom serialization is applied.
let i = 10; // FastBuffer
if (abView.constructor !== Buffer) {
BridgeAR marked this conversation as resolved.
Show resolved Hide resolved
i = arrayBufferViewTypeToIndex(abView);
if (i === -1) {
throw new this._getDataCloneError(
`Unserializable host object: ${inspect(abView)}`);
}
Expand All @@ -313,7 +325,7 @@ class DefaultDeserializer extends Deserializer {
*/
_readHostObject() {
const typeIndex = this.readUint32();
const ctor = arrayBufferViewTypes[typeIndex];
const ctor = arrayBufferViewIndexToType(typeIndex);
const byteLength = this.readUint32();
const byteOffset = this._readRawBytes(byteLength);
const BYTES_PER_ELEMENT = ctor.BYTES_PER_ELEMENT || 1;
Expand Down
8 changes: 8 additions & 0 deletions test/parallel/test-v8-serdes.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,18 @@ circular.circular = circular;
const objects = [
{ foo: 'bar' },
{ bar: 'baz' },
new Int8Array([1, 2, 3, 4]),
new Uint8Array([1, 2, 3, 4]),
new Int16Array([1, 2, 3, 4]),
new Uint16Array([1, 2, 3, 4]),
new Int32Array([1, 2, 3, 4]),
new Uint32Array([1, 2, 3, 4]),
new Float32Array([1, 2, 3, 4]),
new Float64Array([1, 2, 3, 4]),
new DataView(new ArrayBuffer(42)),
Buffer.from([1, 2, 3, 4]),
new BigInt64Array([42n]),
new BigUint64Array([42n]),
undefined,
null,
42,
Expand Down