Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buffer.slice unexpected behavior #9096

Closed
rideg opened this issue Oct 14, 2016 · 1 comment
Closed

Buffer.slice unexpected behavior #9096

rideg opened this issue Oct 14, 2016 · 1 comment
Labels
buffer Issues and PRs related to the buffer subsystem.

Comments

@rideg
Copy link

rideg commented Oct 14, 2016

  • Version: v6.7.0
  • Platform: Linux 3.13.0-96-generic Ubuntu SMP x86_64 GNU/Linux
  • Subsystem: NodeBuffer

It seems if one provides a non integer boundary to the slice function then the later slice calls on the same buffer provide wrong chunks.

Test case to reproduce

it("Should slice Buffer", () => {
    let original = new Buffer("1234567890ABCDEFG", "ascii");
    let chunk1 = original.slice(0, original.length / 3);
    let chunk2 = original.slice(original.length / 3);
    let result = Buffer.concat([chunk2, chunk1]);
    expect(result).to.be.eqls(original);
});

And the result is:

AssertionError: expected 
|Buffer 31 32 33 34 35 36 37 38 39 30 41 42 43 44 45 46| to deeply equal |Buffer 31 32 33 34 35 36 37 38 39 30 41 42 43 44 45 46 `**47**`|

In this test case the result buffer misses the last byte from the original one.

I am not exactly sure what should be the correct behavior but I would be fine either with throwing an exception or round/ceil/floor the provided boundary value.

@vkurchatkin vkurchatkin added the buffer Issues and PRs related to the buffer subsystem. label Oct 14, 2016
@mscdex
Copy link
Contributor

mscdex commented Oct 14, 2016

/cc @nodejs/buffer

thefourtheye added a commit to thefourtheye/io.js that referenced this issue Oct 15, 2016
As shown in nodejs#9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: nodejs#9096
jasnell pushed a commit that referenced this issue Oct 18, 2016
As shown in #9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: #9096
PR-URL: #9101
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Franziska Hinkelmann <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Brian White <[email protected]>
jasnell pushed a commit that referenced this issue Oct 19, 2016
As shown in #9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: #9096
PR-URL: #9101
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Franziska Hinkelmann <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Brian White <[email protected]>
cjihrig added a commit to cjihrig/node that referenced this issue Nov 5, 2016
This is a partial revert of
14d1a8a, which coerced the offset
of Buffer#slice() using the | operator. This causes some edge
cases to be handled incorrectly. This commit restores the old
behavior, but converts offsets to integers using Math.trunc().
This commit does not revert any tests, and adds an additional
regression test.

Refs: nodejs#9096
Refs: nodejs#9101
PR-URL: nodejs#9341
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Brian White <[email protected]>
Reviewed-By: Trevor Norris <[email protected]>
evanlucas pushed a commit that referenced this issue Nov 7, 2016
This is a partial revert of
14d1a8a, which coerced the offset
of Buffer#slice() using the | operator. This causes some edge
cases to be handled incorrectly. This commit restores the old
behavior, but converts offsets to integers using Math.trunc().
This commit does not revert any tests, and adds an additional
regression test.

Refs: #9096
Refs: #9101
PR-URL: #9341
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Brian White <[email protected]>
Reviewed-By: Trevor Norris <[email protected]>
MylesBorins pushed a commit that referenced this issue Nov 18, 2016
As shown in #9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: #9096
PR-URL: #9101
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Franziska Hinkelmann <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Brian White <[email protected]>
MylesBorins pushed a commit that referenced this issue Nov 18, 2016
This is a partial revert of
14d1a8a, which coerced the offset
of Buffer#slice() using the | operator. This causes some edge
cases to be handled incorrectly. This commit restores the old
behavior, but converts offsets to integers using Math.trunc().
This commit does not revert any tests, and adds an additional
regression test.

Refs: #9096
Refs: #9101
PR-URL: #9341
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Brian White <[email protected]>
Reviewed-By: Trevor Norris <[email protected]>
MylesBorins pushed a commit that referenced this issue Nov 19, 2016
As shown in #9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: #9096
PR-URL: #9101
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Franziska Hinkelmann <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Brian White <[email protected]>
MylesBorins pushed a commit that referenced this issue Nov 19, 2016
This is a partial revert of
14d1a8a, which coerced the offset
of Buffer#slice() using the | operator. This causes some edge
cases to be handled incorrectly. This commit restores the old
behavior, but converts offsets to integers using Math.trunc().
This commit does not revert any tests, and adds an additional
regression test.

Refs: #9096
Refs: #9101
PR-URL: #9341
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Brian White <[email protected]>
Reviewed-By: Trevor Norris <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
buffer Issues and PRs related to the buffer subsystem.
Projects
None yet
Development

No branches or pull requests

3 participants