Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream mode does not produce an array of JSON #321

Closed
mcqj opened this issue May 21, 2019 · 5 comments
Closed

Stream mode does not produce an array of JSON #321

mcqj opened this issue May 21, 2019 · 5 comments
Milestone

Comments

@mcqj
Copy link

mcqj commented May 21, 2019

I am not sure what the expected behaviour is.
My output is one JSON object per line with no wrapping array and no comma separators e.g.

{name:"Joe Bloggs",status:"alive"}
{name:"Alice Kay",status:"deceased"}

etc.

@GioLogist
Copy link

GioLogist commented May 31, 2019

+1 on this, but only when using it piped

@ushakov-ruslan
Copy link

ushakov-ruslan commented Jun 13, 2019

The same here. It's a little problem.
Here is an ugly fix but anyway might by helpful so far:

import fs from 'fs';
import {Transform} from 'stream';
import csvtojson from 'csvtojson/v2';

// flag to know if it's the first chunk
let theFirstChunk = true;

// create a read stream
const rs = fs.createReadStream(src);

const csv = csvtojson({
    ignoreEmpty: true,
  })
  .subscribe((item, index) => {
    // transform of json item here what you need
  });

// create a writable stream
const ws = fs.createWriteStream(dest);

// transformation stream
const transform = new Transform({
  transform(chunk, encoding, callback) {

    const json = JSON.parse(chunk);
    let string = `,${JSON.stringify(json)}`;

    if (theFirstChunk) {
      string = '[' + string.replace(/^\,\s*/, '');
      theFirstChunk = false;
    }

    callback(null, string);
  },

  // for the last chunk add ]
  flush(callback) {
    callback(null, ']');
  }

});

rs.pipe(csv).pipe(transform).pipe(ws);

@Keyang
Copy link
Owner

Keyang commented Jun 20, 2019

Duplicate of #307
Will be covered by 2.0.9

@Keyang Keyang added this to the 2.0.9 milestone Jun 20, 2019
@Keyang Keyang closed this as completed Jun 26, 2019
@adams-family
Copy link

Hi, I think that this ticket should not be closed because the array generated is syntactically incorrect, since it contains a comma after the last row:

Received result:
[
{ "Name": "Peter" },
{ "Name": "Anna" },
{ "Name": "Jean" },
]
Expected result:
[
{ "Name": "Peter" },
{ "Name": "Anna" },
{ "Name": "Jean" }
]

Using v 2.0.10.

@cdimascio
Copy link

@adams-family the following issue exists for the trailing comma issue, #333

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants