Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(v5): merge v5 #657

Merged
merged 72 commits into from
Sep 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
2cf4b46
refactor: emit `drain` as soon as capacity is available (#367)
metcoder95 Jul 14, 2023
7b86105
refactor!: drop runTask method (#363)
metcoder95 Sep 21, 2023
b42bb14
chore(deps-dev): Bump @types/node from 20.6.0 to 20.6.2 (#406)
dependabot[bot] Sep 19, 2023
4df0ca4
fix: add signal reason support (#403)
metcoder95 Sep 20, 2023
986659a
chore(deps): Bump actions/checkout from 3 to 4 (#413)
dependabot[bot] Oct 2, 2023
bf92e5a
chore(deps-dev): Bump @types/node from 20.6.2 to 20.8.2 (#417)
dependabot[bot] Oct 2, 2023
352bc84
chore(deps-dev): Bump @types/node from 20.8.2 to 20.8.3 (#419)
dependabot[bot] Oct 15, 2023
a998a69
chore(deps): Bump @babel/traverse (#425)
dependabot[bot] Oct 20, 2023
31f7a2b
fix: do not re-create threads when calling `.destory()` (#430)
alan-agius4 Oct 20, 2023
3d67926
chore(deps-dev): Bump @types/node from 20.8.3 to 20.8.7 (#428)
dependabot[bot] Oct 23, 2023
5c2264c
fix: migrate to EventEmitterAsyncResource from core (#433)
groozin Oct 30, 2023
69cae53
chore(deps): Bump actions/setup-node from 3 to 4 (#437)
dependabot[bot] Nov 6, 2023
0cfa8d7
chore(deps-dev): Bump @types/node from 20.8.7 to 20.8.10 (#440)
dependabot[bot] Nov 6, 2023
c9e2abc
chore(deps-dev): Bump @typescript-eslint/parser from 6.9.1 to 6.10.0 …
dependabot[bot] Nov 13, 2023
ef6e205
chore(deps-dev): Bump @types/node from 20.8.10 to 20.9.0 (#443)
dependabot[bot] Nov 13, 2023
1f85c16
chore(deps-dev): Bump @typescript-eslint/eslint-plugin from 6.9.1 to …
dependabot[bot] Nov 15, 2023
b7895b9
chore(release): 4.2.0
metcoder95 Nov 19, 2023
cc8fca5
chore(deps-dev): Bump @types/node from 20.9.0 to 20.9.2 (#448)
dependabot[bot] Nov 20, 2023
4260e7e
chore(deps-dev): Bump @typescript-eslint/parser from 6.10.0 to 6.11.0…
dependabot[bot] Nov 20, 2023
99ae84b
chore(deps-dev): Bump @typescript-eslint/eslint-plugin from 6.10.0 to…
dependabot[bot] Nov 21, 2023
27cac34
chore(deps-dev): Bump @typescript-eslint/eslint-plugin from 6.11.0 to…
dependabot[bot] Nov 27, 2023
b6e2559
chore(deps-dev): Bump @typescript-eslint/parser from 6.11.0 to 6.12.0…
dependabot[bot] Nov 27, 2023
b08a78e
chore(deps-dev): Bump @types/node from 20.9.2 to 20.10.0 (#455)
dependabot[bot] Nov 27, 2023
be15076
chore(deps-dev): Bump typescript from 5.1.6 to 5.3.2 (#454)
dependabot[bot] Nov 27, 2023
a150938
fix: default minThreads with odd CPU count (#457)
Chocobozzz Dec 3, 2023
6ea35db
chore(deps-dev): Bump @types/node from 20.10.0 to 20.10.4 (#466)
dependabot[bot] Dec 13, 2023
41408c1
chore(deps-dev): Bump ts-node from 9.1.1 to 10.9.2 (#463)
dependabot[bot] Dec 13, 2023
682a931
chore(release): 4.2.1
metcoder95 Dec 13, 2023
d30c6b3
chore(deps-dev): Bump @typescript-eslint/parser from 6.12.0 to 6.14.0…
dependabot[bot] Dec 13, 2023
89ee83a
chore(deps-dev): Bump typescript from 5.3.2 to 5.3.3 (#464)
dependabot[bot] Dec 13, 2023
8deb653
chore(deps-dev): Bump @typescript-eslint/eslint-plugin from 6.12.0 to…
dependabot[bot] Dec 13, 2023
88b40d8
chore(deps): Bump @babel/traverse from 7.22.8 to 7.23.2 (#469)
dependabot[bot] Dec 13, 2023
8dadd43
chore(deps-dev): Bump @types/node from 20.10.4 to 20.10.5 (#470)
dependabot[bot] Dec 19, 2023
d84adb3
chore(deps-dev): Bump @typescript-eslint/eslint-plugin from 6.14.0 to…
dependabot[bot] Dec 28, 2023
c1996d4
chore(deps-dev): Bump @typescript-eslint/parser from 6.14.0 to 6.16.0…
dependabot[bot] Dec 28, 2023
cff5414
chore(deps-dev): Bump @types/node from 20.10.5 to 20.10.6 (#476)
dependabot[bot] Jan 3, 2024
ef07d02
chore(deps-dev): Bump @typescript-eslint/eslint-plugin from 6.15.0 to…
dependabot[bot] Jan 3, 2024
40b5e6b
chore(deps-dev): Bump @types/node from 20.10.6 to 20.10.7 (#479)
dependabot[bot] Jan 12, 2024
a3571fd
feat: use native Node.js histogram support (#482)
clydin Jan 12, 2024
e848054
chore(deps-dev): Bump @types/node from 20.11.0 to 20.11.1 (#485)
dependabot[bot] Jan 16, 2024
79d7401
chore(release): 4.3.0
metcoder95 Jan 16, 2024
aabaa5b
chore(deps-dev): Bump @typescript-eslint/parser from 6.18.1 to 6.19.0…
dependabot[bot] Jan 26, 2024
177a58d
chore: workflows: drop v16.x (#495)
RafaelGSS Jan 27, 2024
6f4ef50
fix(#491): out of bounds histogram value (#496)
metcoder95 Jan 28, 2024
c1f533d
chore(release): 4.3.1
metcoder95 Jan 30, 2024
335e406
chore(deps-dev): Bump @typescript-eslint/eslint-plugin from 6.18.1 to…
dependabot[bot] Jan 31, 2024
be9e83b
fix(#513): forward errors correctly to Piscina (#514)
metcoder95 Feb 16, 2024
86a5b21
chore(release): 4.3.2
metcoder95 Feb 16, 2024
e0686f6
chore(deps-dev): Bump @types/node from 20.11.1 to 20.11.19 (#516)
dependabot[bot] Feb 21, 2024
07b03eb
chore(deps): Bump actions/cache from 3 to 4 (#505)
dependabot[bot] Feb 21, 2024
01726c3
feat: add option to disable run/wait time recording (#518)
clydin Feb 21, 2024
ff8dee0
feat: allow named import usage (#517)
clydin Feb 21, 2024
683cda4
chore(release): 4.4.0
metcoder95 Feb 28, 2024
b2d7e62
chore(deps-dev): Bump @types/node from 20.11.19 to 20.11.24 (#524)
dependabot[bot] Mar 6, 2024
eb3008c
feat!: narrow TS types for histograms
metcoder95 Mar 10, 2024
2877294
chore: add v21.x to CI
metcoder95 Mar 10, 2024
b60d8ba
chore: adjust cache
metcoder95 Mar 10, 2024
4b391ac
chore: update package-lock.json
metcoder95 Mar 10, 2024
4bb872f
Merge remote-tracking branch 'origin/current' into next
metcoder95 May 29, 2024
57570c6
refactor: cleanups
metcoder95 May 29, 2024
6cbc507
fix: lint
metcoder95 May 29, 2024
7e40e2d
Merge remote-tracking branch 'origin/current' into next
metcoder95 May 31, 2024
d701046
fix: merge with current
metcoder95 May 31, 2024
144bd1c
feat!: set FixedQueue as default task queue (#578)
metcoder95 Jun 4, 2024
363b5af
docs: remove runtask references (#581)
metcoder95 Jun 12, 2024
3729907
feat!: drop v16 (#582)
metcoder95 Jun 12, 2024
c5946d1
Merge remote-tracking branch 'origin/current' into next
metcoder95 Jul 21, 2024
267bde4
Merge remote-tracking branch 'origin/current' into next
metcoder95 Sep 15, 2024
53e41f4
chore: reconcile lockfile
metcoder95 Sep 15, 2024
bbb684d
docs: update README
metcoder95 Sep 15, 2024
a828c8d
Merge remote-tracking branch 'origin/current' into next
metcoder95 Sep 18, 2024
a8447c0
fix: bad merge
metcoder95 Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/nodejs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ jobs:
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: v20.x
node-version: v22.x
cache: 'npm'
cache-dependency-path: package.json

- name: Install dependencies
run: npm install
- name: Check linting
Expand All @@ -36,7 +37,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
node-version: [16.x, 18.x, 20.x, 22.x]
node-version: [18.x, 20.x, 22.x]
runs-on: ${{matrix.os}}
steps:
- uses: actions/checkout@v4
Expand All @@ -49,8 +50,9 @@ jobs:
node-version: ${{ matrix.node-version }}
cache: 'npm'
cache-dependency-path: package.json

- name: Install Dependencies
run: npm install

- name: Run Tests
run: npm run test:ci
145 changes: 16 additions & 129 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,14 @@

Written in TypeScript.

For Node.js 16.x and higher.
For Node.js 18.x and higher.

[MIT Licensed][].

## Documentation

- [Website](https://piscinajs.github.io/piscina/)

## Piscina API

### Example
Expand Down Expand Up @@ -382,33 +386,6 @@ an error, the returned `Promise` will be rejected with that error.
If the task is aborted, the returned `Promise` is rejected with an error
as well.

### Method: `runTask(task[, transferList][, filename][, abortSignal])`

**Deprecated** -- Use `run(task, options)` instead.

Schedules a task to be run on a Worker thread.

* `task`: Any value. This will be passed to the function that is exported from
`filename`.
* `transferList`: An optional lists of objects that is passed to
[`postMessage()`] when posting `task` to the Worker, which are transferred
rather than cloned.
* `filename`: Optionally overrides the `filename` option passed to the
constructor for this task. If no `filename` was specified to the constructor,
this is mandatory.
* `abortSignal`: An [`AbortSignal`][] instance. If passed, this can be used to
cancel a task. If the task is already running, the corresponding `Worker`
thread will be stopped.
(More generally, any `EventEmitter` or `EventTarget` that emits `'abort'`
events can be passed here.) Abortable tasks cannot share threads regardless
of the `concurrentTasksPerWorker` options.

This returns a `Promise` for the return value of the (async) function call
made to the function exported from `filename`. If the (async) function throws
an error, the returned `Promise` will be rejected with that error.
If the task is aborted, the returned `Promise` is rejected with an error
as well.

### Method: `destroy()`

Stops all Workers and rejects all `Promise`s for pending tasks.
Expand Down Expand Up @@ -439,12 +416,16 @@ An `'error'` event is emitted by instances of this class when:
- Unexpected messages are sent from from Worker threads.

All other errors are reported by rejecting the `Promise` returned from
`run()` or `runTask()`, including rejections reported by the handler function
`run()`, including rejections reported by the handler function
itself.

### Event: `'drain'`

A `'drain'` event is emitted whenever the `queueSize` reaches `0`.
A `'drain'` event is emitted when the current usage of the
pool is below the maximum capacity of the same.
The intended goal is to provide backpressure to the task source
so creating tasks that can not be executed at immediately can be avoided.


### Event: `'needsDrain'`

Expand Down Expand Up @@ -697,7 +678,7 @@ An example of a custom task queue that uses a shuffled priority queue
is available in [`examples/task-queue`](./examples/task-queue/index.js);

The special symbol `Piscina.queueOptionsSymbol` may be set as a property
on tasks submitted to `run()` or `runTask()` as a way of passing additional
on tasks submitted to `run()` as a way of passing additional
options on to the custom `TaskQueue` implementation. (Note that because the
queue options are set as a property on the task, tasks with queue
options cannot be submitted as JavaScript primitives).
Expand Down Expand Up @@ -744,14 +725,13 @@ const piscina = new Piscina({

// Submit tasks to the pool
for (let i = 0; i < 10; i++) {
piscina.runTask({ data: i }).then((result) => {
piscina.run({ data: i }).then((result) => {
console.log(result);
}).catch((error) => {
console.error(error);
});
}
```
**Note** The `FixedQueue` will become the default task queue implementation in a next major version.

## Current Limitations (Things we're working on / would love help with)

Expand Down Expand Up @@ -877,107 +857,14 @@ documentation that threads are being used. It would be ideal if those
would make it possible for users to provide an existing `Piscina` instance
as a configuration option in lieu of always creating their own.


## Release Notes

### 4.1.0

#### Features

* add `needsDrain` property ([#368](https://github.com/piscinajs/piscina/issues/368)) ([2d49b63](https://github.com/piscinajs/piscina/commit/2d49b63368116c172a52e2019648049b4d280162))
* correctly handle process.exit calls outside of a task ([#361](https://github.com/piscinajs/piscina/issues/361)) ([8e6d16e](https://github.com/piscinajs/piscina/commit/8e6d16e1dc23f8bb39772ed954f6689852ad435f))


#### Bug Fixes

* Fix types for TypeScript 4.7 ([#239](https://github.com/piscinajs/piscina/issues/239)) ([a38fb29](https://github.com/piscinajs/piscina/commit/a38fb292e8fcc45cc20abab8668f82d908a24dc0))
* use CJS imports ([#374](https://github.com/piscinajs/piscina/issues/374)) ([edf8dc4](https://github.com/piscinajs/piscina/commit/edf8dc4f1a19e9b49e266109cdb70d9acc86f3ca))

### 4.0.0

* Drop Node.js 14.x support
* Add Node.js 20.x to CI

### 3.2.0

* Adds a new `PISCINA_DISABLE_ATOMICS` environment variable as an alternative way of
disabling Piscina's internal use of the `Atomics` API. (https://github.com/piscinajs/piscina/pull/163)
* Fixes a bug with transferable objects. (https://github.com/piscinajs/piscina/pull/155)
* Fixes CI issues with TypeScript. (https://github.com/piscinajs/piscina/pull/161)

### 3.1.0

* Deprecates `piscina.runTask()`; adds `piscina.run()` as an alternative.
https://github.com/piscinajs/piscina/commit/d7fa24d7515789001f7237ad6ae9ad42d582fc75
* Allows multiple exported handler functions from a single file.
https://github.com/piscinajs/piscina/commit/d7fa24d7515789001f7237ad6ae9ad42d582fc75

### 3.0.0

* Drops Node.js 10.x support
* Updates minimum TypeScript target to ES2019

### 2.1.0

* Adds name property to indicate `AbortError` when tasks are
canceled using an `AbortController` (or similar)
* More examples

### 2.0.0

* Added unmanaged file descriptor tracking
* Updated dependencies

### 1.6.1

* Bug fix: Reject if AbortSignal is already aborted
* Bug Fix: Use once listener for abort event

### 1.6.0

* Add the `niceIncrement` configuration parameter.

### 1.5.1

* Bug fixes around abortable task selection.

### 1.5.0

* Added `Piscina.move()`
* Added Custom Task Queues
* Added utilization metric
* Wait for workers to be ready before considering them as candidates
* Additional examples

### 1.4.0

* Added `maxQueue = 'auto'` to autocalculate the maximum queue size.
* Added more examples, including an example of implementing a worker
as a Node.js native addon.

### 1.3.0

* Added the `'drain'` event

### 1.2.0

* Added support for ESM and file:// URLs
* Added `env`, `argv`, `execArgv`, and `workerData` options
* More examples

### 1.1.0

* Added support for Worker Thread `resourceLimits`

### 1.0.0

* Initial release!

## The Team

* James M Snell <[email protected]>
* Anna Henningsen <[email protected]>
* Matteo Collina <[email protected]>
* Rafael Gonzaga <[email protected]>
* Robert Nagy <[email protected]>
* Carlos Fuentes <[email protected]>

## Acknowledgements

Expand Down
4 changes: 2 additions & 2 deletions benchmark/piscina-queue-comparison.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ bench
});
const tasks = [];
for (let i = 0; i < QUEUE_SIZE; i++) {
tasks.push(pool.runTask({ a: 4, b: 6 }));
tasks.push(pool.run({ a: 4, b: 6 }));
}
await Promise.all(tasks);
await pool.destroy();
Expand All @@ -28,7 +28,7 @@ bench
});
const tasks = [];
for (let i = 0; i < QUEUE_SIZE; i++) {
tasks.push(pool.runTask({ a: 4, b: 6 }));
tasks.push(pool.run({ a: 4, b: 6 }));
}
await Promise.all(tasks);
await pool.destroy();
Expand Down
2 changes: 1 addition & 1 deletion benchmark/simple-benchmark-fixed-queue.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ async function simpleBenchmark ({ duration = 10000 } = {}) {

async function scheduleTasks () {
while ((process.hrtime.bigint() - start) / 1_000_000n < duration) {
await pool.runTask({ a: 4, b: 6 });
await pool.run({ a: 4, b: 6 });
done++;
}
}
Expand Down
2 changes: 1 addition & 1 deletion benchmark/simple-benchmark.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ async function simpleBenchmark ({ duration = 10000 } = {}) {

async function scheduleTasks () {
while ((process.hrtime.bigint() - start) / 1_000_000n < duration) {
await pool.runTask({ a: 4, b: 6 });
await pool.run({ a: 4, b: 6 });
done++;
}
}
Expand Down
2 changes: 1 addition & 1 deletion examples/react-ssr/pooled.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ fastify.register(require('fastify-piscina'), {
});

// Declare a route
fastify.get('/', async () => fastify.runTask({ name: 'James' }));
fastify.get('/', async () => fastify.run({ name: 'James' }));

// Run the server!
const start = async () => {
Expand Down
2 changes: 1 addition & 1 deletion examples/server/async-sleep-pooled.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ fastify.register(require('fastify-piscina'), {
});

// Declare a route
fastify.get('/', () => fastify.runTask());
fastify.get('/', () => fastify.run());

// Run the server!
const start = async () => {
Expand Down
2 changes: 1 addition & 1 deletion examples/server/sync-sleep-pooled.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ fastify.register(require('fastify-piscina'), {
});

// Declare a route
fastify.get('/', () => fastify.runTask());
fastify.get('/', () => fastify.run());

// Run the server!
const start = async () => {
Expand Down
Loading
Loading