Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout and retry logic to downloading Javy and function-runner #4384

Merged
merged 5 commits into from
Sep 13, 2024

Conversation

jeffcharles
Copy link
Contributor

@jeffcharles jeffcharles commented Aug 27, 2024

WHY are these changes introduced?

CI is sometimes marking tests around this logic as timing out. I am making an educated guess that it's the call to fetch that's taking more than 13 seconds to return because I don't know why ungzipping or running a subprocess's help command (the other two I/O operations) would take more than 13 seconds to complete. Adding an explicit timeout and retries to the downloading operation mean, if the downloads were to hang for whatever reason, we'd abort the download and retry 3 times. Hopefully this should be sufficient to work around this problem (assuming it is a result of the network request that's causing the timeout).

WHAT is this pull request doing?

When downloading the Javy or function-runner binaries, only allow up to 5 seconds for the download to complete and then retry for a given number of times.

I'm also having the CLI download Javy and function-runner to a temporary directory first and then move the file when it's finished being processed so retrying the command won't keep trying to load the same broken binary. This should also help if multiple things are trying to download Javy or function-runner in parallel.

I'm open to suggestions on if that 5 second timeout I'm imposing should be higher to not make life difficult for someone who has a really bad network connection but needs to download and install Javy or function-runner at that point in time. We can get at least one retry in if the timeout is under 12.5 seconds.

How to test your changes?

  1. rm packages/app/src/cli/services/bin/javy (if it exists)
  2. rm packages/app/dist/cli/services/bin/javy (if it exists)
  3. pnpm shopify app build --path <path_to_app_with_shopify_function> --verbose
  4. There should've been a verbose log saying it was downloading Javy
  5. packages/app/dist/cli/services/bin/javy should now exist

Repeat the above steps for running a function to test function-runner.

Measuring impact

How do we know this change was effective? Please choose one:

  • n/a - this doesn't need measurement, e.g. a linting rule or a bug-fix
  • Existing analytics will cater for this addition
  • PR includes analytics changes to measure impact

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes

Copy link
Contributor

Thanks for your contribution!

Depending on what you are working on, you may want to request a review from a Shopify team:

  • Themes: @shopify/advanced-edits
  • UI extensions: @shopify/ui-extensions-cli
    • Checkout UI extensions: @shopify/checkout-ui-extensions-api-stewardship
  • Hydrogen: @shopify/hydrogen
  • Other: @shopify/app-inner-loop

Copy link
Contributor

github-actions bot commented Aug 27, 2024

Coverage report

St.
Category Percentage Covered / Total
🟡 Statements
72.87% (+0.12% 🔼)
8315/11410
🟡 Branches
69.45% (+0.08% 🔼)
4060/5846
🟡 Functions
71.66% (+0.03% 🔼)
2167/3024
🟡 Lines
73.22% (+0.13% 🔼)
7866/10743

Test suite run success

1870 tests passing in 849 suites.

Report generated by 🧪jest coverage report action from 65cb711

@jeffcharles jeffcharles marked this pull request as ready for review August 27, 2024 15:14
@jeffcharles
Copy link
Contributor Author

I've also updated the PR to download the Javy and function-runner binaries to a temporary directory first and then move the binary after it's been fully downloaded and processed so re-running the installation command will retry if something interrupted the download.

@andrewhassan andrewhassan force-pushed the jc.add-timeout-and-retry-to-downloading-binaries branch from 1bd616f to 64829d2 Compare September 4, 2024 13:07
@andrewhassan andrewhassan force-pushed the jc.add-timeout-and-retry-to-downloading-binaries branch from 64829d2 to 657836b Compare September 13, 2024 14:36
Copy link
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/public/common/retry.d.ts
@@ -10,6 +10,7 @@
  * @param performAction - The action to perform.
  * @param recoveryProcedure - The recovery procedure to perform if the action
  * fails the first time.
+ * @param retries - The number of times to retry the action if an error happens.
  * @returns The result of the action.
  */
-export declare function performActionWithRetryAfterRecovery<T>(performAction: () => Promise<T>, recoveryProcedure: () => Promise<unknown>): Promise<T>;
\ No newline at end of file
+export declare function performActionWithRetryAfterRecovery<T>(performAction: () => Promise<T>, recoveryProcedure: () => Promise<unknown>, retries?: number): Promise<T>;
\ No newline at end of file

@andrewhassan andrewhassan requested review from andrewhassan and removed request for andrewhassan September 13, 2024 15:51
@andrewhassan andrewhassan added this pull request to the merge queue Sep 13, 2024
Merged via the queue into main with commit 1d9b883 Sep 13, 2024
36 checks passed
@andrewhassan andrewhassan deleted the jc.add-timeout-and-retry-to-downloading-binaries branch September 13, 2024 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants