Jupyter for vscode continues to be slow (for large notebooks with mardown cells & large outputs) #14459
Comments
|
Thank you for filling this issue and sorry you are running into these issues I would like to get to the bottom of these issues and get you unblocked
Please could you enable logging as follows:
|
|
I have exactly the same issues. The notebooks get especially slow as they get bigger. But many of the problems already exist in an empty notebook. |
|
Same here as @dschaub95 . Could you @DonJayamanne upload a screenshot of Jupyter output panel? I have nothing here. This issue did not present couple of versions of vscode before. It is a recent versions thing. |
|
Okay, I found Jupyter output panel. |
|
@jhancibo @dschaub95 @loftusa |
|
In my experience, it is entirely independent of the variable view. |
|
If you have the Jupyter powertoys extension |
|
I don't use the powertoys extension at all. Maybe it's also important to mention that the problems with jupyter notebooks are even more severe when developing on a remote server (via ssh or Kubernetes). However, they still persist when developing locally. |
|
@dschaub95
please share these logs when you run into perf issues |
|
The output is attached. As far as I could see, the output only changed when the cell started executing. The time between me trying to execute and the actual execution seems not to be logged. |
Same here. And I didn't install powertoys extension. |
|
@DonJayamanne The issue persists after close the variable view completely. |
|
@dschaub95 @jhancibo
Once again thank you for your continued patience ands support |
|
@DonJayamanne Could you recording a video to do those instruction above? I tried and it is abstract to follow each step. For example, when I type Developer: Set log level in command palette, I see nothing pop up. |
Please send the logs from @dschaub95 @jhancibo
Please let me know
Here are the instructions again (please clear the both
|
|
Closing this issue as its been over 4 weeks, since the information was requested. |
|
got one example and the logs are in microsoft/pylance-release#5301 (Note that I'm not disabling all other extensions).
UPDATE: having the same issue again, but nothing in Extension Host (in info level). the pending lasted for about one minutes. I tried it again with debug level for Extension Host, the only logs (from middle of pending to after execution) are as follows. (seems unrelated?) meanwhile, the time shown below is incorrect - the waiting time is more than one minute. and again, pls note that I'm not disabling other extensions during work, which is not consistent with your instruction above. |
|
Hi, i think this issue should not be closed because it is not solved. Or does someone have a solution? When I run notebooks in jupyter lab in the browser everything is instant but in vscode everything runs delayed. |
|
Not sure if this is what you're seeing, but I've noticed a regression of an old bug. I have a code cell that should run in a fraction of a second. I run it. It's stuck for about 1 minute. Then all of a sudden it runs. Very annoying. Because of this and other, numerous bugs, I'm thinking to go back to Jupyter Notebook in a browser. |
|
I don't know if Windows interferes the "vscode ipykernel" differently than the "kernel type" that is started by running jupyter lab. Perhaps there are some Windows processes like Windows Defender and proxy and vpn settings that interfere with the ipykernel when starting a kernel from vscode. Here is an issue that describes some of these things: |
|
I am experiencing the same issue. A few days ago, everything was working fine but after updating to the latest version, Visual Studio Code is using almost 90% of my CPU and the Jupyter Notebook is also running slow. Previously, I could use 3-4 notebooks at a time with my current setup but now running even one notebook is not possible. On the other hand, everything is smooth in Jupyter Lab. "I have attached the cell loading data. It has been stuck for 5 minutes now and is not running."
|
|
In my experience, jupyter notebooks performance degrades very quickly in the size of the notebook. This is especially true for plotly.express plots, and is independent of whether I am using a .ipynb file or the interactive cell views for a .py file. Describing the experience for a .py file: when there are no plots and no LaTeX in the interactive window, everything is snappy. But if I have even just a handful of plots (or many lines of rendered LaTeX from Markdown cells), then it takes multiple seconds between when I press Shift+Enter and when the interactive window starts running the command. If I click "clear all", everything is quick again. This seems to largely depend on how many plots are in the interactive window, not how many are currently visible. Some other observations:
|
|
@JasonGross All that is true, but the bug where it gets stuck on a cell does not depend on notebook size or plot complexity. |
|
It used to be the case that clearing the interactive window would restore performance, but now it seems like slowness accumulates over time even when the interactive window has been cleared --- this is so much worse than the non-pre-release/non-insider-preview state!
Clicking "Clear all" should be instantaneous, it should not take multiple seconds after clicking "run cell" to have the text show up in the interactive window, and it should not take 5+ seconds to start running the code. Profiling the clear all -> shift+enter -> wait for running to complete: am I reading this right that most of the time is not spent in extensions?
|
Are you saying that this is fast in VS Code insiders, and slow in VS Code Stable?
Yes, it looks like the extensions are fine, at least from the logs
Please try the following:
Unfortunately saving the logs is broken in VS Code, hence we will have to resort to capturing screenshots. Below is a screenshot (for reference) of the
|
|
Slightly different issue, shift+enter on some code in the interactive window takes a second or two before it starts running the code. (This time if I clear the interactive window first, it becomes instantaneous; there are a bunch of plots hidden above where I'm scrolled to.) CPU-20240426T065336.682Z.cpuprofile.txt here too is seems that not much happens in extensions
|
No, it is slow in VS Code Insiders, fast in VS Code Stable.
I did not disable extensions, but it should be fine because the extension profile says that no time is spent in extensions other than ssh, right?
Yes, captures were performed while recreating the events shown in the gif. |
|
I have captured a profile from repeating the test in #14459 (comment) a handful of times, over the course of 60 seconds. However, the performance capture seems to be broken, there are no details provided |
What is slow, editing, executing notebooks? Please can you try this with the following extension
|
Okay, I did it for only 40s instead of 60s, here's more captures: total time over 100ms
expanded
I hope you find this more informative than I do. But I can't help but think that this profile doesn't contain the relevant information---I'm waiting 40 seconds for an operation that should only take 4 seconds (or less), and yet this profile only claims to capture 1256ms/.266 = 4.7 seconds. Where are the other 35s spent? |
Executing. I have given up on using The problem is that while text is dumped into the interactive window immediately, VS Code spends an excessive amount of time (sometimes upwards of 5 seconds for one-liners like You can see the behavior in the gifs I've uploaded.
No, I haven't done this. I'll do this next. But why does the CPU profile show that no time is spent in extensions? |
|
I've disabled all extensions except jupyter, python, remote ssh, and wsl extensions
I've established that the problem persists even when I've disabled all extensions except for the ones shown above. |
|
Oh no, I'm so sorry, please can you sort the table is descending order, |
|
Luckily I still have it open, I can resort. I don't suppose there's a quick way to automatically expand everything above 100ms? Also, I think there must be a bug in the profiling code, there's an instance where the total time of a parent node is less than half the total time of it's only child. What might cause this? |
|
Pleaser can you take a screen recording. |
|
I will try to find a way to get the logs from you |
|
Sorry it took so long, the autoscrolling screen capture kept crashing. Apparently it's too much for a single image. resorted expanded profile
high self time
|
|
Hm, it seems to have the performance issue occasionally even with all extensions disabled, but has the performance issue way more reliably with copilot enabled. Is it possible that copilot is blocking cell execution on sending relevant data to ChatGPT or something, but this somehow doesn't show up in any profile because the time is spent in low-level system calls or something? Is there a way to debug this? |
|
Highlighting this in case it got lost:
@DonJayamanne do you have any idea what would cause almost 90% of the time to not show up in the profile at all? |
|
@DonJayamanne For now, thanks a ton for all the help and work! |
|
I've had the same issues as others. I'm working with a particularly large dataset (50 GB in RAM). Turning on Github Copilot completions makes it take >60 seconds to start to run a cell; if I turn off completions, there is no delay at all. I'm running VS Code stable (not insiders) with pylance installed and running. Definitely seems like a Copilot issue (for me at least). |
|
@tlkaufmann @Liam3851 |
|
Is the issue that Jupyter (or VSCode) is exposing some sort of blocking hook on cell execution / send text to the interactive window, Copilot takes advantage of this hook to process the entirety of the notebook / interactive window with an LLM, and blocks the execution of the code until it gets a response? (Also, did "Clear All" become less destructive in the insider's preview / pre-release version of Jupyter? Is Copilot somehow getting access to all the cells that have been executed, even the cleared ones?) |
Can also confirm, disabling Copilot entirely (autocompletion setting didnt make faster on its own) was the answer to much faster performance including shift-enter, decreased lag when entering text, and much faster built-in autocompletions. When/How will we know when the copilot workaround has been resolved? Remote Configuration: MacOS - remotessh - Linux machine vscode detailsVersion: 1.88.1 Commit: e170252f762678dec6ca2cc69aba1570769a5d39 Date: 2024-04-10T17:42:52.765Z Electron: 28.2.8 ElectronBuildId: 27744544 Chromium: 120.0.6099.291 Node.js: 18.18.2 V8: 12.0.267.19-electron.0 OS: Darwin x64 23.4.0
(2 theme extensions excluded) |
|
I'm facing the same issue and I don't even use copilot (haven't installed it). |
|
@SoshyHayami Please can you file a separate issue for that, and please can you provide a sample notebook with all of the necessary dependencies to repro the issue (perhaps a requirements.txt or conda yml file to setup the python envirolnment). |
|
This issue also seems to occur with Microsoft's IntelliCode, so it seems like it might be common to LLM-based code completion extensions |
|
I'm using neither IntelliCode nor copilot, but I do use codeium, so perhaps @JasonGross is on the right track here |
@JasonGross Do you mean that performance is back to normal when IntelliCode is disabled? |
|
For the copilot relevant issues, we are working with Copilot team for a fix and will keep you posted. |
|
I'm going to close this issue since we've taken care of two of the major causes of poor performance and this thread has gotten really long and difficult to navigate. Please use a more specific issue for any other performance issues or file a separate issue with the specific poor performance patterns that you are experiencing. CPU profiles are usually very helpful if it is something new: We'll use microsoft/vscode#211154 to track the issue with copilot from our end. A big thanks to everyone that helped gather diagnostics and help narrow down the issues! |
and others
Every few months I try to use vscode for jupyter because I would really love to just use vscode for everything. Every few months, I am disappointed and switch back to the web version.
There are two reasons for this:
1) Jupyter for vscode continues, stubbornly, to essentially always be more slow than traditional jupyter lab on localhost. Look at the run times in this screenshot. It took me a minute to run imports; when I ran the exact same code on the localhost version, it took 7.7 seconds (pictures attached). This is an extremely consistent theme in vscode jupyter. Cells will sometimes randomly take minutes to run, and will sometimes not even run at all until you press 'shift-enter' on them twice. This has been true for me across multiple computers, in many different dev environments.
Cells also just randomly take forever to run, for god knows what reason. Here is a screenshot of assigning a string to a variable taking 27.4 seconds:
Note that I am not trying to blame the team here, I am just frustrated because this is so close to being a great product, but this one thing holds it back, and it keeps not being fixed for years on end. The very first thing I would do as a product manager if I were in charge of vscode-jupyter is to pause all current tasks and plan, with the team, a multiple-month effort to speed things up, and get cells to run effectively instantly (or as close to the amount of time the python processing of the code takes as possible), every time.
2) Jupyter for vscode sucks at inline documentation, the equivalent of
shift+tabin vscode jupyter. I am aware of the existence of thetrigger parameter hintsandshow hoversettings in the keyboard shortcuts. These are extremely unreliable, and actually show documentation when I press the button maybe 1/5 of the time. When they do show documentation, there is a 'loading' tag for awhile. Browser jupyter, on the other hand, is immediate with this. Basically every time. Below is an example.The other issue with inline documentation is that, as far as I can tell, hover documentation for methods on instantiated variables simply doesn't work. When I am using
pandas, for instance, typingdf.unique(and then pressing theshow hoverhotkey while my typing carat is to the right of the parenthesis pops up a documentation window saying exactly nothing. In contrast, in the web version, typing the same thing produces full documentation, as expected.I don't understand how these two issues aren't your guys's top priority. Everyone I've spoken to who uses jupyter has had exactly the same experience as I have, and everyone I've spoken to who uses jupyter uses the web version exclusively for exactly these issues. Even Kaggle notebooks are better. I love copilot and it'd be great to bring it into my jupyter notebook experience, but it has just never been viable to switch if I don't want a workflow where I have to wait for 30 seconds every time I press
command-enter, or I am frustratingly making a new cell above the current one and typingfunction?just to see documentation.These issues have been ongoing since vscode jupyter started. They are the only things holding me and everyone else I've spoken to back from using it. Without fixing these issues, the whole thing is unusable, and no other features you guys put in matter. Why are you guys working on anything besides this when they are the only things anyone I know cares about?
I should note that this is all running in a docker container with access to 7 of my 8 cpus and 10gb of RAM. I am on a 2022 macbook air. I realize that this is a rant, so thank you for reading it. Nothing personal, I just think this product has a bunch of potential and I hate to see it unusable for so long.
The text was updated successfully, but these errors were encountered: