pkg> st fails second time around with ERROR: MethodError: no method matching Int64(::Dates.DateTime) #4017
Comments
|
Note segfault when trying similar (why not in my original report?), just opened another tab and Julia, again verbatim: |
This comment was marked as duplicate.
This comment was marked as duplicate.
|
This doesn't happen in rc2, so I likely messed up rc3 with my Markdown change. Even if I undo that change of rc3, or try...,, it doesn't seem to take, i.e. I still get the error. I guess the Markdown precompile statements can't be undone? Precompilation is triggered when I added them, and I think should happen each time there's a change, but I'm using a stale pkgimage? Why isn't it updated if I change the source? |
|
@cdoris, are you doing something unusual in CondaPkg, with Julia's internals? You change Pkg, or the REPL mode, not sure if it's disallowed, or not guaranteed to work in later Julia versions. Note this works in my rc2 (where I've not changed Markdown). Also in rc3 works with Markdown alone (that I changed and may not have undone the right way, so can someone reproduce this error with an unchanged rc3), but not with otherwise unchanged CondaPkg: |
|
I think this is a TOML issue. Could Pkg being loaded via @topolarity given you worked on some of the TOML stuff here. |
|
yes, you can have any number of different TOML packages loaded, and unless you depend specifically upon a TOML package, you should not assume anything about it being present |
|
Ok. I've not followed the recent TOML changes in Pkg so I defer to @topolarity I think? |
|
Pkg does depend on both TOML and Dates, and makes sure to load the Dates-enabled TOML parser always. I'm a bit skeptical that we changed anything there - I think it's always been that way. We read As far as I can tell that's actually supposed to be hitting https://github.com/JuliaLang/julia/blob/1463c9991c4a75730e4d6822313180ca323241c0/stdlib/Dates/src/conversions.jl#L20 I'm not sure why it's not hitting that method - that doesn't seem related to TOML to me. |
|
My guess is that there must be two the first |
|
Oh my - so |
|
That seems terribly broken to me. How is Julia allowed to break the identity of a common dependency like that? |
|
I haven't confirmed my suspicion but that is the only way I can imagine hitting that dispatch... |
|
It is permitted, but only in certain configurations (e.g. where there is no dependencies between them, but only an assumption it will exist because of some transitive assumption of being in the other's project files) |
|
Is that defined precisely somewhere? It's not clear to me whether that would include this situation or not... |
|
@PallHaraldsson I'm having trouble reproducing this issue on my end. Can you try to come up with an MWE that works out-of-the-box with a fresh depot (i.e. using |
|
[Probably worth noting is I often do CTRL-C while precompiling packages to avoid OOM... so far always ok, or at best a segfualt and ok after a restart. I'm not sure I did CTRL-C in that case (I even think I didn't), but might have and very probably something got inconsistent in compiled code? I think CTRL-C is supported, or at least is should be... and if not I would like to know and how to then fix after such issues.] I haven't seen this again myself... And maybe CTRL-C is worse if you're deving packages? And deving further fixed things? I'm not sure I can do an MWE, I basically just added some precompiles and/or deleted stuff from PythonCall if I recall. This isn't supposed to happen, obviously, and even if I deved PythonCall to trigger this I've changed it since, but I do have some julias open where this happened, if there's anything helpful with knowing that, I could do there. The problem didn't go away there, I checked in one julia, nor did I expect that... all in RAM. I don't know about JULIA_DEPOT_PATH really, not something I've used, if you can tell me exactly what to do then maybe I can be helpful. This was just so strange to me, that I though opening an issue worthy... even if I can't do more. Though willing. |
|
@topolarity I think we want the trailing |
|
@PallHaraldsson The idea is that if you start Julia like this: $ export _JULIA_BUILTIN_DEPOT="$(dirname $(julia +1.10 -e "println(Sys.BINDIR)"))/local/share/julia:$(dirname $(julia +1.10 -e "println(Sys.BINDIR)"))/share/julia"
$ JULIA_DEPOT_PATH="./temp_depot:${_JULIA_BUILTIN_DEPOT}" julia +1.10That will force Julia to run using a "fresh" registry, compile cache, etc. in "./temp_depot" If you can show a how to create the problem (from start to finish) after starting Julia as above, that means that it isn't affected by any corrupted cache files, etc. and it should be easier for us to reproduce. After you're done, you can just |
|
Hi, if it's needed I think I've got another MWE in JuliaLang/julia#55887? There, with an additional |
|
Ugh, unfortunately that still doesn't reproduce for me. Can you try re-producing with |
|
Sorry, I'm not sure what the Windows equivalent of those commands are. I also realized I can't reproduce that on Fedora... |
|
Oh, on Windows? Let me see... The powershell equivalent looks like this: $_JULIA_BUILTIN_DEPOT = "$(Split-Path -parent $(julia +1.11 -e "println(Sys.BINDIR)"))\local\share\julia;$(Split-Path -parent $(julia +1.11 -e "println(Sys.BINDIR)"))\share\julia"
$env:JULIA_DEPOT_PATH=".\temp_depot;${_JULIA_BUILTIN_DEPOT}"; julia +1.11; $env:JULIA_DEPOT_PATH=$nullwhich will run using a temporary depot in |
|
oops, can't reproduce with a temporary depot too... |
|
OK, at least we know there's something going on with the cache. Can you try running with: $env:JULIA_DEBUG="loading"; julia +1.11; $env:JULIA_DEBUG=$nulland share the full output here? (this will run with the default depot, so the problem should still reproduce) |
|
Ok, I'm not sure if this is what you're asking for, but I can reliably reproduce the error with the below steps, so here goes:
|
|
Woah! Thanks for working so hard to get that MWE, that is extremely helpful. I can reproduce on my machine with those steps now |
|
Oof, we can even end up loading two copies of Pkg. One that works and one that doesn't... (@v1.11) pkg> status
Status `C:\Users\cody\temp_depot\environments\v1.11\Project.toml`
[fa267f1f] TOML v1.0.3
julia> using TOML
julia> using Pkg # loads a *second* copy of Pkg
[ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] (cache misses: wrong dep version loaded (2), mismatched flags (2))
julia> Pkg.status() # the new copy works!
Status `C:\Users\cody\temp_depot\environments\v1.11\Project.toml`
[fa267f1f] TOML v1.0.3
(@v1.11) pkg> status # the old one is still broken
ERROR: MethodError: no method matching Int64(::Dates.DateTime)
The type `Int64` exists, but no method is defined for this combination of argument types when trying to construct it. |
|
How is the old Pkg getting broken by the loading of later packages? That seems like it should only be possible if one of those packages is doing some invalid hacks around accessing private APIs such as Base.loaded_modules or Base.get_extension |
|
Turns out this is a method overload issue - @KristofferC 's spidey senses were tingling for good reason when he pointed out this type piracy The problem is that This specific problem is easy to fix, but it also makes me think more broadly about the correct-ness requirements here. I guess this means that |
Since stdlibs can be duplicated but Base never is, `Base.require_stdlib` makes type piracy even more complicated than it normally would be. To adapt, this changes `TOML.Parser` to be a type defined by the TOML stdlib, so that we can define methods on it without committing type-piracy and avoid problems like Pkg.jl#4017 Resolves JuliaLang/Pkg.jl#4017 (comment) (cherry picked from commit 2a2878c)
…#56005) I was a recent offender in JuliaLang/Pkg.jl#4017 (comment) This PR tries to lay down some guidelines for the behavior that stdlibs and the callers of `require_stdlib` must adhere to to avoid "duplicate stdlib" bugs These bugs are particularly nasty because they are experienced semi-rarely and under pretty specific circumstances (they only occur when `require_stdlib` loads another copy of a stdlib, often in a particular order and/or with a particular state of your pre-compile / loading cache) so they may make it a long way through a pre-release cycle without an actionable bug report.
This is a very strange bug to me, and likely has nothing to do with PythonCall, though seemingly the only thing triggering it, showing verbatim:
Yes, I'm deving the package (and Markdown its dependency, seemingly successfully, just added precompile statements there), but I didn't really change PythonCall, some very minor changes, besides they shouldn't make this happen?!
The text was updated successfully, but these errors were encountered: