Add project hash to manifest metadata and warn when manifest and project are out of sync #2815

IanButterworth · 2021-11-01T03:59:25Z

One of the reasons I was a fan of adding extensible metadata to the manifest was to be able to save a hash of the project state in the manifest, to make it easy to identify when the manifest and project are out of sync. i.e. a project dep entry has been manually added/removed in the toml file, or a compat entry added/changed/removed

Note that the project hash used here only hashes the deps and compat fields of the project, the rest shouldn't affect a resolve, so should be free to change.

One thing this would allow is CI to easily check whether checked-in manifests are valid, as it's easy to forget to update them.
Currently Pkg.instantiate doesn't check compat bounds and just does what the manifest says. This adds a warning.

Todo:

Add tests

julia> import Pkg
[ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b79f]

(Pkg) pkg> activate --temp
  Activating new project at `/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_2QJmtW`

(jl_2QJmtW) pkg> add CSV
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `/private/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_2QJmtW/Project.toml`
  [336ed68f] + CSV v0.9.10
    Updating `/private/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_2QJmtW/Manifest.toml`
  [336ed68f] + CSV v0.9.10
...

(jl_2QJmtW) pkg> compat CSV 0.8
      Compat entry set:
  CSV = "0.8"

(jl_2QJmtW) pkg> instantiate
┌ Warning: The project hash recorded into the manifest when it was resolved does not match the hash of the current project.
│ A project dependency has been added/removed or compat entry has changed. The environment may need to be updated
└ @ Pkg.API ~/Documents/GitHub/Pkg.jl/src/API.jl:1509

(jl_2QJmtW) pkg> up
    Updating registry at `~/.julia/registries/General.toml`
    Updating `/private/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_2QJmtW/Project.toml`
  [336ed68f] ↓ CSV v0.9.10 ⇒ v0.8.5
    Updating `/private/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_2QJmtW/Manifest.toml`
  [336ed68f] ↓ CSV v0.9.10 ⇒ v0.8.5
  [944b1d66] - CodecZlib v0.7.0
  [48062228] - FilePathsBase v0.9.13
  [842dd82b] - InlineStrings v1.0.1
  [69de0a69] ↓ Parsers v2.1.1 ⇒ v1.1.2
  [3bb67fe8] - TranscodingStreams v0.9.6
  [ea10d353] - WeakRefStrings v1.4.1
  [cf7118a7] - UUIDs
  [83775a58] - Zlib_jll v1.2.12+1

(jl_2QJmtW) pkg> instantiate

(jl_2QJmtW) pkg> compat CSV 0.9
      Compat entry set:
  CSV = "0.9"

(jl_2QJmtW) pkg> instantiate
┌ Warning: The project hash recorded into the manifest when it was resolved does not match the hash of the current project.
│ A project dependency has been added/removed or compat entry has changed. The environment may need to be updated
└ @ Pkg.API ~/Documents/GitHub/Pkg.jl/src/API.jl:1509

(jl_2QJmtW) pkg> rm CSV
    Updating `/private/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_2QJmtW/Project.toml`
  [336ed68f] - CSV v0.8.5
    Updating `/private/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_2QJmtW/Manifest.toml`
  [336ed68f] - CSV v0.8.5
...

(jl_2QJmtW) pkg> instantiate

julia> Pkg.is_manifest_current()
true

(jl_2QJmtW) pkg> add CSV
...

(jl_2QJmtW) pkg> compat CSV 0.8
      Compat entry set:
  CSV = "0.8"

julia> Pkg.is_manifest_current()
false

(jl_2QJmtW) pkg> instantiate
┌ Warning: The project hash recorded into the manifest when it was resolved does not match the hash of the current project.
│ A project dependency has been added/removed or compat entry has changed. The environment may need to be updated
└ @ Pkg.API ~/Documents/GitHub/Pkg.jl/src/API.jl:1509

also add is_manifest_current check to instantiate

DilumAluthge · 2021-11-01T04:01:42Z

One thing this would allow is CI to easily check whether checked-in manifests are valid, as it's easy to forget to update them.

This is a good use case for this PR. Although, I am guessing that most people don't check the CI logs unless the CI job fails, so maybe a warning isn't sufficient for CI? We could do something fancy like check for the presence of the CI environment variable, and if we are running on CI, we throw a fatal error instead of just printing a warning.

IanButterworth · 2021-11-01T04:05:04Z

Yeah, that's why I exposed Pkg.is_manifest_current() as part of the API. So you can do

@test Pkg.is_manifest_current()

but that requires people to add that to their tests.

Detecting CI and throwing sounds good to me.. but I'm not sure whether people will be happy with that

src/Types.jl

src/Pkg.jl

IanButterworth · 2021-11-01T16:01:30Z

Now with a warning in Pkg.status as well

(jl_hHdFrB) pkg> st
Status `/private/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_hHdFrB/Project.toml`
  [336ed68f] CSV v0.9.10
Warning The project and manifest may be out of sync. The project hash recorded into the manifest when it was resolved does not match the hash of the current project. A project dependency has been added/removed or compat entry has changed. The environment may need to be updated

(jl_hHdFrB) pkg> instantiate
┌ Warning: The project and manifest may be out of sync. The project hash recorded into the manifest when it was resolved does not match the hash of the current project. A project dependency has been added/removed or compat entry has changed. The environment may need to be updated
└ @ Pkg.API ~/Documents/GitHub/Pkg.jl/src/API.jl:1510

IanButterworth · 2021-11-02T03:53:51Z

Perhaps the warning message could be shorter. This might do

Warning: The project and manifest may be out of sync. A project dependency has been added/removed or compat entry has changed since the manifest was last resolved. The environment may need to be updated.

KristofferC · 2021-11-03T08:35:40Z

Just to point out another way where things can get "out of date" is if you dev a package, and then you work on the deved package and i.e. add a dependency, the original project that deved the package will now have the wrong dependency list in its Manifest and will fail to load the newly added dependency of the deved package.

IanButterworth · 2021-11-03T12:47:04Z

Yeah. That issue's covered at package load time by the x doesn't have y in its dependencies error, I guess?

The issue this tries to overcome is having a manifest which is a false resolve of a project, and instantiate doing as it's told and instantiating the manifest without warning, then the package loading without error, but during runtime hitting some different runtime behavior in a dep because that dep compat narrowed in the project but wasn't re-resolved.

That's a highly possible failure mode of projects with checked in manifests, and this PR would make it possible to check for that during PRs

src/Types.jl

src/Operations.jl

src/Types.jl

Co-authored-by: Fredrik Ekre <[email protected]>

GunnarFarneback · 2023-11-20T15:04:32Z

Yeah, that's why I exposed Pkg.is_manifest_current() as part of the API. So you can do
@test Pkg.is_manifest_current()

Is that actually working? I rather get the impression that it tests if the manifest in the temporary test environment is current, which it kind of trivially is.

julia> using Pkg

julia> Pkg.generate("HashTest")
  Generating  project HashTest:
    HashTest/Project.toml
    HashTest/src/HashTest.jl
Dict{String, Base.UUID} with 1 entry:
  "HashTest" => UUID("21f82d62-6511-4653-97ca-6e7cc2914814")

julia> mkdir("HashTest/test")
"HashTest/test"

julia> write("HashTest/test/runtests.jl",
             """
             using Test, HashTest, Pkg
             @test Pkg.is_manifest_current()
             """)
58

julia> write("HashTest/Project.toml",
             """
             name = "HashTest"
             uuid = "21f82d62-6511-4653-97ca-6e7cc2914814"
             version = "0.1.0"

             [deps]

             [extras]
             Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
             Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

             [targets]
             test = ["Pkg", "Test"]
             """)
225

julia> write("HashTest/Manifest.toml",
             """
             julia_version = "1.9.3"
             manifest_format = "2.0"
             project_hash = "0000000000000000000000000000000000000000"
             """)
106

julia> Pkg.activate("HashTest")
  Activating project at `/tmp/HashTest`

julia> Pkg.test()
┌ Warning: The project dependencies or compat requirements have changed since the manifest was last resolved.
│ It is recommended to `Pkg.resolve()` or consider `Pkg.update()` if necessary.
└ @ Pkg.API ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/Pkg/src/API.jl:1687
     Testing HashTest
      Status `/tmp/jl_Gy6PhH/Project.toml`
  [21f82d62] HashTest v0.1.0 `/tmp/HashTest`
  [44cfe95a] Pkg v1.9.2 `@stdlib/Pkg`
  [8dfed614] Test `@stdlib/Test`
      Status `/tmp/jl_Gy6PhH/Manifest.toml`
  [21f82d62] HashTest v0.1.0 `/tmp/HashTest`
  [0dad84c5] ArgTools v1.1.1 `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [f43a241f] Downloads v1.6.0 `@stdlib/Downloads`
  [7b1f6079] FileWatching `@stdlib/FileWatching`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [b27032c2] LibCURL v0.6.3 `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [ca575930] NetworkOptions v1.2.0 `@stdlib/NetworkOptions`
  [44cfe95a] Pkg v1.9.2 `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA v0.7.0 `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [fa267f1f] TOML v1.0.3 `@stdlib/TOML`
  [a4e569a6] Tar v1.10.0 `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [deac9b47] LibCURL_jll v7.84.0+0 `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll v1.10.2+0 `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll v2.28.2+0 `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll v2022.10.11 `@stdlib/MozillaCACerts_jll`
  [83775a58] Zlib_jll v1.2.13+0 `@stdlib/Zlib_jll`
  [8e850ede] nghttp2_jll v1.48.0+0 `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll v17.4.0+0 `@stdlib/p7zip_jll`
Precompiling project...
  1 dependency successfully precompiled in 0 seconds. 3 already precompiled.
     Testing Running tests...
     Testing HashTest tests passed

GunnarFarneback · 2023-11-20T15:09:06Z

Btw, I think

@test Pkg.is_manifest_current(dirname(@__DIR__))

would be an acceptable API.

IanButterworth · 2023-11-20T15:16:56Z

Oh you're right.. yes, not having a default arg would be a reasonable bug fix I think.

GunnarFarneback · 2023-11-20T15:27:36Z

If you just remove the default argument all you have left is is_manifest_current(::Context), which doesn't seem very user friendly?

IanButterworth · 2023-11-20T15:37:23Z

Yeah, I hadn't remembered the type was ::Context.

Perhaps leave that method, but without a default, and add one that takes a path.

Your suggestion seems a reasonable, although this might be better usage to promote in the docs?

@test Pkg.is_manifest_current(pkgdir(Foo))

IanButterworth · 2023-11-20T16:50:30Z

Maybe we could leave the default arg for the Context method but error if it is run on an unnamed Project.toml and point people to the correct form? That might be nicer as a bugfix

GunnarFarneback · 2023-11-20T19:09:09Z

I don't have a strong opinion about it, but to me it seems useful to be able to verify the manifest-up-to-dateness for any environment, not only packages.

IanButterworth · 2023-11-20T19:11:50Z

True

GunnarFarneback · 2023-11-20T21:40:58Z

See how you like #3701 and if something needs to be changed. It would probably be good with some additional and/or revised tests but I'll wait with that until we are happy with the design changes.

IanButterworth added 2 commits October 31, 2021 23:48

add project hash to manifest metadata. add is_manifest_current

09e0dc5

also add is_manifest_current check to instantiate

precompile: don't instantiate again after an instantiate

ad7667f

fredrikekre reviewed Nov 1, 2021

View reviewed changes

src/Types.jl Outdated Show resolved Hide resolved

fredrikekre reviewed Nov 1, 2021

View reviewed changes

src/Pkg.jl Show resolved Hide resolved

use SHA1

2681963

IanButterworth force-pushed the ib/manifest_project_hash_meta branch from 523fd4f to 2681963 Compare November 1, 2021 15:43

add warning to Pkg.status

818ed58

IanButterworth added 2 commits November 1, 2021 12:19

fix imports

cd1b92c

add tests

d8d542b

IanButterworth requested review from KristofferC and fredrikekre November 2, 2021 02:16

fredrikekre reviewed Nov 3, 2021

View reviewed changes

src/Types.jl Outdated Show resolved Hide resolved

fredrikekre reviewed Nov 3, 2021

View reviewed changes

src/Types.jl Outdated Show resolved Hide resolved

KristofferC reviewed Nov 3, 2021

View reviewed changes

src/Operations.jl Outdated Show resolved Hide resolved

KristofferC reviewed Nov 3, 2021

View reviewed changes

src/Operations.jl Outdated Show resolved Hide resolved

KristofferC reviewed Nov 3, 2021

View reviewed changes

src/Types.jl Outdated Show resolved Hide resolved

IanButterworth and others added 6 commits November 3, 2021 10:52

Apply suggestions from code review

2ba7095

Co-authored-by: Fredrik Ekre <[email protected]>

update warning message

7c4241d

fix ios

1e161ff

better message

bb6b438

Update CHANGELOG.md

af82fea

Update CHANGELOG.md

d24f0fb

IanButterworth merged commit b963d05 into JuliaLang:master Nov 11, 2021

IanButterworth deleted the ib/manifest_project_hash_meta branch November 11, 2021 17:44

This was referenced Dec 30, 2021

Remove precompile resolve #2905

Merged

Resolve after setting compat via Pkg.compat #2907

Merged

GunnarFarneback mentioned this pull request Nov 20, 2023

Revise the API of is_manifest_current. #3701

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add project hash to manifest metadata and warn when manifest and project are out of sync #2815

Add project hash to manifest metadata and warn when manifest and project are out of sync #2815

IanButterworth commented Nov 1, 2021 •

edited

Loading

DilumAluthge commented Nov 1, 2021

IanButterworth commented Nov 1, 2021

IanButterworth commented Nov 1, 2021

IanButterworth commented Nov 2, 2021

KristofferC commented Nov 3, 2021

IanButterworth commented Nov 3, 2021 •

edited

Loading

GunnarFarneback commented Nov 20, 2023

GunnarFarneback commented Nov 20, 2023

IanButterworth commented Nov 20, 2023

GunnarFarneback commented Nov 20, 2023

IanButterworth commented Nov 20, 2023

IanButterworth commented Nov 20, 2023

GunnarFarneback commented Nov 20, 2023

IanButterworth commented Nov 20, 2023

GunnarFarneback commented Nov 20, 2023

Add project hash to manifest metadata and warn when manifest and project are out of sync #2815

Add project hash to manifest metadata and warn when manifest and project are out of sync #2815

Conversation

IanButterworth commented Nov 1, 2021 • edited Loading

DilumAluthge commented Nov 1, 2021

IanButterworth commented Nov 1, 2021

IanButterworth commented Nov 1, 2021

IanButterworth commented Nov 2, 2021

KristofferC commented Nov 3, 2021

IanButterworth commented Nov 3, 2021 • edited Loading

GunnarFarneback commented Nov 20, 2023

GunnarFarneback commented Nov 20, 2023

IanButterworth commented Nov 20, 2023

GunnarFarneback commented Nov 20, 2023

IanButterworth commented Nov 20, 2023

IanButterworth commented Nov 20, 2023

GunnarFarneback commented Nov 20, 2023

IanButterworth commented Nov 20, 2023

GunnarFarneback commented Nov 20, 2023

IanButterworth commented Nov 1, 2021 •

edited

Loading

IanButterworth commented Nov 3, 2021 •

edited

Loading