Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid calling go_path rule in source mode #3121

Merged
merged 4 commits into from
Apr 18, 2022
Merged

Conversation

linzhp
Copy link
Contributor

@linzhp linzhp commented Apr 16, 2022

What type of PR is this?
Performance optimization. This is also a breaking change from original gomock rule implementation, because it changes the types of aux_files attribute.

What does this PR do? Why is it needed?
Running mockgen in source mode does not require all its dependencies in GOPATH. It only need the source file and some aux files. We can copy them to form a GOPATH structure without calling go_path rule.

Before this patch:

Critical path (1.199 s):
       Time Percentage   Description
    0.46 ms    0.04%   action 'Writing file mocks_gomock_gopath~manifest'
     250 ms   20.86%   action 'GoPath mocks_gomock_gopath'
     949 ms   79.10%   action 'Action mocks.go'

After this patch:

Critical path (891 ms):
       Time Percentage   Description
    24.1 ms    2.70%   action 'Action gopath/src/uber.com/gomock/client.go'
     867 ms   97.30%   action 'Action mocks.go'

The experiment can be reproduced in the following workspace:

  1. create a workspace with the following files:
-- BUILD.bazel --
load("@io_bazel_rules_go//go:def.bzl", "go_library")
load("@io_bazel_rules_go//extras:gomock.bzl", "gomock")
load("@bazel_gazelle//:def.bzl", "gazelle")

# gazelle:prefix uber.com/gomock
gazelle(name = "gazelle")

gazelle(
    name = "gazelle-update-repos",
    args = [
        "-from_file=go.mod",
        "-prune",
    ],
    command = "update-repos",
)

go_library(
    name = "client",
    srcs = ["client.go"],
    importpath = "uber.com/gomock",
    visibility = ["//visibility:public"],
    deps = [
        "@go_googleapis//google/bytestream:bytestream_go_proto",
        "@org_golang_google_grpc//:go_default_library",
    ],
)

# gazelle:exclude mocks.go
gomock(
    name = "mocks",
    out = "mocks.go",
    library = ":client",
    package = "gomock",
    source = "client.go",
    interfaces = ["Client"],
)
-- client.go --
package gomock

import (
	_ "github.com/mattn/go-sqlite3"
	_ "google.golang.org/genproto/googleapis/bytestream"
	_ "google.golang.org/grpc"
)

type Client interface {
}
-- go.mod --
module uber.com/gomock

go 1.18

require (
	github.com/mattn/go-sqlite3 v1.14.12
	google.golang.org/genproto v0.0.0-20220414192740-2d67ff6cf2b4
	google.golang.org/grpc v1.45.0
)

require (
	github.com/golang/protobuf v1.5.2 // indirect
	github.com/google/go-cmp v0.5.7 // indirect
	golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f // indirect
	golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8 // indirect
	golang.org/x/text v0.3.7 // indirect
	google.golang.org/protobuf v1.28.0 // indirect
)
-- WORKSPACE --
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "io_bazel_rules_go",
    sha256 = "6c3bc93b2f14db41c2ef53388700fb95cbf7005b7c94e98f5d8bca0748110062",
    strip_prefix = "rules_go-926c47abbb2dd059a4fcd3f810005167cc62ed1c",
    urls = [
        "https://github.com/bazelbuild/rules_go/archive/926c47abbb2dd059a4fcd3f810005167cc62ed1c.zip",
    ],
)

http_archive(
    name = "bazel_gazelle",
    sha256 = "de69a09dc70417580aabf20a28619bb3ef60d038470c7cf8442fafcf627c21cb",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.24.0/bazel-gazelle-v0.24.0.tar.gz",
        "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.24.0/bazel-gazelle-v0.24.0.tar.gz",
    ],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")

go_rules_dependencies()

go_register_toolchains(version = "1.18")

gazelle_dependencies()

http_archive(
    name = "com_google_protobuf",
    sha256 = "d0f5f605d0d656007ce6c8b5a82df3037e1d8fe8b121ed42e536f569dec16113",
    strip_prefix = "protobuf-3.14.0",
    urls = [
        "https://mirror.bazel.build/github.com/protocolbuffers/protobuf/archive/v3.14.0.tar.gz",
        "https://github.com/protocolbuffers/protobuf/archive/v3.14.0.tar.gz",
    ],
)

load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")

protobuf_deps()

go_repository(
    name = "com_github_golang_mock",
    build_file_generation = "on",
    importpath = "github.com/golang/mock",
    sum = "h1:G5FRp8JnTd7RQH5kemVNlMeyXQAztQ3mOWV95KxsXH8=",
    version = "v1.1.1",
)
  1. run go mod tidy
  2. run bazel run //:gazelle-update-repos
  3. run bazel build --profile=/tmp/mocks.gz //:mocks
  4. run bazel analyze-profile /tmp/mocks.gz

@achew22
Copy link
Member

achew22 commented Apr 16, 2022

This looks good to me. Is there any chance we could use add a test? Even just something that used it would provide a lot of confidence for me. We could also add a test in https://github.com/bazelbuild/rules_go/tree/master/tests/integration to confirm it if you're feeling it.

Also, what would you think about explicitly adding some docs onto this that say the API on the gomock rules isn't stable yet? Then we can make some changes, wait for it to solidify and then promote it?

@linzhp
Copy link
Contributor Author

linzhp commented Apr 17, 2022

Good ideas. I added a unit test, instead of an integration test, to generate mocks using the gomock rule, and verify the output. Unit tests should be powerful enough in this case, and they are much faster.

@@ -0,0 +1,33 @@
load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test", "gomock")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this shouldn't be in tests/core since it isn't a part of the core go package, it is a part of extras. Should we put this inside extras/tests/gomock?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying to be consistent with go_embed_data, which is also in extras. I can create a follow up PR to move both tests/core/go_embed_data and tests/core/gomock to tests/extras.

@linzhp linzhp merged commit b5ca1f0 into bazelbuild:master Apr 18, 2022
@linzhp linzhp deleted the gopath branch April 18, 2022 04:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants