Support Apple Silicon

[phst]

What version of rules_go are you using?

Current head, commit hash da0f36c

What version of gazelle are you using?

None

What version of Bazel are you using?

4.0.0

Does this issue reproduce with the latest releases of all the above?

Yes

What operating system and processor architecture are you using?

macOS, ARM64

Any other potentially useful information about your toolchain?

rules_go/go/private/sdk.bzl

Line 232 in a322b06

goos, goarch = "darwin", "amd64"

seems to assume that all macOS machines are x86-64, which is no longer the case. This branch likely should check the output of uname -m or arch, like the Linux branch.

What did you do?

Download the current master of rules_go and run bazel test //... on an Apple Silicon machine.

What did you expect to see?

Tests should succeed.

What did you see instead?

Toolchain resolution fails:

$ bazel test //...
DEBUG: /private/var/tmp/_bazel_p/3d10a82390614c10e7f6b64e3ae2263e/external/bazel_toolchains/rules/rbe_repo/version_check.bzl:68:14: 
Current running Bazel is ahead of bazel-toolchains repo. Please update your pin to bazel-toolchains repo in your WORKSPACE file.
DEBUG: /private/var/tmp/_bazel_p/3d10a82390614c10e7f6b64e3ae2263e/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:125:14: buildkite_config not using checked in configs; Bazel version 4.0.0 was picked/selected but no checked in config was found in map {"0.20.0": ["8.0.0"], "0.21.0": ["8.0.0"], "0.22.0": ["8.0.0", "9.0.0"], "0.23.0": ["8.0.0", "9.0.0"], "0.23.1": ["8.0.0", "9.0.0"], "0.23.2": ["9.0.0"], "0.24.0": ["9.0.0"], "0.24.1": ["9.0.0"], "0.25.0": ["9.0.0"], "0.25.1": ["9.0.0"], "0.25.2": ["9.0.0"], "0.26.0": ["9.0.0"], "0.26.1": ["9.0.0"], "0.27.0": ["9.0.0"], "0.27.1": ["9.0.0"], "0.28.0": ["9.0.0"], "0.28.1": ["9.0.0"], "0.29.0": ["9.0.0"], "0.29.1": ["9.0.0", "10.0.0"], "1.0.0": ["9.0.0", "10.0.0"], "1.0.1": ["10.0.0"], "1.1.0": ["10.0.0"], "1.2.0": ["10.0.0"], "1.2.1": ["10.0.0"], "2.0.0": ["10.0.0"], "2.1.0": ["10.0.0"], "2.1.1": ["10.0.0", "11.0.0"], "2.2.0": ["11.0.0"], "3.0.0": ["11.0.0"], "3.1.0": ["11.0.0"], "3.2.0": ["11.0.0"], "3.3.0": ["11.0.0"], "3.3.1": ["11.0.0"], "3.4.1": ["11.0.0"], "3.5.0": ["11.0.0"], "3.5.1": ["11.0.0"], "3.6.0": ["11.0.0"], "3.7.0": ["11.0.0"]}
ERROR: While resolving toolchains for target //tests/core/cross:darwin_cross: no matching toolchains found for types //go:toolchain
ERROR: Analysis of target '//tests/core/cross:darwin_cross' failed; build aborted: no matching toolchains found for types //go:toolchain
INFO: Elapsed time: 0,243s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 71 targets configured)
FAILED: Build did NOT complete successfully (0 packages loaded, 71 targets configured)
    Fetching @go_googleapis; fetching
    Fetching @gogo_special_proto; fetching

[jayconrod]

It will be good to support Apple Silicon, and I'm happy to take PRs working toward that.

However, I'll note that Bazel doesn't yet officially support Apple Silicon (bazelbuild/bazel#11628), and we don't have arm64 hardware in CI yet. Go support is also close but not there yet (coming in 1.16). So this may not happen right away.

[phst]

It will be good to support Apple Silicon, and I'm happy to take PRs working toward that.

Thanks!

However, I'll note that Bazel doesn't yet officially support Apple Silicon (bazelbuild/bazel#11628), and we don't have arm64 hardware in CI yet. Go support is also close but not there yet (coming in 1.16). So this may not happen right away.

Sure. At least bazelbuild/bazel#11628 is now fixed (i.e. Bazel officially supports AArch64).

[JoshBroomberg]

FYI that Bazel now has an Apple Silicon binary (with support for darwin-arm64): bazelbuild/bazel@8c7e11a that is available for easy download through homebrew. Would be great to work towards toolchain support in rules_go

[phst]

Go support is also close but not there yet (coming in 1.16)

Go 1.16 has been released today with AArch64 support: https://tip.golang.org/doc/go1.16#darwin

[phst]

Once #2829 is fixed, I'll check whether ARM works.

[phst]

It looks like #2829 didn't update all needed dependencies. At least I get the same errors about outdated versions of golang.org/x/net and golang.org/x/sys as before. We still need something like #2827 and #2828. https://github.com/bazelbuild/rules_go/pull/2837/files#diff-47e2b5bc058238377d9e37d788633c409b47cdfce5f9fe4da2937eb7887fb552 only updates golang.org/x/tools and misses everything else.

[phst]

Apart from #2795 (comment), tests now generally succeed on ARM-64, with a few minor exceptions:

• The plugin tests fail with an error that looks similar to plugin: load plugin crash fatal error: runtime: no plugin module data golang/go#33072.
• The Objective-C CGo rules require passing --macos_cpus=arm64 manually. This might be related to Swift host tools compiled with wrong cpu on Apple Silicon bazel#12671.

[jayconrod]

Ah, sorry. In my haste, I didn't realize popular_repos would need to be updated as well. Those don't get updated regularly since they're just there for downstream testing.

I'll send a PR to update them shortly.

[phst]

Thanks! Unfortunately 64bba9f is still not enough; at least I still get ancient versions of golang.org/x/sys, even after bazel clean --expunge. For example, compare the files in the cpu subdirectory with upstream (https://github.com/golang/sys/tree/master/cpu). Probably you also need to update the versions in some dependency, e.g. https://github.com/bazelbuild/bazel-gazelle/blob/master/deps.bzl.

[jayconrod]

@phst Sorry for the oversight. I just tagged rules_go v0.27.0, which updates org_golang_x_tools and adds a dependency on org_golang_x_sys at master as of yesterday. Hopefully that fixes things.

[phst]

Thanks again!
Now all the tests except the plugin tests in the tests/core/go_plugin package pass. Since these tests work with go build and go test, I guess some compiler flag is missing. -dynlink looks like a promising candidate, but passing it explicitly to the compiler doesn't help.

[phst]

I've compared the generated command lines for a go build command and a Bazel build, but couldn't find any meaningful difference. Maybe you could take a look since you're much more familiar with the builder code? Otherwise, could we declare plugins unsupported on ARM-64 for now and file a follow-up bug?

[jayconrod]

Sorry for the very delayed reply. I'm afraid the only update I have to share is that plugins should work on darwin/arm64. supported.go is the code the toolchain uses to decide whether something's supported or not, and darwin/arm64 is in there, so I assume it should work.

I'm afraid I don't have bandwidth to dig into it further than that though.

[phst]

OK, after a few more hours of debugging, I found the problem, and of course it's a trivial one-liner: darwin/arm64 needs to be added to _LINK_PLUGIN_PLATFORMS in go/private/mode.bzl. Otherwise the runtime is built without the necessary -dynlib flag.

[joice]

operating system and processor architecture

$ uname -a
Darwin C02F70LRQ05G 20.4.0 Darwin Kernel Version 20.4.0: Thu Apr 22 21:46:41 PDT 2021; root:xnu-7195.101.2~1/RELEASE_ARM64_T8101 arm64

bazel version

$ bazel version
Build label: 4.0.0-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Sat Jan 23 02:10:29 2021 (1611367829)
Build timestamp: 1611367829
Build timestamp as int: 1611367829

bazel rules_go version

http_archive(
    name = "io_bazel_rules_go",
    sha256 = "69de5c704a05ff37862f7e0f5534d4f479418afc21806c887db544a316f3cb6b",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.27.0/rules_go-v0.27.0.tar.gz",
        "https://github.com/bazelbuild/rules_go/releases/download/v0.27.0/rules_go-v0.27.0.tar.gz",
    ],
)

Error when I tried to build project running bazel build --host_cpu=darwin_arm64 //...

ERROR: While resolving toolchains for target //foo/bar:bar no matching toolchains found for types @io_bazel_rules_go//go:toolchain
ERROR: Analysis of target '//foo/bar:bar' failed; build aborted: no matching toolchains found for types @io_bazel_rules_go//go:toolchain

Any ideas how to solve this? plz

[phst]

Try --macos_cpus=arm64 instead of --host_cpu=darwin_arm64, or try Bazel 4.1, which shouldn't need the --macos_cpus=arm64 flag.

[joice]

Now, I've updated the Bazel version and tried to build again my project, but I got this new error shown bellow. Just in case, I've cloned the bazel project and I'd tried to build the project running bazel build //src:bazel with and without the flags above and I've got the same error raised in my project.

bazel version
Build label: 4.1.0-homebrew
Build target: bazel-out/darwin_arm64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800

ERROR: /Users/joicets/go/work/github.com/bazelbuild/bazel/src/BUILD:100:10: Target '//src:create_embedded_tools' depends on toolchain '@local_config_cc//:cc-compiler-darwin_arm64', which cannot be found: no such target '@local_config_cc//:cc-compiler-darwin_arm64': target 'cc-compiler-darwin_arm64' not declared in package '' defined by /private/var/tmp/_bazel_cache/df96afd62f2908e36efc22992595d94b/external/local_config_cc/BUILD'
ERROR: Analysis of target '//src:bazel' failed; build aborted: Analysis failed

Looks like a problem with C/C++ compiler. Any thoughts?

[thii]

FYI the fix for bazelbuild/bazel#12671 is not included in Bazel 4.1.

[uhthomas]

FWIW I picked up an Apple Silicon machine today and have confirmed the same behaviour @joice is seeing.

$ bazel test //...
ERROR: While resolving toolchains for target //database/badger:go_default_library: no matching toolchains found for types @io_bazel_rules_go//go:toolchain
ERROR: Analysis of target '//database/badger:go_default_library' failed; build aborted: no matching toolchains found for types @io_bazel_rules_go//go:toolchain
INFO: Elapsed time: 0.102s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
    currently loading: @io_bazel_rules_docker//

[phst]

FYI the fix for bazelbuild/bazel#12671 is not included in Bazel 4.1.

Good point, I guess we should wait for Bazel 4.2 then.

[phst]

Looks like a problem with C/C++ compiler.

No idea, sorry. The Go rules work reasonably well on my machine (with Bazel 4.1 and rules_go 0.27).
If this is happening for pure C++ repositories as well, it might be worth reporting an issue against Bazel.

[phst]

One failure mode that I still see even with --macos_cpus=arm64 and Bazel 4.1 (actually seems to be a new one) is that shared library dependencies of test binaries have a wrong filename. For example:

$ bazel test //tests/legacy/examples/cgo:cgo_lib_test
[...]
==================== Test output for //tests/legacy/examples/cgo:cgo_lib_test:
dyld: Library not loaded: bazel-out/darwin_arm64-fastbuild/bin/tests/legacy/examples/cgo/cc_dependency/libc_version_so.so
  Referenced from: /private/var/tmp/_bazel_p/3d10a82390614c10e7f6b64e3ae2263e/sandbox/darwin-sandbox/63/execroot/io_bazel_rules_go/bazel-out/darwin_arm64-fastbuild/bin/tests/legacy/examples/cgo/cgo_lib_test_/cgo_lib_test.runfiles/io_bazel_rules_go/tests/legacy/examples/cgo/cgo_lib_test_/cgo_lib_test
  Reason: image not found
================================================================================

The problem is that this test executable uses a relative filename to reference a dynamic library:

.../execroot/io_bazel_rules_go/bazel-out/darwin_arm64-fastbuild/bin/tests/legacy/examples/cgo/cgo_lib_test_/cgo_lib_test.runfiles/io_bazel_rules_go$ otool -L tests/legacy/examples/cgo/cgo_lib_test_/cgo_lib_test
tests/legacy/examples/cgo/cgo_lib_test_/cgo_lib_test:
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 905.6.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
	bazel-out/darwin_arm64-fastbuild/bin/tests/legacy/examples/cgo/cc_dependency/libc_version_so.so (compatibility version 0.0.0, current version 0.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1775.118.101)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1775.118.101)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

That doesn't work for tests because they are run from the workspace subdirectory of the runfiles root, not the execution root. No idea how to best fix this though; using a relative filename makes sense so that people can e.g. pick a different output base. But to produce binaries that work in both actions and tests, the dynamic linker needs to search both the execution root and the runfiles root.

[Darko-Huma]

 bazel run gazelle                                                                              
(14:52:48) INFO: Current date is 2021-07-13
(14:52:48) ERROR: While resolving toolchains for target //:gazelle-runner: no matching toolchains found for types @io_bazel_rules_go//go:toolchain
(14:52:48) ERROR: Analysis of target '//:gazelle' failed; build aborted: no matching toolchains found for types @io_bazel_rules_go//go:toolchain
(14:52:48) INFO: Elapsed time: 0.082s
(14:52:48) INFO: 0 processes.
(14:52:48) ERROR: Build failed. Not running target

Bazel version
4.1

We just updated the project on 4.1 bazel and now it's not working, strange with 3.7 it was okay.

[luisnunes111]

I had the same issue as @uhthomas and @joice because my bazel was installed for the arm64 arch, but installing x86_64 bazel binary solved the issue.

I'm using rules_go 0.28 and bazel 4.0.0

[ruiyang2015]

got the no matching toolchains found for types @io_bazel_rules_go//go:toolchain error for bazel 4.1.0 rules_go 0.28

[weixiao-huang]

Is there any progress for this topic? Can we use bazel-4.2.1-darwin-arm64 to compile go code in Apple Silicon?

[robfig]

@linzhp is using rules_go on Apple Silicon as his daily driver, so I think we can close this. Any bugs in the support should be opened as separate issues.

[linzhp]

@weixiao-huang Yes, you can use Bazel 4.2.1 and rules_go 0.29 to build Go code in Apple Silicon. For all other people having issues, please upgrade Bazel and rules_go

[movewho]

I upgrade new version of Bazel (4.2.2) and rules_go(0.29), but I still get error
Analysis of target '//:gazelle' failed; build aborted: no matching toolchains found for types @io_bazel_rules_go//go:toolchain

Which version I can use?

[robfig]

Please open a new issue along with enough information to reproduce the problem, ideally in the form of a github repo where we can run a command and see the issue. Thanks

[keith]

Do make sure your version of rules_go is actually 0.29 and not being pinned transitively to an older version via some other repo.

[pjjw]

starting to also get the above after an os update. will file a ticket.

[kunal-rp]

Also hitting this issue. Cloning intel darwin bazel project over to M1 mac, see this error :

ERROR: While resolving toolchains for target @io_bazel_rules_docker//container/go/cmd/zipper:zipper: no matching toolchains found for types @io_bazel_rules_go//go:toolchain
ERROR: Analysis of target '//java/com/task:task_java_image' failed; build aborted: no matching toolchains found for types @io_bazel_rules_go//go:toolchain

arch :

Darwin KRP-Silver.local 21.3.0 Darwin Kernel Version 21.3.0: Wed Jan  5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000 arm64

1. What is the core issue? Is there an problem that go is not supported on arm64 machines running macOs ?

2. I see that for go toolchain there is a platform/arch configuration for my device , does this have to do anything with the issue

[Mercanuis]

I am getting this error again on my M1 with Bazel-5.0.0-homebrew

 ~/Documents/Workspaces/xxx/ [main+*] bazel version
Build label: 5.0.0-homebrew
Build target: bazel-out/darwin_arm64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800

WORKSPACE

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

###############################################################################
#
# Go Rules.
#

http_archive(
    name = "bazel_gazelle",
    sha256 = "de69a09dc70417580aabf20a28619bb3ef60d038470c7cf8442fafcf627c21cb",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.24.0/bazel-gazelle-v0.24.0.tar.gz",
        "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.24.0/bazel-gazelle-v0.24.0.tar.gz",
    ],
)

http_archive(
    name = "io_bazel_rules_go",
    sha256 = "2b1641428dff9018f9e85c0384f03ec6c10660d935b750e3fa1492a281a53b0f",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.29.0/rules_go-v0.29.0.zip",
        "https://github.com/bazelbuild/rules_go/releases/download/v0.29.0/rules_go-v0.29.0.zip",
    ],
)

load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")

gazelle_dependencies()

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_register_toolchains(version = "1.17.2")

go_rules_dependencies()

BUILD (root)

load("@bazel_gazelle//:def.bzl", "gazelle")

# gazelle:prefix github.com/demo/xxx
gazelle(name = "gazelle")

BUILD(under cmd)

load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library")

go_library(
    name = "ils_lib",
    srcs = ["main.go"],
    importpath = "xxx/demo/yyy/cmd",
    visibility = ["//visibility:private"],
)

go_binary(
    name = "ils",
    embed = [":ils_lib"],
    visibility = ["//visibility:public"],
)

Output

 ~/Documents/Workspaces/xxx/ [main+*] bazel run //:gazelle
Starting local Bazel server and connecting to it...
DEBUG: Rule 'bazel_skylib' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1629300223 -0400"
DEBUG: Repository bazel_skylib instantiated at:
  Documents/Workspaces/xxx/WORKSPACE:28:21: in <toplevel>
  /private/var/tmp/_bazel_jjj/039f59c7b0f7992a261100780bd49fd9/external/bazel_gazelle/deps.bzl:43:11: in gazelle_dependencies
  /private/var/tmp/_bazel_jjj/039f59c7b0f7992a261100780bd49fd9/external/bazel_gazelle/deps.bzl:376:18: in _maybe
Repository rule git_repository defined at:
  /private/var/tmp/_bazel_jjj/039f59c7b0f7992a261100780bd49fd9/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
ERROR: Documents/Workspaces/xxx/BUILD:4:8: While resolving toolchains for target //:gazelle-runner: no matching toolchains found for types @io_bazel_rules_go//go:toolchain
ERROR: Analysis of target '//:gazelle' failed; build aborted: 
INFO: Elapsed time: 8.849s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (36 packages loaded, 450 targets configured)
FAILED: Build did NOT complete successfully (36 packages loaded, 450 targets configured)

[vpanta]

Wanted to note: we too have been seeing this issue recently, right after enabling arm bazel (via bumping bazelisk to it's arm version).

We are on [email protected] and [email protected], and folks are getting the same:

bazel build //:gazelle_all_languages
INFO: Invocation ID: 1bd233f1-5d70-47ac-a961-548c62d2eb07
ERROR:xxx/BUILD.bazel:19:15: While resolving toolchains for target //:gazelle_all_languages: no matching toolchains found for types @io_bazel_rules_go//go:toolchain
ERROR: Analysis of target '//:gazelle_all_languages' failed;

We've found that after the change, if they do a bazel clean --expunge and purge our on-disk repository cache, then it works. Maybe the toolchain gets cached as the x64 version along with the repository itself?

Raising it in case there's anything to be done from y'all's perspective for this. We're going to update to [email protected] either way, which should also cause a rebuild of the toolchain.

Our WORKSPACE load for rules_go:

http_archive(
    name = "io_bazel_rules_go",
    sha256 = "2b1641428dff9018f9e85c0384f03ec6c10660d935b750e3fa1492a281a53b0f",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.29.0/rules_go-v0.29.0.zip",
        "https://github.com/bazelbuild/rules_go/releases/download/v0.29.0/rules_go-v0.29.0.zip",
    ],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_register_toolchains(go_version = "1.18")

go_rules_dependencies()

[keith]

Can you submit a new issue for that? We see this as well, another workaround I've been using is to bump the go_version, which seems to force it to fix itself

[Fristi]

I'm also seeing this error as well on an M1 with different combinations

/private/var/tmp/_bazel_markie/35a9a062c2f847c537952b63ca28f9a9/external/io_bazel_rules_docker/container/go/cmd/extract_config/BUILD:27:10: While resolving toolchains for target @io_bazel_rules_docker//container/go/cmd/extract_config:extract_config: no matching toolchains found for types @io_bazel_rules_go//go:toolchain

• Bazel 4 and 5
• Tried latest 3-4 versions of rules_go
• Tried latest 2-3 versions of rules_gazelle
• Tried latest 2-3 versions of rules_docker

This in combination with the latest rules_scala

[Fristi]

Oh my, I just figured something.. after I found this line:
https://github.com/bazelbuild/rules_go/blob/master/go/private/sdk.bzl#L270

I tried in my iterm uname -m which returend x86_64 .. after trying the native terminal from macos, it returns arm64 and after a bazel clean --expunge it works :tada