MSVC toolchain requiring parse_showinclude feature is not documented

[limdor]

Description of the problem:

When trying to build a cc_library using a custom msvc 2019 toolchain that is the same like the autodetected one, but without /showInclude options, it does not build and shows the following error:
ERROR: C:/users/xavib/documents/development/quoniam/src/example/BUILD:3:11: Compiling src/example/library.cpp failed: error while parsing .d file: C:\users\xavib\_bazel_xavib\3r2w35xj\execroot\__main__\bazel-out\x64_windows-fastbuild\bin\src\example\_objs\example\library.d (The system cannot find the file specified)
If I add /showInclude in the manually declared toolchain, then it works.

Feature requests: what underlying problem are you trying to solve with this feature?

I want to have the same toolchain explicitly declared but I want to remove the /showInclude option

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Create toolchain like the autodetected one in Windows with Visual studio 2019, and remove the /showInclude options. Here is the one that I used: limdor/quoniam@8605cd3

What operating system are you running Bazel on?

Windows

What's the output of bazel info release?

4.0.0

Have you found anything relevant by searching the web?

I tried to check where this error about parsing .d files is comming from, and this seems to come from Bazel, and not from Visual Studio: https://github.com/bazelbuild/bazel/search?q=%22error+while+parsing+.d+file%22
The most interesting part is in this part of the code, it seems that the behavior is different if /showInclude is specified or not, maybe someone familiar with the code can say what is the reason for that.

bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppCompileAction.java

Line 1877 in 2f00d4f

// output of cl.exe caused by /showIncludes option.

[meteorcloudy]

Enabling the /showincludes feature is required for MSVC toolchain, because Bazel uses the output to check if the headers included are declared as input. So this failure is WAI.

[limdor]

https://docs.bazel.build/versions/master/windows.html#build-c-with-msvc does not mention anything about it. Is it mentioned somewhere else? How is supposed the user know about it?
If that is intended, would it be ok if I create a feature request at least to show that this is missing?

[meteorcloudy]

If that is intended, would it be ok if I create a feature request at least to show that this is missing?

To report this feature is missing, Bazel has to know if you are using MSVC or mingw-gcc/msys-gcc in advance. Because it's only required for MSVC. I don't think Bazel should try to identify what exact compiler the toolchain is using.

But we definitely could use better documentation about this for users who want to write their own custom toolchain. Feel free to open an issue or PR for this! ;)

[limdor]

@meteorcloudy I created this PR to extend the documentation #13247

Could you elaborate a bit more on what is the problem? Is it in the rules_cc or in Bazel itself? I would not expect Bazel the need to know about the compiler. What would happen if I want to add a custom C++ compiler? Would this not be possible or it would only be possible without using the cc_toolchain?
Just asking because if that is the case it would also be good to add something in the documentation stating what compilers are supported for C++ and what should be done in case that someone wants to add another compiler that is not supported.

[meteorcloudy]

So with gcc style compilers, Bazel depends on the compiler to output a dependency file that declares what actual header files used for the compiling action, see this.

But this feature is not available in MSVC. Instead, MSVC provides this information by printing out the dependencies in stdout (/showIncludes). So basically, the parse_showinclude feature is an equivalent of the dependency_file feature for the MSVC toolchain.

If the parse_showinclude feature is enabled in the toolchain, Bazel tries to discover dependencies by parsing the stdout of MSVC, otherwise it tries to read the dot d file generated by gcc style compiler.
See

bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppCompileAction.java

Line 1839 in 5bff601

if (featureConfiguration.isEnabled(CppRuleClasses.PARSE_SHOWINCLUDES)) {

The problem you encounter here is that when you remove the parse_showinclude feature, Bazel assume the compiler will produce the dot d file, but in fact it still uses MSVC which doesn't generate that file.

[limdor]

This java code is part of Bazel, but this is part of rules cpp. Is this part of the code that in the long term should go in rules_cc in starlark? If that is the case, everything start making sense now.

Should I create another ticket for the documentation or what would be your proposal? Seems that it is not obvious where to document it.

[meteorcloudy]

This java code is part of Bazel, but this is part of rules cpp. Is this part of the code that in the long term should go in rules_cc in starlark? If that is the case, everything start making sense now.

Yes, you're right. The reason it's in Java is because C++ rules are still native rules. In the long term, they will be starlarkified.

Should I create another ticket for the documentation or what would be your proposal? Seems that it is not obvious where to document it.

Yeah, maybe adding it here: https://docs.bazel.build/versions/master/tutorial/cc-toolchain-config.html ?

[limdor]

@aiuto @meteorcloudy thanks for your feedback. I will continue the discussion here.

As for creating a toolchain for MSVC, maybe https://github.com/meteorcloudy/my_tests/tree/master/preconfigure_msvc_toolchain can help? Instead of writing one entirely from scratch, maybe you can start with the auto generated one?

I will take a look, seems like a good starting point, in the end this is exactly what I was looking for when I created #12767

Perhaps the two of you can pair up to turn that private test into a public
sample in the examples repository.

Fine for me, however I'm doing this during my free time, not sure when I will finish it. At work I am only working on Linux.
My plan was:

1. Get a manually configured toolchain working on a project (I took a personal project that is small enough to play around but big enough to check if would scale https://github.com/limdor/quoniam/pull/83/files)
2. Prepare a Bazel example explaining how to create a MSVC toolchain taking as a reference the autodetected one (https://github.com/limdor/bazel-examples)
3. Check how can I port it in some place of the official Bazel documentation. For the moment I had this private repository because I did not know where to put it and how much Bazel wants to maintain a big set of examples.

If you can tell me a bit more about number 3 would be great, I could event prepare porting some of the ones that I currently have if the community is interested.

[meteorcloudy]

Maybe https://github.com/bazelbuild/examples is a good place to store those examples?
Tutorials can be put at https://github.com/bazelbuild/bazel/tree/master/site/docs/tutorial

[limdor]

Thanks, I will see if I can put them somewhere there, for the moment I will play around in my repo and once I have it ready I will create the PR.
Create the toolchain from the generated one was easy, now the problem is to make use of the toolchain without crosstool_top and making sure that the generated one is not available and not selected.
#7260 (comment)

[meteorcloudy]

Sounds good!

making sure that the generated one is not available and not selected.

Check the BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN env var.
https://github.com/bazelbuild/bazel/blob/master/tools/cpp/cc_configure.bzl#L49

[limdor]

Thanks for the support @meteorcloudy, I manage to create the example for linux, now I will do the same for Windows also and then we can see where would be the place to put them in the Bazel repo.
https://github.com/limdor/bazel-examples/tree/master/linux_toolchain

[meteorcloudy]

Wow, that's some really nice documentation! Thank you so much!

[meteorcloudy]

/cc @oquenchil probably can help with reviewing it.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[limdor]

@bazelbuild/triage please keep

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.

[github-actions]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please post @bazelbuild/triage in a comment here and we'll take a look. Thanks!