linux.git/scripts/Makefile.host, branch v6.18.21 Clone of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git kbuild: support building external modules in a separate build directory 2024-11-27T23:11:55+00:00 Masahiro Yamada masahiroy@kernel.org 2024-11-10T01:34:35+00:00 11b3d5175e6bc3779159228e6077be202d2b0069 There has been a long-standing request to support building external modules in a separate build directory. This commit introduces a new environment variable, KBUILD_EXTMOD_OUTPUT, and its shorthand Make variable, MO. A simple usage: $ make -C <kernel-dir> M=<module-src-dir> MO=<module-build-dir> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> 
There has been a long-standing request to support building external
modules in a separate build directory.

This commit introduces a new environment variable, KBUILD_EXTMOD_OUTPUT,
and its shorthand Make variable, MO.

A simple usage:

 $ make -C <kernel-dir> M=<module-src-dir> MO=<module-build-dir>

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
kbuild: add intermediate targets for Flex/Bison in scripts/Makefile.host 2024-09-08T03:15:46+00:00 Masahiro Yamada masahiroy@kernel.org 2024-09-04T23:47:37+00:00 fc41a0a7498636ac0af7c37be80ca8571c2f4173 Flex and Bison are used only for host programs. Move their intermediate target processing from scripts/Makefile.build to scripts/Makefile.host. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> 
Flex and Bison are used only for host programs. Move their intermediate
target processing from scripts/Makefile.build to scripts/Makefile.host.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
kbuild: Use $(obj)/%.cc to fix host C++ module builds 2024-06-25T15:18:57+00:00 Nicolas Schier n.schier@avm.de 2024-06-24T11:12:14+00:00 7ed9d1318c127b3aec77099802a9fdf2480250b4 Use $(obj)/ instead of $(src)/ prefix when building C++ modules for host, as explained in commit b1992c3772e6 ("kbuild: use $(src) instead of $(srctree)/$(src) for source directory"). This fixes build failures of 'xconfig': $ make O=build/ xconfig make[1]: Entering directory '/data/linux/kbuild-review/build' GEN Makefile make[3]: *** No rule to make target '../scripts/kconfig/qconf-moc.cc', needed by 'scripts/kconfig/qconf-moc.o'. Stop. Fixes: b1992c3772e6 ("kbuild: use $(src) instead of $(srctree)/$(src) for source directory") Reported-by: Rolf Eike Beer <eb@emlix.com> Signed-off-by: Nicolas Schier <n.schier@avm.de> Tested-by: Rolf Eike Beer <eb@emlix.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> 
Use $(obj)/ instead of $(src)/ prefix when building C++ modules for
host, as explained in commit b1992c3772e6 ("kbuild: use $(src) instead
of $(srctree)/$(src) for source directory").  This fixes build failures
of 'xconfig':

    $ make O=build/ xconfig
    make[1]: Entering directory '/data/linux/kbuild-review/build'
      GEN     Makefile
    make[3]: *** No rule to make target '../scripts/kconfig/qconf-moc.cc', needed by 'scripts/kconfig/qconf-moc.o'.  Stop.

Fixes: b1992c3772e6 ("kbuild: use $(src) instead of $(srctree)/$(src) for source directory")
Reported-by: Rolf Eike Beer <eb@emlix.com>
Signed-off-by: Nicolas Schier <n.schier@avm.de>
Tested-by: Rolf Eike Beer <eb@emlix.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
kbuild: use $(obj)/ instead of $(src)/ for common pattern rules 2024-05-09T19:33:45+00:00 Masahiro Yamada masahiroy@kernel.org 2024-04-27T14:55:01+00:00 9a0ebe5011f49e932bb0a2cea2034fd65e6e567e Kbuild conventionally uses $(obj)/ for generated files, and $(src)/ for checked-in source files. It is merely a convention without any functional difference. In fact, $(obj) and $(src) are exactly the same, as defined in scripts/Makefile.build: src := $(obj) Before changing the semantics of $(src) in the next commit, this commit replaces $(obj)/ with $(src)/ in pattern rules where the prerequisite might be a generated file. C, assembly, Rust, and DTS files are sometimes generated by tools, so they could be either generated files or real sources. The $(obj)/ prefix works for both cases with the help of VPATH. As mentioned above, $(obj) and $(src) are the same at this point, hence this commit has no functional change. I did not modify scripts/Makefile.userprogs because there is no use case where userspace C files are generated. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> 
Kbuild conventionally uses $(obj)/ for generated files, and $(src)/ for
checked-in source files. It is merely a convention without any functional
difference. In fact, $(obj) and $(src) are exactly the same, as defined
in scripts/Makefile.build:

  src := $(obj)

Before changing the semantics of $(src) in the next commit, this commit
replaces $(obj)/ with $(src)/ in pattern rules where the prerequisite
might be a generated file.

C, assembly, Rust, and DTS files are sometimes generated by tools, so
they could be either generated files or real sources. The $(obj)/ prefix
works for both cases with the help of VPATH.

As mentioned above, $(obj) and $(src) are the same at this point, hence
this commit has no functional change.

I did not modify scripts/Makefile.userprogs because there is no use
case where userspace C files are generated.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
kbuild: mark `rustc` (and others) invocations as recursive 2024-02-29T21:16:38+00:00 Miguel Ojeda ojeda@kernel.org 2024-02-17T00:26:37+00:00 ecab4115c44cc50fc7320bef9c19ac01ad43c785 `rustc` (like Cargo) may take advantage of the jobserver at any time (e.g. for backend parallelism, or eventually frontend too). In the kernel, we call `rustc` with `-Ccodegen-units=1` (and `-Zthreads` is 1 so far), so we do not expect parallelism. However, in the upcoming Rust 1.76.0, a warning is emitted by `rustc` [1] when it cannot connect to the jobserver it was passed (in many cases, but not all: compiling and `--print sysroot` do, but `--version` does not). And given GNU Make always passes the jobserver in the environment variable (even when a line is deemed non-recursive), `rustc` will end up complaining about it (in particular in Make 4.3 where there is only the simple pipe jobserver style). One solution is to remove the jobserver from `MAKEFLAGS`. However, we can mark the lines with calls to `rustc` (and Cargo) as recursive, which looks simpler. This is being documented as a recommendation in `rustc` [2] and allows us to be ready for the time we may use parallelism inside `rustc` (potentially now, if a user passes `-Zthreads`). Thus do so. Similarly, do the same for `rustdoc` and `cargo` calls. Finally, there is one case that the solution does not cover, which is the `$(shell ...)` call we have. Thus, for that one, set an empty `MAKEFLAGS` environment variable. Link: https://github.com/rust-lang/rust/issues/120515 [1] Acked-by: Masahiro Yamada <masahiroy@kernel.org> Link: https://github.com/rust-lang/rust/pull/121564 [2] Link: https://lore.kernel.org/r/20240217002638.57373-1-ojeda@kernel.org [ Reworded to add link to PR documenting the recommendation. ] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> 
`rustc` (like Cargo) may take advantage of the jobserver at any time
(e.g. for backend parallelism, or eventually frontend too). In the kernel,
we call `rustc` with `-Ccodegen-units=1` (and `-Zthreads` is 1 so far),
so we do not expect parallelism. However, in the upcoming Rust 1.76.0, a
warning is emitted by `rustc` [1] when it cannot connect to the jobserver
it was passed (in many cases, but not all: compiling and `--print sysroot`
do, but `--version` does not). And given GNU Make always passes
the jobserver in the environment variable (even when a line is deemed
non-recursive), `rustc` will end up complaining about it (in particular
in Make 4.3 where there is only the simple pipe jobserver style).

One solution is to remove the jobserver from `MAKEFLAGS`. However, we
can mark the lines with calls to `rustc` (and Cargo) as recursive, which
looks simpler. This is being documented as a recommendation in `rustc`
[2] and allows us to be ready for the time we may use parallelism inside
`rustc` (potentially now, if a user passes `-Zthreads`). Thus do so.

Similarly, do the same for `rustdoc` and `cargo` calls.

Finally, there is one case that the solution does not cover, which is the
`$(shell ...)` call we have. Thus, for that one, set an empty `MAKEFLAGS`
environment variable.

Link: https://github.com/rust-lang/rust/issues/120515 [1]
Acked-by: Masahiro Yamada <masahiroy@kernel.org>
Link: https://github.com/rust-lang/rust/pull/121564 [2]
Link: https://lore.kernel.org/r/20240217002638.57373-1-ojeda@kernel.org
[ Reworded to add link to PR documenting the recommendation. ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
rust: Respect HOSTCC when linking for host 2023-10-14T09:26:28+00:00 Matthew Maurer mmaurer@google.com 2023-10-05T21:39:58+00:00 80bac83a739d36d227aa5c54f6ed417f13c086dd Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined, resulting in build failures in hermetic environments where `cc` does not exist. This includes both hostprogs and proc-macros. Since we are setting the linker to `HOSTCC`, we set the linker flavor to `gcc` explicitly. The linker-flavor selects both which linker to search for if the linker is unset, and which kind of linker flags to pass. Without this flag, `rustc` would attempt to determine which flags to pass based on the name of the binary passed as `HOSTCC`. `gcc` is the name of the linker-flavor used by `rustc` for all C compilers, including both `gcc` and `clang`. Signed-off-by: Matthew Maurer <mmaurer@google.com> Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com> Tested-by: Alice Ryhl <aliceryhl@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Acked-by: Miguel Ojeda <ojeda@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> 
Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
resulting in build failures in hermetic environments where `cc` does not
exist. This includes both hostprogs and proc-macros.

Since we are setting the linker to `HOSTCC`, we set the linker flavor to
`gcc` explicitly. The linker-flavor selects both which linker to search
for if the linker is unset, and which kind of linker flags to pass.
Without this flag, `rustc` would attempt to determine which flags to
pass based on the name of the binary passed as `HOSTCC`. `gcc` is the
name of the linker-flavor used by `rustc` for all C compilers, including
both `gcc` and `clang`.

Signed-off-by: Matthew Maurer <mmaurer@google.com>
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Tested-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
kbuild: rust: avoid creating temporary files 2023-07-23T18:15:31+00:00 Miguel Ojeda ojeda@kernel.org 2023-07-23T14:21:28+00:00 df01b7cfcef08bf3fdcac2909d0e1910781d6bfd `rustc` outputs by default the temporary files (i.e. the ones saved by `-Csave-temps`, such as `*.rcgu*` files) in the current working directory when `-o` and `--out-dir` are not given (even if `--emit=x=path` is given, i.e. it does not use those for temporaries). Since out-of-tree modules are compiled from the `linux` tree, `rustc` then tries to create them there, which may not be accessible. Thus pass `--out-dir` explicitly, even if it is just for the temporary files. Similarly, do so for Rust host programs too. Reported-by: Raphael Nestler <raphael.nestler@gmail.com> Closes: https://github.com/Rust-for-Linux/linux/issues/1015 Reported-by: Andrea Righi <andrea.righi@canonical.com> Tested-by: Raphael Nestler <raphael.nestler@gmail.com> # non-hostprogs Tested-by: Andrea Righi <andrea.righi@canonical.com> # non-hostprogs Fixes: 295d8398c67e ("kbuild: specify output names separately for each emission type from rustc") Cc: stable@vger.kernel.org Signed-off-by: Miguel Ojeda <ojeda@kernel.org> Tested-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> 
`rustc` outputs by default the temporary files (i.e. the ones saved
by `-Csave-temps`, such as `*.rcgu*` files) in the current working
directory when `-o` and `--out-dir` are not given (even if
`--emit=x=path` is given, i.e. it does not use those for temporaries).

Since out-of-tree modules are compiled from the `linux` tree,
`rustc` then tries to create them there, which may not be accessible.

Thus pass `--out-dir` explicitly, even if it is just for the temporary
files.

Similarly, do so for Rust host programs too.

Reported-by: Raphael Nestler <raphael.nestler@gmail.com>
Closes: https://github.com/Rust-for-Linux/linux/issues/1015
Reported-by: Andrea Righi <andrea.righi@canonical.com>
Tested-by: Raphael Nestler <raphael.nestler@gmail.com> # non-hostprogs
Tested-by: Andrea Righi <andrea.righi@canonical.com> # non-hostprogs
Fixes: 295d8398c67e ("kbuild: specify output names separately for each emission type from rustc")
Cc: stable@vger.kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
kbuild: remove sed commands after rustc rules 2023-01-22T14:43:33+00:00 Masahiro Yamada masahiroy@kernel.org 2023-01-07T09:18:17+00:00 2185242faddd12a1ba1060be5caf584fe5aba93a rustc may put comments in dep-info, so sed is used to drop them before passing it to fixdep. Now that fixdep can remove comments, Makefiles do not need to run sed. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Miguel Ojeda <ojeda@kernel.org> Tested-by: Miguel Ojeda <ojeda@kernel.org> Reviewed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> 
rustc may put comments in dep-info, so sed is used to drop them before
passing it to fixdep.

Now that fixdep can remove comments, Makefiles do not need to run sed.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
kbuild: specify output names separately for each emission type from rustc 2023-01-22T14:43:33+00:00 Masahiro Yamada masahiroy@kernel.org 2023-01-07T09:18:15+00:00 295d8398c67e314d99bb070f38883f83fe94a97a In Kbuild, two different rules must not write to the same file, but it happens when compiling rust source files. For example, set CONFIG_SAMPLE_RUST_MINIMAL=m and run the following: $ make -j$(nproc) samples/rust/rust_minimal.o samples/rust/rust_minimal.rsi \ samples/rust/rust_minimal.s samples/rust/rust_minimal.ll [snip] RUSTC [M] samples/rust/rust_minimal.o RUSTC [M] samples/rust/rust_minimal.rsi RUSTC [M] samples/rust/rust_minimal.s RUSTC [M] samples/rust/rust_minimal.ll mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory make[3]: *** [scripts/Makefile.build:334: samples/rust/rust_minimal.ll] Error 1 make[3]: *** Waiting for unfinished jobs.... mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory make[3]: *** [scripts/Makefile.build:309: samples/rust/rust_minimal.o] Error 1 mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory make[3]: *** [scripts/Makefile.build:326: samples/rust/rust_minimal.s] Error 1 make[2]: *** [scripts/Makefile.build:504: samples/rust] Error 2 make[1]: *** [scripts/Makefile.build:504: samples] Error 2 make: *** [Makefile:2008: .] Error 2 The reason for the error is that 4 threads running in parallel renames the same file, samples/rust/rust_minimal.d. This does not happen when compiling C or assembly files because -Wp,-MMD,$(depfile) explicitly specifies the dependency filepath. $(depfile) is a unique path for each target. Currently, rustc is only given --out-dir and --emit=<list-of-types> So, all the rust build rules output the dep-info into the default <CRATE_NAME>.d, which causes the path conflict. Fortunately, the --emit option is able to specify the output path individually, with the form --emit=<type>=<path>. Add --emit=dep-info=$(depfile) to the common part. Also, remove the redundant --out-dir because the output path is specified for each type. The code gets much cleaner because we do not need to rename *.d files. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Miguel Ojeda <ojeda@kernel.org> Tested-by: Miguel Ojeda <ojeda@kernel.org> Reviewed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> 
In Kbuild, two different rules must not write to the same file, but
it happens when compiling rust source files.

For example, set CONFIG_SAMPLE_RUST_MINIMAL=m and run the following:

  $ make -j$(nproc) samples/rust/rust_minimal.o samples/rust/rust_minimal.rsi \
                    samples/rust/rust_minimal.s samples/rust/rust_minimal.ll
    [snip]
    RUSTC [M] samples/rust/rust_minimal.o
    RUSTC [M] samples/rust/rust_minimal.rsi
    RUSTC [M] samples/rust/rust_minimal.s
    RUSTC [M] samples/rust/rust_minimal.ll
  mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory
  make[3]: *** [scripts/Makefile.build:334: samples/rust/rust_minimal.ll] Error 1
  make[3]: *** Waiting for unfinished jobs....
  mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory
  make[3]: *** [scripts/Makefile.build:309: samples/rust/rust_minimal.o] Error 1
  mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory
  make[3]: *** [scripts/Makefile.build:326: samples/rust/rust_minimal.s] Error 1
  make[2]: *** [scripts/Makefile.build:504: samples/rust] Error 2
  make[1]: *** [scripts/Makefile.build:504: samples] Error 2
  make: *** [Makefile:2008: .] Error 2

The reason for the error is that 4 threads running in parallel renames
the same file, samples/rust/rust_minimal.d.

This does not happen when compiling C or assembly files because
-Wp,-MMD,$(depfile) explicitly specifies the dependency filepath.
$(depfile) is a unique path for each target.

Currently, rustc is only given --out-dir and --emit=<list-of-types>
So, all the rust build rules output the dep-info into the default
<CRATE_NAME>.d, which causes the path conflict.

Fortunately, the --emit option is able to specify the output path
individually, with the form --emit=<type>=<path>.

Add --emit=dep-info=$(depfile) to the common part. Also, remove the
redundant --out-dir because the output path is specified for each type.

The code gets much cleaner because we do not need to rename *.d files.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
kbuild: refactor host*_flags 2023-01-22T14:43:33+00:00 Masahiro Yamada masahiroy@kernel.org 2023-01-07T09:18:14+00:00 16169a47d5c36046041527faafb5a3f5c86701c6 Remove _host*_flags. No functional change is intended. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Miguel Ojeda <ojeda@kernel.org> Tested-by: Miguel Ojeda <ojeda@kernel.org> 
Remove _host*_flags. No functional change is intended.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Miguel Ojeda <ojeda@kernel.org>