linux.git/scripts/rust_is_available.sh, branch v6.18.21

rust: document `bindgen` 0.71.0 regression

2025-01-09T23:17:00+00:00

`bindgen` 0.71.0 regressed [1] on the "`--version` requires header"
issue which appeared in 0.69.0 first [2] and was fixed in 0.69.1. It
has been fixed again in 0.71.1 [3].

Thus document it so that, when we upgrade the minimum past 0.69.0 in the
future, we do not forget that we cannot remove the workaround until we
arrive at 0.71.1 at least.

Link: https://github.com/rust-lang/rust-bindgen/issues/3039 [1]
Link: https://github.com/rust-lang/rust-bindgen/issues/2677 [2]
Link: https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md#v0711-2024-12-09 [3]
Reviewed-by: Fiona Behrens 
Reviewed-by: Alice Ryhl 
Link: https://lore.kernel.org/r/20241209212544.1977065-1-ojeda@kernel.org
Signed-off-by: Miguel Ojeda

rust: warn on bindgen < 0.69.5 and libclang >= 19.1

2024-11-12T20:26:54+00:00

When testing a `clang` upgrade with Rust Binder, Alice encountered [1] a
build failure caused by `bindgen` not translating some symbols related to
tracepoints. This was caused by commit 2e770edd8ce1 ("[libclang] Compute
the right spelling location") changing the behavior of a function exposed
by `libclang`. `bindgen` fixed the regression in commit 600f63895f73
("Use clang_getFileLocation instead of clang_getSpellingLocation").

However, the regression fix is only available in `bindgen` versions
0.69.5 or later (it was backported for 0.69.x). This means that when
older bindgen versions are used with new versions of `libclang`, `bindgen`
may do the wrong thing, which could lead to a build failure.

Alice encountered the bug with some header files related to tracepoints,
but it could also cause build failures in other circumstances. Thus,
always emit a warning when using an old `bindgen` with a new `libclang`
so that other people do not have to spend time chasing down the same
bug.

However, testing just the version is inconvenient, since distributions
do patch their packages without changing the version, so I reduced the
issue into the following piece of code that can trigger the issue:

    #define F(x) int x##x
    F(foo);

In particular, an unpatched `bindgen` will ignore the macro expansion
and thus not provide a declaration for the exported `int`.

Thus add a build test to `rust_is_available.sh` using the code above
(that is only triggered if the versions appear to be affected), following
what we did for the 0.66.x issue.

Moreover, I checked the status in the major distributions we have
instructions for:

  - Fedora 41 was affected but is now OK, since it now ships `bindgen`
    0.69.5.

    Thanks Ben for the quick reply on the updates that were ongoing.

    Fedora 40 and earlier are OK (older `libclang`, and they also now
    carry `bindgen` 0.69.5).

  - Debian Sid was affected but is now OK, since they now ship a patched
    `bindgen` binary (0.66.1-7+b3). The issue was reported to Debian by
    email and then as a bug report [2].

    Thanks NoisyCoil and Matthias for the quick replies. NoisyCoil handled
    the needed updates. Debian may upgrade to `bindgen` 0.70.x, too.

    Debian Testing is OK (older `libclang` so far).

  - Ubuntu non-LTS (oracular) is affected. The issue was reported to Ubuntu
    by email and then as a bug report [3].

    Ubuntu LTS is not affected (older `libclang` so far).

  - Arch Linux, Gentoo Linux and openSUSE should be OK (newer `bindgen` is
    provided). Nix as well (older `libclang` so far).

This issue was also added to our "live list" that tracks issues around
distributions [4].

Cc: Ben Beasley 
Cc: NoisyCoil 
Cc: Matthias Geiger 
Link: https://lore.kernel.org/rust-for-linux/20241030-bindgen-libclang-warn-v1-1-3a7ba9fedcfe@google.com/ [1]
Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1086510 [2]
Link: https://bugs.launchpad.net/ubuntu/+source/rust-bindgen-cli/+bug/2086639 [3]
Link: https://github.com/Rust-for-Linux/linux/issues/1127 [4]
Co-developed-by: Alice Ryhl 
Signed-off-by: Alice Ryhl 
Link: https://lore.kernel.org/r/20241111201607.653149-1-ojeda@kernel.org
Signed-off-by: Miguel Ojeda

rust: warn about `bindgen` versions 0.66.0 and 0.66.1

2024-07-10T08:28:52+00:00

`bindgen` versions 0.66.0 and 0.66.1 panic due to C string literals with
NUL characters [1]:

    panicked at .cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.66.0/codegen/mod.rs:717:71:
    called `Result::unwrap()` on an `Err` value: FromBytesWithNulError { kind: InteriorNul(4) }

Thus, in preparation for supporting several `bindgen` versions, add a
version check to warn the user about it.

Since some distributions may have patched it (e.g. Debian did [2]),
check if that seems to be the case (after the version check matches),
in order to avoid printing a warning in that case.

We could make it an error, but 1) it is going to fail anyway later
in the build, 2) we would disable `RUST`, which is also painful, 3)
someone could have patched it in a way that still makes our extra check
fail (however unlikely), 4) the interior NUL may go away in the headers
(however unlikely). Thus just warn about it so that users know why it
is failing.

In addition, add a couple tests for the new cases.

Link: https://github.com/rust-lang/rust-bindgen/pull/2567 [1]
Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1069047 [2]
Link: https://lore.kernel.org/r/20240709160615.998336-11-ojeda@kernel.org
Signed-off-by: Miguel Ojeda

rust: start supporting several `bindgen` versions

2024-07-10T08:28:52+00:00

With both the workaround for `bindgen` 0.69.0 and the warning about
0.66.0 and 0.66.1 in place, start supporting several `bindgen` versions,
like it was done for the Rust compiler in a previous patch.

All other versions, including the latest 0.69.4, build without errors.

The `bindgen` project, like Rust, has also agreed to have the kernel
in their CI [1] -- thanks! This should help both projects: `bindgen`
will be able to detect early issues like those mentioned above, and the
kernel will be very likely build with new releases (at least for the
basic configuration being tested).

Link: https://github.com/rust-lang/rust-bindgen/pull/2851 [1]
Tested-by: Benno Lossin 
Tested-by: Andreas Hindborg 
Link: https://lore.kernel.org/r/20240709160615.998336-10-ojeda@kernel.org
Signed-off-by: Miguel Ojeda

rust: work around `bindgen` 0.69.0 issue

2024-07-10T08:28:52+00:00

`bindgen` 0.69.0 contains a bug: `--version` does not work without
providing a header [1]:

    error: the following required arguments were not provided:
      

    Usage: bindgen    -- ...

Thus, in preparation for supporting several `bindgen` versions, work
around the issue by passing a dummy argument.

Include a comment so that we can remove the workaround in the future.

Link: https://github.com/rust-lang/rust-bindgen/pull/2678 [1]
Reviewed-by: Finn Behrens 
Tested-by: Benno Lossin 
Tested-by: Andreas Hindborg 
Link: https://lore.kernel.org/r/20240709160615.998336-9-ojeda@kernel.org
Signed-off-by: Miguel Ojeda

rust: start supporting several compiler versions

2024-07-10T08:28:52+00:00

It is time to start supporting several Rust compiler versions and thus
establish a minimum Rust version.

We may still want to upgrade the minimum sometimes in the beginning since
there may be important features coming into the language that improve
how we write code (e.g. field projections), which may or may not make
sense to support conditionally.

We will start with a window of two stable releases, and widen it over
time. Thus this patch does not move the current minimum (1.78.0), but
instead adds support for the recently released 1.79.0.

This should already be enough for kernel developers in distributions that
provide recent Rust compiler versions routinely, such as Arch Linux,
Debian Unstable (outside the freeze period), Fedora Linux, Gentoo
Linux (especially the testing channel), Nix (unstable) and openSUSE
Tumbleweed. See the documentation patch about it later in this series.

In addition, Rust for Linux is now being built-tested in Rust's pre-merge
CI [1]. That is, every change that is attempting to land into the Rust
compiler is tested against the kernel, and it is merged only if it passes
-- thanks to the Rust project for that!

Thus, with the pre-merge CI in place, both projects hope to avoid
unintentional changes to Rust that break the kernel. This means that,
in general, apart from intentional changes on their side (that we will
need to workaround conditionally on our side), the upcoming Rust compiler
versions should generally work.

For instance, currently, the beta (1.80.0) and nightly (1.81.0) branches
work as well.

Of course, the Rust for Linux CI job in the Rust toolchain may still need
to be temporarily disabled for different reasons, but the intention is
to help bring Rust for Linux into stable Rust.

Link: https://github.com/rust-lang/rust/pull/125209 [1]
Reviewed-by: Finn Behrens 
Tested-by: Benno Lossin 
Tested-by: Andreas Hindborg 
Link: https://lore.kernel.org/r/20240709160615.998336-7-ojeda@kernel.org
Signed-off-by: Miguel Ojeda

kbuild: rust_is_available: check that output looks as expected

2023-08-09T23:18:34+00:00

The script already checks for `$RUSTC` and `$BINDGEN` existing
and exiting without failure. However, one may still pass an
unexpected binary that does not output what the later parsing
expects. The script still successfully reports a failure as
expected, but the error is confusing. For instance:

    $ RUSTC=true BINDGEN=bindgen CC=clang scripts/rust_is_available.sh
    scripts/rust_is_available.sh: 19: arithmetic expression: expecting primary: "100000 *  + 100 *  + "
    ***
    *** Please see Documentation/rust/quick-start.rst for details
    *** on how to set up the Rust support.
    ***

Thus add an explicit check and a proper message for unexpected
output from the called command.

Similarly, do so for the `libclang` version parsing, too.

Link: https://lore.kernel.org/rust-for-linux/CAK7LNAQYk6s11MASRHW6oxtkqF00EJVqhHOP=5rynWt-QDUsXw@mail.gmail.com/
Reviewed-by: Nathan Chancellor 
Reviewed-by: Martin Rodriguez Reboredo 
Link: https://lore.kernel.org/r/20230616001631.463536-11-ojeda@kernel.org
Signed-off-by: Miguel Ojeda

kbuild: rust_is_available: handle failures calling `$RUSTC`/`$BINDGEN`

2023-08-09T23:18:34+00:00

The script already checks if `$RUSTC` and `$BINDGEN` exists via
`command`, but the environment variables may point to a
non-executable file, or the programs may fail for some other reason.
While the script successfully exits with a failure as it should,
the error given can be quite confusing depending on the shell and
the behavior of its `command`. For instance, with `dash`:

    $ RUSTC=./mm BINDGEN=bindgen CC=clang scripts/rust_is_available.sh
    scripts/rust_is_available.sh: 19: arithmetic expression: expecting primary: "100000 *  + 100 *  + "

Thus detect failure exit codes when calling `$RUSTC` and `$BINDGEN` and
print a better message, in a similar way to what we do when extracting
the `libclang` version found by `bindgen`.

Link: https://lore.kernel.org/rust-for-linux/CAK7LNAQYk6s11MASRHW6oxtkqF00EJVqhHOP=5rynWt-QDUsXw@mail.gmail.com/
Reviewed-by: Nathan Chancellor 
Reviewed-by: Martin Rodriguez Reboredo 
Link: https://lore.kernel.org/r/20230616001631.463536-10-ojeda@kernel.org
Signed-off-by: Miguel Ojeda

kbuild: rust_is_available: normalize version matching

2023-08-09T23:18:34+00:00

In order to match the version string, `sed` is used in a couple
cases, and `grep` and `head` in a couple others.

Make the script more consistent and easier to understand by
using the same method, `sed`, for all of them.

This makes the version matching also a bit more strict for
the changed cases, since the strings `rustc ` and `bindgen `
will now be required, which should be fine since `rustc`
complains if one attempts to call it with another program
name, and `bindgen` uses a hardcoded string.

In addition, clarify why one of the existing `sed` commands
does not provide an address like the others.

Reviewed-by: Nathan Chancellor 
Reviewed-by: Masahiro Yamada 
Reviewed-by: Martin Rodriguez Reboredo 
Link: https://lore.kernel.org/r/20230616001631.463536-9-ojeda@kernel.org
Signed-off-by: Miguel Ojeda

kbuild: rust_is_available: fix confusion when a version appears in the path

2023-08-09T23:18:22+00:00

`bindgen`'s output for `libclang`'s version check contains paths, which
in turn may contain strings that look like version numbers [1][2]:

    .../6.1.0-dev/.../rust_is_available_bindgen_libclang.h:2:9: warning: clang version 11.1.0  [-W#pragma-messages], err: false

which the script will pick up as the version instead of the latter.

It is also the case that versions may appear after the actual version
(e.g. distribution's version text), which was the reason behind `head` [3]:

    .../rust-is-available-bindgen-libclang.h:2:9: warning: clang version 13.0.0 (Fedora 13.0.0-3.fc35) [-W#pragma-messages], err: false

Thus instead ask for a match after the `clang version` string.

Reported-by: Jordan Isaacs 
Closes: https://github.com/Rust-for-Linux/linux/issues/942 [1]
Reported-by: "Ethan D. Twardy" 
Closes: https://lore.kernel.org/rust-for-linux/20230528131802.6390-2-ethan.twardy@gmail.com/ [2]
Reported-by: Tiago Lam 
Closes: https://github.com/Rust-for-Linux/linux/pull/789 [3]
Fixes: 78521f3399ab ("scripts: add `rust_is_available.sh`")
Reviewed-by: Martin Rodriguez Reboredo 
Reviewed-by: Ethan Twardy 
Tested-by: Ethan Twardy 
Reviewed-by: Nathan Chancellor 
Link: https://lore.kernel.org/r/20230616001631.463536-8-ojeda@kernel.org
Signed-off-by: Miguel Ojeda