diff options
Diffstat (limited to 'src/doc/rustc-dev-guide/src')
45 files changed, 842 insertions, 328 deletions
diff --git a/src/doc/rustc-dev-guide/src/SUMMARY.md b/src/doc/rustc-dev-guide/src/SUMMARY.md index 360265c0e..adc397fd8 100644 --- a/src/doc/rustc-dev-guide/src/SUMMARY.md +++ b/src/doc/rustc-dev-guide/src/SUMMARY.md @@ -8,7 +8,7 @@ # Building and debugging `rustc` -- [How to Build and Run the Compiler](./building/how-to-build-and-run.md) +- [How to build and run the compiler](./building/how-to-build-and-run.md) - [Prerequisites](./building/prerequisites.md) - [Suggested Workflows](./building/suggested.md) - [Distribution artifacts](./building/build-install-distribution-artifacts.md) @@ -25,7 +25,7 @@ - [Test headers](./tests/headers.md) - [Performance testing](./tests/perf.md) - [Crater](./tests/crater.md) -- [Debugging the Compiler](./compiler-debugging.md) +- [Debugging the compiler](./compiler-debugging.md) - [Using the tracing/logging instrumentation](./tracing.md) - [Profiling the compiler](./profiling.md) - [with the linux perf tool](./profiling/with_perf.md) @@ -57,7 +57,7 @@ # High-level Compiler Architecture - [Prologue](./part-2-intro.md) -- [Overview of the Compiler](./overview.md) +- [Overview of the compiler](./overview.md) - [The compiler source code](./compiler-src.md) - [Bootstrapping](./building/bootstrapping.md) - [Queries: demand-driven compilation](./query.md) @@ -75,7 +75,7 @@ - [Prologue](./part-3-intro.md) - [Command-line arguments](./cli.md) -- [The Rustc Driver and Interface](./rustc-driver.md) +- [rustc_driver and rustc_interface](./rustc-driver.md) - [Example: Type checking](./rustc-driver-interacting-with-the-ast.md) - [Example: Getting diagnostics](./rustc-driver-getting-diagnostics.md) - [Syntax and the AST](./syntax-intro.md) @@ -95,7 +95,7 @@ - [MIR construction](./mir/construction.md) - [MIR visitor and traversal](./mir/visitor.md) - [MIR passes: getting the MIR for a function](./mir/passes.md) -- [Identifiers in the Compiler](./identifiers.md) +- [Identifiers in the compiler](./identifiers.md) - [Closure expansion](./closure.md) - [Inline assembly](./asm.md) @@ -117,6 +117,10 @@ - [Lowering to logic](./traits/lowering-to-logic.md) - [Goals and clauses](./traits/goals-and-clauses.md) - [Canonical queries](./traits/canonical-queries.md) + - [Next-gen trait solving](./solve/trait-solving.md) + - [The solver](./solve/the-solver.md) + - [Canonicalization](./solve/canonicalization.md) + - [Coinduction](./solve/coinduction.md) - [Type checking](./type-checking.md) - [Method Lookup](./method-lookup.md) - [Variance](./variance.md) @@ -164,7 +168,7 @@ - [Profile-guided Optimization](./profile-guided-optimization.md) - [LLVM Source-Based Code Coverage](./llvm-coverage-instrumentation.md) - [Sanitizers Support](./sanitizers.md) -- [Debugging Support in the Rust Compiler](./debugging-support-in-rustc.md) +- [Debugging support in the Rust compiler](./debugging-support-in-rustc.md) --- diff --git a/src/doc/rustc-dev-guide/src/appendix/glossary.md b/src/doc/rustc-dev-guide/src/appendix/glossary.md index 42306dc1c..019d31138 100644 --- a/src/doc/rustc-dev-guide/src/appendix/glossary.md +++ b/src/doc/rustc-dev-guide/src/appendix/glossary.md @@ -23,7 +23,7 @@ Term | Meaning <span id="double-ptr">double pointer</span> | A pointer with additional metadata. See "fat pointer" for more. <span id="drop-glue">drop glue</span> | (internal) compiler-generated instructions that handle calling the destructors (`Drop`) for data types. <span id="dst">DST</span> | Short for Dynamically-Sized Type, this is a type for which the compiler cannot statically know the size in memory (e.g. `str` or `[u8]`). Such types don't implement `Sized` and cannot be allocated on the stack. They can only occur as the last field in a struct. They can only be used behind a pointer (e.g. `&str` or `&[u8]`). -<span id="ebl">early-bound lifetime</span> | A lifetime region that is substituted at its definition site. Bound in an item's `Generics` and substituted using a `Substs`. Contrast with **late-bound lifetime**. ([see more](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.RegionKind.html#bound-regions)) +<span id="ebl">early-bound lifetime</span> | A lifetime region that is substituted at its definition site. Bound in an item's `Generics` and substituted using a `Substs`. Contrast with **late-bound lifetime**. ([see more](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.RegionKind.html#bound-regions)) <span id="empty-type">empty type</span> | see "uninhabited type". <span id="fat-ptr">fat pointer</span> | A two word value carrying the address of some value, along with some further information necessary to put the value to use. Rust includes two kinds of "fat pointers": references to slices, and trait objects. A reference to a slice carries the starting address of the slice and its length. A trait object carries a value's address and a pointer to the trait's implementation appropriate to that value. "Fat pointers" are also known as "wide pointers", and "double pointers". <span id="free-var">free variable</span> | A "free variable" is one that is not bound within an expression or term; see [the background chapter for more](./background.md#free-vs-bound) @@ -42,7 +42,7 @@ Term | Meaning <span id="irlo">IRLO</span> | `IRLO` or `irlo` is sometimes used as an abbreviation for [internals.rust-lang.org](https://internals.rust-lang.org). <span id="item">item</span> | A kind of "definition" in the language, such as a static, const, use statement, module, struct, etc. Concretely, this corresponds to the `Item` type. <span id="lang-item">lang item</span> | Items that represent concepts intrinsic to the language itself, such as special built-in traits like `Sync` and `Send`; or traits representing operations such as `Add`; or functions that are called by the compiler. ([see more](https://doc.rust-lang.org/1.9.0/book/lang-items.html)) -<span id="lbl">late-bound lifetime</span> | A lifetime region that is substituted at its call site. Bound in a HRTB and substituted by specific functions in the compiler, such as `liberate_late_bound_regions`. Contrast with **early-bound lifetime**. ([see more](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.RegionKind.html#bound-regions)) +<span id="lbl">late-bound lifetime</span> | A lifetime region that is substituted at its call site. Bound in a HRTB and substituted by specific functions in the compiler, such as `liberate_late_bound_regions`. Contrast with **early-bound lifetime**. ([see more](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.RegionKind.html#bound-regions)) <span id="local-crate">local crate</span> | The crate currently being compiled. This is in contrast to "upstream crates" which refer to dependencies of the local crate. <span id="lto">LTO</span> | Short for Link-Time Optimizations, this is a set of optimizations offered by LLVM that occur just before the final binary is linked. These include optimizations like removing functions that are never used in the final program, for example. _ThinLTO_ is a variant of LTO that aims to be a bit more scalable and efficient, but possibly sacrifices some optimizations. You may also read issues in the Rust repo about "FatLTO", which is the loving nickname given to non-Thin LTO. LLVM documentation: [here][lto] and [here][thinlto]. <span id="llvm">[LLVM]</span> | (actually not an acronym :P) an open-source compiler backend. It accepts LLVM IR and outputs native binaries. Various languages (e.g. Rust) can then implement a compiler front-end that outputs LLVM IR and use LLVM to compile to all the platforms LLVM supports. diff --git a/src/doc/rustc-dev-guide/src/asm.md b/src/doc/rustc-dev-guide/src/asm.md index 3b6b5e093..b19f2ad46 100644 --- a/src/doc/rustc-dev-guide/src/asm.md +++ b/src/doc/rustc-dev-guide/src/asm.md @@ -277,7 +277,7 @@ pub enum InlineAsmOperand<'tcx> { } ``` -As part of HAIR lowering, `InOut` and `SplitInOut` operands are lowered to a split form with a +As part of THIR lowering, `InOut` and `SplitInOut` operands are lowered to a split form with a separate `in_value` and `out_place`. Semantically, the `InlineAsm` terminator is similar to the `Call` terminator except that it has diff --git a/src/doc/rustc-dev-guide/src/ast-validation.md b/src/doc/rustc-dev-guide/src/ast-validation.md index ff549ca62..f3ac35e8a 100644 --- a/src/doc/rustc-dev-guide/src/ast-validation.md +++ b/src/doc/rustc-dev-guide/src/ast-validation.md @@ -1,6 +1,36 @@ # AST Validation -AST validation is the process of checking various correctness properties about -the AST after macro expansion. +<!-- toc --> -**TODO**: write this chapter. [#656](https://github.com/rust-lang/rustc-dev-guide/issues/656) +## About + +_AST validation_ is a separate AST pass that visits each +item in the tree and performs simple checks. This pass +doesn't perform any complex analysis, type checking or +name resolution. + +Before performing any validation, the compiler first expands +the macros. Then this pass performs validations to check +that each AST item is in the correct state. And when this pass +is done, the compiler runs the crate resolution pass. + +## Validations + +Validations are defined in `AstValidator` type, which +itself is located in `rustc_ast_passes` crate. This +type implements various simple checks which emit errors +when certain language rules are broken. + +In addition, `AstValidator` implements `Visitor` trait +that defines how to visit AST items (which can be functions, +traits, enums, etc). + +For each item, visitor performs specific checks. For +example, when visiting a function declaration, +`AstValidator` checks that the function has: + +* no more than `u16::MAX` parameters; +* c-variadic functions are declared with at least one named argument; +* c-variadic argument goes the last in the declaration; +* documentation comments aren't applied to function parameters; +* and other validations. diff --git a/src/doc/rustc-dev-guide/src/backend/backend-agnostic.md b/src/doc/rustc-dev-guide/src/backend/backend-agnostic.md index ea50cd754..c5eb57e64 100644 --- a/src/doc/rustc-dev-guide/src/backend/backend-agnostic.md +++ b/src/doc/rustc-dev-guide/src/backend/backend-agnostic.md @@ -2,16 +2,21 @@ <!-- toc --> -As of <!-- date-check --> Aug 2022, `rustc_codegen_ssa` provides an -abstract interface for all backends to implement, to allow other codegen -backends (e.g. [Cranelift]). +[`rustc_codegen_ssa`] +provides an abstract interface for all backends to implement, +namely LLVM, [Cranelift], and [GCC]. -[Cranelift]: https://github.com/bytecodealliance/wasmtime/tree/HEAD/cranelift +[Cranelift]: https://github.com/bjorn3/rustc_codegen_cranelift +[GCC]: https://github.com/rust-lang/rustc_codegen_gcc +[`rustc_codegen_ssa`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/index.html -# Refactoring of `rustc_codegen_llvm` +Below is some background information on the refactoring that created this +abstract interface. + +## Refactoring of `rustc_codegen_llvm` by Denis Merigoux, October 23rd 2018 -## State of the code before the refactoring +### State of the code before the refactoring All the code related to the compilation of MIR into LLVM IR was contained inside the `rustc_codegen_llvm` crate. Here is the breakdown of the most @@ -51,7 +56,7 @@ While the LLVM-specific code will be left in `rustc_codegen_llvm`, all the new traits and backend-agnostic code will be moved in `rustc_codegen_ssa` (name suggestion by @eddyb). -## Generic types and structures +### Generic types and structures @irinagpopa started to parametrize the types of `rustc_codegen_llvm` by a generic `Value` type, implemented in LLVM by a reference `&'ll Value`. This @@ -103,7 +108,7 @@ of the backend and it makes more sense to leave their definition to the backend implementor than to allow just a narrow spot via a generic field for the backend's context. -## Traits and interface +### Traits and interface Because they have to be defined by the backend, `CodegenCx` and `Builder` will be the structures implementing all the traits defining the backend's interface. @@ -170,7 +175,7 @@ called. However, when implementing a Rust backend for `rustc`, these methods will need information from `CodegenCx`, hence the additional parameter (unused in the LLVM implementation of the trait). -## State of the code after the refactoring +### State of the code after the refactoring The traits offer an API which is very similar to the API of LLVM. This is not the best solution since LLVM has a very special way of doing things: when diff --git a/src/doc/rustc-dev-guide/src/backend/monomorph.md b/src/doc/rustc-dev-guide/src/backend/monomorph.md index 4a0b55f62..cbc56acfc 100644 --- a/src/doc/rustc-dev-guide/src/backend/monomorph.md +++ b/src/doc/rustc-dev-guide/src/backend/monomorph.md @@ -133,7 +133,7 @@ the substitutions with the identity parameter before being added to the set of collected items - thereby reducing the copies from two (`[u16, u32]` and `[u64, u32]`) to one (`[A, u32]`). -`unused_generic_params` will also invoked during code generation when the +`unused_generic_params` will also be invoked during code generation when the symbol name for `foo` is being computed for use in the callsites of `foo` (which have the regular substitutions present, otherwise there would be a symbol mismatch between the caller and the function). diff --git a/src/doc/rustc-dev-guide/src/backend/updating-llvm.md b/src/doc/rustc-dev-guide/src/backend/updating-llvm.md index 38fbb2e44..a02011149 100644 --- a/src/doc/rustc-dev-guide/src/backend/updating-llvm.md +++ b/src/doc/rustc-dev-guide/src/backend/updating-llvm.md @@ -2,7 +2,7 @@ <!-- toc --> -<!-- date-check: Aug 2022 --> +<!-- date-check: Feb 2023 --> There is no formal policy about when to update LLVM or what it can be updated to, but a few guidelines are applied: @@ -16,9 +16,9 @@ but a few guidelines are applied: There are two reasons we would want to update LLVM: -* A bug could have been fixed! Often we find bugs in the compiler and fix - them upstream in LLVM. We'll want to pull fixes back to the compiler itself as - they're merged upstream. +* A bug could have been fixed! + Note that if we are the ones who fixed such a bug, + we prefer to upstream it, then pull it back for use by rustc. * LLVM itself may have a new release. @@ -172,9 +172,11 @@ so let's go through each in detail. Ideally the above instructions are pretty smooth, but here's some caveats to keep in mind while going through them: -* LLVM bugs are hard to find, don't hesitate to ask for help! Bisection is - definitely your friend here (yes LLVM takes forever to build, yet bisection is - still your friend) +* LLVM bugs are hard to find, don't hesitate to ask for help! + Bisection is definitely your friend here + (yes LLVM takes forever to build, yet bisection is still your friend). + Note that you can make use of [Dev Desktops], + which is an initiative to provide the contributors with remote access to powerful hardware. * If you've got general questions, [wg-llvm] can help you out. * Creating branches is a privileged operation on GitHub, so you'll need someone with write access to create the branches for you most likely. @@ -184,3 +186,4 @@ keep in mind while going through them: [llvm/llvm-project repository]: https://github.com/llvm/llvm-project [`llvm-wrapper`]: https://github.com/rust-lang/rust/tree/master/compiler/rustc_llvm/llvm-wrapper [wg-llvm]: https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm +[Dev Desktops]: https://forge.rust-lang.org/infra/docs/dev-desktop.html diff --git a/src/doc/rustc-dev-guide/src/borrow_check/region_inference/lifetime_parameters.md b/src/doc/rustc-dev-guide/src/borrow_check/region_inference/lifetime_parameters.md index 2dbbc0c1e..bd8a07843 100644 --- a/src/doc/rustc-dev-guide/src/borrow_check/region_inference/lifetime_parameters.md +++ b/src/doc/rustc-dev-guide/src/borrow_check/region_inference/lifetime_parameters.md @@ -43,8 +43,8 @@ only variant of [`ty::RegionKind`] that we use is the [`ReVar`] variant. These region variables are broken into two major categories, based on their index: -[`ty::RegionKind`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.RegionKind.html -[`ReVar`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.RegionKind.html#variant.ReVar +[`ty::RegionKind`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.RegionKind.html +[`ReVar`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.RegionKind.html#variant.ReVar - 0..N: universal regions -- the ones we are discussing here. In this case, the code must be correct with respect to any value of those diff --git a/src/doc/rustc-dev-guide/src/building/bootstrapping.md b/src/doc/rustc-dev-guide/src/building/bootstrapping.md index 543c68c2e..fe34cb500 100644 --- a/src/doc/rustc-dev-guide/src/building/bootstrapping.md +++ b/src/doc/rustc-dev-guide/src/building/bootstrapping.md @@ -1,8 +1,7 @@ -# Bootstrapping the Compiler +# Bootstrapping the compiler <!-- toc --> - [*Bootstrapping*][boot] is the process of using a compiler to compile itself. More accurately, it means using an older compiler to compile a newer version of the same compiler. @@ -16,6 +15,11 @@ version. This is exactly how `x.py` works: it downloads the current beta release of rustc, then uses it to compile the new compiler. +Note that this documentation mostly covers user-facing information. See +[bootstrap/README.md][bootstrap-internals] to read about bootstrap internals. + +[bootstrap-internals]: https://github.com/rust-lang/rust/blob/master/src/bootstrap/README.md + ## Stages of bootstrapping Compiling `rustc` is done in stages. Here's a diagram, adapted from Joshua Nelson's @@ -86,7 +90,7 @@ because one must first build the new compiler with an older compiler and then use that to build the new compiler with itself. For development, you usually only want the `stage1` compiler, which you can build with `./x.py build library`. -See [Building the Compiler](./how-to-build-and-run.html#building-the-compiler). +See [Building the compiler](./how-to-build-and-run.html#building-the-compiler). ### Stage 3 @@ -135,31 +139,6 @@ bootstrapping the compiler. [intrinsics]: ../appendix/glossary.md#intrinsic [ocaml-compiler]: https://github.com/rust-lang/rust/tree/ef75860a0a72f79f97216f8aaa5b388d98da6480/src/boot -## Contributing to bootstrap - -When you use the bootstrap system, you'll call it through `x.py`. -However, most of the code lives in `src/bootstrap`. -`bootstrap` has a difficult problem: it is written in Rust, but yet it is run -before the Rust compiler is built! To work around this, there are two -components of bootstrap: the main one written in rust, and `bootstrap.py`. -`bootstrap.py` is what gets run by `x.py`. It takes care of downloading the -`stage0` compiler, which will then build the bootstrap binary written in -Rust. - -Because there are two separate codebases behind `x.py`, they need to -be kept in sync. In particular, both `bootstrap.py` and the bootstrap binary -parse `config.toml` and read the same command line arguments. `bootstrap.py` -keeps these in sync by setting various environment variables, and the -programs sometimes have to add arguments that are explicitly ignored, to be -read by the other. - -### Adding a setting to config.toml - -This section is a work in progress. In the meantime, you can see an example -contribution [here][bootstrap-build]. - -[bootstrap-build]: https://github.com/rust-lang/rust/pull/71994 - ## Understanding stages of bootstrap ### Overview @@ -407,44 +386,51 @@ usually means something is quite wrong -- or you're trying to compile e.g. the unlikely case that you actually need to invoke rustc in such a situation, you can tell the bootstrap shim to print all env variables by adding `-vvv` to your `x.py` command. -### Directories and artifacts generated by `bootstrap` - -This is an incomplete reference for the outputs generated by bootstrap: - -| Stage 0 Action | Output | -|-----------------------------------------------------------|----------------------------------------------| -| `beta` extracted | `build/HOST/stage0` | -| `stage0` builds `bootstrap` | `build/bootstrap` | -| `stage0` builds `test`/`std` | `build/HOST/stage0-std/TARGET` | -| copy `stage0-std` (HOST only) | `build/HOST/stage0-sysroot/lib/rustlib/HOST` | -| `stage0` builds `rustc` with `stage0-sysroot` | `build/HOST/stage0-rustc/HOST` | -| copy `stage0-rustc` (except executable) | `build/HOST/stage0-sysroot/lib/rustlib/HOST` | -| build `llvm` | `build/HOST/llvm` | -| `stage0` builds `codegen` with `stage0-sysroot` | `build/HOST/stage0-codegen/HOST` | -| `stage0` builds `rustdoc`, `clippy`, `miri`, with `stage0-sysroot` | `build/HOST/stage0-tools/HOST` | - -`--stage=0` stops here. - -| Stage 1 Action | Output | -|-----------------------------------------------------|---------------------------------------| -| copy (uplift) `stage0-rustc` executable to `stage1` | `build/HOST/stage1/bin` | -| copy (uplift) `stage0-codegen` to `stage1` | `build/HOST/stage1/lib` | -| copy (uplift) `stage0-sysroot` to `stage1` | `build/HOST/stage1/lib` | -| `stage1` builds `test`/`std` | `build/HOST/stage1-std/TARGET` | -| copy `stage1-std` (HOST only) | `build/HOST/stage1/lib/rustlib/HOST` | -| `stage1` builds `rustc` | `build/HOST/stage1-rustc/HOST` | -| copy `stage1-rustc` (except executable) | `build/HOST/stage1/lib/rustlib/HOST` | -| `stage1` builds `codegen` | `build/HOST/stage1-codegen/HOST` | - -`--stage=1` stops here. - -| Stage 2 Action | Output | -|--------------------------------------------------------|-----------------------------------------------------------------| -| copy (uplift) `stage1-rustc` executable | `build/HOST/stage2/bin` | -| copy (uplift) `stage1-sysroot` | `build/HOST/stage2/lib and build/HOST/stage2/lib/rustlib/HOST` | -| `stage2` builds `test`/`std` (not HOST targets) | `build/HOST/stage2-std/TARGET` | -| copy `stage2-std` (not HOST targets) | `build/HOST/stage2/lib/rustlib/TARGET` | -| `stage2` builds `rustdoc`, `clippy`, `miri` | `build/HOST/stage2-tools/HOST` | -| copy `rustdoc` | `build/HOST/stage2/bin` | - -`--stage=2` stops here. +Finally, bootstrap makes use of the [cc-rs crate] which has [its own +method][env-vars] of configuring C compilers and C flags via environment +variables. + +[cc-rs crate]: https://github.com/rust-lang/cc-rs +[env-vars]: https://github.com/rust-lang/cc-rs#external-configuration-via-environment-variables + +## Clarification of build command's stdout + +In this part, we will investigate the build command's stdout in an action +(similar, but more detailed and complete documentation compare to topic above). +When you execute `x.py build --dry-run` command, the build output will be something +like the following: + +```text +Building stage0 library artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu) +Copying stage0 library from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu) +Building stage0 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu) +Copying stage0 rustc from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu) +Assembling stage1 compiler (x86_64-unknown-linux-gnu) +Building stage1 library artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu) +Copying stage1 library from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu) +Building stage1 tool rust-analyzer-proc-macro-srv (x86_64-unknown-linux-gnu) +Building rustdoc for stage1 (x86_64-unknown-linux-gnu) +``` + +### Building stage0 {std,compiler} artifacts + +These steps use the provided (downloaded, usually) compiler to compile the +local Rust source into libraries we can use. + +### Copying stage0 {std,rustc} + +This copies the library and compiler artifacts from Cargo into +`stage0-sysroot/lib/rustlib/{target-triple}/lib` + +### Assembling stage1 compiler + +This copies the libraries we built in "building stage0 ... artifacts" into +the stage1 compiler's lib directory. These are the host libraries that the +compiler itself uses to run. These aren't actually used by artifacts the new +compiler generates. This step also copies the rustc and rustdoc binaries we +generated into `build/$HOST/stage/bin`. + +The stage1/bin/rustc is a fully functional compiler, but it doesn't yet have +any libraries to link built binaries or libraries to. The next 3 steps will +provide those libraries for it; they are mostly equivalent to constructing +the stage1/bin compiler so we don't go through them individually. diff --git a/src/doc/rustc-dev-guide/src/building/how-to-build-and-run.md b/src/doc/rustc-dev-guide/src/building/how-to-build-and-run.md index 24786c0c0..6651b3691 100644 --- a/src/doc/rustc-dev-guide/src/building/how-to-build-and-run.md +++ b/src/doc/rustc-dev-guide/src/building/how-to-build-and-run.md @@ -1,4 +1,4 @@ -# How to Build and Run the Compiler +# How to build and run the compiler The compiler is built using a tool called `x.py`. You will need to have Python installed to run it. @@ -52,7 +52,7 @@ If you have already built `rustc` and you change settings related to LLVM, then execute `rm -rf build` for subsequent configuration changes to take effect. Note that `./x.py clean` will not cause a rebuild of LLVM. -## Building the Compiler +## Building the compiler Note that building will require a relatively large amount of storage space. You may want to have upwards of 10 or 15 gigabytes available to build the compiler. @@ -122,16 +122,11 @@ you will likely need to build at some point; for example, if you want to run the entire test suite). ```bash -rustup toolchain link stage1 build/<host-triple>/stage1 -rustup toolchain link stage2 build/<host-triple>/stage2 +rustup toolchain link stage0 build/host/stage0-sysroot # beta compiler + stage0 std +rustup toolchain link stage1 build/host/stage1 +rustup toolchain link stage2 build/host/stage2 ``` -The `<host-triple>` would typically be one of the following: - -- Linux: `x86_64-unknown-linux-gnu` -- Mac: `x86_64-apple-darwin` or `aarch64-apple-darwin` -- Windows: `x86_64-pc-windows-msvc` - Now you can run the `rustc` you built with. If you run with `-vV`, you should see a version number ending in `-dev`, indicating a build from your local environment: diff --git a/src/doc/rustc-dev-guide/src/building/prerequisites.md b/src/doc/rustc-dev-guide/src/building/prerequisites.md index 8fee0a4b5..3dc2ea934 100644 --- a/src/doc/rustc-dev-guide/src/building/prerequisites.md +++ b/src/doc/rustc-dev-guide/src/building/prerequisites.md @@ -29,7 +29,7 @@ laptop. We suggest downloading LLVM from CI so you don't have to build it from s Like `cargo`, the build system will use as many cores as possible. Sometimes this can cause you to run low on memory. You can use `-j` to adjust the number -concurrent jobs. If a full build takes more than ~45 minutes to an hour, you +of concurrent jobs. If a full build takes more than ~45 minutes to an hour, you are probably spending most of the time swapping memory in and out; try using `-j1`. diff --git a/src/doc/rustc-dev-guide/src/building/suggested.md b/src/doc/rustc-dev-guide/src/building/suggested.md index f81daa5bc..2e2592094 100644 --- a/src/doc/rustc-dev-guide/src/building/suggested.md +++ b/src/doc/rustc-dev-guide/src/building/suggested.md @@ -19,51 +19,25 @@ You can also install the hook as a step of running `./x.py setup`! ## Configuring `rust-analyzer` for `rustc` +### Visual Studio Code + `rust-analyzer` can help you check and format your code whenever you save a file. By default, `rust-analyzer` runs the `cargo check` and `rustfmt` commands, but you can override these commands to use more adapted versions -of these tools when hacking on `rustc`. For example, for Visual Studio Code, -you can write: <!-- date-check: nov 2022 --><!-- the date comment is for the edition below --> - -```JSON -{ - "rust-analyzer.checkOnSave.overrideCommand": [ - "python3", - "x.py", - "check", - "--json-output" - ], - "rust-analyzer.rustfmt.overrideCommand": [ - "./build/host/stage0/bin/rustfmt", - "--edition=2021" - ], - "rust-analyzer.procMacro.server": "./build/host/stage0/libexec/rust-analyzer-proc-macro-srv", - "rust-analyzer.procMacro.enable": true, - "rust-analyzer.cargo.buildScripts.enable": true, - "rust-analyzer.cargo.buildScripts.invocationLocation": "root", - "rust-analyzer.cargo.buildScripts.invocationStrategy": "once", - "rust-analyzer.cargo.buildScripts.overrideCommand": [ - "python3", - "x.py", - "check", - "--json-output" - ], - "rust-analyzer.cargo.sysroot": "./build/host/stage0-sysroot", - "rust-analyzer.rustc.source": "./Cargo.toml", -} -``` - -in your `.vscode/settings.json` file. This will ask `rust-analyzer` to use -`./x.py check` to check the sources, and the stage 0 rustfmt to format them. +of these tools when hacking on `rustc`. For example, `x.py setup` will prompt +you to create a `.vscode/settings.json` file which will configure Visual Studio code. +This will ask `rust-analyzer` to use `./x.py check` to check the sources, and the +stage 0 rustfmt to format them. +The recommended `rust-analyzer` settings live at [`src/etc/vscode_settings.json`]. If you have enough free disk space and you would like to be able to run `x.py` commands while rust-analyzer runs in the background, you can also add `--build-dir build-rust-analyzer` to the `overrideCommand` to avoid x.py locking. If you're running `coc.nvim`, you can use `:CocLocalConfig` to create a -`.vim/coc-settings.json` and enter the same settings as above, but replacing -`editor.formatOnSave: true,` with -`"coc.preferences.formatOnSaveFiletypes": ["rust"],`. +`.vim/coc-settings.json` and copy the settings from [`src/etc/vscode_settings.json`]. + +[`src/etc/vscode_settings.json`]: https://github.com/rust-lang/rust/blob/master/src/etc/vscode_settings.json If running `./x.py check` on save is inconvenient, in VS Code you can use a [Build Task] instead: @@ -87,6 +61,35 @@ Task] instead: [Build Task]: https://code.visualstudio.com/docs/editor/tasks + +### Neovim + +For Neovim users there are several options for configuring for rustc. The easiest way is by using +[neoconf.nvim](https://github.com/folke/neoconf.nvim/), which allows for project-local +configuration files with the native LSP. The steps for how to use it are below. Note that requires +Rust-Analyzer to already be configured with Neovim. Steps for this can be +[found here](https://rust-analyzer.github.io/manual.html#nvim-lsp). + +1. First install the plugin. This can be done by following the steps in the README. +2. Run `x.py setup`, which will have a prompt for it to create a `.vscode/settings.json` file. +`neoconf` is able to read and update Rust-Analyzer settings automatically when the project is +opened when this file is detected. + +If you're running `coc.nvim`, you can use `:CocLocalConfig` to create a +`.vim/coc-settings.json` and copy the settings from +[this file](https://github.com/rust-lang/rust/blob/master/src/etc/vscode_settings.json). + +Another way is without a plugin, and creating your own logic in your configuration. To do this you +must translate the JSON to Lua yourself. The translation is 1:1 and fairly straight-forward. It +must be put in the `["rust-analyzer"]` key of the setup table, which is +[shown here](https://github.com/neovim/nvim-lspconfig/blob/master/doc/server_configurations.md#rust_analyzer) + +If you would like to use the build task that is described above, you may either make your own +command in your config, or you can install a plugin such as +[overseer.nvim](https://github.com/stevearc/overseer.nvim) that can [read VSCode's `task.json` +files](https://github.com/stevearc/overseer.nvim/blob/master/doc/guides.md#vs-code-tasks), and +follow the same instructions as above. + ## Check, check, and check again When doing simple refactorings, it can be useful to run `./x.py check` @@ -175,6 +178,27 @@ You can also use `--keep-stage 1` when running tests. Something like this: - Initial test run: `./x.py test tests/ui` - Subsequent test run: `./x.py test tests/ui --keep-stage 1` +## Using incremental compilation + +You can further enable the `--incremental` flag to save additional +time in subsequent rebuilds: + +```bash +./x.py test tests/ui --incremental --test-args issue-1234 +``` + +If you don't want to include the flag with every command, you can +enable it in the `config.toml`: + +```toml +[rust] +incremental = true +``` + +Note that incremental compilation will use more disk space than usual. +If disk space is a concern for you, you might want to check the size +of the `build` directory from time to time. + ## Fine-tuning optimizations Setting `optimize = false` makes the compiler too slow for tests. However, to diff --git a/src/doc/rustc-dev-guide/src/closure.md b/src/doc/rustc-dev-guide/src/closure.md index 5746fd4de..e356f415d 100644 --- a/src/doc/rustc-dev-guide/src/closure.md +++ b/src/doc/rustc-dev-guide/src/closure.md @@ -155,7 +155,7 @@ This uses the stage1 compiler and enables `debug!` logging for the The other option is to step through the code using lldb or gdb. -1. `rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs` +1. `rust-lldb build/host/stage1/bin/rustc test.rs` 2. In lldb: 1. `b upvar.rs:134` // Setting the breakpoint on a certain line in the upvar.rs file` 2. `r` // Run the program until it hits the breakpoint diff --git a/src/doc/rustc-dev-guide/src/compiler-debugging.md b/src/doc/rustc-dev-guide/src/compiler-debugging.md index eac9aeb6d..6052ea58a 100644 --- a/src/doc/rustc-dev-guide/src/compiler-debugging.md +++ b/src/doc/rustc-dev-guide/src/compiler-debugging.md @@ -1,5 +1,4 @@ # Debugging the compiler -[debugging]: #debugging <!-- toc --> @@ -185,7 +184,7 @@ stack backtrace: Cool, now I have a backtrace for the error! -## Getting the the error creation location +## Getting the error creation location `-Z track-diagnostics` can help figure out where errors are emitted. It uses `#[track_caller]` for this and prints its location alongside the error: @@ -341,3 +340,37 @@ error: aborting due to previous error ``` [`Layout`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_target/abi/struct.Layout.html + + +## Configuring CodeLLDB for debugging `rustc` + +If you are using VSCode, and have edited your `config.toml` to request debugging +level 1 or 2 for the parts of the code you're interested in, then you should be +able to use the [CodeLLDB] extension in VSCode to debug it. + +Here is a sample `launch.json` file, being used to run a stage 1 compiler direct +from the directory where it is built (does not have to be "installed"): + +```javascript +// .vscode/launch.json +{ + "version": "0.2.0", + "configurations": [ + { + "type": "lldb", + "request": "launch", + "name": "Launch", + "args": [], // array of string command-line arguments to pass to compiler + "program": "${workspaceFolder}/build/host/stage1/bin/rustc", + "windows": { // applicable if using windows + "program": "${workspaceFolder}/build/host/stage1/bin/rustc.exe" + }, + "cwd": "${workspaceFolder}", // current working directory at program start + "stopOnEntry": false, + "sourceLanguages": ["rust"] + } + ] + } +``` + +[CodeLLDB]: https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb diff --git a/src/doc/rustc-dev-guide/src/compiler-src.md b/src/doc/rustc-dev-guide/src/compiler-src.md index 1f55a9d09..9c7d0bb2e 100644 --- a/src/doc/rustc-dev-guide/src/compiler-src.md +++ b/src/doc/rustc-dev-guide/src/compiler-src.md @@ -27,8 +27,10 @@ The repository consists of three main directories: `proc_macro`, `test`), as well as the Rust runtime (`backtrace`, `rtstartup`, `lang_start`). +- `tests/` contains the compiler tests. + - `src/` contains the source code for rustdoc, clippy, cargo, the build system, - compiler tests, language docs, etc. + language docs, etc. ## Compiler diff --git a/src/doc/rustc-dev-guide/src/constants.md b/src/doc/rustc-dev-guide/src/constants.md index a33a283f3..91d21bd32 100644 --- a/src/doc/rustc-dev-guide/src/constants.md +++ b/src/doc/rustc-dev-guide/src/constants.md @@ -78,5 +78,5 @@ the constant doesn't use them in any way. This can cause [`ty::Const`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.Const.html [`ty::ConstKind`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.ConstKind.html -[`ty::TyKind`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html +[`ty::TyKind`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html [pcg-unused-substs]: https://github.com/rust-lang/project-const-generics/issues/33 diff --git a/src/doc/rustc-dev-guide/src/contributing.md b/src/doc/rustc-dev-guide/src/contributing.md index bc39508a4..383660fc1 100644 --- a/src/doc/rustc-dev-guide/src/contributing.md +++ b/src/doc/rustc-dev-guide/src/contributing.md @@ -202,7 +202,7 @@ Periodically the changes made to subtree based dependencies need to be synchroni repository and the upstream tool repositories. Subtree synchronizations are typically handled by the respective tool maintainers. Other users -are welcome to submit synchronization PRs, however, in order to do so you you will need to modify +are welcome to submit synchronization PRs, however, in order to do so you will need to modify your local git installation and follow a very precise set of instructions. These instructions are documented, along with several useful tips and tricks, in the [syncing subtree changes][clippy-sync-docs] section in Clippy's Contributing guide. @@ -320,7 +320,7 @@ Here are those same steps in detail: merged because CI will be broken. You'll want to write a message on the PR referencing your change, and how the PR should be merged once your change makes it into a nightly. 5. Wait for your PR to merge. -6. Wait for a nightly +6. Wait for a nightly. 7. (optional) Help land your PR on the upstream repository now that your changes are in nightly. 8. (optional) Send a PR to rust-lang/rust updating the submodule. @@ -390,20 +390,20 @@ Just a few things to keep in mind: add a special annotation before specifying the date: ```md - <!-- date-check --> Jul 2022 + <!-- date-check --> Jan 2023 ``` Example: ```md - As of <!-- date-check --> Jul 2022, the foo did the bar. + As of <!-- date-check --> Jan 2023, the foo did the bar. ``` For cases where the date should not be part of the visible rendered output, use the following instead: ```md - <!-- date-check: Jul 2022 --> + <!-- date-check: Jan 2023 --> ``` - A link to a relevant WG, tracking issue, `rustc` rustdoc page, or similar, that may provide diff --git a/src/doc/rustc-dev-guide/src/crates-io.md b/src/doc/rustc-dev-guide/src/crates-io.md index f012c5bb5..403d61a81 100644 --- a/src/doc/rustc-dev-guide/src/crates-io.md +++ b/src/doc/rustc-dev-guide/src/crates-io.md @@ -11,10 +11,9 @@ you should avoid adding dependencies to the compiler for several reasons: - The dependency may have transitive dependencies that have one of the above problems. -As of <!-- date-check --> Aug 2022, -there is no official policy for vetting new dependencies to the compiler. -Decisions are made on a case-by-case basis, -during code review. +<!-- date-check: Feb 2023 --> +Note that there is no official policy for vetting new dependencies to the compiler. +Decisions are made on a case-by-case basis, during code review. ## Permitted dependencies diff --git a/src/doc/rustc-dev-guide/src/diagnostics.md b/src/doc/rustc-dev-guide/src/diagnostics.md index b50b7bd18..d32de068e 100644 --- a/src/doc/rustc-dev-guide/src/diagnostics.md +++ b/src/doc/rustc-dev-guide/src/diagnostics.md @@ -112,10 +112,11 @@ Here are a few examples: - Dead code: this is a lint. While the user probably doesn't want dead code in their crate, making this a hard error would make refactoring and development very painful. -- [safe_packed_borrows future compatibility warning][safe_packed_borrows]: - this is a silencable lint related to safety. It was judged that the making - this a hard (fixed) error would cause too much breakage, so instead a - warning is emitted that eventually will be turned into a hard error. +- [future-incompatible lints]: + these are silencable lints. + It was decided that making them fixed errors would cause too much breakage, + so warnings are instead emitted, + and will eventually be turned into fixed (hard) errors. Hard-coded warnings (those using the `span_warn` methods) should be avoided for normal code, preferring to use lints instead. Some cases, such as warnings @@ -124,7 +125,7 @@ with CLI flags, will require the use of hard-coded warnings. See the `deny` [lint level](#diagnostic-levels) below for guidelines when to use an error-level lint instead of a fixed error. -[safe_packed_borrows]: https://github.com/rust-lang/rust/issues/46043 +[future-incompatible lints]: #future-incompatible-lints ## Diagnostic output style guide @@ -637,12 +638,12 @@ broader meaning than what rustc exposes to users of the compiler. Inside rustc, future-incompatible lints are for signalling to the user that code they have written may not compile in the future. In general, future-incompatible code exists for two reasons: -* the user has written unsound code that the compiler mistakenly accepted. While +* The user has written unsound code that the compiler mistakenly accepted. While it is within Rust's backwards compatibility guarantees to fix the soundness hole (breaking the user's code), the lint is there to warn the user that this will happen in some upcoming version of rustc *regardless of which edition the code uses*. This is the meaning that rustc exclusively exposes to users as "future incompatible". -* the user has written code that will either no longer compiler *or* will change +* The user has written code that will either no longer compiler *or* will change meaning in an upcoming *edition*. These are often called "edition lints" and can be typically seen in the various "edition compatibility" lint groups (e.g., `rust_2021_compatibility`) that are used to lint against code that will break if the user updates the crate's edition. @@ -665,7 +666,7 @@ declare_lint! { Notice the `reason` field which describes why the future incompatible change is happening. This will change the diagnostic message the user receives as well as determine which lint groups the lint is added to. In the example above, the lint is an "edition lint" -(since it's "reason" is `EditionError`) signifying to the user that the use of anonymous +(since its "reason" is `EditionError`), signifying to the user that the use of anonymous parameters will no longer compile in Rust 2018 and beyond. Inside [LintStore::register_lints][fi-lint-groupings], lints with `future_incompatible` diff --git a/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-items.md b/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-items.md index dcaba533e..78fdd032d 100644 --- a/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-items.md +++ b/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-items.md @@ -48,7 +48,7 @@ A new diagnostic item can be added with these two steps: For the naming conventions of diagnostic items, please refer to [*Naming Conventions*](#naming-conventions). -2. <!-- date-check: Aug 2022 --> +2. <!-- date-check: Feb 2023 --> Diagnostic items in code are accessed via symbols in [`rustc_span::symbol::sym`]. To add your newly-created diagnostic item, diff --git a/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-structs.md b/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-structs.md index e26ba5f34..790d74dcf 100644 --- a/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-structs.md +++ b/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-structs.md @@ -23,7 +23,7 @@ pub struct FieldAlreadyDeclared { #[primary_span] #[label] pub span: Span, - #[label(hir_analysis_previous_decl_label)] + #[label(previous_decl_label)] pub prev_span: Span, } ``` @@ -82,7 +82,7 @@ these attributes can also take a value that is the attribute name to look for Other types have special behavior when used in a `Diagnostic` derive: -- Any attribute applied to an `Option<T>` and will only emit a +- Any attribute applied to an `Option<T>` will only emit a subdiagnostic if the option is `Some(..)`. - Any attribute applied to a `Vec<T>` will be repeated for each element of the vector. diff --git a/src/doc/rustc-dev-guide/src/diagnostics/translation.md b/src/doc/rustc-dev-guide/src/diagnostics/translation.md index e36333039..e3ccec7d7 100644 --- a/src/doc/rustc-dev-guide/src/diagnostics/translation.md +++ b/src/doc/rustc-dev-guide/src/diagnostics/translation.md @@ -222,7 +222,7 @@ returned by `Emitter::fluent_bundle`. This bundle is used preferentially when translating messages, the fallback bundle is only used if the primary bundle is missing a message or not provided. -As of <!-- date-check --> June 2022, there are no locale bundles +As of <!-- date-check --> Jan 2023, there are no locale bundles distributed with the compiler, but mechanisms are implemented for loading bundles. diff --git a/src/doc/rustc-dev-guide/src/getting-started.md b/src/doc/rustc-dev-guide/src/getting-started.md index 4e1f520ff..bc294d1b3 100644 --- a/src/doc/rustc-dev-guide/src/getting-started.md +++ b/src/doc/rustc-dev-guide/src/getting-started.md @@ -81,7 +81,7 @@ normally used for: building, testing, creating releases, formatting, etc. [bootstrapping]: ./building/bootstrapping.md -### Configuring the Compiler +### Configuring the compiler In the top level of the repo: @@ -114,6 +114,10 @@ serious development work. In particular, `./x.py build` and `./x.py test` provide many ways to compile or test a subset of the code, which can save a lot of time. +Also, note that `x.py` supports all kinds of path suffixes for `compiler`, `library`, +and `src/tools` directories. So, you can simply run `x.py test tidy` instead of +`x.py test src/tools/tidy`. Or, `x.py build std` instead of `x.py build library/std`. + [rust-analyzer]: ./building/suggested.html#configuring-rust-analyzer-for-rustc See the chapters on [building](./building/how-to-build-and-run.md), diff --git a/src/doc/rustc-dev-guide/src/git.md b/src/doc/rustc-dev-guide/src/git.md index cac3d6534..a426157a2 100644 --- a/src/doc/rustc-dev-guide/src/git.md +++ b/src/doc/rustc-dev-guide/src/git.md @@ -238,18 +238,22 @@ Changes not staged for commit: no changes added to commit (use "git add" and/or "git commit -a") ``` -These changes are not changes to files: they are changes to submodules (more on -this [later](#git-submodules)). To get rid of those, run `git submodule update` -(or run any `x.py` command, which will automatically update the submodules). -Note that, -as of <!-- date-check --> Aug 2022, +These changes are not changes to files: they are changes to submodules (more on this +[later](#git-submodules)). To get rid of those, run `./x.py --help`, which will automatically update +the submodules. + +Some submodules are not actually needed; for example, `src/llvm-project` doesn't need to be checked +out if you're using `download-ci-llvm`. To avoid having to keep fetching its history, you can use +`git submodule deinit -f src/llvm-project`, which will also avoid it showing as modified again. + +Note that, as of <!-- date-check --> Aug 2022, there is a [bug][#77620] if you use worktrees, submodules, and `x.py` in a commit hook. If you run into an error like the following, it's not anything you did wrong: ``` -error: failed to read `/home/joshua/rustc-worktree/src/tools/miri/cargo-miri/Cargo.toml` +error: failed to read `/home/jyn/rustc-worktree/src/tools/cargo/Cargo.toml` Caused by: No such file or directory (os error 2) @@ -260,8 +264,6 @@ There is a workaround in [the issue][#77620-workaround]. [#77620]: https://github.com/rust-lang/rust/issues/77620 [#77620-workaround]: https://github.com/rust-lang/rust/issues/77620#issuecomment-705228229 -(Note that as of Sept 2022 `miri` is a subtree and not a submodule.) - ## Rebasing and Conflicts When you edit your code locally, you are making changes to the version of diff --git a/src/doc/rustc-dev-guide/src/hir.md b/src/doc/rustc-dev-guide/src/hir.md index 8316f7ce3..160b47645 100644 --- a/src/doc/rustc-dev-guide/src/hir.md +++ b/src/doc/rustc-dev-guide/src/hir.md @@ -117,9 +117,9 @@ that `n` must be some HIR expression, you can do [Expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/struct.Expr.html Finally, you can use the HIR map to find the parents of nodes, via -calls like [`tcx.hir().get_parent_node(n)`][get_parent_node]. +calls like [`tcx.hir().get_parent(n)`][get_parent]. -[get_parent_node]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.get_parent_node +[get_parent]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.get_parent ## HIR Bodies diff --git a/src/doc/rustc-dev-guide/src/identifiers.md b/src/doc/rustc-dev-guide/src/identifiers.md index 18a3efefc..1b60b3b0b 100644 --- a/src/doc/rustc-dev-guide/src/identifiers.md +++ b/src/doc/rustc-dev-guide/src/identifiers.md @@ -1,4 +1,4 @@ -# Identifiers in the Compiler +# Identifiers in the compiler If you have read the few previous chapters, you now know that `rustc` uses many different intermediate representations to perform different kinds of analyses. diff --git a/src/doc/rustc-dev-guide/src/implementing_new_features.md b/src/doc/rustc-dev-guide/src/implementing_new_features.md index 9147c1b41..9bd853080 100644 --- a/src/doc/rustc-dev-guide/src/implementing_new_features.md +++ b/src/doc/rustc-dev-guide/src/implementing_new_features.md @@ -33,7 +33,7 @@ like this; for example, the compiler team recommends filing a Major Change Proposal ([MCP][mcp]) as a lightweight way to garner support and feedback without requiring full consensus. -[mcp]: compiler/mcp.md#public-facing-changes-require-rfcbot-fcp +[mcp]: https://forge.rust-lang.org/compiler/mcp.html#public-facing-changes-require-rfcbot-fcp You don't need to have the implementation fully ready for r+ to propose an FCP, but it is generally a good idea to have at least a proof diff --git a/src/doc/rustc-dev-guide/src/llvm-coverage-instrumentation.md b/src/doc/rustc-dev-guide/src/llvm-coverage-instrumentation.md index b29b32a62..8cd765011 100644 --- a/src/doc/rustc-dev-guide/src/llvm-coverage-instrumentation.md +++ b/src/doc/rustc-dev-guide/src/llvm-coverage-instrumentation.md @@ -222,13 +222,9 @@ properly-configured variables in LLVM IR, according to very specific details of the [_LLVM Coverage Mapping Format_][coverage-mapping-format] (Version 6).[^llvm-and-covmap-versions] -[^llvm-and-covmap-versions]: The Rust compiler (as of <!-- date-check --> December 2021) -supports _LLVM Coverage Mapping Format_ Version 5 or 6. Version 5 -was introduced in _LLVM 12_, -which is (as of <!-- date-check: December 2021--> this writing) the minimum LLVM -version supported by the current version of Rust. Version 6 was introduced in -_LLVM 13_, which is currently the default LLVM version for Rust. The Rust -compiler will automatically use the most up-to-date coverage mapping format +[^llvm-and-covmap-versions]: +The Rust compiler (as of <!-- date-check: --> Feb 2023) supports _LLVM Coverage Mapping Format_ 6. +The Rust compiler will automatically use the most up-to-date coverage mapping format version that is compatible with the compiler's built-in version of LLVM. ```rust @@ -301,11 +297,10 @@ $ ./x.py test tests/run-make-fulldeps/coverage --bless ``` [mir-opt-test]: https://github.com/rust-lang/rust/blob/master/tests/mir-opt/instrument_coverage.rs -[coverage-test-samples]: https://github.com/rust-lang/rust/tree/master/tests/run-make-fulldeps/coverage -[`coverage-reports`]: https://github.com/rust-lang/rust/tree/master/tests/run-make-fulldeps/coverage-reports -[`coverage-spanview`]: https://github.com/rust-lang/rust/tree/master/tests/run-make-fulldeps/coverage-spanview +[coverage-test-samples]: https://github.com/rust-lang/rust/tree/master/tests/run-make/coverage +[`coverage-reports`]: https://github.com/rust-lang/rust/tree/master/tests/run-make/coverage-reports [spanview-debugging]: compiler-debugging.md#viewing-spanview-output -[`coverage-llvmir`]: https://github.com/rust-lang/rust/tree/master/tests/run-make-fulldeps/coverage-llvmir +[`coverage-llvmir`]: https://github.com/rust-lang/rust/tree/master/tests/run-make/coverage-llvmir ## Implementation Details of the `InstrumentCoverage` MIR Pass diff --git a/src/doc/rustc-dev-guide/src/mir/debugging.md b/src/doc/rustc-dev-guide/src/mir/debugging.md index 3da17785b..4e1d68938 100644 --- a/src/doc/rustc-dev-guide/src/mir/debugging.md +++ b/src/doc/rustc-dev-guide/src/mir/debugging.md @@ -14,10 +14,10 @@ elements obscured by overlaps, and tooltips to view the MIR statements). This flag takes an optional value: `statement` (the default), `terminator`, or `block`, to generate span highlights with different levels of granularity. -`-Z dump-mir=F` is a handy compiler options that will let you view the MIR for +`-Z dump-mir=F` is a handy compiler option that will let you view the MIR for each function at each stage of compilation. `-Z dump-mir` takes a **filter** `F` which allows you to control which functions and which passes you are -interesting in. For example: +interested in. For example: ```bash > rustc -Z dump-mir=foo ... diff --git a/src/doc/rustc-dev-guide/src/mir/visitor.md b/src/doc/rustc-dev-guide/src/mir/visitor.md index 505b700b3..17186fe75 100644 --- a/src/doc/rustc-dev-guide/src/mir/visitor.md +++ b/src/doc/rustc-dev-guide/src/mir/visitor.md @@ -37,10 +37,10 @@ code that will execute whenever a `foo` is found. If you want to recursively walk the contents of the `foo`, you then invoke the `super_foo` method. (NB. You never want to override `super_foo`.) -A very simple example of a visitor can be found in [`LocalUseCounter`]. -By implementing `visit_local` method, this visitor counts how many times each local is used. +A very simple example of a visitor can be found in [`LocalUseVisitor`]. +By implementing `visit_local` method, this visitor counts how many times each local is mutably used. -[`LocalUseCounter`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir_transform/simplify_try/struct.LocalUseCounter.html +[`LocalUseVisitor`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir_transform/const_debuginfo/struct.LocalUseVisitor.html ## Traversal diff --git a/src/doc/rustc-dev-guide/src/overview.md b/src/doc/rustc-dev-guide/src/overview.md index ca37f94b2..de06c742b 100644 --- a/src/doc/rustc-dev-guide/src/overview.md +++ b/src/doc/rustc-dev-guide/src/overview.md @@ -1,4 +1,4 @@ -# Overview of the Compiler +# Overview of the compiler <!-- toc --> diff --git a/src/doc/rustc-dev-guide/src/profiling/with_perf.md b/src/doc/rustc-dev-guide/src/profiling/with_perf.md index af30b4732..c9bd88ecd 100644 --- a/src/doc/rustc-dev-guide/src/profiling/with_perf.md +++ b/src/doc/rustc-dev-guide/src/profiling/with_perf.md @@ -90,14 +90,15 @@ You can also use that same command to use cachegrind or other profiling tools. If you prefer to run things manually, that is also possible. You first need to find the source for the test you want. Sources for the tests -are found in [the `collector/benchmarks` directory][dir]. So let's go -into the directory of a specific test; we'll use `clap-rs` as an -example: +are found in [the `collector/compile-benchmarks` directory][compile-time dir] +and [the `collector/runtime-benchmarks` directory][runtime dir]. So let's +go into the directory of a specific test; we'll use `clap-rs` as an example: -[dir]: https://github.com/rust-lang/rustc-perf/tree/master/collector/benchmarks +[compile-time dir]: https://github.com/rust-lang/rustc-perf/tree/master/collector/compile-benchmarks +[runtime dir]: https://github.com/rust-lang/rustc-perf/tree/master/collector/runtime-benchmarks ```bash -cd collector/benchmarks/clap-rs +cd collector/compile-benchmarks/clap-3.1.6 ``` In this case, let's say we want to profile the `cargo check` diff --git a/src/doc/rustc-dev-guide/src/rustc-driver-getting-diagnostics.md b/src/doc/rustc-dev-guide/src/rustc-driver-getting-diagnostics.md index bb19ad9d3..3c2102a50 100644 --- a/src/doc/rustc-dev-guide/src/rustc-driver-getting-diagnostics.md +++ b/src/doc/rustc-dev-guide/src/rustc-driver-getting-diagnostics.md @@ -7,34 +7,8 @@ To get diagnostics from the compiler, configure `rustc_interface::Config` to output diagnostic to a buffer, and run `TyCtxt.analysis`. The following was tested -with <!-- date-check: Jan 2023 --> `nightly-2022-12-19` (See [here][example] -for the complete example): - -[example]: https://github.com/rust-lang/rustc-dev-guide/blob/master/examples/rustc-driver-getting-diagnostics.rs +with <!-- date-check: Feb 2023 --> `nightly-2023-02-13`: ```rust -let buffer = sync::Arc::new(sync::Mutex::new(Vec::new())); -let config = rustc_interface::Config { - opts: config::Options { - // Configure the compiler to emit diagnostics in compact JSON format. - error_format: config::ErrorOutputType::Json { - pretty: false, - json_rendered: rustc_errors::emitter::HumanReadableErrorType::Default( - rustc_errors::emitter::ColorConfig::Never, - ), - }, - /* other config */ - }, - /* other config */ -}; -rustc_interface::run_compiler(config, |compiler| { - compiler.enter(|queries| { - queries.global_ctxt().unwrap().take().enter(|tcx| { - // Run the analysis phase on the local crate to trigger the type error. - let _ = tcx.analysis(()); - }); - }); -}); -// Read buffered diagnostics. -let diagnostics = String::from_utf8(buffer.lock().unwrap().clone()).unwrap(); +{{#include ../examples/rustc-driver-getting-diagnostics.rs}} ``` diff --git a/src/doc/rustc-dev-guide/src/rustc-driver-interacting-with-the-ast.md b/src/doc/rustc-dev-guide/src/rustc-driver-interacting-with-the-ast.md index 5b495b4fe..d058a5838 100644 --- a/src/doc/rustc-dev-guide/src/rustc-driver-interacting-with-the-ast.md +++ b/src/doc/rustc-dev-guide/src/rustc-driver-interacting-with-the-ast.md @@ -5,45 +5,8 @@ ## Getting the type of an expression To get the type of an expression, use the `global_ctxt` to get a `TyCtxt`. -The following was tested with <!-- date-check: Jan 2023 --> `nightly-2022-12-19` -(see [here][example] for the complete example): - -[example]: https://github.com/rust-lang/rustc-dev-guide/blob/master/examples/rustc-driver-interacting-with-the-ast.rs +The following was tested with <!-- date-check: Feb 2023 --> `nightly-2023-02-13`: ```rust -let config = rustc_interface::Config { - input: config::Input::Str { - name: source_map::FileName::Custom("main.rs".to_string()), - input: "fn main() { let message = \"Hello, world!\"; println!(\"{}\", message); }" - .to_string(), - }, - /* other config */ -}; -rustc_interface::run_compiler(config, |compiler| { - compiler.enter(|queries| { - // Analyze the crate and inspect the types under the cursor. - queries.global_ctxt().unwrap().take().enter(|tcx| { - // Every compilation contains a single crate. - let hir_krate = tcx.hir(); - // Iterate over the top-level items in the crate, looking for the main function. - for id in hir_krate.items() { - let item = hir_krate.item(id); - // Use pattern-matching to find a specific node inside the main function. - if let rustc_hir::ItemKind::Fn(_, _, body_id) = item.kind { - let expr = &tcx.hir().body(body_id).value; - if let rustc_hir::ExprKind::Block(block, _) = expr.kind { - if let rustc_hir::StmtKind::Local(local) = block.stmts[0].kind { - if let Some(expr) = local.init { - let hir_id = expr.hir_id; // hir_id identifies the string "Hello, world!" - let def_id = tcx.hir().local_def_id(item.hir_id()); // def_id identifies the main function - let ty = tcx.typeck(def_id).node_type(hir_id); - println!("{:?}: {:?}", expr, ty); - } - } - } - } - } - }) - }); -}); +{{#include ../examples/rustc-driver-interacting-with-the-ast.rs}} ``` diff --git a/src/doc/rustc-dev-guide/src/rustc-driver.md b/src/doc/rustc-dev-guide/src/rustc-driver.md index cef50111d..192811da1 100644 --- a/src/doc/rustc-dev-guide/src/rustc-driver.md +++ b/src/doc/rustc-dev-guide/src/rustc-driver.md @@ -1,4 +1,4 @@ -# The Rustc Driver and Interface +# `rustc_driver` and `rustc_interface` The [`rustc_driver`] is essentially `rustc`'s `main()` function. It acts as the glue for running the various phases of the compiler in the correct order, diff --git a/src/doc/rustc-dev-guide/src/solve/canonicalization.md b/src/doc/rustc-dev-guide/src/solve/canonicalization.md new file mode 100644 index 000000000..a14be5216 --- /dev/null +++ b/src/doc/rustc-dev-guide/src/solve/canonicalization.md @@ -0,0 +1,84 @@ +# Canonicalization + +Canonicalization is the process of *isolating* a value from its context and is necessary +for global caching of goals which include inference variables. + +The idea is that given the goals `u32: Trait<?x>` and `u32: Trait<?y>`, where `?x` and `?y` +are two different currently unconstrained inference variables, we should get the same result +for both goals. We can therefore prove *the canonical query* `exists<T> u32: Trait<T>` once +and reuse the result. + +Let's first go over the way canonical queries work and then dive into the specifics of +how canonicalization works. + +## A walkthrough of canonical queries + +To make this a bit easier, let's use the trait goal `u32: Trait<?x>` as an example with the +assumption that the only relevant impl is `impl<T> Trait<Vec<T>> for u32`. + +### Canonicalizing the input + +We start by *canonicalizing* the goal, replacing inference variables with existential and +placeholders with universal bound variables. This would result in the *canonical goal* +`exists<T> u32: Trait<T>`. + +We remember the original values of all bound variables in the original context. Here this would +map `T` back to `?x`. These original values are used later on when dealing with the query +response. + +We now call the canonical query with the canonical goal. + +### Instantiating the canonical goal inside of the query + +To actually try to prove the canonical goal we start by instantiating the bound variables with +inference variables and placeholders again. + +This happens inside of the query in a completely separate `InferCtxt`. Inside of the query we +now have a goal `u32: Trait<?0>`. We also remember which value we've used to instantiate the bound +variables in the canonical goal, which maps `T` to `?0`. + +We now compute the goal `u32: Trait<?0>` and figure out that this holds, but we've constrained +`?0` to `Vec<?1>`. We finally convert this result to something useful to the caller. + +### Canonicalizing the query response + +We have to return to the caller both whether the goal holds, and the inference constraints +from inside of the query. + +To return the inference results to the caller we canonicalize the mapping from bound variables +to the instantiated values in the query. This means that the query response is `Certainty::Yes` +and a mapping from `T` to `exists<U> Vec<U>`. + +### Instantiating the query response + +The caller now has to apply the constraints returned by the query. For this they first +instantiate the bound variables of the canonical response with inference variables and +placeholders again, so the mapping in the response is now from `T` to `Vec<?z>`. + +It now equates the original value of `T` (`?x`) with the value for `T` in the +response (`Vec<?z>`), which correctly constrains `?x` to `Vec<?z>`. + +## `ExternalConstraints` + +Computing a trait goal may not only constrain inference variables, it can also add region +obligations, e.g. given a goal `(): AOutlivesB<'a, 'b>` we would like to return the fact that +`'a: 'b` has to hold. + +This is done by not only returning the mapping from bound variables to the instantiated values +from the query but also extracting additional `ExternalConstraints` from the `InferCtxt` context +while building the response. + +## How exactly does canonicalization work + +TODO: link to code once the PR lands and elaborate + +- types and consts: infer to existentially bound var, placeholder to universally bound var, + considering universes +- generic parameters in the input get treated as placeholders in the root universe +- all regions in the input get all mapped to existentially bound vars and we "uniquify" them. + `&'a (): Trait<'a>` gets canonicalized to `exists<'0, '1> &'0 (): Trait<'1>`. We do not care + about their universes and simply put all regions into the highest universe of the input. +- once we collected all canonical vars we compress their universes, see comment in `finalize`. +- in the output everything in a universe of the caller gets put into the root universe and only + gets its correct universe when we unify the var values with the orig values of the caller +- we do not uniquify regions in the response and don't canonicalize `'static`
\ No newline at end of file diff --git a/src/doc/rustc-dev-guide/src/solve/coinduction.md b/src/doc/rustc-dev-guide/src/solve/coinduction.md new file mode 100644 index 000000000..c682e002d --- /dev/null +++ b/src/doc/rustc-dev-guide/src/solve/coinduction.md @@ -0,0 +1,250 @@ +# Coinduction + +The trait solver may use coinduction when proving goals. +Coinduction is fairly subtle so we're giving it its own chapter. + +## Coinduction and induction + +With induction, we recursively apply proofs until we end up with a finite proof tree. +Consider the example of `Vec<Vec<Vec<u32>>>: Debug` which results in the following tree. + +- `Vec<Vec<Vec<u32>>>: Debug` + - `Vec<Vec<u32>>: Debug` + - `Vec<u32>: Debug` + - `u32: Debug` + +This tree is finite. But not all goals we would want to hold have finite proof trees, +consider the following example: + +```rust +struct List<T> { + value: T, + next: Option<Box<List<T>>>, +} +``` + +For `List<T>: Send` to hold all its fields have to recursively implement `Send` as well. +This would result in the following proof tree: + +- `List<T>: Send` + - `T: Send` + - `Option<Box<List<T>>>: Send` + - `Box<List<T>>: Send` + - `List<T>: Send` + - `T: Send` + - `Option<Box<List<T>>>: Send` + - `Box<List<T>>: Send` + - ... + +This tree would be infinitely large which is exactly what coinduction is about. + +> To **inductively** prove a goal you need to provide a finite proof tree for it. +> To **coinductively** prove a goal the provided proof tree may be infinite. + +## Why is coinduction correct + +When checking whether some trait goals holds, we're asking "does there exist an `impl` +which satisfies this bound". Even if are infinite chains of nested goals, we still have a +unique `impl` which should be used. + +## How to implement coinduction + +While our implementation can not check for coinduction by trying to construct an infinite +tree as that would take infinite resources, it still makes sense to think of coinduction +from this perspective. + +As we cannot check for infinite trees, we instead search for patterns for which we know that +they would result in an infinite proof tree. The currently pattern we detect are (canonical) +cycles. If `T: Send` relies on `T: Send` then it's pretty clear that this will just go on forever. + +With cycles we have to be careful with caching. Because of canonicalization of regions and +inference variables encountering a cycle doesn't mean that we would get an infinite proof tree. +Looking at the following example: +```rust +trait Foo {} +struct Wrapper<T>(T); + +impl<T> Foo for Wrapper<Wrapper<T>> +where + Wrapper<T>: Foo +{} +``` +Proving `Wrapper<?0>: Foo` uses the impl `impl<T> Foo for Wrapper<Wrapper<T>>` which constrains +`?0` to `Wrapper<?1>` and then requires `Wrapper<?1>: Foo`. Due to canonicalization this would be +detected as a cycle. + +The idea to solve is to return a *provisional result* whenever we detect a cycle and repeatedly +retry goals until the *provisional result* is equal to the final result of that goal. We +start out by using `Yes` with no constraints as the result and then update it to the result of +the previous iteration whenever we have to rerun. + +TODO: elaborate here. We use the same approach as chalk for coinductive cycles. +Note that the treatment for inductive cycles currently differs by simply returning `Overflow`. +See [the relevant chapters][chalk] in the chalk book. + +[chalk]: https://rust-lang.github.io/chalk/book/recursive/inductive_cycles.html + + +## Future work + +We currently only consider auto-traits, `Sized`, and `WF`-goals to be coinductive. +In the future we pretty much intend for all goals to be coinductive. +Lets first elaborate on why allowing more coinductive proofs is even desirable. + +### Recursive data types already rely on coinduction... + +...they just tend to avoid them in the trait solver. + +```rust +enum List<T> { + Nil, + Succ(T, Box<List<T>>), +} + +impl<T: Clone> Clone for List<T> { + fn clone(&self) -> Self { + match self { + List::Nil => List::Nil, + List::Succ(head, tail) => List::Succ(head.clone(), tail.clone()), + } + } +} +``` + +We are using `tail.clone()` in this impl. For this we have to prove `Box<List<T>>: Clone` +which requires `List<T>: Clone` but that relies on the impl which we are currently checking. +By adding that requirement to the `where`-clauses of the impl, which is what we would +do with [perfect derive], we move that cycle into the trait solver and [get an error][ex1]. + +### Recursive data types + +We also need coinduction to reason about recursive types containing projections, +e.g. the following currently fails to compile even though it should be valid. +```rust +use std::borrow::Cow; +pub struct Foo<'a>(Cow<'a, [Foo<'a>]>); +``` +This issue has been known since at least 2015, see +[#23714](https://github.com/rust-lang/rust/issues/23714) if you want to know more. + +### Explicitly checked implied bounds + +When checking an impl, we assume that the types in the impl headers are well-formed. +This means that when using instantiating the impl we have to prove that's actually the case. +[#100051](https://github.com/rust-lang/rust/issues/100051) shows that this is not the case. +To fix this, we have to add `WF` predicates for the types in impl headers. +Without coinduction for all traits, this even breaks `core`. + +```rust +trait FromResidual<R> {} +trait Try: FromResidual<<Self as Try>::Residual> { + type Residual; +} + +struct Ready<T>(T); +impl<T> Try for Ready<T> { + type Residual = Ready<()>; +} +impl<T> FromResidual<<Ready<T> as Try>::Residual> for Ready<T> {} +``` + +When checking that the impl of `FromResidual` is well formed we get the following cycle: + +The impl is well formed if `<Ready<T> as Try>::Residual` and `Ready<T>` are well formed. +- `wf(<Ready<T> as Try>::Residual)` requires +- `Ready<T>: Try`, which requires because of the super trait +- `Ready<T>: FromResidual<Ready<T> as Try>::Residual>`, **because of implied bounds on impl** +- `wf(<Ready<T> as Try>::Residual)` :tada: **cycle** + +### Issues when extending coinduction to more goals + +There are some additional issues to keep in mind when extending coinduction. +The issues here are not relevant for the current solver. + +#### Implied super trait bounds + +Our trait system currently treats super traits, e.g. `trait Trait: SuperTrait`, +by 1) requiring that `SuperTrait` has to hold for all types which implement `Trait`, +and 2) assuming `SuperTrait` holds if `Trait` holds. + +Relying on 2) while proving 1) is unsound. This can only be observed in case of +coinductive cycles. Without cycles, whenever we rely on 2) we must have also +proven 1) without relying on 2) for the used impl of `Trait`. + +```rust +trait Trait: SuperTrait {} + +impl<T: Trait> Trait for T {} + +// Keeping the current setup for coinduction +// would allow this compile. Uff :< +fn sup<T: SuperTrait>() {} +fn requires_trait<T: Trait>() { sup::<T>() } +fn generic<T>() { requires_trait::<T>() } +``` +This is not really fundamental to coinduction but rather an existing property +which is made unsound because of it. + +##### Possible solutions + +The easiest way to solve this would be to completely remove 2) and always elaborate +`T: Trait` to `T: Trait` and `T: SuperTrait` outside of the trait solver. +This would allow us to also remove 1), but as we still have to prove ordinary +`where`-bounds on traits, that's just additional work. + +While one could imagine ways to disable cyclic uses of 2) when checking 1), +at least the ideas of myself - @lcnr - are all far to complex to be reasonable. + +#### `normalizes_to` goals and progress + +A `normalizes_to` goal represents the requirement that `<T as Trait>::Assoc` normalizes +to some `U`. This is achieved by defacto first normalizing `<T as Trait>::Assoc` and then +equating the resulting type with `U`. It should be a mapping as each projection should normalize +to exactly one type. By simply allowing infinite proof trees, we would get the following behavior: + +```rust +trait Trait { + type Assoc; +} + +impl Trait for () { + type Assoc = <() as Trait>::Assoc; +} +``` + +If we now compute `normalizes_to(<() as Trait>::Assoc, Vec<u32>)`, we would resolve the impl +and get the associated type `<() as Trait>::Assoc`. We then equate that with the expected type, +causing us to check `normalizes_to(<() as Trait>::Assoc, Vec<u32>)` again. +This just goes on forever, resulting in an infinite proof tree. + +This means that `<() as Trait>::Assoc` would be equal to any other type which is unsound. + +##### How to solve this + +**WARNING: THIS IS SUBTLE AND MIGHT BE WRONG** + +Unlike trait goals, `normalizes_to` has to be *productive*[^1]. A `normalizes_to` goal +is productive once the projection normalizes to a rigid type constructor, +so `<() as Trait>::Assoc` normalizing to `Vec<<() as Trait>::Assoc>` would be productive. + +A `normalizes_to` goal has two kinds of nested goals. Nested requirements needed to actually +normalize the projection, and the equality between the normalized projection and the +expected type. Only the equality has to be productive. A branch in the proof tree is productive +if it is either finite, or contains at least one `normalizes_to` where the alias is resolved +to a rigid type constructor. + +Alternatively, we could simply always treat the equate branch of `normalizes_to` as inductive. +Any cycles should result in infinite types, which aren't supported anyways and would only +result in overflow when deeply normalizing for codegen. + +experimentation and examples: https://hackmd.io/-8p0AHnzSq2VAE6HE_wX-w?view + +Another attempt at a summary. +- in projection eq, we must make progress with constraining the rhs +- a cycle is only ok if while equating we have a rigid ty on the lhs after norm at least once +- cycles outside of the recursive `eq` call of `normalizes_to` are always fine + +[^1]: related: https://coq.inria.fr/refman/language/core/coinductive.html#top-level-definitions-of-corecursive-functions + +[perfect derive]: https://smallcultfollowing.com/babysteps/blog/2022/04/12/implied-bounds-and-perfect-derive +[ex1]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=0a9c3830b93a2380e6978d6328df8f72 diff --git a/src/doc/rustc-dev-guide/src/solve/the-solver.md b/src/doc/rustc-dev-guide/src/solve/the-solver.md new file mode 100644 index 000000000..61e6cad1c --- /dev/null +++ b/src/doc/rustc-dev-guide/src/solve/the-solver.md @@ -0,0 +1,17 @@ +# The solver + +Also consider reading the documentation for [the recursive solver in chalk][chalk] +as it is very similar to this implementation and also talks about limitations of this +approach. + +[chalk]: https://rust-lang.github.io/chalk/book/recursive.html + +The basic structure of the solver is a pure function +`fn evaluate_goal(goal: Goal<'tcx>) -> Response`. +While the actual solver is not fully pure to deal with overflow and cycles, we are +going to defer that for now. + +To deal with inference variables and to improve caching, we use +[canonicalization](./canonicalization.md). + +TODO: write the remaining code for this as well. diff --git a/src/doc/rustc-dev-guide/src/solve/trait-solving.md b/src/doc/rustc-dev-guide/src/solve/trait-solving.md new file mode 100644 index 000000000..71f6581c2 --- /dev/null +++ b/src/doc/rustc-dev-guide/src/solve/trait-solving.md @@ -0,0 +1,114 @@ +# Trait solving (new) + +This chapter describes how trait solving works with the new WIP solver located in +[`rustc_trait_selection/solve`][solve]. Feel free to also look at the docs for +[the current solver](../traits/resolution.md) and [the chalk solver](../traits/chalk.md) +can be found separately. + +## Core concepts + +The goal of the trait system is to check whether a given trait bound is satisfied. +Most notably when typechecking the body of - potentially generic - functions. +For example: + +```rust +fn uses_vec_clone<T: Clone>(x: Vec<T>) -> (Vec<T>, Vec<T>) { + (x.clone(), x) +} +``` +Here the call to `x.clone()` requires us to prove that `Vec<T>` implements `Clone` given +the assumption that `T: Clone` is true. We can assume `T: Clone` as that will be proven by +callers of this function. + +The concept of "prove the `Vec<T>: Clone` with the assumption `T: Clone`" is called a [`Goal`]. +Both `Vec<T>: Clone` and `T: Clone` are represented using [`Predicate`]. There are other +predicates, most notably equality bounds on associated items: `<Vec<T> as IntoIterator>::Item == T`. +See the `PredicateKind` enum for an exhaustive list. A `Goal` is represented as the `predicate` we +have to prove and the `param_env` in which this predicate has to hold. + +We prove goals by checking whether each possible [`Candidate`] applies for the given goal by +recursively proving its nested goals. For a list of possible candidates with examples, look at +[`CandidateSource`]. The most important candidates are `Impl` candidates, i.e. trait implementations +written by the user, and `ParamEnv` candidates, i.e. assumptions in our current environment. + +Looking at the above example, to prove `Vec<T>: Clone` we first use +`impl<T: Clone> Clone for Vec<T>`. To use this impl we have to prove the nested +goal that `T: Clone` holds. This can use the assumption `T: Clone` from the `ParamEnv` +which does not have any nested goals. Therefore `Vec<T>: Clone` holds. + +The trait solver can either return success, ambiguity or an error as a [`CanonicalResponse`]. +For success and ambiguity it also returns constraints inference and region constraints. + +## Requirements + +Before we dive into the new solver lets first take the time to go through all of our requirements +on the trait system. We can then use these to guide our design later on. + +TODO: elaborate on these rules and get more precise about their meaning. +Also add issues where each of these rules have been broken in the past +(or still are). + +### 1. The trait solver has to be *sound* + +This means that we must never return *success* for goals for which no `impl` exists. That would +simply be unsound by assuming a trait is implemented even though it is not. When using predicates +from the `where`-bounds, the `impl` will be proved by the user of the item. + +### 2. If type checker solves generic goal concrete instantiations of that goal have the same result + +Pretty much: If we successfully typecheck a generic function concrete instantiations +of that function should also typeck. We should not get errors post-monomorphization. +We can however get overflow as in the following snippet: + +```rust +fn foo<T: Trait>(x: ) +``` + +### 3. Trait goals in empty environments are proven by a unique impl + +If a trait goal holds with an empty environment, there is a unique `impl`, +either user-defined or builtin, which is used to prove that goal. + +This is necessary for codegen to select a unique method. +An exception here are *marker traits* which are allowed to overlap. + +### 4. Normalization in empty environments results in a unique type + +Normalization for alias types/consts has a unique result. Otherwise we can easily implement +transmute in safe code. Given the following function, we have to make sure that the input and +output types always get normalized to the same concrete type. +```rust +fn foo<T: Trait>( + x: <T as Trait>::Assoc +) -> <T as Trait>::Assoc { + x +} +``` + +### 5. During coherence trait solving has to be complete + +During coherence we never return *error* for goals which can be proven. This allows overlapping +impls which would break rule 3. + +### 6. Trait solving must be (free) lifetime agnostic + +Trait solving during codegen should have the same result as during typeck. As we erase +all free regions during codegen we must not rely on them during typeck. A noteworthy example +is special behavior for `'static`. + +### 7. Removing ambiguity makes strictly more things compile + +We *should* not rely on ambiguity for things to compile. +Not doing that will cause future improvements to be breaking changes. + +### 8. semantic equality implies structural equality + +Two types being equal in the type system must mean that they have the same `TypeId`. + + +[solve]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/solve/index.html +[`Goal`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/solve/struct.Goal.html +[`Predicate`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.Predicate.html +[`Candidate`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/solve/assembly/struct.Candidate.html +[`CandidateSource`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/solve/assembly/enum.CandidateSource.html +[`CanonicalResponse`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/solve/type.CanonicalResponse.html diff --git a/src/doc/rustc-dev-guide/src/tests/compiletest.md b/src/doc/rustc-dev-guide/src/tests/compiletest.md index 9f0c56dc4..5fc6ba809 100644 --- a/src/doc/rustc-dev-guide/src/tests/compiletest.md +++ b/src/doc/rustc-dev-guide/src/tests/compiletest.md @@ -24,6 +24,11 @@ See the [Adding new tests](adding.md) chapter for a tutorial on creating a new test, and the [Running tests](running.md) chapter on how to run the test suite. +Compiletest itself tries to avoid running tests when the artifacts +that are involved (mainly the compiler) haven't changed. You can use +`x test --test-args --force-rerun` to rerun a test even when none of the +inputs have changed. + ## Test suites All of the tests are in the [`tests`] directory. diff --git a/src/doc/rustc-dev-guide/src/tests/running.md b/src/doc/rustc-dev-guide/src/tests/running.md index 96c869109..0a3de6f8b 100644 --- a/src/doc/rustc-dev-guide/src/tests/running.md +++ b/src/doc/rustc-dev-guide/src/tests/running.md @@ -175,27 +175,6 @@ By passing `--pass $mode`, you can reduce the testing time. For each mode, please see [Controlling pass/fail expectations](ui.md#controlling-passfail-expectations). -## Using incremental compilation - -You can further enable the `--incremental` flag to save additional -time in subsequent rebuilds: - -```bash -./x.py test tests/ui --incremental --test-args issue-1234 -``` - -If you don't want to include the flag with every command, you can -enable it in the `config.toml`: - -```toml -[rust] -incremental = true -``` - -Note that incremental compilation will use more disk space than usual. -If disk space is a concern for you, you might want to check the size -of the `build` directory from time to time. - ## Running tests with different "compare modes" UI tests may have different output depending on certain "modes" that diff --git a/src/doc/rustc-dev-guide/src/traits/resolution.md b/src/doc/rustc-dev-guide/src/traits/resolution.md index 9cf753b19..639ebbdec 100644 --- a/src/doc/rustc-dev-guide/src/traits/resolution.md +++ b/src/doc/rustc-dev-guide/src/traits/resolution.md @@ -163,7 +163,7 @@ impl<T: Get> Get for Box<T> { } ``` -What happens when we invoke `get_it(&Box::new(1_u16))`, for example? In this +What happens when we invoke `get(&Box::new(1_u16))`, for example? In this case, the `Self` type is `Box<u16>` – that unifies with both impls, because the first applies to all types `T`, and the second to all `Box<T>`. In order for this to be unambiguous, the compiler does a *winnowing* @@ -248,7 +248,7 @@ because we know that each resolution will resolve to a particular impl. One interesting twist has to do with nested obligations. In general, in codegen, we only to figure out which candidate applies, we do not care about nested obligations, -as these are already assumed to be true. Nonetheless, we *do* currently do fulfill all of them. +as these are already assumed to be true. Nonetheless, we *do* currently fulfill all of them. That is because it can sometimes inform the results of type inference. That is, we do not have the full substitutions in terms of the type variables of the impl available to us, so we must run trait selection to figure diff --git a/src/doc/rustc-dev-guide/src/ty.md b/src/doc/rustc-dev-guide/src/ty.md index 90e17e85f..4a6cdb7c7 100644 --- a/src/doc/rustc-dev-guide/src/ty.md +++ b/src/doc/rustc-dev-guide/src/ty.md @@ -133,7 +133,7 @@ the `ty::Ty` to be a thin pointer-like type. This allows us to do cheap comparisons for equality, along with the other benefits of interning. -[tykind]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html +[tykind]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html ## Allocating and working with types @@ -148,18 +148,62 @@ These methods all return a `Ty<'tcx>` – note that the lifetime you get back is arena that this `tcx` has access to. Types are always canonicalized and interned (so we never allocate exactly the same type twice). -> N.B. -> Because types are interned, it is possible to compare them for equality efficiently using `==` -> – however, this is almost never what you want to do unless you happen to be hashing and looking -> for duplicates. This is because often in Rust there are multiple ways to represent the same type, -> particularly once inference is involved. If you are going to be testing for type equality, you -> probably need to start looking into the inference code to do it right. - You can also find various common types in the `tcx` itself by accessing its fields: `tcx.types.bool`, `tcx.types.char`, etc. (See [`CommonTypes`] for more.) [`CommonTypes`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.CommonTypes.html +<!-- N.B: This section is linked from the type comparison internal lint. --> +## Comparing types + +Because types are interned, it is possible to compare them for equality efficiently using `==` +– however, this is almost never what you want to do unless you happen to be hashing and looking +for duplicates. This is because often in Rust there are multiple ways to represent the same type, +particularly once inference is involved. + +For example, the type `{integer}` (`ty::Infer(ty::IntVar(..))` an integer inference variable, +the type of an integer literal like `0`) and `u8` (`ty::UInt(..)`) should often be treated as +equal when testing whether they can be assigned to each other (which is a common operation in +diagnostics code). `==` on them will return `false` though, since they are different types. + +The simplest way to compare two types correctly requires an inference context (`infcx`). +If you have one, you can use `infcx.can_eq(param_env, ty1, ty2)` +to check whether the types can be made equal. +This is typically what you want to check during diagnostics, which is concerned with questions such +as whether two types can be assigned to each other, not whether they're represented identically in +the compiler's type-checking layer. + +When working with an inference context, you have to be careful to ensure that potential inference +variables inside the types actually belong to that inference context. If you are in a function +that has access to an inference context already, this should be the case. Specifically, this is the +case during HIR type checking or MIR borrow checking. + +Another consideration is normalization. Two types may actually be the same, but one is behind an +associated type. To compare them correctly, you have to normalize the types first. This is +primarily a concern during HIR type checking and with all types from a `TyCtxt` query +(for example from `tcx.type_of()`). + +When a `FnCtxt` or an `ObligationCtxt` is available during type checking, `.normalize(ty)` +should be used on them to normalize the type. After type checking, diagnostics code can use +`tcx.normalize_erasing_regions(ty)`. + +There are also cases where using `==` on `Ty` is fine. This is for example the case in late lints +or after monomorphization, since type checking has been completed, meaning all inference variables +are resolved and all regions have been erased. In these cases, if you know that inference variables +or normalization won't be a concern, `#[allow]` or `#[expect]`ing the lint is recommended. + +When diagnostics code does not have access to an inference context, it should be threaded through +the function calls if one is available in some place (like during type checking). + +If no inference context is available at all, then one can be created as described in +[type-inference]. But this is only useful when the involved types (for example, if +they came from a query like `tcx.type_of()`) are actually substituted with fresh +inference variables using [`fresh_substs_for_item`]. This can be used to answer questions +like "can `Vec<T>` for any `T` be unified with `Vec<u32>`?". + +[type-inference]: ./type-inference.md#creating-an-inference-context +[`fresh_substs_for_item`]: https://doc.rust-lang.org/beta/nightly-rustc/rustc_infer/infer/struct.InferCtxt.html#method.fresh_substs_for_item + ## `ty::TyKind` Variants Note: `TyKind` is **NOT** the functional programming concept of *Kind*. @@ -207,16 +251,16 @@ There are many variants on the `TyKind` enum, which you can see by looking at it - [**And many more**...][kindvars] [wikiadt]: https://en.wikipedia.org/wiki/Algebraic_data_type -[kindadt]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variant.Adt -[kindforeign]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variant.Foreign -[kindstr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variant.Str -[kindslice]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variant.Slice -[kindarray]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variant.Array -[kindrawptr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variant.RawPtr -[kindref]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variant.Ref -[kindparam]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variant.Param -[kinderr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variant.Error -[kindvars]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/enum.TyKind.html#variants +[kindadt]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variant.Adt +[kindforeign]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variant.Foreign +[kindstr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variant.Str +[kindslice]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variant.Slice +[kindarray]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variant.Array +[kindrawptr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variant.RawPtr +[kindref]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variant.Ref +[kindparam]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variant.Param +[kinderr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variant.Error +[kindvars]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/sty/enum.TyKind.html#variants ## Import conventions diff --git a/src/doc/rustc-dev-guide/src/type-checking.md b/src/doc/rustc-dev-guide/src/type-checking.md index 08f15e30b..b60694201 100644 --- a/src/doc/rustc-dev-guide/src/type-checking.md +++ b/src/doc/rustc-dev-guide/src/type-checking.md @@ -1,10 +1,12 @@ # Type checking -The [`rustc_typeck`][typeck] crate contains the source for "type collection" -and "type checking", as well as a few other bits of related functionality. (It -draws heavily on the [type inference] and [trait solving].) +The [`hir_analysis`] crate contains the source for "type collection" as well +as a bunch of related functionality. +Checking the bodies of functions is implemented in the [`hir_typeck`] crate. +These crates draw heavily on the [type inference] and [trait solving]. -[typeck]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/index.html +[`hir_analysis`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/index.html +[`hir_typeck`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_typeck/index.html [type inference]: ./type-inference.md [trait solving]: ./traits/resolution.md diff --git a/src/doc/rustc-dev-guide/src/type-inference.md b/src/doc/rustc-dev-guide/src/type-inference.md index ca88c1686..2bafeb247 100644 --- a/src/doc/rustc-dev-guide/src/type-inference.md +++ b/src/doc/rustc-dev-guide/src/type-inference.md @@ -36,18 +36,16 @@ signature, such as the `'a` in `for<'a> fn(&'a u32)`. A region is ## Creating an inference context -You create and "enter" an inference context by doing something like +You create an inference context by doing something like the following: ```rust,ignore -tcx.infer_ctxt().enter(|infcx| { - // Use the inference context `infcx` here. -}) +let infcx = tcx.infer_ctxt().build(); +// Use the inference context `infcx` here. ``` -Within the closure, -`infcx` has the type `InferCtxt<'a, 'tcx>` for some fresh `'a`, -while `'tcx` is the same as outside the inference context. +`infcx` has the type `InferCtxt<'tcx>`, the same `'tcx` lifetime as on +the `tcx` it was built from. The `tcx.infer_ctxt` method actually returns a builder, which means there are some kinds of configuration you can do before the `infcx` is |