Why parallelise the parser? [LWN.net]

Why parallelise the parser?

Posted Nov 1, 2024 13:58 UTC (Fri) by anton (subscriber, #25547)
Parent article: The performance of the Rust compiler

Does Rust generate huge output from text-macro processing? Or what makes the parsing such a time-consuming operation that parallel processing of that part appears to be a promising way towards smaller elapsed compile times? My expectation is that the compiler spends nearly all of its time in the checking parts and in the LLVM back end.

to post comments

Why parallelise the parser?

Posted Nov 1, 2024 15:49 UTC (Fri) by atnot (guest, #124910) [Link]

My probably outdated memory is that the main point is being able to kick stuff to LLVM sooner. As mentioned the codegen is parallel already, but those threads only get spawned sequentially and all of them have to finish, across all crates, before linking can start. This means that if, say, you get unlucky and a heavy task gets started towards the end, you'll be waiting a while. With a parallel frontend you get to start those tasks earlier, which indeed doesn't affect the bulk of the runtime, but lessens the impact from stagglers. The parser is particularly relevant bottleneck there because you can't really start to divide up the work until you've traced all of the modules and imports and know what you need to compile.