The Compact C Type Format in the GNU toolchain [LWN.net]

The Compact C Type Format in the GNU toolchain

Posted Aug 6, 2019 21:38 UTC (Tue) by markjdb (guest, #94056)
In reply to: The Compact C Type Format in the GNU toolchain by nix
Parent article: The Compact C Type Format in the GNU toolchain

Is there a specification somewhere for the updated format? I recently wrote a new libctf and DWARF->CTF converter for FreeBSD with the aim of not having to run ctfconvert on each CU before linking. The limits you mention are indeed quite annoying and I'd be interested in adopting a newer CTF version if someone's already thought through the details.

to post comments

The Compact C Type Format in the GNU toolchain

Posted Aug 6, 2019 23:00 UTC (Tue) by nix (subscriber, #2304) [Link] (1 responses)

There will be a spec by the end of the month -- and I'll add the plans for v4 into it as well so people can tell what is planned on the format front. v3 is definitely less compact than FreeBSD's format, and v4 will get that compactness back again for all but very large projects that really need the increased type range.

If we could cooperate on future improvements, I'd be very happy! (But I fear that, as usual, code reuse is difficult because libctf is in the GNU toolchain, so it's GPL... how annoying, a planned format oracle that its principal beneficiaries will probably refuse to use :( but that, I suppose, is why the format needs documentation as well.)

The Compact C Type Format in the GNU toolchain

Posted Aug 6, 2019 23:47 UTC (Tue) by markjdb (guest, #94056) [Link]

I would be happy with "just" documentation. :)

The Compact C Type Format in the GNU toolchain

Posted Aug 7, 2019 9:27 UTC (Wed) by movement (subscriber, #871) [Link] (3 responses)

Any reason you didn't use illumos's newer CTF tools, which also don't need a separate ctfconvert step for each .o?

The Compact C Type Format in the GNU toolchain

Posted Aug 8, 2019 5:03 UTC (Thu) by markjdb (guest, #94056) [Link] (2 responses)

They still work by generating CTF for each CU, and then pairwise merging the CTF files. It is faster to generate the type graph directly from DWARF, or at least without using CTF as an intermediate representation.

The Compact C Type Format in the GNU toolchain

Posted Aug 8, 2019 8:53 UTC (Thu) by movement (subscriber, #871) [Link]

I see, thanks.

The Compact C Type Format in the GNU toolchain

Posted Aug 9, 2019 11:35 UTC (Fri) by nix (subscriber, #2304) [Link]

One of our motivations for getting this stuff in GCC is that it's even faster to not generate the DWARF at all, particularly as it is often so large that it overflows caches and now you're incurring disk write time for all that stuff, which if you only want the CTF is completely wasted. (Of course you can build both in if you want to, but you don't *have* to.)