Namespaces for the Python Package Index [LWN.net]

Namespaces for the Python Package Index

Posted May 4, 2023 17:26 UTC (Thu) by mathstuf (subscriber, #69389)
In reply to: Namespaces for the Python Package Index by NYKevin
Parent article: Namespaces for the Python Package Index

Didn't Python have UCS-2 or UTF-16 as its internal encoding for the longest time? That's a lesson Java should have taught anyone to avoid…

to post comments

Namespaces for the Python Package Index

Posted May 5, 2023 2:25 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

No, it was even dumber than that. Python had "narrow" and "wide" builds, which used UCS-2 and UCS-4 respectively (i.e. the language was not aware of surrogate pairs and treated them as two characters). To a first approximation, the Windows builds were narrow and the Linux builds were (mostly) wide (and I have no idea what they did for macOS).

This was all cleaned up in Python 3. Now, strings are sequences of abstract code points, and the encoding is an internal implementation detail. If you want to use "bytes encoded in UTF-8" instead, you can easily do that, but it's just not what the language does by default.