Namespaces for the Python Package Index [LWN.net]

Namespaces for the Python Package Index

Posted May 4, 2023 6:35 UTC (Thu) by NYKevin (subscriber, #129325)
In reply to: Namespaces for the Python Package Index by SLi
Parent article: Namespaces for the Python Package Index

> Flat is better than nested.

(The Zen of Python, PEP 20)

In more explicit terms:

* Nobody likes typing out com.example.foo a hundred million times in their import statements.
* Who is to say whether example.com will still be owned by the same people in a year or ten?
* There needs to be a process for PyPI to reclaim an abandoned, compromised, or maliciously-used name, and that process should not involve IANA and/or UDRP.
* I shouldn't need to buy a whole domain for a hobby project (and I am far more likely to do a hobby project in Python than in Java).

Regardless, this surely is not just a local-only thing, because namespace packages have been around for a very long time, and should totally satisfy that need for people doing their own custom deployments. See [1] for details. If this article is to make any sense at all, it must be talking about some sort of *centralized* standard.

[1]: https://packaging.python.org/en/latest/guides/packaging-n...

to post comments

Namespaces for the Python Package Index

Posted May 4, 2023 7:43 UTC (Thu) by lunaryorn (guest, #111088) [Link] (7 responses)

I once heard the theory that typosquatting is largely not a problem in the Java ecosystem because these reverse DNS group names make artifact names so long that on one actually types them out; people copy them from elsewhere (e.g. search.maven.com, Github, etc) which makes it much harder to trick people into copying wrong artifact names. And there are probably way to many potential typos to attempt typosquatting on Maven Central.

I actually really like the Java naming system; I think it's one of the few things Java really got right from the start. I believe it helps to manage trust because you can selectively delegate trust along the namespace hierarchy. It also supports routing in proxy repositories: You can summarily accept certain root namespaces into your proxy repository, require manual verification for others (e.g. the whole io.github hierarchy), and make sure that your own "com.example.your-company.internal" packages never get resolved from a public repository.

It also moves load off maintainers in a central repository: Sonatype can afford a comparatively strict and cumbersome application process, because they only check group names once, whereas moderation on PyPi or crates.io has no chance but to check each and every package.

All this is somewhat impossible to do with flat packaging as in Rust or Python. Sure these names are harder to type out, but that's what tooling exists for, in my opinion.

Namespaces for the Python Package Index

Posted May 4, 2023 7:52 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (4 responses)

The whole point of Python is to move away from the Java style of doing things (i.e. big enterprisey hierarchical structures). If you want Java, then use Java.

Namespaces for the Python Package Index

Posted May 4, 2023 9:56 UTC (Thu) by SLi (subscriber, #53131) [Link]

That sounds a bit like the purpose is to do it differently only because Java does it this way. As I said, I don't understand the upsides and downsides of the Java system, but I'm sure "Java does it this way, so this can't be good" is not a good justification.

Namespaces for the Python Package Index

Posted May 4, 2023 13:53 UTC (Thu) by lunaryorn (guest, #111088) [Link]

I have a déjà vu. Didn't they say the same (or, for the more "elite" part of the community, rather "If you want Haskell, then use Haskell") about PEP 484 and the whole typing thing? And look where we are today… ;)

Namespaces for the Python Package Index

Posted May 4, 2023 17:26 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (1 responses)

Didn't Python have UCS-2 or UTF-16 as its internal encoding for the longest time? That's a lesson Java should have taught anyone to avoid…

Namespaces for the Python Package Index

Posted May 5, 2023 2:25 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

No, it was even dumber than that. Python had "narrow" and "wide" builds, which used UCS-2 and UCS-4 respectively (i.e. the language was not aware of surrogate pairs and treated them as two characters). To a first approximation, the Windows builds were narrow and the Linux builds were (mostly) wide (and I have no idea what they did for macOS).

This was all cleaned up in Python 3. Now, strings are sequences of abstract code points, and the encoding is an internal implementation detail. If you want to use "bytes encoded in UTF-8" instead, you can easily do that, but it's just not what the language does by default.

Namespaces for the Python Package Index

Posted May 4, 2023 8:45 UTC (Thu) by epa (subscriber, #39769) [Link] (1 responses)

Domain names can and do contain the - character. How do you include that in a Java namespace?

Namespaces for the Python Package Index

Posted May 4, 2023 8:58 UTC (Thu) by lunaryorn (guest, #111088) [Link]

The established convention would be to use an underscore instead.

Namespaces for the Python Package Index

Posted May 4, 2023 10:05 UTC (Thu) by Karellen (subscriber, #67644) [Link]

> Flat is better than nested.

Don't namespaces inherently introduce at least one level of nesting? Given that a DNS-based naming system will only give 2 levels of nesting in the common case (e.g. instead of "mynamespace.mypackage" you'll often end up with "tld.mydomain.mypackage"), is it that much of a big deal?

> Who is to say whether example.com will still be owned by the same people in a year or ten?

Won't you need a process for retiring/reusing namespaces anyway, if the original owner dies? How is this different?

> There needs to be a process for PyPI to reclaim an abandoned, compromised, or maliciously-used name,

Couldn't they just decide to block an arbitrary set of domains within PyPI? So they could reject packages for the "ru.h4x0r" namespace without involving IANA?

> I shouldn't need to buy a whole domain for a hobby project

Then you can use an un-namespaced package name, as you do now. It'll still the existing problems of collisions and squatting, but no worse than the situation currently is. Or PyPI could designate "local." as a private use namespace, mirroring the ".local" special use tld.

https://en.wikipedia.org/wiki/.local

Namespaces for the Python Package Index

Posted May 4, 2023 11:15 UTC (Thu) by ebassi (subscriber, #54855) [Link] (1 responses)

> Flat is better than nested.

30 years of Linux distributions should have thoroughly debunked this mantra; and that's a limited, heavily gated pool of components. Once you open it to a whole ecosystem and minimise the friction towards publishing, you have the perfect recipe for a disaster.

Very, very few people sign up for curating the wild west, and you can only get so far with minimal volunteer resources.

The idea of a flat namespace in which everyone plays nice and nobody squats on a name they just reserved for a rainy day project that never comes is naive at best; it's like thinking that the Internet is still the same as it was in the '90s.

> I shouldn't need to buy a whole domain for a hobby project (and I am far more likely to do a hobby project in Python than in Java)

That would have been a problem before GitHub/GitLab/source hosting services; you can use a reverse domain like com.github.yourusername.yourproject, and satisfy the requirements for a domain. If the verification process is automated, and based on something like a token held in a file accessible via a well-known URL, then you can publish it using something like GitHub or GitLab pages, which both reflect your user name as the namespace.

Namespaces for the Python Package Index

Posted May 4, 2023 16:48 UTC (Thu) by kpfleming (subscriber, #23250) [Link]

Slight tangent, but even that sort of system is subject to abuse... a user on the Bluesky social media network has 'registered' s3.amazonaws.com as their domain, since they were able to place a file there and have it pass the verification check :-)