An overview of structural pattern matching for Python [LWN.net]

An overview of structural pattern matching for Python

Posted May 4, 2022 0:41 UTC (Wed) by tialaramex (subscriber, #21167)
Parent article: An overview of structural pattern matching for Python

Structural pattern matching is really nice.

I wondered why they went with a statement, because expression match feels nicer to me. But PEP 635 says they explicitly considered this question and decided against an expression, and they know much more about Python and about Python programmers than I ever will.

However it is painful to see, also in PEP 635, how problematic new keywords are for Python. If the language wants to go around adding keywords (and it seems it does based on this) it needs to find a way to make that less painful, it got lucky for match but it can't count on always getting lucky.

to post comments

An overview of structural pattern matching for Python

Posted May 4, 2022 6:56 UTC (Wed) by epa (subscriber, #39769) [Link] (11 responses)

I don’t think there is any language that can easily add new keywords. You always end up with ugliness like _bool. Perl started out with an explicit & prefix for subroutine calls but soon dropped it because everyone prefers to see a bare word, even though it will make it hard to add new builtin functions or keywords later.

All in all, I favour the approach of adding the new keyword in a new version of the language, and let everyone change their code where it clashes, and if you don’t want to do that then don’t upgrade.

The context-aware parser in Python is great as a way to do a smooth upgrade, but I would still appreciate a warning if I have used keywords like match and case as variable names. I would still want to change them everywhere to avoid confusing humans, even if the parser can cope.

(Stroustrup wrote about how C’s ‘bool’ as a header file #define does not really solve any of the keyword clash problems and it would have been better to just add the keyword properly. Anyone have a link to that?)

An overview of structural pattern matching for Python

Posted May 4, 2022 7:12 UTC (Wed) by NYKevin (subscriber, #129325) [Link]

> The context-aware parser in Python is great as a way to do a smooth upgrade, but I would still appreciate a warning if I have used keywords like match and case as variable names. I would still want to change them everywhere to avoid confusing humans, even if the parser can cope.

You can probably hack that together with ast.walk() and look for ast.Name instances with a "bad" name. I'm pretty sure that's, like, four lines of code. See https://docs.python.org/3/library/ast.html for documentation.

An overview of structural pattern matching for Python

Posted May 4, 2022 7:43 UTC (Wed) by mpr22 (subscriber, #60784) [Link] (1 responses)

> I don’t think there is any language that can easily add new keywords.

The Edition mechanism in Rust, with its "raw identifier" syntax, looks like the best approach I've seen so far.

An overview of structural pattern matching for Python

Posted May 4, 2022 21:53 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

Right now, Rust's approach lets them smoothly add a keyword in the medium term. This is how Rust added, for example, async.

Suppose Rust decided it must have a "new" keyword. Rust 2024 edition could reserve this keyword, and people writing Rust 2024 would be obliged to write r#new when they meant the identifier named "new" while all existing code still compiles. They could then (any time after Rust 2024 "ships") use the reserved keyword for whatever purpose they had in mind in Rust 2024. Since lots of things in Rust are named new e.g. Vec::new() is a very common function call, and would need to be re-written Vec::r#new() this is awkward but it would work.

However, if things were more urgent, say the "new" feature was so critical that otherwise conservative languages like C all rush to provide it, and Rust must do likewise, raw identifiers aren't enough. As I understand it, the intent in that case is to ship an entire feature to provide this capability, named raw keywords. k#new would be the (awkwardly spelled) new keyword immediately, and then Rust 2024 would bless the actual name "new" for it and do all the tidying up in a few years.

The entire namespace where "raw keywords" could live is reserved, so like __Bool in C this is merely awkward spelling. If anything the biggest problem is picking opportunities to use it.

An overview of structural pattern matching for Python

Posted May 4, 2022 9:29 UTC (Wed) by Wol (subscriber, #4433) [Link]

> I don’t think there is any language that can easily add new keywords. You always end up with ugliness like _bool. Perl started out with an explicit & prefix for subroutine calls but soon dropped it because everyone prefers to see a bare word, even though it will make it hard to add new builtin functions or keywords later.

DataBASIC (depending on your variant) uses REM to mean four different things. Different compilers interpret it differently.

I always used to use it as a variable, but it was also a function (REMainder). Then someone added using it as a statement (REMark). And you could also use it as a label ...

Which variants of REM your compiler accepted depended on the compiler. I think modern versions accept every variation except maybe the variable. Which is a pain, because one of the biggest implementations of its time used it that way ...

Cheers,
Wol

An overview of structural pattern matching for Python

Posted May 4, 2022 18:21 UTC (Wed) by jezuch (subscriber, #52988) [Link] (2 responses)

> I don’t think there is any language that can easily add new keywords.

There's Java :) Kind of, at least. They've recently added a couple of keywords by using a trick: they're context-dependent. See for example "yield" in the new enhanced switch. The latest proposals of pattern-matching switch go even further, adding "when" keyword as a case guard.

The current design is here: https://openjdk.java.net/jeps/427 As a bonus you can compare it with what is described in the article ;)

An overview of structural pattern matching for Python

Posted May 5, 2022 15:22 UTC (Thu) by kokada (subscriber, #92849) [Link] (1 responses)

Well, from the looks of the text it does looks to me that `match` in Python s also context-dependent. The text even gave an example on how you could have both a variable called `match` and a `match` statement one after another (with the amusing resulting `match match:`).

An overview of structural pattern matching for Python

Posted May 7, 2022 1:48 UTC (Sat) by smitty_one_each (subscriber, #28989) [Link]

Right. "Soft" keywords, due to the PEG parser.

An overview of structural pattern matching for Python

Posted May 4, 2022 18:33 UTC (Wed) by bartoc (guest, #124262) [Link] (3 responses)

Algol 68, and Oberon-7 (among other versions) put keywords in a totally separate namespace and reserve the whole space. Everything ALLCAPS in oberon is a keyword, and you must not use such things as identifiers. Similarly algol 68 required surrounding keywords in back-ticks or something.

The middle ground is allowing some kind of stropping to turn what would be a keyword into an identifier, that way you can “update” in an automated fashion

if you have a module system the parser can just parse according to what the module expects.

An overview of structural pattern matching for Python

Posted May 7, 2022 1:51 UTC (Sat) by smitty_one_each (subscriber, #28989) [Link] (2 responses)

If I ever get around to crafting a language, I would achieve this by having editor support for wrapping all keywords in JSON, e.g. {'for':'kw'} so that it looks like `for` but lets the coder use `for` for whatever else is interesting. [Waves hands as to how that would actually work from a UX perspective.]

An overview of structural pattern matching for Python

Posted May 7, 2022 18:30 UTC (Sat) by NYKevin (subscriber, #129325) [Link] (1 responses)

Personally, I don't want to write code in rich text.

An overview of structural pattern matching for Python

Posted May 9, 2022 2:04 UTC (Mon) by smitty_one_each (subscriber, #28989) [Link]

Obviously editor support would be a going-in requirement.