Exciting changes for python [LWN.net]

Exciting changes for python

Posted Jul 14, 2025 18:51 UTC (Mon) by mb (subscriber, #50428)
In reply to: Exciting changes for python by anselm
Parent article: Following up on the Python JIT

Python is a good language and I enjoy programming in it.
But all of my bigger projects eventually ran into performance and/or threading problems. No exceptions.
That's the main reason why I don't use Python for anything that has the potential to outgrow 1kLOC today.

And performance improvements of 50% or 100% don't really help, because compiled languages are *so* much faster.
I have reached almost-compiled performance with Cython in certain projects. But why would I want to write code in Cython, if much better compiled languages with much better type systems exist?

to post comments

Alternatives to Python

Posted Jul 15, 2025 9:53 UTC (Tue) by farnz (subscriber, #17727) [Link]

One thing I've done in the past, with PyCXX when I used C++ and PyO3 now that I use Rust most of the time is to rewrite the "critical path" code from Python into something more amenable to high performance.

The core idea is to use cProfile to find the bits of Python code that are bottlenecking you, then work out what architecture changes are needed to move the bottleneck code into a different language - which, of course, can include moving a chunk of non-bottleneck code across, too - leaving just glue code behind in Python.

Exciting changes for python

Posted Aug 1, 2025 14:09 UTC (Fri) by Stcbus (guest, #178625) [Link] (6 responses)

I am curious how you think Home assistant or even Instagram is handing it.

Exciting changes for python

Posted Aug 1, 2025 15:41 UTC (Fri) by mb (subscriber, #50428) [Link] (5 responses)

I don't know about these specific cases, but it's usually handled by not writing the performance critical parts in pure Python. There are dozens of ways to do that. I don't know which one(s) they choose.
And by throwing hardware resources at the problem, of course.

Exciting changes for python

Posted Aug 4, 2025 11:21 UTC (Mon) by anselm (subscriber, #2796) [Link] (4 responses)

it's usually handled by not writing the performance critical parts in pure Python

The time to do that is when it is actually worth the trouble, and when you know exactly where the performance critical parts are.

Instagram didn't get big and successful by initially writing a super-fast web site in a “performance” language; they got big and successful by quickly building something in Python that people really liked, and then spending time making it faster later when it was worth the trouble. Many other startups which worry a lot about performance up-front never even get to the stage where they're big enough to make worrying about performance actually worth the trouble, because they forget that at the start it is all about quickly building something that people really like. If you don't have users, nobody cares how fast your site is.

Exciting changes for python

Posted Aug 4, 2025 11:34 UTC (Mon) by farnz (subscriber, #17727) [Link]

And note, too, that there's a tradeoff between engineering time and your hosting bills. If your engineers can either add features that'll increase your income by between $5 million/year and $20 million/year, or sort out your performance so that you can reduce the hosting bill by $5 million/year, you'll add features because the extra income pays for the extra hosting costs, plus they increase the chances that you'll outcompete everyone else in your sphere.

This calculus changes somewhat if the compute is in boxes you sold your customers - replacing N smart home in the field with new ones might be too expensive compared to the rewrite - but for Internet applications, it's a very real tradeoff, and you don't start focusing on performance until you're big.

Exciting changes for python

Posted Aug 4, 2025 12:13 UTC (Mon) by mb (subscriber, #50428) [Link] (2 responses)

You say that companies should quickly write Python code do get things going and then rewrite stuff in another language when they realize that it's too slow?
Well, Ok. So? It's fine.
I just don't see how that makes Python "perfectly adequate". It's a business decision. Not a technology decision. It could very well be a technology lock-in by now.

I have just experienced the exact opposite of "Python performance is perfectly adequate for a large number of use cases".
In my experience it's only adequate if the performance critical part is actually written in some other compiled language.

The initial advantage of extremely fast development quickly turns around into extremely slow execution times and extremely slow development times due to dynamic typing getting out of hand. Did you ever do a major refactoring of some large Python code base? It takes years to find all corner cases and error paths. And you can never be sure that you got everything.

Exciting changes for python

Posted Aug 4, 2025 15:09 UTC (Mon) by anselm (subscriber, #2796) [Link] (1 responses)

The initial advantage of extremely fast development quickly turns around into extremely slow execution times and extremely slow development times due to dynamic typing getting out of hand.

I've been writing (mostly) Python code for a living for quite some time now and I'm doing just fine, thank you. In my team we tend to experience neither “extremely slow execution times” nor “extremely slow development times”. Our customers are happy, too. YMMV.

As far as dynamic typing is concerned, there are now quite good type checkers for Python and they're getting better all the time. It's definitely possible to write terrible code in Python just like it is possible to write terrible code in pretty much any other programming language. But OTOH it's also possible in Python – perhaps even more so than in some other popular programming languages – to write good-looking code that is correct, maintainable, memory-safe, and reasonably fast. Also in fairness it must be said that current versions of Python are noticeably faster than they used to be in the past. Will Python code ever be as fast as, say, compiled Rust code? Probably not. Does it matter? It depends. But I wouldn't discount Python out of hand.

Exciting changes for python

Posted Aug 4, 2025 15:57 UTC (Mon) by mb (subscriber, #50428) [Link]

>As far as dynamic typing is concerned, there are now quite good type checkers for Python and they're getting better all the time.

Sure. If you use strict typing then maintainability is fine. But that throws duck typing out of the window and brings every downside from static typing languages into Python. And you still end up with a program written in a slow-by-design language that was at least as hard to write as a program in a compiled language.

>it must be said that current versions of Python are noticeably faster than they used to be in the past.

True. Today sometimes we get only a performance penalty of 5 times instead of 10 times. Maybe.
Are we ever going to reach 1:1 with a compiled language? No. Well, maybe yes, but then every feature that makes Python superior for prototyping has been thrown out of the window.

I think Python is a wonderful language and I write code in Python all the time.
The language is extremely powerful and fun for writing small scripts and prototypes.
But it's important to know the limits and notice when it's time to completely rewrite the project in a proper compiled language before the maintenance dance begins.