PythonHub
2.61K subscribers
2.35K photos
50.4K links
News & links about Python programming.
https://pythonhub.dev/
Download Telegram
Prototype on a laptop, scale to 16 billion rows: one Polars query

The post shows how to prototype a data pipeline locally on 97 million Polymarket orderbook rows, then run the same LazyFrame queries on 16 billion rows using distributed execution. The pipeline pre-aggregates the raw data into small Parquet artifacts in S3 that power a responsive Plotly Dash dashboard without scanning the full dataset on each request.

https://pola.rs/posts/market-data-to-plotly-enterprise-dashboard/
jsonfold: Making Pretty-Printed JSON Compact and Readable in Python

jsonfold is a streaming post-filter that compacts pretty-printed JSON by folding small arrays and objects onto single lines while preserving existing serializers and custom encoders. It processes JSON incrementally without reparsing the document, keeping additional memory usage bounded while producing more compact, human-readable output for large JSON files.

https://medium.com/@yair.lenga/a-streaming-json-formatter-that-works-with-existing-serializers-eced220da37d
Thinking in Python by Bruce Eckel

https://thinkinginpython.com/
django-waffle - the best Feature Flipper for Django!

This is an introduction to django-waffle, a production-ready Django package for controlling feature rollouts through flags, switches, and sampling. It enables gradual releases, A/B testing, and quick kill switches for features that need to be disabled without redeploying

https://www.youtube.com/watch?v=K-Y9XHEZwU8
The Polars vs SQL differences nobody is talking about

This post explores important differences between Polars and SQL that can cause unexpected results when moving between the two, including row ordering, column independence, literals, null handling, and broadcasting. It explains how understanding these different mental models can help developers write safer Polars code and more reliably translate data workflows between Polars and SQL.

https://labs.quansight.org/blog/polars-vs-sql-differences