PythonHub
2.37K subscribers
2.35K photos
49K links
News & links about Python programming.
https://pythonhub.dev/
Download Telegram
You Should Probably Pay Attention to Tokenizers

This article emphasizes the importance of understanding tokenizers in AI applications, particularly for Retrieval-augmented generation (RAG) systems. The author demonstrates how different tokenizers handle various types of text input, including emojis and misspelled words, and explains how tokenization affects embedding quality and overall performance in natural language processing tasks.

https://cybernetist.com/2024/10/21/you-should-probably-pay-attention-to-tokenizers/
From Python to CPU instructions: Part 1

In the first part of a two-part series, we’ll compare the same program written in C and Python to reveal what Python hides from us.

https://dilovan.substack.com/p/from-python-to-cpu-instructions-part
Investigation of a Workbench UI Latency Issue

Netflix engineers investigated a JupyterLab UI latency issue in their Workbench product, tracing it to an unexpected interaction between a resource usage extension and memory allocation. The root cause was identified as the extension's performance degrading linearly with increased virtual memory usage, despite available physical memory.

https://netflixtechblog.com/investigation-of-a-workbench-ui-latency-issue-faa017b4653d
ClickPy

PyPI analytics powered by ClickHouse.

https://clickpy.clickhouse.com/