Classifying all of the pdfs on the internet

The article describes an attempt to classify a massive dataset of 8.4 million PDFs from Common Crawl using various machine learning techniques. The author experiments with different approaches, including deep learning models and traditional machine learning methods like XGBoost, ultimately achieving the best performance with an XGBoost model trained on embeddings, reaching 85.26% accurac...

https://snats.xyz/pages/articles/classifying_a_bunch_of_pdfs.html

Mini-Omni

Mini-Omni is an open-source multimodel large language model that can hear, talk while thinking. Featuring real-time end-to-end speech input and streaming audio output conversational capabilities.

https://github.com/gpt-omni/mini-omni

How to Create a Pre-Commit Hook

A step-by-step guide to developing your own pre-commit hook.

https://stefaniemolin.com/articles/devx/pre-commit/hook-creation-guide/

pipefunc

Lightweight function pipeline (DAG) creation in pure Python for scientific workflows.

https://github.com/pipefunc/pipefunc

smartcut

Cut video files with minimal recoding.

https://github.com/skeskinen/smartcut

Using GPT-4o for web scraping

The article discusses using GPT-4 with OpenAI's structured outputs feature to create an AI-assisted web scraper, exploring its capabilities in parsing complex tables and generating XPaths. While the author found GPT-4 effective at extracting data from various HTML tables, they also noted challenges with merged rows, high API costs, and the need for further refinements to improve accuracy...

https://blancas.io/blog/ai-web-scraper/

Integrating Stripe Into A One-Product Django Python Shop

In the first part of this series, we created a Django online shop with htmx. In this second part, we'll handle orders using Stripe.

https://blog.appsignal.com/2024/09/04/integrating-stripe-into-a-one-product-django-python-shop.html

My Favorite Error Handling Technique

This video presents a surprising “Let it burn” approach to error handling, demonstrating how allowing code to fail fast can result in simpler, clearer, and more robust software. Discover the benefits of this method and its impact on improving overall code quality.

https://www.youtube.com/watch?v=YA0Wq1rcs6U

Pure Python: Build a full stack ChatGPT-like UI. Reflex, Neon Postgres. Deploy with Docker to a VM

This video tutorial demonstrates how to build a full-stack ChatGPT-like UI using Reflex, a Python framework for web development, integrating it with Neon Postgres database and OpenAI. It covers the entire process from setting up the development environment to deploying the application using Docker, GitHub Actions, and Ansible on a virtual machine.

https://www.youtube.com/watch?v=NuNaI__4xiU

kazam

Linux Screen Recorder, Broadcaster, Capture and OCR with AI in mind.

https://github.com/henrywoo/kazam

cookiecutter-uv

A modern cookiecutter template for Python projects that use uv for dependency management.

https://github.com/fpgmaas/cookiecutter-uv

Shades of testing HTTP requests in Python

The post discusses various approaches to testing HTTP requests in Python applications, focusing on mocking external API calls during unit and integration testing.

https://rednafi.com/python/testing_http_requests/

nlp-zero-to-hero

A comprehensive resource for learning Natural Language Processing (NLP) from the basics to advanced topics. It contains Jupyter notebooks covering various NLP concepts, techniques, and implementations, making it a valuable guide for beginners and intermediate learners in the field of NLP.

https://github.com/JUSTSUJAY/nlp-zero-to-hero

PyPI Proxying for Docker Builds

I wanted to improve our CI system by caching PyPI data locally. I saw that there’s a project to do this, but I didn’t see any good examples actually using it.

https://www.robopenguins.com/pypi-proxy/

Multiversion Python Thoughts

A braindump on how to make multi version in Python work.

https://lucumr.pocoo.org/2024/9/9/multiversion-python/