Crawling Pages with Infinite Scroll using Scrapy and Playwright

This post provides a detailed guide on how to scrape infinite scroll websites using Scrapy and Playwright in Python. It covers the setup process, explains how to implement a custom downloader middleware to handle JavaScript rendering, and demonstrates how to extract data from dynamically loaded content, offering a practical solution for web scraping challenges posed by modern web applica...

https://www.xiegerts.com/post/infinite-scroll-scrapy-playwright/

AgentK

An autoagentic AGI that is self-evolving and modular.

https://github.com/mikekelly/AgentK

Unstract

No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents.

https://github.com/Zipstack/unstract

pgai

Bring AI models closer to your PostgreSQL data.

https://github.com/timescale/pgai

emval

emval is a blazingly fast Python email validator written in Rust.

https://github.com/bnkc/emval

Some more batteries to do stuff with Mapping related data structures

This library provides utility functions for manipulating and transforming data structures which have or include Mapping-like characteristics.

https://github.com/erivlis/mappingtools

CSVs Are Kinda Bad. DSVs Are Kinda Good.

The article argues that CSVs (Comma-Separated Values) are problematic due to various edge cases involving delimiters, quotes, and newlines, and proposes using Delimiter-Separated Values (DSV) with ASCII control characters as a more robust alternative. It demonstrates how DSVs can handle complex data without escaping or quoting issues, but acknowledges that the lack of widespread tool sup...

https://matthodges.com/posts/2024-08-12-csv-bad-dsv-good/

goauthentik / authentik

The authentication glue you need.

https://github.com/goauthentik/authentik

Deep-Live-Cam

Real time face swap and one-click video deepfake with only a single image.

https://github.com/hacksider/Deep-Live-Cam

DeltaDB

DeltaDB is a lightweight, fast, and scalable database built on polars and deltalake.

https://github.com/uname-n/deltadb

Automating Boolean-Based SQL Injection with Python

How to be efficiently lazy at finding hidden gems in predictable places – Database Edition.

https://trebledj.me/posts/automating-boolean-sql-injection-with-python/

llm_aided_ocr

Enhance Tesseract OCR output for scanned PDFs by applying Large Language Model (LLM) corrections.

https://github.com/Dicklesworthstone/llm_aided_ocr

From Boring Object-Oriented to INSANE Functional Code

This video demonstrates that there's a place for both object-oriented and functional code. In Python, these two approaches can be combined effectively, allowing you to leverage the strengths of each for the best results.

https://www.youtube.com/watch?v=DvdZv_DD0DY

EDSL

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.

https://github.com/expectedparrot/edsl

Cloudflare R2 x Django. Static Files. User uploads, css, images, js and more. Production-ready.

The video covers setting up and managing Django files, including static and user-uploaded files, using Cloudflare's R2 object storage. It emphasizes best practices for configuring environment variables, securing API keys, and managing static and media files in Django with advanced validation and customization options.

https://www.youtube.com/watch?v=VU3MAN1gs1s

Gemma for Streaming ML with Dataflow

The article demonstrates how to integrate Google's Gemma 2 language model into a Dataflow pipeline for real-time sentiment analysis and response generation in customer support chats. It provides a practical example of using Gemma to process streaming data, including code snippets for creating prompts, running inference, and handling model outputs within a scalable data processing framework.

https://developers.googleblog.com/en/gemma-for-streaming-ml-with-dataflow/