Gemma for Streaming ML with Dataflow

The article demonstrates how to integrate Google's Gemma 2 language model into a Dataflow pipeline for real-time sentiment analysis and response generation in customer support chats. It provides a practical example of using Gemma to process streaming data, including code snippets for creating prompts, running inference, and handling model outputs within a scalable data processing framework.

https://developers.googleblog.com/en/gemma-for-streaming-ml-with-dataflow/

Pare

Pare is the easiest way to deploy Python Lambdas alongside your primary web application.

https://github.com/gauge-sh/pare

Coding a Multimodal (Vision) Language Model from scratch in PyTorch with full explanation

The video explains how to code a visual language model from scratch using PyTorch, covering topics such as vision transformers, contrastive learning, and language models. It provides a detailed walkthrough of implementing various components, including the vision encoder, language model, and how to combine image and text embeddings, with explanations of key concepts like attention mechani...

https://www.youtube.com/watch?v=vAmKB7iPkWw

OpenBB-finance / OpenBB

Investment Research for Everyone, Everywhere.

https://github.com/OpenBB-finance/OpenBB

Saving Scrapy Crawl Stats to PostgreSQL with a Custom Extension and SQLAlchemy

The post explains how to extend Scrapy to save crawl statistics directly into a PostgreSQL database, detailing the implementation process and configuration needed. It provides a guide for integrating Scrapy with PostgreSQL to enhance data management and analysis.

https://www.xiegerts.com/post/scrapy-extension-save-crawlstats-postgres/

Python's Preprocessor

Every now and then you hear outrageous claims such as “Python has no preprocessor”. This is simply not true. In fact, Python has the best preprocessor of all languages - it quite literally allows us to do whatever we want, and a lot more. It’s just a little tricky to (ab)use.

https://pydong.org/posts/PythonsPreprocessor/

PromptMage - simplifies the process of creating and managing LLM workflows

"PromptMage" is designed to offer an intuitive interface that simplifies the process of creating and managing LLM workflows. It facilitates prompt testing and comparison, and version control.

https://github.com/tsterbak/promptmage

llmio

A Lightweight Library for LLM I/O.

https://github.com/badgeir/llmio

Try LangChain with Python and Upstash Vector

The video covers setting up a Lang chain for custom data sets using a variety of tools like Python, Jupyter, and OpenAI, demonstrating API creation, embedding models, and vector database integration. It also discusses implementing rate limiting in a FastAPI application and setting up agents for querying and handling data.

https://www.youtube.com/watch?v=FjKMnszG8Dk

How to build a query language in Python

The article discusses the complexities and challenges of building a custom query language, highlighting key considerations such as syntax design, parsing, and execution. It provides insights and practical tips for developers looking to create or improve their own query languages.

https://jamesg.blog/2024/08/17/build-a-query-language/

RAG_Techniques

This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems.

https://github.com/NirDiamant/RAG_Techniques

Qik

Qik is a command runner that understands the import graph, allowing you to cache commands and only run the things that matter. Can dramatically speed up CI/dev in large monorepos.

https://github.com/Opus10/qik

LabelU

Data annotation toolbox supports image, audio and video data.

https://github.com/opendatalab/labelU