Data eXplore : Data Science, ML, Big Data, LLMs and AI Security
582 subscribers
845 photos
446 videos
1 file
675 links
Exploring Data Science, Big Data Analytics & Visualization, ML/DL, Neural Networks, LLMs with GitHub, Kaggle, HuggingFace and some white papers by big institutions.

Not just data, but science behind data

Paid project? premodi@zohomail.in
β˜… @DataML
Download Telegram
How MiniMax M2.1 made?

When they say that one model writes code better than another, they usually mean the SWE-Bench benchmark. The model gets a real bug from a real project on Github, which it has to read, find the error and fix it. This partially resembles a programmer's daily work.

🟒 How MiniMax-AI became a truly universal AI programmer?

SWE-Bench benchmark has its drawbacks. They found the answer and implemented it in their latest model M2.1.

1️⃣ LANGUAGE BARRIER

Problem: SWE-Bench only works with Python. In the real world, developers deal with Java, Go, TypeScript, Rust, C++ and a bunch of other languages.

⁠☞ Solution: SCALING THE ENVIRONMENT

Behind this vague term lies a huge system that operates with popular languages: JS, TS, Python, Java, Go, C++ and Rust.

For this, more than 100 thousand real tasks with a description of the problem, code and tests were collected from GitHub. This was not easy, as complex languages (Java or C++) require setup and each language has its own frameworks and dependency management systems.

To train the model on such a dataset, MiniMax built an infrastructure capable of running more than 5 thousand isolated execution environments in the shortest possible time - 10 seconds.

2️⃣ "BUG-FIX ONLY" TRAP
The Problem: Most benchmarks are is only about fixing bugs, while programmers also write new functions, refactor and optimize.

☞ The Solution: GOING BEYOND BUG FIXES:

MiniMax-M2.1 was also trained to generate tests, and it turned out that this is a critically important skill.

The previous version, M1, wrote too simple tests and often chose the wrong solutions. M2.1 excelled in this and equaled the results of the powerful competitor Claude Sonnet 4.5.

It also learned to optimize code performance - on SWE-Perf it showed an average increase in efficiency of 3.1%.

And finally, M2.1 was taught to do Code Review, for which an internal benchmark SWE-Review was created.

3️⃣ ENVIRONMENT DEPENDENCY
The Problem: A model's results strongly depend on the environment in which the model operates.

⁠☞ The Solution: GENERALIZATION ON OOD SCAFFOLDS.

The model should equally well follow long instructions and adapt to different ways of managing the context of the dialogue.

The team conducted tests in mini-swe-agent, Droid and Claude Code and if you look at the figures from their comparative table, you can see that the model has become much more flexible and versatile.

On the same SWE-Bench, when using Claude Code, MiniMax-M2.1 scored 74 points, which is higher than the model M2 with its 69.2 points, and almost on a par with Claude Sonnet 4.5 and DeepSeek V3.2.

On another test, OctoCodingBench, the gap is even greater: 26.1 for the new model against 13.3 for the old one.


Seems that concept of an "AI coder" is becoming more and more real. Success of MiniMax-M2.1 showed that it's no longer about writing individual lines of code, but about a comprehensive understanding of the entire development process.

#AI #ML #LLM #MiniMaΡ…

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
12 new advanced types of RAG

β–ͺ️ Mindscape-Aware RAG (MiA-RAG)
β–ͺ️ Multi-step RAG with hypergraph-based memory
β–ͺ️ QuCo-RAG
β–ͺ️ HiFi-RAG
β–ͺ️ Bidirectional RAG
β–ͺ️ TV-RAG
β–ͺ️ MegaRAG
β–ͺ️ AffordanceRAG
β–ͺ️ Graph-O1
β–ͺ️ SignRAG
β–ͺ️ Hybrid RAG for multilingual question answering based on documents
β–ͺ️ RAGPart and RAGMask


β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
Claude Code + Integration with Supabase

Step-by-step tutorial:
How to install and use 2 agents, 8 commands for the database, and an MCP server to automate development for Supabase in Claude Code.

Claude Code Stack for Supabase:

Claude Code Templates offers three ready-made components for integrating with Supabase:

1️⃣ Agents:

Β» Supabase Schema Architect
An expert in database design, migrations, and RLS policies. Automatically analyzes requirements and generates production-ready schemas with a focus on performance.

Β» Supabase Realtime Optimizer
A specialist in WebSocket optimization and real-time. Monitors connections, optimizes subscriptions, and helps maintain scalable real-time without degradation.

2️⃣ Commands:

Β» supabase-schema-sync
Synchronization of local and remote schemas, version control, and automatic detection of schema drift.

Β» supabase-migration-assistant
Generation, management, and application of database migrations with rollback support and conflict resolution.

Β» supabase-performance-optimizer
Analysis of query performance, index recommendations, and optimization of database operations for maximum speed.

Β» supabase-security-audit
Full security audit, validation of RLS policies, and vulnerability detection with automatic fixes.

Β» supabase-backup-manager
Scheduled backups, recovery procedures, and DR planning with testing.

Β» supabase-type-generator
Generation of TypeScript types from the database schema, support for type safety, and automatic updates when the schema changes.

Β» supabase-data-explorer
Interactive data viewing, visual query builder, and export with filters.

Β» supabase-realtime-monitor
Monitoring of real-time connections, performance tracking, and WebSocket diagnostics.


3️⃣ Supabase MCP Server. Direct integration with the Supabase API via MCP. Gives Claude Code native access to the project: executing commands, working with schemas and data, managing real-time, and security without unnecessary intermediaries.

Before installation, you can view all available Supabase components on the official Claude Code Templates website.

Go to aitmpl.com and find supabase


INSTALLATION OPTIONS:
There are several ways to install the Supabase stack for Claude Code. Choose the one that best suits you:

1️⃣ Installing individual components

# Install a specific agent
npx claude-code-templates@latest --agent database/supabase-schema-architect

# Install a specific Supabase command
npx claude-code-templates@latest --command database/supabase-schema-sync

# Install the MCP server
npx claude-code-templates@latest --mcp database/supabase


The components will be installed in:

* πŸ“.claude/commands/
* πŸ“.claude/agents/
* πŸ“.mcp.json

2️⃣ Creating global agents (available in any project)

# Create global agents available from any project
npx claude-code-templates@latest --create-agent database/supabase-schema-architect
npx claude-code-templates@latest --create-agent database/supabase-realtime-optimizer

# Show the list of all global agents
npx claude-code-templates@latest --list-agents

# Update a global agent
npx claude-code-templates@latest --update-agent database/supabase-schema-architect

# Remove a global agent
npx claude-code-templates@latest --remove-agent database/supabase-realtime-optimizer


β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore & @PromptXplore
This media is not supported in your browser
VIEW IN TELEGRAM
Microsoft has really turned the tables 🀯

They've long since released the open-source bitnet.cpp - a framework for inference of 1-bit LLMs.

It allows you to run models with 100B parameters directly on the local CPU, without any GPUs.

- inference is 6.17 times faster
- energy consumption on the CPU is 82.2% lower

And yes, it's 100% open source.

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
Cursor is completely switching to dynamic context for all models

Means the agent (based on any model) will now primarily collect context on its own, rather than using what has been provided.

🟒 How is Dynamic Context different from "Classic" approach?

Static context is a classic approach. You dump all the logs, documentation, chat history, descriptions of all toolboxes, MCP, etc. into the agent's context at once. In general, this works, but the context ends up being filled with a lot of irrelevant information and is constantly overflowing.

Now, Cursor is offering Dynamic context discovery. This involves placing a conditional "table of contents" and links in the context, while the rest is scattered across files, and the agent can add information to itself as needed. For example:

βž– Everyone remembers that when the context overflows, Cursor performs summarization and updates the window, right? Now, in addition to this, Cursor stores chat history as a file. After summarization, the agent receives a link to this file, and if some necessary detail was lost in the summary, he can search the history and supplement himself.

βž– Long responses from tool calls are now also recorded in files, rather than being sent directly into the context. Only a link to the necessary output appears in the context, while the gigantic JSON file sits waiting for the agent to access it and search for what he needs using conditional grep or tail.

βž– The same applies to MCP, Agent Skill, and terminal sessions. Bulky tool descriptions and terminal outputs are stored not in the context, but in files. The context simply says "MCP is available for jira, datadog, figma", and the agent, if he needs something, goes to the detailed description and invokes the tool.

It turns out to be quite nice and practical. On A/B tests, the overall token consumption has decreased by ~46.9%.


And it's also scalable, because here the context transforms from a place where knowledge is stored into an instruction on how to retrieve it. Find details

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
This media is not supported in your browser
VIEW IN TELEGRAM
How to use LLM without losing quality?

DFlash is a way to speed up text generation for large models.

HOW and WHY to use?

Works like this: one model quickly creates a draft, and another one checks it and corrects errors.

- 6.2Γ— faster without losing quality on Qwen3-8B
- 2.5 times faster than EAGLE-3

The idea is simple:

β€’ Diffusion models - generate quickly, but sometimes make mistakes
β€’ Autogenerative (AR) - very accurate, but work slowly
β€’ DFlash combines both approaches:
diffusion - draft β†’ AR - checking and confirmation


Both quickly and accurately, instead of choosing one or the other.

Blog, Code, Models

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
SleepFM model diagnoses 130 diseases by analyzing a single night's sleep.

Stanford trained SleepFM model (fundamental for predicting a range of pathologies) from atrial fibrillation and myocardial infarction to dementia and Parkinson's disease.

πŸ”΄ Why traditional ML models fail despite having gigabytes of data?
Polysomnography is the "gold standard" for studying sleep: a person is fitted with sensors (EEG, ECG, respiration, muscles) and gigabytes of raw signals are recorded.

But in the ML world, this data is used ineffectively. Existing models were trained on small datasets for specific tasks (finding apnea, determining sleep phases).

A huge amount of physiological information about a patient's health was simply ignored, because it's impossible to manually label hundreds of hours of recordings for each disease.

Moreover, if the EEG sensor was mounted slightly differently in one clinic or fell off, the usual model would break down.


🟒 How training on 585k hours possible without human labels?
At the university, they realized that they didn't need human labelers, they needed volumes. They collected a huge dataset of 585,000 hours of sleep recordings from more than 65,000 patients and invented a unique SSL learning algorithm for the future model.

1️⃣ LOO-CL (Leave-One-Out Contrastive Learning)

Instead of teaching the model to predict a diagnosis, they made it solve a puzzle: the system receives input signals from 3 modalities (heart, muscles, respiration) and must predict the embedding of the fourth (brain waves).

This forces the neural network based on 1D CNN and Transformers to learn deep, hidden connections between physiological processes.

2️⃣ The second feature is Channel-Agnostic Attention.

The models don't care about which sensors are connected and in what order. If a channel fails or is absent, attention pooling simply redistributes weights, and inference continues.

3️⃣ SleepFM has learned to read sleep not just for insomnia.

Having received a single night of recordings as input, the model predicts the risk of 130 diseases, and it does this more accurately than specialized models trained with a teacher: the risk of Parkinson's disease is detected in 89% of cases, dementia in 85%, and the probability of a heart attack in 81%.


Such diagnostics could move from labs to smartwatches with development of wearable electronics, and tests shown that noise of sleep signals can hide a patient's entire medical record.

Details #news #AI #ML

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
Convert PDF files into clean data, ready for LLM.

Dolphin is a document parsing framework that converts PDFs into structured formats: Markdown, HTML, LaTeX, and JSON.

Works and Features:

Stage 1️⃣ Detailed analysis of the layout at the page level. Elements and their order are determined according to the natural reading order.

Stage 2️⃣ Parallel parsing of elements using different types of anchors and task-specific prompts.

Key features:

Β» Open-source
Β» A two-stage approach of analyze-then-parse based on a single VLM
Β» Encouraging performance on document parsing tasks
Β» Generation of a sequence of elements in the natural reading order
Β» Heterogeneous anchor prompts for different types of document elements
Β» An efficient parallel parsing mechanism


GitHub

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
On subject of progress: An agent from SakanaAI took a confident first place in a coding competition.

🟒 How did a "wrapper" beat 800 human experts?

Last year, an agent from OpenAI only took second place in the same competition.

This year, about 800 people participated in the AtCoder Heuristic Contest. The ALE-Agent from the Japanese laboratory outperformed everyone and took the top spot with a significant lead. The cost of solution was approximately Β£1,300.

Interestingly, authors of this year's optimization task themselves expected a classic approach using annealing and constructive heuristics, but the Sakana agent took a different path. He suddenly implemented the virtual power heuristic, which allowed him to escape local optima even better than human experts.

Agent is a rather clever wrapper over (in this case) GPT-5.2 high and Gemini 3 Pro high.


Sakana themselves never really shone in terms of models, but they learned to work competently with inference time scaling and here's the result.

In a word, well done! Read Here

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
Binary search + rescoring in int8.

The simple strategy to search through 40 million texts in ~200 ms using only a CPU server, 8GB of RAM, and 45GB of disk space.

If you want to try it out immediately, there's a demo of 40 million texts from Wikipedia. No login or other hassles required.

🟒 The Inference Strategy:

☞ Embed the query with a dense model into a regular fp32 vector

☞ Quantize the fp32 embedding into a binary format, which is 32 times smaller

☞ Retrieve, for example, 40 documents (about 20 times faster than an fp32 index) using an approximate or exact binary index

☞ Load the int8 embeddings for these top-40 documents from the disk

☞ Rescoring: fp32 embedding of the query Γ— 40 int8 embeddings

☞ Sort these 40 documents by the new score and take the top-10

☞ Load the titles and texts of the top-10 documents

The documents are embedded once, and then these embeddings are used in two representations:

1️⃣ A binary index (I used IndexBinaryFlat for exact search and IndexBinaryIVF for approximate search)

2️⃣ An int8 view, i.e., a way to quickly read int8 embeddings from the disk by document ID

➑️ In end, instead of fp32 embeddings, you store:

- a binary index (32 times smaller)
- int8 embeddings (4 times smaller)

Plus, Only the binary index is kept in memory, so the RAM savings are also x32 compared to fp32 search.
For comparison: a regular fp32 retrieval on such a task would require about 180GB of RAM, 180GB of disk for embeddings, and would be 20–25 times slower.

A binary retrieval with int8 rescoring fits into about 6GB of RAM and ~45GB of disk for embeddings.

For example, if you load 4 times more documents through the binary index and then rescoring them in int8, you can recover about 99% of the quality of fp32 search (compared to ~97% for pure binary search)


HFblog
β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
Data eXplore : Data Science, ML, Big Data, LLMs and AI Security
Live stream scheduled for
Voicechat 2 of Saturday series with industry pros.

Q/A with QA Tester

πŸ“… Time: TODAY, Jan 10, 2026 | 4:30 PM UTC (10 PM IST)
🎧 Listen Only: Join Livestream

🎀 Want to Speak or Ask?
Comment for Speaker Link

#DataXplore #AI #ML #VoiceChat

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
Ralph Mode for Deep Agents

What if we give the agent a task and let it run endlessly?

Developed Ralph Mode based on Deep Agents specifically for such an experiment.

Ralph Mode cycles the agent repeatedly, with each pass using a clean context, and the file system is used as memory. You can start it, step away, and then stop it with Ctrl+C when you're done (or set limits in advance).

This video shows how to run Ralph Mode together with Deep Agents and automatically compile an entire Python course.


Video, Repo

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
The most comprehensive review of RL that I've seen.

It was written by Kevin Murphy from Google DeepMind, who has over 128k citations.

How it differs from other materials on RL:

β†’ There's a bridge between classical RL and the current era of LLMs:

A separate chapter on LLMs and RL, which discusses:

RLHF, RLAIF, and reward modeling
PPO, GRPO, DPO, RLOO, REINFORCE++
Training reasoning models
Multi-turn RL for agents

Scaling computations for inference (test-time compute scaling)

β†’ The basics are explained very clearly

All major algorithms like value-based methods, policy gradients, and actor-critic are explained with mathematical rigor.

β†’ Model-based RL and world models are also well-covered

There's Dreamer, MuZero, MCTS, and more on the list - this is exactly where the field is heading now.

β†’ A section on multi-agent RL

Game theory, Nash equilibrium, and MARL for LLM agents.


β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
This media is not supported in your browser
VIEW IN TELEGRAM
Everyone is talking about n8n, but it's worth taking a closer look at Sim.

This is an open-source platform for building AI agents:

βœ“ Next.js + Bun + PostgreSQL + Zustand stack
βœ“ You can connect any AI model
βœ“ You can deploy it on your own server

GitHub

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
Tencent introduced a diffusion language model: 6X faster than classic LLMs

WeDLM-8B Instruct does not use autoregression like regular LLMs,
but a diffusion method for text generation.

What does this provide?
πŸš€ In mathematical reasoning tasks, the model works 3–6 times faster
than Qwen3-8B even with vLLM optimizations - while maintaining quality.

This release breaks the old myth that "diffusion models are not suitable for precise text tasks".

In practice, WeDLM shows that such an approach can compete
and even outperform transformers in inference speed.


The model is open and available under the Apache 2.0 license:

GitHub, HuggingFace

β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’β€’
πŸ€– Data Science, ML & Big Data with @DataXplore
30 terms from the field of agent-based AI that AI development engineers should

πŸ€– @DataXplore
Do you want to learn AI on real projects?
In this repository, there are 29 projects with Generative AI, Machine Learning, and Deep Learning.

With full code for each one. This is pure gold: https://github.com/KalyanM45/AI-Project-Gallery

πŸ€– @DataXplore
This media is not supported in your browser
VIEW IN TELEGRAM
Stokes' theorem is a classic of vector analysis.

Essentially, it states that the linear integral of a vector field over a closed contour is equal to the surface integral of the rotor of this field over the surface bounded by this contour.

πŸ€– @DataXplore