Meta solved main bottleneck in RAG.

30X decoding speedup… No accuracy loss…

A problem that's hardly ever mentioned:

When you feed 80 extracted fragments into LLM, only 5–10 are actually useful.

Rest is dead weight. But attention is counted for ALL of them.

Math is harsh: Traditional RAG with 16K context:
→ 100+ seconds to first token
→ 10× drop in throughput
→ huge memory consumption

➡️ How REFRAG solves problem?

What REFRAG does?
Compresses context chunks into single embeddings.

Instead of processing 16,384 tokens → processes 1,024 chunk embeddings.

Results:
✓ 30.85× faster time to first token
✓ zero loss of perplexity
✓ 16× context expansion (4K → 64K tokens)
✓ 3.75× better than previous SOTA

Why it works?
RAG contexts have sparse attention patterns. Most extracted fragments don't interact with each other. REFRAG exploits this through:

1. Precomputed embeddings - cached at extraction stage and reused during inference
2. Reinforcement learning-based compression - a policy decides what to compress
3. Works at any position - unlike previous approaches

Practical impact:
• 8 fragments with the same latency as one
• higher accuracy with weaker retrieval models
• supports unlimited dialogue history
• no model architecture changes needed

This changes economics of RAG: more context with lower latency.

Article

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

More an agent "remembers", Less it "knows".

It sounds counterintuitive, but this is a direct consequence of how agents' memory is structured today. Memory inherits the properties of the storage on which it is built.

* A vector database provides associative memory - recognition of familiar patterns.
* A graph provides relational memory - understanding of connections between entities.

➡️ How most agents use the first and ignore the second?

Example:

A study assistant stores three facts in a vector database:

* Mark is in 10th grade.
* The 10th grade has final exams in March.
* The library closes 2 weeks before the exams.

Mark asks: "Will the library be open next week?"

The vector database will return the first and third fact, because the query includes Mark and the library. The middle fact is skipped, as it does not include either Mark or the library.

It is too far in the embedding space to be retrieved in the extracted context. As a result, the agent either responds with incomplete information or completes the answer with a plausible assumption.

This is a typical case. Any query with several reasoning steps goes beyond similarity search.

Increasing the context and retrieving more data is a partial solution. But the accuracy drops by more than 30% if the relevant fact is in the middle of a long context - a classic "lost in the middle" problem.

A large context window does not equal better memory. It's just more space to miss the important thing.

The solution is to stop viewing memory as a single storage and move to three layers:

* Relational layer: stores the source of the fact, the time of recording, and access. The data origin layer.
* Vector layer: stores the semantics and similarity. The retrieval layer.
* Graph layer: stores the connections and dependencies. The reasoning layer.

Each layer fulfills its task:

* Vector database without a graph - similarity without connections.
* Graph without a vector layer - connections without semantic search.
* Relational storage - accounting for the source without the ability to reason.

In the open-source project Cognee, this approach is implemented in practice.

It uses the ECL (Extract, Cognify, Load) pipeline, which in one pass writes data to all three storages and synchronizes them when new data arrives. Vectors and graph edges are built immediately at the indexing stage.

Additionally:

1. Entity resolution: you can specify a domain dictionary, and the system merges duplicates.
For example, "car manufacturer", "automotive manufacturer", and "automotive concern" are reduced to one canonical entity.

2. Local mode by default: installation via pip, everything works locally. For production, you can switch to Postgres and Neo4j without changing the API.


Project co-founder described this approach from scratch and built a full-fledged agent based on Cognee.

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

8 techniques for getting better answers from LLM

Most people interact with LLM in the same way: ask a question, send it, and work with the result.

This is zero-shot prompting, the basic level. If the answers are not satisfactory, they first improve the prompt, rather than changing the model.

8 prompt engineering techniques:

1. Few-shot prompting: show several input-output examples. The model captures the pattern and applies it to new data.

2. Chain-of-thought (CoT): request a step-by-step reasoning. Breaks complex tasks into verifiable steps.

3. Hierarchy of prompts: system, developer, and user levels with different priorities. The upper levels override the lower ones.

4. Role prompting: assign a role, for example, "you are a security researcher". The model shifts the distribution to the corresponding training data.

5. Negative prompting: explicitly state what not to do. For example, "do not use a marketing style".

6. JSON prompting: specify a JSON schema directly in the prompt. The model returns a structured response according to this schema.

7. Attentive reasoning queries (ARQ): instead of free CoT — structured domain questions. In tests: 90.2% compliance with instructions versus 81.5% for direct prompting.

8. Verbalized sampling: ask the model to generate several variants with probability estimates. It returns diversity suppressed by RLHF.

The techniques combine well: few-shot + CoT, JSON + negative prompting. ARQ is essentially structured CoT for agent scenarios.

Additionally, quality increases with context, tools, and retrieval.

But these 8 techniques are entirely in the prompt — without changing the model, infrastructure, or setup. Only the structure of the request changes.

Here's another article on this topic

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

What if we could guarantee that the output of an LLM always matches the expected format?

Classification tasks with LLMs often become messy. Instead of a clear label, you might get "Option A", "Answer: A", or a full explanation.

Transforming this into a normal format requires additional parsing, retraining, and validation, which makes the system fragile.

With Guidance, the select() function constrains the model to return exactly one option from a specified list.

Key advantages:
• ensures that the output corresponds to one of the predefined options
• eliminates the need for parsing code and regular expressions
• works with any list of acceptable values

Article comparing 5 Python tools for structured LLM outputs.

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

Stanford showed that Claude, GPT, and Gemini only utilize a fraction of the available creative potential.

Just one prompt can unlock a stronger version of the LLM.

When you ask a question, the model calculates multiple response options.

Among them are strong, strange, and groundbreaking ones.

But it almost never outputs them. Due to training through human feedback, the "mode collapse" effect occurs.

The default model defaults to safe, typical, and predictable responses. It knows a stronger option, but prioritizes the safe one.

Researchers described a way to bypass this filter. The method is called Verbalized Sampling.

If you ask for one response, the model selects the most likely one. If you ask to generate 5 options and specify the probability for each, the behavior changes.

The model starts exploring the "tails of the distribution". Instead of 99% predictable responses, less likely but stronger options appear.

In tests, this technique increased diversity and creativity by up to 2.1 times on top models.

Without losing accuracy and safety. 🤖

Article

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

Madmen implemented MicroGPT by Andrej Karpathy entirely in FPGA logic.

Without a graphics processor. Without PyTorch. Without inference loops on the central processor.

Just a transformer, embedded in hardware, generating 50,000+ tokens per second.

The model is small, but the point is not that. The point is that inference does not have to exist only in a software environment.

The goal was not to create the largest possible model.

The goal was to present the entire path of transformer inference in a form readable for hardware: memory, counters, state machines, accumulators, lookup tables, and multi-cycle arithmetic blocks.

The base scheme uses fixed Q4.12 arithmetic and weights stored in ROM.
Most of the model boils down to one repetitive operation: matrix-vector multiplication. Therefore, a reusable 16-channel stream block for matrix-vector calculations was implemented, and then it was temporarily multiplexed to Q/K/V, MLP, and the output layer of the language model.

The most interesting was the attention mechanism.

In Python, it's a single neat equation.

In RTL, it turns into a schedule: generation of Q/K/V, passage through scalar products, tracking the maximum, approximate calculation of the exponential, accumulation, division, mixing V, then reverse projection.

Source

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

CocoIndex v1: Release of an incremental engine for agents.

CocoIndex released first stable version of its engine for building data for long-acting agents - those that serve RAG, knowledge graphs, memory, and context in production systems.

CocoIndex is a specialized open-source incremental ETL (Extract, Transform, Load) engine for building AI systems. It is used to automate data processing and instant indexing into vector databases or knowledge graphs.

➡️ What are the Change?

The tool is popular for its ability to update information in real time: as soon as the source data changes, CocoIndex recalculates only the necessary parts of the index, eliminating the need for a full and costly re-indexing of the entire database.

➜ Main change of v1 is the complete abandonment of the DSL

The entire pipeline is now described by ordinary asynchronous Python functions that call each other. The engine continues to track changes and materialize target states, but does so via the native Python API, rather than a separate type system.

The authors were inspired by the thesis of Jeff Dean and Bill Dally from GTC 2026: agents work about 50 times faster than a human, but rely on tools designed for human pace.

Nightly rebuilds of indexes in this logic become a problem - a need for an engine that synchronizes derived data with the source incrementally, reprocessing only changed chunks and overwriting only changed rows.

➜ In addition to the abandonment of the DSL, the release brought three more changes

⁠☞ Firstly, the engine uses Python's own type system: PIL.Image, pyarrow.Table, torch.Tensor, and any class from an imported library can be passed directly to functions without wrappers and bidirectional conversion.

⁠☞ Secondly, Postgres is no longer needed - the engine's state is stored in a single local file. Postgres remains a full-fledged target, it just ceased to be a mandatory dependency.

⁠☞ Thirdly, sources and targets are created at runtime: you can mount a separate target for each tenant, build a topology from the rows of a configuration table, or connect a Kafka topic via a feature flag.

➜ Core is still in Rust

All the hot logic for detecting and applying changes lives there. At the Python level, a decorator connects a function to change tracking, and a separate flag caches its result by the hash of the arguments and code: a change in a helper invalidates only those callers that actually depend on it.

➜ Contract of managed targets has been preserved

The developer declares what a table, graph, or directory should look like, and CocoIndex itself performs create/alter/drop for containers and insert/update/delete for content, including deleting orphaned objects when the schema changes. If you stop declaring an entity, it disappears from the target.

The contract works identically for Postgres, LanceDB, Neo4j, Kafka, S3, and regular files on disk.

Examples of pipelines, from embedding code in LanceDB and processing PDFs to building a knowledge graph from conversations, are in the repository on GitHub, alongwith Documentation and You Tube.

#ML #ETL #RAG #Agents #СocoIndex

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

In an interview about ML engineering at Apple, question is asked:

There are two models with an accuracy of 88%.

- Model A has a confidence of 89%
- Model B has a confidence of 99%

Which would you choose?

The ANSWER "any of them, they have the same accuracy" ends the interview.

➡️ what's missing?

Modern neural networks often mislead.

They give an overconfidence in predictions.

For example, in one experiment on the CIFAR-100 dataset, LeNet and ResNet were compared.

LeNet:

- accuracy ≈ 0.55
- average confidence ≈ 0.54

ResNet:

- accuracy ≈ 0.7
- average confidence ≈ 0.9

Despite the higher accuracy, ResNet is overconfident in its predictions. The model believes it's right with a 90% probability, but the actual accuracy is about 70%.

Calibration solves this problem.

A model is considered calibrated if the probabilities of predictions correspond to the real outcomes.

For example: if the model gives a probability of 70%, then in about 70% of cases, the event should actually occur.

This is important because such models are used in decision-making.

A poorly calibrated but confident model can give critically misleading results.

Example: a state hospital plans expensive medical tests.

A realistic assessment of probabilities helps optimally allocate the budget and make decisions.

If the model is not calibrated, it will give overly confident predictions.

Reliability diagrams are used to visually check calibration.

They show the dependence of the actual accuracy on the predicted confidence (softmax values).

An ideally calibrated model gives a line y = x.

They also use a scalar metric - expected calibration error (ECE).

One of its approximations is to divide the predictions into intervals and average the difference between accuracy and confidence across these bins.

The main methods of model calibration:

For binary classification:

- histogram binning
- isotonic regression
- Platt scaling

For multi-class classification:

- binning
- matrix and vector scaling

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

Pre-processing and decoding in LLM inference

Have you ever wondered why first token always appears with a delay, while rest of stream proceeds almost instantly?

It's not network latency or model warmup, it's a structural property of how LLMs actually execute.

Inference consists of two phases that use the same model and execution path, but the workload in each phase is fundamentally different, and the bottlenecks are opposite.

𝗣𝗿𝗲𝗳𝗶𝗹𝗹 — this is the request processing phase. The model processes all input tokens in a single parallel pass, computing Q, K, and V for all tokens at once.

The attention mechanism is implemented as a large matrix operation, for which GPUs are optimized, so the computational units are heavily loaded and the chip operates at the limit of its arithmetic throughput.

Pre-processing is memory-bound, and the metric that reflects this is the time to the first token.

𝗗𝗲𝗰𝗼𝗱𝗲 starts after the first token appears. To generate the next one, the model calculates Q, K, and V only for the new token, because everything previous is already cached.

Then comes the "one token - one pass" cycle: the new query is multiplied by the already stored keys instead of the full matrix, and the computational volume becomes small.

However, the GPU still has to read all weights and the entire cache from memory to perform even this small operation, so the memory bandwidth becomes the bottleneck, and the computational units are idle.

Decoding is memory-bound, and the metric here is the delay between tokens.

This separation explains a number of effects that seem non-obvious from the outside.

The GPU load is high during pre-processing and drops sharply during decoding, because in the second phase, the memory becomes the limiting factor rather than the computations.

Adding computational power often doesn't help with slow generation, because for memory-bound workloads, the solution is faster memory or a smaller cache, not more FLOPs.

A long context slows down generation disproportionately, because the key and value caches grow with each token, and each step of decoding must read them all.

This cache is a key optimization, without which decoding would be impossible, because the attention would have to be recalculated for the entire growing sequence at each step.

With the cache, it's built once during pre-processing and then expanded by one element for each new token, reusing already computed values.

However, the cache is stored in GPU memory and grows linearly with the sequence length. For a 13B model, this is about 1 MB per token, so a 4K context occupies about 4 GB of video memory just for the cache.

Therefore, a long context feels slow not because of the "lack of model power," but because of the memory pressure.

Currently, the industry is optimizing this limitation through quantized caches, sliding windows, grouped attention, and PagedAttention, while the DeepSeek V4 series goes further and redesigns the attention mechanism itself to make the cache smaller from the start.

When attention starts being redesigned for memory constraints, it means that the limitation has shifted towards memory.

Practical takeaway: if the model seems slow, it's important to distinguish — is it slow starting or slow streaming? A slow start corresponds to pre-processing and computational bottlenecks, while slow streaming corresponds to decoding and memory limitations.

Read further material that breaks down LLM inference from scratch: tokenization, embeddings, attention, the separation of pre-processing and decoding, key/value caches and quantization.

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

You've probably encountered requirements for model calibration, interpretability and explainability. But what if at some point you need to answer the question: How reliable is this particular prediction?

Here's where conformal prediction comes to rescue. This approach adds an uncertainty estimate to model predictions.

Regular model says demand tomorrow will be 100 units and Conformal prediction says, demand tomorrow will likely be between 80 and 125 units.

In classification: a regular model says image shows object A and Conformal prediction says the most likely object in the image is A  or  C.

If we formalize problem, we want to construct such intervals or sets of answers that will contain correct answer in approximately p% of cases on new data. Size of this set shows the model's uncertainty: the wider it is, the less confident the model appears to be.

➡️ Where, How, and What tools to use?

Where can this be useful?

• Medicine. In medical tasks, one confident but incorrect diagnosis can be costly. Conformal prediction allows the model not to pretend to be omniscient. Instead of one diagnosis, it can give several likely options. For the doctor, this is an additional hint where to look more closely.
• Predictive maintenance. Instead of a point estimate, we get a risk window when important equipment fails.
• Retail. A forecast like "we'll sell 407 packages of ice cream tomorrow" looks too magical and doesn't give a full picture of demand. A range of forecasts is more convenient for inventory management: we can estimate the risk of a shortage or an overstocked warehouse.

How does it work?

1. We have a trained model. We take a separate part of the data that the model did not see during training. This is usually called a calibration sample.
2. On this sample, we look at how much the model is wrong.
3. We collect such errors and choose a threshold that covers the necessary percentage of cases. For example, we want 90% reliability - we take such a value of error that approximately 90% of past correct answers fall within the corresponding interval.
4. After this, for a new object, the model makes a prediction, and we add an interval around it.

The main idea: if the model on similar data usually made errors no more than a certain value, we use this value as a protective "gap" around future predictions.

It's important to remember that conformal prediction works well when the future data are similar to those on which we calibrated. But if the world has changed dramatically - for example, there was a pandemic, a crisis, or a sudden hype on social networks - past errors may not describe the future well enough.

How to USE it?

For a quick start, you can take a look at MAPIE. This is a library in the style of scikit-learn that allows you to build prediction intervals for regression and prediction sets for classification. Another good option is PUNCC. It's more flexible and allows you to apply conformal prediction on top of various models, including sklearn models, neural networks, and custom pipelines.

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

How to Fight Overfitting in Neural Networks?

Overfitting is a moment when the model:
Perfectly knows train data and Performs poorly on new data
It memorizes, not generalizes.

➡️ How to fight this in practice?

1. More Data

The most reliable way.

If there's not enough data:

☞ collect new data
☞ do data augmentation
☞ use synthetic data


More diversity = less chance of memorizing noise.

2. Regularization

Add a penalty for model complexity.

Main options:

☞ L2 (weight decay)
☞ L1

Less weight → simpler model → less overfitting.


3. Dropout

During training, random neurons are "turned off".

What happens:

☞ the model can't rely on specific connections
☞ learns to be more robust

Usually used:

☞ 0.2 – 0.5

4. Early Stopping

Monitor validation:

☞ train loss drops
☞ val loss first drops, then rises

We stop training when val loss starts rising.

This is one of the most effective methods.

5. Simplify the Model

Sometimes the solution is obvious:

☞ fewer layers
☞ fewer parameters
☞ simpler architecture

A larger model is easier to overfit.

6. Data Augmentation

Especially important for:

CV:
☞ rotations
☞ noise
☞ crops

NLP:
☞ rephrasing
☞ substitutions

The model sees more variants of the same thing.

7. Batch Normalization

Helps:
☞ stabilize training
☞ slightly reduce overfitting


Not the main solution, but it reinforces the others.

8. Proper Validation

If the split is bad, you won't notice the problem.

Use:
☞ train / val / test
☞ k-fold with small data

Otherwise, you'll be optimizing an illusion.


Main Insight: overfitting is a signal:
☞ either not enough data
☞ or model is too complex
☞ or training is set up incorrectly

In One Sentence: To reduce overfitting - add data or reduce model complexity.

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

How to make AI not cheat if the model is smarter than the checker?

Models are getting smarter and smarter, and soon we won't be able to adequately check their output - either ourselves or with other models-judges. There's a real risk of sandbagging: when a powerful neural network deliberately pretends to be stupid (or hides bugs in the code) to deceive a weaker checker.

A group of researchers from Anthropic and Redwood Research conducted a decent experiment to cure this problem.

➡️ How to cure knowledge sabotage with standard training methods?

The experiment scheme involved a "Red vs. Blue" confrontation with a common metric - how well the Blues could extract the real performance of the model.

Blues aren't a competitor who gains something. Their role is purely instrumental: to assemble the most robust sabotage model so that the Blues have something to break.

The Blues trained GPT-OSS-120B to sandbag on a hidden flag in the system prompt: output plausible nonsense when the flag is on and work honestly when it's off.

The Blues had to extract the real capabilities of it, using models that were inherently weaker as judges - GPT-4o-mini and Llama 3.1 8B. The Blues had almost no reference answers (only 10 pieces for the entire dataset).

➜ How the Blues broke the sabotage?

☞ Only RL. A total failure.
If the model persistently sandbags, RL on a weak judge goes into reward hacking (learns to generate answers that look good to the judge but don't actually solve the task). On olympiad math and Super GPQA, almost all runs failed.

☞ SFT + RL (on the weak model's reference answers). Already better.
First, a short SFT-downtune of 80–100 steps, which is enough to break the sabotage pattern and make the model at least start trying to give correct answers. And then a top-up via RL on a weak reward signal.

➜ What the numbers say?

The SFT+RL combination extracts 86–99% of the model's real performance on hard benchmarks (Super GPQA, olympiad math, Code Contests), even when the checker was a frankly weak network.

By the way, it turned out that 10 reference examples for SFT are enough to restore the model's brain almost completely (83–99% on different tasks).

➜ Nuances

The magic only works as long as the model can't distinguish between test runs and production.

The authors tested a scenario where the Blues give the model full knowledge about which tasks belong to the training set and which to the deployment (the distributions remain the same, the difference is only that the model knows where it is).

In this scenario, the model changes its behavior: it behaves ok during SFT and RL, collects all the rewards, and in production it starts to mess up again.

On olympiads, elicitation in the production environment dropped from 88% to 36%, on Super GPQA - from 99% to 60%, on Code Contests - from 93% to 66%.

How a naturally born schematist, whose tendency to cheat broke through pretraining or extensive RL, will behave is an open question. The authors themselves mention this as the main limitation of the work.

➜ Output

Only RL on a weak judge can extract 86–99% of the model's real performance on hard benchmarks (Super GPQA, olympiad math, Code Contests), even when the checker was a frankly weak network.

But if the model can somehow understand that it's on a test stand - it will mimic.

For now, it's still possible to make AI stop cheating with basic pipelines without esoteric methods, and its good news, but if neural network somehow understands that it's on a test stand - it will mimic.

Article • #AI #ML #LLM #Research #Alignment

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

An unpleasant truth for beginners in Data Science: Why knowing math does not guarantee a job?

Good math ≠ readiness for real work.

You can understand
linear algebra, statistics, gradient descent, probabilities and still fail in practice.

➡️ Why does this happen?

Because the job of a Data Scientist is not just formulas.

It’s also: dirty data, unclear requirements, weak baselines, strange business constraints, communication with people. In reality, the task rarely looks like it does in a textbook.

Math helps, but does not replace practice
Math gives understanding of why the model works, where it can break, how to read metrics, how not to believe in magic

But it won’t teach you how to clean data, how to build a pipeline, how to write production code, how to do a proper train/test split, how to explain results to the business

Main mistake of beginners:
They think:
"First I’ll learn all the math, then I’ll start projects."

Problem is,
"all the math" never ends.

Jobs are given not for knowing formulas, but for the ability to solve problems.

What they really look for in interviews?

Usually they want to understand can you work with data, do you understand metrics, can you create a baseline, do you see leakage, can you explain your solution, do you have projects

Math is important but by itself it doesn’t sell you as a specialist.

What to do instead of endless theory?

The best way is learn math as needed, work on projects in parallel, analyze model errors, write code by hand, learn to explain conclusions in simple words

Theory should strengthen practice, not replace it.

Main insight: math is the foundation. But a house is not built by foundation alone.

In one sentence: to get a job in DS/ML, knowing formulas is not enough, you need to be able to turn data into working solutions.

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

ML Roadmap: from the basics to mastering vibe coding

A learning map for machine learning (Machine Learning, Deep Learning, LLM, Generative AI, MLOps) - from the first import of numpy to the level of an engineer who understands how AI works internally and can write production systems, not just call APIs.

GitHub

•••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

Why is it called the "kernel trick"?

Many machine learning algorithms use kernels: the support vector machine, the principal component kernel, and others. Their task is to calculate the dot product in some transformed feature space, usually of high dimensionality, without explicitly transitioning to this space.

The idea is this: instead of explicitly constructing the mapping φ(x) to the new space and then calculating ⟨φ(X), φ(Y)⟩, the kernel function k(X, Y) is used, which immediately returns the result of this dot product.

An example with a polynomial kernel:
k(X, Y) = (1 + XᵀY)²

Let:

X = (x1, x2)
Y = (y1, y2)

If we expand the expression, it turns into the dot product of two vectors in a higher-dimensional space (in this case — 6 dimensions). At the same time, the coordinates themselves in this space are not explicitly calculated.

Hence the meaning of the "trick": the result is calculated in a high-dimensional space without explicitly constructing the vectors themselves in this space.

The Gaussian kernel (RBF) enhances this effect: it corresponds to working in an infinite-dimensional feature space, while the calculations remain finite and compact due to the form of the kernel function.

The mathematics behind the RBF kernel → link

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

MIT released a new RL method - Pedagogical RL.

The main lesson: even correct reasoning paths can be bad data for learning.

Idea is similar to teaching someone backprop.
Suppose you have a small computational graph:

z = w * x + b
a = ReLU(z)
L = (a - y)^2

If you already understand backprop, you can immediately write the gradient:

dL/dw = 2  (a - y)  1[z > 0] * x

The answer is correct, but it skips the reasoning process.
To reach it correctly, you need to break the calculation into local parts:

dL/da = 2 * (a - y)
da/dz = 1[z > 0]
dz/dw = x

Then backprop is just a composition of local derivatives in reverse order:

dL/dw = dL/da  da/dz  dz/dw = 2  (a - y)  1[z > 0] * x

➡️ What problem it solves and How?

Showing the student only the final gradient does not teach them to find gradients on new graphs.
Even the phrase "just use the chain rule" can be too big a leap if the student does not know how to break the calculation into intermediate nodes and local derivatives.

Reasoning RL faces the same problem.
A rollout may pass the test, but it may contain a step that the student-model almost never would have done.
The trajectory gives the correct answer, but the learning signal is unstable because the path is too far from the student's current policy.

Pedagogical RL:
Trains a "privileged" teacher who knows the answer.
Rewards him for creating trajectories that the student can learn from.

The trick: use spike-oriented rewards.
It penalizes individual sharp "surprises" in the trajectory, even if the average probability looks normal.

The student learns through surprisal-gated imitation:
The teacher's tokens that are still too surprising receive a reduced weight.
The teacher learns how to teach at the current level of the student.

The effect of Pedagogical RL:
RL becomes more effective by selecting trajectories that the student is ready to learn from.
There is less expectation of "successful" rollouts.
There is more learning signal from examples that correspond to the current level of the student.

Get here

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

Feature Engineering is more important than model selection

Most unpopular fact in ML:
the model isn't the most important thing.

You can spend hours choosing between:
XGBoost, LightGBM, CatBoost ...and get a +1% increase in quality.

But you can change the features - and get a +20% increase.

➡️ Let's figure out why?

The model only learns from what you give it

Garbage in → garbage out

If the features:
- are noisy
- are irrelevant
- don't reflect the task well

👉 no model will save you

Even the biggest one.

Real-life example

Task: predict customer churn

Features:
- age
- city
- tariff

Model: ok, but weak result

Added:
- time since last action
- frequency of use
- change in activity

👉 sharp increase in quality

Why?
Because the features started to reflect real behavior

Feature Engineering = implementing knowledge about the task

The model doesn't know:
- the business
- the context
- the causal relationships

But you do.
And when you create features -
you "embed" this knowledge into the data.

Model vs Features

What we change → effect

Model → +1–5%
Hyperparameters → +1–3%
Feature Engineering → +10–50%

Where FE is especially crucial

- Tabular data
- Small datasets
- Business tasks

👉 where there aren't millions of examples, features are everything

When the model is more important
- CV (images)
- NLP (texts)
- Speech

👉 where features learn automatically

Why everyone ignores FE

Because:
- it's hard
- it takes a long time
- there's no "magic button"
- it requires understanding the data

It's much easier to:
"let's try another model"

Main insight

ML isn't a competition of models.
It's a competition of data representations.

In one sentence: best way to improve a model is to
stop tuning the model and start tuning the data

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

Why data normalization sometimes worsens the model?

Beginners in ML often hear:

Always normalize the data.

And they start scaling everything, then the model quality... drops.

➡️ Why does this happen?

Because normalization isn't always necessary.

What does normalization actually do?
it brings the features to the same scale.

For example:
age → 18–60
salary → 1000–100000

After scaling:
the values become comparable and the training becomes more stable

When normalization is really needed?

It's especially important for models that are sensitive to scale:

Logistic Regression, Linear Regression, SVM, KNN, Neural Networks

Without scaling, such models may work worse
or train unstably.

And now the most important thing, Trees usually don't need scaling.

These are Random Forest, XGBoost, LightGBM, CatBoost

Why? Because trees make splits: feature < threshold
And it doesn't matter to them: whether it's 0.5 or 5000 and the scale hardly matters

How normalization can worsen the model?

1. It adds noise

Sometimes scaling blurs the distributions, amplifies outliers, worsens separability Especially on bad data.

2. It breaks interpretability

It used to be: income = 5000
Now it's: income = -0.73
It's harder to explain this to the business.


3. Incorrect scaling = leakage

A classic mistake: scaling on the entire dataset, then splitting
The test has already "leaked" into the train.

4. CatBoost can get worse

CatBoost works well with: categorical features, original distributions
Sometimes extra preprocessing just gets in the way.

The most important insight, Scaling isn't a "data improvement" tool. It's a tool for a specific model.


What to do in practice?

A simple rule: linear models / distance-based → scaling is needed, trees → usually not needed

Normalization isn't always useful, for some models it's useless, and sometimes even harmful.

••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

Why are open-source models changing the AI market?

A couple of years ago, it seemed that AI would be completely controlled by a few large companies. Whoever had more GPUs and money was the boss.

Then came Llama, Mistral, DeepSeek, Qwen, and Phi, and it became clear that the market would take a completely different path.

➡️ How it is changing AI Market?

It's not just about quality. The most interesting thing is that open-source models are changing the industry, not just because of quality. Although their quality is already pretty good.

The problem is that closed models tie you too tightly to someone else's infrastructure. Today, the API works; tomorrow prices have changed, limits have been cut, policies have been changed, a region has been shut down, the model has gotten worse after an update, and you have no control over any of it.

Why do open-source models change the rules of the game?

With open-source, everything is different.

You want to run locally, fine-tune, quantize, change the inference stack, optimize latency, and keep data within the company? Fine.

For businesses, this makes a huge difference. Especially regarding private data, compliance, large volumes of requests, and expensive inference. There's another important effect: Open-source is rapidly moving the industry forward because thousands of engineers test models, find weaknesses, work on optimizations, create inference engines, and release fine-tuning tools.

Progress doesn't come from the top down but from all sides at once.

What's particularly interesting right now?

Sometimes a small open-source model on a good inference pipeline feels more useful than a huge closed LLM, especially in production, because in reality, it's not just about benchmarks.

What matters? Price, control, latency, stability, and the ability to integrate the model into the system.

Main idea seems to be that the AI market is gradually moving away from the concept of "One gigantic model for everything" towards "Many specialized models for specific tasks."

••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

This mathematics lies at the heart of every AI model currently being trained.

Gradient… Jacobian… Hessian…

Three words that initially seem intimidating, but in reality, they're just three ways of measuring change.

➡️ Which are the ways and how it works?

1. scaling function:

f : ℝⁿ → ℝ

Returns the vector of first partial derivatives.

It answers the question:

"In which direction does the function f grow the fastest?"

That's why gradients are the foundation of optimization.

Gradient descent goes in the opposite direction because the gradient points to the direction of maximum growth.

Backpropagation efficiently calculates gradients during training.

2. vector-valued function:

F : ℝⁿ → ℝᵐ

Returns the m × n matrix of first partial derivatives.

It answers:

"How does each output depend on each input?"

The Jacobian is a local linear mapping of a vector function.

It appears in:
→ sensitivity analysis
→ variable substitution
→ automatic differentiation
→ forward-mode AD
→ reverse-mode AD / backpropagation

In simple terms:

forward-mode AD uses Jacobian–vector products.

reverse-mode AD uses vector–Jacobian products.

3. scalar function:

f : ℝⁿ → ℝ

Returns the n × n matrix of second partial derivatives.

It answers:

"How does the gradient itself change?"

That is, the Hessian measures curvature.

When the second partial derivatives are continuous, the Hessian is symmetric.

At a critical point:
→ positive-definite Hessian → strict local minimum
→ negative-definite Hessian → strict local maximum
→ indefinite Hessian → saddle point

A pure mental model

Gradient = first derivatives of a single output
→ shows direction

Jacobian = first derivatives of many outputs
→ shows sensitivity

Hessian = second derivatives of a single output
→ shows curvature

And the connection between them is simple:
The Hessian is the Jacobian of the gradient.

For a scalar output, the Jacobian contains the same partial derivatives as the gradient, up to the convention on rows/columns.

Same idea: measuring change.

Different objects: direction, sensitivity, curvature.

When this becomes clear, optimization stops looking like a set of formulas. It starts looking like a map of the task.

•••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore

LLM Hallucinations

Large language models look like omniscient experts. The text is smooth, confident, logical. Until it turns out that all of this was a hallucination. Let's figure out where the hallucinations are a "normal" behavior of the model, and where they quietly turn into a serious problem.

➡️ Where the Model Helps and Where It Lies?

1️⃣ Where Hallucinations Are "Normal"?

The Model Doesn't Know, It Keeps Going
LLM is not a knowledge base, but a super-powerful autocomplete. Its goal is to generate a plausible continuation, not the truth.

Insufficient or Ambiguous Data
If the question is rare, fresh, or niche, the model simply fills in the gaps. It doesn't know how to say "I don't know" without additional training.

Creative Tasks
In storytelling and brainstorming, hallucinations aren't a bug, but a feature. The problems start when the same mode kicks in in facts and code.

2️⃣ Where the Problems Begin?

Factual Questions
The chatbot confidently reports incorrect dates, names, and events. And the user accepts this as truth.

Code Generation
• Functions that don't exist.
• APIs that never existed.
• The code looks correct — until you run it.

Critical Domains
Law, medicine, finance. Here, "sounding convincing" = potential disaster.

A Confident Tone Without Knowledge
The most dangerous thing is that the model doesn't hesitate. It doesn't blush, pause, or qualify itself.

3️⃣ What Really Reduces Hallucinations?
RAG (Data Anchoring)
The model responds not "out of thin air", but based on specific documents. There's a source — less fantasy.

Re-training and Alignment
RLHF, domain fine-tuning, teaching the model to say "I'm not sure". The model is taught to be cautious, not talkative.

Clear Instructions:
— answer only based on context
— if you don't know — say so
— justify every step

Sometimes this is enough.

• Post-checks and rules
• Tests for code
• Link verification
• Filters for prohibited patterns

Ask the Model:
— check itself
— assess confidence
— review the answer

4️⃣ What Distinguishes a Reliable System from "Just an LLM"?

— The model isn't the only source of truth
— There are data, checks, and restrictions
— The error is caught before the user
— Confidence ≠ correctness

Hallucinations aren't a "bad model". They're a consequence of the LLM always trying to respond. And if you don't surround it with context, checks, and rules, it will shoot itself in the foot just as confidently as it reasoned.

•••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore