Seems like GPT-4 is back on top with their latest update. Will have to wait and see how it compares in real time tests.

Microsoft just casually shared their new Phi-3 LLMs less than a week after Llama 3 release. Based on the benchmarks in technical report (https://arxiv.org/abs/2404.14219), even the smallest Phi-3 model beats Llama 3 8B despite being less than half the size.

Phi-3 has "only" been trained on 5x fewer tokens than Llama 3 (3.3 trillion instead of 15 trillion)

Phi-3-mini less has "only" 3.8 billion parameters, less than half the size of Llama 3 8B.

Despite being small enough to be deployed on a phone (according to report), it matches the performance of the much larger Mixtral 8x7B and GPT-3.5. (Phi-3 mini can be quantized to 4-bits, so it only requires ≈ 1.8GB of memory.)

What is the secret sauce? According to the technical report, it's dataset quality over quantity: "heavily filtered web data and synthetic data".

Next to the 4k context-window version, there's also a phi-3-mini-128K model that supports up to 128k tokens.

Fun fact: Phi-3 uses the same tokenizer with a vocabulary size of 32,064 as Llama 2.

Ai / Computer Vision Bootcamp🚀

Learn Ai / Computer Vision from Basics to Deployment from IITian and COEPian.

✅ Build Face Recognition ☺️
✅ Build Ai Object detection 🏍️🚇✈️
✅ Building Social Distancing App
✅ Build Automated Invoice reader 📃
✅Image Classification 🍘🥗
✅ Build Application of Computer Vision in Healthcare, Automotive, retail, Manufacturing and Security, Surveillance. 📸

+40 Hrs sessions.
+12 Weeks.
+13 Tools & Technology.
+7 Projects.
+7 Homework Assignments.
+5 Case studies
+5 Skills.
+5 Domains.

📌 Remote and weekend sessions.
📌 Starting from basics.
📌 Get Certificate.

Duration: 3 months

Attend the 1st FREE session on 11th May: https://chat.whatsapp.com/BibIwuuUEWrGEWdZHYluNe

For registrations: https://aiindia.ai/cv-bootcamp/

Early result of Gemma 2 on the leaderboard, matching Llama-3-70B.

- Full data at leaderboard.lmsys.org
- Chat with Gemma 2 at chat.lmsys.org
- Gemma 2 blog goo.gle/3RLQXUa

The matrix calculus for Deep Learning. Very well written. https://explained.ai/matrix-calculus/

Best article on GenAI getting started
https://blog.bytebytego.com/p/where-to-get-started-with-genai

How Much GPU Memory Needed To Server A LLM ?

This is a common question that consistnetly comes up in interview or during the disscusiion with your business stakeholders.

And it’s not just a random question — it’s a key indicator of how well you understand the deployment and scalability of these powerful models in production.

As a data scientist understanding and estimating the require GPU memory is essential.

LLM's (Large Language Models) size vary from 7 billion parameters to trillions of parameters. One size certainly doesn’t fit all.

Let’s dive into the math that will help you estimate the GPU memory needed for deploying these models effectively.

𝐓𝐡𝐞 𝐟𝐨𝐫𝐦𝐮𝐥𝐚 𝐭𝐨 𝐞𝐬𝐭𝐢𝐦𝐚𝐭𝐞 𝐆𝐏𝐔 𝐦𝐞𝐦𝐨𝐫𝐲 𝐢𝐬

General formula, 𝐦 = ((𝐏 * 𝐬𝐢𝐳𝐞 𝐩𝐞𝐫 𝐩𝐚𝐫𝐚𝐦𝐞𝐭𝐞𝐫)/𝐦𝐞𝐦𝐨𝐫𝐲 𝐝𝐞𝐧𝐬𝐢𝐭𝐲) * 𝐨𝐯𝐞𝐫𝐡𝐞𝐚𝐝 𝐟𝐚𝐜𝐭𝐨𝐫

Where:
- 𝐦 is the GPU memory in Gigabytes.
- 𝐩 is the number of parameters in the model.
- 𝐬𝐢𝐳𝐞 𝐩𝐞𝐫 𝐩𝐚𝐫𝐚𝐦𝐞𝐭𝐞𝐫 typically refers to the bytes needed for each model parameter, which is typically 4 bytes for float32 precision.
- 𝐦𝐞𝐦𝐨𝐫𝐲 𝐝𝐞𝐧𝐬𝐢𝐭𝐲 (q) refer to the number of bits typically processed in parallel, such as 32 bits for a typical GPU memory channel.
- 𝐨𝐯𝐞𝐫𝐡𝐞𝐚𝐝 𝐟𝐚𝐜𝐭𝐨𝐫 is often applied (e.g., 1.2) to account for additional memory needed beyond just storing parameters, such as activations, temporary tensors, and any memory fragmentation or padding.

𝐒𝐢𝐦𝐩𝐥𝐢𝐟𝐢𝐞𝐝 𝐅𝐨𝐫𝐦𝐮𝐥𝐚:

M = ((P * 4B)/(32/Q)) * 1.2

With this formula in hand, I hope you'll feel more confident when discussing GPU memory requirements with your business stakeholders.

#LLM

Uber used RAG and AI agents to build its in-house Text-to-SQL, saving 140,000 hours annually in query writing time. 📈
Here’s how they built the system end-to-end:

The system is called QueryGPT and is built on top of multiple agents each handling a part of the pipeline.

1. First, the Intent Agent interprets user intent and figures out the domain workspace which is relevant to answer the question (e.g., Mobility, Billing, etc).

2. The Table Agent then selects suitable tables using an LLM, which users can also review and adjust.

3. Next, the Column Prune Agent filters out any unnecessary columns from large tables using RAG. This helps the schema fit within token limits.

4. Finally, QueryGPT uses Few-Shot Prompting with selected SQL samples and schemas to generate the query.

QueryGPT reduced query authoring time from 10 minutes to 3, saving over 140,000 hours annually!

Link to the full article: https://www.uber.com/en-IN/blog/query-gpt/?uclick_id=6cfc9a34-aa3e-4140-9e8e-34e867b80b2b

"Agents are not enough." New Microsoft research explores that for the latest wave of agents, differentiated by GenAI, to succeed, they need to work together with Sims and Assistants (see diagram on page 3):

Agents are nothing new, evolving from early agents (1950s) to expert systems (1980s), reactive agents (1990s), and more recently multi-agent systems and cognitive architectures.

While frameworks like AutoGen help modern agents tackle complex tasks in narrow domains, challenges like generalization, scalability, and coordination persist.

To help tackle challenges and improve standardization, privacy, personalization, and trust, the research advocates for an ecosystem centered on Agents, Sims, and Assistants.

1\ Agents:

→ Narrow and purpose-driven modules that are trained to do a specific task. Each agent can be autonomous, but with an ability to interface with other agents.

2\ Sims:

→ Representations of the user, built from their profile, preferences, and behaviors, capturing key aspects of who the user is.
→ Sims can act on the user’s behalf, interacting with agents to accomplish tasks, guided by the user’s Assistant.

3\ Assistants:

→ Programs that interact directly with users, deeply understand them, and can call Sims or Agents to handle tasks reactively or proactively.
→ Assistants act as private agents, accessing personal information and fine-tuned to the user, enabling them to perform tasks on the user's behalf.

Interaction

→ Agents, Sims, and Assistants work together with high degree of synergy.
→ The Assistant, deeply understanding the user, co-creates and manages Sims with user input, reflecting different facets of the user’s life.
→ Sims engage specialized Agents to complete tasks effectively, ensuring precision and personalization, which enhances user satisfaction.

P.S. Paper attached with link dives deeper: https://www.arxiv.org/pdf/2412.16241