Article 34: Sequence Modeling – RNN, LSTM, and GRU 🧠📚
Standard Neural Networks assume that all inputs and outputs are independent of each other. But in language, the word "bank" means something different depending on the previous words ("river bank" vs. "money bank"). Sequence models have a memory to remember past information.
1. Recurrent Neural Networks (RNNs) 🔄
An RNN processes a sequence one step at a time. It keeps an internal state (a hidden memory) that gets passed from one step to the next.
The Math (Step-by-Step) 🧮
At time step t,
More - Click here...
2. The Big Problem with Standard RNNs ⚠️
Standard RNNs work well for short sequences, but they fail on long sequences due to the Vanishing Gradient Problem.
3. Long Short-Term Memory (LSTM) 🚀🧠
To fix the vanishing gradient problem, the LSTM architecture was invented. An LSTM adds a Cell State that acts like a highway for information to flow unchanged, controlled by three mathematical Gates,
More - Click here...
4. Gated Recurrent Unit (GRU) ⚡️
A GRU is a streamlined version of the LSTM. It merges the cell state and hidden state and reduces the three gates down to Two Gates,
GRUs have fewer parameters than LSTMs. It making them faster to train while offering nearly identical performance.
Summary 📝
RNNs process sequence data step-by-step using a hidden memory. Standard RNNs suffer from Vanishing Gradients, so we use LSTMs (3 gates) or GRUs (2 gates) to retain long-term dependencies effectively.
In the next article (Article 35), we start Phase 10: Advanced AI & Transformers, diving into Attention Mechanisms and Transformers! ⚡️🔥 🙊😁
✍️ @TheInfinityAI
Standard Neural Networks assume that all inputs and outputs are independent of each other. But in language, the word "bank" means something different depending on the previous words ("river bank" vs. "money bank"). Sequence models have a memory to remember past information.
1. Recurrent Neural Networks (RNNs) 🔄
An RNN processes a sequence one step at a time. It keeps an internal state (a hidden memory) that gets passed from one step to the next.
The Math (Step-by-Step) 🧮
At time step t,
The network takes the current input and the previous memory
It combines them to make the new memory
It uses new memory to produce an output
More - Click here...
2. The Big Problem with Standard RNNs ⚠️
Standard RNNs work well for short sequences, but they fail on long sequences due to the Vanishing Gradient Problem.
📉 Vanishing Gradient
When backpropagating through many time steps (Backpropagation Through Time), the gradients get multiplied repeatedly by small numbers.
😵 The Result
The gradient becomes nearly zero. The model forgets what happened at the beginning of the text or series.
3. Long Short-Term Memory (LSTM) 🚀🧠
To fix the vanishing gradient problem, the LSTM architecture was invented. An LSTM adds a Cell State that acts like a highway for information to flow unchanged, controlled by three mathematical Gates,
Forget Gate - Decides what information from the past to throw away.
Input Gate - Decides what new information to store in the cell state.
Output Gate - Decides what the next hidden state should be based on the updated cell state.
More - Click here...
4. Gated Recurrent Unit (GRU) ⚡️
A GRU is a streamlined version of the LSTM. It merges the cell state and hidden state and reduces the three gates down to Two Gates,
Reset Gate - Decides how much past information to forget.
Update Gate - Combines the jobs of the input gate and forget gate.
GRUs have fewer parameters than LSTMs. It making them faster to train while offering nearly identical performance.
Summary 📝
RNNs process sequence data step-by-step using a hidden memory. Standard RNNs suffer from Vanishing Gradients, so we use LSTMs (3 gates) or GRUs (2 gates) to retain long-term dependencies effectively.
In the next article (Article 35), we start Phase 10: Advanced AI & Transformers, diving into Attention Mechanisms and Transformers! ⚡️🔥 🙊😁
✍️ @TheInfinityAI
Telegram
Infinity CS
Article 35: Attention Mechanisms and Transformers ⚡️🤖🧠
In 2017, Google published the famous paper "Attention Is All You Need." It introduced the Transformer which replaced RNNs and LSTMs entirely. Transformers can process an entire sentence at the exact same time (parallel processing), making them much faster and more accurate.
1. What is Attention? 👀🔍
When you read a sentence like: "The animal didn't cross the street because it was too tired," what does "it" refer to? Humans know "it" refers to the animal.
An Attention Mechanism allows the model to look at all words in a sentence simultaneously and assign weights (importance) to the connections between them. It directly connects the word "it" to "animal."
2. The Math Behind Self-Attention 🧮🧠
To compute self-attention, the model converts every word vector into three vectors using learned weight matrices
Learn more about "The Scaled Dot-Product Attention Formula"
3. Multi-Head Attention 🧠🧠🧠
Instead of calculating attention just once, Transformers use Multi-Head Attention.
4. Key Components of Transformer Architecture 🏗🤖
A Transformer model consists of an Encoder and a Decoder
Summary 📝✨
Attention Mechanisms allow models to learn which words are important to one another and capture relationships across an entire sequence. 🔗🧠
Transformers use,
Together, these ideas form the foundation of many modern Generative AI and Large Language Models (LLMs).
In the final article (Article 36), we explore Large Language Models (LLMs) and Generative AI! 🤖✨🚀
✍️ @TheInfinityAI
In 2017, Google published the famous paper "Attention Is All You Need." It introduced the Transformer which replaced RNNs and LSTMs entirely. Transformers can process an entire sentence at the exact same time (parallel processing), making them much faster and more accurate.
1. What is Attention? 👀🔍
When you read a sentence like: "The animal didn't cross the street because it was too tired," what does "it" refer to? Humans know "it" refers to the animal.
An Attention Mechanism allows the model to look at all words in a sentence simultaneously and assign weights (importance) to the connections between them. It directly connects the word "it" to "animal."
2. The Math Behind Self-Attention 🧮🧠
To compute self-attention, the model converts every word vector into three vectors using learned weight matrices
Query (Q) - What the word is looking for.
Key (K) - What the word represents.
Value (V) - The actual information content of the word.
Learn more about "The Scaled Dot-Product Attention Formula"
3. Multi-Head Attention 🧠🧠🧠
Instead of calculating attention just once, Transformers use Multi-Head Attention.
The model runs the attention formula multiple times (e.g., 8 or 16 heads) in parallel.
Each head focuses on a different type of relationship (e.g., one head tracks grammar, another tracks pronoun references, another tracks action object links).
4. Key Components of Transformer Architecture 🏗🤖
A Transformer model consists of an Encoder and a Decoder
Positional Encoding - Because Transformers process all words at once, they do not inherently know word order. Positional encodings add mathematical sine and cosine values to input vectors to inject word positions.
Encoder - Takes input text, processes it through Multi-Head Attention and Feed-Forward Neural Networks, and outputs rich contextual representations. (Used in models like BERT).
Decoder - Uses masked self-attention to generate text word-by-word by predicting the next token. (Used in models like GPT).
Summary 📝✨
Attention Mechanisms allow models to learn which words are important to one another and capture relationships across an entire sequence. 🔗🧠
Transformers use,
🔎 Query, Key, and Value (Q, K, V)
🧠 Self-Attention
👥 Multi-Head Attention
📍 Positional Encoding
⚡️ Parallel Processing
Together, these ideas form the foundation of many modern Generative AI and Large Language Models (LLMs).
In the final article (Article 36), we explore Large Language Models (LLMs) and Generative AI! 🤖✨🚀
✍️ @TheInfinityAI
Telegram
Infinity CS
1
Article 36: Large Language Models (LLMs) and Generative AI 🤖🚀
Generative AI refers to models that create new content like text, images or audio rather than just classifying existing data. LLMs are Decoder only Transformers trained on massive textual datasets to predict and generate text.
1. How LLMs Work (Next-Token Prediction) 🔮
At its core, an LLM performs a probability calculation, given a sequence of words (tokens), it predicts the most likely next token.
more...
2. The Training Pipeline of an LLM 🏗
Building a production-grade LLM involves three major phases
3. Key Paradigms in Modern Generative AI ✨
Summary 📝📚
LLMs utilize Decoder-based Transformer architectures to predict text sequentially. They undergo Pre-training, Supervised Fine-Tuning (SFT), and Alignment (RLHF/DPO) to become instruction-following assistants. Frameworks like RAG and architectural techniques like Mixture of Experts (MoE) extend these models into production systems. 💬🤝🚀🌍
Congratulations, we have completed the entire A to Z Machine Learning and AI Roadmap from fundamental statistics to modern Generative AI models now. 🗺🤖
✍️ @TheInfinityAI
Generative AI refers to models that create new content like text, images or audio rather than just classifying existing data. LLMs are Decoder only Transformers trained on massive textual datasets to predict and generate text.
1. How LLMs Work (Next-Token Prediction) 🔮
At its core, an LLM performs a probability calculation, given a sequence of words (tokens), it predicts the most likely next token.
more...
2. The Training Pipeline of an LLM 🏗
Building a production-grade LLM involves three major phases
I. Pre-training (Unsupervised / Self-Supervised)
II. Supervised Fine-Tuning (SFT)
III. Alignment (RLHF & DPO)
3. Key Paradigms in Modern Generative AI ✨
Retrieval-Augmented Generation (RAG)
Diffusion Models (Image Generation)
Mixture of Experts (MoE)
Summary 📝📚
LLMs utilize Decoder-based Transformer architectures to predict text sequentially. They undergo Pre-training, Supervised Fine-Tuning (SFT), and Alignment (RLHF/DPO) to become instruction-following assistants. Frameworks like RAG and architectural techniques like Mixture of Experts (MoE) extend these models into production systems. 💬🤝🚀🌍
Congratulations, we have completed the entire A to Z Machine Learning and AI Roadmap from fundamental statistics to modern Generative AI models now. 🗺🤖
✍️ @TheInfinityAI
Telegram
Infinity CS
👏2❤1🎉1
⚠️ Telegram SMS Fee Update
Telegram has introduced an SMS verification fee in some cases to reduce fake/scam accounts and cover high SMS verification costs. It may apply to both new registrations and existing-account logins, depending on the number/provider.
Thanks me later 😪
Telegram has introduced an SMS verification fee in some cases to reduce fake/scam accounts and cover high SMS verification costs. It may apply to both new registrations and existing-account logins, depending on the number/provider.
📌 Current update: Existing accounts may now be able to log in without the SMS fee, while the fee is still being requested for new account registrations.
Thanks me later 😪
Linux gives you control. Antigravity CLI takes that power to the next level. 🔥✨
AWS Certified Cloud Practitioner.pdf
608.9 KB
📚 AWS Certified Cloud Practitioner (CLF-C02)
A useful reference to understand what to study and what to focus on for the CLF-C02 exam
Refer this also: Limited Discount
Use the promo code to get 50% off AWS Certified AI Practitioner. Pass the AWS Certified AI Practitioner exam by September 30, and you’ll qualify for a free AWS Certified Cloud Practitioner exam any time before November 30, 2026. This is a limited time offer!
@TheInfinityAI
Sharing the official AWS exam guide for anyone preparing for the AWS Certified Cloud Practitioner certification. It covers the exam structure, key domains, important AWS services, security, cloud concepts, pricing, billing, and support.
A useful reference to understand what to study and what to focus on for the CLF-C02 exam
Refer this also: Limited Discount
Use the promo code to get 50% off AWS Certified AI Practitioner. Pass the AWS Certified AI Practitioner exam by September 30, and you’ll qualify for a free AWS Certified Cloud Practitioner exam any time before November 30, 2026. This is a limited time offer!
@TheInfinityAI
AI Engineering Buildcamp: From RAG to Agents – Course & Scholarship
The AI Engineering Buildcamp: From RAG to Agents is an 8-week, cohort-based online program taught by Alexey Grigorev. It is built for developers, data scientists, and machine learning engineers who want to move past basic tutorials and learn how to build production-grade Artificial Intelligence (AI) systems.
This is a very competitive program. To stand out, tell them how you will use these AI skills to help your community or society. If you can pay even a small part of the fee yourself, you have a much better chance of getting selected.
@TheInfinityAI
The AI Engineering Buildcamp: From RAG to Agents is an 8-week, cohort-based online program taught by Alexey Grigorev. It is built for developers, data scientists, and machine learning engineers who want to move past basic tutorials and learn how to build production-grade Artificial Intelligence (AI) systems.
Scholarship Details 🗓
Normal Cost: $1,799 USD.
Offer: Full and partial scholarships.
Application: Link
Updates & Blog: Link
Deadline: September 13, 2026.
This is a very competitive program. To stand out, tell them how you will use these AI skills to help your community or society. If you can pay even a small part of the fee yourself, you have a much better chance of getting selected.
@TheInfinityAI
Google Docs
Scholarship Application: AI Engineering Buildcamp - From RAG to Agents (Cohort 4)
Apply for a scholarship to join AI Engineering Buildcamp: From RAG to Agents, a 8-week, hands-on learning experience led by Alexey Grigorev. This course takes you from the fundamentals of LLM and RAG to building production-ready AI agents with monitoring…
Infinity CS pinned «Hands-On AI_ RAG using LlamaIndex Part 01 to Part 04 Part 01: https://t.me/AIMLDeepThaught/959 Part 02: https://t.me/AIMLDeepThaught/967 Part 03: https://t.me/AIMLDeepThaught/975 Part 04: https://t.me/AIMLDeepThaught/979 1. Introduction 01. Overcome the…»
Hey everyone! If you are a university student interested in DevOps or AI, here is a great chance for you! 🎓
Normally, AWS Certification exam costs around $100. But now, you can get a 100% FREE AWS Certification exam Voucher through the AWS Student Rewards program.
🔗 AWS Student Rewards - https://lnkd.in/ee7fUSvV
Here is how you can get it 👇
1️⃣ Sign up or Sign in with your free AWS Builder ID
2️⃣ Verify your student status using your university email or Student ID
3️⃣ Get 1 year of AWS Skill Builder Premium for free.
4️⃣ Collect badges 🏆
5️⃣ Unlock rewards 🎁
👉 Collect 7 badges = $10 AWS credits
👉 Collect 14 badges = $20 AWS credits
👉 Collect 21 badges = $100 AWS Certification Exam Voucher
You can use this free voucher to do one of these exams 🎯
● AWS Certified Cloud Practitioner
● AWS Certified AI Practitioner
This is a valuable opportunity for students to learn new skills and get a globally recognized certificate without spending money. Trying this out 🚀
Normally, AWS Certification exam costs around $100. But now, you can get a 100% FREE AWS Certification exam Voucher through the AWS Student Rewards program.
🔗 AWS Student Rewards - https://lnkd.in/ee7fUSvV
Here is how you can get it 👇
1️⃣ Sign up or Sign in with your free AWS Builder ID
2️⃣ Verify your student status using your university email or Student ID
3️⃣ Get 1 year of AWS Skill Builder Premium for free.
4️⃣ Collect badges 🏆
5️⃣ Unlock rewards 🎁
👉 Collect 7 badges = $10 AWS credits
👉 Collect 14 badges = $20 AWS credits
👉 Collect 21 badges = $100 AWS Certification Exam Voucher
You can use this free voucher to do one of these exams 🎯
● AWS Certified Cloud Practitioner
● AWS Certified AI Practitioner
This is a valuable opportunity for students to learn new skills and get a globally recognized certificate without spending money. Trying this out 🚀
ඔයාගේ සාමාන්ය Photo එකක් 1985 Retro Photo එකක් වෙන්නේ කොහොමද?
මේ දවස්වල Social Media එකේ ගොඩක් දෙනෙක් තමන්ගේ සාමාන්ය photo එකක් upload කරලා ඒක 1980 ගණන්වල කැමරාවකින් ගත්ත photo එකක් වගේ හදාගන්නවා ඔයා දැකලා ඇති. Denim jacket එකක්, curly hairstyle එකක්, ලොකු කණ්නාඩි, අර පරණ ෆොටෝ වල තියෙන බොඳ වුණු පාට... මේවා දැක්කම ඇත්තටම 1985 දී ගත්ත photo එකක් වගේමයි. 😎
හැබැයි මෙතන තියෙන රසවත්ම ප්රශ්නය තමයි, “AI එක මේ photo එක ඇතුළෙන් ඇත්තටම ඉගෙනගන්නේ මොනවාද? ඒක ඇතුළේ වැඩ කරන්නේ කොහොමද?” කියන එක. මේක නිකන්ම ෆිල්ටර් එකක් නෙවෙයි. මේක තේරුම් ගන්න නම් අපි AI එකේ pixels වල ඉඳන් අලුත් image එකක් හැදෙන තැනට යන ගමන බලන්න ඕනේ. 🧠
🧠
මුලින්ම AI එකට Photo එකක් කියන්නේ Photo එකක් නෙවෙයි 🖼
අපිට මේක ලස්සන ෆොටෝ එකක් වුණාට, AI එකට මේක පේන්නේ ඉලක්කම් ගොඩක් විදියට විතරයි. ෆොටෝ එකේ තියෙන හැම පුංචි තිතක්ම (Pixel එකක්ම) රතු, කොළ සහ නිල් කියන වර්ණ තුනේ එකතුවකින් තමයි හැදෙන්නේ.
සාමාන්ය ෆොටෝ එකක් ගත්තොත්, ඒක ඇතුළේ මේ වගේ ඉලක්කම් මිලියන ගණනක් තියෙනවා. AI එකේ ගමන පටන් ගන්නේ අන්න ඒ දැවැන්ත ඉලක්කම් ගොඩෙන්. මුලින්ම AI එකට "මෙතන මනුස්සයෙක් ඉන්නවා" කියලා පේන්නේ නෑ.
එහෙනම් “මේක මනුස්සයෙක්” කියලා AI එක දන්නේ කොහොමද?
මෙතනදි තමයි Neural Network එළියට එන්නේ. මේ ඉලක්කම් ගොඩ එකපාරටම තේරුම් ගන්නවා වෙනුවට, AI එක ඒක පියවරෙන් පියවර (Layers හරහා) වෙනස් කරනවා.
මුල් පියවර වලදී edges අඳුරගන්නවා. ඊළඟට හැඩතල අඳුරගන්නවා. අන්තිම වෙනකොට, "මෙතන මුහුණක් තියෙනවා", "මෙතන කොණ්ඩයක් තියෙනවා" කියන එක AI එක තේරුම් ගන්නවා. මේ සංකීර්ණ තොරතුරු ඔක්කොම එකතු කරලා AI එක විශේෂ ඉලක්කම් ලැයිස්තුවක් හදනවා. තාක්ෂණිකව මේකට කියන්නේ Embedding එකක් කියලා. මේ Embedding එක ඇතුළේ ඔයාගේ මුහුණේ හැඩය, ඇස් වල පිහිටීම වගේ දේවල් ඉලක්කම් විදියට ගබඩා වෙලා තියෙනවා.
Transformer සහ "Attention" කියන මැජික් එක
ChatGPT වගේ දේවල් මේ තරම් දක්ෂ වෙන්න ප්රධානම හේතුවක් තමයි Transformer කියන තාක්ෂණය. මේකේ තියෙන ප්රධානම අදහස තමයි Attention (අවධානය යොමු කිරීම). සමීකරණ නැතුව මේක සරලව තේරුම් ගනිමු;
- ෆොටෝ එකක තියෙන කණ්නාඩිය, මුහුණට ගොඩක් සම්බන්ධයි.
- කොණ්ඩය, ඔළුවට සම්බන්ධයි.
- ජැකට් එක, ඇඟට සම්බන්ධයි.
Transformer එක කරන්නේ මේ එකිනෙකට තියෙන සම්බන්ධතාවය කොච්චර ප්රබලද කියලා ගණනය කරන එකයි. ඒ නිසා AI එකට පුළුවන් ෆොටෝ එකේ තියෙන දේවල් වල සන්දර්භය (context) හරියටම තේරුම් ගන්න.
පරණ ක්රමය සහ අලුත් ක්රමය (Diffusion vs. GPT-4o)
මේ ෆොටෝ ජෙනරේට් කරන විදිය ගැන කතා කරද්දී ගොඩක් අය හිතන්නේ එකම ක්රමයයි තියෙන්නේ කියලා.
Traditional Diffusion (DALL·E වගේ) - මේ ක්රමයේදී කරන්නේ ෆොටෝ එකකට වැලි දැම්මා වගේ සම්පූර්ණයෙන්ම බොඳ කරලා (Noise එකතු කරලා), ඊට පස්සේ ඒ වැලි ටික අයින් කරන ගමන් අලුත් ෆොටෝ එකක් මතු කරගන්න එක.
අලුත් ක්රමය (GPT-4o Architecture) - OpenAI ආයතනය පැහැදිලි කරන විදියට, එයාලගේ අලුත්ම මාදිලි වැඩ කරන්නේ ටිකක් වෙනස් විදියට. ඒකෙදි Text සහ Images දෙකම එකම මොළයකින් (end-to-end trained multimodal network) තමයි process කරන්නේ. ඒ කියන්නේ වචන කෑලි වලට කඩනවා වගේම, ෆොටෝ එකත් කුඩා "Tokens" වලට කඩලා තමයි AI එක තේරුම් ගන්නේ. මේක ගොඩක් වේගවත් වගේම ස්වභාවිකයි.
අවසානයේ AI එක “1985” කියලා දන්නේ කොහොමද?
AI එකට 1985 කියන්නේ නිකන්ම අවුරුද්දක් නෙවෙයි. එයාව ට්රේන් කරද්දී, 1985 කියන වචනයත් එක්ක තියෙන ෆොටෝස් වල අර බොඳ වුණු පාට (film grain), ලොකු කොණ්ඩ මෝස්තර, සහ පරණ විලාසිතා තියෙනවා කියලා එයා ඉගෙනගෙන තියෙනවා.
ඒ නිසා ඔයාගේ ෆොටෝ එකේ තියෙන මුහුණේ හැඩය (Embedding එක) අරගෙන, ඒකට 1985 විලාසිතාවන් එකතු කරලා අලුත්ම ෆොටෝ එකක් නිර්මාණය කරනවා.
මේක Photoshop වලින් කරනවා වගේ කෑලි කපලා අලවනවා නෙවෙයි. මේක බිලියන ගණනක් ගණනය කිරීම් හරහා නිර්මාණය වෙන අලුත්ම යථාර්ථයක්. 🤯
මේ දවස්වල Social Media එකේ ගොඩක් දෙනෙක් තමන්ගේ සාමාන්ය photo එකක් upload කරලා ඒක 1980 ගණන්වල කැමරාවකින් ගත්ත photo එකක් වගේ හදාගන්නවා ඔයා දැකලා ඇති. Denim jacket එකක්, curly hairstyle එකක්, ලොකු කණ්නාඩි, අර පරණ ෆොටෝ වල තියෙන බොඳ වුණු පාට... මේවා දැක්කම ඇත්තටම 1985 දී ගත්ත photo එකක් වගේමයි. 😎
හැබැයි මෙතන තියෙන රසවත්ම ප්රශ්නය තමයි, “AI එක මේ photo එක ඇතුළෙන් ඇත්තටම ඉගෙනගන්නේ මොනවාද? ඒක ඇතුළේ වැඩ කරන්නේ කොහොමද?” කියන එක. මේක නිකන්ම ෆිල්ටර් එකක් නෙවෙයි. මේක තේරුම් ගන්න නම් අපි AI එකේ pixels වල ඉඳන් අලුත් image එකක් හැදෙන තැනට යන ගමන බලන්න ඕනේ. 🧠
🧠
මුලින්ම AI එකට Photo එකක් කියන්නේ Photo එකක් නෙවෙයි 🖼
අපිට මේක ලස්සන ෆොටෝ එකක් වුණාට, AI එකට මේක පේන්නේ ඉලක්කම් ගොඩක් විදියට විතරයි. ෆොටෝ එකේ තියෙන හැම පුංචි තිතක්ම (Pixel එකක්ම) රතු, කොළ සහ නිල් කියන වර්ණ තුනේ එකතුවකින් තමයි හැදෙන්නේ.
සාමාන්ය ෆොටෝ එකක් ගත්තොත්, ඒක ඇතුළේ මේ වගේ ඉලක්කම් මිලියන ගණනක් තියෙනවා. AI එකේ ගමන පටන් ගන්නේ අන්න ඒ දැවැන්ත ඉලක්කම් ගොඩෙන්. මුලින්ම AI එකට "මෙතන මනුස්සයෙක් ඉන්නවා" කියලා පේන්නේ නෑ.
එහෙනම් “මේක මනුස්සයෙක්” කියලා AI එක දන්නේ කොහොමද?
මෙතනදි තමයි Neural Network එළියට එන්නේ. මේ ඉලක්කම් ගොඩ එකපාරටම තේරුම් ගන්නවා වෙනුවට, AI එක ඒක පියවරෙන් පියවර (Layers හරහා) වෙනස් කරනවා.
මුල් පියවර වලදී edges අඳුරගන්නවා. ඊළඟට හැඩතල අඳුරගන්නවා. අන්තිම වෙනකොට, "මෙතන මුහුණක් තියෙනවා", "මෙතන කොණ්ඩයක් තියෙනවා" කියන එක AI එක තේරුම් ගන්නවා. මේ සංකීර්ණ තොරතුරු ඔක්කොම එකතු කරලා AI එක විශේෂ ඉලක්කම් ලැයිස්තුවක් හදනවා. තාක්ෂණිකව මේකට කියන්නේ Embedding එකක් කියලා. මේ Embedding එක ඇතුළේ ඔයාගේ මුහුණේ හැඩය, ඇස් වල පිහිටීම වගේ දේවල් ඉලක්කම් විදියට ගබඩා වෙලා තියෙනවා.
Transformer සහ "Attention" කියන මැජික් එක
ChatGPT වගේ දේවල් මේ තරම් දක්ෂ වෙන්න ප්රධානම හේතුවක් තමයි Transformer කියන තාක්ෂණය. මේකේ තියෙන ප්රධානම අදහස තමයි Attention (අවධානය යොමු කිරීම). සමීකරණ නැතුව මේක සරලව තේරුම් ගනිමු;
- ෆොටෝ එකක තියෙන කණ්නාඩිය, මුහුණට ගොඩක් සම්බන්ධයි.
- කොණ්ඩය, ඔළුවට සම්බන්ධයි.
- ජැකට් එක, ඇඟට සම්බන්ධයි.
Transformer එක කරන්නේ මේ එකිනෙකට තියෙන සම්බන්ධතාවය කොච්චර ප්රබලද කියලා ගණනය කරන එකයි. ඒ නිසා AI එකට පුළුවන් ෆොටෝ එකේ තියෙන දේවල් වල සන්දර්භය (context) හරියටම තේරුම් ගන්න.
පරණ ක්රමය සහ අලුත් ක්රමය (Diffusion vs. GPT-4o)
මේ ෆොටෝ ජෙනරේට් කරන විදිය ගැන කතා කරද්දී ගොඩක් අය හිතන්නේ එකම ක්රමයයි තියෙන්නේ කියලා.
Traditional Diffusion (DALL·E වගේ) - මේ ක්රමයේදී කරන්නේ ෆොටෝ එකකට වැලි දැම්මා වගේ සම්පූර්ණයෙන්ම බොඳ කරලා (Noise එකතු කරලා), ඊට පස්සේ ඒ වැලි ටික අයින් කරන ගමන් අලුත් ෆොටෝ එකක් මතු කරගන්න එක.
අලුත් ක්රමය (GPT-4o Architecture) - OpenAI ආයතනය පැහැදිලි කරන විදියට, එයාලගේ අලුත්ම මාදිලි වැඩ කරන්නේ ටිකක් වෙනස් විදියට. ඒකෙදි Text සහ Images දෙකම එකම මොළයකින් (end-to-end trained multimodal network) තමයි process කරන්නේ. ඒ කියන්නේ වචන කෑලි වලට කඩනවා වගේම, ෆොටෝ එකත් කුඩා "Tokens" වලට කඩලා තමයි AI එක තේරුම් ගන්නේ. මේක ගොඩක් වේගවත් වගේම ස්වභාවිකයි.
අවසානයේ AI එක “1985” කියලා දන්නේ කොහොමද?
AI එකට 1985 කියන්නේ නිකන්ම අවුරුද්දක් නෙවෙයි. එයාව ට්රේන් කරද්දී, 1985 කියන වචනයත් එක්ක තියෙන ෆොටෝස් වල අර බොඳ වුණු පාට (film grain), ලොකු කොණ්ඩ මෝස්තර, සහ පරණ විලාසිතා තියෙනවා කියලා එයා ඉගෙනගෙන තියෙනවා.
ඒ නිසා ඔයාගේ ෆොටෝ එකේ තියෙන මුහුණේ හැඩය (Embedding එක) අරගෙන, ඒකට 1985 විලාසිතාවන් එකතු කරලා අලුත්ම ෆොටෝ එකක් නිර්මාණය කරනවා.
මේක Photoshop වලින් කරනවා වගේ කෑලි කපලා අලවනවා නෙවෙයි. මේක බිලියන ගණනක් ගණනය කිරීම් හරහා නිර්මාණය වෙන අලුත්ම යථාර්ථයක්. 🤯
This media is not supported in your browser
VIEW IN TELEGRAM
(==) or (===) 😕
Infinity CS
Photo
Do you know about Wireshark? 🦈
If you are working in the IT field, you have definitely heard about Wireshark. Generally, it captures and shows the data (network packets) going through our network. Actually, it is a network protocol analyzer. But this is not only a tool that shows just data lines. Then we will explore the hidden things inside Wireshark.
This tool was created by Gerald Combs in 1998 and the 1st name was Ethereal. But in 2006, Gerald Combs moved to another company and rebranded the name as Wireshark.
What is the meaning of the Wireshark logo? Do you know about it? 🕵️♂️
The logo is a shark fin. A shark can detect the scent of blood from far away in the ocean, this tool can capture even the smallest piece of data travelling through network cables, which is why it was named "Wireshark".
Most users and beginners think we can see the data only in text format. But we can also hear Voice over IP (VoIP) calls if there is not encrypted. We can gather the voices as audio clips and we can replay them. And also we can access the unencrypted photos and files going through the network using Wireshark. 🎧
Is this a hacking tool or an X-ray machine? 🩻
Hackers (kids) in the films use Wireshark for hacking like password stealing, that's not what it was actually designed for. Network engineers use Wireshark as an X-ray machine to solve tasks like finding OSPF routing errors, DNS problems and issues during data transmission between IPv4 and IPv6 within large networks.
If you are curious about what the background apps on your computer are actually doing, setup Wireshark and take a closer look! 🔍
@TheInfinityAI
If you are working in the IT field, you have definitely heard about Wireshark. Generally, it captures and shows the data (network packets) going through our network. Actually, it is a network protocol analyzer. But this is not only a tool that shows just data lines. Then we will explore the hidden things inside Wireshark.
This tool was created by Gerald Combs in 1998 and the 1st name was Ethereal. But in 2006, Gerald Combs moved to another company and rebranded the name as Wireshark.
What is the meaning of the Wireshark logo? Do you know about it? 🕵️♂️
The logo is a shark fin. A shark can detect the scent of blood from far away in the ocean, this tool can capture even the smallest piece of data travelling through network cables, which is why it was named "Wireshark".
Most users and beginners think we can see the data only in text format. But we can also hear Voice over IP (VoIP) calls if there is not encrypted. We can gather the voices as audio clips and we can replay them. And also we can access the unencrypted photos and files going through the network using Wireshark. 🎧
Is this a hacking tool or an X-ray machine? 🩻
Hackers (kids) in the films use Wireshark for hacking like password stealing, that's not what it was actually designed for. Network engineers use Wireshark as an X-ray machine to solve tasks like finding OSPF routing errors, DNS problems and issues during data transmission between IPv4 and IPv6 within large networks.
If you are curious about what the background apps on your computer are actually doing, setup Wireshark and take a closer look! 🔍
@TheInfinityAI
Telegram
Infinity CS
Exploring the world of CS, AI, and ML 🧠. Sharing high-quality resources, tools, and interesting tech breakthroughs daily. Join us to learn and build together!
❤1
Cython is a method that allows you to write code in an easy-to-use language like Python while achieving the execution speed of a complex language like C. It is an essential tool for anyone pursuing a career in fields such as Data Science or Machine Learning.
What is Cython?
In today’s world, Python is a fundamental component of AI, Machine Learning, and Data Science. It is easy to learn, and its code is highly readable. However, Python’s primary drawback is its execution speed; it operates significantly slower compared to languages like C or C++. Cython was developed as a solution to this issue—it is a technology that converts Python code directly into C code (a C-extension) for execution.
1. It is not just a compiler; it is a new language 📝
Many people mistakenly believe that Cython is merely a tool. In reality, Cython is a superset of Python—meaning it includes everything found in Python, plus additional features. If you take a standard Python file (typically with a .py extension), save it with a .pyx extension, and compile it using Cython, it will generate complete C code.
2. Bringing C-style Static Typing to Python ⚙️
A primary reason for Python’s relatively slow execution is its dynamic typing (meaning the data type of a variable is determined only when the code is actually running). With Cython, however, you can use special keywords like cdef and cpdef to explicitly define types—such as specifying that a variable is an int or a double, right when you write the code (static typing). This approach can boost execution speed by up to 100x, especially within for loops.
3. A Solution to Python’s Biggest Hurdle: The GIL 🔓
Python does not support true multi-threading (performing multiple tasks simultaneously) because of the Global Interpreter Lock (GIL), which ensures that only one thread executes at any given time. By using Cython to operate at the C level, you can bypass the GIL by wrapping code blocks in a with nogil: statement. This enables genuine hardware-level parallel processing, often facilitated by technologies like OpenMP.
How do libraries we all use—such as Pandas, Scikit-learn, and SciPy—manage to process massive datasets with such high speed? While they appear to be Python libraries on the surface, their core algorithms are actually implemented using Cython. Therefore, if you feel that Python’s speed is insufficient while building AI/ML models or large data pipelines, be sure to try Cython before switching to another language.
@TheInfinityAI
What is Cython?
In today’s world, Python is a fundamental component of AI, Machine Learning, and Data Science. It is easy to learn, and its code is highly readable. However, Python’s primary drawback is its execution speed; it operates significantly slower compared to languages like C or C++. Cython was developed as a solution to this issue—it is a technology that converts Python code directly into C code (a C-extension) for execution.
1. It is not just a compiler; it is a new language 📝
Many people mistakenly believe that Cython is merely a tool. In reality, Cython is a superset of Python—meaning it includes everything found in Python, plus additional features. If you take a standard Python file (typically with a .py extension), save it with a .pyx extension, and compile it using Cython, it will generate complete C code.
2. Bringing C-style Static Typing to Python ⚙️
A primary reason for Python’s relatively slow execution is its dynamic typing (meaning the data type of a variable is determined only when the code is actually running). With Cython, however, you can use special keywords like cdef and cpdef to explicitly define types—such as specifying that a variable is an int or a double, right when you write the code (static typing). This approach can boost execution speed by up to 100x, especially within for loops.
3. A Solution to Python’s Biggest Hurdle: The GIL 🔓
Python does not support true multi-threading (performing multiple tasks simultaneously) because of the Global Interpreter Lock (GIL), which ensures that only one thread executes at any given time. By using Cython to operate at the C level, you can bypass the GIL by wrapping code blocks in a with nogil: statement. This enables genuine hardware-level parallel processing, often facilitated by technologies like OpenMP.
How do libraries we all use—such as Pandas, Scikit-learn, and SciPy—manage to process massive datasets with such high speed? While they appear to be Python libraries on the surface, their core algorithms are actually implemented using Cython. Therefore, if you feel that Python’s speed is insufficient while building AI/ML models or large data pipelines, be sure to try Cython before switching to another language.
@TheInfinityAI
Telegram
Infinity CS
Exploring the world of CS, AI, and ML 🧠. Sharing high-quality resources, tools, and interesting tech breakthroughs daily. Join us to learn and build together!