https://twitter.com/srush_nlp/status/1720113524121235577
Fine grained control of KV caching is one of the reasons I work on haloop.
Install today: https://www.youtube.com/watch?v=2-G5bomAkfs
Fine grained control of KV caching is one of the reasons I work on haloop.
Install today: https://www.youtube.com/watch?v=2-G5bomAkfs
X (formerly Twitter)
Sasha Rush (@srush_nlp) on X
I got excited about a bunch of fast LLM generators (vLLM, MLC, etc) but none of them implement prefix caching / kv storage? This seems like a benchmark failure where everyone optimized Tok/Sec, and use cases all have massive prompts. Find myself back using…
New work from IDSIA. Transformer's largest bottleneck is a feedforward MLP block that expands hidden dimension four times and shrinks it back.
Instead of running all parts of this network on every request, sparse gating decides what subnetwork to run depending on the input. This is straightforward at inference time but hard to backpropagate at training time.
Sparsely-gated Mixtures of Experts date back to Ivakhnenko and Lapa (see Section 4) , and now feature an open source implementation from Robert.
https://arxiv.org/abs/2310.10837
Instead of running all parts of this network on every request, sparse gating decides what subnetwork to run depending on the input. This is straightforward at inference time but hard to backpropagate at training time.
Sparsely-gated Mixtures of Experts date back to Ivakhnenko and Lapa (see Section 4) , and now feature an open source implementation from Robert.
https://arxiv.org/abs/2310.10837
🔥1
Btw does anybody know how to use Triton kernels with torch.compile without graph breaks?
ICLR 2024 has a track that accepts anonymized blog posts. Submissions are now open:
https://x.com/fpedregosa/status/1722197379418665247?
https://x.com/fpedregosa/status/1722197379418665247?
X (formerly Twitter)
Fabian Pedregosa on X
📢 Attention machine learning enthusiasts!📢
The #ICLR2024 blog post track is now accepting submissions!. This is a great opportunity to share your insights on the latest #MachineLearning research and present it at one of the main ML conferences.
https:/…
The #ICLR2024 blog post track is now accepting submissions!. This is a great opportunity to share your insights on the latest #MachineLearning research and present it at one of the main ML conferences.
https:/…
Alec Radford on language models. He gave this lecture right after he completed his work on GPT-2.
https://www.youtube.com/watch?v=BnpB3GrpsfM
https://www.youtube.com/watch?v=BnpB3GrpsfM
YouTube
L11 Language Models -- guest instructor: Alec Radford (OpenAI) --- Deep Unsupervised Learning SP20
Course homepage:
https://sites.google.com/view/berkeley-cs294-158-sp20/home
Lecture Instructor: Alec Radford (OpenAI)
Course Instructors: Pieter Abbeel, Aravind Srinivas, Peter Chen, Jonathan Ho, Alex Li, Wilson Yan
CS294-158-SP20: Deep Unsupervised Learning…
https://sites.google.com/view/berkeley-cs294-158-sp20/home
Lecture Instructor: Alec Radford (OpenAI)
Course Instructors: Pieter Abbeel, Aravind Srinivas, Peter Chen, Jonathan Ho, Alex Li, Wilson Yan
CS294-158-SP20: Deep Unsupervised Learning…
https://nips.cc/virtual/2023/events/journal_track_2023
My VAE paper is on NeurIPS Journal Track. See you in New Orleans!
My VAE paper is on NeurIPS Journal Track. See you in New Orleans!
👍2🔥1
RNNs are officially back. This paper is such a good read and the experiments are actually serious, we've been doing 350M models so far with our fast LSTM on 6B token runs.
Recipe for success: use VERY large hidden state, do not materialize activations early, use CUTLASS, do final runs on 300B tokens.
https://arxiv.org/abs/2312.00752
Recipe for success: use VERY large hidden state, do not materialize activations early, use CUTLASS, do final runs on 300B tokens.
https://arxiv.org/abs/2312.00752
Ha, selective scan is not using CUTLASS, it's written using cub! https://github.com/state-spaces/mamba/blob/main/csrc/selective_scan/selective_scan_fwd_kernel.cuh
GitHub
mamba/csrc/selective_scan/selective_scan_fwd_kernel.cuh at main · state-spaces/mamba
Mamba SSM architecture. Contribute to state-spaces/mamba development by creating an account on GitHub.
But what about BERT? Monarch matrices have you covered https://www.youtube.com/live/IS59IwGLvVs?si=3yvBYGsOSx3jU2tE
YouTube
Monarch Mixer: Making Foundation Models More Efficient - Dan Fu | Stanford MLSys #86
Episode 86 of the Stanford MLSys Seminar Series!
Monarch Mixer: Making Foundation Models More Efficient
Speaker: Dan Fu
Abstract:
Machine learning models are increasingly being scaled in both sequence length and model dimension to reach longer contexts…
Monarch Mixer: Making Foundation Models More Efficient
Speaker: Dan Fu
Abstract:
Machine learning models are increasingly being scaled in both sequence length and model dimension to reach longer contexts…
Word2vec received a test of time award at NeurIPS https://papers.nips.cc/paper_files/paper/2013/hash/9aa42b31882ec039965f3c4923ce901b-Abstract.html
Happy New Year!I have started curating a collection of new year speeches. Try talking to one!
https://wilab.org.ua/watch/
https://wilab.org.ua/watch/
❤1
Mamba is the first large scale-trained RNN that speaks Ukrainian. It's a large architectural shift from transformers: RNNs use constant memory and have linear complexity (down from quadratic!) when generating sequences.
RNNs is a recipe to get efficient machine learning systems on the end-user device.
I've forked it so you can run it on CPU (tested on Mac) without cuda dependencies:
RNNs is a recipe to get efficient machine learning systems on the end-user device.
I've forked it so you can run it on CPU (tested on Mac) without cuda dependencies:
git clone https://github.com/proger/mamba-cpu
cd mamba-cpu
pip install -e .
python benchmarks/benchmark_generation_mamba_simple.py --model-name "state-spaces/mamba-130m" --prompt "Мій пес написав цей код на Python і вийшло таке:" --topp 0.9 --temperature 0.7 --repetition-penalty 1.2
👍3