Since most of the success of large language models arise from making the underlying transformer model bigger, and training it on more text (trillions of tokens), and training it using more compute for longer duration, transformer-based language model offers an intriguing possibility just like the balloons did in early 1900’s: Without having to figure out the principles behind human intelligence, we could build machines that solve more cognitive and human-like tasks simply by building them bigger, and training them with more data, compute, and human feedback.

….

Once the basic principles behind dirigibles were figured out, they had a favorable scaling law going for them — to go further distances, and to carry heavier payloads you simply had to make them bigger, and give them more powerful engines. This was purely an engineering task.

Our implementation achieves speedups of up to 665x over naive sequential application, allowing training nonlinear RNNs at unprecedented scales. To showcase this, we apply ParaRNN to adaptations of LSTM and GRU architectures, successfully training models of 7B parameters that attain perplexity comparable to similarly-sized Transformers and Mamba2 architectures.

While NetHack is complex in comparison to other RL benchmarks, it still contains only a tiny fraction of the complexity of the real world (its source code is 4.2MB, which provides an upper bound on its Kolmogorov complexity). As long as we can’t reliably solve this game for which we can easily collect lifetimes worth of data, have access to detailed textual resources (and even the underlying source code), and large-scale datasets of human gameplay, I think AGI remains a ways off.

🪖 Research Engineer в области RL

Вы будете работать над практическими R&D-проектами, от робототехники и управления беспилотным транспортом до рекомендательных систем и макроэкономического моделирования.

🧪Research Scientist в области RL

Вы будете заниматься написанием статей на top-tier конференции (NeurIPS, ICLR, ICML), проводить эксперименты и предлагать новые исследовательские идеи.