[PROMPT]:
Tesla's quarterly earnings report exceeded expectations, with revenue reaching $25 billion. Elon Musk commented that

[GENERATION]:
the new generation will be an important focus on the company.
"I think if this is the case, it would have a better time for these companies
to try and get us back," said Ramesen.

[PROMPT]:
The Federal Reserve announced an interest rate decision today, keeping rates at 5.25%. Fed Chair Powell stated that

[GENERATION]:
inflation remains the only source of a short-term reduction in the price of
capital value. The current balance is that you can see more than the difference
between these two options. This isn't to say that this is the question.
The U.S. government has given us its own way:
"Even if that were not in the world, I would have had a good deal with the EU
and the West are in the same place."

[PROMPT]:
According to a new report by Reuters, global markets are responding to the latest trade negotiations between the US and China. Analysts say that

[GENERATION]:
the deal will be reached with the end of March this year, and China has refused
to sign up for their promises in terms of a future-rich economy for Britain to
do more to its own people.
In what the EU needs to pay to get the European Union, it can take on a larger
scale for Britain because Australia would not support the rest of the world's
people. The UK is one of only two countries considering whether they have a
chance to be able to make the next step."

[PROMPT]:
A team of researchers studying the human brain has found evidence that

[GENERATION]:
memory formation involves complex interactions between the brain and brain regions.
The research is published in PLoS ONE, available online this week by National
Institute for Science. The study was conducted in 2011.

  ЛУЧШАЯ #1: Alzheimer's Treatment (научный стиль!)

[PROMPT]:
A landmark clinical trial has shown promising results for a new Alzheimer's treatment. The drug, developed by Eli Lilly,

[GENERATION]:
reduced cognitive decline by 10% in the U.S., and placebo-treated mice (Kleman 2012; McFarland, 2005; Davis 2001).

"If you can find that it or not, let alone do what the patients see fit," Dr. Gettler says. "This patient will be available for any medical reasons."

  ЛУЧШАЯ #2: Amazon Q3 Earnings

[PROMPT]:
Amazon reported Q3 earnings with net income of $9.9 billion, up 240% from last year. The company's cloud division AWS continues to

[GENERATION]:
expand the stock market in 2014 and will continue to develop further growth.

  ЛУЧШАЯ #3: OpenAI Breakthrough

[PROMPT]:
OpenAI has announced a breakthrough in artificial intelligence research. Their new model demonstrates unprecedented capabilities in

[GENERATION]:
the area of the AI to explore, identify and respond rapidly to those on these challenges in developing a world-wide network of networks.

In addition to the recent rise in the development of the digital age, researchers have found that there was no immediate response for increased communication between satellite and satellite channels in China. The report found that a UAW might be in line with a specific attack on the ground and its ability to communicate with each other.

ВЫВОД

Step 8500 показывает прогресс:
- Научные ссылки в правильном формате
- Цитаты с именами докторов
- Короткие ответы стали точнее
- Val Loss упал: 1.35 → 1.33

  ╔══════════════════════════════════════════════════════════════╗
  ║                    PIR 270M PRETRAIN                         ║
  ║                      ЗАВЕРШЁН!                               ║
  ╠══════════════════════════════════════════════════════════════╣
  ║  ✅ Step: 39,000 / 39,000 (100%)                            ║
  ║  📉 Final Loss: 1.1040                                      ║
  ║  ⏱️  Total Time: 26.07 hours                                ║
  ║  💾 Tokens Trained: 4.00B                                   ║
  ║  🏆 PPL: 3.0                                                ║
  ╚══════════════════════════════════════════════════════════════╝

● PIR 270M vs GPT-2 — Полное сравнение

  ┌───────────────────────┬────────────────────┬─────────────────┬─────────────────┐
  │                       │      PIR 270M      │   GPT-2 124M    │   GPT-2 345M    │
  ├───────────────────────┼────────────────────┼─────────────────┼─────────────────┤
  │ Параметры             │ 272M               │ 124M            │ 345M            │
  ├───────────────────────┼────────────────────┼─────────────────┼─────────────────┤
  │ Архитектура           │ Parallel Scan O(T) │ Attention O(T²) │ Attention O(T²) │
  ├───────────────────────┼────────────────────┼─────────────────┼─────────────────┤
  │ Hardware для обучения │ 1x A100            │ 8x A100         │ 8x A100         │
  ├───────────────────────┼────────────────────┼─────────────────┼─────────────────┤
  │ Время обучения        │ ~50 часов          │ ~4 дня          │ ~4 дня          │
  └───────────────────────┴────────────────────┴─────────────────┴─────────────────┘

  ┌──────────────────┬──────────┬────────────┬────────────┐
  │     Метрика      │ PIR 270M │ GPT-2 124M │ GPT-2 345M │
  ├──────────────────┼──────────┼────────────┼────────────┤
  │ WikiText-2 PPL ↓ │ 5.14     │ ~30        │ ~22        │
  ├──────────────────┼──────────┼────────────┼────────────┤
  │ Val Loss ↓       │ 1.10     │ ~2.85      │ ~2.5       │
  └──────────────────┴──────────┴────────────┴────────────┘
  ↓ = меньше лучше

  ┌─────────────────┬──────────────┬───────────────┬───────────────┐
  │     Метрика     │   PIR 270M   │  GPT-2 124M   │  GPT-2 345M   │
  ├─────────────────┼──────────────┼───────────────┼───────────────┤
  │ Forward 512 tok │ 20,550 tok/s │ ~25,000 tok/s │ ~15,000 tok/s │
  ├─────────────────┼──────────────┼───────────────┼───────────────┤
  │ Генерация       │ 57 tok/s     │ ~70 tok/s     │ ~45 tok/s     │
  ├─────────────────┼──────────────┼───────────────┼───────────────┤
  │ ms/token        │ 17.5 ms      │ ~14 ms        │ ~22 ms        │
  └─────────────────┴──────────────┴───────────────┴───────────────┘

  ┌──────────────┬──────────┬────────────┬────────────┐
  │   Метрика    │ PIR 270M │ GPT-2 124M │ GPT-2 345M │
  ├──────────────┼──────────┼────────────┼────────────┤
  │ Distinct-1 ↑ │ 0.39     │ ~0.35      │ ~0.38      │
  ├──────────────┼──────────┼────────────┼────────────┤
  │ Distinct-2 ↑ │ 0.82     │ ~0.75      │ ~0.78      │
  └──────────────┴──────────┴────────────┴────────────┘
  ↑ = больше лучше

  ┌───────────┬──────────┬───────────────┬────────────┐
  │   Тест    │ PIR 270M │  GPT-2 124M   │ GPT-2 345M │
  ├───────────┼──────────┼───────────────┼────────────┤
  │ LAMBADA ↑ │ 1.6%     │ ~1-2%         │ ~3-5%      │
  ├───────────┼──────────┼───────────────┼────────────┤
  │ BoolQ ↑   │ 25%      │ ~50% (random) │ ~50%       │
  └───────────┴──────────┴───────────────┴────────────┘
  Base models не обучены отвечать на вопросы

  Главный результат
  ┌──────────────────┬──────────┬──────────┐
  │                  │ PIR 270M │  GPT-2   │
  ├──────────────────┼──────────┼──────────┤
  │ PPL лучше в      │ 4-6 раз  │ baseline │
  ├──────────────────┼──────────┼──────────┤
  │ GPU для обучения │ 1        │ 8        │
  ├──────────────────┼──────────┼──────────┤
  │ Сложность        │ O(T)     │ O(T²)    │
  └──────────────────┴──────────┴──────────┘

Архитектура без attention работает.

[A] The physical activity of a brain. The brain is responsible for processing and recording the brain's flow, which can be seen as a function of memory. It is often used to describe emotions such as emotions or feelings of pain or other stress...

[A] A Bitcoin Blockchain(ТОЧКА)org is a digital currency that allows you to create and use bitcoin, while using bitcoins, the process can be used for transactions by people who are interested in buying or selling.

[A] This paper is a computer science course that uses computers to read and write. It is the best of all tools, including machine learning,
machine learning algorithms, data science, software development and other technologies.

[A] The political process is a system of government, and it is not intended to be used in any way. It can lead to a more democratic society and an effective system of governance... every citizen must first vote for the people to vote, which will also have a right to control their own party.