​​GPT-3: Language Models are Few-Shot Learners

#openAI train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting
Their model applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model.

Achieves strong performance on many NLP datasets, including translation, q&a, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic.

Also, they find that GPT-3 can generate samples of news articles in which human evaluators have difficulty distinguishing from articles written by humans.

175 billion parameters! And on some tasks, it is not performed
It is all you need to know about


paper: https://arxiv.org/abs/2005.14165.pdf

#nlp #gpt #gpt3 #language #model

​​GPT-3 application for website form generation

Turns out #GPT3 model is capable of generating #JSX code (which is HTML layout for #React ) given the description of the required blocks to generate.

Author reports that there are exceptions, given current output limit of the model of 512 tokens.

Why this is important: one might suppose that in the future programmers will just write specifications and tests for the AI to generate the code. Given the speed of progress that won’t be surprising at all.

And probably the more sophisticated models will be capable of using hard output limit to produce a code for the output generation but that obviously is still an area for active research.

More realistic evaluation is that the upcoming code generation tools is that it will just allow more people to build products, following #nocode movement.

Twitter thread: https://twitter.com/sharifshameem/status/1282676454690451457

#codegeneration #NLU

Sharif Shameem improved the original app, which is now capable of generating real applications, as he demostrates with a simple ToDo app.

#GPT3 #codegeneration

​​how gpt3 works. a visual thread

short thread with cool animations how gpt-3 works by jay alammar

collected twitter thread: https://threader.app/thread/1285498971960598529


#nlp #transformers #gpt3 #jayalammar

#GPT3 attracted lots of attention. Let’s try new format of discussing the matter in the comments, provided by peerboard.

For accessing the comments, just click the link below ⬇️⬇️⬇️, authorize with the telegram and follow the discussion.

Applying GPT-3 to generate neural network code

Matt Shumer used GPT-3 to generate code for a machine learning model, just by describing the dataset and required output.

#GPT3 #inception #codegeneration #NLU #NLP

english to regex

generating regex by just describing it and providing an example (apparently powered by gpt-3)


web page: https://losslesshq.com

#regext #gpt3

​​Philosopher AI — website to generate text with #GPT3

Tool to generate text on different topics. Sensible topics such as sex, religion or even nationality are blocked.

Great way to spread the awareness on #ai and to show nontechnical friends that #Skynet is not a problem to be concerned with yet.

Website: https://philosopherai.com/philosopher/humanity-on-mars-73ac00

#nlu #nlp

​​🔥New breakthrough on text2image generation by #OpenAI

DALL·E: Creating Images from Text

This architecture is capable of understanding style descriptions as well as complex relationship between objects in context.

That opens whole new perspective for digital agencies, potentially threatening stock photo sites and new opportunies for regulations and lawers to work on.

Interesting times!

Website: https://openai.com/blog/dall-e/

#GAN #GPT3 #openai #dalle #DL

​​Summarizing Books with Human Feedback

#OpenAI fine-tuned #GPT3 to summarize books well enough to be human-readable. Main approach: recursively split text into parts and then meta-summarize summaries.

This is really important because once there will be a great summarization #SOTA we won't need editors to write posts for you. And researchers ultimatively will have some asisstance interpreting models' results.

BlogPost: https://openai.com/blog/summarizing-books/
ArXiV: https://arxiv.org/abs/2109.10862

#summarization #NLU #NLP

​​AI Generated Pokemon Sprites with GPT-2

Author trained #GPT2 model to generate #pokemon sprites, encoding them as the lines of characters (including color). Surprisingly, results were decent, so this leaves us wonder if #GPT3 results would be better.

YouTube: https://www.youtube.com/watch?v=Z9K3cwSL6uM
GitHub: https://github.com/MatthewRayfield/pokemon-gpt-2
Article: https://matthewrayfield.com/articles/ai-generated-pokemon-sprites-with-gpt-2/
Example: https://matthewrayfield.com/projects/ai-pokemon/

#NLU #NLP #generation #neuralart