Textless NLP: Generating expressive speech from raw audio
Generative Spoken Language Model (GSLM), the first high-performance NLP model that leverages recent breakthroughs in representation learning, allowing it to work directly from only raw audio signals, without any labels or text.
Link
#deeplearning #nlp #selfsupervised #facebook
@pythonicAi
Generative Spoken Language Model (GSLM), the first high-performance NLP model that leverages recent breakthroughs in representation learning, allowing it to work directly from only raw audio signals, without any labels or text.
Link
#deeplearning #nlp #selfsupervised #facebook
@pythonicAi
Meta
Textless NLP: Generating expressive speech from raw audio
We’re introducing GSLM, the first language model that breaks free completely of the dependence on text for training. This “textless NLP” approach learns to generate expressive speech using only raw audio recordings as input.
Summarizing Books with Human Feedback
OpenAI fine-tuned GPT3 to summarize books well enough to be human-readable. Main approach: recursively split text into parts and then meta-summarize summaries.
This is really important because once there will be a great summarization SOTA we won't need editors to write posts for you. And researchers ultimatively will have some asisstance interpreting models' results.
Read more
ArXiV
#deeplearning #nlp #gpt #transformer
@pythonicAi
OpenAI fine-tuned GPT3 to summarize books well enough to be human-readable. Main approach: recursively split text into parts and then meta-summarize summaries.
This is really important because once there will be a great summarization SOTA we won't need editors to write posts for you. And researchers ultimatively will have some asisstance interpreting models' results.
Read more
ArXiV
#deeplearning #nlp #gpt #transformer
@pythonicAi
Openai
Summarizing books with human feedback
Scaling human oversight of AI systems for tasks that are difficult to evaluate.