Generative modeling with sparse transformers
- ID
- 892
- Status
- new
- Published
- 23 Apr 2019, 3:00 PM
- Fetched
- 27 Jun 2026, 7:47 PM
- Provider
- OpenAI News
- Category
- ai-labs
- Original URL
- https://openai.com/index/sparse-transformer
- Source URL
- https://openai.com/news/rss.xml
Excerpt
We’ve developed the Sparse Transformer, a deep neural network which sets new records at predicting what comes next in a sequence—whether text, images, or sound. It uses an algorithmic improvement of the attention mechanism to extract patterns from sequences 30x longer than possible previously.
Summary
No summary yet. It will appear after the daemon summarizes this item.