Tales from the Terrordrome
Este perfil es de un servidor federado y podría estar incompleto. Explorar más contenido en la instancia original.
Tales from the Terrordrome
Este perfil es de un servidor federado y podría estar incompleto. Explorar más contenido en la instancia original.
r/Place 2023: The Good Ending (peertube.io) en
Retentive Network: A Successor to Transformer for Large Language Models (arxiv.org) en
This is an exciting new paper that replaces attention in the Transformer architecture with a set of decomposable matrix operations that retain the modeling capacity of Transformer models, while allowing parallel training and efficient RNN-like inference without the use of attention (it doesn't use a softmax)....
Extending Context Window of Large Language Models via Positional Interpolation (arxiv.org) en
Interesting technique to increase the context window of language models by finetuning on a small number of samples after pretraining....