@Lenguador@kbin.social avatar

Lenguador

@Lenguador@kbin.social

Este perfil es de un servidor federado y podría estar incompleto. Explorar más contenido en la instancia original.

Retentive Network: A Successor to Transformer for Large Language Models (arxiv.org) en

This is an exciting new paper that replaces attention in the Transformer architecture with a set of decomposable matrix operations that retain the modeling capacity of Transformer models, while allowing parallel training and efficient RNN-like inference without the use of attention (it doesn't use a softmax)....

Lenguador,
@Lenguador@kbin.social avatar

This looks amazing, if true. The paper is claiming state of the art across literally every metric. Even in their ablation study the model outperforms all others.

I'm a bit suspicious that they don't extend their perplexity numbers to the 13B model, or provide the hyper parameters, but they reference it in text and in their scaling table.

Code will be released in a week https://github.com/microsoft/unilm/tree/master/retnet

Artificial Muscles Flex for the First Time: Ferroelectric Polymer Innovation in Robotics (scitechdaily.com) en

A new ferroelectric polymer that efficiently converts electrical energy into mechanical strain has been developed by Penn State researchers. This material, showing potential for use in medical devices and robotics, overcomes traditional piezoelectric limitations.

Lenguador,
@Lenguador@kbin.social avatar

So, taking the average bicep volume as 1000cm3, this muscle could: exert 1 tonne of force, contact 8% (1.6cm for a 20cm long bicep), and require 400kV and must be above 29 degrees Celcius.

Maybe someone with access to the paper can double check the math and get the conversion efficiency from electrical to mechanical.

I expect there's a good trade-off to be made to lower the force but increase the contraction and lower the voltage. Possibly some kind of ratcheting mechanism with tiny cells could be used to overcome the crazy high voltage requirement.

Hardwiring ViT Patch Selectivity into CNNs using Patch Mixing (arxiv.org) en

Vision transformers (ViTs) have significantly changed the computer vision landscape and have periodically exhibited superior performance in vision tasks compared to convolutional neural networks (CNNs). Although the jury is still out on which model type is superior, each has unique inductive biases that shape their learning and...

Lenguador,
@Lenguador@kbin.social avatar

I find the link valuable. Despite the proliferation of AI in pop culture, actual discussion of machine learning research is still niche. The community on Reddit is quite valuable and took a long time to form.

  • Todo
  • Suscrito
  • Moderado
  • Favoritos
  • random
  • noticiascr
  • CostaRica
  • Todos las revistas