Pinned Loading

1. RWKV-LM RWKV-LM Public

   RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference,…

   Python 12.1k 830

2. ChatRWKV ChatRWKV Public

   ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

   Python 9.3k 684

3. LinearAttentionArena LinearAttentionArena Public

   Here we will test various linear attention designs.

   Python 53 6

4. nanoRWKV nanoRWKV Public

   Forked from karpathy/nanoGPT

   RWKV in nanoGPT style

   Python 165 12

5. RWKV-CUDA RWKV-CUDA Public

   The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )

   Cuda 201 33

6. SmallInitEmb SmallInitEmb Public

   LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence

   Python 45 3