A general dedicated to the discussion and development of local language models.

Previous Threads: >>98296924 & >>98289826

►News
>(01/06) DynaTemp merged in SillyTavern staging and koboldcpp experimental
>(01/04) 2bit Quants > https://github.com/ggerganov/llama.cpp/pull/4773
>(01/04) Discussion on Layer Selective Rank Reduction (LASER) for overall LLM improvements >>98243179
>(01/02) TabbyAPI/exllamav2 implemented CFG support. >>98220295
>(12/31) CUDA 12.3 Build of Koboldcpp-1.53 released https://github.com/kalomaze/koboldcpp/releases
>(12/28) Mixtral finetuning issue was discovered and fixed in Transformers >>98158030
>(12/23) Discussion about using LoRAs as an alternative to QMoE compression for Mixtral >>98055798
>(12/23) llama.cpp server support added to SillyTavern 1.11.1; Koboldcpp 1.53 released

►FAQ: https://rentry.org/er2qd
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png
►Jarted QRD: https://rentry.org/jarted

►Getting Started
https://rentry.org/llama-mini-guide
https://rentry.org/8-step-llm-guide
https://rentry.org/llama_v2_sillytavern
https://rentry.org/lmg-spoonfeed-guide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
General Purpose: https://hf.co/spaces/HuggingFaceH4/open_llm_leaderboard
https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard
Programming: https://hf.co/spaces/bigcode/bigcode-models-leaderboard
Censorbench: https://codeberg.org/jts2323/censorbench

►ERP/RP Datasets
https://rentry.org/qib8f
https://rentry.org/ashh2

►Alpha Calculator
https://desmos.com/calculator/ffngla98yc

►Text Gen. UI, Inference Engines
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/lmg-anon/mikupad
https://github.com/turboderp/exui
https://github.com/ggerganov/llama.cpp
https://github.com/turboderp/exllamav2