Trying to maintain this old repo, answering some friends' request that have Google AI Pro plan and don't know where to waste the colab credits.
So we resurrected and reworked this project completely, with better attribution and sd-scripts as a submodule.
github.com/Linaqruf/kohya-tr…
github.com/KohakuBlueleaf/To…
Tokenomos is open sourced
It resolved the frontend (scheduler and kv cache/state manager system for continuous batching) overhead in vLLM and make small model inference few time faster even with batch size 512 on RTX5090
These number are from RTX5090
Guess how I get nearly 3x speed over vLLM on small model?
Better GPU kernel? Weirdo quantize trick?
NO, all the model code are fully borrow from vLLM. All I did is just FULLY rewrite the frontend and now the CPU is no longer the bottleneck
DeepGHS × Laxhar Lab
We’re pleased to announce our collaboration with Laxhar Lab, the team behind the Noob model series, to develop a next-generation anime base model.
Together, we aim to advance open-source anime AI.
Explore Laxhar Lab's models: huggingface.co/Laxhar
The thing about PRs in the age of coding agents is that I can build almost any feature or fix any bug with significantly less effort than it takes to read over and test a PR. There are exceptions, but I think the way we manage open source projects is going to have to change.