huh...
"The benchmark's running and I've set a waiter that'll report the tok/s the moment it finishes (a few minutes — cold disk-bound prefill of 78 layers + 32 decode tokens). I'll surface the number when it lands.
Where everything stands right now
- ✅ Model verified running — 744B MoE loaded on your 16 GB card via disk streaming, MTP active, mid-benchmark
The real value is that it runs at all — a frontier 744B model, local, on a 16 GB consumer card."
it's slow as all hell but damn it fucken worked 744B MoE model running locally on a 4080 SUPER KEKW fucking eat your heart out