Prefill is survivable if you cache well. But what kills me is the context. Qwen 27 needs a ton of room for KV Cache. I guess not an issue on a 128 GB Halo or Spark, but if you are running of consumer/prosumer GPUs it's miserable to be compacting every 120k tokens.

I find that 35B-A3B is much easier to run on my M4 Max (both prefill and generation)

It's well known 35b is much faster (on any hardware) and quite a bit dumber

This really very much depends on how you are using it, I think. If you intend to leave it to solve long context problems and write whole prototypes, the 27B is going to be much better.

But if you are sort of pair-programming with the model, the speed obviously matters and I think then the 35B is acceptably smart, and when it's wrong it'll be wrong much more quickly. It seems very good on SQL and PHP, and I assume on typical JS and Python.

I would rather work that way, so I hope they do produce a small MoE model.

I had no idea. Where can we learn stuff like this?

Learn about MoE models and also just look at the benchmarks of the two models. For example https://artificialanalysis.ai/models/comparisons/qwen3-6-27b... clearly shows both the intelligence and speed differences. It's a bit degenerate but you can get some useful info from e.g. the /r/localllama subreddit

How are you running it on a Strix Halo? The weights aren't out yet, are they?

I interpret @syntaxing as meaning they are looking forward to running Qwen3.8-27B, but are frustrated by prefill times with other models, such as Qwen3.6-27B.

I meant Qwen3.6. Unsloth supposedly has early preview of the model and the VRAM requirement is the same so most people expect similar model size and type.

What type of agentic tasks are you using it for (eg how complex)?

For personal stuff, I use it with AnythingLLM. It replaced any Google search for me. For coding, I run opencode though I have been debating switching to Pi. I would argue it’s at Sonnet 3 level.