phazonoverload's comments

phazonoverload · 2026-09-02T13:35:03

I will give it a look!

phazonoverload · 2026-09-02T13:27:09

I haven't, but that's just because it really isn't my personal usage pattern.

phazonoverload · 2026-09-02T13:25:03

A literal manual typo. Good catch, will fix.

phazonoverload · 2026-09-02T13:23:05

The beauty of this is that you can just swap out the platform and everything remains as it's the same backend. You make a really good point, one that I haven't really considered, but I also only have so many hours in the day to be fiddling around with stuff.

manmal · 2026-09-02T15:36:35

I think I'd use Discord instead. Clankers are happy to set it all up for you.

sunaookami · 2026-09-03T08:17:44

Discord is even worse than Telegram in the privacy aspect though!?

manmal · 2026-09-04T07:21:33

Depends what actors you trust. They are equally bad but personally I‘m way more comfortable with the Discord story.

phazonoverload · 2026-09-02T13:20:48

Hehe I really am just working it out as I go along - I promise it is fairly painless. Hugging Face allow you to specify your machine and then browse models that fit. And then you can just vibe out 'oh this one is a bit slow let me try another' etc

phazonoverload · 2026-09-02T10:08:27

It is not

phazonoverload · 2026-09-02T10:07:04

Running it depends on RAM, which is what I wrote, bandwidth is important for speed. I chose my words carefully, but you are absolutely right.

phazonoverload · 2026-09-02T13:33:57

Coming back to this a few hours later, I've decided to add a section to explain this to the blog post. Thank you for flagging it.

phazonoverload · 2026-09-02T10:06:18

My perf sucks compared to yours. Added it to the post - same model averages 325 tok/s in processing prompts, and 34 tok/s in token generation. What am I doing wrong..?

argee · 2026-09-02T15:14:11

Wow, that's just about half the perf. I'm not sure what you're doing differently, though our hardware is a bit different: I am on a Macbook Pro M4 Pro, while you're on a Mac Mini.

I would try a different version of the model from HuggingFace while ensuring it's MLX. I'm also using LM Studio, not oMLX, and I've seen some threads like these:

https://www.reddit.com/r/LocalLLaMA/comments/1spuwir/omlx_10...

phazonoverload · 2026-09-02T10:05:32

I'm the author - hello! Added to the post! Qwen averages 325 tok/s in processing prompts, and 34 tok/s in token generation. That isn't instant, but it's quick enough that I never really think about it.

phazonoverload · 2026-09-02T10:05:15

I'm the author - hello! I talk about it in the blog post - knowing what's being run, knowing where it's being run, and not having anyone else control it.