seems unusably slow, and is this for short context?

Given these numbers it has some potential to become quite usable for unattended workloads, especially if decode can be batched across multiple sessions (ideally enough of them to get some reuse of the sparsely streamed weights). (Of course this ultimately makes prefill times explode as you try and increase the workload even further. But that's arguably the natural bottleneck on any interesting local LLM inference, being a compute bound step.)

idk I thought the same until I got personal amex support at work, now it’s my roofline on how useful/trusted an agent could be. Now I book all my travel through them, and it’s an absolute blessing

Is LinkedIn a normal social network? It seems like an absolute dumpster fire to me (it even has shorts now!)

I’m quite happy that my friends have labelled me as “an absolute idiot with anything but computers.” It’s much easier to relate to people at that level, adopt a happy-go-lucky attitude, and being wrong about 99% of things. being the smart one isn’t such a great label, and is (mostly) all in your head

It’s 27b active params vs 13b active params, so you’d expect it to be 2x more expensive when serving multiple users

And the goalposts must move once again..

I think this comes in with the wrong assumptions from the start. The thing that US vs Apple taught us is to not demand access publicly, this puts companies in an awkward spot. If approached more tacitly, gag order etc the company has nothing to gain but everything to lose.. and more likely to comply. I hugely doubt that intentional backdoors don’t exist for the most powerful countries / people

Bit of a counter point, but I ran a very nice wfh setup at a previous job, 4k monitor and ergo keyboards. I was a bit dismayed when I joined a new co and found that we had 1080p monitors and a generic Logitech keyboard. After a week of the text looking pixelated I entirely got used to it and think maybe complaints about monitors being a health hazard and rsi are a tad overblown by a vocal minority.

Stylistically ugly code that is globally consistent (ideally across the entire ecosystem) >> locally beautiful code that is inconsistent with the rest of the codebase / up to authors taste. No one likes black formatting, but it’s at least consistent. “Your car can be any color you like as long as it’s black”

With regards to your example, adding a comma to the final element should preserve multi-line. You just don’t know the formatting rules yet apparently