4GB VRAM

Mmmmm… I would wait a few days, and try a GGUF quantization of Kimi Linear once its better supported: https://huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct

Otherwise you can mess with Qwen 3 VL now, in the native llama.cpp UI. But be aware that Qwen is pretty sycophantic like ChatGPT: https://huggingface.co/unsloth/Qwen3-VL-30B-A3B-Instruct-GGUF/blob/main/Qwen3-VL-30B-A3B-Instruct-UD-Q4_K_XL.gguf

If you’re interested, I can work out an optimal launch command. But to be blunt, with that setup, you’re kinda better off using free LLM APIs with a local chat UI.

Really raises the question of how profitable these scam ads have become.

It does, doesn’t it?

If Meta is literally making billions from ads, how much are the scammers buying them making?

People keep coming back to these obviously booby-trapped websites to claw at a thin veneer of simulated friendship because they’ve got nothing better to do with their lives.

Eh, part of it is FOMO. I can’t blame older folks if all their friends and kids and grandkids are on (say) Facebook, and that’s the most advanced means they have of communicating with them that they’re capable of using. What are they supposed to do?

Yeah, exactly… In other words, ‘general server buildout.’

That’s (as I understand it) a misconception.

Apple attaches their laptop RAM the same way all smartphones do. It’s a wide bus with LPDDR, which makes it an unusual configuration amongst laptops, but it’s technically conventional. And relatively cheap.

AMD’s Strix Halo chips are the same. Apple could use LPCAMM to make the memory upgradable if they wanted, they just… don’t.

When we talk ‘packaging’, we’re talking putting chips on advanced substrates with denser wires than one could possibly get on a motherboard (or a ‘mini’ motherboard which is kinda what Apple/smartphone RAM is packaged on), stuff silicon fabs have to do:

https://www.tsmc.com/english/dedicatedFoundry/services/advanced-packaging

And HBM falls into this bucket. The way its hooked up to the processor is physically different than PC RAM sticks, or Apple’s RAM. This is mostly not done on consumer stuff because its very expensive, and most of TSMC’s advanced packaging production capacity is reserved for server stuff.

Ah. Yeah. And its on the fab to do that.

I always though it’d be cool for CPUs to switch to packaged RAM, too. Samsung apparently tried to do it with Wide I/O for mobile ARM stuff, but it never caught on.

Plenty of folks do AMD. A popular homelabsetup is 32GB AMD MI50 GPUs, which are quite cheap on eBay. Even Intel is fine these days!

But what’s your setup, precisely? CPU, RAM, and GPU.

It must be bugged and busy-looping in the background.

…Is it that interchangeable?

TBH I know little of memory fabs and HBM ICs, but I know (say) TSMC can’t just switch from a power-optimized process to a high frequency one at the drop of a hat.

They can ALL be run on RAM, theoretically. I bought 128GB so I can run GLM 4.5 with the experts offloaded to CPU, with a custom trellis/K quant mix; but this is a ‘personal use’ tinkerer setup basically no one but hobbyists will touch.

Qwen Next is good at that because its very low active parameter.

…But they aren’t actually deployed that way. They’re basically always deployed on cloud GPU boxes that serve dozens/hundreds of people at once, in parallel.

AFAIK the only major model actually developed for CPU inference is one of the esoteric Gemma releases, aimed at mobile. And the bitnet experiments, which aren’t very big so far.

(In case it’s not obvious, this is my special interest, and I’m happy to ramble on about how to set up ‘niche gaming rig hybrid models’ for anyone interested).

I just got a 2x64GB 6000 kit before its price skyrocketed by like $130. I saw other kits going up, but had no clue I timed it so well.

…Also, why does “AI” need so much CPU RAM?

In actual server deployments, pretty much all inference work is done in VRAM (read: HBM/GDDR); they could get by with almost no system RAM. And honestly most businesses are too dumb to train anything that extensively. ASICs that would use, say, LPDDR are super rare, and stuff like Hybrid/IGP inference is the realm of a few random folks with homelabs… Like me.

I think ‘AI’ might be an overly broad term for general server buildout.

There are difficult ‘AI’ tools.

Look up controlnet workflows or VACE, just to start, much less little niches in vapoursynth pipelines or image editing layers. You could spend days training them, messing with the implementation, then doing the manual work of carefully and deliberately applying them. This has, in fact, has been happening in film production for awhile, just in disguise.

Same with, say, LLMs used in game mods where appropriate, like the Rimworld mod. That’s careful creative expression.

…As usual, it’s tech bros fucking everything up by dumbing it down to zero-option prompt box and then shoving that in front of as many people as possible to try and monopolize their attention.

In other words, I agree with the author that what I hate about ‘AI art’ is the low effort ‘sloppiness.’ It’s gross, like rotten fast food. It makes me sad. And that’s 99.999% of all AI art.

…But it doesn’t have to be like that.

It’s like saying the concept of the the fediverse sucks because Twitter/Facebook suck, even if 99.999% of what folks see is the slop of the later. It’s not fair to the techniques, and it’s not holding the jerks behind mass slop proliferation accountable.