I have one server with a cheap MI50 instinct. Those come for really cheap on eBay. And it’s got really good memory bandwidth with HBM2. They worked ok with ollama until recently when they dropped support for some weird reason but a lot of other software still works fine. Also older models work fine on old ollama.

The other one runs an RTX 3060 12GB. I use this for models that only work on nvidia like whisper speech recognition.

I tend to use the same models for everything so I don’t have the delay of loading the model. Mainly uncensored ones so it doesn’t choke when someone says something slightly sexual. I’m in some very open communities so standard models are pretty useless with all their prudeness.

For frontend i use OpenWebUI and i also run stuff directly against the models like scripts.

I mainly use it for Spanish which I have a basic proficiency in. It just accompanies me on my learning journey. It may be wrong sometime but not often. Like the other reply said, LLMs are good at languages, it’s what they were originally designed for until people found out they could do more (but not quite as well).

And as for filtering, I just use it as a news feed sanitizer with a whole bunch of rules. It will miss things sometimes but it’s also my ruleset that’s not perfect. I often come across the unfiltered sources anyway and even if it misses something, it’s only news. Nothing really important to me.

I work with AI and use it personally, but I have my own servers running local models which solves tons of privacy concerns. The inaccuracy is another problem but not a big one for me as I know it and will simply fact check. Also, I don’t really use it for knowledge anyway. Just to filter news to my interest, help with summaries and translation etc.

People use AI as some all-knowing oracle but an LLM is not meant for that at all.

I know, but at least they are making a case which most news outlets probably would but don’t have the means for a long legal fight. They might get some precedent out of this from which the whole news industry will benefit.

I don’t think AI training should ever be fair use. These companies are making billions of other people’s work and giving nothing back.

The problem I’ve been seeing with email on my own domain is that some services refuse it, saying “please enter a real email address” 🤬 some others just silently refuse to send a confirmation code so I can’t register either (I think tinder did this). Especially the “not a real email address” really pissed me off.

And with proton I got “Anonimisation services are forbidden” once at least.

I forget which services, but it’s Hella annoying…

The marketshare of Google and Microsoft on email is really becoming a problem.

True, the NYT does seem to have a significant impact on OpenAI though.

Very very good point. Google’s ubiquity makes that very difficult.

Haha especially the angry bird is genius