Send CathInfo's owner Matthew a gift from his Amazon wish list:
https://www.amazon.com/hz/wishlist/ls/25M2B8RERL1UO

« previous next »
Pages: 1 2 [3]

Author Topic: Youtube version of my AI update  (Read 2342 times)

0 Members and 1 Guest are viewing this topic.

Offline St Giles

  • Supporter
Re: Youtube version of my AI update
« Reply #10 on: Yesterday at 07:26:27 PM »
Is it worth getting a GPU with ECC (error correction)? I don't know if GPU's regularly come with the equivalent of "hot" or "dead" pixels like camera sensors, but that doesn't matter as much in pictures and games as getting a perfect AI output without compounding errors. RTX Pro x000 series are the AI specific GPU's, though they are of course more expensive. Google AI says the 4000 is better than the 3060, but worse than the 3090 performance wise. I'm not sure how important ECC is.
Logged

Offline Matthew

  • Mod
Re: Youtube version of my AI update
« Reply #11 on: Yesterday at 10:53:53 PM »
Quote from: St Giles on Yesterday at 07:26:27 PM
Is it worth getting a GPU with ECC (error correction)? I don't know if GPU's regularly come with the equivalent of "hot" or "dead" pixels like camera sensors, but that doesn't matter as much in pictures and games as getting a perfect AI output without compounding errors. RTX Pro x000 series are the AI specific GPU's, though they are of course more expensive. Google AI says the 4000 is better than the 3060, but worse than the 3090 performance wise. I'm not sure how important ECC is.

That's taken care of in software. The model is loaded into the VRAM, Ollama (or equivalent software) takes care of the actual legwork of running the LLM on the video card. I'm sure it has some kind of error handling.
But I haven't heard ECC as being an issue or a thing with GPUs being used for AI.
Logged
Pages: 1 2 [3]
« previous next »