Nerd alert!
Yep, I did it.
After months/years of shunning AI altogether, then slowly embracing it for trivial tasks such as turning photos into cartoons, I’ve gone as all-in as one can without dedicating a GPU to the cause.
“Look, Ma, I’m a cartoon!” – via ChatGPT
It all began when my IT brethren constantly extolled the value of AI to me. “Hey, just run that through ChatGPT“, they said. Admittedly, I’ve been doing some of that now, for how long it’s difficult to say but at least a few months. Don’t get me wrong, it is still not my “go to”. But, that said, it does have its place (and I kinda hope it stays there, but we all know it’s not going to).
Then one day, discussion turned towards Ollama and hosting one’s own chatbot. After a period of time where I became more and more intrigued, I began Googling and – long story short – I ended up with an instance of Ollama behind Open WebUI running in a Docker container within a VM on my Proxmox hypervisor. Not at all an ideal setup for an AI chatbot but, for the time being, it works.
I’ve also played with Ollama (without the GUI, since I hadn’t yet discovered it at that point) on a dedicated machine running Ubuntu Desktop. This was quite entertaining since, when I gave it a particularly unique task, it woke from the dead and exercised fans inside the chassis that I didn’t even know it had 😮😂. Alas, it has an AMD GPU in the box and I hear Nvidia is the way to go. So, maybe if and when I get more serious about all this, I’ll find an Nvidia GPU to stuff into the thing. Time will tell.
It might not be human but it sure is chatty…
One thing that I did already learn is that the model I’m currently running – llama3:8b – is a “text only” model, meaning I can’t use it to generate cute images like the one above. I was hoping to at least try it so I could compare processing times between my local setup and the big boys. I guess that will be a task for another day.
In the meantime, I am hoping to spend some cycles becoming more knowledgeable about the whole “AI” thing, keeping in mind I already have several irons in the fire and this just adds to them.
One thing is certain: the home lab is never boring!
Good stuff!!! Funny, I haven’t really gotten in to it either. Is it because we are… *gasp* OLD?!!?! I might try something like this to see if it really is, in fact, the coolest thing since sliced bread.
Honestly, I don’t think it is because we are old; rather, I think it has more to do with the fact that younger generations have had so much at their fingertips for as long as they can remember and so they have never had a need to think with their own brain. They just google or otherwise use tech to get the answers they need/want and that is how they have always operated.
So, in short, I guess it is in fact because we are old.