Hey. Yeah you. No don’t look over your shoulder. I’m not talking to the guy behind you. Look, we’ve been meaning to tell you that you’re doing a pretty good job out there. Proud of you. Keep up the good work.

  • 4 Posts
  • 135 Comments
Joined 1 year ago
cake
Cake day: November 18th, 2024

help-circle
  • My company is doing small trial runs and trying to get feedback on if stuff is helpful. They are obviously pushing things because they are hopeful, but most people report that AI is helpful about 45% of the time. I’m sorry your leadership just dove in head first. That’s sound like such a pain.





  • Honestly all it is for my family is eat the big meal and watch the game. I usually nap during the latter part because I ate so much.

    If I’m at my moms then we might do some games. Sometimes we do what is called an “auction party”. Where everyone who shows up brings two things they don’t want anymore. Then we all get the same amount of fake money and sit in the same room and bid on it. I can be real funny with gag items going through long bidding wars. It can also be pretty helpful to get an item you actually care about.

    During a covid auction party a half used bottle of hand sanitizer had an absolutely riotous bidding war over it.





  • I used to set some coins on the table (five or so) and then if I get it right I move a coin to the other side. If I get one wrong I move them all back. I can’t move on until I’ve moved all the coins to the other side. This was generally for music practice but it seems pretty applicable here.







  • I am young and have a computer science degree, and I still struggle at times. I get it.

    For games, I’d try to install steam and run them through steam if thats how you’d normally do it on windows. Then for me the main setting to play with (on a game by game basis) is setting the game to use proton (in the compatibility settings of the game) and whether or not to use steam input for controller support.

    If you are trying to install a non steam game, maybe look into lutris. Though I’m on the techy side, and I hear a lot of people like heroic game launcher on the less techy side.

    Good luck. I think it’s fair to run out of energy while trying get the right combo, but if ya stick to it I’m confident you’ll find the set up that works for you.





  • Donations would be great. I dream of mine would be to get a large enough following to live a decent life off of donations and just make art. Kinda far fetched, but it would be great.

    I know one person I’ve found that did that is Chris from Airwindows. His art is a bit technical, but he makes interesting audio software and shares it all open source. I donate a bit to him.


  • Yeah setting up openwebui with llamacpp is pretty easy. I would start with building llamacpp by cloning it from github and then following the short guide for building it linked on the readme. I don’t have a Mac, but I’ve found building it to be pretty simple. Just one or two commands for me.

    Once its built just run llama-sever with the right flags telling it to load model. I think it can take huggingface links, but I always just download gguf files. They have good documentation for llama-server on the readme. You also specify a port when you run llama-server.

    Then you just add http://127.0.0.1:PORT_YOU_CHOSE/v1 as one of your openai api connections in the openwebui admin panel.


    Separately, if you want to be able to swap models on the fly, you can add llama-swap into the mix. I’d look into this after you get llamacpp running and are somewhat comfy with it. You’ll absolutely want it though coming from ollama. At this point its a full replacement IMO.


  • With 128GB of ram on a Mac, GLM 4.5 Air is going to be one of your best options. You could run it anywhere from Q5 to Q8 depending on how you wanna manage your speed to quality ratio.

    I have a different system that likely runs it slower than yours will, and I get 5 T/s generation which is just about the speed I read at. (Using q8)

    I do hear that ollama may be having issues with that model though, so you may have to wait for an update to it.

    I use llamacpp and llama-swap with openwebui, so if you want any tips on switching over I’d be happy to help. Llamacpp is usually one of the first projects to start supporting new models when they come out.

    Edit: just reread your post. I was thinking it was a newer Mac lol. This may be a slow model for you, but I do think it’ll be one of the best your can run.