Local LLM Utility

Objective

Setup a local LLM and a generic application for carrying out common LLM tasks.

  • OLlama Allows you to host the models locally on your won machine.

  • Anything LLM A generic LLM utility application that can be used with local and remotely hosted LLM.

ollama-anythingllm

Setup

  1. Install OLlama

Download and install Ollama on your machine.

You can Watch this video to learn more about Ollama.

  1. Install AnythingLLM

Download and install

Try it out

  1. Launch OLlama

Open a terminal windown on your machine. Run the command below. This will download the gemma2 model weights to the file system cache.

Watch this video to learn more about how to use other models with OLlama

ollama run gemma2
  1. Setup AnythingLLM to use Ollama gemma2 setup

  • Launch the AnythingLLM desktop application

  • Provide email address to register

  • Setup AnythingLLM to use local OLlama

anything-ollama-setup

  • Click on Save changes and you are done !!!
  1. Create workspace

  • Think of this as a session e.g., name it ’test'

  • Interact with local LLM !!

launch-anythingLLM