Chat with Captain Furguson the Cat

Or how to run LLMs offline in your browser.

Hide/ShowCaptain Furguson the Cat

Why

Real reasons:

  • I personally don’t like sharing personal information with online chat bots
  • The cost of online chat bots could be preventive
  • Running on your computer means you only pay for electricity
  • Accessibility, not everyone has access to ChatGPT, having the ability to run the models on your machine makes them more accessible
  • Many companies have restrictions on what you can share in with online chat bots.
  • You can use it offline

Demo

Try Offline GPT Store

How does it work

But you can just use the output (bin and wasm files):

Models

  • Llama-2-7b-chat-hf-q4f16_1
  • gemma-2b-it-q4f32_1

Browse all models on Hugging face: https://huggingface.co/mlc-ai

Browser Support

Browser Support

Show me the code

Clone the code base on Hiku Dev

Reach out

If you have any suggestions or feature requests feel free to reach out.