10 Open-Source AI Developer Tools You Can Self-Host
Paying for several AI subscriptions adds up quickly, and sending client code or private documents to someone else’s servers is not always an option. The good news is that a lot of serious AI tooling is now open source. You can run it on your own laptop, a spare PC or a small server, and keep your data where it is.
This guide covers ten self-hostable tools that are worth knowing about if you build websites, write code or automate work for clients. For each one you will find what it is for, who it suits and anything to watch out for, including licences, because “open source” does not always mean the same thing.
The Short Version
| Tool | What it does | Good starting point for |
|---|---|---|
| Ollama | Downloads and runs AI models locally | Everyone: it is the foundation for most of the others |
| llama.cpp | Fast engine that runs models on ordinary hardware | Tinkerers who want full control |
| Open WebUI | ChatGPT-style chat in your browser | A friendly front end for local models |
| Continue | AI coding assistant for VS Code and JetBrains | Developers who want local code help |
| Tabby | Self-hosted code completion server | Small teams sharing one assistant |
| Aider | AI pair programmer in the terminal | Developers comfortable with Git |
| AnythingLLM | Chat with your own documents | Searching PDFs, notes and docs privately |
| Flowise | Drag-and-drop builder for AI apps | Prototyping chatbots without much code |
| n8n | Workflow automation with AI steps | Automating client and business tasks |
| Langfuse | Tracks and debugs AI app behaviour | Anyone shipping an AI feature to real users |
Before You Start: What Hardware Do You Need?
You do not need a data centre. A small model with around 7 to 8 billion parameters, compressed (“quantised”) to 4-bit, is roughly a 5 GB download and runs on most modern laptops with 16 GB of RAM. It will be slower on a CPU alone and much faster with a dedicated graphics card. Bigger models give better answers but need far more memory, so start small and move up only if you need to.
One honest expectation to set: small local models are very useful for drafting, summarising and everyday coding help, but they are not as strong as the best paid cloud models on long or complicated tasks.
1. Ollama: The Easiest Way to Run Models Locally
Ollama is where most people should begin. You install it, type a command such as ollama run llama3.2, and it downloads the model and opens a chat in your terminal. It also runs a local API on your machine, which is what lets the other tools on this list talk to your models.
Best for: anyone trying local AI for the first time. Licence: MIT. Watch out for: model downloads are large, so check your disk space before pulling several.
We have a full walkthrough in How to Run Local AI on Your PC for Free.
2. llama.cpp: The Engine Under the Hood
llama.cpp is the project that made running large language models on normal computers practical. Many tools, including Ollama, build on its work. Using it directly gives you more control over settings and performance, at the cost of a more technical setup.
Best for: developers who like to fine-tune performance or run models on unusual hardware. Licence: MIT.
3. Open WebUI: A ChatGPT-Style Interface for Your Own Models
Typing into a terminal gets old quickly. Open WebUI gives you a clean chat interface in the browser, with conversation history, document uploads and support for several users. It connects to Ollama and to any service with an OpenAI-compatible API.
Best for: giving yourself, your family or a small team a private chat assistant. Licence: free to self-host; its licence is based on BSD but adds rules about keeping the Open WebUI branding, so read it before you rebrand it for a client.
4. Continue: AI Coding Help Inside Your Editor
Continue is an extension for VS Code and JetBrains IDEs. You can chat about your code, ask it to explain a function, or get autocomplete suggestions. The useful part is that you choose the model: a local one through Ollama for privacy, or a cloud model when you need more power.
Best for: WordPress and web developers who want Copilot-style help without sending every file to a third party. Licence: Apache 2.0.
5. Tabby: A Shared Coding Assistant for Small Teams
Tabby runs as a server, so a small agency can host one coding assistant and let every developer connect to it from their editor. It focuses on code completion and can use your own repositories as context.
Best for: teams of two or more who want one central, private setup. Licence: mostly Apache 2.0, with some enterprise features under a separate licence. Watch out for: it works best with a GPU on the server.
6. Aider: Pair Programming in the Terminal
Aider works in your terminal alongside Git. You describe a change, it edits the files and commits the result with a clear message, so every AI change is easy to review or undo. It can use local models, though it performs best with stronger cloud models.
Best for: developers who already live in the command line. Licence: Apache 2.0.
7. AnythingLLM: Chat With Your Own Documents
AnythingLLM lets you load PDFs, Word files, notes and web pages, then ask questions about them. Answers are based on your documents rather than the model’s general knowledge. It comes as a desktop app or a Docker container, and it can use Ollama for fully offline use.
Best for: searching project documentation, contracts or research notes privately. Licence: MIT. Watch out for: answers are only as good as the documents you give it, so still check anything important.
8. Flowise: Build AI Apps by Dragging Boxes
Flowise is a visual builder. You connect blocks for a model, a document store, memory and tools, and you get a working chatbot or AI workflow you can embed on a website. It is a quick way to show a client a prototype before writing custom code.
Best for: prototyping website chatbots and document assistants. Licence: Apache 2.0.
9. n8n: Workflow Automation With AI Built In
n8n is an automation tool in the same family as Zapier and Make, but you can host it yourself. It connects hundreds of apps and has AI steps, including an AI agent node, so you can build things like “summarise every new support email and post it to Slack”.
Best for: freelancers who automate tasks for clients. Licence: “fair-code” (the Sustainable Use License). It is free to self-host for your own or your clients’ internal use, but it is not open source in the strict sense, so check the terms before offering it as a paid hosted service.
10. Langfuse: See What Your AI App Is Actually Doing
Once real people use an AI feature you built, you need to know what it is answering, how much it costs and where it goes wrong. Langfuse records each request and response so you can debug problems and compare prompts.
Best for: anyone shipping an AI chatbot or feature to clients. Licence: the core is MIT, with some enterprise features licensed separately.
A Simple Setup to Start With
- Install Ollama and pull one small model.
- Add Open WebUI if you want a chat window in your browser.
- Install the Continue extension in your code editor and point it at the same model.
- Only add AnythingLLM, Flowise or n8n when you have a specific job for them.
That covers chat, coding help and private documents without a monthly bill. Add more tools only when a real project needs them; self-hosting is easier to manage when you keep it small.
Things to Keep in Mind
- Security: if you expose a tool to the internet, put it behind a login and HTTPS. Do not leave an open AI server on a public IP.
- Updates: these projects move fast. Update regularly and read the release notes.
- Model licences: the tool’s licence and the model’s licence are separate. Check the model’s terms before using it in commercial work.
Related guides on AiTalkings:
- How to Run Local AI on Your PC for Free
- Best AI Code Assistants for Web Developers
- How to Set Up AI Automation for Your Business
Frequently Asked Questions
Are these tools really free?
The software is free to download and run. Your costs are your own hardware, electricity and, if you use a server, hosting. Some tools also sell paid cloud or enterprise versions.
Can I use them without a graphics card?
Yes, with small models. Expect slower answers on a CPU. A dedicated GPU makes a large difference.
Is self-hosted AI completely private?
If both the tool and the model run on your machine, your prompts do not leave it. If you connect a tool to a cloud model, your data goes to that provider.




