# Open-source self-hosted AI & platform — agency delivery

> Platform engineering for self-hosted AI workloads: model serving with vLLM and Ollama, Kubernetes operations, and reproducible Terraform/OpenTofu environments.

For AI agents: curated site map at [https://www.devolute.org/llms.txt](https://www.devolute.org/llms.txt). Canonical HTML (preferred for search engines): [https://www.devolute.org/en/agency/open-source-self-hosted-ai-platform](https://www.devolute.org/en/agency/open-source-self-hosted-ai-platform).

- Language: en
- Last modified: 2026-05-06
- Markdown URL: https://www.devolute.org/en/agency/open-source-self-hosted-ai-platform.md

## Self-hosted AI & platform engineering with clear operational boundaries

Inference serving lanes, GitOps-managed Kubernetes baselines, and Infrastructure as Code—designed for predictable cost, security, and compliance.

## Unifying model serving, Kubernetes, and Infrastructure as Code

Running open-weight models in production is rarely bottlenecked by model weights—it is bottlenecked by GPU scheduling, secrets management, network segmentation, and repeatable environment provisioning. We combine vLLM/Ollama serving architectures, Kubernetes platform engineering, and Terraform/OpenTofu automation into a single operable stack.

## What we deliver

- **Self-hosted AI model serving**: Production inference lanes with vLLM, Ollama, Llama, and Mistral—isolated from batch workloads with clear latency and cost budgets.
- **Kubernetes platform engineering**: Helm, GitOps release pipelines, GPU node pools, network policies, and secrets management your team can run day to day.
- **Infrastructure as Code (Terraform & OpenTofu)**: Reproducible dev, staging, and production environments with state locking, CI plan/apply reviews, and disaster recovery runbooks.

## Engineering standards in production

Platform discipline for long-term ownership

- **Separate inference and batch paths**: Prevent heavy background jobs or fine-tuning runs from degrading interactive user latency.
- **Full-stack observability**: GPU utilisation, token throughput, queue depth, and service SLO alerts wired into Prometheus and Grafana.
- **Clean operational handover**: Documented GitOps repositories, IaC modules, and incident playbooks transferred directly to your internal engineers.

## Choosing the right architectural scope

- **Dedicated layer support**: When you specifically need LLM serving setup, a Kubernetes platform audit, or a Terraform-to-OpenTofu migration.
- **End-to-end AI platform rollout**: When hardware sizing, IaC provisioning, Kubernetes baselines, and model serving must ship together for a new environment.
- **Sovereign enterprise stack**: When self-hosted AI is combined with PostgreSQL, Keycloak IAM, and strict European data residency requirements.

## FAQ

- **Do we need Kubernetes for every self-hosted AI deployment?**: No. We size the platform to your team and workload—from lean single-node or Coolify-managed setups for smaller footprints to full GitOps Kubernetes clusters when multi-node GPU scaling and strict multi-tenancy are required.
- **Should we standardize on Terraform or OpenTofu?**: Both work well in production; we help you choose based on licence governance, existing provider modules, and CI/CD integration.
- **Can you train our ops team during the rollout?**: Yes. We pair with your infrastructure engineers on upgrades, failure drills, and capacity planning so you are never locked into an external operator.

## Related services & guides

- [Self-hosted AI, Kubernetes & IaC — best-practices guide](https://www.devolute.org/en/insights/open-source/self-hosted-ai-kubernetes-iac-best-practices)
- [Open-source AI infrastructure](https://www.devolute.org/en/agency/open-source-ai-infrastructure)
- [Kubernetes platform engineering](https://www.devolute.org/en/agency/kubernetes-platform-engineering)
- [Coolify self-hosted PaaS consulting](https://www.devolute.org/en/agency/coolify-self-hosted-paas-consulting-deployment)
- [Terraform / OpenTofu infrastructure](https://www.devolute.org/en/agency/terraform-opentofu-infrastructure)

## Discuss your self-hosted AI & platform project

Scope model serving, Kubernetes baselines, and IaC automation with our engineering team.

[Contact us](https://www.devolute.org/en/agency/open-source-self-hosted-ai-platform#Contact)