AI Sovereignty in Switzerland: What It Actually Costs and What Makes Sense

Every few weeks, someone asks us the same question: "Can you host GPUs so we can run AI workloads in Switzerland?"

The answer is more nuanced than yes or no. We have been having this conversation with Swiss companies across industries since 2024, from agencies building AI-powered products to enterprises with strict compliance requirements. The pattern is always the same: there is a vague sense that AI workloads should run locally, but the economics and the actual requirements are rarely discussed openly.

This post is that open discussion. What GPU hosting actually costs, where existing hardware already covers more than people think, and why the most important question is usually not "where does the model run?" but "where does my data live?"

Everyone asks, nobody has concrete requirements

This is the first thing we noticed. The interest in sovereign AI infrastructure is real and growing. Companies hear about data protection concerns with US cloud providers, read about European AI regulations, and conclude that they need local GPU compute.

But when we ask "What model do you want to run? What latency do you need? What throughput?", the answer is usually "We do not know yet" or "We are still evaluating." There is rarely a specific model, a specific dataset, or a specific SLA requirement behind the inquiry.

This is not a criticism. The AI space moves so fast that most organizations are still figuring out where AI fits in their products and workflows. But it means that the infrastructure decision, which involves significant capital investment, is being driven by a general feeling rather than concrete requirements.

Our approach: start with what you actually need today, not what you might need in two years.

The honest economics of GPU hardware

Let us talk numbers, because this is where most conversations get uncomfortable.

Entry-level: workstation GPUs (what exists today). Cards like the NVIDIA RTX A2000 with 12 GB VRAM are already in some datacenter servers. They are not designed for large-scale AI inference, but they handle more than people expect: computer vision models, image classification, video transcoding, CAD rendering, and quantized language models in the 7B parameter range. If your use case fits within 12 GB VRAM, you can start today without new hardware procurement.

Mid-range: dedicated AI cards. The NVIDIA L4 with 24 GB VRAM is the current sweet spot for inference at reasonable cost. A server with multiple L4 cards runs in the range of CHF 50-80k depending on configuration. This covers most production inference workloads for models up to 13B parameters (or larger models with quantization).

High-end: training and large model inference. The NVIDIA A100 (80 GB HBM) and H100 (80 GB HBM3) are the industry standard for serious AI workloads. A single GPU-optimized server with 8x H100 cards costs upwards of CHF 300k. Under full load, one H100 draws around 700W. A fully equipped server with cooling needs significant power and rack space.

And then there is the procurement process. Since November 2023, US export regulations require documentation of end customer and intended use for high-end NVIDIA GPUs. Vendors like Supermicro run an approval process before shipping. This adds weeks to delivery timelines.

The CAPEX reality for infrastructure providers. GPU hosting is not like CPU hosting where utilization can be shared across many customers. GPUs are expensive, power-hungry, and the technology moves fast. An H100 purchased today may be outperformed by next year's architecture at a lower price point. For a Swiss infrastructure provider to invest six figures in GPU hardware, there needs to be planning security through concrete customer commitments. Without that, the economics do not work.

We are honest about this. We are not going to pretend we have a warehouse full of H100 servers waiting for customers. What we do have is the platform and operational capability to run GPU workloads on our managed Kubernetes infrastructure as soon as the hardware is in place, and the willingness to procure hardware when there is a concrete project behind it.

What works today on existing infrastructure

The gap between "we have nothing" and "we need H100 clusters" is wider than most people think. There is a middle ground that covers many real-world use cases:

Quantized language models. A 7B parameter model quantized to 4-bit runs in under 4 GB VRAM. That fits on the entry-level GPUs we already have in our datacenters. The quality is surprisingly good for focused tasks: summarization, classification, code completion, document Q&A. It is not GPT-4, but for many business applications it does not need to be.

Computer vision and classification. Custom-trained models for image recognition, quality inspection, document processing. These are typically small (under 1 GB) and run efficiently on workstation GPUs. Several of our customers already run these workloads on our platform.

Video processing. Modern GPUs have dedicated encode/decode engines that handle video transcoding without touching the CUDA cores used for AI inference. This means you can run video processing and lightweight model inference on the same GPU without contention.

Embedding generation. Turning documents into vector embeddings for RAG systems is GPU-accelerated but does not need high-end hardware. Embedding models are small (typically under 2 GB) and batch processing can run on any GPU with enough VRAM.

The point: if your use case fits within the constraints of existing hardware, you can be in production within weeks, not months. No new procurement, no export approvals, no six-figure investment decisions. Talk to us about what is already available.

Apple Silicon: the unexpected middle tier

One development worth mentioning: Apple Silicon Macs with unified memory are becoming a viable option for local AI inference. A Mac Studio with an M4 Ultra can be configured with up to 192 GB of unified memory that is shared between CPU and GPU.

For AI inference, this means you can load models that would require an A100 (80 GB) or even an H100 on NVIDIA hardware. The inference speed is slower than dedicated NVIDIA GPUs with HBM3 memory, but for many use cases, latency of a few seconds per generation is perfectly acceptable.

The economics are different too. A high-end Mac Studio costs a fraction of an equivalent NVIDIA GPU server, uses significantly less power, and does not require special cooling or rack infrastructure. For organizations that need local inference of larger models but cannot justify six-figure hardware investments, this is a pragmatic path.

We are evaluating offering hosted Mac infrastructure, similar to what other Swiss providers already offer in the dedicated Apple Silicon space. If this is relevant for your use case, reach out.

The real insight: data sovereignty matters more than model hosting

Here is where we want to reframe the conversation entirely.

When a company says "we need AI sovereignty", what they usually mean is: "we cannot send our sensitive data to a US cloud provider." That is a valid concern. But the solution is not necessarily running your own GPU cluster.

Think about what actually happens in most AI applications:

1. Your documents, customer data, internal knowledge base, these are your sensitive assets
2. They get chunked, embedded, and stored in a vector database
3. When a user asks a question, the relevant chunks are retrieved (RAG)
4. The chunks plus the question are sent to a language model for a response

Steps 1-3 involve your data and should absolutely run on infrastructure you control. A managed Kubernetes cluster in a Swiss datacenter, with pgvector for embeddings and proper backup and access controls, gives you full sovereignty over your data.

Step 4, the model inference, is where the nuance lies. The model itself is stateless. It receives a prompt, generates a response, and forgets everything. If you strip personally identifiable information from the RAG context before sending it to the model, the sovereignty concern is significantly reduced.

The pragmatic architecture for most Swiss companies:

• Data layer: Swiss-hosted, on your managed Kubernetes cluster. Vector database, document storage, embedding pipeline, RAG retrieval. Full control, full sovereignty.
• Model layer: Use the best model for the job. That might be a cloud API (OpenAI, Anthropic, Google) for general-purpose reasoning, a hosted open-source model for specific tasks, or a local quantized model for the most sensitive operations.

This hybrid approach gives you data sovereignty where it matters (your data never leaves Switzerland) while leveraging the best available models regardless of where they run. And it costs a fraction of running your own GPU cluster.

When you actually need local GPU inference

There are legitimate cases where cloud model APIs are not an option:

Regulated industries with strict data residency. Healthcare, finance, government. When regulations explicitly require that no data leaves the country, even temporarily, you need local inference. Note: most regulations we encounter are about data storage and processing, not about transient API calls. Read your actual compliance requirements, not the summary.

Air-gapped environments. Defense, critical infrastructure. No internet connectivity means no cloud APIs. This is a clear case for local inference on dedicated hardware.

Latency-critical real-time processing. If you need sub-100ms inference latency at high throughput (manufacturing quality inspection, real-time video analysis), network round-trips to cloud APIs add unacceptable delay. Local GPUs with optimized serving (vLLM, Triton) solve this.

Cost optimization at scale. If you are running millions of inference calls per day, the per-token cost of cloud APIs can exceed the amortized cost of owning GPUs. But this threshold is higher than most people think, and it keeps moving as cloud providers reduce pricing.

For these cases, we design and operate GPU node pools on our managed Kubernetes platform. The workloads run on the same platform, with the same observability, GitOps, and operational support as everything else.

How we approach it

We are not going to sell you GPU hardware you do not need yet. Our approach:

1. Start with the data layer. Get your RAG pipeline, vector database, and document processing running on Swiss infrastructure. This is where sovereignty actually matters and where you get immediate value.
2. Use cloud models pragmatically. For most use cases, cloud APIs are the fastest path to production. Evaluate data sensitivity per use case, not as a blanket policy.
3. Upgrade to local inference when the use case is concrete. When you know which model, which throughput, and which latency you need, we procure hardware and set up GPU compute on your managed cluster. We are transparent about costs so you can make an informed decision.
4. Explore the middle ground. Quantized models on existing GPUs, Apple Silicon for mid-range workloads, and optimized inference frameworks can cover more than you expect.

If you are early in your AI journey and wondering where to start, the answer is almost always: start with your data, not with your GPUs. Schedule a call and we will map out what makes sense for your specific situation.