-
On the read axis alone, self-hosting a vector DB loses at 6.76 million vectors even at zero labor cost
The split between self-hosting a vector DB and buying dedicated nodes closes on one number, the allowable labor budget. L = P k r + W + A − (self-hosted infrastructure − s S), where P is the monthly rate per dedicated node, k the shard count, r the replica factor, W the managed write bill, A managed-only add-ons, and s S the managed storage bill. On this post's baseline, which sets W = 0 and A = 0, L goes negative past 6,764,154 vectors, so self-hosting loses even if you book zero monthly labor. On-demand and dedicated nodes split where corpus GB × QPS = 8.0 × k × r, which at $16 per RU is 0.40% × k × r of the on-demand default rate limit.
-
The 16.22% Reindex Break-Even Is Not Your Number
In RAG reindexing, the break-even between incremental and full reindexing is a rewritten-chunk share, and it closes into one formula, q* = (π + c + ρ) ÷ (π + 1 + ρ). π is parsing cost, c is full-reindex write cost, and ρ is embedding cost, each divided by incremental write cost, and all three come off vendor price lists. Depending on how the parse cache, the load path, and the embedding cache combine, that value lands anywhere from the 3% range to the 90% range.
-
A 31.6% Hit Rate Is Both a Win and a Loss
The break-even hit rate for prompt caching is not one number. At the same 31.6%, hits bunched into a session's early turns cost 14.4% more than caching only the stable prefix, while hits bunched into the late turns come in 28.3% under. Two axes move the threshold and a third makes it meaningless, so compare measured costs directly instead of converting to one.
-
Agent Caching Changes the Coefficient, Not the Exponent
Putting a breakpoint only on the stable prefix — the common setup — saves less as the agent gets longer. Caching the history reverses that, but it caps out at 90% and cost still grows with the square of the turn count. A 200-turn session costs $36 even when cached perfectly.
-
Self-Hosting a 70B Model Is Chartering a Bus - You Need a 25% Load Factor to Beat Taxis
Self-hosting is a fixed cost and the API is metered, so the deciding number is not monthly tokens but load factor. Add a realistic latency target and a standby node to a long-context workload and the required load factor passes 100%, at which point no amount of volume makes self-hosting cheaper.