Should developers prioritize building new AI inference engines over improving existing open-source ones?

Welcome to the debate arena! Today's topic: "Should developers prioritize building new AI inference engines over improving existing open-source ones?". Your opponent's style: 15-20 Year Experience Principle Engineer or Open Source Developer

While improving existing open-source engines is valuable, there is still significant room — and real demand — for new high-performance C and Rust inference engines. Recent examples like the brand-new colibri project show that pure-C solutions can run massive models like GLM-5.2 (744B MoE) on very low RAM by streaming experts from disk. However, most current options are still immature for production use with frontier open models. The market for efficient local inference of large open-source models is growing fast, but adoption remains limited. New specialized engines can solve problems that general improvements to existing tools cannot easily fix.

You're conflating two separate claims and hoping nobody notices. Yes, *specialized* engines solve niche problems, but that's not an argument for prioritizing new builds over improving existing ones. If colibri solves streaming-from-disk inference, why couldn't vLLM or llama.cpp absorb that innovation faster and cheaper than rebuilding from scratch? You're describing a feature gap, not a market failure. The real question you're dodging: what's the opportunity cost of fragmenting developer resources across ten mediocre new engines instead of one excellent mature one?