NeMo Switchyard is a new option in a growing field.
Model routing has emerged as a way to keep AI inferencing costs down by examining prompts and directing them to the most appropriate model — for example, the cheapest one able to respond effectively. This means that requests can be handled more efficiently, giving more accurate results and lower runtime costs, a concern in an era of rising AI expenditure.
Nvidia sees the potential for model routing and has just launched NeMo Switchyard, its take on the technology. Switchyard provides a library for applying multiple routing approaches, enabling developers to apply a system-of-models approach and build more efficient, controllable agents.
Interest in model routing is growing — both as a tool and as an investment: Cloudflare recently introduced a model router as part of a new suite of enterprise AI tools, while payment services company Stripe is looking to buy OpenRouter, according to The Wall Street Journal.
Companies will certainly be looking at routing options more carefully in the future as they work out how best to marry their infrastructures with their AI demands.
