AI trends that need more attention

Over the past couple of weeks, there's been some news in the AI space that have underscored some of the things I think need most to happen for AI to become truly useful and not just another black box that we get overcharged for.

Now, I've touched before on what those things need to be. They need to be open models with ethical training which we can host on our own hardware and which handle the most specific possible use cases.

Now I'm thinking of another parameter brought up by the more recent news, and that's the freedom to not be dependent on any one particular set of model capacities or any one model, period.

And the thing that I came across recently that brought this up for me is a piece on the Next web about a company called N8N.

And I quote, the pitch for N8N is that the platform N8N itself sits between the models and everything a company already runs. So stitching large language models, deterministic code, and human sign off into workflows that survive contact with production.

What it deliberately does not do is make a model of its own. To continue quoting the design goal Oberhauser said was that a user should be able to own their data, self, host and avoid lock in no matter what happens in the outside world.

For large organizations, he argued that freedom is the whole appeal. So the idea here with this product is that you have a way to decouple whatever it is that you're doing from any particular model.

And that's pretty important given how token prices are jumping all over the place and how any commercial model hosted behind an API is at a risk of having its plug pulled at any moment.

So decoupling the work from any one particular model you use makes sense on a bunch of levels.

For one, you can take advantage of advances in the models themselves if there's some capability a newer model has, and you don't want to miss out on it because your workflow is tied to an older model. You can do that if you have enough separation of concern.

There's also another obvious advantage. If the model provider gets too expensive or unreliable, you can always switch. And third, that means you can make self hosting your models a much more viable option going forward.

If it doesn't matter where the model lives or who provides it, you can switch to a local version of it when the capacity is there, or when a svelte enough version of the model comes out to handle your use case on local hardware.

And on that note, I encountered some other news about another and I think underappreciated aspect of making it easier to take control of models and run them. The ability to make the models smaller.

This was a piece about an outfit named Eigen AI and they take open source models from the big providers and essentially compress them to run more economically without losing too much accuracy. Open models are routinely subjected to this kind of process. It's called quantization.

And many of your favorite models all exist in what are called lower quantization forms, smaller, more compact forms that make it possible to run them on commodity hardware and also to make them run faster on the same high end hardware.

The more efficiently that you can perform this compression without compromising the model's capabilities, the better.

And Eigen AI claims they have techniques that have made it appealing enough to do this to be bought out by one of the major European cloud companies, Nabius, the better to make it cheaper to run models at scale on their hardware and more of those models and faster.

Now the bad news about Eigen's approach is that it's proprietary. The only way you get to use models quantized through their system is through their services.

But that leaves a door open for other non proprietary ways to further enhance quantization techniques and to work more effectively with those giant multi billion parameter models that we're all getting hyped about.

Truth be told, I don't think the future is in efficient quantizations of big multi billion parameter models. I do think it is in models that are smaller by design because they're more focused.

But if you have several of those made even more efficient with good quantization, efficient enough to fit on local hardware, and you gang them together into a workflow that might turn out to be more powerful and efficient than a single monolithic does it all model.

And if you have something like N8N's decoupled approach to coordinating between them, that makes it even easier to generate a flexible workflow that isn't marry to any one model or provider or having bigger models automatically better ones.

These are good trends and I hope they continue and that's it for this episode. If you liked it, leave a comment below. And don't forget to follow Dev with Serdar and InfoWorld on Facebook, YouTube and InfoWorld.com.