y, i am just looking at changes and battling thru code with the of help of AI to see if it was C++ part’s fault or ggml.C.
Will test these changes of course. This is core feature.
From perspective of any Crystal bindings I don’t think the underlaying libraries’ code in llama.ccp are mature enough. IDK. Changing so fast.
Dealing with http API is much easiers and ultimately more scalable.
I am new in AI, and only more high level stuff. Nitty gritty is > me, so if !c++ do crystal. :-) end.