Do you run the models locally? No local model for me manages to get function cal... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		okeuro49 3 months ago \| parent \| context \| favorite \| on: So you wanna build a local RAG? Do you run the models locally? No local model for me manages to get function calling right.

spacecadet 3 months ago [–]

Are you using LangGraph tool nodes? I run very small, non-RLHF instruction models with maybe a 2% failure rate on response format matching tool definition. I would also guess you do not have your orchestration and pipe configured correctly.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact