### 🗣️ [#403](https://github.com/ikawrakow/ik_llama.cpp/discussions/403) - Tool Calling and Structured Response (Json Mode) support | **Author** | `mtcl` | | :--- | :--- | | **Created** | 2025-05-10 | | **Updated** | 2025-05-30 | --- #### Description Hey Team, Amazing work here. as compared to llama.cpp the biggest feature that I see missing is support for tool calling. D oyou have any plans to include it in the future roadmap? Or am i missing something and it alredy exists? I am forced to use other frameworks, even though i like inferencing speeds from ik_llama.cpp, just beacuse i cant live without these features and want to swap it out natively in the openai's python client in my project implementation. I know tha i can prompt the model in a particular way to force it to produce a json response. I am not looking for that. Thank you in advance! --- #### 🗣️ Discussion 👤 **ikawrakow** replied the **2025-05-10** at **08:30:16**:
Hey @mtcl, we are a very small team, so cannot do everything that `llama.cpp` does. Hence, the strategy is to focus on few things, but do these things really well. Please enter a feature request in the Issues. I'll label it with "help wanted" and we will see what happens. > 👤 **mtcl** replied the **2025-05-10** at **08:33:02**:
> No worries my friend. I have a workaround here that I've written. > > https://github.com/Teachings/FastAgentAPI > > It acts as a wrapper and get me by. Thank you for your hard work! > > 👤 **cmoncure** replied the **2025-05-30** at **19:58:13**:
> Before I try and get this running, can you educate me on the mechanics of tool calling within the LLM response? I understand that the LLM may request a call as part of its TG phase, and then the call runner injects the result into the LLM response. Is this correct? > > I have some questions about this. Suppose I want to ask the LLM a question about a long document. > > What's the difference in outcome between: > 1) Including the question and document in the prompt, and enduring the long PP time > 2) Including the question in the prompt, and having the LLM retrieve the document instantly via tool call during TG, then going on to complete the response? > > Do all injected tokens need to undergo a form of 'PP during TG'? That would make the most sense, actually... --- 👤 **KCS-Mack** replied the **2025-05-18** at **22:28:59**:
This is great, will give it a try!