Try an interactive version of this dialog: Sign up at solve.it.com, click Upload, and pass this URL.

Note: 84

Lisette Tool loop

This notebook is to develop from scratch the chat class used in lisette. Here is my plan:

  • Research and understand Lisette
  • Impliment basic tool use with an llm chat in Lisette
  • Re-create chat class from scratch

Code: 28 () 

from fastcore.utils import *
from dialoghelper import *
from IPython.display import HTML,Markdown

Code: 7 () 

from lisette import *

Code: 6 () 

fc_tool_info()

Code: 9 () 

from fastcore.tools import *

Note: 253

Tools available from fastcore.tools:

  • &rg: Run the rg command with the args in argstr (no need to backslash escape)
  • &sed: Run the sed command with the args in argstr (e.g for reading a section of a file)
  • &view: View directory or file contents with optional line range and numbers
  • &create: Creates a new file with the given content at the specified path
  • &insert: Insert new_str at specified line number
  • &str_replace: Replace first occurrence of old_str with new_str in file
  • &strs_replace: Replace for each str pair in old_strs,new_strs
  • &replace_lines: Replace lines in file using start and end line-numbers

Code: 24 () 

%cd /usr/local/lib/python3.12/site-packages/lisette

Output: 36 

/usr/local/lib/python3.12/site-packages/lisette

Prompt: 114

I want to learn how lisette's tool loop works. Can you please read the documentation at https://lisette.answer.ai/core.html to see how to use it, and then use your tools to investigate the source in the current directory and show me how it's all put together? As you use your tools, detail what you're doing and exactly what you find out along the way.

Assistant: 18,187

I'd be happy to help you explore how lisette's tool loop works! Let me start by reading the documentation, then we'll investigate the source code together.

Let me first check out the documentation:

Copied!
{
  "id": "toolu_013aAWPEwPhdt9XAiZTQKm4m",
  "call": {
    "function": "read_url",
    "arguments": {
      "url": "https://lisette.answer.ai/core.html"
    }
  },
  "result": "'[ ](./index.html) [ lisette ](./index.html)\\n\\n__\\n\\n  1. [Core](./core.html)\\n\\n[ ](./index.html)\\n\\n  * [ Lisette](./index.html)\\n\\n  * [ Core](./core.html)\\n\\n## On this page\\n\\n  * LiteLLM\\n    * Deterministic outputs\\n      * patch_litellm\\n    * Completion\\n    * Messages formatting\\n      * remove_cache_ckpts\\n      * mk_msg\\n      * Caching\\n      * Reconstructing formatted outputs\\n      * fmt2hist\\n      * `mk_msgs`\\n      * mk_msgs\\n    * Streaming\\n      * stream_with_complete\\n    * Tools\\n      * lite_mk_func\\n      * ToolResponse\\n    * Search\\n    * Citations\\n      * cite_footnotes\\n      * cite_footnote\\n  * Chat\\n    * Chat\\n    * Examples\\n      * History tracking\\n      * Synthetic History Creation\\n      * Chat.print_hist\\n      * random_tool_id\\n      * mk_tc\\n      * mk_tc_req\\n      * mk_tc_result\\n      * mk_tc_results\\n      * Images\\n      * Prefill\\n      * Streaming\\n      * Tool use\\n      * Thinking w tool use\\n      * Search\\n      * Multi tool calling\\n      * Tool call exhaustion\\n    * Async\\n      * AsyncChat\\n      * astream_with_complete\\n      * AsyncChat\\n      * Examples\\n    * Async Streaming Display\\n      * mk_tr_details\\n      * AsyncStreamFormatter\\n      * adisplay_stream\\n    * Streaming examples\\n      * Tool call\\n      * Thinking tool call\\n      * Multiple tool calls\\n      * Search\\n      * Caching\\n\\n  * [__Report an issue](https://github.com/AnswerDotAI/lisette/issues/new)\\n\\n## Other Formats\\n\\n  * [ __CommonMark](core.html.md)\\n\\n# Core\\n\\nLisette Core\\n\\n# LiteLLM\\n\\n## Deterministic outputs\\n\\nLiteLLM `ModelResponse(Stream)` objects have `id` and `created_at` fields that are generated dynamically. Even when we use [`cachy`](https://github.com/answerdotai/cachy) to cache the LLM response these dynamic fields create diffs which makes code review more challenging. The patches below ensure that `id` and `created_at` fields are fixed and won\u2019t generate diffs.\\n\\n* * *\\n\\n[source](https://github.com/AnswerDotAI/lisette/blob/main/lisette/core.py#L24)\\n\\n### patch_litellm\\n\\n>\\n```\\n>      patch_litellm (seed=0)\\n```\\n\\n_Patch litellm.ModelResponseBase such that`id` and `created` are fixed._\\n\\n```\\npatch_litellm()\\n```\\n\\n__\\n\\n## Completion\\n\\nLiteLLM provides an convenient unified interface for most big LLM providers. Because it\u2019s so useful to be able to switch LLM providers with just one argument. We want to make it even easier to by adding some more convenience functions and classes.\\n\\nThis is very similar to our other wrapper libraries for popular AI providers: [claudette](https://claudette.answer.ai/) (Anthropic), [gaspard](https://github.com/AnswerDotAI/gaspard) (Gemini), [cosette](https://answerdotai.github.io/cosette/) (OpenAI).\\n\\n```\\n# litellm._turn_on_debug()\\n```\\n\\n__\\n\\n```\\nms = [\"gemini/gemini-2.5-flash\", \"claude-sonnet-4-5\", \"openai/gpt-4.1\"]\\nmsg = [{\\'role\\':\\'user\\',\\'content\\':\\'Hey there!\\', \\'cache_control\\': {\\'type\\': \\'ephemeral\\'}}]\\nfor m in ms:\\n    display(Markdown(f\\'**{m}:**\\'))\\n    display(completion(m,msg))\\n```\\n\\n__\\n\\n**gemini/gemini-2.5-flash:**\\n\\nHey there! How can I help you today?\\n\\n  * id: `chatcmpl-xxx`\\n  * model: `gemini-2.5-flash`\\n  * finish_reason: `stop`\\n  * usage: `Usage(completion_tokens=153, prompt_tokens=4, total_tokens=157, completion_tokens_details=CompletionTokensDetailsWrapper(accepted_prediction_tokens=None, audio_tokens=None, reasoning_tokens=143, rejected_prediction_tokens=None, text_tokens=10), prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=None, text_tokens=4, image_tokens=None))`\\n\\n**claude-sonnet-4-5:**\\n\\nHello! How can I help you today?\\n\\n  * id: `chatcmpl-xxx`\\n  * model: `claude-sonnet-4-5-20250929`\\n  * finish_reason: `stop`\\n  * usage: `Usage(completion_tokens=12, prompt_tokens=10, total_tokens=22, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None), cache_creation_input_tokens=0, cache_read_input_tokens=0)`\\n\\n**openai/gpt-4.1:**\\n\\nHello! How can I help you today? \ud83d\ude0a\\n\\n  * id: `chatcmpl-xxx`\\n  * model: `gpt-4.1-2025-04-14`\\n  * finish_reason: `stop`\\n  * usage: `Usage(completion_tokens=10, prompt_tokens=10, total_tokens=20, completion_tokens_details=CompletionTokensDetailsWrapper(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0, text_tokens=None), prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=0, cached_tokens=0, text_tokens=None, image_tokens=None))`\\n\\n## Messages formatting\\n\\nLet\u2019s start with making it easier to pass messages into litellm\u2019s `completion` function (including images, and pdf files).\\n\\n* * *\\n\\n[source](https://github.com/AnswerDotAI/lisette/blob/main/lisette/core.py#L99)\\n\\n### remove_cache_ckpts\\n\\n>\\n```\\n>      remove_cache_ckpts (msg)\\n```\\n\\n_remove cache checkpoints and return msg._\\n\\n* * *\\n\\n[source](https://github.com/AnswerDotAI/lisette/blob/main/lisette/core.py#L110)\\n\\n### mk_msg\\n\\n>\\n```\\n>      mk_msg (content, role=\\'user\\', cache=False, ttl=None)\\n```\\n\\n_Create a LiteLLM compatible message._\\n\\n| **Type** | **Default** | **Details**  \\n---|---|---|---  \\ncontent |  |  | Content: str, bytes (image), list of mixed content, or dict w \u2018role\u2019 and \u2018content\u2019 fields  \\nrole | str | user | Message role if content isn\u2019t already a dict/Message  \\ncache | bool | False | Enable Anthropic caching  \\nttl | NoneType | None | Cache TTL: \u20185m\u2019 (default) or \u20181h\u2019  \\n  \\nNow we can use mk_msg to create different types of messages.\\n\\nSimple text:\\n\\n```\\nmsg = mk_msg(\"hey\")\\nmsg\\n```\\n\\n __\\n\\n```\\n{\\'role\\': \\'user\\', \\'content\\': \\'hey\\'}\\n```\\n\\nWhich can be passed to litellm\u2019s `completion` function like this:\\n\\n```\\nmodel = ms[1]\\n```\\n\\n__\\n\\n```\\nres = completion(model, [msg])\\nres\\n```\\n\\n __\\n\\nHey! How\u2019s it going? What\u2019s on your mind?\\n\\n  * id: `chatcmpl-xxx`\\n  * model: `claude-sonnet-4-5-20250929`\\n  * finish_reason: `stop`\\n  * usage: `Usage(completion_tokens=16, prompt_tokens=8, total_tokens=24, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None), cache_creation_input_tokens=0, cache_read_input_tokens=0)`\\n\\nWe\u2019ll add a little shortcut to make examples and testing easier here:\\n\\n```\\ndef c(msgs, **kw):\\n    msgs = [msgs] if isinstance(msgs,dict) else listify(msgs)\\n    return completion(model, msgs, **kw)\\n```\\n\\n__\\n\\n```\\nc(msg)\\n```\\n\\n__\\n\\nHey! How\u2019s it going? What\u2019s on your mind?\\n\\n  * id: `chatcmpl-xxx`\\n  * model: `claude-sonnet-4-5-20250929`\\n  * finish_reason: `stop`\\n  * usage: `Usage(completion_tokens=16, prompt_tokens=8, total_tokens=24, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None), cache_creation_input_tokens=0, cache_read_input_tokens=0)`\\n\\nLists w just one string element are flattened for conciseness:\\n\\n```\\ntest_eq(mk_msg(\"hey\"), mk_msg([\"hey\"]))\\n```\\n\\n__\\n\\n(LiteLLM ignores these fields when sent to other providers)\\n\\nText and images:\\n\\n```\\nimg_fn = Path(\\'samples/puppy.jpg\\')\\nImage(filename=img_fn, width=200)\\n```\\n\\n__\\n\\n![](00_core_files/figure-html/cell-14-output-1.jpeg)\\n\\n```\\nmsg = mk_msg([\\'hey what in this image?\\',img_fn.read_bytes()])\\nprint(json.dumps(msg,indent=1)[:200]+\"...\")\\n```\\n\\n__\\n\\n```\\n{\\n \"role\": \"user\",\\n \"content\": [\\n  {\\n   \"type\": \"text\",\\n   \"text\": \"hey what in this image?\"\\n  },\\n  {\\n   \"type\": \"image_url\",\\n   \"image_url\": \"data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/4gxUSU...\\n```\\n\\n```\\nc(msg)\\n```\\n\\n__\\n\\nThis image shows an adorable **Cavalier King Charles Spaniel puppy**! The puppy has the breed\u2019s characteristic features:\\n\\n  * **Coloring** : Brown (chestnut) and white coat\\n  * **Sweet expression** : Large, dark eyes and a gentle face\\n  * **Setting** : The puppy is lying on grass near some purple flowers (appear to be asters or similar blooms)\\n\\nThe puppy looks very young and has that irresistibly cute, innocent look that Cavalier puppies are famous for. The photo has a professional quality with nice lighting and composition, capturing the puppy\u2019s endearing personality perfectly!\\n\\n  * id: `chatcmpl-xxx`\\n  * model: `claude-sonnet-4-5-20250929`\\n  * finish_reason: `stop`\\n  * usage: `Usage(completion_tokens=139, prompt_tokens=104, total_tokens=243, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None), cache_creation_input_tokens=0, cache_read_input_tokens=0)`\\n\\nLet\u2019s also demonstrate this for PDFs\\n\\n```\\npdf_fn = Path(\\'samples/solveit.pdf\\')\\nmsg = mk_msg([\\'Who is the author of this pdf?\\', pdf_fn.read_bytes()])\\nc(msg)\\n```\\n\\n__\\n\\nThe author of this PDF is **Jeremy Howard** from fast.ai. He explicitly introduces himself in the document with \u201cHi, I\u2019m Jeremy Howard, from fast.ai\u201d and goes on to describe his work co-founding fast.ai with Rachel Thomas eight years ago.\\n\\n  * id: `chatcmpl-xxx`\\n  * model: `claude-sonnet-4-5-20250929`\\n  * finish_reason: `stop`\\n  * usage: `Usage(completion_tokens=59, prompt_tokens=1610, total_tokens=1669, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None), cache_creation_input_tokens=0, cache_read_input_tokens=0)`\\n\\n### Caching\\n\\nSome providers such as Anthropic require manually opting into caching. Let\u2019s try it:\\n\\n```\\ndef cpr(i): return f\\'{i} \\'*1024 + \\'This is a caching test. Report back only what number you see repeated above.\\'\\n```\\n\\n__\\n\\n```\\ndisable_cachy()\\n```\\n\\n__\\n\\n```\\nmsg = mk_msg(cpr(1), cache=True)\\nres = c(msg)\\nres\\n```\\n\\n __\\n\\n1\\n\\n  * id: `chatcmpl-xxx`\\n  * model: `claude-sonnet-4-5-20250929`\\n  * finish_reason: `stop`\\n  * usage: `Usage(completion_tokens=5, prompt_tokens=3, total_tokens=8, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None), cache_creation_input_tokens=2070, cache_read_input_tokens=0)`\\n\\nAnthropic has a maximum of 4 cache checkpoints, so we remove previous ones as we go:\\n\\n```\\nres = c([remove_cache_ckpts(msg), mk_msg(res), mk_msg(cpr(2), cache=True)])\\nres\\n```\\n\\n __\\n\\n2\\n\\n  * id: `chatcmpl-xxx`\\n  * model: `claude-sonnet-4-5-20250929`\\n  * finish_reason: `stop`\\n  * usage: `Usage(completion_tokens=5, prompt_tokens=2073, total_tokens=2078, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=2070, text_tokens=None, image_tokens=None), cache_creation_input_tokens=2074, cache_read_input_tokens=2070)`\\n\\nWe see that the first message was cached, and this extra message has been written to cache:\\n\\n```\\nres.usage.prompt_tokens_details\\n```\\n\\n __\\n\\n```\\nPromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=2070, text_tokens=None, image_tokens=None)\\n```\\n\\nWe can add a bunch of large messages in a loop to see how the number of cached tokens used grows.\\n\\nWe do this for 25 times to ensure it still works for more than >20 content blocks, [which is a known anthropic issue](https://docs.claude.com/en/docs/build-with-claude/prompt-caching).\\n\\nThe code below is commented by default, because it\u2019s slow. Please uncomment when working on caching.\\n\\n```\\n# h = []\\n# msg = mk_msg(cpr(1), cache=True)\\n\\n# for o in range(2,25):\\n#     h += [remove_cache_ckpts(msg), mk_msg(res)]\\n#     msg = mk_msg(cpr(o), cache=True)\\n#     res = c(h+[msg])\\n#     detls = res.usage.prompt_tokens_details\\n#     print(o, detls.cached_tokens, detls.cache_creation_tokens, end=\\'; \\')\\n```\\n\\n__\\n\\n```\\nenable_cachy()\\n```\\n\\n__\\n\\n### Reconstructing formatted outputs\\n\\nLisette can call multiple tools in a loop. Further down this notebook, we\u2019ll provide convenience functions for formatting such a sequence of toolcalls and responses into one formatted output string.\\n\\nFor now, we\u2019ll show an example and show how to transform such a formatted output string back into a valid LiteLLM history.\\n\\n```\\nfmt_outp = \\'\\'\\'\\nI\\'ll solve this step-by-step, using parallel calls where possible.\\n\\n<details class=\\'tool-usage-details\\'>\\n\\n```json\\n{\\n  \"id\": \"toolu_01KjnQH2Nsz2viQ7XYpLW3Ta\",\\n  \"call\": { \"function\": \"simple_add\", \"arguments\": { \"a\": 10, \"b\": 5 } },\\n  \"result\": \"15\"\\n}\\n```\\n\\n</details>\\n\\n<details class=\\'tool-usage-details\\'>\\n\\n```json\\n{\\n  \"id\": \"toolu_01Koi2EZrGZsBbnQ13wuuvzY\",\\n  \"call\": { \"function\": \"simple_add\", \"arguments\": { \"a\": 2, \"b\": 1 } },\\n  \"result\": \"3\"\\n}\\n```\\n\\n</details>\\n\\nNow I need to multiply 15 * 3 before I can do the final division:\\n\\n<details class=\\'tool-usage-details\\'>\\n\\n```json\\n{\\n  \"id\": \"toolu_0141NRaWUjmGtwxZjWkyiq6C\",\\n  \"call\": { \"function\": \"multiply\", \"arguments\": { \"a\": 15, \"b\": 3 } },\\n  \"result\": \"45\"\\n}\\n```\\n\\n</details>\\n\\'\\'\\'\\n```\\n\\n__\\n\\nWe can split into chunks of (text,toolstr,json):\\n\\n```\\nsp = re_tools.split(fmt_outp)\\nfor o in list(chunked(sp, 3, pad=True)): print(\\'- \\', o)\\n```\\n\\n__\\n\\n```\\n-  [\"\\\\nI\\'ll solve this step-by-step, using parallel calls where possible.\\\\n\\\\n\", \\'<details class=\\\\\\'tool-usage-details\\\\\\'>\\\\n\\\\n```json\\\\n{\\\\n  \"id\": \"toolu_01KjnQH2Nsz2viQ7XYpLW3Ta\",\\\\n  \"call\": { \"function\": \"simple_add\", \"arguments\": { \"a\": 10, \"b\": 5 } },\\\\n  \"result\": \"15\"\\\\n}\\\\n```\\\\n\\\\n</details>\\', \\'{\\\\n  \"id\": \"toolu_01KjnQH2Nsz2viQ7XYpLW3Ta\",\\\\n  \"call\": { \"function\": \"simple_add\", \"arguments\": { \"a\": 10, \"b\": 5 } },\\\\n  \"result\": \"15\"\\\\n}\\']\\n-  [\\'\\\\n\\\\n\\', \\'<details class=\\\\\\'tool-usage-details\\\\\\'>\\\\n\\\\n```json\\\\n{\\\\n  \"id\": \"toolu_01Koi2EZrGZsBbnQ13wuuvzY\",\\\\n  \"call\": { \"function\": \"simple_add\", \"arguments\": { \"a\": 2, \"b\": 1 } },\\\\n  \"result\": \"3\"\\\\n}\\\\n```\\\\n\\\\n</details>\\', \\'{\\\\n  \"id\": \"toolu_01Koi2EZrGZsBbnQ13wuuvzY\",\\\\n  \"call\": { \"function\": \"simple_add\", \"arguments\": { \"a\": 2, \"b\": 1 } },\\\\n  \"result\": \"3\"\\\\n}\\']\\n-  [\\'\\\\n\\\\nNow I need to multiply 15 * 3 before I can do the final division:\\\\n\\\\n\\', \\'<details class=\\\\\\'tool-usage-details\\\\\\'>\\\\n\\\\n```json\\\\n{\\\\n  \"id\": \"toolu_0141NRaWUjmGtwxZjWkyiq6C\",\\\\n  \"call\": { \"function\": \"multiply\", \"arguments\": { \"a\": 15, \"b\": 3 } },\\\\n  \"result\": \"45\"\\\\n}\\\\n```\\\\n\\\\n</details>\\', \\'{\\\\n  \"id\": \"toolu_0141NRaWUjmGtwxZjWkyiq6C\",\\\\n  \"call\": { \"function\": \"multiply\", \"arguments\": { \"a\": 15, \"b\": 3 } },\\\\n  \"result\": \"45\"\\\\n}\\']\\n-  [\\'\\\\n\\', None, None]\\n```\\n\\n* * *\\n\\n[source](https://github.com/AnswerDotAI/lisette/blob/main/lisette/core.py#L139)\\n\\n### fmt2hist\\n\\n>\\n```\\n>      fmt2hist (outp:str)\\n```\\n\\n_Transform a formatted output into a LiteLLM compatible history_\\n\\nSee how we can turn that one formatted output string back into a list of Messages:\\n\\n```\\nfrom pprint import pprint\\n```\\n\\n __\\n\\n```\\nh = fmt2hist(fmt_outp)\\npprint(h)\\n```\\n\\n__\\n\\n```\\n[Message(content=\"I\\'ll solve this step-by-step, using parallel calls where possible.\", role=\\'assistant\\', tool_calls=[ChatCompletionMessageToolCall(function=Function(arguments=\\'{\"a\":10,\"b\":5}\\', name=\\'simple_add\\'), id=\\'toolu_01KjnQH2Nsz2viQ7XYpLW3Ta\\', type=\\'function\\')], function_call=None, provider_specific_fields=None),\\n {\\'content\\': \\'15\\',\\n  \\'name\\': \\'simple_add\\',\\n  \\'role\\': \\'tool\\',\\n  \\'tool_call_id\\': \\'toolu_01KjnQH2Nsz2viQ7XYpLW3Ta\\'},\\n Message(content=\\'\\', role=\\'assistant\\', tool_calls=[ChatCompletionMessageToolCall(function=Function(arguments=\\'{\"a\":2,\"b\":1}\\', name=\\'simple_add\\'), id=\\'toolu_01Koi2EZrGZsBbnQ13wuuvzY\\', type=\\'function\\')], function_call=None, provider_specific_fields=None),\\n {\\'content\\': \\'3\\',\\n  \\'name\\': \\'simple_add\\',\\n  \\'role\\': \\'tool\\',\\n  \\'tool_call_id\\': \\'toolu_01Koi2EZrGZsBbnQ13wuuvzY\\'},\\n Message(content=\\'Now I need to multiply 15 * 3 before I can do the final division:\\', role=\\'assistant\\', tool_calls=[ChatCompletionMessageToolCall(function=Function(arguments=\\'{\"a\":15,\"b\":3}\\', name=\\'multiply\\'), id=\\'toolu_0141NRaWUjmGtwxZjWkyiq6C\\', type=\\'function\\')], function_call=None, provider_specific_fields=None),\\n {\\'content\\': \\'45\\',\\n  \\'name\\': \\'multiply\\',\\n  \\'role\\': \\'tool\\',\\n  \\'tool_call_id\\': \\'toolu_0141NRaWUjmGtwxZjWkyiq6C\\'},\\n Message(content=\\'.\\', role=\\'assistant\\', tool_calls=None, function_call=None, provider_specific_fields=None)]\\n```\\n\\n### [`mk_msgs`](https://lisette.answer.ai/core.html#mk_msgs)\\n\\nWe will skip tool use blocks and tool results during caching\\n\\nNow lets make it easy to provide entire conversations:\\n\\n* * *\\n\\n[source](https://github.com/AnswerDotAI/lisette/blob/main/lisette/core.py#L162)\\n\\n### mk_msgs\\n\\n>\\n```\\n>      mk_msgs (msgs, cache=False, cache_idxs=[-1], ttl=None)\\n```\\n\\n_Create a list of LiteLLM compatible messages._\\n\\n| **Type** | **Default** | **Details**  \\n---|---|---|---  \\nmsgs |  |  | List of messages (each: str, bytes, list, or dict w \u2018role\u2019 and \u2018content\u2019 fields)  \\ncache | bool | False | Enable Anthropic caching  \\ncache_idxs | list | [-1] | Cache breakpoint idxs  \\nttl | NoneType | None | Cache TTL: \u20185m\u2019 (default) or \u20181h\u2019  \\n  \\nWith [`mk_msgs`](https://lisette.answer.ai/core.html#mk_msgs) you can easily provide a whole conversation:\\n\\n```\\nmsgs = mk_msgs([\\'Hey!\\',\"Hi there!\",\"How are you?\",\"I\\'m doing fine and you?\"])\\nmsgs\\n```\\n\\n __\\n\\n```\\n[{\\'role\\': \\'user\\', \\'content\\': \\'Hey!\\'},\\n {\\'role\\': \\'assistant\\', \\'content\\': \\'Hi there!\\'},\\n {\\'role\\': \\'user\\', \\'content\\': \\'How are you?\\'},\\n {\\'role\\': \\'assistant\\', \\'content\\': \"I\\'m doing fine and you?\"}]\\n```\\n\\nBy defualt the last message will be cached when `cache=True`:\\n\\n```\\nmsgs = mk_msgs([\\'Hey!\\',\"Hi there!\",\"How are you?\",\"I\\'m doing fine and you?\"], cache=True)\\nmsgs\\n```\\n\\n __\\n\\n```\\n[{\\'role\\': \\'user\\', \\'content\\': \\'Hey!\\'},\\n {\\'role\\': \\'assistant\\', \\'content\\': \\'Hi there!\\'},\\n {\\'role\\': \\'user\\', \\'content\\': \\'How are you?\\'},\\n {\\'role\\': \\'assistant\\',\\n  \\'content\\': [{\\'type\\': \\'text\\',\\n    \\'text\\': \"I\\'m doing fine and you?\",\\n    \\'cache_control\\': {\\'type\\': \\'ephemeral\\'}}]}]\\n```\\n\\n```\\ntest_eq(\\'cache_control\\' in msgs[-1][\\'content\\'][0], True)\\n```\\n\\n__\\n\\nAlternatively, users can provide custom `cache_idxs`. Tool call blocks and results are skipped during caching:\\n\\n```\\nmsgs = mk_msgs([\\'Hello!\\',\\'Hi! How can I help you?\\',\\'Call some functions!\\',fmt_outp], cache=True, cache_idxs=[0,-2,-1])\\nmsgs\\n```\\n\\n __\\n\\n```\\n[{\\'role\\': \\'user\\',\\n  \\'content\\': [{\\'type\\': \\'text\\',\\n    \\'text\\': \\'Hello!\\',\\n    \\'cache_control\\': {\\'type\\': \\'ephemeral\\'}}]},\\n {\\'role\\': \\'assistant\\', \\'content\\': \\'Hi! How can I help you?\\'},\\n {\\'role\\': \\'user\\',\\n  \\'content\\': [{\\'type\\': \\'text\\',\\n    \\'text\\': \\'Call some functions!\\',\\n    \\'cache_control\\': {\\'type\\': \\'ephemeral\\'}}]},\\n Message(content=\"I\\'ll solve this step-by-step, using parallel calls where possible.\", role=\\'assistant\\', tool_calls=[ChatCompletionMessageToolCall(function=Function(arguments=\\'{\"a\":10,\"b\":5}\\', name=\\'simple_add\\'), id=\\'toolu_01KjnQH2Nsz2viQ7XYpLW3Ta\\', type=\\'function\\')], function_call=None, provider_specific_fields=None),\\n {\\'role\\': \\'tool\\',\\n  \\'tool_call_id\\': \\'toolu_01KjnQH2Nsz2viQ7XYpLW3Ta\\',\\n  \\'name\\': \\'simple_add\\',\\n  \\'content\\': \\'15\\'},\\n Message(content=\\'\\', role=\\'assistant\\', tool_calls=[ChatCompletionMessageToolCall(function=Function(arguments=\\'{\"a\":2,\"b\":1}\\', name=\\'simple_add\\'), id=\\'toolu_01Koi2EZrGZsBbnQ13wuuvzY\\', type=\\'function\\')], function_call=None, provider_specific_fields=None),\\n {\\'role\\': \\'tool\\',\\n  \\'tool_call_id\\': \\'toolu_01Koi2EZrGZsBbnQ13wuuvzY\\',\\n  \\'name\\': \\'simple_add\\',\\n  \\'content\\': \\'3\\'},\\n Message(content=\\'Now I need to multiply 15 * 3 before I can do the final division:\\', role=\\'assistant\\', tool_calls=[ChatCompletionMessageToolCall(function=Function(arguments=\\'{\"a\":15,\"b\":3}\\', name=\\'multiply\\'), id=\\'toolu_0141NRaWUjmGtwxZjWkyiq6C\\', type=\\'function\\')], function_call=No<TRUNCATED>"
}

Great! Now I can see the documentation. Let me explore the source code to understand how the tool loop works. First, let me see what files are in the current directory:

Copied!
{
  "id": "toolu_01BmANHMXAZnZGknU6u6nyP8",
  "call": {
    "function": "view",
    "arguments": {
      "path": "."
    }
  },
  "result": "'Directory contents of /usr/local/lib/python3.12/site-packages/lisette:\\n/usr/local/lib/python3.12/site-packages/lisette/core.py\\n/usr/local/lib/python3.12/site-packages/lisette/__init__.py\\n/usr/local/lib/python3.12/site-packages/lisette/_modidx.py\\n/usr/local/lib/python3.12/site-packages/lisette/__pycache__\\n/usr/local/lib/python3.12/site-packages/lisette/__pycache__/core.cpython-312.pyc\\n/usr/local/lib/python3.12/site-packages/lisette/__pycache__/_modidx.cpython-312.pyc\\n/usr/local/lib/python3.12/site-packages/lisette/__pycache__/__init__.cpython-312.pyc'"
}

The main code is in core.py. From the documentation, I can see that the Chat class is the key component that handles tool loops with the max_steps parameter. Let me look at the Chat class implementation:

Copied!
{
  "id": "toolu_01MgZeytiEAcvCxsuLb5HQzt",
  "call": {
    "function": "rg",
    "arguments": {
      "argstr": "-n \"^class Chat\""
    }
  },
  "result": "'core.py:229:class Chat:\\n'"
}

Now let me view the Chat class to see its structure:

Copied!
{
  "id": "toolu_01JTL75kBNiMpZFz3AKxzWJt",
  "call": {
    "function": "view",
    "arguments": {
      "path": "core.py",
      "view_range": "[229, 320]",
      "nums": "True"
    }
  },
  "result": "'   229 \u2502 class Chat:\\n   230 \u2502     def __init__(\\n   231 \u2502         self,\\n   232 \u2502         model:str,                # LiteLLM compatible model name \\n   233 \u2502         sp=\\'\\',                    # System prompt\\n   234 \u2502         temp=0,                   # Temperature\\n   235 \u2502         search=False,             # Search (l,m,h), if model supports it\\n   236 \u2502         tools:list=None,          # Add tools\\n   237 \u2502         hist:list=None,           # Chat history\\n   238 \u2502         ns:Optional[dict]=None,   # Custom namespace for tool calling \\n   239 \u2502         cache=False,              # Anthropic prompt caching\\n   240 \u2502         cache_idxs:list=[-1],     # Anthropic cache breakpoint idxs, use `0` for sys prompt if provided\\n   241 \u2502         ttl=None,                 # Anthropic prompt caching ttl\\n   242 \u2502     ):\\n   243 \u2502         \"LiteLLM chat client.\"\\n   244 \u2502         self.model = model\\n   245 \u2502         hist,tools = mk_msgs(hist,cache,cache_idxs,ttl),listify(tools)\\n   246 \u2502         if ns is None and tools: ns = mk_ns(tools)\\n   247 \u2502         elif ns is None: ns = globals()\\n   248 \u2502         self.tool_schemas = [lite_mk_func(t) for t in tools] if tools else None\\n   249 \u2502         store_attr()\\n   250 \u2502     \\n   251 \u2502     def _prep_msg(self, msg=None, prefill=None):\\n   252 \u2502         \"Prepare the messages list for the API call\"\\n   253 \u2502         sp = [{\"role\": \"system\", \"content\": self.sp}] if self.sp else []\\n   254 \u2502         if sp:\\n   255 \u2502             if 0 in self.cache_idxs: sp[0] = _add_cache_control(sp[0])\\n   256 \u2502             cache_idxs = L(self.cache_idxs).filter().map(lambda o: o-1 if o>0 else o)\\n   257 \u2502         else:\\n   258 \u2502             cache_idxs = self.cache_idxs\\n   259 \u2502         if msg: self.hist = mk_msgs(self.hist+[msg], self.cache, cache_idxs, self.ttl)\\n   260 \u2502         pf = [{\"role\":\"assistant\",\"content\":prefill}] if prefill else []\\n   261 \u2502         return sp + self.hist + pf\\n   262 \u2502 \\n   263 \u2502     def _call(self, msg=None, prefill=None, temp=None, think=None, search=None, stream=False, max_steps=2, step=1, final_prompt=None, tool_choice=None, **kwargs):\\n   264 \u2502         \"Internal method that always yields responses\"\\n   265 \u2502         if step>max_steps: return\\n   266 \u2502         if not get_model_info(self.model).get(\"supports_assistant_prefill\"): prefill=None\\n   267 \u2502         if _has_search(self.model) and (s:=ifnone(search,self.search)): kwargs[\\'web_search_options\\'] = {\"search_context_size\": effort[s]}\\n   268 \u2502         else: _=kwargs.pop(\\'web_search_options\\',None)\\n   269 \u2502         res = completion(model=self.model, messages=self._prep_msg(msg, prefill), stream=stream, \\n   270 \u2502                          tools=self.tool_schemas, reasoning_effort = effort.get(think), tool_choice=tool_choice,\\n   271 \u2502                          # temperature is not supported when reasoning\\n   272 \u2502                          temperature=None if think else ifnone(temp,self.temp),\\n   273 \u2502                          **kwargs)\\n   274 \u2502         if stream:\\n   275 \u2502             if prefill: yield _mk_prefill(prefill)\\n   276 \u2502             res = yield from stream_with_complete(res,postproc=cite_footnotes)\\n   277 \u2502         m = res.choices[0].message\\n   278 \u2502         if prefill: m.content = prefill + m.content\\n   279 \u2502         self.hist.append(m)\\n   280 \u2502         yield res\\n   281 \u2502 \\n   282 \u2502         if tcs := m.tool_calls:\\n   283 \u2502             tool_results=[_lite_call_func(tc, ns=self.ns) for tc in tcs]\\n   284 \u2502             self.hist+=tool_results\\n   285 \u2502             for r in tool_results: yield r\\n   286 \u2502             if step>=max_steps-1: prompt,tool_choice,search = final_prompt,\\'none\\',False\\n   287 \u2502             else: prompt = None\\n   288 \u2502             yield from self._call(\\n   289 \u2502                 prompt, prefill, temp, think, search, stream, max_steps, step+1,\\n   290 \u2502                 final_prompt, tool_choice, **kwargs)\\n   291 \u2502     \\n   292 \u2502     def __call__(self,\\n   293 \u2502                  msg=None,          # Message str, or list of multiple message parts\\n   294 \u2502                  prefill=None,      # Prefill AI response if model supports it\\n   295 \u2502                  temp=None,         # Override temp set on chat initialization\\n   296 \u2502                  think=None,        # Thinking (l,m,h)\\n   297 \u2502                  search=None,       # Override search set on chat initialization (l,m,h)\\n   298 \u2502                  stream=False,      # Stream results\\n   299 \u2502                  max_steps=2, # Maximum number of tool calls\\n   300 \u2502                  final_prompt=_final_prompt, # Final prompt when tool calls have ran out \\n   301 \u2502                  return_all=False,  # Returns all intermediate ModelResponses if not streaming and has tool calls\\n   302 \u2502                  **kwargs):\\n   303 \u2502         \"Main call method - handles streaming vs non-streaming\"\\n   304 \u2502         result_gen = self._call(msg, prefill, temp, think, search, stream, max_steps, 1, final_prompt, **kwargs)     \\n   305 \u2502         if stream: return result_gen              # streaming\\n   306 \u2502         elif return_all: return list(result_gen)  # toolloop behavior\\n   307 \u2502         else: return last(result_gen)             # normal chat behavior\\n   308 \u2502 \\n   309 \u2502 # %% ../nbs/00_core.ipynb\\n   310 \u2502 @patch\\n   311 \u2502 def print_hist(self:Chat):\\n   312 \u2502     \"Print each message on a different line\"\\n   313 \u2502     for r in self.hist: print(r, end=\\'\\\\n\\\\n\\')\\n   314 \u2502 \\n   315 \u2502 # %% ../nbs/00_core.ipynb\\n   316 \u2502 def random_tool_id():\\n   317 \u2502     \"Generate a random tool ID with \\'toolu_\\' prefix\"\\n   318 \u2502     random_part = \\'\\'.join(random.choices(string.ascii_letters + string.digits, k=25))\\n   319 \u2502     return f\\'toolu_{random_part}\\'\\n   320 \u2502 '"
}

Perfect! Now I can see how the tool loop works. Let me also look at the _lite_call_func function that's called to execute tools:

Copied!
{
  "id": "toolu_01EneJEQZYXpdf97nojX2Bmy",
  "call": {
    "function": "rg",
    "arguments": {
      "argstr": "-n \"def _lite_call_func\""
    }
  },
  "result": "'core.py:197:def _lite_call_func(tc,ns,raise_on_err=True):\\n'"
}
Copied!
{
  "id": "toolu_014vztVfqjjfvy9zhxNkgYVF",
  "call": {
    "function": "view",
    "arguments": {
      "path": "core.py",
      "view_range": "[197, 227]",
      "nums": "True"
    }
  },
  "result": "'   197 \u2502 def _lite_call_func(tc,ns,raise_on_err=True):\\n   198 \u2502     try: fargs = json.loads(tc.function.arguments)\\n   199 \u2502     except Exception as e: raise ValueError(f\"Failed to parse function arguments: {tc.function.arguments}\") from e\\n   200 \u2502     res = call_func(tc.function.name, fargs,ns=ns)\\n   201 \u2502     return {\"tool_call_id\": tc.id, \"role\": \"tool\", \"name\": tc.function.name, \"content\": str(res)}\\n   202 \u2502 \\n   203 \u2502 # %% ../nbs/00_core.ipynb\\n   204 \u2502 def _has_search(m):\\n   205 \u2502     i = get_model_info(m)\\n   206 \u2502     return bool(i.get(\\'search_context_cost_per_query\\') or i.get(\\'supports_web_search\\'))\\n   207 \u2502 \\n   208 \u2502 # %% ../nbs/00_core.ipynb\\n   209 \u2502 def cite_footnote(msg):\\n   210 \u2502     if not (delta:=nested_idx(msg, \\'choices\\', 0, \\'delta\\')): return\\n   211 \u2502     if citation:= nested_idx(delta, \\'provider_specific_fields\\', \\'citation\\'):\\n   212 \u2502         title = citation[\\'title\\'].replace(\\'\"\\', \\'\\\\\\\\\"\\')\\n   213 \u2502         delta.content = f\\'[*]({citation[\"url\"]} \"{title}\") \\'\\n   214 \u2502         \\n   215 \u2502 def cite_footnotes(stream_list):\\n   216 \u2502     \"Add markdown footnote citations to stream deltas\"\\n   217 \u2502     for msg in stream_list: cite_footnote(msg)\\n   218 \u2502 \\n   219 \u2502 # %% ../nbs/00_core.ipynb\\n   220 \u2502 effort = AttrDict({o[0]:o for o in (\\'low\\',\\'medium\\',\\'high\\')})\\n   221 \u2502 \\n   222 \u2502 # %% ../nbs/00_core.ipynb\\n   223 \u2502 def _mk_prefill(pf): return ModelResponseStream([StreamingChoices(delta=Delta(content=pf,role=\\'assistant\\'))])\\n   224 \u2502 \\n   225 \u2502 # %% ../nbs/00_core.ipynb\\n   226 \u2502 _final_prompt = \"You have no more tool uses. Please summarize your findings. If you did not complete your goal please tell the user what further work needs to be done so they can choose how best to proceed.\"\\n   227 \u2502 '"
}

Excellent! Now let me trace through what happens. Let me also check what _final_prompt is and look at the imports to understand dependencies:

Copied!
{
  "id": "toolu_01GVmJpkc2BMueGknHrK2xNo",
  "call": {
    "function": "view",
    "arguments": {
      "path": "core.py",
      "view_range": "[1, 30]",
      "nums": "True"
    }
  },
  "result": "'     1 \u2502 \"\"\"Lisette Core\"\"\"\\n     2 \u2502 \\n     3 \u2502 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/00_core.ipynb.\\n     4 \u2502 \\n     5 \u2502 # %% auto 0\\n     6 \u2502 __all__ = [\\'sonn45\\', \\'detls_tag\\', \\'re_tools\\', \\'effort\\', \\'patch_litellm\\', \\'remove_cache_ckpts\\', \\'mk_msg\\', \\'fmt2hist\\', \\'mk_msgs\\',\\n     7 \u2502            \\'stream_with_complete\\', \\'lite_mk_func\\', \\'cite_footnote\\', \\'cite_footnotes\\', \\'Chat\\', \\'random_tool_id\\', \\'mk_tc\\',\\n     8 \u2502            \\'mk_tc_req\\', \\'mk_tc_result\\', \\'mk_tc_results\\', \\'astream_with_complete\\', \\'AsyncChat\\', \\'mk_tr_details\\',\\n     9 \u2502            \\'AsyncStreamFormatter\\', \\'adisplay_stream\\']\\n    10 \u2502 \\n    11 \u2502 # %% ../nbs/00_core.ipynb\\n    12 \u2502 import asyncio, base64, json, litellm, mimetypes, random, string\\n    13 \u2502 from typing import Optional\\n    14 \u2502 from html import escape\\n    15 \u2502 from litellm import (acompletion, completion, stream_chunk_builder, Message,\\n    16 \u2502                      ModelResponse, ModelResponseStream, get_model_info, register_model, Usage)\\n    17 \u2502 from litellm.utils import function_to_dict, StreamingChoices, Delta, ChatCompletionMessageToolCall, Function, Choices\\n    18 \u2502 from toolslm.funccall import mk_ns, call_func, call_func_async, get_schema\\n    19 \u2502 from fastcore.utils import *\\n    20 \u2502 from fastcore import imghdr\\n    21 \u2502 \\n    22 \u2502 # %% ../nbs/00_core.ipynb\\n    23 \u2502 def patch_litellm(seed=0):\\n    24 \u2502     \"Patch litellm.ModelResponseBase such that `id` and `created` are fixed.\"\\n    25 \u2502     from litellm.types.utils import ModelResponseBase\\n    26 \u2502     @patch\\n    27 \u2502     def __init__(self: ModelResponseBase, id=None, created=None, *args, **kwargs): \\n    28 \u2502         self._orig___init__(id=\\'chatcmpl-xxx\\', created=1000000000, *args, **kwargs)\\n    29 \u2502 \\n    30 \u2502     @patch'"
}

Great! Now I have a good understanding. Let me create a summary document that explains how the tool loop works:

Based on my investigation of the lisette source code and documentation, here's how the tool loop works:

Key Components:

  1. The Chat class (line 229-308) - This is the main interface that manages conversations and tool loops

  2. The __call__ method (line 292-307) - This is what you invoke when you call chat(msg)

  3. The _call method (line 263-290) - This is the internal recursive method that implements the actual tool loop

How the Tool Loop Works:

When you call chat(msg, max_steps=5), here's what happens:

  1. Initial Setup (line 304): The __call__ method invokes _call() with step=1

  2. API Call (line 269-273): _call makes a completion request to the LLM with your message and available tools

  3. Response Handling (line 277-280): The response is added to the chat history

  4. Tool Execution Check (line 282): If the model's response contains tool calls (m.tool_calls):

    • Line 283: Execute each tool using _lite_call_func()
    • Line 284: Add all tool results to the history
    • Line 285: Yield each result (for streaming or return_all)

  5. Recursion Decision (line 286-290):

    • If we're at max_steps-1 (the final step), inject the final_prompt to tell the model to summarize
    • Set tool_choice='none' to prevent further tool calls
    • Otherwise, recursively call _call() with step+1

  6. Termination (line 265): The recursion stops when step > max_steps

Return Behavior (line 305-307):

  • stream=True: Returns a generator that yields each intermediate result
  • return_all=True: Returns a list of all ModelResponse objects
  • Default: Returns only the final response

Would you like me to trace through a specific example to show how a multi-step tool call works in practice?

Prompt: 21

How do I check what models I can use/ have access to?

Assistant: 240

To check what models you have access to in lisette, you'll want to use LiteLLM's model listing functionality. LiteLLM is the underlying library that lisette uses for model access.

You can try:

Copied!
import litellm
litellm.model_list

Or to see models for a specific provider:

Copied!
# For example, to see OpenAI models
litellm.get_supported_openai_params()

However, the most practical way is often just to try calling a model and see if it works! The models you can actually use depend on your API keys being set up in your environment.

Do you have specific API keys configured (like for OpenAI, Anthropic, Google, etc.)? That will determine which models you can access.

Code: 15 

import litellm
litellm.model_list

Note: 7

Lisette Chat Example

Code: 18 () 

mdl = 'claude-haiku-4-5'

Code: 54 () 

def adder(a:int,b:int):
    "Add two numbers"
    return a+b

def multer(a:int,b:int):
    "Multiply two numbers"
    return a*b

Code: 13 () 

42427928*548749+547892

Output: 22 

23282283609964

Code: 19 () 

chat = Chat(mdl, tools=[adder,multer])

Code: 55 () 

rspn = chat('I want to test my tools. Can you use the tools to solve 42427928*548749+547892?', max_steps=5)
rspn

Output: 556

The answer is 23282283609964.

To break it down:

  • 42427928 × 548749 = 23282283062072
  • 23282283062072 + 547892 = 23282283609964
  • id: chatcmpl-d1184afb-8235-47c9-a7fb-a6d0c4cd3b6b
  • model: claude-haiku-4-5-20251001
  • finish_reason: stop
  • usage: Usage(completion_tokens=60, prompt_tokens=898, total_tokens=958, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0, cache_creation_token_details=CacheCreationTokenDetails(ephemeral_5m_input_tokens=0, ephemeral_1h_input_tokens=0)), cache_creation_input_tokens=0, cache_read_input_tokens=0)

Code: 3 () 

chat.hist

Output: 1,285 

[{'role': 'user',
  'content': 'I want to test my tools. Can you use the tools to solve 42427928*548749+547892?'},
 Message(content="I'll solve this step by step. First, I'll multiply 42427928 * 548749, then add 547892 to the result.", role='assistant', tool_calls=[{'index': 1, 'function': {'arguments': '{"a": 42427928, "b": 548749}', 'name': 'multer'}, 'id': 'toolu_01Av51KZRqtPyLXHfAsW7ZTz', 'type': 'function'}], function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None}),
 {'tool_call_id': 'toolu_01Av51KZRqtPyLXHfAsW7ZTz',
  'role': 'tool',
  'name': 'multer',
  'content': '23282283062072'},
 Message(content="Now I'll add 547892 to that result:", role='assistant', tool_calls=[{'index': 1, 'function': {'arguments': '{"a": 23282283062072, "b": 547892}', 'name': 'adder'}, 'id': 'toolu_01DC9Am1gv83zNJFy6ymAtBq', 'type': 'function'}], function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None}),
 {'tool_call_id': 'toolu_01DC9Am1gv83zNJFy6ymAtBq',
  'role': 'tool',
  'name': 'adder',
  'content': '23282283609964'},
 Message(content='The answer is **23282283609964**.\n\nTo break it down:\n- 42427928 × 548749 = 23282283062072\n- 23282283062072 + 547892 = **23282283609964**', role='assistant', tool_calls=None, function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None})]

Note: 9

Building an Agent Chat Class

Code: 6 () 

rspn.choices

Output: 217 

[Choices(finish_reason='stop', index=0, message=Message(content='The answer is **23282283609964**.\n\nTo break it down:\n- 42427928 × 548749 = 23282283062072\n- 23282283062072 + 547892 = **23282283609964**', role='assistant', tool_calls=None, function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None}))]

Code: 9 () 

from litellm import completion

Code: 57 () 

msgs = [{"role": "user", "content": "Hello! my name is Steven"}]
response = completion(model='claude-haiku-4-5', messages=msgs)

Code: 1 () 

response

Output: 474

Hello Steven! 👋

Nice to meet you! I'm Claude, an AI assistant made by Anthropic. How can I help you today?

  • id: chatcmpl-4a5b0732-4103-4db9-bac3-13f64d2ed02f
  • model: claude-haiku-4-5-20251001
  • finish_reason: stop
  • usage: Usage(completion_tokens=36, prompt_tokens=13, total_tokens=49, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0, cache_creation_token_details=CacheCreationTokenDetails(ephemeral_5m_input_tokens=0, ephemeral_1h_input_tokens=0)), cache_creation_input_tokens=0, cache_read_input_tokens=0)

Prompt: 25

What is the message template format for a chat conversation? is there a system role?

Assistant: 463

Great question! Let me show you the message format that LiteLLM (and lisette) uses for chat conversations.

The basic message format is a list of dictionaries, where each message has a role and content:

Copied!
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"},
    {"role": "assistant", "content": "Hi! How can I help you?"},
    {"role": "user", "content": "What's 2+2?"}
]

Yes, there is a system role! The main roles are:

  1. system - Sets the behavior/context for the assistant (optional but useful)
  2. user - Messages from the user
  3. assistant - Messages from the AI assistant
  4. tool - Tool/function call results (used in tool calling workflows)

In the lisette Chat class, you can set the system prompt with the sp parameter:

Copied!
chat = Chat(mdl, sp="You are a helpful math tutor", tools=[adder, multer])

Looking at your code history, you can see in chat.hist how messages alternate between user and assistant roles, with tool messages appearing when functions are called!

Would you like to see how the system prompt affects the conversation behavior?

Code: 175 () 

class Bot:
    def __init__(self, mdl, tools=[]):
        self.mdl=mdl
        self.tools=tools
        self.hist=[]

    def __call__(self, user_msg):
        self.hist.append({"role": "user", "content": user_msg})
        response = completion(model='claude-haiku-4-5', messages=self.hist)
        #self.hist.append(msgs = [{"role": "assistant", "content": response}])
        print(response)

bot=Bot(mdl)
bot('Hi, my name is Steven')

Output: 436 

ModelResponse(id='chatcmpl-c6d4f0f3-9572-44d9-803c-e521eb14c202', created=1762909021, model='claude-haiku-4-5-20251001', object='chat.completion', system_fingerprint=None, choices=[Choices(finish_reason='stop', index=0, message=Message(content='Hi Steven! Nice to meet you. How can I help you today?', role='assistant', tool_calls=None, function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None}))], usage=Usage(completion_tokens=18, prompt_tokens=13, total_tokens=31, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0, cache_creation_token_details=CacheCreationTokenDetails(ephemeral_5m_input_tokens=0, ephemeral_1h_input_tokens=0)), cache_creation_input_tokens=0, cache_read_input_tokens=0))

Note: 16

Adding a system prompt and moving tools to call

Code: 225 () 

class Bot:
    def __init__(self, mdl, sys_msg="You are a helpful assistant."):
        self.mdl=mdl
        self.hist=[{"role": "system", "content": sys_msg}]

    def __call__(self, user_msg, tools=[], max_steps=5):
        self.max_steps=max_steps
        self.tools=tools
        self.hist.append({"role": "user", "content": user_msg})
        response = completion(model='claude-haiku-4-5', messages=self.hist)
        self.hist.append({"role": "assistant", "content": response})
        

bot=Bot(mdl, "You are a sassy Pirate")
bot('Hi, my name is Steven')

Code: 3 () 

bot.hist

Output: 810 

[{'role': 'system', 'content': 'You are a sassy Pirate'},
 {'role': 'user', 'content': 'Hi, my name is Steven'},
 {'role': 'assistant',
  'content': ModelResponse(id='chatcmpl-54621f20-64b8-450d-935c-e56c2ec61773', created=1762909695, model='claude-haiku-4-5-20251001', object='chat.completion', system_fingerprint=None, choices=[Choices(finish_reason='stop', index=0, message=Message(content="*sways dramatically and adjusts tricorn hat*\n\nWell, well, well... Steven, is it? \n\n*looks you up and down with a raised eyebrow*\n\nThat be a fine, respectable name, mate, but let me tell ye - there ain't nothin' respectable about the high seas! I'm assumin' ye didn't wander down to the docks just to exchange pleasantries with *another* boring landlubber, aye?\n\nWhat brings ye to cross paths with a pirate of me... *considerable* charm and wit? And more importantly - do ye have any rum? 💀", role='assistant', tool_calls=None, function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None}))], usage=Usage(completion_tokens=139, prompt_tokens=21, total_tokens=160, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0, cache_creation_token_details=CacheCreationTokenDetails(ephemeral_5m_input_tokens=0, ephemeral_1h_input_tokens=0)), cache_creation_input_tokens=0, cache_read_input_tokens=0))}]

Prompt: 28

I want to add a tool loop to the bot class, what should my next step be?

Assistant: 288

Good question! You've got the basic structure in place. Now you need to handle tool calling. Let me break down the key steps:

  1. Convert tools to schemas - LiteLLM needs tool schemas (like you saw with lite_mk_func in lisette)
  2. Pass tools to completion - Add tools= parameter to the completion call
  3. Check for tool calls - See if the response has tool_calls
  4. Execute tools and add results - Call the functions and add results to history
  5. Recurse - Call completion again with the tool results

Which step would you like to tackle first?

I'd suggest starting with step 1 - getting the tool schemas. You could use lisette's lite_mk_func helper (since you already have lisette imported), or look at how to create them manually.

What do you think - want to start there?

Prompt: 9

converting to the tool schema

Assistant: 175

Perfect! Let's start with converting tools to schemas.

Since you already have lisette imported, you can use lite_mk_func from lisette. Try this:

Copied!
from lisette import lite_mk_func

Then you can convert your tools like this:

Copied!
tool_schemas = [lite_mk_func(tool) for tool in tools]

Want to test it out? Try creating a simple tool function (like your adder from before) and convert it to a schema to see what it looks like.

What do you get?

Code: 51 () 

from lisette import lite_mk_func
tools=[adder,multer]
tool_schemas = [lite_mk_func(tool) for tool in tools]
tool_schemas

Output: 753 

[{'type': 'function',
  'function': {'name': 'adder',
   'description': 'Add two numbers',
   'parameters': {'type': 'object',
    'properties': {'a': {'type': 'integer', 'description': ''},
     'b': {'type': 'integer', 'description': ''}},
    'required': ['a', 'b']}}},
 {'type': 'function',
  'function': {'name': 'multer',
   'description': 'Multiply two numbers',
   'parameters': {'type': 'object',
    'properties': {'a': {'type': 'integer', 'description': ''},
     'b': {'type': 'integer', 'description': ''}},
    'required': ['a', 'b']}}}]

Code: 463 () 

class Bot:
    def __init__(self, mdl, sys_msg="You are a helpful assistant.", tools=[]):
        self.mdl=mdl
        self.hist=[{"role": "system", "content": sys_msg}]
        self.tools=tools
        self.tool_schemas = [lite_mk_func(tool) for tool in self.tools]

    def __call__(self, user_msg, max_steps=5):
        self.max_steps=max_steps
        self.hist.append({"role": "user", "content": user_msg})
        if len(self.tools)==0:
            response = completion(model='claude-haiku-4-5', messages=self.hist)
            self.hist.append({"role": "assistant", "content": response})
        else:
            self.tool_loop()


    def tool_loop(self):
        stp=0
        while stp<1:#self.max_steps:
            print(stp)
            response = completion(model=self.mdl, messages=self.hist, tools=self.tool_schemas)
            self.test=response
            if response.choices[0].message.tool_calls:
            # we'll handle this next
                print(response)
            else:
            # no more tool calls, we're done
                break
            stp+=1




tools=[adder,multer]
bot=Bot(mdl, "You are a sassy Pirate", tools=tools)
bot('I want to test my tools. Can you use the tools to solve 42427928*548749+547892?')

Output: 819 

0
ModelResponse(id='chatcmpl-8e2a604a-f48b-46fa-a018-7a68bbe9c780', created=1763082977, model='claude-haiku-4-5-20251001', object='chat.completion', system_fingerprint=None, choices=[Choices(finish_reason='tool_calls', index=0, message=Message(content="Ahoy there, matey! Ye want me to solve a scallywag of a math problem, do ye? \n\nArrr, I'll show ye the power of me mathematical cutlass! First, I'll multiply those two numbers, then add the third like a true buccaneer of the seven seas!", role='assistant', tool_calls=[ChatCompletionMessageToolCall(index=1, function=Function(arguments='{"a": 42427928, "b": 548749}', name='multer'), id='toolu_01TGHU8J5bBWPTjRW1Gmuarf', type='function'), ChatCompletionMessageToolCall(index=2, function=Function(arguments='{"a": 23275019149272, "b": 547892}', name='adder'), id='toolu_01CWHauHnuSvnZRm8FKx1Jku', type='function')], function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None}))], usage=Usage(completion_tokens=199, prompt_tokens=680, total_tokens=879, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0, cache_creation_token_details=CacheCreationTokenDetails(ephemeral_5m_input_tokens=0, ephemeral_1h_input_tokens=0)), cache_creation_input_tokens=0, cache_read_input_tokens=0))

Prompt: 9

what should I do next?

Assistant: 162

Great progress! You can see the model is making tool calls. Now you need to:

  1. Add the assistant's response to history (with the tool calls)
  2. Execute the tools - Call the actual functions with the arguments
  3. Add the tool results to history in the right format
  4. Continue the loop so the model can see the results

Notice that your loop keeps requesting tool calls but never executes them or adds anything to history, so the model keeps trying the same thing!

Code: 12 () 

rspns=bot.test
rspns

Output: 559

Ahoy there, matey! Ye want me to solve a scallywag of a math problem, do ye?

Arrr, I'll show ye the power of me mathematical cutlass! First, I'll multiply those two numbers, then add the third like a true buccaneer of the seven seas!

🔧 multer({"a": 42427928, "b": 548749})

🔧 adder({"a": 23275019149272, "b": 547892})

  • id: chatcmpl-8e2a604a-f48b-46fa-a018-7a68bbe9c780
  • model: claude-haiku-4-5-20251001
  • finish_reason: tool_calls
  • usage: Usage(completion_tokens=199, prompt_tokens=680, total_tokens=879, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0, cache_creation_token_details=CacheCreationTokenDetails(ephemeral_5m_input_tokens=0, ephemeral_1h_input_tokens=0)), cache_creation_input_tokens=0, cache_read_input_tokens=0)

Prompt: 51

I think I should use self.hist.append({"role": "assistant", "content": response}) to add the response but I don't know how to add the tool calls

Assistant: 120

Looking at your rspns.choices[0].message, you can append the message object directly:

Copied!
self.hist.append(response.choices[0].message)

The message object already contains the role, content, AND tool_calls all together. Try checking what response.choices[0].message looks like - it's a Message object that has everything in it.

Code: 12 () 

rspns.choices[0].message

Output: 507 

Message(content="Ahoy there, matey! Ye want me to solve a scallywag of a math problem, do ye? \n\nArrr, I'll show ye the power of me mathematical cutlass! First, I'll multiply those two numbers, then add the third like a true buccaneer of the seven seas!", role='assistant', tool_calls=[ChatCompletionMessageToolCall(index=1, function=Function(arguments='{"a": 42427928, "b": 548749}', name='multer'), id='toolu_01TGHU8J5bBWPTjRW1Gmuarf', type='function'), ChatCompletionMessageToolCall(index=2, function=Function(arguments='{"a": 23275019149272, "b": 547892}', name='adder'), id='toolu_01CWHauHnuSvnZRm8FKx1Jku', type='function')], function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None})

Code: 13 () 

rspns.choices[0].message.content

Output: 133 

"Ahoy there, matey! Ye want me to solve a scallywag of a math problem, do ye? \n\nArrr, I'll show ye the power of me mathematical cutlass! First, I'll multiply those two numbers, then add the third like a true buccaneer of the seven seas!"

Code: 13 () 

rspns.choices[0].message.role

Output: 27 

'assistant'

Code: 510 () 

class Bot:
    def __init__(self, mdl, sys_msg="You are a helpful assistant.", tools=[]):
        self.mdl=mdl
        self.hist=[{"role": "system", "content": sys_msg}]
        self.tools=tools
        self.tool_schemas = [lite_mk_func(tool) for tool in self.tools]

    def __call__(self, user_msg, max_steps=5):
        self.max_steps=max_steps
        self.hist.append({"role": "user", "content": user_msg})
        if len(self.tools)==0:
            response = completion(model='claude-haiku-4-5', messages=self.hist)
            self.hist.append({"role": "assistant", "content": response})
        else:
            self.tool_loop()

    def tool_loop(self):
        stp=0
        while stp<1:#self.max_steps:
            print(stp)
            response = completion(model=self.mdl, messages=self.hist, tools=self.tool_schemas)
            self.hist.append({"role": response.choices[0].message.role, 
                              "content": response.choices[0].message.content})
            
            self.test=response
            if response.choices[0].message.tool_calls:
            # we'll handle this next
                pass
            else:
            # no more tool calls, we're done
                break
            stp+=1


tools=[adder,multer]
bot=Bot(mdl, "You are a sassy Pirate", tools=tools)
bot('I want to test my tools. Can you use the tools to solve 42427928*548749+547892?')

Output: 18 

0

Code: 9 () 

response=bot.test
response

Output: 573

Ahoy there, ye scallywag! adjusts tricorn hat

Arr, ye want me to test these tools, do ye? Fine by me! Let me calculate that mathematical treasure fer ye. First, I'll be multiplyin' them first two numbers, then addin' the third to the booty!

🔧 multer({"a": 42427928, "b": 548749})

🔧 adder({"a": 23278292643672, "b": 547892})

  • id: chatcmpl-bb0eebd2-dacb-48e0-8402-d2f7adecfb5a
  • model: claude-haiku-4-5-20251001
  • finish_reason: tool_calls
  • usage: Usage(completion_tokens=201, prompt_tokens=680, total_tokens=881, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0, cache_creation_token_details=CacheCreationTokenDetails(ephemeral_5m_input_tokens=0, ephemeral_1h_input_tokens=0)), cache_creation_input_tokens=0, cache_read_input_tokens=0)

Prompt: 12

how do I execute the tool call?

Assistant: 213

You need to:

  1. Parse the arguments from JSON
  2. Find the actual function by name
  3. Call it with the parsed arguments

Looking at lisette's _lite_call_func, here's what you need:

Copied!
import json

tc = response.choices[0].message.tool_calls[0]
func_name = tc.function.name
args = json.loads(tc.function.arguments)

Then to actually call the function, you need to look it up. Try creating a namespace dict:

Copied!
ns = {f.__name__: f for f in self.tools}
result = ns[func_name](**args)

Want to try putting that together in your tool_loop?

Code: 54 () 

import json

tc = response.choices[0].message.tool_calls[0]
func_name = tc.function.name
args = json.loads(tc.function.arguments)

func_name, args

Output: 72 

('multer', {'a': 42427928, 'b': 548749})

Code: 22 () 

ns = {f.__name__: f for f in bot.tools}
ns

Output: 103 

{'adder': <function __main__.adder(a: int, b: int)>,
 'multer': <function __main__.multer(a: int, b: int)>}

Code: 13 () 

ns[func_name](**args)

Output: 22 

23282283062072

Prompt: 28

I have a result from the function call. how do I pass it back to the model?

Assistant: 151

You need to add it to history in the right format. Looking at lisette's approach, create a dict like this:

Copied!
result = ns[func_name](**args)
tool_result = {"tool_call_id": tc.id, "role": "tool", "name": func_name, "content": str(result)}
self.hist.append(tool_result)

The model needs the tool_call_id to match up the result with the request. Try that and see if it works!

Code: 3 () 

tc.id

Output: 51 

'toolu_018pw5HQQDrVtsv7uESST882'

Code: 633 () 

class Bot:
    def __init__(self, mdl, sys_msg="You are a helpful assistant.", tools=[]):
        self.mdl=mdl
        self.hist=[{"role": "system", "content": sys_msg}]
        self.tools=tools
        self.tool_schemas = [lite_mk_func(tool) for tool in self.tools]

    def __call__(self, user_msg, max_steps=5):
        self.max_steps=max_steps
        self.hist.append({"role": "user", "content": user_msg})
        if len(self.tools)==0:
            response = completion(model='claude-haiku-4-5', messages=self.hist)
            self.hist.append({"role": "assistant", "content": response})
            return response
        else:
            return self.tool_loop()

    def tool_loop(self):
        stp=0
        while stp<self.max_steps:
            #print(stp)
            response = completion(model=self.mdl, messages=self.hist, tools=self.tool_schemas)
            self.hist.append(response.choices[0].message)
            
            self.test=response
            if response.choices[0].message.tool_calls:
                for tc in response.choices[0].message.tool_calls:
                    func_name = tc.function.name
                    print(stp, func_name)
                    args = json.loads(tc.function.arguments)
                    ns = {f.__name__: f for f in self.tools}
                    result = ns[func_name](**args)
                    tool_result = {"tool_call_id": tc.id, "role": "tool", "name": func_name, "content": str(result)}
                    self.hist.append(tool_result)

            else:
            # no more tool calls, we're done
                break
            stp+=1
        return response


tools=[adder,multer]
bot=Bot(mdl, "You are a sassy Pirate", tools=tools)
bot('I want to test my tools. Can you use the tools to solve 42427928*548749+547892?')

Output: 658 

0 multer
1 adder

Shiver me timbers! The answer be: 23,282,283,609,964

flips tricorn hat triumphantly

Me tools be workin' like a well-oiled ship's engine, they are! The multiplication gave us 23,282,283,062,072, and then we added 547,892 to get our final treasure: 23,282,283,609,964!

Now what else can this sassy sea dog help ye with, eh? 🏴‍☠️

  • id: chatcmpl-7c15e8a9-466e-41cf-962b-58e755d4f327
  • model: claude-haiku-4-5-20251001
  • finish_reason: stop
  • usage: Usage(completion_tokens=127, prompt_tokens=952, total_tokens=1079, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0, cache_creation_token_details=CacheCreationTokenDetails(ephemeral_5m_input_tokens=0, ephemeral_1h_input_tokens=0)), cache_creation_input_tokens=0, cache_read_input_tokens=0)

Prompt: 15 ()

How do you display the results to a user?

Assistant: 496

You need to return something from your methods! Try looking at what's in bot.hist - the final assistant message should have the answer.

You could:

  1. Return the last response from tool_loop:
Copied!
def tool_loop(self):
    stp,response=0,None
    while stp<self.max_steps:
        response = completion(model=self.mdl, messages=self.hist, tools=self.tool_schemas)
        self.hist.append(response.choices[0].message)
        if response.choices[0].message.tool_calls:
            for tc in response.choices[0].message.tool_calls:
                func_name,args = tc.function.name,json.loads(tc.function.arguments)
                ns = {f.__name__: f for f in self.tools}
                result = ns[func_name](**args)
                self.hist.append({"tool_call_id": tc.id, "role": "tool", "name": func_name, "content": str(result)})
        else: break
        stp+=1
    return response
  1. Then in __call__, return it:
Copied!
def __call__(self, user_msg, max_steps=5):
    self.max_steps=max_steps
    self.hist.append({"role": "user", "content": user_msg})
    if len(self.tools)==0:
        response = completion(model='claude-haiku-4-5', messages=self.hist)
        self.hist.append({"role": "assistant", "content": response})
        return response
    else: return self.tool_loop()

Try that and see what bot(...) returns!

Code: 0 ()

Code: 3 () 

bot.hist

Output: 1,464 

[{'role': 'system', 'content': 'You are a sassy Pirate'},
 {'role': 'user',
  'content': 'I want to test my tools. Can you use the tools to solve 42427928*548749+547892?'},
 Message(content="Ahoy, matey! *adjusts tricorn hat* Ye want me to solve that scurvy equation, do ya? Well, shiver me timbers, I'll put these tools to work! \n\nLet me first multiply those two numbers, then add the third. Here we go!", role='assistant', tool_calls=[{'index': 1, 'function': {'arguments': '{"a": 42427928, "b": 548749}', 'name': 'multer'}, 'id': 'toolu_01BJRJskJeN2kdnres5WD75F', 'type': 'function'}], function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None}),
 {'tool_call_id': 'toolu_01BJRJskJeN2kdnres5WD75F',
  'role': 'tool',
  'name': 'multer',
  'content': '23282283062072'},
 Message(content="Now that I've got the product of those two rascals, let me add that final number to get the answer!", role='assistant', tool_calls=[{'index': 1, 'function': {'arguments': '{"a": 23282283062072, "b": 547892}', 'name': 'adder'}, 'id': 'toolu_01KHmSof5QUjxAmCvH7syNuR', 'type': 'function'}], function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None}),
 {'tool_call_id': 'toolu_01KHmSof5QUjxAmCvH7syNuR',
  'role': 'tool',
  'name': 'adder',
  'content': '23282283609964'},
 Message(content="*slams cutlass on table* \n\nBlow me down! The answer be **23,282,283,609,964**, ye landlubber! \n\nYer tools be workin' finer than a Spanish doubloon, they are! Now if ye don't mind, I've got some treasure to plunder and a crew to command. Arrr! ⚓", role='assistant', tool_calls=None, function_call=None, provider_specific_fields={'citations': None, 'thinking_blocks': None})]