Tool Calling· Lesson 7 of 7
Parallel Tool Calls
In one line · Sometimes the model requests several tools at once, not one after another — and you can run them together to save time.
If every tool needlessly waits for the previous one, your response gets slow. Running independent ones in parallel cuts latency noticeably.
If the model wants three unrelated pieces of info, there's no reason to fetch them in turn. Request them all at the same moment, like sending three messages together.
Like sending three people to fetch three things at the same time instead of one person making three trips — as long as none of the errands depends on another's result.
A user asks 'compare price, weather, and stock for 3 cities' — the model requests 3 tools at once, you run them in parallel, return them all, and it finishes in one reply.
The model can return multiple tool-call blocks in a single reply, each with its own id. Your code then runs them together (Promise.all in JavaScript, say) and returns one result per id in the same next message — each result tied to its id so the model knows which belongs to which. The golden rule: parallelism is only for independent calls — if one call needs another's output (fetch the customer first, then their invoices), that's two sequential loop turns, not a parallel call. And if one of the parallel calls fails, return the successes plus that one's error — don't sacrifice the batch.
🧪 Try it yourself: a user asks 'get the weather for Cairo, Alexandria, and Aswan, then recommend the best one for a trip tomorrow'. Split the task: which calls can run in parallel? (the three cities' weather) And which step must wait? (the recommendation needs all three results). Write out the model's first-turn reply (3 tool calls with 3 ids) and the results message you'd send back.
Top mistakes: (1) running dependent calls in parallel — the second executes with data that hasn't arrived. (2) Parallel writes to the same resource (two updates to one record) — a race with a random winner. (3) Returning results without correct id mapping, scrambling the answers. (4) One tool fails so you throw away the whole batch. (5) Assuming the model always parallelizes — that's its call, and your code must handle both shapes: one request or several.
Independent reads: weather for 3 cities, data from 3 different sources, several checks on one request — that's parallelism's natural ground.
Any chain with dependencies (one step's output feeding the next) must stay sequential, as must writes to the same resource — parallelism there buys race-condition bugs, not speed.