LLM

References

LLM Base Class

Community LLMs

Fake LLM

Fake Streaming LLM

Python asyncio library

Calling thread is NOT blocked when async function is invoked

When you call an async function with await, the calling thread does not block; instead, it yields control back to the event loop, allowing other tasks to run while waiting for the async operation to complete. This enables efficient concurrency within a single thread, where the event loop manages multiple tasks without blocking the execution of other code.

However, it’s important to note that while the calling thread isn’t blocked, you must still await the async function for it to run, and the event loop manages when the coroutine resumes.