Infino
Infino is an open-source observability platform that stores both metrics and application logs together.
Key features of Infino include:
- Metrics Tracking: Capture time taken by LLM model to handle request, errors, number of tokens, and costing indication for the particular LLM.
- Data Tracking: Log and store prompt, request, and response data for each LangChain interaction.
- Graph Visualization: Generate basic graphs over time, depicting metrics such as request duration, error occurrences, token count, and cost.
Installation and Setup
First, you'll need to install the infinopy Python package as follows:
pip install infinopy
If you already have an Infino Server running, then you're good to go; but if
you don't, follow the next steps to start it:
- Make sure you have Docker installed
- Run the following in your terminal:
docker run --rm --detach --name infino-example -p 3000:3000 infinohq/infino:latest
Using Infino
See a usage example of InfinoCallbackHandler.
from langchain.callbacks import InfinoCallbackHandler