Exposing usage data from API response

[derekkraan]

Hi @brainlid, first of all thanks for this project, it has been very useful to us so far.

We are running into the issue that we want to be able to track our token usage on OpenAI. This is given as part of the response, but I believe LangChain doesn't do anything with this information yet.

I am wondering if you would consider a PR to expose this data somehow.

And if you are, whether you have a preferred way to do this. We would probably be happy with simply making the raw response available somehow, as a trap door. But if you want to structure this data and translate it per API, we could also talk about that.

Thanks, Derek

[brainlid]

Hi @derekkraan! I'm glad you've found it helpful! Yes, I am open to a PR for this. I've been thinking about it as I've been working with more models.

For ChatGPT, Anthropic (Claude), and Bumblebee, I think we get the token counts at the end of the a completion. It's a form of metadata, so I can see some type of metadata being exposed on the chain at the completion. Not sure yet though what makes the most sense.

Yes, let's talk about. I'm converting this to a discussion item.