[BBD] OpenAI Token Usage Wrong

[monochrome]

I used OpenAI to auto-tag two images. IMatch says I've used 1110 tokens, but OpenAI's "Activity" tab says 6161 tokens.

Looking at https://www.photools.com/help/imatch/index.html#ai-services.htm?dl=ppt, the docs do not mention the number of input tokens that are consumed by the image itself, as described here: https://platform.openai.com/docs/guides/vision#calculating-costs . It could be that these are not returned by the API or included in the IMatch calculation (I have no idea where IMatch gets the number from.)

[Mario]

IMatch has no idea how OpenAI calculates tokens. This may even vary per model used.

There is also the possibility that IMatch has to retry a prompt with a larger limit, when OpenAI returns an error that states that the token limit was exceeded. Using a too large token limit (max_tokens) would waste tokens, but a too small max_tokens may cause OpenAI to fail with the tokens exceeded error. This depends on the prompt you use and the length of the reply. IMatch retries with a doubled max_token limit when this error is returned. IMatch reties a maximum of three times before it considers the request a failure.

Anyway, OpenAI returns a number of headers with each response and IMatch uses the "prompt_tokens" and "completion_tokens" elements to update the token counter it then shows in the AutoTagger dialog. If a retry is needed, the tokens used for both attempts are accumulated.

As I wrote in the help system, this token count may different from what OpenAI counts.
There have been numerous discussions in the OpenAI community, on reddit, on slashdot about this. The way they count tokens, surcharges and whatnot is a bit intransparent.

See https://platform.openai.com/docs/api-reference/introduction for more information. If you think I have something overlooked, let me know.

I've made a quick check. Reset the counter in AutoTagger and ran a prompt.
Open AI returned:
1. Input tokens: 476
2. Output tokens: 93
and the counter now shows 569 tokens. So at least this parts seems to be correct.

I have frankly never bothered about this too much since I can always keep track of the total money spend via the dashboard.

Can you provide your prompt so I can see what OpenAI returns for it?

[Mario]

Any news on this? Else I'll close this bug report.

[Mario]

From what I have read on the web, OpenAI and Mistral (and all other AIs) may charge extra tokens for certain processes or tasks not reflected in the prompt token count, response token count they return via their APIs.

But that is the only data AutoTagger can use to calculate the approximate prompt usage counter shown.