The facts on input tokens
- A 'token' is usually considered 4 characters, which is 3/4 of a word on average.
- OpenAI, Anthropic, and other AI services charge in part based on input tokens: the amount of text in characters that you ship to their API.
- Reducing the size of the text you send to their API (your 'prompt') is the best lever you have for controlling your costs.
-
Compressing a prompt without losing context
iswas difficult, but now you have vrbose.ai .