OpenAI team, thanks for the work you're doing. I'm thrilled with...
I appreciate the efforts of the OpenAI team and the continuous advancements they are making. The recent news has sparked my interest, and I have some questions regarding the Assistants API pricing structure that I couldn't find documented elsewhere.
Charging Process
I am curious about when charges are incurred when using the Assistants API. Are we billed upon initiating a run or when adding a message to the conversation thread?
Token Calculation
Furthermore, I would like clarification on how tokens are calculated. Is the charge based on the entire conversation thread for each interaction (run)? Additionally, how is token calculation handled for lengthy threads that may have been truncated in the background?
According to the documentation, charges are applied for the complete conversation during each run.
Knowledge Retrieval
How does token calculation work when integrating knowledge retrieval? The size of the information plays a significant role in this process. It can either be processed entirely as tokens or segmented into smaller chunks, converted into embeddings, and then a maximum cost is determined based on the document size. The model will iterate through these chunks until an appropriate answer is identified.
Estimating Token Usage
Is there a way to estimate the number of tokens required before initiating each run? Utilizing libraries such as tiktoken and analyzing the average token usage from previous runs along with the knowledge base can provide insights. Establishing a limit beforehand and subsequently reviewing the calculated costs proves to be a beneficial approach.
While the Assistants API streamlines backend operations, the clarity on pricing benefits may not be readily apparent. Developers may relinquish some control, potentially resulting in unforeseen costs. This aspect should be considered when leveraging Assistants for various tasks.
Powered by Discourse, best viewed with JavaScript enabled.
