Reading Your AI Bill: What Tokens Are, Why Costs Balloon and Three Ways to Stay on Budget

For anyone who uses AI to write code or hands work to an AI agent, beginners included. You do not need to know how models work. Every term comes with an everyday picture, and the calculation is marked so you can skip it if you do not need it.

Numbers come from public sources, listed at the end. Examples marked "illustration" are reconstructed and name no real person or company. Every price in this post is an assumption chosen to make the arithmetic easy, not a current price list, because prices change by provider and over time. Nobody fully understands how a model works inside, so the pictures are memory aids only. Billing, however, is documented, and the post follows the documentation.

---

Quick summary#

• AI bills by the token, a piece of text the model reads in and writes out. The bill is input tokens times the input price, plus output tokens times the output price.
• In a conversation with many turns, the whole history is sent again at every turn. So cost grows faster than the number of turns. In the post's worked example, 15 turns send 322,500 input tokens but only 7,500 output tokens, and doubling the turns raises input by about 3.4 times.
• Prompt caching can sharply cut the cost of re-reading. In the example, cost falls to about 31%. But rules and prices differ between providers, and the cache expires if you pause for long.
• Three ways to stay on budget: short sessions, then a new session from a summary, cheaper models for routine work, spending caps and monitoring.
• The trend is toward usage-based pricing. Gartner (24 June 2026) forecasts that AI coding costs could exceed an average developer's salary by 2028, and GitHub moved Copilot to token-based credits on 1 June 2026.
• Cheapest per token is not cheapest per finished task. Measure cost and rework together.

---

Hard words, plain meaning#

---

After this post you can explain#

1. What a token is, and why the same content can take different numbers of tokens depending on the language and how text is split.
2. Why a long conversation gets more expensive faster than the number of turns, and what caching does to soften it.
3. Three ways to keep cost on budget, and why cheapest per token is not necessarily cheapest per finished task.

---

1. A picture first: the taxi meter and the notebook that must be reread#

Imagine you hire a very good answerer who remembers nothing between questions. Each time you ask, they must reread the whole notebook of everything said so far before writing an answer. They charge by the pages read plus the pages written, the way a taxi meter ticks by the kilometre.

Question one: the notebook is thin, quick and cheap to read. Question fifteen: the notebook has grown, and each reread costs more. What you add each turn may be only a few lines, but what has to be reread keeps growing.

This picture is only a memory aid. What is clearly documented is the billing: AI services charge for the tokens sent in and the tokens produced, and in multi-turn conversations the earlier history is part of the input at every turn (see also The context window).

Takeaway: the bill does not grow with the number of sentences you type. It grows with the number of times the notebook is reread multiplied by how thick it is.

---

2. What a token is#

2.1 Pieces of text, not words#

A model does not read letter by letter or word by word, but in pieces. A common word may be one piece, a rare word may be cut into several, and punctuation is a piece too. How text is cut differs between models, so I give no specific splits here, because they would be easy to get wrong.

A rule of thumb often quoted for English is about 4 characters per token. I have not verified this against the original page (OpenAI's tokenizer page returned a 403 error when I opened it) and treat it as a rough assumption.

2.2 Same content, different token counts#

Petrov, La Malfa, Torr and Bibi (2023) compared the tokenized length of the same text translated into many languages and found gaps of up to about 15 times in some cases, even for tokenizers built to support many languages. They argue this can lead to higher costs for some language communities.

What this means for you: if you ask AI in Vietnamese or Japanese, the number of tokens per character may be higher than in English. I could not run a token counter in the environment where I wrote this (no tokenizer library was installed), so I give no figure of my own for Vietnamese or Japanese. The right approach is to use the token counter of the tool you actually use and measure your own text. The study measured 2023-era models, and newer ones may differ.

Takeaway: do not guess token counts from word counts. Measure with the provider's counter, on your own content.

---

3. Input and output prices#

Most providers set different prices for input and output tokens, and output is usually more expensive per token. An example from the price table in Anthropic's documentation, which I read on 30 October 2026 (price per million tokens):

In these three rows output is 5 times input, and the small model is many times cheaper than the large one. Treat this as an example from one provider at one moment. Prices change, model names change, and other providers price differently.

Still, do not conclude that "output is pricey so ignore input". The calculation in section 4 shows that in a long conversation input is the largest part of the bill, because it keeps being repeated.

---

4. Why the bill balloons: a script you can run#

Picture it: each turn the notebook grows a little and is reread.

4.1 Assumptions (illustration, you can skip the calculation)#

This is an illustrative example with made-up numbers, not a measurement from any particular tool:

• At the start you send 6,000 tokens of instructions and files.
• Each turn adds 1,500 tokens of new content (what you type, tool output).
• Each turn the model writes 500 tokens, which join the history.
• Assumed prices: $2 per million input tokens, $10 per million output tokens.
• With caching: rereading the cached part costs 0.1 times the input price, writing the new part costs 1.25 times. These two factors are the defaults in Anthropic's documentation (some models have a lower read factor).

4.2 The script, rerunnable#

# Illustrative numbers only. Change them to match your own tool and price list.
BASE = 6000           # tokens of instructions and files sent at the start
NEW_IN = 1500         # new tokens added each turn (your message, tool output)
OUT = 500             # tokens the model writes each turn
PRICE_IN = 2.00       # dollars per million input tokens (assumption)
PRICE_OUT = 10.00     # dollars per million output tokens (assumption)
READ = 0.10           # cache read costs 0.1 x input price (assumption)
WRITE = 1.25          # cache write costs 1.25 x input price (assumption)

def run(turns, base=BASE, new_in=NEW_IN, out=OUT):
    sent = 0                      # input tokens sent in total
    cost_plain = 0.0              # no caching: everything at full input price
    cost_cache = 0.0              # with caching: read old part cheaply, write new part
    context = base                # tokens already in the conversation
    prev_in = 0                   # what the previous call sent (it is in the cache)
    for t in range(1, turns + 1):
        this_in = context + new_in            # whole history is sent again
        sent += this_in
        cost_plain += this_in * PRICE_IN / 1e6
        if t == 1:
            cost_cache += this_in * PRICE_IN * WRITE / 1e6
        else:
            cost_cache += prev_in * PRICE_IN * READ / 1e6            # read what was cached
            fresh = this_in - prev_in                                # last output + new content
            cost_cache += fresh * PRICE_IN * WRITE / 1e6             # write the new part
        cost_out = out * PRICE_OUT / 1e6
        cost_plain += cost_out
        cost_cache += cost_out
        prev_in = this_in
        context = this_in + out               # output joins the history
    return sent, turns * out, cost_plain, cost_cache

for turns in (15, 30):
    sent, out_total, plain, cache = run(turns)
    print(f"{turns} turns: input sent {sent:,} tokens, output {out_total:,} tokens, "
          f"no cache ${plain:.4f}, with cache ${cache:.4f}")

# restart halfway: a fresh session instead of one long history (no caching in this line)
first = run(8)
fresh_short = run(7, base=1000)   # starts from a 1,000-token summary
fresh_full = run(7)               # starts again from the full 6,000-token base
one_long = run(15)
print(f"one session of 15 turns:               ${one_long[2]:.4f}")
print(f"8 turns + fresh 7 (1,000 summary):     ${first[2] + fresh_short[2]:.4f}")
print(f"8 turns + fresh 7 (full 6,000 again):  ${first[2] + fresh_full[2]:.4f}")

The real output when I ran it with Python 3.14.8 (I also recomputed it separately with a different snippet and the two agree):

15 turns: input sent 322,500 tokens, output 7,500 tokens, no cache $0.7200, with cache $0.2212
30 turns: input sent 1,095,000 tokens, output 15,000 tokens, no cache $2.3400, with cache $0.5197
one session of 15 turns:               $0.7200
8 turns + fresh 7 (1,000 summary):     $0.4260
8 turns + fresh 7 (full 6,000 again):  $0.4960

4.3 Reading the result#

• Input dominates. With 15 turns, the total input sent is 322,500 tokens against only 7,500 output tokens, so input is 43 times larger. Without caching, input is $0.645 of the $0.72 total (about 90%), and output is $0.075.
• Cost grows faster than turns. Doubling the turns (15 to 30) takes input from 322,500 to 1,095,000 tokens, about 3.4 times, and uncached cost from $0.72 to $2.34, about 3.25 times.
• Caching cuts the rereading sharply. With 15 turns, cost drops to $0.2212, about 31% of $0.72. With 30 turns it is $0.5197, about 22% of $2.34. It still grows with turns, only more slowly.
• A fresh session at the right moment helps. Splitting 15 turns into 8, then 7 in a new session that starts from a 1,000-token summary, costs $0.426, about 41% lower than $0.72 (same uncached accounting). If the new session has to resend the full 6,000 starting tokens, it is $0.496, about 31% lower.

4.4 Limits of the calculation#

This is a simplified model. It ignores real things: the cache expires if you pause for long (Anthropic's documentation gives a default lifetime of 5 minutes, refreshed each time it is used), a minimum length for caching (from 512 to 4,096 tokens depending on the model), other charges (tools, search), and whether your tool uses caching automatically at all. I have not verified this for specific tools. Your absolute numbers will differ. What matters is the shape of the curve: many turns in one history make cost climb faster than the number of turns.

Takeaway: the main dial is not typing less, but keeping the notebook shorter and rereading it fewer times.

---

5. Three ways to stay on budget#

Picture managing the team's lunch fund: pick the right place for each meal, do not reorder the whole menu every time, and keep a record.

Way 1. Short, fresh sessions with a summary#

When a task is done, do not drag the same conversation into an unrelated one. Ask the agent to summarize what must carry over (decisions made, relevant files, work left), then open a new session. In the example this cuts about 31% to 41% of the uncached cost. A handover summary template is in The context window. Gartner likewise suggests training developers to send only relevant context and to summarize or trim input.

Way 2. Cheaper models for routine work#

Simple, repetitive work (renaming, generating sample data, formatting) often does not need the strongest model. With the assumed prices divided by 4 ($0.50 input and $2.50 output per million tokens), the same 15 turns cost $0.18 instead of $0.72. Gartner advises routing simple, high-volume tasks to smaller models and saving stronger ones for complex work. But keep the balance section below in mind: if a cheaper model needs more turns or leaves mistakes, total cost may not fall.

Way 3. Caps and monitoring#

• Set spending caps per person or team. According to GitHub's post of 27 April 2026, organizations can pool credits and administrators can set budgets at the enterprise, cost center and user level, and when the pool runs out they can allow extra use at published rates or cap spending.
• Look at the tool's usage page every week instead of waiting for the monthly invoice. GitHub says a preview bill was available from early May 2026.
• Gartner suggests setting token thresholds, escalation rules and automated monitoring, and reviewing high-token workflows in sprint retrospectives.

Takeaway: the three work together. Short sessions reduce the tokens, cheaper models reduce the price per token, and caps plus monitoring stop you being surprised.

---

6. Why this topic is heating up: the move to usage-based pricing#

6.1 Gartner's forecast (24 June 2026)#

According to a reprint of Gartner's release on DevOps Digest (Gartner's own page returned a 403 error when I opened it), Gartner forecasts that by 2028 AI coding costs will exceed an average developer's salary. The reasons listed include rising token consumption, the shift from per-seat licenses to usage-based billing, weak governance of usage (agents running on their own, bloated context windows) and vendors not yet having mature cost-optimization tools. The reprint gives no percentages or dollar amounts for that comparison, so treat it as a forecast of direction, not a measured number.

6.2 GitHub Copilot moves to token-based credits (1 June 2026)#

According to GitHub's post of 27 April 2026, which I read through a page-summarizing tool, from 1 June 2026 Copilot usage consumes GitHub AI Credits, calculated from token consumption (input, output and cached tokens) at each model's published API rates. Monthly plans include credits equal to the plan price (for example Pro $10, Business $19 per user per month). Code completions and next edit suggestions do not consume credits. The fallback to a cheaper model when a limit is reached is removed. The page does not state how many dollars one credit is worth (secondary sources give different figures), so I do not use that number.

6.3 Dollar figures per developer: rough reference only#

Blogs such as CloudZero, Eden AI or Exceeds AI give ranges from a few hundred dollars per developer per month to much higher for heavy agent users. These are companies that sell cost-management tools, so they have a commercial interest, and the methods are unclear. I do not draw conclusions from those numbers. Look at your own team's real bill.

Takeaway: the direction is clear, from fixed prices to pay for use. The only certain approach is to measure your own usage.

---

7. Where the budget swells#

Picture a tap running when nobody is looking.

7.1 Illustration: one morning, two ways#

This is an illustrative situation that names no real person or product. Numbers follow the model in section 4.

Way one, stretching one conversation:

• 09:00. A developer asks an agent to fix a display bug and keeps going in the same conversation.
• 09:45. The first task is done after about 15 turns. The developer asks for an unrelated task in the same conversation, believing the agent "already understands the project".
• 11:30. About 30 turns in total. With the post's assumed numbers, uncached cost is about $2.34 instead of $0.72 for 15 turns.

Way two, close the task and open a new session:

• 09:45. The first task is done. The developer asks the agent to summarize the decisions made and the relevant files in about 1,000 tokens, then opens a new session for the second task.
• 11:30. Both tasks are done, each in a short history. With the assumed numbers, two 15-turn sessions cost about $1.44 (two times $0.72) instead of $2.34, about 38% lower. This is multiplication from the script's results, not a real measurement.

What to do: make opening a new session at the boundary between tasks a habit, and check the usage page at the end of the day. This illustration proves nothing. The evidence is in how billing is documented by providers and in Gartner's recommendations above.

---

8. Exercise with an answer key#

What to do: run the script in section 4.2 and compare.

1. Before running, guess: is the cost of 15 uncached turns close to $0.5, $1 or $2?
2. Run it and note four numbers: input sent, output, uncached cost, cached cost.
3. Change exactly one assumption, rerun and compare. Three suggestions: change NEW_IN from 1500 to 3000 (a tool prints a long log), change the number of turns from 15 to 30, or divide both prices by 4.

Answer key to check against (same assumptions, Python 3.14.8):

How to read it: doubling the new content per turn raises cost by about 50%, doubling the turns raises it by about 3.25 times, and a model 4 times cheaper cuts cost exactly 4 times (because prices are multipliers). Compare your guess from step 1 with the real result: the gap tells you whether you tend to guess low or high, and which part you overlook when you forget that history is resent.

A real exercise (30 minutes): open the usage page of the tool you actually use. Pick a task you just finished and note the number of turns, input tokens, output tokens and cost. If the page does not show them, note that. It is a finding too.

---

9. Checklist: eight questions#

1. Does my tool bill by token or at a fixed price, and does the plan have a usage allowance?
2. Do I know this week's usage, or am I just waiting for the invoice?
3. How many turns is the current conversation, and is the task still related to the first turns?
4. Am I sending a whole file or a long log when I need only part of it?
5. Does this task need the strongest model, or is a cheaper one enough?
6. Does the tool set a maximum number of steps or a spending cap for the agent?
7. Is caching on, and how long does it last?
8. How does the cost of this task compare with the rework and the risk of getting it wrong?

---

10. A balanced view: what is known and what is not#

Fairly solid: AI services bill by input and output tokens, output is usually more expensive per token, earlier conversation history is sent again at later turns, and caching can lower the price of rereading (according to Anthropic's and GitHub's documentation on billing).

Unclear:

• Your team's real cost. The model in section 4 is simplified with assumed prices.
• How much your tool uses caching on its own. I did not verify each tool.
• The token gap between Vietnamese, Japanese and English on today's models. The study I cite is from 2023 and I had no counter to measure it myself.
• Gartner's 2028 forecast is a forecast, not a measurement.

Do not overstate. Cheapest per token is not necessarily cheapest per finished task. A weaker model may need more turns or leave mistakes, and rework costs more than tokens. Do not choose a model by price without measuring quality. The post Measuring an AI feature covers measuring quality, cost and trust together.

Do not dismiss it. Cost is a real axis of daily agent use, and usage-based pricing is spreading. Put cost into your team's one-page AI use policy, together with asking for evidence as in Make AI Output Prove Itself, so saving money does not lower quality.

Do not use it to compare colleagues. Token use depends on the kind of work and does not measure a person's productivity.

---

11. Conclusion#

The AI bill does not grow with the number of sentences you type, but with how many times the history is reread multiplied by how thick it is. Three things keep cost within reach: open a new session at the boundary between tasks, use the right size of model for each task, and set caps and check usage every week. Run the script once with your own numbers, then compare it with your real usage page.

---

References#

Figures come from public sources as of 30 October 2026. Prices and billing rules change, so open the originals before quoting.

• Anthropic, Prompt caching (cache write and read factors, minimum length, default lifetime, example price table; read through a page-summarizing tool)
• Gartner, Gartner Predicts AI Coding Costs Will Surpass Average Developer's Salary by 2028 as Token Consumption Surges (24 June 2026, the original page returned a 403 error when I opened it) and the reprint on DevOps Digest (the source I actually read)
• GitHub, GitHub Copilot is moving to usage-based billing (27 April 2026, read through a page-summarizing tool; secondary sources give different values for one credit, so I do not use it)
• Petrov, La Malfa, Torr, Bibi, Language Model Tokenizers Introduce Unfairness Between Languages (2023)
• Per-developer cost ranges from blogs of vendors that sell cost-management tools, as a rough reference only: CloudZero, Eden AI