How ChatGPT actually works: tokens, context windows, fast vs thinking models and why answers vary

Most owners first use ChatGPT like a search box, and then lose trust when it confidently gives a wrong GST due date or forgets a price you told it twenty messages ago. Both failures are predictable once you understand what the tool is doing underneath. This lesson gives you a working mental model — tokens, context windows, model choice and randomness — so you can tell in advance when ChatGPT will save your team hours and when it will quietly cost you money.

What you need to know

It predicts text; it does not look things up. ChatGPT runs on a large language model trained on a very large amount of text to predict what comes next. When you ask something, it produces the most plausible continuation of your words based on patterns it learned. That is why it writes fluently in seconds, and also why it can produce an answer that sounds right but is false. A fluent sentence is not a checked fact. Unless web search is switched on and actually used in that reply, the answer comes from its training, which stops at a cut-off date.

Tokens are the unit behind everything. The model reads and writes in tokens — small chunks of words. A common rule of thumb for English is that one token is about four characters, or roughly three-quarters of a word. Hindi, Marathi, Tamil and other Indian scripts often need more tokens for the same meaning. Tokens matter because the amount the model can hold in one conversation, your plan's usage limits and any API charges are all counted in them.

Quick arithmetic you will use often:

• A 20-page supplier agreement at about 500 words per page = 10,000 words. At 0.75 words per token, that is 10,000 ÷ 0.75 ≈ 13,300 tokens.
• A 60-page tender document at 450 words per page = 27,000 words ≈ 36,000 tokens.

The context window is its working memory. Everything in the current chat — your instructions, the text of files it reads, its own earlier replies — has to fit into a context window measured in tokens. Different models and plans have different limits, and OpenAI changes them, so check the current figure in OpenAI's help pages rather than relying on a number you read somewhere. In very long chats, earlier material can be summarised, dropped or simply given less attention. Practical rules that follow:

• One job per chat. Start a fresh chat when you change topic.
• In long chats, restate the key facts (price list, customer name, deadline) right next to your request.
• For long documents, ask about specific sections instead of "review everything".

Fast models and thinking models. ChatGPT offers more than one model, and the line-up and names change frequently. Broadly there are models tuned to respond quickly — good for drafting, rewording, translation and short replies — and reasoning or "thinking" models that work through a problem in steps before answering — better for analysis, comparisons, calculations and planning. Some plans switch automatically; others let you pick. A decision rule:

• Rewriting, emails, captions, first drafts: fast model.
• Comparing three vendor quotations, checking pricing logic, planning a 12-week project, reviewing a spreadsheet formula: thinking model.
• Anything with numbers you will act on: thinking model, and you still check the arithmetic yourself.

Why the same question gets different answers. Generation contains an element of chance, so two runs of the same prompt rarely match word for word. This is useful when you want options ("give me five subject lines") and a problem when you want consistency (a standard reply to a refund request). You reduce variation with precise instructions, fixed output templates and examples of what good looks like — covered in lessons 05 and 08.

Knowledge cut-off versus search. Built-in knowledge stops at a date. For anything that changes — tax rates, platform commissions, government scheme rules, interest rates, prices — either use search and open the cited sources, or go straight to the official portal (GST portal, CBIC, DGFT, RBI, Udyam). Treat an unsourced answer about a rate or rule as a lead to verify, never as the answer.

Tools change how much you can trust the output. Depending on your plan, ChatGPT can search the web, read uploaded files, run code for data analysis and create images. When data analysis is used, calculations are done by running code rather than by predicting digits in text, which is far more reliable for totals and percentages. When you see it searching, you get sources you can open.

Where it is strong and where it is weak.

• Strong: first drafts, rewriting for a different reader, summarising text you supply, turning messy notes into structure, brainstorming, explaining a concept simply.
• Weak without help: current facts, exact figures, very local knowledge (a specific mandi rate, a municipal rule in your ward), anything about your business it has not been told, and legal or tax conclusions.

Step-by-step method

1. List the ten tasks you and your team would most like help with this month — replies, quotations, reports, research, translations.
2. Label each task as Drafting, Analysis, Current facts, or Needs private data.
3. For each Drafting task, note that a fast model is enough. For each Analysis task, note that you will use a thinking model and check the numbers.
4. For each Current facts task, write the official source you will verify against (for example, the GST portal for return due dates).
5. For any document you plan to paste, estimate pages × words per page ÷ 0.75 to get tokens. If it is large, plan to work section by section.
6. Run a consistency test: send the same prompt in three new chats and compare. Note what varies — tone, length, facts.
7. Run a cut-off test: ask about a recent change in your industry without search, then with search, and compare both answers with an official source.
8. Write a one-page "trust card": what you let ChatGPT do directly, what you always verify, and what you never use it for.

My ChatGPT trust card

• Tasks I let it draft directly (fast model): __, , __
• Tasks where I use a thinking model and check numbers: __, __
• Facts I always verify, and where: __ → (portal) ; → __
• Tasks that need my private data (see lesson 03 first): ____
• Largest document I work with: __ pages ≈ __ tokens → section by section? yes / no
• Consistency test done (3 runs)? yes / no — what varied: ____
• Cut-off test done? yes / no — result: ____
• I start a new chat when: ____
• Things I never use ChatGPT for: ____

Common mistakes

• Running one endless chat for the whole week and then wondering why it "forgot" the price list from Monday.
• Asking for a current GST rate, TDS threshold or platform fee without search and copying the answer into an invoice or a customer message.
• Using a fast model to compare quotations with many numbers and trusting its totals without recalculating.
• Pasting a very long document and asking "any problems?" — you get a vague, generic list instead of specific, checkable points.
• Judging the whole tool on one bad answer, when the real cause was a vague prompt or the wrong model.
• Assuming a polished, confident tone means the content has been checked.

Ask the AI Business Tutor

• "I run a [type of business] in [city] with [number] staff. Here are ten tasks I want help with: [list]. Classify each as drafting, analysis, current facts or private data, suggest whether a fast or thinking model suits it, and tell me which official source I should verify each current-fact task against."