# LLM APIs in TypeScript

> Call Claude from TypeScript with the official SDK: messages, system prompts, history, streaming, stop reasons, errors, retries and cost.

AI Engineer roadmap · LLM foundations · 20 min read · https://levelcode.org/roadmaps/ai-engineer/llm-apis-typescript

Every AI feature you build starts with the same move: your code sends a request to a model and handles what comes back. This lesson teaches that move properly in TypeScript, using Anthropic's official `@anthropic-ai/sdk` package, on Node and on edge runtimes. You will set up a client, send messages with a system prompt, keep a multi-turn conversation, stream responses, read usage and stop reasons, handle errors and retries, and estimate cost. Other providers' APIs follow a very similar shape, so the concepts transfer. (If you build in Python, the Python version of this lesson covers the same ground.)

## Setup: SDK, key and client

You need Node 22+ (a supported LTS release) and an API key from the Claude Console. Create a project and install the SDK, plus `tsx` to run TypeScript files directly:

```bash
mkdir llm-demo && cd llm-demo
npm init -y
npm pkg set type=module
npm install @anthropic-ai/sdk
npm install -D typescript tsx @types/node
```

**Never put the key in your source code.** Export it as an environment variable, which the SDK reads automatically:

```bash
export ANTHROPIC_API_KEY="sk-ant-..."
```

In a real project, keep the key in your deployment platform's secret store, or in a `.env` file that is listed in `.gitignore`. Now create a client. One client per process is enough because it reuses HTTP connections, so create it once at module level:

```ts
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic(); // reads process.env.ANTHROPIC_API_KEY
const MODEL = "claude-opus-5-5";
```

`claude-opus-5-5` is the most capable current Opus model. `claude-sonnet-5-5` and `claude-haiku-5-5` are cheaper and faster options. Use model IDs exactly as written, with no date suffix.

> [!WARNING]
> This code belongs on a server, never in browser JavaScript. Anything shipped to a browser can be read by its user, including your key. The SDK refuses to run in a browser by default for exactly that reason; don't override it. Have the browser call your own backend instead.

## Your first request

The core endpoint is **Messages**. You send a list of messages and get one assistant message back. Save this as `first.ts` and run `npx tsx first.ts`:

```ts
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();
const MODEL = "claude-opus-5-5";

const response = await client.messages.create({
  model: MODEL,
  max_tokens: 1024,
  messages: [{ role: "user", content: "In one sentence, what is a context window?" }],
});

console.log(response.stop_reason, response.usage.input_tokens, response.usage.output_tokens);
for (const block of response.content) {
  if (block.type === "text") console.log(block.text);
}
```

```text
end_turn 19 58
A context window is the maximum amount of text, measured in tokens, that a model can take into account at once, including both your input and its response.
```

Three parameters are required:

- `model`: which model to run.
- `max_tokens`: a hard cap on how many tokens the model may generate in this response, including any thinking. It is a ceiling, not a target. If the model hits it, the response is cut off mid-sentence.
- `messages`: the conversation so far. The first message must have role `user`.

`response.content` is a **discriminated union array** of content blocks, not a string. Current models think before answering, so a `thinking` block may come before the `text` block, and later lessons add `tool_use` blocks. TypeScript won't let you read `content[0].text` without narrowing, and that's a good thing. A helper with a type guard keeps call sites clean:

```ts
function textOf(message: Anthropic.Message): string {
  return message.content
    .filter((block): block is Anthropic.TextBlock => block.type === "text")
    .map((block) => block.text)
    .join("");
}
```

You can tune how hard the model thinks with `output_config: { effort: ... }`, using `"low"`, `"medium"`, `"high"`, `"xhigh"` or `"max"`. Low effort is faster and cheaper for simple tasks; higher effort helps on hard ones. Some older tutorials pass `temperature` or a thinking `budget_tokens`, or pre-fill the start of the assistant's reply. Current Claude models reject all three with a 400 error, so leave them out.

## System prompts and conversations

The `system` parameter sets the model's role, rules and context for the whole conversation. It is separate from `messages`:

```ts
const reply = await client.messages.create({
  model: MODEL,
  max_tokens: 1024,
  system: "You are a concise assistant for a cycling shop. Answer in at most three sentences.",
  messages: [{ role: "user", content: "Do I need tubeless tyres?" }],
});
console.log(textOf(reply));
```

The API is **stateless**. The model remembers nothing between calls, so to hold a conversation you send the whole history every time, alternating `user` and `assistant` turns. Type the history with the SDK's `Anthropic.MessageParam` rather than inventing your own interface:

```ts
const history: Anthropic.MessageParam[] = [
  { role: "user", content: "My name is Priya and I ride gravel." },
  { role: "assistant", content: "Nice to meet you, Priya! Gravel is great fun." },
  { role: "user", content: "What tyre width would you suggest for me?" },
];
```

Wrap the bookkeeping in a class. Two details matter. First, append the assistant's **entire** `response.content` (the block array), not just its text, so the history is exactly what the model produced. Second, only commit the new turns after the call succeeds, so a failed request doesn't leave a dangling user message:

```ts
class Chat {
  private history: Anthropic.MessageParam[] = [];

  constructor(
    private readonly system: string,
    private readonly model: string = MODEL,
  ) {}

  async send(userText: string): Promise<string> {
    const userTurn: Anthropic.MessageParam = { role: "user", content: userText };
    const response = await client.messages.create({
      model: this.model,
      max_tokens: 2048,
      system: this.system,
      messages: [...this.history, userTurn],
    });
    this.history.push(userTurn, { role: "assistant", content: response.content });
    return textOf(response);
  }
}

const chat = new Chat("You are a friendly cycling coach.");
console.log(await chat.send("My name is Priya."));
console.log(await chat.send("What's my name?")); // works because the history was re-sent
```

> [!IMPORTANT]
> Because every call re-sends the whole history, a long conversation costs more with each turn. Turn 50 pays for turns 1 to 49 again. Keep the history append-only (don't rewrite earlier turns or the system prompt between calls), and cap or summarise very long conversations.

## Reading the response: stop reasons and usage

Before you use a response, check **why the model stopped**:

| `stop_reason` | Meaning | What to do |
| --- | --- | --- |
| `end_turn` | Finished naturally | Use the content |
| `max_tokens` | Hit your `max_tokens` cap | Content is truncated: raise the cap or ask for less |
| `stop_sequence` | Generated one of your `stop_sequences` | Use the content |
| `refusal` | Declined for safety reasons | Don't parse the content as an answer; show a fallback |
| `model_context_window_exceeded` | Ran out of context window | Shorten or summarise the input |
| `tool_use` | Wants to call a tool | Covered in the tool use lesson |

```ts
function answerOrThrow(response: Anthropic.Message): string {
  switch (response.stop_reason) {
    case "end_turn":
    case "stop_sequence":
      return textOf(response);
    case "max_tokens":
      throw new Error("Response truncated; raise max_tokens or ask for less");
    case "refusal":
      return "Sorry, I can't help with that request.";
    default:
      throw new Error(`Unexpected stop_reason: ${response.stop_reason}`);
  }
}
```

`response.usage` reports what you are billed for. `input_tokens` covers the uncached input, `output_tokens` covers everything generated (thinking included), and `cache_creation_input_tokens` and `cache_read_input_tokens` report prompt caching, which the cost lesson covers. Log usage for every call. It is your only reliable view of spend.

## Streaming, on Node and at the edge

A long answer can take many seconds to finish. **Streaming** delivers text as it is generated. `client.messages.stream()` returns a `MessageStream` you can iterate, plus `finalMessage()` for the assembled result:

```ts
const stream = client.messages.stream({
  model: MODEL,
  max_tokens: 4096,
  system: "You are a concise technical writer.",
  messages: [{ role: "user", content: "Explain HTTP caching in two paragraphs." }],
});

stream.on("text", (delta) => process.stdout.write(delta));
const final = await stream.finalMessage();
console.log(`\n[${final.stop_reason}, ${final.usage.output_tokens} output tokens]`);
```

`finalMessage()` resolves to the same `Anthropic.Message` you would get without streaming (or rejects if the stream fails), so you can still check `stop_reason`, read `usage` and push `final.content` onto your history. Stream anything user-facing, and anything with a large `max_tokens`: for requests it estimates will run very long, the SDK refuses to make a non-streaming call and throws an error telling you to stream.

The SDK is built on the standard `fetch` API, so it also runs on edge and serverless runtimes such as Cloudflare Workers, Deno and Bun. There is often no `process.env`, so pass the key explicitly. Here is a Workers-style handler that streams plain text to the browser:

```ts
import Anthropic from "@anthropic-ai/sdk";

interface Env {
  ANTHROPIC_API_KEY: string;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const client = new Anthropic({ apiKey: env.ANTHROPIC_API_KEY });
    const { question } = (await request.json()) as { question: string };

    const stream = client.messages.stream({
      model: "claude-sonnet-5-5",
      max_tokens: 1024,
      messages: [{ role: "user", content: question }],
    });

    const encoder = new TextEncoder();
    const body = new ReadableStream<Uint8Array>({
      async start(controller) {
        try {
          for await (const event of stream) {
            if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
              controller.enqueue(encoder.encode(event.delta.text));
            }
          }
          controller.close();
        } catch (err) {
          controller.error(err);
        }
      },
    });
    return new Response(body, { headers: { "content-type": "text/plain; charset=utf-8" } });
  },
};
```

## Errors, retries and timeouts

Network calls fail. The SDK throws typed errors, all subclasses of `Anthropic.APIError`. Check them with `instanceof`, most specific first, and never by matching message text:

| Error class | Cause | Retry? |
| --- | --- | --- |
| `Anthropic.APIConnectionError` (and `APIConnectionTimeoutError`) | Network failure or timeout | Yes |
| `Anthropic.RateLimitError` (429) | Too many requests or tokens per minute | Yes, after a delay |
| `Anthropic.InternalServerError` (any 5xx, including 529 overloaded) | Server-side problem | Yes |
| `Anthropic.BadRequestError` (400) | Invalid request: bad parameter, malformed history | No, fix the code |
| `Anthropic.AuthenticationError` (401), `PermissionDeniedError` (403) | Bad key or missing permission | No |
| `Anthropic.NotFoundError` (404) | Wrong model ID or endpoint | No |

The SDK **already retries** connection errors, 408, 409, 429 and 5xx responses twice, with exponential backoff, honouring the server's `retry-after` header. Tune it on the client, or per request through the second argument:

```ts
const tunedClient = new Anthropic({ maxRetries: 4, timeout: 60_000 }); // timeout in ms

const quick = await tunedClient.messages.create(
  {
    model: "claude-haiku-5-5",
    max_tokens: 1024, // leave room for thinking; it counts toward max_tokens
    messages: [{ role: "user", content: "Reply with OK." }],
  },
  { maxRetries: 0, timeout: 10_000 },
);
```

Your own code decides what happens once retries run out. Catch the specific errors, show a useful message, and log the request ID so you can trace the failure:

```ts
async function safeAsk(question: string): Promise<string> {
  try {
    const response = await client.messages.create({
      model: MODEL,
      max_tokens: 1024,
      messages: [{ role: "user", content: question }],
    });
    return answerOrThrow(response);
  } catch (err) {
    if (err instanceof Anthropic.RateLimitError) {
      return "We're busy right now. Please try again in a minute.";
    }
    if (err instanceof Anthropic.APIConnectionError) {
      return "Couldn't reach the AI service. Check your connection.";
    }
    if (err instanceof Anthropic.APIError) {
      console.error(`Claude API error ${err.status} (request ${err.requestID}): ${err.message}`);
      return "Something went wrong on our side.";
    }
    throw err; // not an API error: a bug in our code
  }
}
```

Every specific class extends `APIError`, so it must be checked last. Don't wrap 400-class errors in a retry loop. Sending the same invalid request again will fail the same way.

## Cost math

You pay per million tokens (MTok), with separate input and output prices:

| Model | Input $/MTok | Output $/MTok |
| --- | --- | --- |
| `claude-opus-5-5` | 4.00 | 20.00 |
| `claude-sonnet-5-5` | 2.00 | 10.00 |
| `claude-haiku-5-5` | 0.10 | 0.50 (prompts up to 100K tokens) |

```ts
const PRICES: Record<string, { input: number; output: number }> = {
  "claude-opus-5-5": { input: 4.0, output: 20.0 },
  "claude-sonnet-5-5": { input: 2.0, output: 10.0 },
  "claude-haiku-5-5": { input: 0.1, output: 0.5 },
};

function costUsd(model: string, usage: Anthropic.Usage): number {
  const price = PRICES[model];
  if (!price) throw new Error(`No price for ${model}`);
  return (usage.input_tokens * price.input + usage.output_tokens * price.output) / 1_000_000;
}
```

For a call with 1,500 input and 600 output tokens on Opus 5.5, that is `1,500 × 4 / 1M + 600 × 20 / 1M = $0.006 + $0.012 = $0.018`. Output dominates even though there are fewer output tokens, because each one costs five times more. To estimate before sending, count the tokens (the endpoint is free):

```ts
const count = await client.messages.countTokens({
  model: MODEL,
  system: "You are a concise technical writer.",
  messages: [{ role: "user", content: "Explain HTTP caching in two paragraphs." }],
});
console.log(count.input_tokens);
```

Don't estimate Claude token counts with another provider's tokenizer package; counts differ between model families.

## Practice

Build `chat.ts`, a terminal chat program that:

1. takes a system prompt from the first command-line argument (with a sensible default);
2. streams each reply as it is generated;
3. keeps the full conversation history;
4. after each reply, prints the stop reason, the tokens used and the running cost in dollars;
5. warns if a reply was truncated or refused;
6. survives API errors without crashing or corrupting the history, and exits cleanly on `quit` or Ctrl-C.

**Solution**

```ts
import Anthropic from "@anthropic-ai/sdk";
import * as readline from "node:readline/promises";
import { stdin as input, stdout as output } from "node:process";

const MODEL = "claude-sonnet-5-5";
const PRICES: Record<string, { input: number; output: number }> = {
  "claude-opus-5-5": { input: 4.0, output: 20.0 },
  "claude-sonnet-5-5": { input: 2.0, output: 10.0 },
  "claude-haiku-5-5": { input: 0.1, output: 0.5 },
};

function costUsd(model: string, usage: Anthropic.Usage): number {
  const price = PRICES[model];
  if (!price) throw new Error(`No price for ${model}`);
  return (usage.input_tokens * price.input + usage.output_tokens * price.output) / 1_000_000;
}

const system = process.argv[2] ?? "You are a helpful, concise assistant.";
const client = new Anthropic();
const history: Anthropic.MessageParam[] = [];
let total = 0;

const rl = readline.createInterface({ input, output, prompt: "\nyou> " });
rl.on("SIGINT", () => rl.close()); // Ctrl-C ends the loop below
rl.prompt();

for await (const line of rl) {
  const userText = line.trim();
  if (userText === "quit" || userText === "exit") break;
  if (!userText) {
    rl.prompt();
    continue;
  }

  const userTurn: Anthropic.MessageParam = { role: "user", content: userText };
  try {
    output.write("claude> ");
    const stream = client.messages.stream({
      model: MODEL,
      max_tokens: 4096,
      system,
      messages: [...history, userTurn],
    });
    stream.on("text", (delta) => output.write(delta));
    const final = await stream.finalMessage();

    history.push(userTurn, { role: "assistant", content: final.content });
    const callCost = costUsd(MODEL, final.usage);
    total += callCost;
    console.log(
      `\n[${final.stop_reason} | in ${final.usage.input_tokens} / out ${final.usage.output_tokens} tokens` +
        ` | $${callCost.toFixed(4)} this turn, $${total.toFixed(4)} total]`,
    );
    if (final.stop_reason === "max_tokens") console.log("[warning: reply was truncated]");
    if (final.stop_reason === "refusal") console.log("[warning: the model declined this request]");
  } catch (err) {
    if (!(err instanceof Anthropic.APIError)) throw err;
    console.log(`\n[API error: ${err.message}; turn discarded, try again]`);
  }
  rl.prompt();
}

rl.close();
console.log(`\nBye. Session cost: $${total.toFixed(4)}`);
```

```bash
npx tsx chat.ts "You are a patient TypeScript tutor."
```

Iterating `rl` with `for await` buffers lines while a reply is streaming and ends cleanly when input closes (Ctrl-D, or `rl.close()` from the SIGINT handler). The user turn is only pushed onto `history` after `finalMessage()` resolves, so a failed request is discarded and the next request is still well-formed. The SDK has already retried transient errors before the `catch` sees them. Watch the "in" token count climb each turn: that is the cost of re-sending history.

## Key takeaways

- Create one `new Anthropic()` client per process. It reads `ANTHROPIC_API_KEY` from the environment (pass `apiKey` explicitly on edge runtimes), and it never belongs in browser code.
- `messages.create` needs `model`, `max_tokens` and `messages`. `content` is a typed union array, so narrow on `block.type`.
- The API is stateless: send the full `Anthropic.MessageParam[]` history each call, push the assistant's whole `content`, and commit turns only after success.
- Check `stop_reason` (`max_tokens`, `refusal` and others) before trusting content, and log `usage` on every call.
- Stream with `messages.stream()` and `finalMessage()`. The same code works on Node and fetch-based edge runtimes.
- The SDK retries transient errors (connection, 429, 5xx) for you. Narrow errors with `instanceof`, never retry 400s, and compute cost as tokens × price per million.

## Quiz

1. Your chatbot forgets everything the user said in earlier turns. What is the most likely bug?

   A. The client was created without a session ID
   B. max_tokens is too low
   C. Each request sends only the latest user message instead of the full conversation history
   D. The system prompt is missing

   **Answer: C.** The Messages API is stateless. The model only knows what is in the current request, so you must re-send every earlier user and assistant turn. There is no server-side session to enable.

2. Why does TypeScript reject console.log(response.content[0].text)?

   A. content is a union of block types (text, thinking, tool_use and more), and only text blocks have .text, so you must narrow on block.type first
   B. content is a string, not an array
   C. The SDK returns content as a Promise
   D. text is a private field

   **Answer: A.** content is a discriminated union array. The first block may be a thinking block or a tool_use block, so the compiler forces you to check block.type === "text" before reading .text, which also prevents a real runtime bug.

3. In a catch block, you check err instanceof Anthropic.APIError before err instanceof Anthropic.RateLimitError. What happens on a 429?

   A. The RateLimitError branch runs because it is more specific
   B. The generic APIError branch runs, because RateLimitError is a subclass of APIError and the first matching check wins
   C. Neither branch runs; 429s are swallowed by the SDK
   D. TypeScript refuses to compile the code

   **Answer: B.** instanceof is true for subclasses, so the first check matches every API error. Order checks from most specific (RateLimitError, APIConnectionError) to least specific (APIError).

4. A call on claude-sonnet-5-5 ($2 input / $10 output per million tokens) uses 10,000 input tokens and 1,000 output tokens. What does it cost?

   A. $0.012
   B. $0.12
   C. $0.002
   D. $0.03

   **Answer: D.** Input: 10,000 x $2 / 1,000,000 = $0.02. Output: 1,000 x $10 / 1,000,000 = $0.01. Total $0.03. Output tokens are priced five times higher, so they matter even when there are fewer of them.