The Model Era, 2022

What you will do

1. see which local models are installed ollama list

   A local model runner (ollama-style) keeps a small registry of downloaded model files on disk — nothing installed yet is normal.

2. download a model into the local registry ollama pull atlas-7b

   Pulling a model fetches its weights once; after that it runs entirely offline, with no per-request network call.

3. confirm the model is installed, and read its quantization ollama list

   Q4_0 means the weights are compressed to ~4 bits per parameter instead of the original 16/32 — smaller and faster, at a small accuracy cost. This is quantization.

4. run inference locally and get a reply ollama run atlas-7b "hello"

   Local inference means the prompt and the reply never leave this machine — relevant for privacy and for offline use.

5. see which models are currently loaded in memory ollama ps

   A pulled model sits on disk; a running one is loaded into RAM/VRAM. Loading is the slow part — that is why runners keep recently-used models warm.

6. delete a model to free disk space ollama rm atlas-7b

   Multi-gigabyte model files add up fast — rm is the same trade-off as uninstalling any large local app.

7. pull a second, smaller model ollama pull nimbus-3b

   A 3B-parameter model is a fraction of the size of a 7B one — faster and lighter, usually at a real quality cost. Picking a model is a size/quality/speed trade-off.

8. pull a larger model and compare its footprint ollama pull forge-13b

   Bigger parameter counts generally mean better quality and slower, hungrier inference — there is no free lunch, only a trade-off you get to choose.

9. weigh local models against a hosted API compare

   Local vs hosted is not "better/worse" — it is a trade of cost, latency, privacy and control against convenience and raw capability.

10. see the shape of a hosted chat-completions request/response curl -X POST /v1/chat/completions

    Nearly every hosted LLM API converged on the same rough JSON shape: a "messages" array in, a "choices[0].message.content" reply out, plus a token-count "usage" block.

11. see what a malformed request looks like curl -X POST /v1/chat/completions -d '{}'

    APIs validate their schema before ever touching the model — a missing required field fails fast with a 400, not a wasted (and billed) generation.

12. see text broken into tokens, not words tokens "the quick brown fox"

    Costs and context limits are counted in TOKENS. Tokens are sub-word pieces — sometimes a whole word, sometimes a fragment of one.

13. see tokenizer quirks on numbers and long words tokens "12345 unbelievably"

    Tokenizers often split numbers into digit groups and long/rare words into pieces — which is why arithmetic on big numbers, and rare vocabulary, can trip a model up in ways short common words never do.

14. understand the context window and why it is not free to grow ctx

    Self-attention (popularised by a landmark 2017 paper) lets every token look at every other token — powerful, but its cost grows roughly with the square of the sequence length.

15. see how system / user / assistant roles shape a conversation roles

    Around 2022, instruction-tuning and human feedback turned raw next-token predictors into assistants that reliably follow the system role's instructions.

16. teach a model a pattern with examples, not rules fewshot

    Few-shot prompting updates no weights at all — the "learning" lives entirely in the examples sitting in the prompt for that one request.

17. see deterministic sampling: temperature 0 temp 0.0

    temperature=0.0 always takes the single most probable next token — same input, same output, every time. Useful for tests, code, and structured data.

18. contrast with creative sampling: temperature 1 temp 1.0

    temperature (and top_p, nucleus sampling) control how much of the probability distribution below the top choice gets a chance to be picked — higher means more variety, less predictability.

19. force a reply into strict, parseable JSON json

    Structured output / JSON mode constrains generation to a schema, so an app can trust response.name and response.age exist instead of regex-scraping prose.

20. see a model request a tool instead of guessing an answer tool call

    Function/tool calling: the model emits structured INTENT (a name + JSON arguments) — it never executes anything itself. Your application decides whether, and how, to actually run it.

21. complete the round trip: tool output back to the model tool result

    request -> tool_call -> your code executes the real function -> the result is appended as a message -> the model writes the final answer using real data instead of a guess.

22. turn text into a vector embed "sofia bulgaria"

    An embedding places text at a point in a high-dimensional space where geometric distance approximates semantic distance — similar meaning, nearby points.

23. measure semantic distance with cosine similarity sim "king" "queen"

    Cosine similarity measures the angle between two vectors, ignoring their length — it is the standard way to compare embeddings for "how related are these two meanings."

24. ask a question before any document has been retrieved — watch it guess rag query "what is your return policy?"

    A model with no retrieval is answering from patterns in its training data, not from YOUR actual documents. A confident-sounding wrong answer is a hallucination — and this is exactly what grounding exists to prevent.

25. see prompt injection as a security problem, and how to defend against it inject

    Any text an app pulls in (a scraped page, a user upload, a tool result) can contain words that LOOK like instructions. Treating untrusted content as data, never as commands, is the core defense.

26. see a reply arrive token by token instead of all at once stream

    Streaming lets the client render the first token almost immediately instead of waiting for generation of the entire reply to finish — a real latency-vs-perceived-latency trick.

27. measure whether a model is actually good enough eval

    Exact-match scoring works for facts and code. Open-ended writing usually needs an LLM-as-judge with a rubric, or human raters — evaluation is rarely one single number.

28. add a real document to the retrieval corpus — RAG stage 1 rag ingest returns-policy.md

    Retrieval-Augmented Generation starts by loading source documents the model should be allowed to actually cite, instead of relying on what it memorised during training.

29. split ingested documents into retrievable pieces — RAG stage 2 rag chunk

    Chunking trades granularity for context: chunks too large waste the context window and dilute relevance; chunks too small lose surrounding meaning.

30. embed every chunk into a vector store — RAG stage 3 rag embed

    Once every chunk has a vector, retrieval at query time becomes a nearest-neighbour search over those vectors — no re-reading the whole corpus per question.

31. Boss missionrun the full pipeline end to end — ingest, chunk, embed, then a grounded, cited answer rag query "how many days do i have to return an item?"

    This is the entire RAG loop: ingest -> chunk -> embed -> retrieve the most relevant chunk at query time -> answer grounded in, and citing, that retrieved text instead of a guess.

Certificate

Nearby eras

All 25 eras in the Terminal Academy