Learn AI applications by measuring simple baselines

Your starting point

You can work with small lists of text. No paid model account or live AI service is needed for these first projects.

• Functions, arrays or lists, and basic file input
• Willingness to label a small set of examples consistently

Your finish line

You can define an AI task, compare it with a baseline, preserve evidence, and inspect failures before adding model access.

Stack up four small wins

1. STEP 1 OF 4

   Define one task and its errors

   Choose three synthetic support categories and write examples that should match or be marked uncertain.

   Bro tip: Include an uncertain example beside the obvious categories. Ask someone to apply the written label rules before showing them your code.

   What you’ll learn

   • Input and output contracts
   • Labels and unsupported cases
   • Baseline behavior
   You’re ready to move on when

   Another person can apply your label rules to the same examples.

   I can do this step

2. STEP 2 OF 4

   Measure a deterministic baseline

   Build the local support-message triage project and compare rules with a majority-label baseline.

   Bro tip: Read a few wrong answers before changing a rule. Keep the final held-out cases separate while you work on development examples.

   What you’ll learn

   • Held-out examples
   • Confusion matrices
   • Error inspection
   You’re ready to move on when

   The report includes failures and does not tune on its final held-out examples.

   Open project: Support-message triage baseline

   I can do this step

3. STEP 3 OF 4

   Retrieve evidence before generating

   Build a local finder over a few invented help documents and show the passages behind each result.

   Bro tip: Try one question the documents cannot answer. Inspect the empty result as carefully as a match, without filling the gap yourself.

   What you’ll learn

   • Document chunks
   • Lexical ranking
   • Source identifiers
   You’re ready to move on when

   An unsupported question produces a useful empty result instead of an invented answer.

   Open project: Local evidence finder

   I can do this step

4. STEP 4 OF 4

   Compare changes with a repeatable evaluation

   Build an evaluation workbench accepting saved or manually entered outputs, then optionally connect a model later.

   Bro tip: Inspect one case that got worse even if the average improved. Keep the cases and scoring rules fixed for both candidate sets.

   What you’ll learn

   • Rubrics
   • Dataset versions
   • Baseline versus candidate
   You’re ready to move on when

   Two candidate sets are scored with the same cases and rules, with regressions visible.

   Open project: AI evaluation workbench

   I can do this step

Your build lineup

1. Support-message triage baseline

   Classify invented support messages with simple rules and measure where those rules fail.

2. Local evidence finder

   Search a few invented help documents and show the exact passages supporting each result.

3. AI evaluation workbench

   Compare saved candidate outputs against the same cases and an explicit human-readable rubric.

Watch it. Then build it.

Start with a worked example, then connect the recording to your own small, checkable build. Videos open on their creator’s or publisher’s website.

Read Vibe Bro’s written lesson briefing ↗

1. Beginner · Video course · About 2 hours

   Machine Learning for Everybody – Full Course

   freeCodeCamp.org · Kylie Ying

   Introduces features, classification, regression, preparing data, and several supervised and unsupervised learning methods.

   For step 1: Define one task and its errors

   Before watching: Basic Python functions and lists for the coding examples; school algebra and graph reading help with the explanations. Start with the introductory sections.

   Watch, then try

   Identify the input features, expected label, and possible error for your support-message task. Write the three label rules and measure a majority-label baseline before experimenting with a fitted model.

   The demonstrations use Python and Colab, with later neural-network sections. Our starter begins with local rules and saved examples; model training and hosted tools are optional extensions.

   Watch course ↗ · Practice: Support-message triage baseline

   Publisher’s course details · checked October 28, 2026

2. Beginner · Video

   Machine Learning Fundamentals: The Confusion Matrix

   StatQuest with Josh Starmer

   Explains how classification outcomes become a confusion matrix, including examples with more than two classes.

   For step 2: Measure a deterministic baseline

   Before watching: Basic counting and the distinction between a predicted label and a recorded label. Reviewing train/test separation and cross-validation first is helpful.

   Watch, then try

   Build a three-category matrix from your held-out support messages. Label its actual and predicted axes, inspect the off-diagonal errors, and compare your rules with the majority-label baseline on the same cases.

   This is a conceptual lesson with worked examples. Implement the table in your existing language and keep individual failed cases visible alongside the aggregate counts.

   Watch video ↗ · Practice: Support-message triage baseline

Video links selected October 28, 2026. Selection checks cover publisher metadata and topic fit.

The docs behind it

These references explain the underlying concepts and APIs. The prompt templates and project plans are original examples to adapt to your own setup.

• scikit-learn — Dummy classifier baselines
• scikit-learn — Classification metrics and scoring
• scikit-learn — Text feature extraction

Documentation links checked October 27, 2026.

Keep the momentum going

• AI fallback policyCoding term
• Build a representative model evaluationCopyable prompt
• Confusion matrixCoding term