diff options
| -rw-r--r-- | TESTING.md | 329 |
1 files changed, 329 insertions, 0 deletions
diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 0000000..7178006 --- /dev/null +++ b/TESTING.md @@ -0,0 +1,329 @@ +# Manual Test Cases + +Run these against the extension loaded from `Zig.novaextension/` with Nova in +development mode. Each case lists the example project to open (under `examples/`), +the steps to follow, and the expected result. + +--- + +## Setup + +Before running any case: + +1. Confirm the extension is active: **Nova → Extensions → Extension Library → + Zig** shows as installed and enabled. +2. Confirm `zig` and `zls` are on `PATH` (or configured in global preferences). +3. Unless stated otherwise, open the project folder directly as a Nova workspace + (not a single file). + +--- + +## T01 — Activation + +**Project:** any `.zig` file or workspace containing one + +| Step | Expected | +|---|---| +| Open a `.zig` file cold (no prior Nova session) | Extension Console shows ZLS starting; no error dialogs | +| Check the task list | **Current Zig File** is present | +| Open a workspace with `build.zig` | After a few seconds, **Zig Build: \<step\>** tasks appear | +| Open a folder with no `.zig` files | Extension does not activate; no ZLS process | + +--- + +## T02 — Syntax highlighting + +**Project:** `examples/snippets` + +Open each file and verify highlighting is applied: + +| File | Check | +|---|---| +| `syntax-tour.zig` | Keywords (`fn`, `const`, `pub`), string literals, comments, numeric literals all coloured distinctly | +| `fibonacci.zig` | Function signatures, control flow (`if`, `while`), types (`u64`) highlighted | +| `error-example.zig` | Error set declarations, `try`, `catch` highlighted | +| `examples/zon-examples/full-types.zon` | `.field = value` pairs highlighted; strings and numbers coloured | +| `examples/zon-examples/invalid.zon` | File opens without Nova crashing | + +--- + +## T03 — Code folding + +**Project:** `examples/snippets/syntax-tour.zig` + +| Step | Expected | +|---|---| +| Click the fold widget next to a `fn` body | Function body collapses | +| Click the fold widget next to a `struct` or `union` | Declaration collapses | +| Click the fold widget next to a `if`/`for`/`while` block | Block collapses | + +--- + +## T04 — ZLS: completion and hover + +**Project:** `examples/multi-step` (open workspace root) + +| Step | Expected | +|---|---| +| Open `src/main.zig`, type `std.` | Completion popover appears with `std` members | +| Hover over `std.debug.print` | Documentation tooltip appears | +| Place cursor on a symbol and press **⌥F12** (Find References) or use the menu | References panel lists results | +| Place cursor on a function call and press **⌘↓** (Go to Definition) | Navigates to the definition site | +| Save a file with trailing whitespace | File is formatted on save (if Build On Save is off, trigger Format Document manually) | + +--- + +## T05 — ZLS: inline diagnostics + +**Project:** `examples/snippets/error-example.zig` (open as a single file) + +| Step | Expected | +|---|---| +| Open the file | ZLS highlights the intentional error with a red underline | +| Hover over the underlined region | Error message appears in the tooltip | +| Fix the error and save | Red annotation disappears | + +--- + +## T06 — Auto-discovered tasks: basic + +**Project:** `examples/multi-step` + +| Step | Expected | +|---|---| +| Open the workspace | Task list eventually contains **Zig Build: install**, **run**, **test**, **check-fmt**, **gen-docs**, **run\_server**, **bench**, **release-macos** | +| Select **Zig Build: run** and press ▶ | `zig build run` executes; output appears in the task report | +| Select **Zig Build: bench** and press ▶ | `zig build bench` executes successfully | +| Select **Zig Build: check-fmt** and press ▶ | Exits cleanly (sources are already formatted) | + +--- + +## T07 — Auto-discovered tasks: cache invalidation + +**Project:** `examples/multi-step` + +| Step | Expected | +|---|---| +| Note the current discovered step list | Baseline | +| Add a new step to `build.zig` (e.g. a blank `b.step("smoke", "…")`) and save | Within a few seconds the task list updates; **Zig Build: smoke** appears | +| Revert the change and save | **Zig Build: smoke** disappears from the list | + +--- + +## T08 — Auto-discovered tasks: discovery disabled + +**Project:** `examples/multi-step` + +| Step | Expected | +|---|---| +| Open **Project Settings → Zig → Tasks → Discover Build Steps** and set to Disabled | **Zig Build: \<step\>** tasks all disappear from the list | +| **Current Zig File** task | Still present and functional | +| Re-enable discovery | **Zig Build: \<step\>** tasks reappear | + +--- + +## T09 — Current Zig File: run + +**Project:** `examples/snippets/fibonacci.zig` (open as a single file, or with the file active in any workspace) + +| Step | Expected | +|---|---| +| Select **Current Zig File** and press ▶ | `zig run fibonacci.zig` executes; program output appears in the task report | +| Switch to a different `.zig` file | The task now targets the new file (name in the task report changes) | + +--- + +## T10 — Current Zig File: clean + +**Project:** `examples/multi-step`, open `src/main.zig` + +| Step | Expected | +|---|---| +| Ensure `.zig-cache` exists in `examples/multi-step/` (run any build first) | Baseline | +| Select **Current Zig File** and trigger Clean | `.zig-cache` and `zig-out` inside `examples/multi-step/` are removed; no warning dialog | + +--- + +## T11 — Zig Package: build and run + +**Project:** `examples/multi-step`, using the pre-configured task in `.nova/Tasks/Zig Package.json` (Run Step = `run`) + +| Step | Expected | +|---|---| +| Press ⌘B (Build) | `zig build` runs; exits 0; binary appears in `zig-out/bin/` | +| Press ▶ (Run) | `zig build run` runs; program output appears in task report | +| Set **Optimize** to `ReleaseFast` and press ▶ | Command contains `-Doptimize=ReleaseFast` (visible in task report header or Extension Console) | +| Set **Target** to `aarch64-macos` and press ⌘B | Command contains `-Dtarget=aarch64-macos` | +| Add `verbose` to **User Options** and press ▶ | Command contains `-Dverbose` | +| Add `--verbose` to **Build Arguments** and press ▶ | Command contains `--verbose` after the `-D` flags | +| Add `--help` to **Program Arguments** and press ▶ | `--` separator and `--help` appear after the step name | +| Set **Console** to External Terminal and press ▶ | macOS Terminal opens a new window/tab and runs the program | + +--- + +## T12 — Zig Package: build-options project + +**Project:** `examples/build-options` + +| Step | Expected | +|---|---| +| Create a **Zig Package** task with **Run Step** blank | ▶ runs `zig build`; program prints using the default option values | +| Add a User Option matching one of the options declared in `build.zig` | Re-run; program output reflects the new value | + +--- + +## T13 — Zig Test: all tests + +**Project:** `examples/test-suite`, using one of the pre-configured tasks + +| Step | Expected | +|---|---| +| Select **Zig Test (all)** and press ▶ | `zig build test` runs; both math and strings tests pass; no inline errors in editor | +| Select **Zig Test (all)** and press ⌘B | Same command without `--summary` or `--test-filter`; exits 0 | +| Set **Summary** to **All** and press ▶ | Output is more verbose (every step printed) | + +--- + +## T14 — Zig Test: filtered + +**Project:** `examples/test-suite`, **Zig Test (math)** task (Test Filter = `math`) + +| Step | Expected | +|---|---| +| Press ▶ | Only math tests run; strings tests are skipped | +| Change the filter to a string that matches no test name (e.g. `zzz`) | Task exits 0 but reports 0 tests run | + +--- + +## T15 — Zig Test: inline error annotations + +**Project:** `examples/test-suite` + +| Step | Expected | +|---|---| +| Introduce a deliberate test failure in `src/math.zig` (e.g. change an expected value) and save | | +| Run **Zig Test (all)** | Task exits non-zero; the failing line in `math.zig` gets a red gutter annotation | +| Click the annotation | Error message is shown | +| Revert the change and re-run | Annotation clears | + +--- + +## T16 — Zig Watch + +**Project:** `examples/multi-step`, pre-configured **Zig Watch** task + +| Step | Expected | +|---|---| +| Press ⌘B (Build) | `zig build --watch` starts; task report shows initial build output and then idles | +| Edit `src/lib.zig` (e.g. change a return value) and save | Zig detects the change and rebuilds automatically; new output appears in the task report | +| Stop the task | Process terminates cleanly | +| Set **Debounce** to `500` and restart | `--debounce 500` appears in the command | +| Set **Incremental** to **On** and restart | `-fincremental` appears in the command | + +--- + +## T17 — Zig Debug: auto-detection + +**Project:** `examples/multi-step` — create a **Zig Debug** task with **Program** blank + +| Step | Expected | +|---|---| +| Press ▶ (Run/Debug) | Extension reads `build.zig.zon`, finds `.name = "multi-step"`, probes `zig-out/bin/multi-step` | +| If binary exists (run Build first) | Debugger launches and the program runs under lldb-dap | +| If binary does not exist | Warning dialog asks you to fill in the Program field manually | + +--- + +## T18 — Zig Debug: breakpoints + +**Project:** `examples/multi-step` — **Zig Debug** task, **Optimize** = Debug + +| Step | Expected | +|---|---| +| Set a breakpoint in `src/main.zig` by clicking the gutter | Red dot appears | +| Press ▶ | Program pauses at the breakpoint; Variables panel shows local state | +| Step over (F10) | Execution advances one line | +| Continue (F5) | Program runs to completion or the next breakpoint | + +--- + +## T19 — Zig Debug: stop on entry + +**Project:** `examples/multi-step` — **Zig Debug** task + +| Step | Expected | +|---|---| +| Enable **Stop On Entry** | | +| Press ▶ | Debugger pauses at the very first instruction before any user code runs | + +--- + +## T20 — Clean: safety guards + +Create a **Zig Package** task in any project and test each guard: + +| Scenario | How to trigger | Expected | +|---|---|---| +| Working Directory = `/` | Set the field to `/` | Clean shows a warning and does nothing | +| Working Directory = `$HOME` | Set the field to your home directory | Clean shows a warning and does nothing | +| Working Directory outside workspace | Set an absolute path outside the open folder | Clean shows a warning and does nothing | +| Valid cwd | Normal configuration | `.zig-cache`, `zig-cache`, `zig-out` are removed | + +--- + +## T21 — Clean: zig build uninstall + +**Project:** a project whose `build.zig` registers an `uninstall` step (add one temporarily if needed) + +| Step | Expected | +|---|---| +| Run Build once to populate the step cache | | +| Trigger Clean | Extension Console shows `zig build uninstall` was called before `rm -rf` | + +--- + +## T22 — Monorepo: sub-package working directory + +**Project:** `examples/monorepo` (open workspace root) + +| Step | Expected | +|---|---| +| Discover tasks at root | **Zig Build: run**, **test-all**, **gen-docs** appear | +| Create a **Zig Package** task; set **Working Directory** to `packages/alpha` | Build/Run operates on `packages/alpha/build.zig` | +| Create a **Zig Test** task; set **Working Directory** to `packages/beta` | Tests run from `packages/beta/` | + +--- + +## T23 — Inline compiler errors + +**Project:** `examples/snippets/error-example.zig` via a **Zig Package** or **Current Zig File** task + +| Step | Expected | +|---|---| +| Run the task against a file with a compile error | Error annotation appears at the correct line and column in the editor | +| The annotation text | Matches the `error:` message from the compiler | +| Fix the error and re-run | Annotation clears | + +--- + +## T24 — Settings: custom executable paths + +**Scope:** Nova Preferences → Extensions → Zig + +| Step | Expected | +|---|---| +| Set **Zig Executable** to an absolute path (e.g. `/usr/local/bin/zig`) | Tasks use that binary instead of PATH discovery | +| Set it to a non-existent path | First task run shows a warning about missing `zig` | +| Clear the field | PATH discovery resumes | + +--- + +## T25 — Settings: workspace overrides global + +**Project:** any workspace + +| Step | Expected | +|---|---| +| Set a global preference (e.g. Discover Build Steps = disabled) | Workspace inherits; no discovered tasks | +| In Project Settings, set Discover Build Steps to **Enabled** | Discovered tasks reappear for this workspace only | +| Open a different workspace | Still uses the global disabled setting | |