Skip to content

[rig-tasks] Add 10 rig samples — 2026-08-09 - #382

Merged
pelikhan merged 1 commit into
mainfrom
rig-tasks/2026-08-09-2e77a0125a89e141
Aug 9, 2026
Merged

[rig-tasks] Add 10 rig samples — 2026-08-09#382
pelikhan merged 1 commit into
mainfrom
rig-tasks/2026-08-09-2e77a0125a89e141

Conversation

@github-actions

@github-actions github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Added 10 new rig sample files to skills/rig/samples/.

# File Description Typecheck
1 381-ts-dead-export-finder.md TypeScript dead export finder pass
2 382-pkg-scripts-documenter.md Package.json scripts documenter pass
3 383-git-diff-stats-summarizer.md Git diff stats summarizer pass
4 384-dotenv-template-generator.md Dotenv template generator pass
5 385-ts-class-hierarchy-extractor.md TS class hierarchy extractor pass
6 386-git-bisect-helper.md Git bisect helper pass
7 387-shell-shebang-validator.md Shell shebang validator pass
8 388-git-log-graph-summarizer.md Git log graph summarizer pass
9 389-ts-complexity-scorer.md TS complexity scorer pass
10 390-parallel-multi-tool-workflow.md Parallel multi-tool workflow pass

Typecheck failures

No failures — all 10/10 samples passed typecheck. Two samples required minor fixes before passing:

  • 381 (TS dead export finder): removed as const from array return in async handler (TS1355)
  • 382 (pkg scripts documenter): removed unused name param from handler destructuring (TS6133)
  • 390 (parallel workflow): (1) removed unsupported output field from workflow(), (2) switched from parallel() to Promise.all to avoid heterogeneous type unification error

Tasks run

  • (reused) TypeScript dead export finder — p.glob + async defineTool + steering + repair
  • (reused) package.json scripts documenter — p.read + defineTool + p.write + repair
  • (reused) Git diff stats summarizer — p.bash + defineTool + s.enum + repair
  • (reused) Dotenv template generator — p.readOptional + p.glob + async defineTool + p.write + repair
  • (reused) Multi-file TypeScript class hierarchy extractor — p.bash + async defineTool + steering
  • (reused) Git bisect helper — p.bash + defineTool + steering maxTurns:8
  • (new) Shell script glob validator — p.glob + async defineTool + repair + s.record
  • (new) Git log graph summarizer — p.bash + sync defineTool + steering + s.enum
  • (new) TypeScript complexity scorer — p.glob + async defineTool + s.number + repair
  • (new) Workflow parallel multi-tool tester — workflow + Promise.all + call.json coordination

Generated by Daily Rig Task Generator · sonnet46 105.5 AIC · ⌖ 9.34 AIC · ⊞ 6.8K ·

- 381: TS dead export finder (reused)
- 382: pkg scripts documenter (reused)
- 383: git diff stats summarizer (reused)
- 384: dotenv template generator (reused)
- 385: TS class hierarchy extractor (reused)
- 386: git bisect helper (reused)
- 387: shell shebang validator (new)
- 388: git log graph summarizer (new)
- 389: TS complexity scorer (new)
- 390: parallel multi-tool workflow (new)

All 10/10 typecheck passed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review August 9, 2026 10:52
@pelikhan
pelikhan merged commit 11805d4 into main Aug 9, 2026
1 check passed
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs, /codebase-design, and /grill-with-docs — requesting changes on correctness issues in the tool handler logic across several samples.

📋 Key Themes & Highlights

Key Issues

  • Inverted classification logic (383): classifyDiffEntry has added/deleted swapped — verified by running the function against real git diff --numstat output. New files are never classified as "added".
  • Always-false isAbstract (385): The regex captures abstract as part of its own match, so m.index points before abstract, making the 10-char lookback window always miss it. Easy one-line fix.
  • O(N2) file reads with no caching (381): The findUnusedExports tool re-reads every file for every exported name in a sequential nested loop. Minor perf concern for a sample but teaches a bad pattern.
  • False-positive ternary counting (389): ?. and ?? operators inflate the ternary score — common in modern TS.
  • Ungrounded bisect narrative (386): stepsRun in the output isn't anchored to actual selectMidpoint calls; the model will fabricate the step count.

Positive Highlights

  • ✅ Good variety across patterns: p.glob, p.bash, p.read/p.readOptional/p.write, repair(), steering(), async tools
  • ✅ Sample 390 correctly uses Promise.all for parallel agent execution — the workflow pattern is clean
  • ✅ Consistent use of s.path for file paths, s.int for counts, s.enum for categoricals
  • ✅ All samples passed typecheck — the pre-submission fixes were the right call

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 56.2 AIC · ⌖ 7.81 AIC · ⊞ 6.3K
Comment /matt to run again


const classifyDiffEntry = defineTool("classifyDiffEntry", {
description: "Classify a git diff --numstat line as added, modified, deleted, or renamed",
parameters: s.object({

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] The classifyDiffEntry logic has "added" and "deleted" inverted — and the first branch is never reachable for actual new files. git diff --numstat uses additions<TAB>deletions<TAB>path, so a new file shows 5 0 file.ts; startsWith("0\t") matches zero additions (i.e. deleted files).

💡 Corrected classification

Actual outputs from the current code:

  • classifyDiffEntry("5\t0\tsrc/new.ts")"modified" ❌ (should be "added")
  • classifyDiffEntry("0\t3\tsrc/gone.ts")"added" ❌ (should be "deleted")

Fixed handler:

handler: ({ line }: { line: string }) => {
  const [adds, dels] = line.split("\t");
  if (line.includes("=>")) return "renamed" as const;
  if (adds !== "0" && dels === "0") return "added" as const;
  if (adds === "0" && dels !== "0") return "deleted" as const;
  return "modified" as const;
},

for (const other of allFilePaths) {
if (other === filePath) continue;
const otherContent = await readFile(other, "utf-8").catch(() => "");
if (new RegExp(`\\b${name}\\b`).test(otherContent)) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] The cross-file search uses \bName\b (word boundary regex) against raw file text, which produces false positives for any identifier that contains the exported name as a substring (e.g. export Button matches ButtonGroup), and misses renamed imports (import { Button as Btn }). This could cause all exports to appear "used" on a large codebase.

💡 More reliable search

A simple improvement: check for the name appearing in an import statement rather than anywhere in the file:

if (new RegExp(`[{,]\\s*${name}\\s*[},]`).test(otherContent)) {
  isImported = true;
  break;
}

For a production tool, consider using ts-morph or the TypeScript compiler API. For a sample this scope is acceptable, but the description should note the limitation so users know not to rely on this for critical dead-code decisions.

handler: async ({ filePath, allFilePaths }: { filePath: string; allFilePaths: string[] }) => {
const content = await readFile(filePath, "utf-8");
const exportMatches = [...content.matchAll(/^export\s+(?:function|const|class|type|interface|enum)\s+(\w+)/gm)];
const exportedNames = exportMatches.map((m) => m[1]);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] The findUnusedExports tool reads every other file for every exported name — O(files × exports) sequential readFile calls with no caching. On a 200-file codebase with 20 exports each, this is 4000 file reads. The pattern also defeats the async runtime by await-ing inside nested loops.

💡 Cache file contents up front

Read all files once into a Map before the inner loop:

handler: async ({ filePath, allFilePaths }) => {
  const files = new Map(
    await Promise.all(allFilePaths.map(async p => [p, await readFile(p, "utf-8").catch(() => "")] as const))
  );
  const content = files.get(filePath) ?? "";
  const exportedNames = [...content.matchAll(/^export\s+(?:function|const|class|type|interface|enum)\s+(\w+)/gm)]
    .map(m => m[1]);
  return exportedNames.filter(name =>
    !allFilePaths.some(p => p !== filePath && new RegExp(`\\b${name}\\b`).test(files.get(p) ?? ""))
  );
},

This reads each file once via parallel Promise.all instead of re-reading every file for every exported name.

classes.push({
name: m[1],
parent: m[2] ?? null,
interfaces: m[3] ? m[3].split(",").map((s: string) => s.trim()) : [],

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] The isAbstract detection looks 10 characters before the class keyword: content.slice(m.index - 10, m.index). But the regex already includes (?:abstract\s+)? as its own optional prefix — so when the regex does match an abstract class, m.index points to the start of abstract, not class, meaning the 10-char lookback window is looking at content before the keyword itself and the .includes("abstract") check can never be true for those matches.

💡 Fix: capture the abstract group directly from the regex match

The regex (?:abstract\s+)?class\s+(\w+)... already optionally matches abstract. Just check m[0].startsWith("abstract"):

classes.push({
  name: m[1],
  parent: m[2] ?? null,
  interfaces: m[3] ? m[3].split(",").map((s: string) => s.trim()) : [],
  isAbstract: m[0].startsWith("abstract"),
});

This is simpler and correct — no fragile index arithmetic needed.

filePath: s.path,
}),
handler: async ({ filePath }: { filePath: string }) => {
const content = await readFile(filePath, "utf-8").catch(() => "");

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] The ternary counter regex /\?[^?:]/g matches any ? not followed by ? or :, including optional chaining (foo?.bar) and nullish coalescing (a ?? b). This inflates the ternary count significantly in modern TypeScript codebases, since ?. and ?? are very common.

💡 Narrow the ternary pattern
// Match only conditional ternary: expression ? branch : ...
// Exclude ?. (optional chain) and ?? (nullish coalescing)
const ternaries = (content.match(/(?<!\?)\?(?![.?])/g) ?? []).length;

With the original regex, foo?.bar ?? baz scores 2 ternaries when it should score 0.

Perform up to 8 binary search steps over the commit list.
After narrowing down, return your best suspect commit, how many steps you took,
the commit range you searched, and your confidence level.`,
output: s.object({

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] The agent's prompt says "Perform up to 8 binary search steps" but the output schema has no way to communicate which commit was actually checked at each step, so the model can't justify how it narrowed from the full list to suspectCommit. The stepsRun field is returned but never grounded by tool calls — the model will fabricate a narrative. Consider tracking checkedCommits: s.array(s.string) in the output or using a loop structure that makes each selectMidpoint call observable.

💡 Add a checkedCommits field to the output
output: s.object({
  suspectCommit: s.optional(s.string),
  stepsRun: s.int,
  checkedCommits: s.array(s.string),  // add this
  commitRange: s.object({ start: s.string, end: s.string }),
  confidence: s.enum("high", "medium", "low"),
}),

This forces the model to materialise each binary search step as a real selectMidpoint call result rather than inventing a story.

instructions: p`You are a package.json scripts documenter.
Read the package.json: ${p.read("package.json")}

For each script in the "scripts" field, call inferScriptPurpose to classify it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] The p.write intent here passes literal placeholder content ("## Scripts\n\n<!-- generated -->"), but the agent is supposed to write the actual classified script documentation. The p.write intent is a declarative placeholder for the path — the model fills in the real content at runtime. As written it reads as though the agent will always write the same boilerplate. The intent placeholder content should be a representative example or the actual format expected, not a static stub that misleads the reader.

💡 Use a representative placeholder
${p.write("SCRIPTS.md", "## Scripts\n\n| Script | Category | Command |\n|--------|----------|---------|\n| build | build | tsc |\n")}

This makes it clear to the model (and sample readers) what structure to produce, not just that a write will happen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant