Validate a SKILL.md with open source checks, then run the task

Every check reads the same few properties

A validator reads the YAML frontmatter and compares the name against the folder the file sits in. It follows the paths and links the body mentions. It counts the tokens the skill adds to an agent's context. The specification puts the reference check in one line: "Use the skills-ref reference library to validate your skills: skills-ref validate ./my-skill. This checks that your SKILL.md frontmatter is valid and follows all naming conventions."

The GitHub Action spells out what valid frontmatter means, field by field. The name field "Must be 1-64 characters, lowercase alphanumeric with hyphens, no leading/trailing/consecutive hyphens, must match directory name". The description field "Must be 1-1024 characters, non-empty". The optional fields carry limits too: license "Must be a string if provided", compatibility "Must be 1-500 characters if provided", metadata "Must be a key-value mapping with string values", allowed-tools "Must be a space-delimited string". The block has to sit between --- markers and parse as YAML.

A check can tell you a description runs 47 characters. It cannot tell you whether a support agent holding four other skills matches a customer's sentence to it. The specification says an agent loads "only the name and description of each available skill" at startup. Every count in the body is spent after the decision the skill needed to win. A token count is worth reading because of the three load tiers. It never answers whether the file gets chosen.

The two questions left to a person

Whether the steps reach the right answer is the first. A body can satisfy every field rule and still send an agent to approve a refund the policy excludes, or to summarise a ticket instead of routing it. No check compares instructions against an outcome. That needs a task, a run, and someone who knows what correct output looks like.

Whether the description triggers is the second. A validator measures length; the matching happens in each client, against the words of a request. Passing and loading when somebody needs it are separate facts about the same file. How to write a SKILL.md covers phrasing that description. The test of whether it works is further down.

Three checks, in the order that catches the most

Start with the reference library, skills-ref validate ./skills/refund-policy, while the file is open in an editor. It is the cheapest, and catches the naming failure earlier.

Then put the Action in CI, on every changed skill folder. It reads the same rules and adds file reference checking, where validate-references confirms the files named in the body exist. It also returns valid, errors and warnings, so a workflow gates on the result instead of a person reading a log.

The third belongs to the project. A skill lives at skills/<name>/SKILL.md in a Kortix project, and the skills table in kortix.yaml maps a skill name to a repository folder (kortix.yaml reference). The kortix validate command reads the files around that manifest. Its errors include "a skills folder with no SKILL.md, or a SKILL.md whose frontmatter name: is not the skill name". It exits 0 with no errors, 1 with one or more, and 2 when the file is missing or unreadable. Neither other check knows where your project keeps its folders, or which agent may load them.

The skill-validator reads the properties those three skip: closed code fences, orphaned files, external links and a token count per file. Its README opens: "Spec compliance is table stakes."

What each warning is telling you

The Action reports four warnings, and they are not one size.

"Very short descriptions (< 50 characters)" is the one to treat as an error on any skill another person relies on. The description is all an agent sees before it decides, so a thin one is a discovery bug no later check will catch. "Empty body content" is an error in all but name, since a skill without instructions is a description with a file around it. "Body exceeding 500 lines" is a decision for review, because the specification recommends keeping the main file under 500 lines and moving reference material into separate files. "Unknown fields in frontmatter" is usually a client extension or a typo of a real field, and one lookup settles which.

The fail-on-warning input is a choice per folder: on where a thin description would cost someone an hour, off while a new skill's body is still being cut.

Put the check on the merge path

A check that runs when someone remembers is not a check; two places make it structural.

The first is CI. Point path at the changed folder and turn validate-references on. Set fail-on-warning where it matters. Then the rules become a failing job on the pull request. The second is the project's gate. The kortix ship command runs the same manifest checks before it commits. The change request merge runs the schema check on the merged manifest. An error blocks that merge, and a warning never does (kortix.yaml reference).

Session work reaches main through a change request. Merge is default-deny for agents. A skill that changes what the company does arrives as a diff somebody reads, with a validation result attached.

What only a run can tell you

Two tests sit outside every validator.

The first is a trigger test. Collect the sentences people type when they need the skill. Run each against an agent that has it loaded, and record whether it loaded. The guidance on descriptions suggests three runs per query and a trigger rate above 0.5 as a reasonable default for a query that should fire. A description that triggers on none of your five sentences leaves a file nobody reaches. One that fires on the wrong sentence takes the next skill's place.

The second is the run itself. Give the agent a real task you have done by hand. Read the output against the exit criterion in the body. One isolated sandbox per session. Each session has its own isolated machine and branch. A wrong answer here has no syntax to fix, and another pass of the validator will not explain it. The diff shows which step the body left out, or which decision it left to the model's judgement.

A passing file, a trigger rate and one diff are three kinds of evidence about one folder. Only the last two say whether anyone should load it. Kortix keeps each skill as a file in a git repo you own. Open source. Read it, fork it, audit it.