The previous article (Part 1 - making and using Skills the easy way) was designed to get you working with Skills as fast as possible, but inevitably there will come a point where you've created your perfect Skill and it does exactly what you want. That's great! And now, as with all good things you create, you decide to share it. However, the Skill's fine, but will it work well for other people? Not sure about this, you turn to adding best practices and quality-of-life additions... if there are any? (There are.)
You can read about all the best practices on the Skill documentation site's best practices page, but I have compiled three tips in particular here!
1. Skill descriptions matter... like, a lot
Skills can be quite long and involve multiple custom scripts, reference material, and all sorts of other paraphernalia, as needed. If you have a lot of them, that's a lot for your LLM to store in memory. This can be a lot of tokens! So, the LLM doesn't do this. Instead, it keeps the name and description of the Skill on hand and checks that against your prompts. This is known as "progressive disclosure".
This means that the name and description of your Skill are probably the most important parts of it! You need to keep both:
- concise: this is going to be held in memory every time, so make it short!
- comprehensive: if the LLM can do the task without using your Skill, it will, so avoid ambiguity.
- complementary to the LLM: don't waste sentences describing things the LLM will know about anyway, like what a "PDF" is.
This is just a quick overview. The Skills specification has a much more in-depth piece about how to get your Skill description just right, and you can also just ask your LLM about it. The LLM should know about the best practices available, and you can always copy and paste the relevant text from them into your prompt if you need to.
2. You can develop a Skill after installation
Provided you have the rights and access, there is nothing to stop you iterating on an installed Skill. Perhaps you want to add some reference material, or perhaps you have some speciality context that needs to be added to your workflow when certain factors arise. You can even ask an LLM to edit the Skill in its installed directory if you want. Of course, you should be aware of the licence your Skill came with if you got it from somewhere else before modifying it, but otherwise the flexibility available is unmatched.
3. You can build tests into your Skill
It is fairly common for AI advocates to compare Skills and prompting to programming because they are both acts of writing instructions in such a way that a computer can understand them and perform operations. Prompts and Skills are nicely analogous to scripts and functions. However, there is another aspect of programming that you may not be aware is available within Skills: tests!
For example, we could take the Skill from earlier and add tests to it, which should be done in the form of an evals.json file within the Skill directory.
blog-post-editorial-review/
├── SKILL.md
└── evals/
├── evals.json
└── files/
└── acronym_expansion.md
acronym_expansion.md
# Using CFD for simulation
CFD can predict fluid flow around a product. The CFD results can then be reviewed.
evals.json
{
"skill_name": "blog-post-editorial-review",
"evals": [
{
"id": 1,
"prompt": "Review the attached Markdown article and expand acronyms on first use where appropriate.",
"expected_output": "The reviewed article expands CFD on its first prose occurrence, keeps later occurrences abbreviated, and records one acronym expansion in the review report.",
"files": [
"evals/files/acronym-expansion.md"
],
"assertions": [
{
"id": "expands-cfd-on-first-use",
"description": "The reviewed article contains the exact text 'Computational fluid dynamics (CFD) can predict fluid flow around a product.'",
"required": true
}
]
}
]
}
Once all this is in your Skill, you can ask your LLM to run the tests for you and get some kind of output. This can be very useful if you have any key functionality that needs to be checked. However, this functionality does have obvious limitations and issues (what if your LLM just lies to you, for example?), and there are more in-depth and rigorous instructions available in the Skill documentation. The dedicated documentation page on the topic has far more details.
This is Part 2 of a 2-part series on AI Skills. If you haven't read Part 1 you can find it below.
