The starting point
SHAREPOINT.md they say, SKILLS.md they say. The first thing I noticed on my test site with Copilot in SharePoint was that there's no Agent Assets library sitting there waiting for you. No Skills folder, nothing to upload into. I assumed I'd have to create it manually.
I didn't. I asked Copilot, in the chat panel, to set it up. Initially it created the AgentAssets folder but then complained that the 'Make "New Folder" command available?' setting wasn't enabled. It was. After a short delay it went ahead and created the structure.
Once that library existed, installing a skill is exactly as simple as it sounds. I pulled a permission-report, part of the PnP community's sharepoint-skills repository, downloaded the SKILL.md, and dropped it into Agent Assets/Skills/permission-report/. No registration step, no activation. Copilot finds it by the name field the next time something matches.
Asking the question
This is the skill's whole job: a read-only access and sharing review, written out as a self-contained HTML report, flagging Anyone links, external users, broad groups, and anyone with Full Control. Previously, to achieve this kind of extract meant PnP PowerShell, looping every list and item, checking HasUniqueRoleAssignments. I asked Copilot for a permission report on the site, and it produced one, first try. Correct scope, sensible summary counts, saved into an Access Reports folder with the filename convention the skill specifies. Cool stuff, and genuinely fast.
Extending the question
The report didn't name names, group membership wasn't expanded, just "SiteName Owners: Full Control." Reasonable, since the skill's own instructions only ask it to capture "users, groups, sharing principals, and links," not crawl into what each group actually contains.
So I asked a follow-up: who's actually in that group. And here's where it went wrong. Copilot answered, but it didn't run the skill again to do it. It told me the group had zero members. I'm a member of that group. It also gave me a link to the group that didn't resolve correctly.
"I don't have that information," would have been better. The manual alternative, a PnP PowerShell script against the group directly, would at least have returned the real member list instead of inventing one.
Asking the same question twice
Then I ran the exact same request, same wording, no changes, twice, about seven minutes apart.
Same skill. Same words. Roughly three times as much content reviewed the second time, and a different report layout, one run added a narrative "Findings" section the other didn't have at all. Nothing about the request changed between the two runs. The interpretation of "whole site" apparently did.
That is a real issue where a Skill should be defensible and repeatable. Ask the same question twice and get materially different answers, and it isn't.
Fixing the skill
Each fix named an exact item that had been left open before: this exact list of libraries, this exact section order, this exact folder-check to skip. That turned out to be the pattern, wherever the skill's own wording left something to be worked out fresh each run, it often changed, the moment I replaced the open-ended phrasing with a fixed, named answer, it stopped. I included an instruction to produce the same section structure, as well as specific column names I expected as output.
SHAREPOINT.md
Worth explaining properly, since it's new and easy to confuse with skills. SHAREPOINT.md is a separate Markdown file in the same library that Copilot reads as standing context before every prompt, not just when a skill happens to fire. Think of it as the site's own README, not a task's instructions: what this site is for, its structure, its conventions, who owns it.
After adding my fixes, the library list as a stated fact in SKILL.md, the deterministic-use instruction in SKILL.md, I ran the identical-request test a third and fourth time. The report came back nearly word-for-word identical both times: Documents, AgentAssets, Site Assets. The "Findings" section appeared consistently in both.
One more addition to the SHAREPOINT.md file were the company colours, which will apply to every Skill on the site.
## Conventions
- Use these brand colours for headers, page chrome, and any decorative
styling in generated reports: slate blue-grey (#5C6B73) and a darker
navy blue (#1B2A41).
- Do not apply brand colours to risk-coded or status-coded elements —
leave semantic colour coding (e.g. red/amber/green risk flags) as
each skill's own instructions define it.The second line is important because the Permission Report's own instructions already use colour with meaning attached, red for high risk, green for low. A blanket "always use our brand colours" rule, could quietly repaint a Full Control risk flag in calm corporate blue instead of red.
What this actually means
Skills are a real shift in who gets to do SharePoint governance. Every audit I've written PowerShell for, alerts, content types, term store, permissions, required someone who could script on the organisation's behalf. A site owner was always the subject of that governance, never the one performing it. A skill like this one puts a real capability directly in that person's hands.
Writing a skill more like code, with explicit branches and explicit refusal conditions, helps. Permission Report already does this well, "if no narrower scope is clear, state that plainly and do not guess" is about as close to a coded conditional as a natural-language instruction gets. But what is clear is that several factors contribute to the specificity of the outcome, the language used, the SKILL itself, and the SHAREPOINT.md file.
So the conclusion isn't "not audit-grade yet." It's narrower than that: a skill's reliability is a direct function of how much of it you've pinned down. Every fix in this piece was small, specific, and aimed at one named gap at a time. It seems that is what building a Skill that you can present at a Governance committee, month after month, requires.