Review the audit log for agent traffic

MaintenanceIntermediate~10 min

Before you start

Plugin version 1.7.4 or newer for the is_agent_identity flag. The Activity Log read requires the manage-options capability, so run this from an administrator connection, not from a least-privilege agent user.

The Activity Log records everything, which is its virtue and its problem. Your own admin work is in there next to the agent’s, in one stream, and the question you usually want answered is specifically about the agents: what have they been doing, which abilities are they leaning on, and is any of it being refused.

Version 1.7.4 put a flag on every row that makes the split possible. This prompt uses it to give you an agent-only view of your own log. It reads and reports, nothing else.

The prompt
I want to know what the agents on this site have been doing. Read only, change
nothing.
Read the Activity Log for [the last seven days], newest first. Every row now
carries an is_agent_identity flag, so split the rows on it: agent traffic in
one group, human traffic in the other. Tell me the count on each side before
you go any further.
For the agent rows, group them by ability and give me the counts, busiest
first. Then group the same rows by status so I can see how many succeeded, how
many errored, and how many were denied.
Pull out every denied and every errored row and show me those in full: the
ability, the acting user, the timestamp, and whatever detail the log recorded.
Finish with a short read on the pattern. Is any single agent identity behaving
differently from the others, reaching for abilities the rest do not touch, or
collecting denials nobody else is getting? Say what you saw, and say plainly if
nothing stands out.

Why this is safe to run

One ability, and it is a read. The Activity Log reports history that has already happened, and reading it cannot rewrite a row of it.

The flag is what makes the split trustworthy. Before 1.7.4 you could filter by username and hope that told you something, but usernames answer the wrong question: they identify the account, not the kind of connection the call came through. The is_agent_identity boolean reflects either the per-user application-password marker that already existed or a per-OAuth-client flag set on the Connection tab, so a connection you deliberately created for an agent is marked as one either way, and the log says so on every row.

Know what the log is not, before you plan around it. It keeps identifier-only values by design: ability names, statuses, user ids and logins, the argument keys that were passed, timestamps, and a short detail such as an object id, a slug, or an error code. It does not keep argument values, content, file paths, or network addresses. So this recipe can tell you that an agent updated four posts and which four, and it cannot tell you what the new text was or where the request came from. Those are deliberate omissions, and they are the reason the log is safe to hand to a prompt in the first place.

The last thing is who runs it. The read is gated behind manage-options, a capability a well-scoped agent user has no business holding, since an agent that could read the audit log could read the record of every other agent too. So this one is an administrator’s prompt, run from your own connection.

Already connected? Skip the paste. From an admin connection, ask your agent for the “review the audit log for agent traffic” recipe, or just ask what the other agents have been up to this week, and it runs the same split against your own log.

How it works

  1. Read the Activity Log over the window you care about, newest first.
  2. Split the rows on the is_agent_identity flag so agent calls sit apart from human ones.
  3. Group the agent rows by ability and by status.
  4. Pick out anything denied or errored and read those separately.
  5. Report the pattern, and name any agent identity behaving unlike the others.

Questions

What makes a row count as agent traffic?
The is_agent_identity boolean, added to every log row in 1.7.4. It is true when the caller is flagged as an agent identity, which comes from one of two places: the per-user application-password marker that already existed, or a per-OAuth-client flag you set on the Connection tab. So a connection is marked at whichever level it was created, and the log tells you which side of that line each call came from.
Why can't I just filter by username?
You can, and before 1.7.4 that was the only option, but it answers a different question. A username tells you which account was used; the agent flag tells you whether that call came through a connection you set up for an agent. The same account can be both, and an OAuth client can be marked as an agent without changing anything about the user behind it. The flag is the one that survives that.
Will the log tell me what the agent actually wrote?
No, on purpose. The log keeps identifier-only values: the ability name, the status, the acting user id and login, the argument keys that were passed, the timestamp, and a short detail such as an object id, a slug, or an error code. It never stores the argument values, the content itself, file paths, or network addresses. So it will tell you that a post was updated and which one, not what the new text said.
Why does this need an admin connection?
The Activity Log read is gated behind the manage-options capability, which a properly scoped agent user should not hold. That is deliberate: an agent that could read the audit log could read the record of every other agent's activity. Run this one from your own administrator connection.