Much of the answer is in lowering prompt entropy with a) more specific prompts b) MCP connectors to actual data that can prove/disprove the efficacy of an approach and c) project/skill files that give guardrails to repeat tasks. But even then, d) notice when you are going in circles, is a must!
Thanks for sharing James! Totally agree with 1 and 3. Do you have specific things you like to call out in your skills and project files? Even when I focus on adding info to these, I feel like Claude will run into hurdles and then try to brute force its way past what I defined (like with the access roles).
Unfortunately I’m still skeptical of data governance when it comes to MCPs. This will be unlocked eventually but as a data team we are still thinking about all the guardrails that need to be in place first when dealing with sensitive customer data.
Good questions… my favourite skills right now are: one for writing PRs (using diffs + session info for context, and nice html/markdown formatting), one to build a staging model, and one to check for and finish off shared dependency upgrades. All comparatively simple tasks with minimal business logic. All boring and repetitive!
Would check out hooks and settings.json customisations for data protection if you haven’t already, but for sure this feels substantially unresolved at present!
I've been building a staff AE agent to review and comment on PRs. It's very much a WIP. I've given it very specific things to look at in skills during a review. It also has running window context of last 10 days PRs and last 4 weekly pr summary.
Much of the answer is in lowering prompt entropy with a) more specific prompts b) MCP connectors to actual data that can prove/disprove the efficacy of an approach and c) project/skill files that give guardrails to repeat tasks. But even then, d) notice when you are going in circles, is a must!
Thanks for sharing James! Totally agree with 1 and 3. Do you have specific things you like to call out in your skills and project files? Even when I focus on adding info to these, I feel like Claude will run into hurdles and then try to brute force its way past what I defined (like with the access roles).
Unfortunately I’m still skeptical of data governance when it comes to MCPs. This will be unlocked eventually but as a data team we are still thinking about all the guardrails that need to be in place first when dealing with sensitive customer data.
Good questions… my favourite skills right now are: one for writing PRs (using diffs + session info for context, and nice html/markdown formatting), one to build a staging model, and one to check for and finish off shared dependency upgrades. All comparatively simple tasks with minimal business logic. All boring and repetitive!
Would check out hooks and settings.json customisations for data protection if you haven’t already, but for sure this feels substantially unresolved at present!
I've been building a staff AE agent to review and comment on PRs. It's very much a WIP. I've given it very specific things to look at in skills during a review. It also has running window context of last 10 days PRs and last 4 weekly pr summary.