I've Reached AI Overwhelm.

The problems I'm seeing with using AI in my workflow and how I'm planning to fix them

It’s official. I’m overwhelmed and frustrated by AI.

I’ve been dedicating most of my days for the last few months to successfully working Claude Code into my analytics engineering workflow. For most of the process, it was fun, new, and exciting.

I felt like I was flying through my work at record speed. Claude was my coding companion who helped me solve anything and everything!

Don’t get me wrong, Claude is still helping me work faster and more efficiently.

However, now that the shininess has worn off, I’m starting to see it for who it really is.

Over-eager and profligate.

So what does this mean for AI and analytics engineering?

---

Today’s newsletter is sponsored by Greybeam.

Thanks to them, this post is free for everyone to read! The best way to support my work is to check out the cost savings you can get with Greybeam and click the links below.

Your snowflake bill is bigger than it should be.

Most cost optimization advice stops at “set auto-suspend to 60 seconds.” This guide doesn’t. Greybeam’s free Snowflake Cost Optimization Guide walks you through 10 checks that can cut 30–60% of your costs with the exact SQL to run against your account today.

The greatest cloud savings emerge from costs never born. This guide helps you find the spend most teams overlook, fix it, and put guardrails in place so it stays that way.

Get the free guide · Get a demo

---

2 Major Problems with AI

I’ve said this before, but I really want to emphasize this now: human discernment is so important.

This week, I used Claude Code to make some basic changes to a few metrics. The changes involved adding an additional app mode for two very specific metrics. Instead of updating the three metrics that I needed, it decided to update all ecomm metrics.

I kept going back and forth with it, specifying that I only wanted it to update the two related metrics and touch nothing else. For some reason, Claude wasn’t understanding and kept overstepping.

Instead of just updating the metrics myself, I ended up wasting an hour trying to reason with it, only to still end up with the wrong outcome.

No matter how much context I gave it, and how specific I tried to be, we were running in circles. This is the perfect example of where you can land with AI when you’re not careful.

There are even rumors going around that Claude has gotten dumber in recent weeks…

I see two main problems:

• AI agents wanting to do too much, too fast, all at once

• AI agents burning costs and wasting resources

Problem #1: Overstepping boundaries

AI agents are like overeager circus monkeys wanting to please their masters. They will do something just for the sake of doing it, like change metrics you didn’t ask to be changed.

This can quickly become overwhelming when you’re trying to complete a very small, specific piece of work, and it starts touching all areas of your code base. Not only does this make for bad data changes, but it also frustrates us as analytics engineers.

I’ve realized that I need to always utilize plan mode to get the result I want. It’s much easier to narrow the scope in a plan before AI makes the actual changes to your code. Plan mode allows you to reason back and forth and dump the context that you need in one brain dump before the work gets done. This then helps provide all the context upfront.

I used to think plan mode was just for bigger projects like complex data modeling, but I’ve found it necessary for smaller-scoped work like the “simple” metric change I mentioned above.

Not to mention, better scoping of your work also gives greater insight into what AI does and doesn’t have access to.

Today, I used my notebook validation skill to open a Snowflake notebook, and Claude tried to brute force a permissions override from the very clearly scoped service account to my personal user, despite very clear directions not to do this.

This is why I never auto-accept its actions and use plan mode before implementing any changes.

Problem #2: Increasing costs

With AI being shoved in our faces every second of every day, the easy thing to do is default to AI. Instead of adding a simple test myself, I ask AI to do it. Instead of writing the SQL myself, I ask AI to do it.

It can be easy to default to AI for every analytics engineering task, despite the true level of effort being quite small.

Well, AI can do it, so why would I ever do it myself!?

Because of costs, that’s why.

The costs of using AI for the small, very human-capable tasks are quite high.

It’s high on the environment.

It’s high on our mental health and cognitive abilities.

It’s high on our’s or our company’s wallets (whether they realize it or not yet).

How you use Claude Code affects how much you pay. Resuming a chat session, for example, has been shown to increase token usage my 15%. How you use it with data is even more risky!

If you are thoughtful about the rollout of your data warehouse with AI, you will be closely monitoring behaviors and costs. However, what happens when companies roll out widespread data access through Snowflake MCP via Claude Code? Claude will be able to run any gnarly query against your data warehouse without many guardrails. How will long-running and inefficient queries be managed?

It’s become more and more important to use tools and strategies to monitor and control your data warehouse costs with AI. Trust me, you don’t want a surprise bill from Snowflake. I’ve learned that lesson the hard way.

Rawdogging without AI

Frankly, once you learn how to harness AI to do a lot more, I think we all need to then scale it back to figure out what we can rawdog without AI.

I don’t know about you, but I don’t want all the skills I built years developing to disappear. The AI fatigue on the brain is real! I plan to utilize AI for complex planning, dynamic code changes, and documentation. This is where I see it make the biggest difference in my work.

I plan to keep my skills sharp by manually opening a GitHub PR every now and then and continuing to write one-off SQL queries.

As analytics engineers, we need to be honest with ourselves about what’s working and what’s not. Continue learning and developing AI in your workflows, but do so where it makes sense. Don’t overcomplicate the simple things.

In the meantime, start really thinking about governance and how these problems will scale when your greater organization gets access to your data for AI usage.

This is where the real difficult problems will start. They’re better solved on a small scale in your personal workflow first, before stakeholders start facing these same problems.

Have a great week!

Madison