How AI Changes 4 Core Data Roles
What are the core skills required for data analysts, data engineers, analytics engineers, and data scientists in the age of AI?
When I first realized I wanted a career in data, I wanted to be a data scientist. I was interning at a fashion company at the time, and someone in the office had introduced me to the idea. I loved math and solving problems- it seemed like the perfect fit!
In my pursuit of data science, Capital One recruited me to join a 6-month coding bootcamp, which led to a full-time data engineering role. Instead of attending a data science graduate program, I accepted the offer.
I worked as a data engineer for two years before realizing it wasn’t quite the right fit for me.
While working as a data engineer, I became interested in data analysis. I wanted to interact with the business and see how my work directly impacted revenue. Unfortunately, I didn’t pass the interview process for the data analyst role, leading me on a search for a data analyst role outside of Capital One.
While browsing data analyst roles on LinkedIn, I came across a role for an analytics engineer. The requirements were a perfect blend of the skills I had as a data engineer and the experience I sought as a data analyst.
Ever since that moment, I’ve been sharing my journey learning the ins and outs of analytics engineering. Since I first became an analytics engineer, the industry has shifted. Each data role no longer looks like it once did because of AI.
In this article, I’ll discuss the main differences between the 4 most popular data roles and how each one is changing because of AI.
Today’s newsletter is sponsored by Liquibase.
The best way to support my work is to check out the powerful data governance features offered by Liquibase and book a demo.
Data Analyst
Data analysts work closely with business stakeholders to define metrics and the different dimensions to slice those metrics by. They are typically focused on a particular area of the business like product, growth, or finance.
Super power: Making the most of the data available to them to get insights to business questions fast!
Technical skills:
SQL
Excel
BI tools like Looker, Tableau, Power BI, etc.
A/B testing and experimentation
Data analysts don’t focus on loading data, testing data quality, or transforming it within the warehouse, but rather find a way to use the data available to them by creating dashboards and reports in BI tools and using SQL to produce ad-hoc queries.
How is this changing?
Dashboarding is still important and necessary, but quickly changing. I’ve found that stakeholders are starting to use dashboards as a way to download filtered datasets that they then use with Claude. How the data is displayed and how metrics are calculated are less important than having a clean set of data to provide them.
This means data modeling, the core skill of analytics engineering, is going to become more important for analysts to learn going forward.
Join the waitlist for my data analyst to analytics engineer course.
Data Engineer
Data engineers set a stable foundation for data systems. They build infrastructure to support ETL pipelines (typically using Python), focusing on data quality from ingestion to orchestration. They deploy and manage cloud infrastructure that supports data tooling.
Super power: Creating stable infrastructure that ensures performant, quality analytics pipelines.
Technical skills:
Python
SQL
Cloud tools like AWS, Azure, and GCP
Database management
Data warehouse performance (indexes, partitions, etc.)
Data engineers don’t necessarily interact with the business but create a stable foundation for analytics engineers to build business solutions. They closely work with other engineers to define contracts on expectations around the source system data.
How is this changing?
AI thrives at building and applying logic to your systems. This means that it can do pretty well at pure data engineering work. Data engineers no longer need to write Python scripts and pipelines from scratch. AI is doing this for them.
As a data engineer, you need to know how to use these AI tools to do the building for you so you can focus on more complex data engineering problems like handling large amounts of data in a cost-effective and performant manner.
Data Scientist
Data scientists build predictive/ML models (which are different than data models!) and find statistically significant patterns in large-scale datasets. They have strong statistical knowledge, as all of the data work they do is centered around this.
Super power: Uncovering hidden patterns in data, building off of them in a way that allows the business to predict future behaviors.
Technical skills:
Python (libraries such as NumPy, Pandas, ScikitLearn, Matplotlib)
R
basic SQL for data cleaning
Like data analysts, data scientists work cross-functionally to understand the needs of the business. While they do a lot of deep research-like work, they also focus on the value it will have for their stakeholders.
How is this changing?
Similar to data engineers, AI is replacing a lot of data scientists’ technical-heavy work. Having the sharpest Python skills no longer matter as much as understanding the business well enough to tweak the model based on how it changes. Again, you need to be able to use AI tools to speed up your development!
Analytics Engineer
Analytics engineers own the analytics pipeline. They build dimensional data models from start to finish, documenting and testing them. They architect the data warehouse for analytics use cases, managing governance and cost, while also managing ETL pipelines.
Super power: Taking business requirements and translating them into scalable, reusable data models.
Technical skills:
SQL
dimensional data modeling
modern data stack tools like dbt, Fivetran, Snowflake, etc.
data governance
cloud data warehousing
Analytics engineers work closely with data analysts to understand what models need to be built to support stakeholder needs. Rather than being told what to build, they are pivotal in deciding what data models need to be built to fulfill the business’s needs.
How is this changing?
Data modeling and data governance are becoming more and more important with the increased adoption of AI- two areas analytics engineers thrive in. I believe data analysts will need to uplevel to analytics engineers in order to stick around.
Analytics engineers need to understand how stakeholders and AI agents will be using the data models they are building. They will also need to shift their focus to semantic/context layers and how these need to be build to support business goals.
The future of data jobs
Overall, you need to know how to leverage AI agents to work faster. Everyone is using them and if you don’t, you will fall behind. This being said, it’s still important to understand the basics of data. AI agents aren’t data experts, you are. You need to allow them to build while providing them context and direction.
Prioritizing foundational knowledge is still necessary, and exactly why you don’t see entry level data professionals replacing senior ones.
The biggest shift will come with data analysts. Data analysts need to learn more analytics engineering skills like dbt and data modeling in order to still provide value in the age of AI. Otherwise, I believe they will eventually be replaced by self-service analytics.
Join the waitlist for my data analyst to analytics engineer course.
That’s most likely awhile away, but the shift will happen eventually. Build analytics engineering skills up now while there’s time!
Have a great week!
Madison
Looking to upskill your analytics engineering skills?
Hire me to train your data team in analytics engineering principles
Buy my ebook on the ABCs of analytics engineering
Invest in a paid newsletter subscription for our upcoming Data Pipeline Summer challenge and customized learning plan
Join the waitlist for my data analyst to analytics engineer course