Data Engineer vs Analytics Engineer
The differences in responsibilities, skills, and tools used
One of the questions I’m asked most often is what is the difference between a data engineer and an analytics engineer?
And that’s a great question.
It’s often unclear from the posts you may read on LinkedIn, job descriptions, and data-related articles on Medium.
While this line is blurring more and more, there’s a general idea of what each of these roles does. Some people may disagree with what I have to say, but from my own experience, this pretty much sums it up.
Responsibilities of a Data Engineer
Let's start by talking about the role of a data engineer. A data engineer is someone who builds the infrastructure to support the storage and movement of data.
They're not really looking at the data itself but rather focusing on how they can support the data.
This includes using Python to write data pipelines, Spark to process data, and cloud technologies like AWS to deploy infrastructure. A data engineer works closely with other developers like software engineers.
When I worked as a data engineer, I first focused solely on DevOps and deploying infrastructure using AWS. I used different pipeline tools like Data Dog and Jenkins to monitor and deploy code. I wrote a bunch of Python scripts that cleaned up our AWS environments and helped manage EC2 clusters.
In a different data engineering role, I focused on building applications like Python APIs to help capture our data. Here, I was building infrastructure and processing tools, not really looking at the data itself.
However, also in this role, I frequently work with SQL and dbt. This was out of the ordinary for data engineers who usually work with software development languages and cloud platforms like AWS and Azure.
At the end of the day, it really depends on the company you work for and how they label their roles. This is why I wrote a whole article on finding the perfect role for you based on the job description rather than the title.
Typical data engineering skills include:
data ingestion
orchestration
data warehousing
managing/building infrastructure
DevOps
Tools/languages typically used by data engineers:
Python
AWS
Git
Bash
Spark
Hadoop
Responsibilities of an Analytics Engineer
I like to think of an analytics engineer as the baby between a data engineer and a data analyst. Analytics engineers inherit some qualities from both of their data parents.
Some analytics engineers look a little more like their data engineer parent, others look more like their data analyst parent.
My main point is that analytics engineers are technically savvy but can also speak to the business about its needs.
An analytics engineer takes data from its source and transforms it, making it super easy for the data analyst or business user to analyze, visualize, and take action on it.
Not only do they work directly with the data, trying to understand it and model it in a way that can easily mimic business processes and systems, but they also move it around. Analytics engineers often take ownership of ingesting, transforming, and scheduling data models.
They really do own the modern data stack!
This being said I think the most telling skill of an analytics engineer is their ability to understand the business and how the data needs to be modeled to mimic its needs. In turn, they make it easy to streamline metrics, populate dashboards, and produce reliable reports.
Typical analytics engineering skills include:
orchestration
business communication & understanding
Tools/languages typically used by analytics engineers:
SQL
dbt
Reverse ETL (Census, Hightouch, Rivery, etc.)
Choosing what’s best for you
The best way to know what’s best for you is to try it. Luckily, there are so many free resources on both data engineering and analytics engineering (*cough cough* this newsletter).
I recommend reading a few different articles on the tools, languages, and skills I mentioned above. Try to learn a few of them and see how it goes. What do you enjoy? What do you dislike? This will help point you in a certain direction.
Here are some of my favorite newsletters for data engineering knowledge:
Check these out and see if the topics interest you. Luckily, data engineering and analytics engineering often overlap. Reading these newsletters along with
is a great way to keep up-to-date on the latest happenings in the data world.Generally, if you like technical challenges that require a lot of coding and building, data engineering will be for you. If you like figuring out how to translate business logic into data, analytics engineering will be more your speed.
As always, if you have any questions, I am here to answer them.
You can also book time with me on my calendar to discuss a career transition or any questions you may have about analytics engineering.
Until next week!
Madison Mae
That's an outstanding explanation! It's how we separate these two roles at the place where I work.
Very good and simple intro. However, based on type of organisation, complexity of process, data sources, infra, tools - both roles gets into different dimension of complexity.
However apart from role, looks at what makes organisation successful wrt to data and analytics game. Tune capabilities, skills and presentation to have longer and successful career.