The traditional data team consisted of data engineers and data analysts. Where data engineers were responsible for supplying the analysts with the data they required. This meant Extracting the data from several different sources, in several different languages and formats, Transforming them to a usable form, and loading them into the data warehouse or datamart for the Analyst to use.
This meant that the Engineer had to have a lot of skills to bring the data from start to the endpoint, with a lot of steps and meetings in between.
With the continually changing landscape of data and visualization, we’re not only seeing new tools coming to the scene but a new way of handling data inside of data teams as well. This change has also paved the way for a new role inside of those teams: The Analytics Engineer.
The Analytics Engineer has jumped into the gap that’s been created by the change of how we handle data. With the introduction of cloud-focused data infrastructures like Snowflake, more and more businesses are also shifting from ETL towards a more ELT-focussed approach. This means that the last part of the data journey, inside of the data structures, is the Transformation step instead of Loading the data.
We see that the moment the raw data hits the database and has been extracted from its source, the data engineer can sign off and the analytics engineer takes over. This means that the engineer can focus on extracting and maintaining the infrastructure and pipelines that are needed to facilitate all of the data movement. The Analytics Engineer then takes the data and, in agreement with the Analysts, Transforms the data to the desired granularity and form.
With the addition of the Analytics Engineer to a team, the Data Engineer can focus on the things that they do best; getting the raw data into 1 source.
This also means that the Data Analyst has a more direct sparring partner where the data is concerned, someone that is more able to help them with their specific data needs.
Because of this, we can only see good things coming from this change and are happily looking forward to seeing this change take its hold inside of businesses.