(2017) Rise of the Data Engineer (medium.com)

submitted 1 year ago by Reader9@programming.dev to c/data_engineering@programming.dev

5 comments fedilink hide all child comments

This article helped defined the “data engineer” role so I’d say it belongs here!

Although some time has passed, I find it very relevant: SQL is used more than ever, graphical ETL tools that don’t output code are rare and vendors are still trying to convince executives to trust all their data to proprietary data warehouses.

The author Maxime Beauchemin also wrote Airflow and Superset so they have some experience worth listening to.

you are viewing a single comment's thread
view the rest of the comments

[-] Reader9@programming.dev 3 points 1 year ago

That seems like a good idea by removing ambiguity about what the necessary skills are.

When joining a new company, I once asked a wise colleague “are you a data engineer or a backend engineer?”. They replied “I’m a software engineer” and ever since I have given the same answer, for reasons similar to your post.

I have also seen “data engineer” used at facebook to indicate someone who writes SQL but not other programming languages, another potential reason not to use this as a job title IMO.

this post was submitted on 08 Aug 2023

8 points (100.0% liked)

Data Engineering

379 readers

2 users here now

A community for discussion about data engineering

Icon base by Delapouite under CC BY 3.0 with modifications to add a gradient

founded 1 year ago

MODERATORS

ericjmorey@programming.dev