this post was submitted on 12 Nov 2025
52 points (100.0% liked)

Privacy

43184 readers
417 users here now

A place to discuss privacy and freedom in the digital world.

Privacy has become a very important issue in modern society, with companies and governments constantly abusing their power, more and more people are waking up to the importance of digital privacy.

In this community everyone is welcome to post links and discuss topics related to privacy.

Some Rules

Related communities

much thanks to @gary_host_laptop for the logo design :)

founded 6 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] sem@lemmy.ml 21 points 3 days ago (1 children)

By working as a data engineer in ad-tech companies and seeing how big is actually an amount of information collected about you.

[–] MeowerMisfit817@lemmy.world 3 points 2 days ago (2 children)

Oh, I never seen anyone say they worked as a data manager. Never even heard about this job.

How's it? What do you do?

[–] fushuan@lemmy.blahaj.zone 1 points 2 days ago

*data engineer

I'm also one but I don't work for advertising. Most data engineers work for consulting companies that work for banks. We program automatic data processing pipelines. For example, bank transactions are stored somewhere, all the historic data, that needs processing to then be graphed out for exec number 3, or for whatever.

Other companies might send you files that need to be automatically processed, cleaned, and put correctly where then other tools can pull that data correctly.

We basically do all the background work concerning data manipulation. File processing, databases.. all that stuff. And by databases it can be normal ones like posture to distributed ones like hdfs/hive/athena/whatever.

Ad world is basically the same but with tracking info instead of transactions.

If you are interested in day to day work, it's a mix of coding SQL processes, then porting them to spark/pyspark for distributed massive processing. There are new shiny tools for those that don't know much of the technical side to manage, sorta.

[–] sem@lemmy.ml 3 points 2 days ago (1 children)

There are petabytes of collected data (even in a relatively small ad-tech companies I had a chance to work on; on a facebook/google scale it is much more). Someone should write all the cleaning, processing, de-duplication and matching (aka fingerprinting) steps as well as make this data usable by AI / Machine Learning guys, who will make models that predicts what ad to show to each user based on the available data. I'm working on processing, cleaning, matching and preparing of these data.

[–] MeowerMisfit817@lemmy.world 1 points 2 days ago (1 children)
[–] sem@lemmy.ml 2 points 2 days ago (1 children)

I know that I'm working on a "dark side"... But ad-tech are offering really interesting tasks about building very complex infrastructure besides they are paying well.

[–] sem@piefed.blahaj.zone 1 points 2 days ago

Another sem 👽