Senior Data Engineer for Comprehensive Project
Budget / Salary$30–250
TypeFreelance project
LocationRemote
Posted1 hour ago
Hi, I am looking for an experienced Senior Data Engineer / Data Engineering Architect to design and implement a complete end-to-end, production-style data engineering project that covers a broad range of modern data engineering technologies.
This is not intended to be a small ETL project. I want to build a realistic, enterprise-level data platform that demonstrates how the different components of a modern data engineering ecosystem work together.
The project should cover the complete data lifecycle:
Data Sources → Ingestion → Streaming → Batch Processing → Data Lake → Transformation → Data Warehouse → Data Quality → Orchestration → Analytics → Monitoring → Deployment
I also want the freelancer to explain the architecture and technologies clearly throughout the project so that I can understand why each technology is being used, what problem it solves, and how the components integrate with each other.
Technical Stack: Python, SQL, Apache Kafka, Spark/PySpark, Databricks, Delta Lake, Airflow, AWS or Azure, Snowflake, dbt, Hive, REST APIs, ETL/ELT, batch and real-time/streaming data pipelines, data lakes/lakehouses, data warehousing, dimensional data modeling, data quality, CI/CD, Git/GitHub, Docker, monitoring, and ideally Terraform.
This is not intended to be a small ETL project. I want to build a realistic, enterprise-level data platform that demonstrates how the different components of a modern data engineering ecosystem work together.
The project should cover the complete data lifecycle:
Data Sources → Ingestion → Streaming → Batch Processing → Data Lake → Transformation → Data Warehouse → Data Quality → Orchestration → Analytics → Monitoring → Deployment
I also want the freelancer to explain the architecture and technologies clearly throughout the project so that I can understand why each technology is being used, what problem it solves, and how the components integrate with each other.
Technical Stack: Python, SQL, Apache Kafka, Spark/PySpark, Databricks, Delta Lake, Airflow, AWS or Azure, Snowflake, dbt, Hive, REST APIs, ETL/ELT, batch and real-time/streaming data pipelines, data lakes/lakehouses, data warehousing, dimensional data modeling, data quality, CI/CD, Git/GitHub, Docker, monitoring, and ideally Terraform.
Apply on Freelancer →
Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.