job description
Are you a passionate Data Engineer looking to push the boundaries of modern data architecture? Clicktek Consulting Inc is seeking a highly skilled Data Engineer to join our dynamic team in Metro Manila. In this role, you will be at the forefront of cloud-based data transformation, leveraging the Microsoft Fabric ecosystem to build scalable, robust, and efficient data solutions.
You will play a pivotal role in designing and maintaining both batch and real-time streaming pipelines. We are looking for a technical expert who thrives in a collaborative environment and is eager to optimize data workflows using Data Factory, Dataflows Gen2, and Spark notebooks. If you are driven by innovation and have a deep understanding of the Microsoft data stack, we want to hear from you.
Responsibility
- Design, develop, and maintain high-performance batch and streaming data pipelines within the Microsoft Fabric ecosystem.
- Configure and monitor automated data workflows utilizing Microsoft Data Factory and Dataflows Gen2.
- Develop and optimize complex data transformations using Spark notebooks for large-scale datasets.
- Collaborate with cross-functional teams to define data requirements and translate them into scalable architectural designs.
- Ensure data quality, integrity, and security across all pipeline stages.
- Troubleshoot and resolve performance bottlenecks in data integration processes.
- Maintain comprehensive documentation of data models, pipeline logic, and infrastructure configurations.
Qualifications
- Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or a related field.
- Minimum 3+ years of professional experience in data engineering or cloud-based data integration.
- Proficiency in the Microsoft Fabric suite, including Data Factory and Dataflows Gen2.
- Strong expertise in PySpark or Scala for building efficient Spark notebooks.
- Solid understanding of SQL, data modeling, and ETL/ELT design patterns.
- Familiarity with cloud data warehousing concepts and big data technologies.
- Strong analytical, problem-solving, and communication skills.