job description
Are you a seasoned Big Data Technical Lead with a passion for architecting scalable data solutions? INFOSIGHT CONSULTING is seeking a visionary leader to drive our next-generation data platforms from the tropical hub of Bali, Indonesia.
In this high-impact role, you will spearhead the design, development, and optimization of our Hadoop-based ecosystems, leveraging cutting-edge technologies like Hive, Spark, and SQL to unlock actionable insights. You’ll collaborate with cross-functional teams to solve complex data challenges while mentoring junior engineers and shaping our data strategy.
This is a contract position offering competitive compensation (USD 8K–12K/month), the flexibility to work remotely from Bali’s vibrant tech communities (Canggu, Ubud, Denpasar, etc.), and the opportunity to work on transformative projects for global clients.
If you thrive in dynamic environments and want to combine technical excellence with a lifestyle in paradise, we’d love to hear from you.
Responsibility
- Architect and optimize large-scale Hadoop ecosystems (HDFS, YARN) for high-performance data processing.
- Design and implement ETL pipelines using Spark, Hive, and SQL to ingest, transform, and analyze petabyte-scale datasets.
- Lead a team of data engineers, providing technical guidance and code reviews to ensure best practices.
- Collaborate with data scientists and analysts to build scalable ML/data models on distributed frameworks.
- Monitor and tune cluster performance, ensuring 99.9% uptime and cost-efficient resource utilization.
- Develop data governance policies and security protocols for sensitive datasets.
- Evaluate and integrate emerging big data tools (e.g., Kafka, Flink, Delta Lake) to future-proof our stack.
- Document technical designs and present solutions to stakeholders in clear, business-aligned terms.
Qualifications
- 10+ years of hands-on experience in big data engineering, with at least 3 years in a leadership role.
- Expertise in Hadoop ecosystem (HDFS, MapReduce, YARN) and distributed computing principles.
- Proficiency in Spark (PySpark/Scala), HiveQL, and advanced SQL for complex queries.
- Experience with cloud platforms (AWS EMR, GCP Dataproc, or Azure HDInsight) and containerization (Docker, Kubernetes).
- Strong problem-solving skills in data modeling, partitioning, and optimization for large datasets.
- Familiarity with data warehousing (Snowflake, Redshift) and real-time processing (Kafka, Flink).
- Excellent communication skills to bridge technical and non-technical teams.
- Bachelor’s/Master’s degree in Computer Science, Engineering, or related field (or equivalent experience).