Senior Data Engineer

Senior Data Engineer

Job Type: Full Time

Location : Melbourne, Australia

We are seeking highly skilled Senior Data Engineers (5 positions) with 8-12 years of experience to join our team. This role is ideal for professionals with expertise in AWS technologies, ETL processes, and big data frameworks. You will be responsible for developing, executing, and optimizing ETL jobs in an AWS environment, ensuring seamless data pipelines and operational excellence.

Key Responsibilities:

  • Design, develop, and manage ETL workflows using Spark and Scala for data transformation and processing.
  • Migrate, test, and deploy ETL jobs in the AWS environment.
  • Utilize AWS services such as EMR, S3, Athena, and SQL to build and manage scalable data solutions.
  • Ensure efficient data storage, processing, and retrieval within AWS infrastructure.
  • Run and execute ETL jobs from Day 1, ensuring reliability and accuracy across multiple runs.
  • Triage, debug, and reconcile ETL workflows to identify and resolve issues efficiently.
  • Work on building and improving ETL frameworks to support business requirements.
  • Utilize tools like Autosys for scheduling and automating ETL job execution.
  • Engage with multiple capability teams to debug and resolve job-related issues.
  • Conduct knowledge transfer (KT) sessions to offshore teams, ensuring smooth operational handoffs.

Preferred Qualifications:

  • Proven experience in running, debugging, testing/reconciling, and deploying ETL jobs.
  • Hands-on experience migrating ETL jobs to AWS environments.
  • Familiarity with building reusable ETL frameworks for scalability and efficiency.
  • Ability to troubleshoot and resolve complex data pipeline issues in collaboration with cross-functional teams.
  • Proficient in using Autosys or similar tools for job scheduling and monitoring

If you’re a passionate Data Engineer looking for an exciting opportunity to work on cutting-edge technologies, apply now and be a part of our dynamic team!