Key Responsibilities:
- Develop and maintain data pipelines and ETL/ELT/analytics engineering workflows to support advanced search and retrieval capabilities.
- Collaborate with data teams to understand requirements and automate deployment and monitoring.
- Optimize data storage and troubleshoot issues to enhance performance.
- Work closely with data engineers and relevant parties to ensure sprints are planned out properly and completed timely.
- Perform BAU monitoring, investigations and resolutions.
- Assist with data governance and data management initiatives.
- Production and application support from a day-to-day basis.
Skillset Requirements
- Possess a degree in Computer Science/Information Technology or related fields.
- At least 5 years of hands-on experience in software or data engineering with proficiency in Python.
- Strong experience in unit and integration testing.
- Familiarity with DevOps practices and Agile methodologies.
- Experience with AWS and Kubernetes (K8s).
- Familiarity with data platforms such as Snowflake, Databricks, Apache Spark, Apache Hive, open table formats (Delta Lake, Apache Iceberg), and vector databases.
- Experience with orchestration tools such as Apache Airflow, Dagster, Prefect, and Temporal.
- Familiarity with GitHub workflows and Datadog.
- Good written and verbal communication skills.
- Agile, fast learner and able to adapt to changes.
