At emagine, we are looking for skilled Data Engineers to enhance our clients team. The ideal candidates will possess a strong background in data engineering, with expertise in data processing and analytics. You will be responsible for designing, implementing, and maintaining data pipelines and architectures that support our clients business needs.
Main Responsibilities:
Design and develop scalable data pipelines using Apache Spark.
Implement data serialization formats, particularly Avro, for efficient data storage and transmission.
Utilize Databricks for data processing and analysis.
Collaborate with data scientists and analysts to gather and address data requirements.
Ensure data quality and integrity across the data lifecycle.
Optimize data workflows for performance and scalability.
Monitor and troubleshoot data pipeline issues.
Stay updated with the latest industry trends in data engineering.
Key Requirements:
Bachelor's degree in Computer Science, Engineering, or a related field.
Proven experience as a Data Engineer or in a similar role.
Strong proficiency in Apache Spark and its ecosystem.
Experience with Avro for data serialization.
Familiarity with Databricks functionalities.
Knowledge of SQL and NoSQL databases.
Excellent problem-solving skills and attention to detail.
Ability to collaborate effectively in a team environment.
Nice to Have:
Experience with cloud platforms (AWS, Azure, GCP).
Knowledge of data warehousing concepts and tools.
Familiarity with ETL processes and tools.
Start: Asap
Onsite/remote: Minimum 3 days onsite per week
Duration: 6 months
