Job description
Conexus are partnered with a leading European utilities company on a long term project. To support the progression of their new internal platform, we are searching for a German-speaking, freelance Cloud Native Data Engineer to join them in Berlin.
Key Responsibilities:
- Design, develop, and maintain scalable data architectures, including databases, data lakes, and data warehouses, to efficiently handle mass data processing within defined time frames.
- Implement cloud-native principles to promote decentralised data ownership and architecture, including designing and validating decentralised data architectures through Proof of Concepts (POCs).
- Design and implement data models aligned with business requirements, collaborating with data scientists and analysts to structure data effectively.
- Develop and maintain ETL (Extract, Transform, Load) processes to move and transform data from various sources into the data infrastructure, ensuring seamless integration into decentralised data products.
- Implement and enforce data quality standards and governance policies, including developing and maintaining data documentation for meta data, lineage, and data dictionaries.
- Assist in migrating existing applications from OpenShift to EDP Kubernetes environments (based on RKE2), ensuring smooth transitions.
- Document POC results, architecture decisions, and lessons learned to facilitate knowledge sharing and future reference.
Profile Requirements:
The ideal candidate is a senior Cloud Native Data Engineer with a strong technical background in data engineering, particularly in implementing scalable data architectures. They should have excellent collaboration skills and an innovative mindset to contribute effectively to the organisation's data strategy.
- Proven experience in designing and implementing scalable data architectures, including extensive experience with databases, data lakes, data warehouses, and message brokers like Kafka.
- Hands-on experience with ETL processes and data integration from various sources, with familiarity with modern data technologies and cloud services.
- Proficiency in designing data models that meet business needs, with knowledge of Data Mesh concepts.
- Ability to evaluate and recommend the adoption of new tools and technologies, staying updated on emerging trends in data engineering.
- An innovative mindset to propose enhancements to the organisation's data architecture.
Skill Requirements:
- Minimum of 5 years of experience as a Cloud Native application engineer, with a strong understanding of rearchitecting monolithic architectures into microservices-based Cloud Native architectures.
- Proficiency in at least one programming language, such as Java or Scala.
- Experience with Big Data technologies and frameworks, including workflow orchestration (AirFlow, Oozie), data integration/ingestion (Nifi, Flume), messaging/data streaming (Kafka, RabbitMQ), data processing (Spark, Flink), RDBMS (PostgreSQL, MySQL), NoSQL (MongoDB, Cassandra, Neo4j), and time series databases (InfluxDB, OpenTSDB, TimescaleDB, Prometheus), or their cloud counterparts.
- Proficiency in deployment and containerisation tools like Docker, Kubernetes, Helm, OpenShift, and CI/CD & DevOps tools such as Azure DevOps, GitHub Actions, GitOps, Gitlab, along with Bash/Shell scripting and Linux.
- Familiarity with agile development methodologies and tools like Scrum, SAFE, JIRA, and Confluence.
- Proficiency in both German and English is essential.
If this role is of interest, please respond with your latest CV for your consideration.
