Hi,
I'm excited to share that one of our clients is hiring for a Senior Data Engineer
in South Africa.
It's a fully remote job.
Below are the job details. If you're interested, please send your CV to apply.
Title: Senior Data Engineer
Location: South Africa
Duration: 6 months contract (possibility of an extension)
Job Type: Fully Remote
Purpose of the role:
This role is integral to the design, development, and maintenance of our data solutions, ensuring scalability, reliability, and performance in our cloud-driven ecosystem. The ideal candidate will possess strong technical expertise in data engineering, coupled with a proven track record in leading technical teams and implementing data-driven projects in an Agile environment.
Key Responsibilities & Outputs
- ETL/ELT Pipeline Development: Build and maintain efficient ETL/ELT pipelines using tools such as Apache Airflow and PySpark.
- Data Modelling and Design: Develop database schemas, dimensional models (Kimball/Inmon), and support data normalization for relational and NoSQL databases.
- Data Warehousing: Participate in the development and maintenance of data warehouses, data lakes, and data lakehouses.
- Database Management: Work with relational database systems such as Azure SQL, PostgreSQL, and Google Big Query, as well as NoSQL systems like MongoDB and Google Firestore.
- File Format Expertise: Handle structured, semi-structured, and big data file formats including Avro, CSV, Parquet, ORC, and Delta.
- API Integration: Develop and maintain APIs for seamless data integration and workflows, with an understanding of REST and microservices architectures.
- Code Maintenance: Oversee the maintenance and optimization of the codebase using Git for version control.
- Testing and Quality Assurance: Implement integration testing and ensure high-quality deliverables for all new data processing scenarios.
- Collaboration: Provide technical design and coding assistance to team members, ensuring the successful delivery of project milestones.
- Analysis: Assess and integrate new data sources into pipelines to meet evolving business needs.
Education, Experience & Knowledge
- Strong proficiency in Python and SQL(PostgreSQL or SQL Server preferred).
- Hands-on experience with Apache Airflow and PySpark.
- Familiarity with Databricks(preferred but not mandatory).
- Working knowledge of cloud platforms such as Azure, GCP, or AWS.
- Experience with data warehousing concepts, dimensional modelling, and database normalisation.
- Understanding of big data file formats such as Avro, Parquet, ORC, and Delta.
- Proficiency in working with APIs, REST, and microservices architectures.
Education & Experience:
- Bachelor’s degree in computer science, Data Science, or related fields.
- 5+ years of progressive experience in data engineering, cloud computing, and technology implementation.
- Experience managing multi-shore projects and working within cloud ecosystems (SaaS/PaaS).
- Proven experience leading technical teams and mentoring team members.
Soft Skills & Management Expertise:
- Demonstrated ability to lead and mentor teams in an Agile/Scrum environment.
- Strong communication and collaboration skills to work with cross-functional teams.
- Problem-solving mindset with a focus on continuous improvement and innovation.
- Excellent organisational and project management abilities.
