Responsibilities:
- Develop and maintain robust and scalable data architecture aligned with organizational goals.
- Design and implement data models to support business requirements and analytical needs.
- Create, optimize, and manage end-to-end data pipelines for ETL processes.
- Implement automation for efficient and error-free data workflows.
- Oversee and optimize database systems for performance, organization, and indexing.
- Select and implement appropriate database technologies based on project requirements.
- Integrate data from diverse sources, including APIs, databases, and external systems.
- Implement solutions for real-time data integration when required.
- Develop and deploy automation scripts and tools to streamline ETL processes.
- Implement data quality checks and validation processes to ensure accuracy and reliability.
- Proactively address data quality issues and discrepancies.
- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver effective solutions.
- Collaborate on cross-functional projects to meet business objectives.
- Establish and enforce data security measures to safeguard sensitive information.
- Ensure compliance with relevant data protection regulations.
- Provide guidance and mentorship to junior data engineers.
- Contribute to the development of best practices and standards within the data engineering team.
- Monitor and optimize the performance of data systems to ensure high availability and responsiveness.
- Investigate and resolve data-related issues, troubleshoot problems in data pipelines.
- Keep abreast of emerging technologies, tools, and trends in data engineering.
Qualifications:
- Bachelor’s or advanced degree in Computer Science, Engineering, or a related field.
- 6 years of proven experience in data engineering roles.
- Proficiency in relevant programming languages, e.g., Python, Java, and experience with database technologies, e.g., SQL, NoSQL.
- Strong expertise in designing and optimizing ETL processes and data pipelines.
- Strong experience in cloud technologies.
- Excellent problem-solving and troubleshooting skills.
- Strong communication and collaboration skills.