Trusted listing
Data Engineer
Smarterise Integrated Solution
Lagos, Nigeria
Permanent
Published 1 month ago · Expires 3 weeks from now
Job description
We invite applications from suitably qualified candidates.
Position Overview:
- As a Data Engineer at Smarterise, you will play a crucial role in designing, building, and maintaining our data infrastructure to support various data-driven initiatives and analytics projects. You will collaborate closely with cross-functional teams including data scientists, analysts, and business stakeholders to ensure that our data systems are optimized for performance, scalability, and reliability.
- Data Pipeline Development: Design, develop, and maintain robust data pipelines to ingest, process, and transform large volumes of structured and unstructured data from diverse sources such as sensors, IoT devices, and operational systems.
- Data Modeling: Work with analysts to design and implement scalable data models that meet the requirements of analytics and reporting applications. Ensure data integrity, consistency, and accuracy throughout the data lifecycle.
- Data Integration: Integrate data from disparate sources and formats, including time-series data, geospatial data, and streaming data, into our centralized data lake or data warehouse environment.
- Performance Optimization: Optimize data pipelines and queries for performance, scalability, and efficiency. Identify and address bottlenecks in data processing and storage to meet SLAs and business requirements.
- Data Quality Assurance: Implement data quality checks and validation processes to ensure that incoming data meets predefined standards for accuracy, completeness, and consistency.
- Infrastructure Management: Manage and monitor data infrastructure components such as databases, ETL tools, and cloud services. Implement best practices for security, access control, and disaster recovery.
- Collaboration and Communication: Collaborate with cross-functional teams to understand their data requirements and provide technical expertise and guidance. Communicate effectively with stakeholders to align data engineering solutions with business objectives.
- Continuous Improvement: Stay updated on emerging technologies, tools, and trends in data engineering and apply them to improve our data infrastructure and processes. Drive continuous improvement initiatives to enhance data quality, performance, and reliability.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Proven experience (3-5years) in data engineering roles, preferably in the energy or utilities industry.
- Proficiency in programming languages such as Python, Java, or Scala.
- Hands-on experience with big data technologies such as Hadoop, Spark, Kafka, and NoSQL databases.
- Strong understanding of data modeling concept...