+44 (0)207 549 4040 Speak to a consultant now

Apply for this job

Attach a resume file. Accepted file types are DOC, DOCX, PDF, HTML, and TXT.

We are uploading your application. It may take a few moments to read your resume. Please wait!

Ref: #50355

Principal Data Engineer

  • Practice Data

  • Technologies Business Intelligence Jobs and Data Recruitment

  • Location Remote, - None Specified -

  • Salary 105000

  • Type Permanent

Who are we looking for?

 

We want to work with people who are passionate about building and maintaining reliable, performant data pipelines, and have practical experience in doing so. You should have a solid professional background in data engineering, and a deep understanding of databases. Given the versatile nature of the role, we're looking for someone who can learn fast, enjoys working with others, and is a pragmatic decision-maker.

 

What are you going to do?

 

You will contribute to a variety of exciting projects ranging from designing robust and fully automated ETL/ELT processes to building tools for improving company-wide productivity with data. You have a passion for designing, implementing and operating stable, scalable and efficient solutions to flow data from disparate sources into the data lake. You, as the Data Engineer, will influence data architecture and technology decisions for engineering best practices.

 

  • Build scalable and reliable near real-time data pipeline that collects, transforms, loads and process data from various internal and external data sources,
  • Build a scalable distributed data store that will be a central source of truth,
  • Own data quality for the pipelines you build and make them auditable,
  • Collaborate with data scientists to make data available for ad-hoc querying, real-time dashboards, and production machine learning models,
  • Build self-service tools that help the business users to extract, analyze and visualize data faster and perform advanced analysis,
  • Evaluate new technologies and build prototypes for continuous improvements in Data Engineering.

 

What do you need to do the job?

 

  • BS/MS with a quantitative focus (e.g. Computer Science, Economics, Mathematics, Physics, Statistics) or equivalent practical experience,
  • Experience with AWS is a prerequisite
  • Experience in Big Data, server-less and data streaming technologies (Hadoop, Hive, AWS lambda, Kinesis, Kafka, Spark),
  • Experience in designing ETL / data pipeline solutions (AWS Glue/Apache Airflow preferred),
  • Experience in Amazon Redshift and/or Snowflake highly preferred,
  • Experience in designing, implementing and maintaining production-grade ETL processes and data pipelines,
  • Experience with git, GitHub, and the pull request workflow,
  • Experience with Shell Scripting,
  • Understanding of the application of Continuous Integration/ Continuous Delivery (CI/CD).
  • Experience in building large-scale, distributed systems using Python, Java or Scala,
  • An eye for automation and instrumentation in all data-related aspects,
  • Positive outlook towards work, strong work ethic and ability to work in a team environment.

 

Bonus Points for;

 

  • Experience in a position of leadership
  • Proven experience with Snowflake
  • Proven experience with Kubernetes
  • Proven experience deploying and maintaining production worthy Airflow infrastructure
  • Experience with AWS Sagemaker and AWS ML services
  • Experience in working with requirements provided by data scientists
Apply now