senior Data Engineer ( AWS )

Senior Data Engineer ( AWS / Redshift / Python )

• Outside IR35
• £450-£500 Per Day
• Hybrid - London ( 1 day in the office a week )
• 3-6 Month Contract

You will help to establish the infrastructure in AWS to handle enterprise-scale volumes of data from numerous sources, build out the data lake and data warehouse for operational and analytical discovery as well as helping to develop data pipelines for a number of different applications.

Key Responsibilities

• Improving the maturity of the data estate with better use of CloudFormation and CI/CD
• Help to build and support an enterprise-level data warehouse in Redshift
• Build ETL/ELT data pipelines with AWS Glue and Lambda to clean, transform and load data from its raw format to multiple endpoints and for multiple use cases
• Maintain and improve data governance within the cloud estate
• Produce data exports and queries for teams throughout the business

Essential Skills & Experience

• Excellent experience working with AWS, including Lambda, Glue, SQS, SNS, S3, ECS, CloudWatch and Step Functions
• Good experience working with Amazon Redshift or PostgreSQL
• Skilled in SQL
• Proficient in developing with Python
• Good experience of CI/CD and AWS SAM
• Understanding of the AWS Well-Architected Framework
• Experience working in a fast-paced agile environment
• Understanding of how to architect resilient, highly available and efficient data pipelines in Databricks with Pyspark.

Desirable Skills & Experience

• Experience working with DynamoDB, including DynamoDB Streams
• Familiarity with business intelligence tools such as Tableau or QuickSight
• Understanding of the Kimball methodology for data warehousing
• Awareness of GDPR/PECR/DPA
• Knowledge of Amazon Kinesis, IAM, Athena, Glue
• Proficient in building Spark/PySpark jobs to process big data volumes on structured and semi-structured data
• AWS certification is a plus