Join us as a Data Platform Engineer
- This is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiences
- You’ll be simplifying the bank through developing innovative data driven solutions, inspiring to be commercially successful through insight, and keeping our customers and the bank safe and secure
- Participating actively in the data engineering community, you’ll deliver opportunities to support our strategic direction while building your network across the bank
- We’re recruiting for multiple roles across a range to levels, up to and including experienced managers
What you'll do
We’ll look to you to drive value for the customer through modelling, sourcing and data transformation. You’ll be working closely with core technology and architecture teams to deliver strategic data solutions, while driving Agile and DevOps adoption in the delivery of data engineering.
We’ll also expect you to be:
- Delivering the automation of data engineering pipelines through the removal of manual stages
- Design, development, testing and implementation of data streaming applications to support our business customers through the software development life cycle
- Developing comprehensive knowledge of the bank’s data structures and metrics, advocating change where needed for product development
- Educating and embedding new data techniques into the business through role modelling, training and experiment design oversight
- Delivering data engineering strategies to build a scalable data architecture and customer feature rich dataset for data scientists
- Developing solutions for streaming data ingestion and transformations in line with streaming strategy
- Identify ways to improve data reliability, efficiency and quality
- Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for the processes dependent on it.
The skills you'll need
To be successful in this role, you’ll need to be a programmer and Data Engineer with a qualification in Computer Science or Software Engineering. You’ll also need a strong understanding of data usage and dependencies with wider teams and the end customer, as well as a proven track record in extracting value and features from large scale data.
You’ll need experience of deploying and managing distributed data/ETL pipelines (batch mode and real time streaming) hosted on Hadoop, Spark, Kafka, Informatica and MongoDB. You will also have experience of managing data engineering tooling/orchestration such as Streamsets and Informatica PWC/BDM/IICS on premise/cloud infrastructure.
You’ll also demonstrate:
- Experience of developing real time data streaming pipelines using Change Data Capture (CDC), Kafka and Streamsets/NiFi/Flume/Flink
- Experience with Change Data Capture tooling such as IBM Infosphere, Oracle Golden Gate, Attunity, Debezium
- Experience of ETL technical design, automated data quality testing, QA and documentation, data warehousing, data modelling and data wrangling
- Expertise in Unix and DevOps automation tools like Terraform and Puppet and experience in deploying applications to at least one of the major public cloud provider such AWS, GCP or Azure
- Extensive experience using RDMS and one of the no-sql database such as MongoDB, ETL pipelines, Python, Java APIs using spring boot and writing complex SQLs
- A good understanding of cloud warehouse such as Snowflake
- A good understanding of modern code development practices
- Good critical thinking and proven problem solving abilities
- Good understanding DevOps working model such as using GitHub, BitBucket, TeamCity along with Jira integration