Big Data Engineer

  • BAE Systems Applied Intelligenceq
  • Guildford, UK
  • 05/05/2022
Full time Data Engineering Big Data Artificial Intelligence Statistics

Job Description

Job details

Job type
Full-time

Full Job Description

The team:

This role is within the behavioural analytics team of the NNCC (National Network Cyber Centre) project. NNCC is a product sold internationally with the goal of identifying malicious behaviour from advanced threats in network traffic. It is a country level cyber solution and operates at the speed and scale of the national Internet.Using cutting-edge technology, the behavioural analytics team research, prototype, develop and engineer complex analytics, on incredibly large datasets, to identify anomalies, and detect and correlate threats across the cyber kill chain. The analytics team works as both a long-term development team for the core NNCC product, as well as a data lab focussed research team for specific clients. They rapidly spin-up prototypes to serve immediate operational needs.

The role:·

  • To design, implement and test scalable and performant analytic pipelines - with guidance from a Data Scientist and Solution Architect.
  • To contribute to the continuous improvement of the engineering processes and architecture for the analytics solution.
  • To detect and analyse performance bottlenecks in new and existing solutions.
  • To tune analytic implementations to cope with the solution scale – we ingest millions of events per second.
  • To communicate and act on solution improvement ideas.
  • To rapidly learn new technologies.

Your experience:

  • Knowledge of the following is essential
  • Python, our code base is Python based
  • Knowledge of the following big data technologies would be extremely advantageous
  • Spark / PySpark (all data analysis on a Spark cluster)·HDFS (forms our persistence backbone)
  • HBase / Elasticsearch databases (used to store entity data)
  • Kafka(for messaging / subsystem buffering)Knowledge of the following would be desirable but not essential·Ansible (our solutions are installed using Ansible)
  • Airflow/ Jupyter (for job control and one off tasks·A JVM language e.g. Java / Scala
  • Unix shell scripting (BASH for example)·Data formats such as Parquet and Avro

The company:

BAE Systems Digital Intelligence is home to 4,800 digital, cyber and intelligence experts. We work collaboratively across 16 countries to collect, connect and understand complex data, so that governments, nation states, armed forces and commercial businesses can unlock digital advantage in the most demanding environments.BAE Digital Intelligence has a core team of big data professionals, specialising in many areas of technology and supporting a wide range of clients. During your career as a big data engineer you will get the opportunity to move between clients - gaining experience in many areas

Life at BAE Systems Applied Intelligence

We are embracing Hybrid Working. This means you and your colleagues may be working in different locations, such as from home, another BAE Systems office or client site, some or all of the time, and work might be going on at different times of the day. By embracing technology, we can interact, collaborate and create together, even when we’re working remotely from one another. Hybrid Working allows for increased flexibility in when and where we work, helping us to balance our work and personal life more effectively, and enhance wellbeing.

Diversity and inclusion are integral to the success of BAE Systems Applied Intelligence. We are proud to have an organisational culture where employees with varying perspectives, skills, life experiences and backgrounds – the best and brightest minds – can work together to achieve excellence and realise individual and organisational potential.

About BAE Systems Applied Intelligence

We use our intelligence-led insights to help defend Governments, Nations and Societies from cyber-attacks and financial crime. Our customers depend on our evolving capabilities to help them safely grow their organisations. Our unprecedented access to threat intelligence, world-leading analysts and market-leading technology means we can help them to adapt, evolve and stay ahead of the criminals.

Division overview: Government

At BAE Systems Applied Intelligence, we pride ourselves in being a leader in the cyber defence industry, and Government contracts are an area we have many decades of experience in. Our Government business unit houses four of our critical client groups: UK National Security, Global National Security, defence and central government. Government and key infrastructure networks are critical targets to defend as the effects of these networks being breached can be devastating.

As a member of the Government business unit, you will defend the connected world and ensure the protection of nations. We all have a role to play in defending our clients, and this is yours.