JOB DETAILS

The role
 
The Data Engineer is the designer, builder and manager of the information or "big data" infrastructure, preparing data for analytical or operational use.  You will design, construct, test and maintain a data pipeline to pull together information from different source systems; integrating, consolidating and cleansing data; and structure it for use in individual analytics applications.  You will work proactively to:
 

• Design, construct, install, test and maintain highly scalable data management systems.
• Analyse new and existing data sources to assess their applicability to address the business issue.
• Extract and manipulate data from a variety of sources and subsequently cleanse, standardize, scale, bin, categorise, tokenise, stem and transform its order to get the data into a state suitable for further analysis.
• Determine whether a given problem can be usefully addressed with analytical techniques and, if so, what further information or data is required to make this possible.
• Provide data in a ready-to-use form to the Data Scientists who are looking to apply statistics, machine learning and analytic approaches to solve critical business problems.
• Ensure systems meet business requirements and industry practices.
• Research opportunities for data acquisition and new uses for existing data.
• Develop data set processes for data modeling, mining and production.
• Integrate new data management technologies and software engineering tools into existing structures.
• Create custom software components and analytics applications.
• Employ a variety of languages and tools to integrate systems.
• Install and update disaster recovery procedures.
• Recommend ways to improve data reliability, efficiency and quality.
• Collaborate with data scientists to determine what data is needed for analysis.

 
In addition to your focus on client engagements, you will contribute to the definition and enhancement of data engineering disciplines within the practice.  
 
You bring to the role
 

• A sound understanding of digital and cognitive technologies and analytics, information management and business process based solutions.
• Demonstrable industry knowledge; understanding the way your primary industry functions and how data can be collected, analyzed and utilized; maintaining flexibility in the face of big data developments. Experience in financial services, telecommunications and retail is not mandatory but highly regarded.
• Experience across different approaches to data architecture and applications.
• A proven ability to:
• Extract knowledge from structured and unstructured data sets.
• Work with an existing lifecycle management framework to collect metadata, follow coding standards, use version control, complete documentation and write and execute unit tests.
• Determine the appropriate approach including data collection methods, sampling methods, same sizes and data processing pipelines to formulate, execute and analyse a sound and reproducible experiment. This includes the ability to recognise and construct a closed loop feedback system. 
• Learn patterns and extract answers from data using algorithms that can build a model based on input data without being explicitly programmed to do so.
• Effectively collaborate with data scientists to establish their needs.
• Appropriately communicate discovered information to consumers, clearly using visual variables such as shape, colour, hue, orientation, etc.
• Possess the following traits:
• Intellectual curiosity; exploring new territories and finding creative and unusual ways to solve data management problems.
• Patience, as nothing will work the first time.
• Focus; enjoy working in the detail and understanding the intricacies of how and why a data pipeline works as it does.
• Experience with a range of technical skills that could include:
• Big Data technologies such as Hadoop, Spark Streaming, Storm, NiFi, HBase, Hive, MapReduce, Pig, Zeppline Notebooks, Kafka, Ranger, Ambari.
• Programming languages such as Java, Python, Scala, MatLab, Ruby, SAS, R.
• SQL-based technologies (e.g. PostgreSQL and MySQL)
• NoSQL technologies (e.g. Cassandra and MongoDB)
• Database architectures
• Data modelling tools (e.g. ERWin, Enterprise Architect and Visio)
• Data warehousing solutions
• Statistical analysis and modelling
• Predictive modelling, NLP and text analysis
• Machine learning
• Data mining
• A disciplined approach to problem solving and an ability to critically assess a range of information to differentiate true business needs as opposed to user requests.
• Excellent interpersonal, oral and written communication skills.
• Proven ability to develop and manage enduring client relationships, engendering a sense of trust and respect.

 
What we offer you

Some of the smartest colleagues in the country 

Hardcore enterprise projects 

Lots more 


 
Please apply for this position by submitting your confidential application online.