Hadoop Data Architect - London, HBase, BigTable

Hadoop Data Architect - London, HBase, BigTable

My client are recruiting a data architect/modeller. Your primary responsibility will be to design a coherent data model across the company, and you will be expected to take ownership of their data. They have crowdsourced the world's largest scientific catalogue with over 50 million unique articles contributed by almost two million users over a period of four years. These semantically rich articles are accompanied by terabytes of usage logs gathered through their suite of Science 2.0 research tools, allowing them to provide our users with personalised research services.

This is a hands-on role; you'll work with their agile development team on implementing a well-architectured system that is stable, durable, and amenable to change. Their software environment is based around open source technologies including the Hadoop ecosystem, Java, MySQL, and we deploy onto both their own hardware and Amazon Web Services. You should be happy taking responsibility, be comfortable with change (ideally backed up with experience of refactoring/change in place of distributed systems), questioning and have a conservative approach to risk.

Responsibilities

Avoid unnecessary data duplication
Enforce the data model
Unify back end data for all clients
Carry ultimate responsibility for all data
What you'll be doing

Designing non-relational and relational schemas
Designing data flow architecture
Configuring and running operational systems
Tuning, monitoring and scaling systems
Coordinating with technical teams
What you should bring

5+ years of commercial experience
2+ years of HBase or BigTable, and distributed systems experience
Production Hadoop application, including configuration management, scaling and availability
Abstract data representation skills - modelling
Agile development experience, preferably XP
Unix skills, preferably Linux, some scripting or programming experience
Ideally you'll have used cloud services, preferably AWS