Content Intake Engineer

The Company:

Our client create and host software in the cloud which is used thousands of institutions across 104 countries and enables institutions to compare written documents, against a database of 17 billion pages of content identifying any material which has been copied and pasted from an existing source.

The Role:

Reporting to the Content & Integration Partner Manager this role will work with various teams within the company to provide the engineering expertise required to regularly crawl acquired content.

Required duties include:

• Creation of specific crawling applications to load partner content.
• Ensure current partner content is being crawled correctly.
• Establish monitoring process to ensure crawling runs successfully.
• Investigating failures in content crawling and help produce reports for management.
• On boarding and configuring new crawlers for new partners.
• Testing new crawling target data to ensure crawling will work.
• Working with the ICAM on key priorities and technical point of contact for partners.
• Liaising with the ICAM and key partners to help the content acquisition process.
• Work with the SEU team.
• Creation and provision of reports as needed on the crawling process including the maintenance of key online documents.

The Individual:

• Proficient in Linux/command line/shell.
• Good understanding of protocols like FTP.
• XML parsing using appropriate open source toolkits.
• Understanding of and ability to write scripts that can crawl and download a webpage.
• Good understanding of XML and HTML.
• Fluent in appropriate scripting languages; perl, python, shell scripts, etc.
• Knowledge of cron and comfortable putting entries in a crontab.
• Good organisational and prioritisation skills.
• Comfortable working as part of a team that operates across multiple timezones.
• Proficient at troubleshooting issues/technical barriers for partners.
• Able to follow through on problem resolution from initial solution identification.
• Ability to adapt to flexible work schedule and evidence of working within a team.
• Well developed interpersonal and communication skills both verbal and written.
• Potential to work in a project based changing environment.
• A good degree in relevant subject.