The position for Content Intake Engineer in Newcastle upon Tyne is no longer available
Content Intake Engineer
Our client create and host software in the cloud which is used thousands of institutions across 104 countries and enables institutions to compare written documents, against a database of 17 billion pages of content identifying any material which has been copied and pasted from an existing source.
Reporting to the Content & Integration Partner Manager this role will work with various teams within the company to provide the engineering expertise required to regularly crawl acquired content.
Required duties include:
• Creation of specific crawling applications to load partner content.
• Ensure current partner content is being crawled correctly.
• Establish monitoring process to ensure crawling runs successfully.
• Investigating failures in content crawling and help produce reports for management.
• On boarding and configuring new crawlers for new partners.
• Testing new crawling target data to ensure crawling will work.
• Working with the ICAM on key priorities and technical point of contact for partners.
• Liaising with the ICAM and key partners to help the content acquisition process.
• Work with the SEU team.
• Creation and provision of reports as needed on the crawling process including the maintenance of key online documents.
• Proficient in Linux/command line/shell.
• Good understanding of protocols like FTP.
• XML parsing using appropriate open source toolkits.
• Understanding of and ability to write scripts that can crawl and download a webpage.
• Good understanding of XML and HTML.
• Fluent in appropriate scripting languages; perl, python, shell scripts, etc.
• Knowledge of cron and comfortable putting entries in a crontab.
• Good organisational and prioritisation skills.
• Comfortable working as part of a team that operates across multiple timezones.
• Proficient at troubleshooting issues/technical barriers for partners.
• Able to follow through on problem resolution from initial solution identification.
• Ability to adapt to flexible work schedule and evidence of working within a team.
• Well developed interpersonal and communication skills both verbal and written.
• Potential to work in a project based changing environment.
• A good degree in relevant subject.