Senior Data Engineer

As a data engineer for Agtuary, you will be working with complex geospatial, JSON and array-based data, such as biodiversity layers, climate data and soil maps. We have built up a very comprehensive internal library capable of manipulating a lot of this data, however our main challenges is the total volume of data, and maintaining ingestion pipelines. One such problem you will be facing is mapping JSON-based property sales data, to GeoJSON-based biodiversity information, and ensuring that these are correctly mapped to individual properties saved in our database, for the frontend/API to consume and display to the user.

Key Responsibilities

- Developing and maintaining back-end features and data for a complex map-based property data platform
- Working with a team of other engineers to prioritise and build features in a timely manner
- Work with our CEO, COO and sales team to ensure a well-defined product feedback loop, to implement new features while also delivering fixes

Requirements

- 5+ years experience with Python and Numpy
- 3+ years experience with Node.js/Express (or similar back-end technologies/frameworks)
- 3+ years experience with PostgreSQL and/or similar SQL databases
- Have previous experience with pipeline-like libraries/tools such as Dagster and Airflow
- Ability to write and maintain high quality code
- Have previously performed data modelling for complex relational data
- Have previously worked with complex and large (100mb+) JSON files
- Being a highly capable engineer and able to work and communicate with a team of other developers

Nice to haves

- Have worked with Geospatial data such as GeoJSON, Shapefiles
- Have knowledge/able to use QGIS
- Have basic data science knowledge and have used other Python libraries such as SciPy