Data Engineer Capital Markets

📄 You will start with a one-year contract with a view to permanent employment

💰 Scaling in scale G, with a gross annual salary between €64,000 and €106,000 (36 hours, incl. 13th month and holiday allowance), depending on knowledge and experience

⏰ 36-hour work week, working days in consultation (e.g., 4x9)

🌴 199.5 vacation hours per year (on top of your standard roster-free day)

💻 A laptop and a smartphone

📚 Extensive training and education opportunities

🚆 Full reimbursement of public transport costs (2nd class)

🏡 Home office allowance for work-from-home days

💪 Contribution to a gym membership

🚲 Attractive secondary employment conditions such as a bicycle and device scheme and an allowance for your home workspace

As an all-round Data Engineer within the Data-driven Capital Markets (DAK) department, you design, build, and manage reliable data pipelines and datasets for the supervision of capital markets. Additionally, you help further professionalize our development and deployment processes and the use of Databricks. You unlock, structure, and prepare data for data analysts, supervisors, and other users. Within our Azure cloud environment, you work with large volumes of structured and unstructured data, ranging from millions to billions of records. The data is located in, among others, Databricks, SQL databases, and Azure Storage. You develop reusable ETL processes and ensure that data is discoverable, verifiable, and suitable for analysis and supervision. You improve development and deployment processes with version control, automated testing, and CI/CD in Azure DevOps. With Declarative Automation Bundles, you capture Databricks code, configurations, and resources and deploy them in a controlled manner to various environments. In doing so, you work on repeatable and better manageable changes and deployments. When developing data solutions, you apply the applicable frameworks for security, privacy, and governance. You contribute to monitoring and cost-conscious use of the Databricks platform, in collaboration with the responsible platform, architecture, and security specialists. Additionally, you develop sustainable data products and actively contribute to knowledge sharing, innovation, and standardization within the team.

• Developing and managing processes for data ingestion, transformation, and processing within our Azure cloud environment and on the Databricks platform using, among others, Databricks Jobs, Azure Data Factory, and PySpark/SQL.

• Setting up and improving automated development, testing, and deployment processes with CI/CD in Azure DevOps.

• Capturing and controlled deployment of Databricks code, configurations, and resources with Declarative Automation Bundles.

• Developing and managing data pipelines and deployment processes for machine learning applications and contributing to the further professionalization of MLOps within the Databricks platform.

• Applying frameworks and standards for data quality, security, privacy, and governance, including the use of Unity Catalog.

• Monitoring and improving the operation and reliability of data pipelines and contributing to cost-conscious use of the Databricks platform.

• Actively contributing to architectural issues regarding scalability, security, data governance, and cloud setup.

• Collaborating with data analysts, data management specialists, and platform engineers to translate technical and functional issues into future-proof solutions.

• Identifying opportunities for improvement and actively contributing to innovation, standardization, and knowledge sharing within the team.