Data Engineer - Machine Learning on Azure
GRiT Solutions
Remoto
There's no secret to become number-one, it takes willingness to improve and determination to pursue it. At GRiT, it's all about the journey getting there. We trust talent and intelligence are important, yet passion and perseverance are key to achieving any goal.
We are Tech Passionate People with a Purpose, delivering solutions while building strong long-term relations.
Recognizing each person's unique contribution to our collective success, we embrace inclusion and diversity, committed to creating a respectful and socially responsible environment.
People. Technology. Innovation. All-around it.
Are you GRiTTY enough to be part of our journey? Take the first step by applying now and let's create our future together.
Job Description: Design and upkeep of ETL/ELT pipelines on Azure Databricks, with PySpark and Azure Data Factory, across Data Lakehouse architectures. Preparation of datasets for Machine Learning, plus hands-on support for training, assessment and productionization of classification, regression, clustering and time series forecasting models. Close work with business and technical teams, bringing models into production workflows.
Must-Have
- 2 to 3 years of experience in data engineering.
- Hands-on Azure Databricks, Python, SQL and PySpark.
- Azure Data Lake Storage experience, including Parquet and Delta Lake.
- Full data lifecycle ownership, from ingestion and transformation through modeling to delivery.
- Classical Machine Learning: libraries, feature engineering, model training, validation and evaluation.
- Sound command of overfitting, data leakage, train/test split and cross-validation.
- Batch scoring and model integration into data pipelines.
- Git and solid engineering practices, with scalable solutions mindful of performance and cost.
- Proactivity, analytical thinking and business-facing communication, including conversion of requirements into technical solutions.
- Portuguese language.
- Azure Machine Learning exposure: training jobs, pipelines, model registry and inference endpoints.
- Working knowledge of Azure Data Factory.
- Model monitoring, data drift detection and retraining routines.
- Delivery of models through APIs or online endpoints.
- Familiarity with Unity Catalog, data governance and access management.
- Agile ways of working, namely Scrum, with Jira and Confluence.
- Continuous Training and Certifications
- GRiT Prizes: GRiT Baby Gift, GRiT Birthday Prize, GRiT Recommendation Prize
- Exclusive Partnerships/Discounts: (Gym, Health, Well-being, Lifestyle, Financial, etc…)
- Unique Events
- Welcoming Office (Fruit, Coffee, Pool Table, etc.)
- Competitive Compensation
- Pay for Performance