Senior Data Engineer (IRC289061)

GlobalLogic is seeking a Senior Data Engineer to join a dynamic startup specializing in pet insurance. You will design and implement scalable data pipelines using Azure Databricks and Spark for an advanced pet insurance claims clearing solution built on Azure cloud services.

• Design, implement, and optimize scalable data pipelines using Databricks, Spark, and Azure data services
• Develop and maintain config-driven ETL/ELT solutions for batch and streaming data
• Ensure data governance, lineage, and compliance using Unity Catalog and Azure Key Vault
• Work with Delta tables and Lakehouse architecture for efficient data processing
• Collaborate with developers, analysts, and data scientists to deliver trusted datasets
• Integrate data pipelines with event-based and microservice architectures
• Design and maintain optimized data models and schemas
• Identify and resolve performance bottlenecks for cost efficiency
• Participate in architecture discussions and sprint planning
• Perform code reviews and provide technical mentorship
• Evaluate and enhance data engineering tools and processes

• Strong hands-on experience with Azure Databricks (DLT Pipelines, Delta Live Tables, Unity Catalog, Time Travel)
• Proficiency in Apache Spark with PySpark, Scala, or Java
• Proven expertise in Azure data ecosystem: Databricks, ADLS Gen2, Azure SQL, Azure Functions, Azure Data Factory, Azure CosmosDB
• Solid understanding of Lakehouse architecture and Delta Lake concepts
• Experience designing config-driven ETL/ELT pipelines with CDC and event-based processing
• Proficiency with RDBMS (MS SQL, MySQL, PostgreSQL) and NoSQL databases
• Strong data modeling and database performance optimization skills
• Experience with JSON, Parquet, and ORC file formats
• Familiarity with machine learning and AI integration
• Hands-on CI/CD pipeline experience (Azure DevOps, GitLab)
• Understanding of data governance, lineage, and cloud security
• Strong analytical and problem-solving skills
• Excellent teamwork and communication skills
• Upper-Intermediate English (spoken and written)

Collaborative and supportive work environment. Dynamic startup project with cutting-edge technology stack. Opportunity to work on innovative pet insurance solution leveraging Azure cloud services and low-code paradigm.