Databricks Engineer (POS-181)

About Us:

As a Senior Databricks Engineer at Kenility, you’ll join a tight-knit family of creative developers, engineers, and designers who strive to develop and deliver the highest quality products into the market.

Technical Requirements:

• Bachelor’s degree in Computer Science, Software Engineering, or a related field.
• Over 7 years of professional experience in Data Engineering or Big Data roles.
• At least 5 years of practical experience working with Databricks, Apache Spark (using PySpark or Scala), and Delta Lake.
• Solid understanding of contemporary data architectures, including Lakehouse and Data Mesh.
• Skilled in SQL and Python, with a strong foundation in distributed data processing.
• Hands-on experience with major cloud platforms such as AWS, Azure, or GCP.
• Familiar with CI/CD pipelines, version control using Git, and DataOps methodologies.
• In-depth knowledge of data governance, cataloging, and security frameworks.
• Demonstrated ability to lead technical initiatives or act as a technical reference within a team.
• Minimum Upper Intermediate English (B2) or Proficient (C1).

Tasks and Responsibilities:

• Bachelor’s degree in Computer Science, Software Engineering, or a related field.
• Oversee the complete lifecycle of data solutions, including the development of ETL/ELT workflows and batch/streaming pipelines using Databricks and Apache Spark.
• Architect Lakehouse-based data solutions aligned with business goals and technical specifications.
• Collaborate closely with analysts, data scientists, and fellow engineers to deliver robust data and machine learning capabilities.
• Define and uphold coding standards, best practices, and performance tuning for Databricks workflows.
• Develop real-time streaming solutions with Spark Structured Streaming and Kafka.
• Enhance Delta Lake data models and enforce data governance through tools such as Unity Catalog or Atlan.
• Coordinate data integration across multi-cloud environments using both native and third-party connectors.
• Participate in code reviews and provide mentorship to junior team members.
• Automate infrastructure setup and deployment through tools like Terraform and the Databricks CLI.
• Maintain a secure, compliant, and scalable data platform.

Soft Skills:

• Responsibility
• Proactivity
• Flexibility
• Great communication skills