We're looking for a Senior Data Platform Engineer to join the Data Platform team, which owns the infrastructure every data team at Lusha runs on and sits between DevOps and the Data groups. You'll be the Databricks admin (Unity Catalog, compute, governance, cost), the Confluent (Kafka) and CDC admin, and own Airflow, Elasticsearch, the databases, the data S3 estate, and the Kubernetes cluster underneath. Your job is to make the platform fast, cheap, governed, and boring: if it's broken, you fix it, if it's manual, you automate it.
Hands-on experience using AI coding tools daily, and building or integrating LLM and agent tooling into real workflows, is a must for this role.
This role is based in Tel Aviv. We work in a hybrid model, with 3 days a week in the office.
This might be for you if:
You enjoy designing, building, and running large scale batch and streaming data pipelines in production, and owning what you ship, including when it breaks
You like administering and evolving core platforms: Databricks, Confluent Kafka and CDC pipelines, Airflow, and Elasticsearch
You care about reliability, performance, and cost of a shared data platform used across R&D
You enjoy setting the engineering standard for data work: testing, observability, cost tagging, documentation
You like driving architecture decisions across Databricks, Kafka, Elasticsearch, and AWS
You actively use AI tools and build self-service tooling, including MCP servers and agent interfaces, so teams can operate the platform safely
Requirements
5+ years hands-on building and operating large scale data pipelines, batch and streaming, in production
Deep Spark and Databricks experience: Delta Lake, Unity Catalog, job and cluster tuning, Structured Streaming or DLT
Streaming and CDC experience: Kafka (Confluent a plus), Debezium or equivalent, schema evolution and its failure modes
Experience writing Airflow DAGs and operating the platform itself
Expert Python and SQL, clean, tested, performant
Strong data modeling and architecture judgment, with the ability to explain tradeoffs in scale, performance, and cost
Infra literacy: comfortable with AWS (S3, IAM, networking basics), Terraform, Docker and Kubernetes, and CI/CD
Proven experience leading cross-team technical initiatives end to end, from ambiguous ask to shipped and adopted
Proven experience using AI coding tools daily, with LLM and agent tooling (MCP, embeddings, vector search) built into real workflows
Nice to have:
Elasticsearch at scale: indexing pipelines, cluster operations, reindex and rollout strategies
FinOps or cloud cost management for data platforms
Vault or similar secrets management, and Kubernetes operations beyond deploying your own service
Experience inheriting a system with real debt and making it boring
5+ years hands-on building and operating large scale data pipelines, batch and streaming, in production
Deep Spark and Databricks experience: Delta Lake, Unity Catalog, job and cluster tuning, Structured Streaming or DLT
Streaming and CDC experience: Kafka (Confluent a plus), Debezium or equivalent, schema evolution and its failure modes
Experience writing Airflow DAGs and operating the platform itself
Expert Python and SQL, clean, tested, performant
Strong data modeling and architecture judgment, with the ability to explain tradeoffs in scale, performance, and cost
Infra literacy: comfortable with AWS (S3, IAM, networking basics), Terraform, Docker and Kubernetes, and CI/CD
Proven experience leading cross-team technical initiatives end to end, from ambiguous ask to shipped and adopted
Proven experience using AI coding tools daily, with LLM and agent tooling (MCP, embeddings, vector search) built into real workflows
Nice to have:
Elasticsearch at scale: indexing pipelines, cluster operations, reindex and rollout strategies
FinOps or cloud cost management for data platforms
Vault or similar secrets management, and Kubernetes operations beyond deploying your own service
Experience inheriting a system with real debt and making it boring
