How Nexxen Optimized Spark Data Platform Cost by 44%, Regaining Control of Pipeline Health & Operations

Overview

Behind the advertising technology stands a large-scale data platform responsible for processing real-time bidding, audience segmentation, ad delivery, and campaign measurement. Ensuring this platform operates with high performance, reliability, and efficiency is fundamental to Nexxen’s ability to deliver differentiated value to its customers.

The on-prem data platform consists of >50K cores running Spark pipelines that supports the mission-critical adtech workloads with strict SLA requirements. While on-prem deployment provides strong control over data governance, security, and locality, it also limits rapid infrastructure elasticity. As data volumes, pipeline complexity, and business demand continued to grow, ensuring consistent performance, cost efficiency, and end-to-end operational visibility became a strategic priority for Nexxen’s data platform organization.

To elevate platform observability and optimization capabilities, Nexxen introduced definity across its Spark environment. This provided unified, full-platform execution visibility and enabled rapid detection of performance bottlenecks, workload inefficiencies, and resource waste — laying the foundation for systematic platform improvement.

The Challenge

As Nexxen’s data platform grew in scale and complexity, maintaining consistent performance and reliability across its on-prem Spark environment became increasingly difficult. Limited observability and fragmented tooling made it hard to understand workload behavior, identify inefficiencies, and proactively optimize the platform.

This led to several key challenges:

• Limited visibility
  Monitoring was fragmented and not unified across the Spark environment. Teams lacked a clear end-to-end view of workload behavior, performance health, and resource utilization.
• Lack of deep waste analysis
  Existing tools did not provide task-level or resource-level inefficiency insights. This prevented identification of high-impact optimization opportunities and slowed improvement cycles.
• SLA-critical workloads under pressure
  Adtech pipelines require short and strict SLAs. Inefficiencies led to cluster congestion, increasing risk to business-critical delivery timelines.
• Manual and reactive operations
  Without comprehensive execution insight, performance tuning depended heavily on manual investigation and reactive troubleshooting — consuming valuable engineering time.

“We were operating a massive, business-critical platform with limited insight. definity gave us actionable visibility and allowed us to rapidly take control of performance and cost - and then expand that into job stability and health“
Elad Wertzberger, VP of Engineering

Why definity

Nexxen chose definity because it delivered the following core advantages seamlessly across its Spark platform:

• ‍Zero-Effort, Rapid Adoption
  definity was deployed across the Spark platform with minimal operational overhead, instantly providing full-stack visibility without requiring code changes or modifications to existing pipelines
• Underutilized Resource Detection & Heatmaps
  definity created a real-time heatmap of resource waste across the data platform, enabling teams to quickly identify and prioritize the highest-ROI optimizations.‍
• Exposed Hidden Inefficiencies
  definity helped uncover and resolve hidden inefficiencies, including “orphaned vCores” caused by imbalanced memory-to-vCore allocation, as well as data skew patterns that led to computational imbalance.‍
  
• Deep Observability Context
  definity enabled true monitoring and root-cause analysis of application behavior, health, and SLAs, providing end-to-end context across jobs, clusters, and workloads to quickly pinpoint performance bottlenecks and reliability risks.

These capabilities enabled Nexxen to rapidly unlock measurable performance gains, reduce infrastructure waste, and allow engineers to focus on core business functions.

The Impact

By applying deep, full-stack observability to its Spark platform, Nexxen quickly translated insight into measurable operational impact:

• Pinpointed 35% optimization opportunities within the first week, leveraging definity’s Cost & Health Assessment
• Captured 44% cost savings in the main Spark on-prem environment by optimizing compute and memory provisioning, machine types, and query inefficiencies
• Captured >9x cost-savings ROI in less than 5 months
• Freed infrastructure capacity to support immediate business growth without added hardware
• Reduced 70% of engineering effort spent on manual troubleshooting, analysis, and optimizations

This impact established a clear, fast-return business case for ongoing optimization at scale.

“definity provided a turnkey solution to optimize Spark cost BIG TIME. The integration couldn't be easier – we were up and running in days
‍Dennis Meyer, Director Data Engineering

Looking Forward

With proven results at scale, Nexxen plans to expand definity across its broader data platform, extending cost optimization, performance reliability, and operational intelligence across all data operations.

‍