Deprecating the Collector Batch Processor: What You Need to Know
The deprecation of the Collector Batch Processor is a significant shift in how telemetry data is handled in OpenTelemetry. This change addresses the need for more efficient data processing by moving batching capabilities directly into exporters. By doing so, it ensures that requests are appropriately sized, improving compression and reducing latency without exceeding limits.
Exporter-level batching is now the standard, allowing exporters to manage their own queuing and batching behavior. Requests first attempt to enter a queue with a limited capacity. If the queue is full, it can either fail immediately or block the caller, depending on your configuration. Key parameters include timeout, which sets a maximum delay at 200ms, and send_batch_size, which defines the minimum item count at 8192. The queue_size parameter limits the number of items in the queue to 1000, while num_consumers allows for up to 10 concurrent processing threads. This integration streamlines the process, making it easier to manage telemetry data effectively.
In production, you need to be aware of the upcoming changes. Exporter-level batching will be enabled by default in most exporters starting with release v0.171.0 in February 2027, with the batch processor being removed in v0.177.0 in May 2027. Transitioning to the new queue_batch processor is essential to avoid disruptions. Ensure you configure your exporters correctly to take advantage of the new batching features and avoid potential bottlenecks in your telemetry pipeline.
Key takeaways
- →Understand the shift from the batch processor to exporter-level batching for improved performance.
- →Configure `queue_size` to manage the capacity of your telemetry data effectively.
- →Utilize `send_batch_size` to define the minimum number of items for batching.
- →Monitor the transition timeline: batch processor removal is planned for May 2027.
- →Adjust your exporters to leverage the new `queue_batch` processor for optimal data handling.
Why it matters
This change impacts how telemetry data is processed, directly affecting performance and latency in production environments. Proper configuration will ensure efficient data handling and prevent bottlenecks.
Code examples
1receivers:
2 host_metrics:
3 scrapers:
4 cpu:
5 {}
6processors:
7 batch:
8exporters:
9 otlp:
10 ...
11sending_queue:
12 enabled:
13 true
14# batch not enabled
15service:
16 pipelines:
17 metrics:
18 receivers:
19 [host_metrics]
20 processors:
21 [batch]
22 exporters:
23 [otlp]1exporters:
2 <exporter>:
3 sending_queue:
4 queue_size:
5 1000
6 sizer:
7 requests
8 block_on_overflow:
9 false
10 wait_for_result:
11 false
12 num_consumers:
13 10
14 batch:
15 enabled:
16 true
17 min_size:
18 8192
19 flush_timeout:
20 200ms1processors:
2 batch:
3 # deprecated processor
4 timeout:
5 200ms
6 # maximum delay
7 send_batch_size:
8 1000
9 # minimum item count
10 send_batch_max_size:
11 5000
12 # maximum item countWhen NOT to use this
The official docs don't call out specific anti-patterns here. Use your judgment based on your scale and requirements.
Want the complete reference?
Read official docsOpenAI & Anthropic-compatible inference API — no GPU provisioning needed. 55+ models, pay-per-token with no minimums. VPC + zero data retention by default.
Try Serverless Inference →Mastering Instrumentation Quality for Full-Stack Observability
To achieve true observability, you need to measure and improve your instrumentation quality. Each service gets a quality score based on automated checks, giving you a clear view of where improvements are needed. Dive in to learn how to leverage this for better insights.
Unlocking Performance: Pyroscope 2.0 for Continuous Profiling at Scale
Pyroscope 2.0 revolutionizes continuous profiling, providing insights into why your code is slow or costly. With data co-location and stateless queriers, it optimizes performance and storage efficiency. Dive in to see how it can transform your observability strategy.
Scaling Alloy: Mastering Your Central Telemetry Gateway
Scaling Alloy as a telemetry gateway is crucial for managing application observability effectively. With Horizontal Pod Autoscaling set to target 70% CPU and 90% memory, you can ensure your system remains responsive under load. Dive into the specifics of configuration and real-world production lessons.
Get the daily digest
One email. 5 articles. Every morning.
No spam. Unsubscribe anytime.