Stream Operator Autoscaling State Simulation Dataset

#Multivariate State Prediction #Replica Count Prediction #Resource Utilization Prediction #Service Latency Prediction #Stream Processing Resource Scheduling #Autoscaling Simulation #Operator Performance Analysis #System Capacity Planning

5270
0.4GB
JSON
CATL
MOBIUSI INC

Updated:2026-11-04

AI Analysis & Value Prop

This dataset represents stream operator autoscaling processes as simulation sequences, covering input traffic changes, resource constraints, scaling actions, state migration overhead, and operator performance parameters. It provides future-state targets such as replica count, CPU and memory utilization, migration progress, backlog events, and service latency. Use it to build and evaluate state prediction models and simulation learning environments for stream processing systems.

Dataset Insights

Sample Examples

{}

Technical Specifications

Field	Type	Description
file_name	string	File name
input_stream	object	Describes the input event-stream volume and its changing characteristics at the current time step.
scaling_action	object	Describes the scaling action performed or planned for the current time step.
simulation_step	integer	The discrete time-step index within the current simulation sequence.
prediction_horizon	integer	The number of steps into the future to predict from the current time step.
operator_performance	object	Describes the processing capacity, resource consumption, and baseline latency characteristics of a single operator replica.
resource_constraints	object	Describes the compute, memory, and network resource limits available to the operator, as well as replica count constraints.
predicted_replica_count	integer	The predicted future number of operator replicas, based on input traffic, resource constraints, scaling actions, and operator performance parameters.
predicted_backlog_events	number	The predicted number of pending events based on the input rate, processing capacity, and scaling process.
state_migration_overhead	object	Describes the state volume to migrate, migration bandwidth, and checkpoint overhead during scaling.
predicted_cpu_utilization	number	The predicted CPU utilization based on future workload and replica configuration, expressed on a scale from 0 to 1.
predicted_memory_utilization	number	The predicted memory utilization based on the future replica count and per-replica memory consumption, expressed on a scale from 0 to 1.
predicted_migration_progress	number	The predicted proportion of state migration completed at a future time step, expressed on a scale from 0 to 1.
predicted_service_latency_ms	number	The predicted service latency in milliseconds, accounting for queue backlog, state migration, and operator processing overhead.

Compliance Statement

Authorization Type	Proprietary - Commercial AI Training License (No Redistribution)
Commercial Use	Requires exclusive subscription or authorization contract (monthly or per-invocation charging)
Privacy and Anonymization	No PII, no real company names, simulated scenarios follow industry standards
Compliance System	Compliant with China's Data Security Law / EU GDPR / supports enterprise data access logs

Frequently Asked Questions

What does the dataset contain?: The data is organized by simulation step and prediction horizon. It includes input traffic, resource constraints, scaling actions, state migration overhead, and operator performance parameters, along with prediction targets for replica count, CPU and memory utilization, migration progress, backlog events, and service latency.

Which modeling tasks is the dataset suited for?: It supports multivariate state prediction for stream processing systems, including predictions of replica count, resource utilization, migration progress, event backlog, and service latency. It can also be used to model simulation learning environments and evaluate autoscaling strategies.

What value ranges and units are used?: Predicted CPU utilization, memory utilization, and state migration progress are represented on a scale from 0 to 1. Service latency is measured in milliseconds. Other object fields describe traffic, resource limits, actions, migration overhead, and operator performance; refer to the accompanying schema documentation for their specific subfields.

How can I obtain the data, and what are the licensing terms?: Check the dataset product page for available download or delivery options, version details, and licensing terms. Before use, verify that the license covers your intended research, training, or commercial deployment scenario.

Can't find the data you need?

Post a request and let data providers reach out to you.

Get this Dataset

Verified for Enterprise Use

Cite this Work

@dataset{Mobiusi2026,
  title={Stream Operator Autoscaling State Simulation Dataset},
  author={MOBIUSI INC},
  year={2026},
  url={https://www.mobiusi.com/datasets/ec1a77db47bf0d9e2b084dd62ccda421},
  urldate={2026-10-27},
  keywords={stream processing dataset,operator autoscaling,state prediction,autoscaling simulation data,replica count prediction,resource utilization prediction,state migration prediction,service latency prediction},
  version={1.0}
}

Using this in research? Please cite us.

placeholder

Scene

Domain

Modality

Task