DARTWIC under load: what can it handle?

Overview

I recently released the first public version of DARTWIC, the data acquisition and control software I’ve been building over the past two years.

It started around rocket testing. I wanted one place to connect equipment, write the logic that operates it, see what was happening, and look back at the data afterward. That grew into DARTWIC: an engine for acquisition and control, an automation language called DCode, and an interface for building and operating your system.

With the first release out, I wanted to see how much I could squeeze out of it. So I ran stress tests with simulated devices, hundreds of tasks, calculations on every input, event overrides, recording, and telemetry. This post walks through the setup and the results, including where performance fell short.

Machine Specs

These measurements came from the same Windows desktop:

The device runs measured 15.28 and 25.72 seconds after warmup. The networking campaign used roughly one- and two-minute runs, boundary repeats, and a ten-minute WebSocket run. Other applications remained on the machine, and host load varied. These were headless engine tests with client queries; interface rendering, physical bus timing, and actuator response were outside the measurements.

This was standard Windows, not a real-time operating system. A fast average does not establish a hard deadline or long-duration reliability.

Channels and Automation in DARTWIC

A channel gives a signal a name the rest of the system can use. A hardware plugin can publish a pressure reading, a controller can command a valve channel, and the interface can show those same signals on a plot or schematic. Recorded channel data lets you look back at what happened afterward. RAPID manages the channel data and its access and recording paths.

A task is a piece of work the engine runs, such as reading a device, calculating its next command, or writing an output. In this test, each simulated device had three tasks ordered read → control → write. Task groups enforce that order; the target was 100 cycles per second for each device.

A channel calculation transforms an incoming value. The stress setup attached a clamp calculation to every input. An event checks a condition and reacts when it changes; each device had a warning event that could override an output, then release it when the condition ended. Command authority defines how ordinary automation, event overrides, and manual operator control interact.

DCode is where those calculations, tasks, sequences, and event reactions come together. Templates reuse the same logic across a configuration table instead of maintaining hundreds of nearly identical scripts. Supported DCode can compile to native C++; the stress workload used that backend throughout. The engine runs independently of the interface, so the process connected to the equipment owns execution.

The next three sections follow that path: first the shared DCode setup, then the device benchmark with acquisition, control, events, and recording running together, and finally the channel database benchmarks and a bare arithmetic reference. The isolated tests show how much faster channel access and local math can be when they are not carrying the rest of the running system. They help separate the cost of the underlying operations from the cost of scheduling and servicing a busy workload on this machine.

Automation Setup in DCode

The stress test used one shared template file. A generated YAML file supplied the device names, input channels, target rate, and thresholds. For one module, the configuration looks like this:

modules:
  - prefix: stress_0000
    hz: 100
    maximum: 1000
    threshold: 800
inputs:
  - { channel: stress_0000_in_1, maximum: 1000 }
  - { channel: stress_0000_in_2, maximum: 1000 }
  - { channel: stress_0000_in_3, maximum: 1000 }
  - { channel: stress_0000_in_4, maximum: 1000 }
  - { channel: stress_0000_in_5, maximum: 1000 }

The calculation clamps each input. The control task latches its output at the waveform endpoints, then commands all five outputs. This is the shared calculation and control setup from the test:

modules = yaml "./instances.yaml#modules"
inputs = yaml "./instances.yaml#inputs"

template(inputs) channel_calculation {params.channel}:
    return clamp(value, 1, {params.maximum})

template(modules) task_periodic {params.prefix}_control {params.hz}:
    start:
        latched = 0
    task elapsed_seconds:
        if |{params.prefix}_in_5| == 1:
            latched = 0
        if |{params.prefix}_in_5| == {params.maximum}:
            latched = 1
        command |{params.prefix}_out_1| = latched
        command |{params.prefix}_out_2| = latched
        command |{params.prefix}_out_3| = latched
        command |{params.prefix}_out_4| = latched
        command |{params.prefix}_out_5| = latched

The warning event fires when all five inputs exceed the threshold, or the fifth input reaches 1. Its reaction overrides the first output to 1 or 2. When firing ends, it releases that override:

template(modules) event {params.prefix}_warning:
    title: "Stress limit {params.prefix}"
    type: warning
    system: "STRESS"
    subsystem: "DEVICE"
    separation_seconds: 0
    channels: |{params.prefix}_in_5|, |{params.prefix}_out_1|
    trigger:
        if |{params.prefix}_in_5| == 1:
            return 1
        if |{params.prefix}_in_1| > {params.threshold} and |{params.prefix}_in_2| > {params.threshold} and |{params.prefix}_in_3| > {params.threshold} and |{params.prefix}_in_4| > {params.threshold} and |{params.prefix}_in_5| > {params.threshold}:
            return 1
    on_firing:
        if value == 1:
            input_name = "{params.prefix}_in_5"
            if |{input_name}| == 1:
                override |{params.prefix}_out_1| = 2
            else:
                override |{params.prefix}_out_1| = 1
        if value == 0:
            free |{params.prefix}_out_1|

The native example-device plugin supplies the read and write tasks. The harness adds task groups to the same DCode file, keeping each module’s three stages in order. One module’s group entries look like this:

task_group stress_group_0:
    run: stress_0000_read
    run: stress_0000_control
    run: stress_0000_write

For the 150-module run, the harness distributed the modules across 32 groups. The script used execution_mode: native_cpp in its script configuration. Increasing the module count expanded the shared templates and group membership; it didn’t require writing another control script for every device. These excerpts depend on the simulated plugin and its channel setup; they aren’t a standalone hardware driver.

I ran this through DARTWIC’s designed real-time execution path: CAESAR tasks, native C++ DCode, and fixed RAPID channels with prepared bindings. Supported prepared numeric operations reuse their storage without routine framework heap allocation or global locks in the hot path, reducing sources of timing variation. Channel-local synchronization still applies; arbitrary plugin code, device I/O, and general host calls have their own costs. The deterministic automation guide explains that boundary.

Device Benchmark

The full workload used the DCode above and the native example-device plugin. Every simulated module had five inputs, five outputs, read/control/write tasks, a calculation on each input, and a synchronous warning event. Both configurations kept recording enabled.

The I/O channels are part of the fixed-channel total, not an additional set. Inputs followed a known waveform. Normal control issued commands while event reactions took and released overrides. The simulated write callback checked the result, including whether ordinary control slipped through an active override.

The 100-module run achieved 99.25–99.83 Hz. At 150 modules, that fell to 89.14–94.93 Hz. The larger run checked 1,776,547 history points without invalid inputs or outputs, timestamp reversals, waveform gaps, or output mismatches. Both runs reported zero recording loss, write errors, and failed task cycles.

The scheduling misses still matter: 4,629 missed releases in the 100-module run and 93,336 in the 150-module run, counted over their bracketing task-counter windows. A missed release is different from a failed callback. These are capacity measurements under the combined workload, not a guarantee that every cycle meets a 10 ms deadline.

Channel Database Benchmarks

The device test includes much more than reading and writing channel values. To look at RAPID itself, I ran reads and writes separately across 10,000 channels and eight threads, for 15 seconds per direction, with recording disabled. Dynamic names perform a named lookup; fixed handles retain a prepared channel reference.

The overview below compares named access, fixed handles, and the separate history-recorder result. Reads and writes are in millions of operations per second; history recording is in millions of rows per second. They are independent tests on a shared numeric scale. The Reads and Writes tabs let you inspect channel access on their own scales too.

Download workload and RAPID results ↗

The recorder result accepted about 5 million rows per second for one minute, with every row committed and indexed after drain and zero loss. It used 131,072-row producer buffers and measured prepared sample admission, excluding live channel mutation. Longer and other high-rate trials encountered backlog or loss; that one-minute configuration is not an indefinite recording guarantee.

The bare C++ math baseline completed about 27.37 million module cycles per second on one thread. Each cycle reproduced the local read, clamp, event-condition, control, and write arithmetic using ordinary memory. It excluded the engine, channel synchronization and authority, scheduled releases, recording, diagnostics, and transport.

Those isolated results show substantial headroom in channel access and arithmetic compared with the complete device workload. They suggest there is room to improve how the rest of the system is serviced. They do not establish how much faster a full engine can run, or identify one bottleneck by themselves: task scheduling, calculations, reactions, recording, client traffic, device I/O, and competing host work all contribute.

You will not get isolated RAPID benchmark speeds under a full workload. Module cycles, channel operations, and recorded rows are different units of work. Treat the raw math as an arithmetic reference and the channel tests as subsystem measurements, then test the actual read → control → output path your project needs.

Networking and Distributed Services

DARTWIC already supports external clients, TEMPEST peers, and engine nodes. A client queries an engine and subscribes to telemetry; a peer can also provide data and handlers that the engine can address. This gives a project a way to share signals and put work on separate nodes.

Python, Node.js, C++, and React clients let you build your own tools around the engine. Engine plugins add drivers and native services, while interface plugins add project-specific controls. The example workspace, hardware-driver guide, and client guides are starting points for those integrations.

The next architectural split I want to explore is:

The existing node system is available today; these three dedicated services are a proposed next step. Moving work between processes or machines may reduce contention while adding communication costs. The telemetry benchmarks below measure the current engine, not that proposed split.

Both telemetry campaigns kept 25 simulated modules at 100 Hz running with recording enabled: 250 I/O channels, 75 read/control/write tasks, 125 input channel calculations, 25 events, and eight task groups. Generated event calculations brought the registered calculation count to 175. The engine had 934 local channels before peer diagnostics were added.

Client Telemetry Benchmark

Each load client subscribed to the same 250 I/O channels at 5 Hz, using complete snapshots. Thirty-two clients therefore meant 8,000 subscriptions to 250 unique channels. All load sessions shared one Node process, in addition to the operator measurement connection. WebSocket runs lasted about two minutes; ZeroMQ/Node runs about one, with different heartbeat policies.

The consistently healthy short runs reached 16 WebSocket clients and eight ZeroMQ clients. At 16 and 32 ZeroMQ clients, complete delivery fell to 3.39 Hz and 1.29 Hz, even while local reads stayed above 97 Hz. Delivery and control timing are different parts of the result, and these measurements include the load generator’s limits.

Thirty-two WebSocket clients passed three short runs, then fell to 4.49 Hz delivery and 94.30 Hz minimum reads during the ten-minute run, with transport warnings and dropped telemetry. Mixing eight WebSocket clients with four peers also missed the control threshold, despite client delivery staying near 5 Hz.

The earlier campaign encountered heavier unrelated background activity. Those runs and repeats remain in the downloadable evidence, separately from the main campaign.

Before fresh runs, I required 20 seconds below 45% whole-host CPU. Activity during measurement remained recorded. These were loopback tests on one shared desktop with other applications still running. A short pass does not establish sustained capacity; none of the reported levels completed the full sustained-qualification sequence.

Peer Telemetry Benchmark

Each lightweight protocol simulator sent 250 changing channels at 5 Hz and received the engine’s complete local export. It also maintained normal connection and empty ARGUS-board traffic, but ran no control workload or recording store. This loaded the tested engine without running eight more engines on the same machine.

Download all communication results and test details ↗

The peer tests passed through eight connections, the sweep cap. At that level the engine held 959 local channels plus 2,000 imported channels: 2,959 total. Every peer received all 959 exported channels. Engine CPU averaged 18.96% of the eight-logical-processor host, and the slowest read task achieved 96.78 Hz.

That run still recorded 10,785 missed task releases, with zero failed cycles in its bracketing counter window. Eight peers establishes a tested level, not a measured maximum or sustained-capacity guarantee. Physical network links, cross-machine clocks, and remote control workloads were outside this test. The downloadable evidence includes all 39 main and earlier communication trials, including failures, repeats, counters, and method notes.

Conclusion

Honestly, I’m not sure why you’d need this many channels and tasks, but it’s fun to know you can build something this busy. The full device workload, isolated channel access, and bare arithmetic show different parts of the system. Local work and telemetry can also reach their limits at different points.

There is room to explore how execution, storage, and communication share the machine, but the useful result for a project is the one its complete workload can maintain. Start with a simulated workspace, then check the hardware and traffic your system will actually use.

As always, test your setup on your own hardware before using it in any critical operation. Check the full read → control → output path, its timing under load, and how it responds when something fails. Your hardware, drivers, and workload are the ones that need to pass.