Step A — Data Preparation: Chemical-Reactors-21d-AIAO

+------------------------------------------------------------------------------
| Program Name:         step_a_prepare.py
| Program Version:      2026-09-10 10:49:28
| Date:                 2026-09-11
| Time:                 08:49:55
| Input 1 - Raw CSV:    chem_reactors_21d.csv
| Input 2 - Glossary:   chem_reactors_schema.csv
| Input 3 - Time stamp: Timestamp  (e.g. "2024-01-01 00:00:00")  -> 1 min grid
+------------------------------------------------------------------------------

Assembly: single file.

Data table: 30,240 samples at 1 min, 2024-01-01 00:00:00 → 2024-01-21 23:59:00; 78 tags kept (78 AI, 0 AO), 12 dropped. Timestamps absent from the raw files: 0. Values filled by interpolation (gaps ≤ 5 samples): 163,759. Bad-value strings replaced: 0.

Where the data is good and bad

Why each flagged tag was flagged

Overall report - objective, A-E map, inputs, what ran, results, next steps.

Bad-tags report - every flagged tag on one page · Fixed-tags report - actual, bad zone, fixed for every repaired tag.

Bad-tags detail - every flagged tag on one page, worst first, with all sub-plots.

Pick a tag:   grey = missing, red ticks = bad-value strings, orange = frozen, magenta circles = spikes, green dots = filled by interpolation

Timestamp formats seen

filefirst timestamp as written
chem_reactors_21d.csv2024-01-01 00:00:00

What was repaired

Rules applied: gaps<=5 interpolate; slow tags held; frozen unknown; spikes flag.
Values filled by interpolation: 163,759 · repair actions: 0 · samples still unknown after repair: 8 (Steps B and C skip unknown samples; nothing is invented over a long gap.)

no repairs were needed

The raw values on the same time grid are kept in raw_on_grid.parquet; the pink crosses on each tag panel are the raw values that were changed.

Tags dropped (12)

tag AI/AO Description EU status missing_pct std
A_R1_pressure_setpoint dropped: constant 0 0
A_R1_temp_setpoint dropped: constant 0 0
A_R2_pressure_setpoint dropped: constant 0 0
A_R2_temp_setpoint dropped: constant 0 0
A_R3_pressure_setpoint dropped: constant 0 0
A_R3_temp_setpoint dropped: constant 0 0
B_R1_pressure_setpoint dropped: constant 0 0
B_R1_temp_setpoint dropped: constant 0 0
B_R2_pressure_setpoint dropped: constant 0 0
B_R2_temp_setpoint dropped: constant 0 0
B_R3_pressure_setpoint dropped: constant 0 0
B_R3_temp_setpoint dropped: constant 0 0

Issues located in time (43)

Gaps, bad values, frozen stretches, spikes and slow-updating tags. Step B should look at these before choosing tags and windows.

tag AI/AO Description EU type start end samples
A_R1_ambient_temp_effect AI A_R1 Ambient temperature effect deg C spike 2024-01-01 00:00:00 2024-01-01 00:00:00 1
A_R2_ambient_temp_effect AI A_R2 Ambient temperature effect deg C spike 2024-01-01 00:00:00 2024-01-01 00:00:00 1
B_R1_ambient_temp_effect AI B_R1 Ambient temperature effect deg C spike 2024-01-01 00:00:00 2024-01-01 00:00:00 1
B_R2_ambient_temp_effect AI B_R2 Ambient temperature effect deg C spike 2024-01-01 00:00:00 2024-01-01 00:00:00 1
B_R3_ambient_temp_effect AI B_R3 Ambient temperature effect deg C spike 2024-01-01 00:00:00 2024-01-01 00:00:00 1
A_R1_ambient_temp_effect AI A_R1 Ambient temperature effect deg C spike 2024-01-01 00:01:00 2024-01-01 00:01:00 1
A_R2_ambient_temp_effect AI A_R2 Ambient temperature effect deg C spike 2024-01-01 00:01:00 2024-01-01 00:01:00 1
A_R3_ambient_temp_effect AI A_R3 Ambient temperature effect deg C spike 2024-01-01 00:01:00 2024-01-01 00:01:00 1
B_R1_ambient_temp_effect AI B_R1 Ambient temperature effect deg C spike 2024-01-01 00:01:00 2024-01-01 00:01:00 1
B_R2_ambient_temp_effect AI B_R2 Ambient temperature effect deg C spike 2024-01-01 00:01:00 2024-01-01 00:01:00 1
B_R3_ambient_temp_effect AI B_R3 Ambient temperature effect deg C spike 2024-01-01 00:01:00 2024-01-01 00:01:00 1
A_R1_ambient_temp_effect AI A_R1 Ambient temperature effect deg C spike 2024-01-01 00:02:00 2024-01-01 00:02:00 1
A_R2_ambient_temp_effect AI A_R2 Ambient temperature effect deg C spike 2024-01-01 00:02:00 2024-01-01 00:02:00 1
A_R3_ambient_temp_effect AI A_R3 Ambient temperature effect deg C spike 2024-01-01 00:02:00 2024-01-01 00:02:00 1
B_R1_ambient_temp_effect AI B_R1 Ambient temperature effect deg C spike 2024-01-01 00:02:00 2024-01-01 00:02:00 1
B_R2_ambient_temp_effect AI B_R2 Ambient temperature effect deg C spike 2024-01-01 00:02:00 2024-01-01 00:02:00 1
B_R3_ambient_temp_effect AI B_R3 Ambient temperature effect deg C spike 2024-01-01 00:02:00 2024-01-01 00:02:00 1
A_R1_ambient_temp_effect AI A_R1 Ambient temperature effect deg C spike 2024-01-01 00:03:00 2024-01-01 00:03:00 1
A_R2_ambient_temp_effect AI A_R2 Ambient temperature effect deg C spike 2024-01-01 00:03:00 2024-01-01 00:03:00 1
A_R3_ambient_temp_effect AI A_R3 Ambient temperature effect deg C spike 2024-01-01 00:03:00 2024-01-01 00:03:00 1
B_R1_ambient_temp_effect AI B_R1 Ambient temperature effect deg C spike 2024-01-01 00:03:00 2024-01-01 00:03:00 1
B_R2_ambient_temp_effect AI B_R2 Ambient temperature effect deg C spike 2024-01-01 00:03:00 2024-01-01 00:03:00 1
B_R3_ambient_temp_effect AI B_R3 Ambient temperature effect deg C spike 2024-01-01 00:03:00 2024-01-01 00:03:00 1
A_R1_ambient_temp_effect AI A_R1 Ambient temperature effect deg C spike 2024-01-01 00:04:00 2024-01-01 00:04:00 1
A_R2_ambient_temp_effect AI A_R2 Ambient temperature effect deg C spike 2024-01-01 00:04:00 2024-01-01 00:04:00 1
A_R3_ambient_temp_effect AI A_R3 Ambient temperature effect deg C spike 2024-01-01 00:04:00 2024-01-01 00:04:00 1
B_R1_ambient_temp_effect AI B_R1 Ambient temperature effect deg C spike 2024-01-01 00:04:00 2024-01-01 00:04:00 1
B_R2_ambient_temp_effect AI B_R2 Ambient temperature effect deg C spike 2024-01-01 00:04:00 2024-01-01 00:04:00 1
B_R3_ambient_temp_effect AI B_R3 Ambient temperature effect deg C spike 2024-01-01 00:04:00 2024-01-01 00:04:00 1
A_R3_ambient_temp_effect AI A_R3 Ambient temperature effect deg C spike 2024-01-01 00:05:00 2024-01-01 00:05:00 1
A_R1_ambient_temp_effect AI A_R1 Ambient temperature effect deg C spike 2024-01-21 23:58:00 2024-01-21 23:58:00 1
A_R2_ambient_temp_effect AI A_R2 Ambient temperature effect deg C spike 2024-01-21 23:58:00 2024-01-21 23:58:00 1
A_R3_ambient_temp_effect AI A_R3 Ambient temperature effect deg C spike 2024-01-21 23:58:00 2024-01-21 23:58:00 1
B_R1_ambient_temp_effect AI B_R1 Ambient temperature effect deg C spike 2024-01-21 23:58:00 2024-01-21 23:58:00 1
B_R1_reaction_rate AI B_R1 Reaction rate 1/min missing 2024-01-21 23:58:00 2024-01-21 23:59:00 2
B_R2_ambient_temp_effect AI B_R2 Ambient temperature effect deg C spike 2024-01-21 23:58:00 2024-01-21 23:58:00 1
B_R3_ambient_temp_effect AI B_R3 Ambient temperature effect deg C spike 2024-01-21 23:58:00 2024-01-21 23:58:00 1
A_R1_ambient_temp_effect AI A_R1 Ambient temperature effect deg C spike 2024-01-21 23:59:00 2024-01-21 23:59:00 1
A_R2_ambient_temp_effect AI A_R2 Ambient temperature effect deg C spike 2024-01-21 23:59:00 2024-01-21 23:59:00 1
A_R3_ambient_temp_effect AI A_R3 Ambient temperature effect deg C spike 2024-01-21 23:59:00 2024-01-21 23:59:00 1
B_R1_ambient_temp_effect AI B_R1 Ambient temperature effect deg C spike 2024-01-21 23:59:00 2024-01-21 23:59:00 1
B_R2_ambient_temp_effect AI B_R2 Ambient temperature effect deg C spike 2024-01-21 23:59:00 2024-01-21 23:59:00 1
B_R3_ambient_temp_effect AI B_R3 Ambient temperature effect deg C spike 2024-01-21 23:59:00 2024-01-21 23:59:00 1

Tags needing attention (top 20 by bad values, missing, spikes)

tag AI/AO Description EU status missing_pct bad_values spikes longest_flat_min update_interval_min
A_R3_reactor_pressure AI A_R3 Reactor pressure bar ok 0.01 0 0 2 1
B_R1_reaction_rate AI B_R1 Reaction rate 1/min ok 0.01 0 0 2 1
A_R1_ambient_temp_effect AI A_R1 Ambient temperature effect deg C ok 0 0 7 1 1
A_R2_ambient_temp_effect AI A_R2 Ambient temperature effect deg C ok 0 0 7 1 1
A_R3_ambient_temp_effect AI A_R3 Ambient temperature effect deg C ok 0 0 7 2 1
B_R1_ambient_temp_effect AI B_R1 Ambient temperature effect deg C ok 0 0 7 1 1
B_R2_ambient_temp_effect AI B_R2 Ambient temperature effect deg C ok 0 0 7 1 1
B_R3_ambient_temp_effect AI B_R3 Ambient temperature effect deg C ok 0 0 7 1 1
A_R1_agitator_speed_rpm AI A_R1 Agitator speed rpm ok 0 0 0 1 1
A_R1_conversion_rate AI A_R1 Conversion % ok 0 0 0 1 1
A_R1_coolant_flow_rate AI A_R1 Coolant flow rate m3/h ok 0 0 0 1 1
A_R1_feed_flow_rate AI A_R1 Feed flow rate m3/h ok 0 0 0 1 1
A_R1_motor_current AI A_R1 Agitator motor current A ok 0 0 0 1 1
A_R1_power_consumption_kw AI A_R1 Power consumption kW ok 0 0 0 1 1
A_R1_pressure_setpoint dropped: constant 0 0 0 3.024e+04 NaN
A_R1_reaction_rate AI A_R1 Reaction rate 1/min ok 0 0 0 1 1
A_R1_reactor_pressure AI A_R1 Reactor pressure bar ok 0 0 0 1 1
A_R1_reactor_temp AI A_R1 Reactor temperature deg C ok 0 0 0 1 1
A_R1_selectivity AI A_R1 Selectivity % ok 0 0 0 1 1
A_R1_temp_setpoint dropped: constant 0 0 0 3.024e+04 NaN

Files written

data.parquet (the corrected table Step B reads), data.csv, raw_on_grid.parquet, repairs.csv, glossary_used.csv, data_quality.csv, issues.csv, manifest.json — in /Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO

Next: Step B (Explore) reads this folder, shows trends and clusters, and writes scope.json.

© 2026 CTRL Designer LLC. All rights reserved.
Yahya Nazer · info@ctrlDesigner.com