Chemical-Reactors-21d-AIAO — Step A overall report

+------------------------------------------------------------------------------
| Program Name:         step_a_prepare.py
| Program Version:      2026-09-10 10:49:28
| Date:                 2026-09-11
| Time:                 08:49:53
| Input 1 - Raw CSV:    chem_reactors_21d.csv
| Input 2 - Glossary:   chem_reactors_schema.csv
| Input 3 - Time stamp: Timestamp  (e.g. "2024-01-01 00:00:00")  -> 1.0 min grid
+------------------------------------------------------------------------------

CTRL Designer LLC

1. Objective and where we are

Objective: turn raw plant history into decisions, in five steps. Each step writes a folder the next one reads, so any result can be traced back to the file it came from.
StepNameWhat it doesProgramOutput folderState
APrepareRaw historian exports become one clean, regular Data table. Merge files, normalise timestamps, replace bad-value strings, put on a regular grid, fill short gaps, apply the repair rules, and record every change.step_a_prepare.py + A-Prepare-GUI-v01.pyA-Data/<project>/you are here
BExploreTrend and cluster the clean data. The engineer chooses which tags, which period, and which window counts as normal, and writes it down as the scope.step_b_explore.py + B-Explore-GUI-v01.pyB-Scope/<project>/next
CAnalyzeOperability (PCA, T2/SPE, events), oscillation and slowness per tag, control-loop metrics, batch phases - all driven by the scope from B.step_c_analyze.py + C-Analyze-GUI-v01.pyC-Analysis/<project>/
DAI reviewAn AI reads the results and the scope and drafts findings, anomalies and questions for the engineer.step_d_review.py + D-Review-GUI-v01.pyD-Review/<project>/
EReportThe engineer edits, decides, and either closes the study or returns to B with a new scope.to buildE-Report/<project>/

2. Input data

FileRoleSizeLocation
chem_reactors_21d.csvdata43.1 MB/Users/yahyanazer/Dropbox/__C_2026_Work/CTRL-Designer/B-CDL-Data-Analysis/A-Raw-Data/Chem-Reactors/chem_reactors_21d.csv
chem_reactors_schema.csvglossary (TAG - Description - EU)5 kB/Users/yahyanazer/Dropbox/__C_2026_Work/CTRL-Designer/B-CDL-Data-Analysis/A-Raw-Data/Chem-Reactors/chem_reactors_schema.csv

Timestamp formats found in the raw files:

filefirst timestamp as written
chem_reactors_21d.csv2024-01-01 00:00:00

Raw data root: /Users/yahyanazer/Dropbox/__C_2026_Work/CTRL-Designer/B-CDL-Data-Analysis/A-Raw-Data

3. What the program did

#OperationResult
1Read the raw files and detect the timestamp columnsingle file; formats seen: 1
2Replace PI bad-value strings (Bad, No Data, Shutdown, …) with unknown0 values
3Put every tag on one regular time grid30,240 samples at 1 min; 0 timestamps were absent
4Fill gaps of at most 5 samples by time interpolation163,759 values filled
5Score every tag: missing, bad values, update interval, longest flat run, spikes90 tags scored
6Drop tags that are constant or mostly missing12 dropped, 78 kept
7Apply the repair rulesgaps<=5 interpolate; slow tags held; frozen unknown; spikes flag → 0 actions
8Locate every issue in time43 issues: spike 42, missing 1
9Attach descriptions and units from the glossary78 AI, 0 AO tags labelled
10Draw the evidence and write the reportsavailability map, issue timeline, per-tag panels, 4 HTML reports
Repair rules applied: gaps<=5 interpolate; slow tags held; frozen unknown; spikes flag. 0 actions; 8 samples left unknown on purpose — long gaps and stuck sensors are never invented.

4. Results

FileWhat it isSizeLocation
prepare_report.htmlwhat Step A did, with the charts25 kB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/prepare_report.html
bad_tags_report.htmlevery flagged tag: trend + zoom per bad zone13 kB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/bad_tags_report.html
fixed_tags_report.htmlACTUAL / BAD / FIXED per repaired tag12 kB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/fixed_tags_report.html
data.parquetthe corrected table Step B reads23.6 MB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/data.parquet
data.csvsame table for Excel / Seeq43.7 MB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/data.csv
raw_on_grid.parquetraw values on the same grid (before/after)22.4 MB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/raw_on_grid.parquet
repairs.csvevery repair action with its reason0 kB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/repairs.csv
data_quality.csvper-tag metrics and status8 kB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/data_quality.csv
issues.csvevery issue located in time3 kB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/issues.csv
glossary_used.csvTAG - Description - EU as used4 kB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/glossary_used.csv
manifest.jsoninputs, settings, counts - the reproducibility record1 kB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/manifest.json
quality_overview.pngtags x time availability map221 kB/Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO/quality_overview.png

Project folder: /Volumes/YN-4T/WIP/CTRL-Designer/B-CDL-Data-Analysis/A-Data/Chemical-Reactors-21d-AIAO

5. Next steps

  1. Review the 20 flagged tags in the bad-tags and fixed-tags reports. Decide which spikes are process events and which are data faults - that judgement belongs to a person, not the tool.
  2. If the repairs are not right, re-run Step A with different rules (spike flag/clip/remove, frozen unknown/keep).
  3. Step B (Explore): read data.parquet from this folder, trend and cluster the tags, choose the tags, the period and the reference (normal) window, and write scope.json into B-Scope/Chemical-Reactors-21d-AIAO/.
  4. Then Step C analyses only what the scope names.

© 2026 CTRL Designer LLC. All rights reserved.
Yahya Nazer · info@ctrlDesigner.com