Public energy data usually reaches an analyst as a file, a report or an API response. By that point, the publisher has already made decisions about what to measure, which population to include, how to organize the observations and when to release them. Those decisions remain part of the data even when they are not obvious from its column names. Consequently, downloading a report is only the beginning of understanding whether it can support a particular analysis.
A18 Analytics is building Ask Bhalo around the work that follows that download. The objective is to make public energy information available as governed data products, with the definitions, quality information and source history needed to use it responsibly. A consumer should be able to understand what a measure represents, how it was prepared and which reporting boundaries apply without having to reconstruct every decision from the publisher’s website.
The process can be understood through seven connected stages: discover, capture, preserve, validate, standardize, govern and publish. Each addresses a different responsibility, although validation and governance continue throughout the lifecycle. Using the Ontario Electricity Demand product as the operational example, the following account explains how A18 connects these stages and how the same approach is being extended to OEB sources during Ask Bhalo’s pre-launch development.
Discover: Establish what the source can support
Discovery begins with identifying an authoritative publisher and understanding the reports it makes available. For Ask Bhalo’s initial Ontario electricity coverage, the IESO Data Directory provides the starting point for market and system reporting, while OEB Open Data supports the discovery of distributor information. Registering these sources establishes what is available for consideration, but a separate decision determines which reports should enter development for a particular product.
This separation allows the catalogue to document a broader range of sources while ingestion remains selective. Before a source becomes a product dependency, its measures, reporting periods, units and level of detail need to be understood. An hourly system report and an annual distributor return may concern the same sector, yet they describe different aspects of it. Discovery therefore includes deciding what analytical purpose a source can serve and where its definitions place limits on that purpose.
For Ontario Electricity Demand, the initial implementation uses the IESO Hourly Ontario and Market Demand report. Related sources can be registered for later development without becoming requirements of the current product. As a result, the scope of the release remains explicit, and the presence of a report in the registry does not imply that its data has already been ingested or certified for use.
Capture: Bring the selected publication into the platform
Once a source has been selected, an adapter brings its published data into the ingestion process. The adapter provides the connection between the publisher’s reporting structure and the platform’s shared lifecycle. Because publishers organize and release information differently, this connection must accommodate the source’s own structure while allowing subsequent processing to follow a consistent approach.
Ask Bhalo’s adapter interface is designed independently of IESO-specific types, allowing additional publishers to enter the same lifecycle without inheriting IESO’s reporting assumptions. OEB, for example, can use the common process while retaining its annual reporting periods, distributor identities and source-specific fields. This is the basis for expanding coverage without requiring every new publisher to resemble the first one.
Successful capture establishes that the platform has obtained a source artifact for processing. Its fitness for publication still depends on the stages that follow. Keeping those responsibilities separate matters because a technically successful ingestion can contain a publisher gap, an unresolved definition or a structure that requires further examination before its values can support a consumer product.
Preserve: Retain the evidence behind the data
After capture, the raw artifact is preserved immutably and identified through a content hash. Retaining that version gives the platform a reference against which subsequent transformations can be understood. If a publisher revises a report, the revised content can be distinguished from the earlier artifact rather than being treated as though the source had always contained the new values.
This becomes particularly useful when examining how a published observation reached an analytical output. The source version identifies the file content, the pipeline version records the transformation applied, and the product version describes the contract presented to the consumer. Together, these distinctions make it possible to explain whether a change originated in the publication, in its processing or in the definition of the product itself.
Preservation also provides a basis for handling republications. In OEB source development, revised reporting releases are retained as identifiable versions, allowing their differences to remain part of the record. The commercial interface provides curated outputs, while the preserved source artifacts support the lineage and verification behind those outputs.
Validate: Assess the data against its reporting context
Validation examines the captured information in relation to what the source and product are expected to contain. Completeness, for example, depends on the period being assessed and the publisher’s release cycle. An open market day may legitimately contain only some of its hourly observations, whereas an absent observation in a closed period requires a different interpretation. Ask Bhalo therefore evaluates completeness against publication expectations and the detail represented by each row.
The Ontario Electricity Demand implementation illustrates this distinction. An expected partial period can remain operationally healthy, while the missing hour-ending 01 observation on 1 May 2025 in the IESO demand file is retained as a source gap. The platform does not fill that omission with an invented observation. Instead, the missingness remains visible so that any later analytical treatment can be understood separately from the publisher’s record.
Quality outcomes are retained for each ingestion, alongside information about whether an issue originated with the publisher, within Ask Bhalo’s processing, through both, or from an unresolved cause. Critical failures can prevent curated publication. Although validation is described here as one stage, its responsibility continues as data is standardized and metrics are prepared, because a source that passes an initial check can still be transformed incorrectly or used in an unsupported calculation.
Standardize: Create a consistent structure while preserving meaning
Standardization maps source fields into an explicit canonical structure, making the data easier to work with across the platform. The central requirement is to retain what each observation represents. In data modelling, this is its grain: an hourly actual observation, a forecast defined by its issue time and target hour, and an annual distributor measure each have a different grain, even when they contribute to a related subject.
Ask Bhalo preserves these differences within its data assets. The catalogue can bring related assets together as part of a consumer product, but the relationship does not establish that their rows can be combined without further interpretation. Hourly system demand in megawatts and distributor consumption in kilowatt-hours describe different measurement concepts. A shared topic or a similar field name is therefore insufficient evidence for treating them as equivalent.
The same discipline applies to transformations already performed by the source. Where an OEB publication states that blanks were populated with zeros, Ask Bhalo retains the published zero while recording the transformation and uncertainty around its origin. Organization identity also receives explicit treatment through legal names, known aliases and documented predecessor relationships, so historical observations remain associated with the organization valid during their reporting period. Unresolved identities enter a review queue, allowing uncertainty to remain visible until there is evidence for a match.
Govern: Define the conditions under which the product can be used
Governance connects the prepared data to a defined analytical purpose. Each product carries intended uses, source dependencies, business definitions, quality expectations and limits on interpretation. These requirements influence the earlier stages as well: the intended use shapes source selection, while agreed definitions guide field mappings and validation. Describing governance as a stage makes its responsibilities easier to explain, but those responsibilities are embedded throughout development and publication.
A18 separates publisher datasets, curated assets, governed metrics and consumer products within this structure. A metric requires its own definition and evidence, including a formula where it is calculated. Ask Bhalo distinguishes publisher-reported values from calculations using verified official equations, deterministic derived measures and modelled outputs. For a derived result, its inputs, aggregation method and treatment of exceptional cases must remain traceable, allowing the consumer to understand what the platform contributed to the final value.
These controls also determine which relationships the product can support. The implementation described here does not certify a transformation that makes IESO system demand and OEB distributor measures directly comparable. Related datasets can still be discoverable together, provided that their limitations remain explicit. Similarly, an unresolved sign convention or source definition keeps the relevant capability under validation until documentation or reproducible evidence establishes its meaning.
Freshness, health and maturity form another part of this agreement. The hourly IESO actuals product has a daily publication rhythm and a 36-hour product freshness target, while annual OEB reporting must be assessed against its own cadence. Operational health concerns the current published contract, whereas maturity and implemented coverage describe different aspects of progress. A Beta product can therefore be healthy against its stated scope while additional capabilities remain in development.
Publish: Make the governed product available for use
Publication makes the curated product available through its supported consumer interfaces. Product pages, downloads and the commercial API use a shared product layer, carrying the same definitions and boundaries into each access route. BI tools and notebooks can consequently work with consistent product identifiers while users retain access to the context needed to interpret the data.
For Ontario Electricity Demand, the initial delivery connects selected IESO reporting to curated demand data and derived daily, monthly and annual actual-demand metrics. The consumer offering includes the information required to understand its scope and assess its quality. Product-level lineage also allows operators to identify which offerings are affected when a supporting source changes, fails a quality rule or arrives late, connecting the internal processing record to what users experience.
Publication status remains distinct from catalogue visibility. A product listed as “Data Contracted” or “Coming Soon” communicates its development state without promising accessible rows, a functioning API or a completed history window. Data access, API access and download availability are recorded separately, and historical coverage reflects what has actually been loaded and what the user is entitled to access. This keeps the catalogue aligned with the product’s current capabilities as the pre-launch programme develops.
The lifecycle continues after publication because the next source release must be captured, preserved and assessed against the same obligations. A publisher revision can change the evidence, a pipeline change can affect its preparation, and a broader product release can alter the supported scope. Keeping those changes identifiable is part of maintaining a dependable product over time.
What this means for the energy data user
By the time a public source reaches an Ask Bhalo product, it has acquired a documented relationship to an analytical purpose. The original publisher remains identifiable, the preparation steps can be traced, and the definitions and quality conditions remain attached to the output. This allows users to begin with a clearer account of what the information represents and where further judgement is required, while the platform maintains the preparation work across recurring releases.
At this stage, the evidence concerns the operational foundation: the Ontario Electricity Demand implementation, its publication lifecycle and the controls supporting it. The work on OEB extends that approach through cataloguing, profiling and product design, with further offerings subject to their own validation and release requirements. User time savings and broader commercial impact remain to be established through adoption and measurement as Ask Bhalo moves into wider use.
A18’s objective is to expand the catalogue while preserving this accountability for every product it contains. The development sequence provides a practical way to do that, connecting source discovery to a consumer offering whose meaning and limitations can be explained. Follow the technical build as Ask Bhalo develops its next products, with further articles examining the modelling decisions, quality controls and interfaces behind their release.