Keet notes, chapter 5: methodology and the quality gates

Chapter 5 of Maria Keet’s ontology-engineering textbook is the methodology chapter, and our study passes through it kept arriving at the same place from different directions. Macro-level methodologies, the OOPS!/TIPS pitfall work, competency-question discipline, and the tooling survey each answer a version of one question: what, besides a reasoner that finds no contradictions, has to be true before an ontology is safe to operate on? The converging lesson: green is not the verdict. Consistency checks stop at the class layer, and the expensive failures live in relation semantics, modeling choices, loose question intake, and tooling nobody can find.

Methodology has to survive as artifacts

Keet’s macro-methodology discussion makes a point that maps directly onto company-building: the model is not enough, because the decisions behind it have to survive inspection, reuse, and handoff. A methodology is not infrastructure if it disappears when the first operator leaves.

The practical form is a machine-readable artifact that records what the system is for, what it must answer, which standards or reuse candidates shaped the design, and who owns the review as the business changes. A competency-question register stored as data, with the question, the model area it depends on, and a review owner, gives the next operator something to inspect instead of a methodology buried in meeting notes. Our test is blunt: reasoning that cannot survive in an artifact another operator can inspect is not yet part of the system.

The relation layer can be wrong while the checks stay green

The chapter’s sharpest distinction is between consistency and semantic quality. A subproperty can be scoped too broadly. A domain/range pairing can carry the wrong business meaning. A property chain can combine two valid relations into an inference the business never intended. None of that is guaranteed to break a class-consistency check.

The failure shape we used in the study run: a subproperty meant as “can coordinate with” drifts under a broader property that downstream systems treat as “can act on behalf of.” The reasoner stays green. A workflow engine can now route approvals or task authority to the wrong party, and the damage comes from false trust rather than visible breakage. That gap is why the study run opened ONT-007, a practical RBox and property-chain audit path that would warn when relations are mathematically coherent but semantically suspect. It is an opened issue, not a shipped check; until it ships, relation review stays a human job here.

Pitfalls are mental-model errors, not syntax errors

Section 5.2.4 on OOPS! and TIPS extends the same argument to modeling choices. An ontology can pass formal checks and still encode weak semantics: a workflow state modeled as a kind of customer, an inverse with the wrong meaning, a catch-all class hiding distinctions that matter. Keet’s pizza example looks small, an intersection placed inside one existential accidentally models a single combined topping, but the point is that the error started in the author’s mental model, the reasoner never objected, and the wrong meaning got embedded.

TIPS matters alongside OOPS because suspicion alone is not repair: authors also need guidance on expressing the intended meaning correctly. Our gap here became ONT-008, a pitfall-scanning surface with consistent grading and a repair path, separate from contradiction findings. Also planned work, not a live gate.

Competency questions need an authoring contract

Section 5.2.5 exposed a different kind of gap. The platform already tracks competency questions as explicit artifacts, validates answer-path structure, filters malformed generated questions, and preserves lifecycle metadata. The machinery exists. What contributors do not have is a contract for writing a question so it becomes a reusable check: the question in controlled language, the business object or relation it is about, the expected answer form (yes/no, list, count, threshold breach, explanation), the executable target (query, validation rule, report, decision step), and the review owner.

The contract is also the LLM-safety mechanism. A model allowed to generate free-form competency questions can produce plausible noise all day; a model filling a governed template with an answer form and an executable destination has a narrower job that a reviewer can actually check. If a contributor cannot name the answer form, target, and owner, the question is still a draft note.

Hidden tooling behaves like missing tooling

Section 5.2.6 closes the loop on discoverability. The repo behind ONT-011 is not empty: governing design notes, ontology sources, validation surfaces, generated summaries, and change reports all exist. What does not exist is an entry point. When finding the right artifact requires knowing the repo layout, operators do one of three things: guess (semantic drift), interrupt an insider (dependence), or stop using the surface (undocumented habit).

Status is not discovery. A readiness endpoint says whether something is healthy; it does not say where the governing design note for a vertical lives, which ontology file is canonical, or which report explains the latest change. The standard ONT-011 would have to meet, if it gets built, is one map from task to artifact: an operator with a plain question like “what governs appointment scheduling for dental?” should land on a page that names the governing documents, the canonical files, the validation surface, and the current summaries. Like the other two issues, it may never ship in this shape; the gap it names is real either way.

If you run a business

The study-note vocabulary is ontology-specific; the standard is not. A clean dashboard can mask drift underneath it. A plausible label can encode the wrong operating rule. A capable toolchain nobody can navigate keeps the company dependent on the people who built it. Chapter 5’s answer, translated: keep the green light, but make the verdict come from artifacts another operator can inspect, audits of the layer where meaning lives, named repair paths for common mistakes, contracts for what enters the system, and a front door that lets the next person find what governs their work. Those gates are what make a system inheritable instead of merely impressive.

---

Source evidence used in this note: Maria Keet’s ontology-engineering textbook, Chapter 5 (macro-level methodologies, Sections 5.2.4-5.2.6); smb-ontology-platform/docs/plans/2026-03-31-keet-ontology-engineering-progress-tracker.md (2026-04-02 through 2026-04-11 entries); smb-ontology-platform/docs/issues/ONT-008-add-ontology-pitfall-scanning-and-tips-style-authoring-guidance.md; and smb-ontology-platform/docs/issues/ONT-011-add-ontology-tooling-catalog-and-documentation-export-index.md. ONT-007, ONT-008, and ONT-011 are opened issues, not shipped capabilities. This chapter note consolidates five earlier study posts on methodology artifacts, RBox governance, pitfall scanning, competency-question contracts, and tooling discoverability.