The brain: collections, records and pages

Draft for review

The brain is where your company's structured data lives: leads, tickets, roadmap items, incidents, and the pages that show them. People and agents read and write the same records under the same rules. Every write is kept: who made it, for whom, from which run, and every earlier version, so a bad write can be rolled back.

This guide describes the brain as it works today on the golive branch. To follow along, start the demo company (see Quickstart) and sign in as the person each step names. What is not possible yet is listed at the end.

Collections#

A collection is one kind of record with a schema: its fields, their types, which are required, and how sensitive the data is. Modules bring their collections with them (the CRM module brings leads and contacts, Support brings tickets). OrchKernel has two of its own: modules and views (the pages, see Pages are records too).

Open Brain in the sidebar (under Company). The left column lists every collection. Under each name is a line such as "Kept here · Customer PII", and on the right a number or a lock:

Type in Search collections to narrow the list. It matches names, descriptions and synonyms: as Alice, pipeline leaves Candidates, Forecasts and Leads.

Click a collection to open it. The head shows its name, the same two facts as badges (Kept here, Customer PII), its description and synonyms ("aka prospects, pipeline, customers, deals"), and on the right one of Add record (you may write), Read only or No access. On a phone the list and the collection are separate screens; ‹ All collections goes back.

The Schema & policy tab shows how the collection is built. For Leads:

Below it is one line per field: name, type (enum values listed), required, data class, synonyms and description. Under the table, after "e.g.", are example questions agents use to find the collection.

Who sees which records#

Every read runs as you. A collection's read teams decide whether you see it at all, and its row rules decide which rows. Being an admin does not bypass this.

In the demo, on Brain › Leads:

Writes follow the write rules. Alice can edit her own leads but not Sam's, and she cannot create a lead owned by someone else ("alice may not create a row owned by someone else in leads"). A new record with no owner is owned by whoever wrote it.

Browsing and filtering records#

The Data tab shows the records you may read, in a table built from the schema: money is formatted ($12,000.00), dates read as days (Mon, Oct 5), people as names. The last two columns are the version (v) and who wrote it (By). Click a column head to sort by it; click again to reverse. Numbers sort as text, so Value puts $12,000.00 before $9,000.00 (a bug).

To filter, as Alice on Brain › Leads:

1. Choose Add condition. A row appears: a field list, an operator list and a value box.
2. Pick stage, keep eq, and type contacted.
3. Choose Run. The table shows Litware only, and the count reads "1 of 7 rows you can read · as Alice Chen".

Operators are eq, ne, contains, gt, gte, lt, lte, is_null and not_null. All conditions must match. typed filter shows the filter as JSON, the form agents and the API use. Remove a condition with its ×, then choose Run again.

A record: provenance, history and links#

Click a row to open the record in a panel. Its title is the collection, the first 8 characters of the record's id and its version (Leads 96d4ef85 V1). It holds:

• Edit, Flag as suspicious and Delete, shown to anyone who may write to the collection, even on rows they may not change (see Flagging a bad write).
• The fields.
• Provenance: written by (the person or agent), acting for (the person an agent worked for), run (a link to the agent's run on the Work page), skill, source (direct, or rollback when the version came from a rollback) and at.
• History: every version, newest first, each with who wrote it, its source and when. Open fields on a version to see its values. Every version but the current one has Roll back to vN.
• Linked from: records in other collections that point at this one. On Alice's Litware lead it shows a Contacts record for Dana Brooks, CTO, whose lead field names Litware.

Adding and editing records#

To add a lead, as Alice on Brain › Leads:

1. Choose Add record. A sheet opens, titled "New leads record", with one input per field. Each label gives the field name and type (name * · string); * marks a required field. Enums are a list, dates a date picker, yes or no a checkbox, and an owner a list of people and agents ("Alice Chen (alice)").
2. Type Woodgrove Bank in name and pick new in stage. Leave owner empty.
3. Choose Create. "Record created" appears and Woodgrove Bank is in the table at version 1, owned by Alice Chen.

To edit:

1. Click the Tailspin Toys row, then Edit. The fields become a form.
2. Change value to 950000. Money is typed in cents (the box shows cents when empty): 950000 is $9,500.00.
3. Choose Save. "Saved new version" appears. A moment later the panel shows V2 with value $9,500.00, and v1 is still in History.

Every save makes a new version; nothing is overwritten. The server refuses a field the schema does not know ("unknown field budget"), a value outside an enum, and a missing required field.

Emptying a field in the form and saving does not clear it: the save makes a new version with the old value still there (a bug). Through the API, sending the field as null clears it.

Rolling back#

Rolling back makes a new version with the fields of an earlier one. The bad version stays in History, so the rollback can itself be undone.

1. As Alice, open Litware, choose Edit, set stage to qualified, and choose Save. The record is at V2.
2. In History, on v1, choose Roll back to v1. "Rolled back to v1" appears.
3. A moment later the record is at V3 with stage Contacted, and Provenance shows source rollback. History lists V3, V2 and V1.

Only someone who may write the record can roll it back. Sam, who can read Alice's leads, sees Roll back to v1 but is refused: "brain: access denied: sam may not modify row ...".

Rolling back also clears any flags on the record (next section).

Delete opens a box, "Delete?" with "Soft-delete this record?", and Cancel or Delete. Choosing Delete shows "Deleted" and removes the record from the table. The delete is a new version marked deleted; the earlier versions stay. The Brain page does not show deleted records, so restoring one takes a rollback to an earlier version through the API or CLI (see For developers).

Flagging a bad write#

When a record looks wrong and you cannot or should not fix it yourself, flag it. The owner is told.

1. Sign in as Sam, open Brain › Leads and click Tailspin Toys (Alice's lead).
2. Choose Flag as suspicious. The browser asks "Why is this write suspicious?". Type Value looks wrong; they quoted 9,000 and choose OK.
3. "Flagged; the owner has been notified" appears.
4. Sign in as Alice and open Inbox. Under Items, a new item reads "Record leads:6ed34b48-... flagged: Value looks wrong; they quoted 9,000" (the id is the full record id).
5. Alice opens Tailspin Toys on Brain › Leads and rolls it back to the right version. The flag is cleared.

The flag goes to the record's owner; if it has none, to the person the writer acted for, or else to the writer. Only people who may write to the collection see Flag as suspicious. The Events page lists the flag as Record flagged.

Things to know:

• The record does not show that it is flagged, and there is no list of open flags on the screen.
• The inbox item names the record by its id, not its name, and does not link to it.
• Sam also sees Edit, Delete and Roll back on Alice's lead. Saving, deleting or rolling back is refused with a raw error ("brain: access denied: sam may not modify row ... in leads"), because only the owner writes a lead.

Search#

The Search box at the top of the sidebar searches the records and knowledge you may read, as you type. As Alice, litware finds the Litware lead, and clicking it opens its Lead page. fabrikam shows "Nothing you may read matches.", because Fabrikam Labs is Sam's lead. Without an embedding model (a model that lets search match by meaning), as in the demo, search matches your words only and says "Matched on your words only: search by meaning is not set up here." The workspace tour covers the box in full.

Collections kept in another system#

A module can keep a collection in a system your company already runs, such as a CRM, instead of in OrchKernel. The admin chooses this when enabling the module: the enable sheet offers "Keep records in " for each collection the connection maps (see Modules and Connections).

Such a collection is federated:

• Its line reads Kept in and it shows no record count.
• Every read and write goes to the vendor through the connection, as you. Nothing is copied into OrchKernel. Each call is an event (Call to another system).
• A read stops at 5,000 rows or 60 seconds. The table, and any page that reads the collection, then shows "partial: the first 5,000 from . Rows past them were not read."
• Deleting or rolling back one of its records is refused; change it in the vendor instead.
• The Schema & policy tab shows its external mapping.
• Where a collection is kept can change only while nothing is stored for it in OrchKernel.

The demo keeps every collection in OrchKernel, so none of this shows there, and this section was not checked in the demo.

Changing a schema: the Schema Steward#

No agent changes a schema directly. Changes arrive as proposals, and a person applies or rejects them on Governance › Schema.

The Schema Steward is a built-in agent that looks for structure that is needed or no longer needed and proposes changes. From its code:

It never applies anything. An admin starts it with Run steward now on Governance › Schema; only admins see that button.

Getting a new field#

The Steward's way to a new field is to see people already keeping it in a text field. In the demo:

1. As Alice, edit the notes of three of your leads to add a line each: Budget: 20000 on Contoso Health, Budget: 15000 on Litware, Budget: 9000 on Tailspin Toys. Each value must start its own line.
2. As Ada, open Governance, then Schema, and choose Run steward now. "Steward proposed 1 change(s)" appears. Under Open a card marked Proposed reads add field on leads, "3 records carry budget: inside notes; a real field would make it queryable", proposed by Schema Steward. details shows the field as JSON. Ada also gets an inbox item, "Steward proposes on leads: add field budget".
3. Apply it, as below. The Leads schema goes to version 2 and every lead has an empty budget field. The values in the notes are not copied over; fill them in.

A person can also propose a change through the API (POST /schema/proposals, see For developers). There is no form for it on the screen.

Applying or rejecting#

Applying a schema change is a critical action, one that needs an admin:

1. As Sam, open Governance › Schema and choose Apply on the budget card. The screen says "Applied", but the card stays under Open: the change is waiting for Ada (a bug: the message should say it was sent for approval).
2. As Ada, open Inbox. Under Waiting on you, a card Apply a schema change reads "Sam Ortiz asks to apply a schema change.", "Proposal 712b2529" (the start of the proposal id) and "critical actions require an admin". Choose Approve. "Approved" appears.
3. On Governance › Schema, the card is under Decided, marked Applied.

Reject asks "Why?" and records your answer on the card ("proposed by Eli Novak · Not needed"). If the card still shows under Open, reload the page.

Anyone signed in can propose a change to any collection, even one they cannot read: Eli, an engineer, can propose a field on leads. Proposing changes nothing; applying is what is governed.

Pages are records too#

The pages in the sidebar between Curation and Company (Leads, Pipeline, Marketing, Support operations and so on) are not code. Each one is a record in the views collection, so it has provenance, versions, history and rollback like any other record. One renderer draws every page from its record.

A page record holds:

The sidebar groups pages by section, sections in alphabetical order (Context, Marketing, Product, Sales, Support for Alice), then by order. An admin can hide or move a module's pages from the module's Pages tab without editing the page record (see Modules).

The block kit#

A page is built only from these blocks. A page cannot add its own styles or new kinds of block.

A button that asks an agent opens a new conversation with that agent with the text filled in. On Alice's Litware Lead page, Email the contact opens New conversation with Sales Assistant with "Email the contact at Litware" ready to send.

Who sees a page#

A page never grants access. Every block reads as the person viewing it:

• Alice and Sam open the same Pipeline, and each sees only the leads they may read. Its subtitle says so: "Every number counts only the leads you may read". For Alice, Open pipeline is $21,000.00 (Litware and Tailspin Toys).
• A page none of whose collections you may read is left out of your sidebar, and opening its address says "This page is paused or no longer exists." Alice opening Engineering (/v/tech.overview) gets this.
• A block reading a collection you may not read, on a page you can otherwise see, says "You don't have access to ...". Alice on Support operations sees "You don't have access to Qa reviews."

Who sees which pages in the demo is in the workspace tour.

Editing a page and rolling it back#

An admin edits a page as a record. As Ada:

1. Open Marketing in the sidebar. At the bottom, choose Edit page (it is followed by the page id, marketing.overview, and only admins see it). It opens Brain › Views, the list of every page record, not the one page.
2. Click the Marketing row, then Edit. spec is a JSON text box.
3. Change name to Marketing overview and choose Save. The sidebar entry now reads Marketing overview.
4. In History, choose Roll back to v1. The sidebar reads Marketing again.

Who may write a page record:

A spec that reads a collection that does not exist is refused when saved ("block 0 (table) reads unknown collection 'nope'").

A new page made by a person is published at once, for everyone who may read its data, with no approval. In the demo, when Sam wrote a Follow-ups list page in the Sales section (through the API), it appeared in Alice's pages straight away, and Alice could not edit it.

Asking the Designer for a page#

The Designer is a built-in agent whose one job is writing pages. As its code describes it: you ask it in words ("Give the support team a queue sorted by age"); it reads the collections and existing pages, drafts a page from the block kit, posts the draft in the conversation ("Drafted the page "...". Approve to publish it."), and writes it to views. That write needs your approval. Approving publishes the page, and it appears in the sidebar under its section. Asking it to change an existing page makes a new version of that page.

In the demo this does not work:

• Only admins work with the Designer; as Ada it is under No team in Talk to an agent on the Threads page. Sam does not see it.
• On a server that was restarted (which includes every ok demo serve), asking it fails at once: the conversation shows "No native skill registered as designer" and the task is marked Failed (a bug). The Schema Steward and the Architect fail the same way when asked in a conversation; Run steward now still works.
• Even once that is fixed, the Designer needs a real model; the demo's stub planner cannot draft a page.

Until this is fixed, an admin edits pages by hand as above.

Not possible yet#

• Clearing a field from the edit form: the saved version keeps the old value. Through the API, sending the field as null clears it.
• Seeing which records are flagged, or a list of open flags, on the screen.
• Restoring a deleted record from the screen.
• Proposing a new field, or any schema change, from the screen: only the Steward's own proposals, or the API.
• Asking the Steward for a particular field: it proposes only from what it notices.
• Changing who may read or write a collection from the screen: read and write teams and row rules come from the module.
• Moving a field's values into a new field (the budget values stay in notes).
• A visual page editor: pages are edited as JSON, or by asking the Designer.
• Opening a page's own record from Edit page: it opens the whole Views list.
• Sorting a number or money column by value: it sorts as text.

For developers#

API#

All under /api, with Authorization: Bearer <token>. Reads and writes run as the caller; 403 denied when the caller may not read or write.

To restore a deleted record, find it with include_deleted: true, read its history, and roll back to the last version before the delete:

curl -X POST -H "Authorization: Bearer $TOKEN" -H 'content-type: application/json' \
  http://127.0.0.1:8080/api/brain/collections/leads/records/<id>/rollback \
  -d '{"to_version": 1}'

A field to propose, for example:

{ "collection": "leads",
  "change": { "change": "add_field",
              "field": { "name": "industry", "type": "enum",
                         "values": ["health", "retail", "software"],
                         "description": "The lead's industry" } },
  "rationale": "We segment the pipeline by industry" }

A page record is written like any record, to views, with the page id as the record id (crm.lead, org.roadmap). A minimal list page:

{ "id": "sales.followups",
  "fields": { "name": "Follow-ups", "kind": "list", "collection": "leads",
              "section": "Sales", "order": 60, "icon": "list",
              "spec": { "blocks": [ { "type": "table",
                                      "columns": ["name", "next_follow_up"],
                                      "source": { "collection": "leads",
                                                  "filter": { "op": "not_null", "field": "next_follow_up" } } } ] } } }

The spec grammar is in RFC-0002 and, exactly as the Designer is told it, in crates/ok-kernel/src/designer.rs. Expressions $record.<field>, $record.id, $me.id, $today and $param.<name> may appear in titles and filter values.

CLI#

The CLI reads and writes the state file directly. With the demo running, use it to read; make changes in the workspace or through the API.

ok --state orchkernel-demo/state.db --as alice brain query leads --filter '{"op":"eq","field":"stage","value":"won"}'
ok --state orchkernel-demo/state.db --as alice brain history leads <id>
ok --state orchkernel-demo/state.db --as alice search litware
ok brain collections | describe <c> | upsert <c> --fields '<json>' [--id <id>]
ok brain rollback <c> <id> <version> | flag <c> <id> --reason <text> | flags
ok schema proposals | apply <id> | reject <id> | steward

Events#