Skip to content

docs: ADR 0005 competency mastery storage, without history - #714

Draft
jesperhodge wants to merge 7 commits into
openedx:mainfrom
jesperhodge:jesperhodge/competency-adr-5-mvp
Draft

docs: ADR 0005 competency mastery storage, without history#714
jesperhodge wants to merge 7 commits into
openedx:mainfrom
jesperhodge:jesperhodge/competency-adr-5-mvp

Conversation

@jesperhodge

Copy link
Copy Markdown
Contributor

Draft. Replaces the ADR 0005 in #657, which decided storage and history together.

Stacked on #713 (ADR 0004), so the diff here is only what ADR 0005 adds. Retarget to main once #713 merges.

What changed

The whole history and audit-trail question moves out to a future ADR. What is left is the MVP storage decision:

  1. Materialize a status row at all three levels including the leaf: at most one row per learner and node, updated in place, absent until attempted.
  2. That write is an upsert against the unique index, since two workers handling a redelivered event can both find no row and both insert.
  3. All three tables in the default database, unpartitioned, read and written on the primary. This is what makes ADR 0004's single transaction and group row lock work.
  4. No schema conventions of their own: BigAutoField primary key, real FK to settings.AUTH_USER_MODEL.

Plus an Assumptions section (the deployment routes the grade write to the same alias; nothing redirects the recorder's reads to a replica) and an Out of Scope section that records the audit requirement, what the MVP can and cannot answer, and the constraints the future history ADR inherits.

All seven rejected alternatives survive, reworked. Several of their cons cited the HISTORY table and had to be re-derived; most came back stronger against ADR 0004 than they were against history.

Changes to ADRs 0002 and 0003

Two of these are forward-compatibility, deliberately taken now because both are free today and a migration on a billion-row table later:

  • CompetencyMasteryStatuses.rank. The seeded statuses are listed highest-first and there is no ordering column, so ADR 0004 Decision 2's "store the higher of the two" cannot be a single UPDATE. Doing it in Python would need a leaf row lock that ADR 0004's rejected alternative 1 explicitly claims the design does not need.
  • modified on the three learner status tables, with created narrowed to the row's first write. Under in-place updates created no longer dates the current status. Without this, no MVP-era row can be dated, which makes a later history table's backfill meaningless.
  • CompetencyRuleProfile.archived said archived profiles keep learner status history resolvable. It keeps rows resolvable.
  • ADR 0003's Context sized learner data as learners x criteria x time. The time factor was append-only residue.
  • ADR 0003 Decision 5's "how history is retained is not decided here" now points at where it is deferred to.

ADR 0001 and ADR 0004 need no changes.

Open questions for review

  • Staff-facing reads have no index. Every learner status index leads with the learner, so "every learner's status for competency X" is exactly the non-indexed query the ADR's own Context calls painful at this row count. Named in Out of Scope rather than decided.
  • User deletion / retirement. Decision 4 keeps a real FK to the user model but does not say what happens on delete. Named in Out of Scope.
  • Native MySQL partitioning is foreclosed, not merely deferred: MySQL needs every partitioning column in every unique key, and the surrogate id and the unique (learner, node) key share none. Stated and accepted rather than glossed as reversible.

Verification

sphinx-build over docs/ succeeds with zero warnings. doc8 is clean except one pre-existing trailing-whitespace error on main, untouched here.

🤖 Generated with Claude Code

jesperhodge and others added 7 commits August 7, 2026 10:57
Adds ADR 0004 covering how learner competency mastery is recorded under
concurrent, out-of-order grade-change events without a per-event
serialization cost.

Adjusts ADRs 0002 and 0003 to match: learner status is stored as an
in-place ACTIVE row plus a paired append-only HISTORY table, with a
unique index on the leaf HISTORY advance that serves as the idempotency
key for the append.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Whether learner status history is stored as separate append-only tables,
and whether those are advance-only, is still under discussion. Remove the
decision from ADRs 0002, 0003, and 0004 rather than commit to it.

ADR 0004 now states only that the competency mastery status writes and
the roll-ups commit in the same transaction as the subsection grade,
without specifying what else that transaction may carry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Monotonicity is a property of the automatic path, not an invariant of
the stored data. Grade changes and competency criteria rule changes
never lower a status, but staff can, through Django admin or as a
deliberate instructor correction, and a direct edit cascades to the
ancestors above the edited node.

Add an Open Questions section recording that the cascade decision needs
confirmation, and that whether a cascade may overwrite a hand-set
ancestor status is still undecided.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The row lock taken by the merge UPDATE is held until the transaction
commits, not for the duration of the statement. Mechanism 2's lock
ordering argument depends on that, so state it correctly.

Give the deadlock-freedom argument its missing premise, that the
criteria tree gives every node exactly one parent, and close the gap
where one grade change advances several leaves at once by fixing their
lock order.

Attribute the READ COMMITTED default to Django's MySQL backend, which
is what actually sets it, and drop the unenforced claim that higher
isolation levels are unsupported.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Break the dense Context and Decision paragraphs into one idea each, cut
the nested parentheticals, and drop the write-skew jargon in favor of
the plain description already alongside it. Replace mechanism 3's vague
"generalized as needed to other places" with what it means: further
entry points will call the same function.

Content is unchanged. Every mechanism, the monotonicity carve-out for
direct staff edits, and the isolation-level argument all say what they
said before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Decides MVP storage only: one row per learner and node at all three
levels, in the default database on one connection, with this repo's
default primary key and user foreign key. The history and audit-trail
question is deferred to a future ADR, with the constraints it inherits
recorded here.

ADR 0002 gains a rank column on CompetencyMasteryStatuses and a modified
timestamp on the learner status tables, both of which are free now and a
migration on a billion-row table later.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Aug 11, 2026
@openedx-webhooks

Copy link
Copy Markdown

Thanks for the pull request, @jesperhodge!

This repository is currently maintained by @axim-engineering.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

🔘 Update the status of your PR

Your PR is currently marked as a draft. After completing the steps above, update its status by clicking "Ready for Review", or removing "WIP" from the title, as appropriate.


Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@github-project-automation github-project-automation Bot moved this to Needs Triage in Contributions Aug 11, 2026
@mphilbrick211 mphilbrick211 moved this from Needs Triage to Waiting on Author in Contributions Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

open-source-contribution PR author is not from Axim or 2U

Projects

Status: Waiting on Author

Development

Successfully merging this pull request may close these issues.

3 participants