From 0f58e6d99270ad39a05d8b435f1eef8133aebfd4 Mon Sep 17 00:00:00 2001 From: Brian Mesick Date: Wed, 5 Feb 2025 15:49:38 -0500 Subject: [PATCH 1/2] docs: Mark OEP-26 accepted, update This OEP has been accepted and largely implemented for a long time. This brings the OEP up to date with current reality. --- .../oep-0026-arch-realtime-events.rst | 41 +++++----- .../oep-0026/caliper-realtime-events.rst | 62 +--------------- .../oep-0026/xapi-realtime-events.rst | 74 +++---------------- .../oep-0052-arch-event-bus-architecture.rst | 2 + 4 files changed, 38 insertions(+), 141 deletions(-) diff --git a/oeps/architectural-decisions/oep-0026-arch-realtime-events.rst b/oeps/architectural-decisions/oep-0026-arch-realtime-events.rst index 824a0fa45..16be5a8ae 100644 --- a/oeps/architectural-decisions/oep-0026-arch-realtime-events.rst +++ b/oeps/architectural-decisions/oep-0026-arch-realtime-events.rst @@ -8,31 +8,45 @@ OEP-26: Real-time Events +-----------------+--------------------------------------------------------+ | Last Modified | 2019-01-16 | +-----------------+--------------------------------------------------------+ -| Authors | Nimisha Asthagiri , | -| | Danial Malik , | -| | Ed Zarecor | +| Authors | - Nimisha Asthagiri , | +| | - Danial Malik , | +| | - Ed Zarecor | +-----------------+--------------------------------------------------------+ -| Arbiter | Brian Mesick | +| Arbiter | Ty Hob | +-----------------+--------------------------------------------------------+ -| Status | Provisional | +| Status | Accepted | +-----------------+--------------------------------------------------------+ | Type | Architecture | +-----------------+--------------------------------------------------------+ | Created | 2018-07-11 | +-----------------+--------------------------------------------------------+ +| Last Updated | 2025-02-05 | ++-----------------+--------------------------------------------------------+ | `Review Period` | 2018-11-29 - 2018-12-20 | +-----------------+--------------------------------------------------------+ + .. contents:: :local: :depth: 3 +Status +****** + +As of 2025-02-05, this OEP has been accepted. It has been implemented in Open edX as the `event_routing_backends`_ plugin, which is currently in use for delivering real-time analytics to Aspects and to deliver real-time events to external systems via xAPI and Caliper. While that project adheres closely to this OEP, there are also other event-based communication mechanisms within the platform that are covered by other OEPs. + +`OEP-41 `_ and `OEP-52 `_ cover the general format and conventions for generic asynchronous event messaging across and within services. While this OEP concerns itself learner-specific event sharing. There is also some overlap, for instance when event-routing-backends is configured to send events to an event bus, however this OEP is not considered superceded at this time. + +Where possible this document has been updated to match the implemented reality. However some sections may be out of date. The `event_routing_backends`_ plugin is the authoritative source for the current implementation. + +.. _event_routing_backends: https://github.com/openedx/event-routing-backends/ + Context ******* -Currently, the Open edX system captures LMS interactions in persisted event streams (tracking logs) of documented proprietary JSON structures, as specified in `Events in the Tracking Logs`_. The events are emitted by backend servers and frontend apps, including web and mobile apps. The logs are shared and used *asynchronously* for various data analyses and processing. +The Open edX system captures LMS interactions in persisted event streams (tracking logs) of documented proprietary JSON structures, as specified in `Events in the Tracking Logs`_. The events are emitted by backend servers and frontend apps, including web and mobile apps. The logs are shared and mostly used *asynchronously* for various data analyses and processing, though through different log handlers / forwarders they can be used for more real-time applications. -Of late, there are emerging use cases that require notifying external systems of these LMS events in real-time, preferably in industry standard formats (such as xAPI_ and Caliper_). By **real-time** events, we mean pushing data as fast as possible in an automated and synchronous communication at a speed within a few hundred milliseconds (to satisfy `human perception`_). Use cases, such as Adaptive Learning, do require this sub-second communication time. For other use cases, however, **near-real-time** communication (with a delay of multiple minutes) is sufficient. Unless explicitly called out in the document, near-real-time is included when we refer to real-time. Both are orders of magnitude faster than the current **batch processing** available via the tracking logs. +There are emerging use cases that require notifying external systems of these LMS events in real-time, preferably in industry standard formats (such as xAPI_ and Caliper_). By **real-time** events, we mean pushing data as fast as possible in an automated and synchronous communication at a speed within a few hundred milliseconds (to satisfy `human perception`_). Use cases, such as Adaptive Learning, do require this sub-second communication time. For other use cases, however, **near-real-time** communication (with a delay of multiple minutes) is sufficient. Unless explicitly called out in the document, near-real-time is included when we refer to real-time. Both are orders of magnitude faster than the **batch processing** of tracking logs. This OEP proposes an evolutionary change to Open edX's eventing system that will enable near realtime event-based feedback loops. Events have the potential to connect users, organizations, and learning services in ways that enrich learning on the platform. @@ -42,7 +56,6 @@ This document describes a design proposal for supporting standard real-time even .. _xAPI: https://xapi.com/ .. _Caliper: https://www.imsglobal.org/activity/caliper .. _human perception: https://www.pubnub.com/blog/how-fast-is-realtime-human-perception-and-technology/ -.. _Gartner reports: https://hackernoon.com/by-2020-50-of-managed-apis-projected-to-be-event-driven-88f7041ea6d8 Use cases @@ -98,7 +111,7 @@ Use Case: External Service **Requirement:** Real-time (*sub-second*) or near-real-time (*minutes*) communications depending on integration type -As we move towards creating a more extensible platform for Open edX, we want to provide integration options between the Open edX platform and external services. External services that enhance the learning, authoring, or LMS administrative experiences will benefit greatly from a real-time eventing framework. Some examples include an external social learning service and a commercial learning analytics tool. *Theoretically, the other use cases are specific cases of this* *use case*. +As we move towards creating a more extensible platform for Open edX, we want to provide integration options between the Open edX platform and external services. External services that enhance the learning, authoring, analytics, or LMS administrative experiences will benefit greatly from a real-time eventing framework. Some examples include an external social learning service and a learning analytics tool. *Theoretically, the other use cases are specific cases of this* *use case*. .. image:: oep-0026/use_cases.png @@ -139,7 +152,7 @@ For details on integrating with Caliper, please see the :ref:`caliper_realtime_e Anonymized User ID ================== -Users will be identified to external systems using a UUID that is associated uniquely with a single user and the external system type with which the UUID can be shared. This decision overrides :ref:`oep-32` and is captured in `ADR 0001-externalid.rst`_. +By default users will be identified to external systems using a UUID that is associated uniquely with a single user and the external system type with which the UUID can be shared. This decision overrides :ref:`oep-32` and is captured in `ADR 0001-externalid.rst`_. Some external systems may require a different identifier for the user to support, for example, xAPI specifies a `mbox` field that is an email address. The system will allow for the configuration of a different identifier for each external system type to allow for flexibility of integration with external systems. .. _ADR 0001-externalid.rst: https://github.com/openedx/edx-platform/blob/e1c8fb51bec2be3aa8da9750cb52b01728d1a740/openedx/core/djangoapps/external_user_ids/docs/decisions/0001-externalid.rst @@ -225,12 +238,6 @@ Decisions & Consequences * **Event-driven APIs at Scale** - The Context_ section describes the motivation and recent use cases for supporting real-time events. This capability has the potential to dramatically improve how external services can integrate and extend the Open edX platform. Real-time events will provide a mechanism for loosely coupled integrations with a reliable, standardized contract. - A big consideration and concern that is sorely missing from this version of the OEP is explicit recommendations on the infrastructure that will be used to support scalability. On one hand, the advantage is that this agnostic approach allows Open edX instances to reuse the core capabilities (and modular subcomponents) without being tied to a specific scalable technology. On the other hand, we run the risk of needing to reimplement initial implementations if a chosen technology's design is fundamentally counter to our choice of boundaries. - -* **Emphasis on user privacy** - We are taking a conservative approach by minimizing the PII that is sent to consumers. The trade-off is that consumers may find the received user identifiers limiting. However, at this time, it's unclear whether adaptive engines, which are written generically for all users, need PII to be effective. They need the ability to bind events together and track pathways and progress for users, but they can do so with any unique identifier - hence the introduction of the :ref:`oep-26-user-id`. - - For Enterprise and other use cases, sharing PII may be required. We have chosen to keep those use cases in mind, but not target them initially, with the understanding that future work would be needed to address those needs. - * **Implementation of an LRS is out of scope** - As mentioned in :ref:`oep-26-lrs`, creating an Open edX specific LRS not planned. As a consequence, adaptive engines may need to maintain their own LRS if they need to refer back to previous events. Given our business research to date, it seems many adaptive engines are already maintaining their own custom-optimized storage of event data. @@ -253,7 +260,7 @@ Here are a list of current Open edX frameworks that are related to "eventing" bu * **Event-driven microservices architecture** - Using event-driven asynchronous messaging between microservices is a design pattern we have adopted for the Open edX system. Such a loosely coupled architecture provides optimal scalability, resiliency and responsiveness, per recommendations in the `Reactive Manifesto`_. - As summarized in Pivotal's `Messaging Patterns for Event-Driven Microservices`_, there are various integration frameworks for supporting eventing frameworks. We currently use the "Asynchronous Command Calls" (2nd) pattern via Django Celery. We may eventually trial and adopt the "Event Firehose" (1st) pattern, using a technology such as Apache Kafka. + As summarized in Pivotal's `Messaging Patterns for Event-Driven Microservices`_, there are various integration frameworks for supporting eventing frameworks. By default we currently use the "Asynchronous Command Calls" (2nd) pattern via `Django Celery`_. As of February of 2025 both event-tracking and event-routing-backends can be configured to use the Kafka or Redis event bus implementations for sending real-time events as defined in :ref:`oep-52`. However, the requirements for that infrastructure are different from those captured in this OEP. This OEP requires support for standardized protocol(s) and configuration and access control (including PII considerations) for external consumers. Although it is theoretically possible for microservices to use this OEP's framework for inter-communication, it is not the intention. Events exchanged between microservices are within a firewall and can use non-standard schemas. diff --git a/oeps/architectural-decisions/oep-0026/caliper-realtime-events.rst b/oeps/architectural-decisions/oep-0026/caliper-realtime-events.rst index 5565b29bd..18ba5f398 100644 --- a/oeps/architectural-decisions/oep-0026/caliper-realtime-events.rst +++ b/oeps/architectural-decisions/oep-0026/caliper-realtime-events.rst @@ -144,66 +144,10 @@ We can provide context in events in the following way: Open edX events *************** -Currently, the Open edX system supports and maintains events that are sent to tracking logs, as described in `Tracking Log Events `__. +Currently, the Open edX system supports and maintains events that are sent to tracking logs, as described in `Tracking Log Events`_. Please see the `Open edX Caliper Events`_ document for a detailed view of the mapping between the above Open edX events and their equivalent Open edX Caliper formats. -Prioritized List of Events -========================== - -For this first iteration, we will focus primarily on the following events: - -- **Enrollment events** - - + `edx.course.enrollment.activated `_. - Whenever a learner enrolls in a course. - + `edx.course.enrollment.deactivated `_. - Whenever a learner unenrolls from a course. - -- **Problem interaction events** - - + `edx.grades.problem.submitted `_. - Whenever a learner submits any problem. - + `problem_check `_. - Whenever a learner's answer to a problem is checked. - + `showanswer `_. - Whenever a learner is shown the answer to a problem. - + `edx.problem.hint.demandhint_displayed `_. - Whenever a learner requests a hint to a problem. - -- **Video events** - - + `edx.video.loaded `_. - Whenever a learner loads a video. - + `edx.video.played `_. - Whenever a learner plays a video. - + `edx.video.stopped `_. - Whenever a learner stops a video. - + `edx.video.paused `_. - Whenever a learner pauses a video. - + `edx.video.position.changed `_. - Whenever a learner navigates to a different position in a video. - -- **Course navigation events** - - + `edx.ui.lms.sequence.outline.selected `_. - Whenever a learner navigates to a subsection in the course. - + `edx.ui.lms.sequence.next_selected `_. - Whenever a learner navigates to the next content in the course. - + `edx.ui.lms.sequence.previous_selected `_. - Whenever a learner navigates to the previous content in the course. - + `edx.ui.lms.sequence.tab_selected `_. - Whenever a learner navigates to another unit within a subsection. - + `edx.ui.lms.link_clicked `_. - Whenever a learner clicks on any link in the course. - -.. _Tracking Log Events: https://edx.readthedocs.io/projects/devdata/en/latest/internal_data_formats/tracking_logs/index.html - -Event Field Mapping -=================== - -Please see the `Open edX Caliper Events`_ document for a detailed view of the mapping between the above Open edX events and their equivalent Open edX Caliper formats. - - -.. _Open edX Caliper Events: https://github.com/openedx/event-routing-backends/blob/d4cb7c1e2c1b5c80f807206cf47eb358f388627c/docs/event-mapping/Caliper_mapping.rst +.. _Tracking Log Events: https://docs.openedx.org/en/latest/developers/references/internal_data_formats/index.html +.. _Open edX Caliper Events: https://github.com/openedx/event-routing-backends/blob/master/docs/event-mapping/Caliper_mapping.rst .. |ecosystem| image:: ./caliper_ecosystem.png :width: 6.5in diff --git a/oeps/architectural-decisions/oep-0026/xapi-realtime-events.rst b/oeps/architectural-decisions/oep-0026/xapi-realtime-events.rst index 84f2dfe67..0d44feda4 100644 --- a/oeps/architectural-decisions/oep-0026/xapi-realtime-events.rst +++ b/oeps/architectural-decisions/oep-0026/xapi-realtime-events.rst @@ -41,11 +41,7 @@ xAPI Actor ========== Although the **Actor** field can be either an *Agent* or a *Group*, we will primarily support only the *Agent* type, which is used for individuals performing an activity (`xAPI Verb`_ on an `xAPI Object`_). -An Actor can be identified using `Friend of a Friend (FOAF)`_ vocabulary with either: (1) `email address`_, (2) `hash of email address`_, (3) `OpenID URI`_, or (4) `account`_ with a *homepage*-scoped identifier. One of these is sent along with the Actor's "name". To be mindful of learner privacy, we will initially take a conservative approach and only send #3, with an Open edX anonymized unique identifier of the learner (:ref:`oep-26-user-id`). - -In the future, if certain external systems require `Personally Identifiable Information (PII)`_, like the learner's email address or name, then those may be conditionally sent with appropriate permissions. Adaptive engines, however, do not need PII. - -Initially, we will exclude the "name" field. However, if we find that xAPI JSON parsers assume this field always exists, then we can include the field but provide a non-PII value, such as a copy of the :ref:`oep-26-user-id`. +An Actor can be identified using `Friend of a Friend (FOAF)`_ vocabulary with either: (1) `email address`_, (2) `hash of email address`_, (3) `OpenID URI`_, or (4) `account`_ with a *homepage*-scoped identifier. One of these is sent along with the Actor's "name". As of February 2025 we send #4 as the default, with an Open edX anonymized unique identifier of the learner (:ref:`oep-26-user-id`) however the system can be configured to use #1 or #2 if necessary. Example ------- @@ -56,10 +52,13 @@ Here is an example of an **Actor** JSON value that we would generate: "actor": { "objectType": "Agent", - "openid": "https://openedx.org/users/user-v1:", - "name": "https://openedx.org/users/user-v1:" # only include this field if necessary + "account": { + "name": "", + "homePage": "https://openedx.org/" + } } + See `Deep Dive: Actor/Agent`_ for more information on xAPI Actors. .. _Friend of a Friend (FOAF): http://xmlns.com/foaf/spec/ @@ -175,62 +174,7 @@ See `Deep Dive: Result`_ for more information on xAPI Result. Open edX Events *************** -Currently, the Open edX system supports and maintains events that are sent to tracking logs, as described in `Tracking Log Events`_. - -Prioritized List of Events -========================== - -For this first iteration, we will focus primarily on the following events: - -- **Enrollment events** - - + `edx.course.enrollment.activated `_. - Whenever a learner enrolls in a course. - + `edx.course.enrollment.deactivated `_. - Whenever a learner unenrolls from a course. - -- **Problem interaction events** - - + `edx.grades.problem.submitted `_. - Whenever a learner submits any problem. - + `problem_check `_. - Whenever a learner's answer to a problem is checked. - + `showanswer `_. - Whenever a learner is shown the answer to a problem. - + `edx.problem.hint.demandhint_displayed `_. - Whenever a learner requests a hint to a problem. - -- **Video events** - - + `edx.video.loaded `_. - Whenever a learner loads a video. - + `edx.video.played `_. - Whenever a learner plays a video. - + `edx.video.stopped `_. - Whenever a learner stops a video. - + `edx.video.paused `_. - Whenever a learner pauses a video. - + `edx.video.position.changed `_. - Whenever a learner navigates to a different position in a video. - -- **Course navigation events** - - + `edx.ui.lms.sequence.outline.selected `_. - Whenever a learner navigates to a subsection in the course. - + `edx.ui.lms.sequence.next_selected `_. - Whenever a learner navigates to the next content in the course. - + `edx.ui.lms.sequence.previous_selected `_. - Whenever a learner navigates to the previous content in the course. - + `edx.ui.lms.sequence.tab_selected `_. - Whenever a learner navigates to another unit within a subsection. - + `edx.ui.lms.link_clicked `_. - Whenever a learner clicks on any link in the course. - -.. _Tracking Log Events: https://edx.readthedocs.io/projects/devdata/en/latest/internal_data_formats/tracking_logs/index.html - -Event Field Mapping -=================== - -Please see the `Open edx xAPI Events`_ document for a detailed view of the mapping between the above Open edX events and their equivalent Open edX xAPI formats. +Currently, the Open edX system supports and maintains events that are sent to tracking logs, as described in `Tracking Log Events`_. The authoritative source for the mapping between these events and their xAPI equivalents is the `event-routing-backends XAPI transform`_ documentation. -.. _Open edx xAPI Events: https://docs.google.com/spreadsheets/d/1oTClCxuUj1vCzytbmjDaHWFmcI6JZDqqJtZmYVwnOTA/view +.. _Tracking Log Events: https://docs.openedx.org/en/latest/developers/references/internal_data_formats/index.html +.. _event-routing-backends XAPI transform: https://github.com/openedx/event-routing-backends/blob/master/docs/event-mapping/xAPI_mapping.rst diff --git a/oeps/architectural-decisions/oep-0052-arch-event-bus-architecture.rst b/oeps/architectural-decisions/oep-0052-arch-event-bus-architecture.rst index 0ccd9cc25..8831613f5 100644 --- a/oeps/architectural-decisions/oep-0052-arch-event-bus-architecture.rst +++ b/oeps/architectural-decisions/oep-0052-arch-event-bus-architecture.rst @@ -1,3 +1,5 @@ +.. _oep-52: + OEP-52: Event Bus Architecture ############################## From 4ad0f68107d00f2c7cc86328881f955bdc69c883 Mon Sep 17 00:00:00 2001 From: Brian Mesick Date: Tue, 18 Feb 2025 11:03:16 -0500 Subject: [PATCH 2/2] docs: Add change history to OEP-26 and OEP-30 --- .../oep-0026-arch-realtime-events.rst | 101 ++++++++++++++++++ .../oep-0030-arch-pii-markup-and-auditing.rst | 52 +++++++++ 2 files changed, 153 insertions(+) diff --git a/oeps/architectural-decisions/oep-0026-arch-realtime-events.rst b/oeps/architectural-decisions/oep-0026-arch-realtime-events.rst index 16be5a8ae..cc771da76 100644 --- a/oeps/architectural-decisions/oep-0026-arch-realtime-events.rst +++ b/oeps/architectural-decisions/oep-0026-arch-realtime-events.rst @@ -273,3 +273,104 @@ Here are a list of current Open edX frameworks that are related to "eventing" bu Although it is possible for this OEP's real-time eventing framework to send events targeted to IoT and personal devices, those events will not be translated nor customized for each individual recipient, nor be adaptive to the individual's policies and time sensitivities. ACE would be a better alternative for those requirements. .. _Automated Communication Engine (ACE): https://edx-ace.readthedocs.io/en/latest/index.html + +Change History +************** + +2025-02-05 +========== + +* Mark as accepted +* Update to reflect current implementation +* `PR #678 `_ + +2023-11-06 +========== + +* Fixed various typos +* `PR #537 `_ + +2023-04-14 +========== + +* Replace tCRIL references with Axim references +* `PR #464 `_ + +2022-11-04 +========== + +* Change Ed Zarecor from arbiter to author, add Brian Mesick as Arbiter +* Clarify language and update to reflect current thoughts +* Remove references to Validator Processor +* Change implementation of an LRS from "Deferring implementation" to "Out of scope" +* `PR #405 `_ + +2022-09-27 +========== + +* Change arbiter from Brian Wilson to Ed Zarecor +* `PR #386 `_ + +2022-09-16 +========== + +* Change Github organization references from edx to openedx +* `PR #376 `_ + +2021-01-20 - 2022-08-02 +======================= + +* RsT style cleanup and minor content updates +* `PR #365 `_ +* `PR #248 `_ +* `PR #245 `_ +* `PR #206 `_ + +2021-04-09 +========== + +* Change http URLs to https +* `PR #200 `_ + +2021-01-20 +========== + +* Clarify when we should build on Django admin vs settings or a self-service portal +* `PR #156 `_ + +2020-12-07 +========== + +* Update example code +* `PR #175 `_ + +2020-09-20 - 2020-09-30 +======================= + +* Cleaning up links and references +* `PR #162 `_ +* `PR #163 `_ + +2020-06-04 +========== + +* Significant updates to Caliper documentation +* `PR #152 `_ + +2019-05-25 +========== + +* Update title to remove prefixing zeros for consistency +* `PR #115 `_ + +2019-05-15 +========== + +* Update anonymized user ID section to unique user id as part of adding OEP-32 +* `PR #103 `_ + +2019-01-19 +========== + +* Initial publication +* `PR #73 `_ diff --git a/oeps/architectural-decisions/oep-0030-arch-pii-markup-and-auditing.rst b/oeps/architectural-decisions/oep-0030-arch-pii-markup-and-auditing.rst index 96f5ef6d0..46ec3b55c 100644 --- a/oeps/architectural-decisions/oep-0030-arch-pii-markup-and-auditing.rst +++ b/oeps/architectural-decisions/oep-0030-arch-pii-markup-and-auditing.rst @@ -325,3 +325,55 @@ django-scrub-pii .. _Code Annotations: https://github.com/openedx/code-annotations + + +Change History +************** + +2025-02-04 +========== + +* Mark as accepted +* Update to reflect current implementation +* `PR #676 `_ + +2022-09-16 +========== + +* Change Github organization references from edx to openedx +* `PR #376 `_ + +2021-01-20 - 2022-08-02 +======================= + +* RsT style cleanup and minor content updates +* `PR #365 `_ +* `PR #248 `_ +* `PR #245 `_ +* `PR #206 `_ + +2020-02-19 +========== + +* Fix code blocks that weren't rendering, and typos +* `PR #134 `_ + +2019-01-30 +========== + +* Update to match reference implementation, add links to it +* Break our Organization tool into its own section +* Change double colons to single colons in annotation tokens +* `PR #101 `_ + +2018-11-06 +========== + +* Change status from Provisional to Draft +* `PR #85 `_ + +2018-10-09 +========== + +* Initial draft +* `PR #81 `_