Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TODO: Learner analytics impl #4248

Closed
wants to merge 47 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
01607f0
strings for learner analytics
Sarthak2601 Dec 29, 2021
e081f2f
platform parameter impl for learner analytics
Sarthak2601 Dec 29, 2021
fcdb50c
nit
Sarthak2601 Jan 2, 2022
d5b79e4
nit
Sarthak2601 Jan 2, 2022
8a1fd8b
event action enum update
Sarthak2601 Jan 2, 2022
c6f0658
addition of contexts
Sarthak2601 Jan 3, 2022
5b12c6d
nit
Sarthak2601 Jan 3, 2022
e7ec38c
Merge branch 'develop' into learner-analytics-strings
Sarthak2601 Jan 8, 2022
eb519de
admin control strings
Sarthak2601 Jan 19, 2022
d557a7c
device id correction
Sarthak2601 Jan 19, 2022
3049fb9
Merge branch 'develop' into learner-analytics-strings
Sarthak2601 Jan 19, 2022
30e8a85
Merge branch 'learner-analytics-strings' into learner-analytics-platf…
Sarthak2601 Jan 19, 2022
17cd55d
Merge branch 'learner-analytics-platform-parameters' into learner-ana…
Sarthak2601 Jan 19, 2022
564944f
exhaustive when fix.
Sarthak2601 Jan 20, 2022
271e08b
exhaustive when fix.
Sarthak2601 Jan 20, 2022
238aeda
todo formatting
Sarthak2601 Jan 20, 2022
b5d025c
collapsed contexts, added spacing, added comments
Sarthak2601 Jan 25, 2022
add6525
Merge branch 'develop' into learner-analytics-proto-impl
Sarthak2601 Jan 25, 2022
e02b9d9
event action removal + nits
Sarthak2601 Feb 2, 2022
369d2db
tests + dev options event logs fixes post event action removal
Sarthak2601 Feb 3, 2022
7b1971f
nits
Sarthak2601 Feb 3, 2022
1314060
removal of method for event action formatted string
Sarthak2601 Feb 3, 2022
073628a
Merge branch 'develop' into learner-analytics-proto-impl
Sarthak2601 Feb 3, 2022
05f4ddc
nits, null context changes.
Sarthak2601 Feb 6, 2022
3f16b82
nits
Sarthak2601 Feb 6, 2022
6ae9bb6
Merge branch 'develop' into learner-analytics-proto-impl
Sarthak2601 Feb 6, 2022
bad71c0
reserved fixes and help index fix
Sarthak2601 Feb 9, 2022
ac244f6
Merge branch 'develop' into learner-analytics-proto-impl
Sarthak2601 Feb 9, 2022
8440952
bazel imports
Sarthak2601 Feb 9, 2022
37d84a0
bazel build fixes
Sarthak2601 Feb 9, 2022
a1c05ea
test fixes
Sarthak2601 Feb 10, 2022
86df24e
nit
Sarthak2601 Feb 10, 2022
6aa8ebf
logging identifier controller, module + uuid wrapper, real impl
Sarthak2601 Feb 22, 2022
f370245
logging identifier controller tests, fake uuid, tests
Sarthak2601 Feb 22, 2022
117daa1
sync status manager + fake
Sarthak2601 Mar 2, 2022
6f00044
logging methods, test setup
Sarthak2601 Mar 3, 2022
686aa92
profile management, tests
Sarthak2601 Mar 4, 2022
b383ec4
sync status update.
Sarthak2601 Mar 9, 2022
7a2d9d0
lifecycle observer
Sarthak2601 Mar 9, 2022
323d96e
Merge branch 'develop' into learner-analytics-proto-impl
Sarthak2601 Mar 14, 2022
f43a073
Merge commit '323d96eca1c91acd4cb25fa843a99156e9eb8db2' of github.com…
BenHenning Mar 14, 2022
5c91087
Merge branch 'develop' into learner-analytics-impl
BenHenning Mar 14, 2022
a84e679
Post-merge fixes + Bazel support.
BenHenning Mar 14, 2022
864121d
Lots of reorganizing & changes.
BenHenning Mar 15, 2022
cebfc40
Lint fixes.
BenHenning Mar 15, 2022
7c36e21
Merge branch 'develop' into learner-analytics-impl
BenHenning Mar 15, 2022
978e9ba
Post-merge fix (proper merge of maven_install).
BenHenning Mar 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions app/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -751,6 +751,7 @@ kt_android_library(
"//domain/src/main/java/org/oppia/android/domain/oppialogger/loguploader:worker_module",
"//domain/src/main/java/org/oppia/android/domain/profile:profile_management_controller",
"//model:arguments_java_proto_lite",
"//domain/src/main/java/org/oppia/android/domain/oppialogger/analytics:prod_module",
"//app/src/main/java/org/oppia/android/app/testing/activity:test_activity",
"//third_party:androidx_databinding_databinding-adapters",
"//third_party:androidx_databinding_databinding-common",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ import org.oppia.android.domain.hintsandsolution.HintsAndSolutionDebugModule
import org.oppia.android.domain.onboarding.ExpirationMetaDataRetrieverModule
import org.oppia.android.domain.oppialogger.ApplicationStartupListener
import org.oppia.android.domain.oppialogger.LogStorageModule
import org.oppia.android.domain.oppialogger.LoggingIdentifierModule
import org.oppia.android.domain.oppialogger.analytics.ApplicationLifecycleModule
import org.oppia.android.domain.oppialogger.exceptions.UncaughtExceptionLoggerModule
import org.oppia.android.domain.oppialogger.loguploader.LogUploadWorkerModule
import org.oppia.android.domain.platformparameter.PlatformParameterModule
Expand All @@ -44,6 +46,7 @@ import org.oppia.android.util.caching.CachingModule
import org.oppia.android.util.gcsresource.GcsResourceModule
import org.oppia.android.util.locale.LocaleProdModule
import org.oppia.android.util.logging.LoggerModule
import org.oppia.android.util.logging.SyncStatusModule
import org.oppia.android.util.logging.firebase.DebugLogReportingModule
import org.oppia.android.util.logging.firebase.FirebaseLogUploaderModule
import org.oppia.android.util.networking.NetworkConnectionDebugUtilModule
Expand All @@ -52,6 +55,7 @@ import org.oppia.android.util.parser.html.HtmlParserEntityTypeModule
import org.oppia.android.util.parser.image.GlideImageLoaderModule
import org.oppia.android.util.parser.image.ImageParsingModule
import org.oppia.android.util.system.OppiaClockModule
import org.oppia.android.util.system.UserIdProdModule
import org.oppia.android.util.threading.DispatcherModule
import javax.inject.Provider
import javax.inject.Singleton
Expand Down Expand Up @@ -93,9 +97,10 @@ import javax.inject.Singleton
DeveloperOptionsModule::class, PlatformParameterSyncUpWorkerModule::class,
NetworkConnectionUtilDebugModule::class, NetworkConfigProdModule::class, AssetModule::class,
LocaleProdModule::class, ActivityRecreatorProdModule::class,
UserIdProdModule::class, ApplicationLifecycleModule::class,
// TODO(#59): Remove this module once we completely migrate to Bazel from Gradle as we can then
// directly exclude debug files from the build and thus won't be requiring this module.
NetworkConnectionDebugUtilModule::class
NetworkConnectionDebugUtilModule::class, LoggingIdentifierModule::class, SyncStatusModule::class
]
)
interface ApplicationComponent : ApplicationInjector {
Expand Down
3 changes: 3 additions & 0 deletions domain/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ kt_android_library(
"//domain/src/main/java/org/oppia/android/domain/oppialogger:startup_listener",
"//domain/src/main/java/org/oppia/android/domain/oppialogger/exceptions:controller",
"//domain/src/main/java/org/oppia/android/domain/oppialogger/loguploader:worker_factory",
"//domain/src/main/java/org/oppia/android/domain/profile:profile_management_controller",
"//domain/src/main/java/org/oppia/android/domain/state:state_deck",
"//domain/src/main/java/org/oppia/android/domain/state:state_graph",
"//domain/src/main/java/org/oppia/android/domain/state:state_list",
Expand Down Expand Up @@ -191,6 +192,8 @@ TEST_DEPS = [
"//domain/src/main/java/org/oppia/android/domain/testing/oppialogger/loguploader:fake_log_uploader",
"//testing",
"//testing/src/main/java/org/oppia/android/testing/data:data_provider_test_monitor",
"//testing/src/main/java/org/oppia/android/testing/logging:fake_user_id_generator",
"//testing/src/main/java/org/oppia/android/testing/logging:user_id_test_module",
"//testing/src/main/java/org/oppia/android/testing/network:network",
"//testing/src/main/java/org/oppia/android/testing/network:test_module",
"//testing/src/main/java/org/oppia/android/testing/platformparameter:test_module",
Expand Down
2 changes: 2 additions & 0 deletions domain/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ dependencies {
'androidx.appcompat:appcompat:1.0.2',
'androidx.exifinterface:exifinterface:1.0.0-rc01',
'androidx.lifecycle:lifecycle-livedata-ktx:2.2.0-alpha03',
'androidx.lifecycle:lifecycle-extensions:2.0.0',
'androidx.lifecycle:lifecycle-runtime-ktx:2.2.0-alpha03',
'androidx.work:work-runtime-ktx:2.4.0',
'com.google.dagger:dagger:2.24',
'com.google.firebase:firebase-analytics-ktx:17.5.0',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,29 @@ package org.oppia.android.domain.exploration

import androidx.lifecycle.LiveData
import androidx.lifecycle.MutableLiveData
import androidx.lifecycle.Transformations
import org.oppia.android.app.model.AnswerOutcome
import org.oppia.android.app.model.CheckpointState
import org.oppia.android.app.model.EphemeralState
import org.oppia.android.app.model.Exploration
import org.oppia.android.app.model.ExplorationCheckpoint
import org.oppia.android.app.model.HelpIndex
import org.oppia.android.app.model.Profile
import org.oppia.android.app.model.ProfileId
import org.oppia.android.app.model.UserAnswer
import org.oppia.android.domain.classify.AnswerClassificationController
import org.oppia.android.domain.exploration.lightweightcheckpointing.ExplorationCheckpointController
import org.oppia.android.domain.hintsandsolution.HintHandler
import org.oppia.android.domain.oppialogger.LoggingIdentifierController
import org.oppia.android.domain.oppialogger.OppiaLogger
import org.oppia.android.domain.oppialogger.exceptions.ExceptionsController
import org.oppia.android.domain.profile.ProfileManagementController
import org.oppia.android.domain.topic.StoryProgressController
import org.oppia.android.domain.translation.TranslationController
import org.oppia.android.util.data.AsyncDataSubscriptionManager
import org.oppia.android.util.data.AsyncResult
import org.oppia.android.util.data.DataProvider
import org.oppia.android.util.data.DataProviders.Companion.toLiveData
import org.oppia.android.util.data.DataProviders.Companion.transformAsync
import org.oppia.android.util.locale.OppiaLocale
import org.oppia.android.util.system.OppiaClock
Expand Down Expand Up @@ -50,7 +55,9 @@ class ExplorationProgressController @Inject constructor(
private val oppiaClock: OppiaClock,
private val oppiaLogger: OppiaLogger,
private val hintHandlerFactory: HintHandler.Factory,
private val translationController: TranslationController
private val translationController: TranslationController,
private val loggingIdentifierController: LoggingIdentifierController,
private val profileManagementController: ProfileManagementController
) : HintHandler.HintMonitor {
// TODO(#179): Add support for parameters.
// TODO(#3622): Update the internal locking of this controller to use something like an in-memory
Expand Down Expand Up @@ -100,6 +107,15 @@ class ExplorationProgressController @Inject constructor(

/** Indicates that the current exploration being played is now completed. */
internal fun finishExplorationAsync() {
oppiaLogger.createFinishExplorationContext(
oppiaLogger.createExplorationDetailsContext(
getSessionId() ?: "",
explorationProgress.currentExplorationId,
explorationProgress.currentExploration.version.toString(),
explorationProgress.stateDeck.getCurrentState().name,
oppiaLogger.createLearnerDetailsContext(getLearnerId() ?: "")
)
)
explorationProgressLock.withLock {
check(explorationProgress.playStage != ExplorationProgress.PlayStage.NOT_PLAYING) {
"Cannot finish playing an exploration that hasn't yet been started"
Expand Down Expand Up @@ -688,4 +704,41 @@ class ExplorationProgressController @Inject constructor(
lastPlayedTimestamp
)
}

private fun getSessionId(): String? {
return Transformations.map(
loggingIdentifierController.getSessionId().toLiveData(),
::processGetSessionIdResult
).value
}

private fun processGetSessionIdResult(sessionIdResult: AsyncResult<String>): String {
if (sessionIdResult.isFailure()) {
oppiaLogger.e(
"ExplorationProgressController",
"Failed to retrieve session id",
sessionIdResult.getErrorOrNull()!!
)
}
return sessionIdResult.getOrDefault("")
}

private fun getLearnerId(): String? {
// TODO: This isn't going to work since the live data won't be processed.
return Transformations.map(
profileManagementController.getProfile(explorationProgress.currentProfileId).toLiveData(),
::processGetProfileResult
).value?.learnerId
}

private fun processGetProfileResult(profileResult: AsyncResult<Profile>): Profile {
if (profileResult.isFailure()) {
oppiaLogger.e(
"ExplorationProgressController",
"Failed to retrieve profile",
profileResult.getErrorOrNull()!!
)
}
return profileResult.getOrDefault(Profile.getDefaultInstance())
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package org.oppia.android.domain.oppialogger

import javax.inject.Qualifier

/**
* Corresponds to an injectable application-level [Long] that provides a static seed that may be
* used for generating seeds that must be consistent for the lifetime of the application (but not
* necessarily across application instances).
*
* Tests may override the value corresponding to this qualifier in the Dagger graph to ensure
* deterministic behavior for corresponding random functions that depend on it.
*/
@Qualifier annotation class ApplicationIdSeed
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,25 @@ Package for providing logging support.
load("@dagger//:workspace_defs.bzl", "dagger_rules")
load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_android_library")

kt_android_library(
name = "application_id_seed",
srcs = [
"ApplicationIdSeed.kt",
],
visibility = ["//domain:__subpackages__"],
deps = [
"//third_party:javax_inject_javax_inject",
],
)

kt_android_library(
name = "oppia_logger",
srcs = [
"OppiaLogger.kt",
],
visibility = ["//domain:__subpackages__"],
deps = [
":logging_identifier_controller",
"//domain/src/main/java/org/oppia/android/domain/oppialogger/analytics:controller",
"//model:event_logger_java_proto_lite",
"//utility/src/main/java/org/oppia/android/util/logging:console_logger",
Expand All @@ -27,13 +39,36 @@ kt_android_library(
)

kt_android_library(
name = "storage_module",
name = "logging_identifier_controller",
srcs = [
"LoggingIdentifierController.kt",
],
visibility = ["//:oppia_api_visibility"],
deps = [
":application_id_seed",
"//domain/src/main/java/org/oppia/android/domain/util:extensions",
"//third_party:com_google_firebase_firebase-installations",
"//utility/src/main/java/org/oppia/android/util/data:async_data_subscription_manager",
"//utility/src/main/java/org/oppia/android/util/data:data_providers",
"//utility/src/main/java/org/oppia/android/util/locale:oppia_locale",
"//utility/src/main/java/org/oppia/android/util/system:user_id_generator",
],
)

kt_android_library(
name = "prod_module",
srcs = [
"LogStorageModule.kt",
"LoggingIdentifierModule.kt",
],
visibility = [
"//:oppia_prod_module_visibility",
"//domain/src/main/java/org/oppia/android/domain/oppialogger:__subpackages__",
],
visibility = ["//domain/src/main/java/org/oppia/android/domain/oppialogger:__subpackages__"],
deps = [
":application_id_seed",
":dagger",
"//utility/src/main/java/org/oppia/android/util/system:oppia_clock",
],
)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
package org.oppia.android.domain.oppialogger

import android.content.Context
import android.provider.Settings
import com.google.firebase.installations.FirebaseInstallations
import org.oppia.android.domain.util.getSecureString
import org.oppia.android.util.data.AsyncDataSubscriptionManager
import org.oppia.android.util.data.AsyncResult
import org.oppia.android.util.data.DataProvider
import org.oppia.android.util.data.DataProviders
import org.oppia.android.util.locale.OppiaLocale
import org.oppia.android.util.system.UserIdGenerator
import java.io.File
import java.security.MessageDigest
import java.util.Random
import java.util.concurrent.atomic.AtomicReference
import javax.inject.Inject
import javax.inject.Singleton

private const val SESSION_ID_DATA_PROVIDER_ID = "LoggingIdentifierController.session_id"
private const val FIREBASE_ID_DATA_PROVIDER_ID = "LoggingIdentifierController.firebase_id"

/** Controller that handles logging identifiers related operations. */
@Singleton
class LoggingIdentifierController @Inject constructor(
private val context: Context,
private val dataProviders: DataProviders,
private val asyncDataSubscriptionManager: AsyncDataSubscriptionManager,
@ApplicationIdSeed private val applicationIdSeed: Long,
private val userIdGenerator: UserIdGenerator,
private val machineLocale: OppiaLocale.MachineLocale
) {
private val learnerIdRandom by lazy { Random(applicationIdSeed) }

// TODO(#4249): Replace this with a StateFlow & the DataProvider with a StateFlow-converted one.
private var sessionId = AtomicReference(computeSessionId())

/**
* Creates and returns a unique identifier which will be used to identify the current learner.
*
* Each call to this function will return a unique learner ID, so it's up to the caller to ensure
* long-lived IDs are properly persisted.
*/
fun createLearnerId(): String = machineLocale.run {
"%08x".formatForMachines(learnerIdRandom.nextInt())
}

/**
* Returns a data provider that provides a one-time string which acts as a unique device ID for
* the current installation environment.
*
* Note that the returned ID is *not* guaranteed to be unique across devices, or even across app
* installations. It's an approximation for a hardware-based unique ID which is expected to be
* suitable for short-term user studies.
*/
fun getDeviceId(): DataProvider<String> = retrieveApproximatedUniqueDeviceId()

/**
* Returns an in-memory data provider pointing to a class variable of [sessionId].
*
* This ID is unique to each session. A session starts when an exploration begins.
*/
fun getSessionId(): DataProvider<String> {
return dataProviders.createInMemoryDataProvider(SESSION_ID_DATA_PROVIDER_ID) {
return@createInMemoryDataProvider sessionId.get()
}
}

/**
* Regenerates [sessionId] and notifies the data provider.
*
* The [sessionId] is generally updated when:
* 1. An exploration is started/resumed/started-over.
* 2. Inactivity duration exceeds 30 mins.
*/
fun updateSessionId() {
sessionId.set(computeSessionId())
asyncDataSubscriptionManager.notifyChangeAsync(SESSION_ID_DATA_PROVIDER_ID)
}

private fun computeSessionId(): String = userIdGenerator.generateRandomUserId()

private fun retrieveApproximatedUniqueDeviceId(): DataProvider<String> {
// TODO(#4249): Replace this with a StateFlow, too, for simplicity.
// Per https://developer.android.com/training/articles/user-data-ids and
// https://stackoverflow.com/a/2785493/3689782 there's no reliable way to compute a device ID.
// The following is an unreliable approximation that's more or less tied to the installation of
// the app, though *an* ID is at least guaranteed to be returned.
val fidResult = AtomicReference(AsyncResult.pending<String>())
FirebaseInstallations.getInstance().id.addOnCompleteListener { task ->
val fid = if (task.isSuccessful) task.result else null
val deviceId = fid ?: retrieveSecureAndroidId() ?: computeInstallTimeBasedId()
val hashedId = machineLocale.run {
MessageDigest.getInstance("SHA-1")
.digest(deviceId.toByteArray())
.joinToString("") { "%02x".formatForMachines(it) }
.substring(startIndex = 0, endIndex = 12)
}
fidResult.set(AsyncResult.success(hashedId))
asyncDataSubscriptionManager.notifyChangeAsync(FIREBASE_ID_DATA_PROVIDER_ID)
}
return dataProviders.createInMemoryDataProviderAsync(FIREBASE_ID_DATA_PROVIDER_ID) {
fidResult.get()
}
}

private fun retrieveSecureAndroidId(): String? =
context.getSecureString(Settings.Secure.ANDROID_ID)

private fun computeInstallTimeBasedId(): String {
// Use the package's install time as a (poor) proxy for a device ID since it's at least
// guaranteed to be present.
val appInfo = context.packageManager.getApplicationInfo("org.oppia.android", /* flags= */ 0)
return machineLocale.run { "%02x".formatForMachines(File(appInfo.sourceDir).lastModified()) }
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package org.oppia.android.domain.oppialogger

import dagger.Module
import dagger.Provides
import org.oppia.android.util.system.OppiaClock

/** Provider to return any constants required during operations on logging identifiers. */
@Module
class LoggingIdentifierModule {
@Provides
@ApplicationIdSeed
fun provideApplicationIdSeed(oppiaClock: OppiaClock): Long = oppiaClock.getCurrentTimeMs()
}
Loading