Package 'EIEvent'

Evidence Identification Event Processing Engine

Description

Extracts observables from a sequence of events. Uses a prolog-like rule language to do the extraction, written in JSON.

Details

The DESCRIPTION file:

Package:	EIEvent
Version:	0.6-2
Date:	2022/07/06
Title:	Evidence Identification Event Processing Engine
Authors@R:	person(given = "Russell", family = "Almond", role = c("aut", "cre"), email = "[email protected]", comment = c(ORCID = "0000-0002-8876-9337"))
Author:	Russell Almond
Maintainer:	Russell Almond <[email protected]>
Depends:	R (>= 3.0), methods, utils, Proc4 (>= 0.4), mongo
Imports:	jsonlite, mongolite, futile.logger, R.utils
Suggests:	knitr, rmarkdown, tidyr
Description:	Extracts observables from a sequence of events. Uses a prolog-like rule language to do the extraction, written in JSON.
License:	Artistic-2.0
URL:	https://pluto.coe.fsu.edu/Proc4
Collate:	AAGenerics.R Contexts.R Events.R Status.R Condition.R Predicates.R RuleTables.R testRules.R EIEngine.R Runners.R
VignetteBuilder:	knitr
Repository:	https://ralmond.r-universe.dev
RemoteUrl:	https://github.com/ralmond/EIEvent
RemoteRef:	HEAD
RemoteSha:	8d302ed7510c5ff280033253b881efdbdb20987e

Index of help topics:

Conditions              Conditional query operators for Rules.
Context                 Constructor for the Context object
Context-class           Class '"Context"'
ContextSet-class        Class '"ContextSet"'
EIEngine                Creator for the EIEngine class.
EIEngine-class          Class '"EIEngine"'
EIEvent-package         Evidence Identification Event Processing Engine
EITest-class            Class '"EITest"'
Event                   Event object constructor
Event-class             Class '"Event"'
Predicates              Functions that modify state when rule is
                        triggered.
Rule                    Constructor for EIEvent Rule Objects
Rule-class              Class '"Rule"'
RuleTable-class         Class '"RuleTable"'
RuleTest                Constructor for EITest or Rule Test.
RuleTest-class          Class '"RuleTest"'
Status                  Status (state) object constructor
Status-class            Class '"Status"'
TestSet-class           Class '"TestSet"'
Timer                   Constructor for Timer objects.
Timer-class             Class '"Timer"'
UserRecordSet-class     Class '"UserRecordSet"'
applicableContexts      Finds context sets to which a given context
                        belongs.
asif.difftime           More flexible constructor for creating difftime
                        objects.
buildMessages           These functions build messages for Trigger
                        Rules.
checkCondition          Checks to see if a condition in a EIEvent Rule
                        is true.
cid                     Accessor functions for context objects.
doLoad                  Construct an Engine and Load the Rules.
doRunrun                Runs the EIEvent engine using the supplied
                        configuration.
doc                     Meta-data accessors for Rules and Contexts.
executePredicate        Executes the predicate of an EIEvent Rule.
flag                    Accessor functions for context objects.
getJS                   Gets a field from an object in Javascript
                        notation.
handleEvent             Does the processing for a new event.
loadContexts            Loads a set of contexts from a matrix
                        description
loadRulesFromList       Functions for loading rules into database.
mainLoop                Runs the 'EIEngine' as a server.
matchContext            Find or replace contexts in a context set
queryResult             Accessor Functions for RuleTest objects.
removeJS                Removes a field from a state object.
ruleType                Accessors for Rule objects.
runRule                 Runs a specific rule in a particular
                        application.
runStatusRules          Runs all of the appropriate rules of the given
                        type.
runTest                 Runs a test case with a given set of rules
setJS                   Sets a field in a status object in Javascript
                        notation.
setTimer                Manipulates a Timer inside of a Status
start                   Functions for manipulating timer objects.
testRule                Functions for testing rule queries.
verb                    Accessor for verb and object field of events
                        and rules.

The package runs a EIEngine which is a server process for processing events according the the EI-Event rules stored in a database.

Configuration

The “quick start” document https://pluto.coe.fsu.edu/Proc4/EIQuickStart.pdf describes most of the configuration steps. The directory file.path(help(library="EIEvent")$Path,"conf") contains a number of files to assist with configuration.

The process assumes that Mongo (https://www.mongodb.com/) is installed on the system (or that it is available via a network). The files setupMongo.js and setupUsers.js (in the conf directory) are to be run in the mongo shell. In particular, the setupMongo.js sets up the indexes. Running these scripts is option, but recommended. See also the configuration instructions in the Proc4 package.

It is also strongly recommended to set up a directory for configuration scripts for the packages in the Proc4 family. The recommended location on Unix systems is /usr/local/share/Proc4. The files EIEvent, EIEvent.R, EIini.R, and EILoader.R should be copied to that directory (or to a bin subdirectory), and edited for local prferences. In particular, the EIini.R file needs to be updated with local details about the database and passwords, as well as directories for configuration files. The other files need to be updated to point to the EIini.R file.

The EILoader.R script loads a scoring model into the engine (see the following section). The EIEvent.R script runs the scoring engine (see Launching and termination, below). The shell script EIEvent runs the EIEvent.R script as a server process.

Rule Sets and Context Sets

In many respects, the EIEngine is an interpreter for a rule-based programming langugage. The document Rules of Evidence (https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf) describes that language specification. Programs are stored in the database. So there are two steps: loading the rules and contexts (described here), and executing the program on the event queue (described below).

The engine processes a queue of Event objects (see the next section) using the following steps:

1. When the first event for a user arrives, that user is defined an initial Status based on the default status for that application.

2. The applicableContexts for the current status is determined by consulting the ContextSet associated with the application. The idea is that a specific context (e.g., a game level) belongs to several larger context sets (e.g., the set of levels with a comon solution strategy).

3. The RuleTable is consulted to find the set of rules applicable to the current event and context.

4. The Conditions of the rules are run. If the condition is true, the the Predicates of the rules are run as well.

5. If running the rules updates the Status, then the updated status is saved.

6. If a trigger rule is run and P4Message is built (!send), then the message is sent to the ListenerSet.

7. The event is marked as processed, and then loop goes on to process the next event.

The program consists of three parts:

RuleTable

A set of Rule objects which are all stored in the database.

ContextSet

A set of context descriptions that play a role in determining when rules are applicable.

A defaul Status

The initial status for the application.

The script EILoader.R illustrates the necessary steps. Note that doLoad provides a more sophisticated interface to those steps.

Assuming the EIEngine is in an object called eng the following steps load the details.

1. Load in the rules from a JSON file: ruleList <- lapply( fromJSON( filename , FALSE), parseRule). The function parseRule creates a rule object from the output of fromJSON.

2. Clear the old rules using eng$clearAllRules().

3. Load the new rules using eng$loadRules(ruleList).

4. Load the context set from a datafile: conMat <- read.csv(filename).

5. Create the initial context: initCon <- data.frame(CID="*INITIAL*", Name="*INITIAL*", Number=0).

6. Clear Old contexts: eng$clearContexts().

7. Load context mattrix: eng$addContexts(conMat).

8. Load initial context: eng$addContexts(initCon).

9. Clear old records including default: eng$clearStatusRecords(TRUE).

10. Create a new blank status: defaultRec <- eng$newUser("*DEFAULT*").

11. Initialize fields on that record:

    • Flag: flag(defaultRec, name ) <- value.

    • Observable: obs(defaultRec, name ) <- value.

    • Timers: timer(defaultRec, name ) <- Timer( name).

12. Save it back to the database: defaultRec <- eng$saveStatus(defaultRec).

Events and the Event Queue

Objects of class Event are stored in a database collection (by default, the Events collection in the EIRecords database). Each event has a processed flag and the timestamp ensure that at any time the system can find the oldest unprocessed event. When run in server mode, new events can be added to the collection and the EIEngine will process them as they are added.

A single cycle of the mainLoop consists of the following steps.

1. Fetch the next event with eve <- eng$fetchNextEvent().

2. Process the event with out <- handleEvent(eng,eve).

3. If an error was generated (out is of class try-error), then save the error in the database using eng$setError(eve, out).

4. Mark the event as processed using eng$setProcessed(eve).

Events can be added to the system using the mongoimport external function. To call this from R, use system2("mongoimport", sprintf('-d %s -c Events --jsonArray', eng$dbname), stdin=eventFile)

Listeners and Messages

The output of the EI process is through the Listener objects. Trigger rules generate objects of P4Message (see !send) which are sent to the ListenerSet associated with the EIEngine. When a trigger rule fires, the !send predicate creates a message which is then sent to the ListenerSet though the notifyListeners method. This, in turn, calls the receiveMessage method on each of the listeners.

One of the most common actions in response to a listener firing is to insert or update a record in another database. This is the mechanism by which the EI process can send its output into the input queue for the EA process.

Launching and termination

Because the program is already stored in the database, launching the EIEngine as a server process takes three steps:

1. Create an instance of EIEngine (eng).

2. Create set the active flag for the process to true by calling, eng$activate().

3. Run the mainLoop process. This will run until the active flag is cleared.

The script EIEvent.R runs through these steps. It calls the EIini.R script to get details of configuration, then creates the engine and runs the main loop. The bash shell script EIEvent runs the EIEvent.R script in a server process (R --slave).

When run from the command line, EIEvent takes four arguments. (These should be specified as name=value). They are as follows:

app (Required)

This is the name of the application to be run. It is usually a URL-like string such as: ecd://epls.coe.fsu.edu/P4test

.

level (Default INFO)

This is the default logging level (see Logging below).

clean (default FALSE)

If true, then old statuses and messages will be cleaned out before starting. Otherwise, the current run will be a continuation of the old run.

evidence (Optional, filename)

If this argument is supplied, then the contents of the file will be read into the event queue before processing. The value of eng$processN will be set to the number of events, so the function will stop after the events are processed.

The arguments are only relevant when the process is run as a server. The EIEvent.R script can also be run line-by-line in an R developement environment (e.g., R Studio), in which case the arguments are replaced by constants set at the beginning of the script.

Unless an event file is given, the script will run an infinite loop. On a *nix system, this usually means that it should run as a background process, e.g., the call should be nohup EIEvent args &. This requires a graceful way to close the process down. This is done by setting the active field for the record corresponding to the application to false in the AuthorizedApps collection of the Proc4 database. A mongo shell call to do this is: db.AuthorizedApps.update({app:{"$regex":"appname"}}, {"$set":{active:false}});, where “appname” can be any string that uniquely identifies the application.

If the event file is given, it should be a JSON file will a collection of Event objects. The events will be loaded into the database and processed. The script will stop when the events are done processing. Often it is more convenient to run this in the interactive mode as R will still be active after the completion, so that the results can be inspected.

Logging and Error Handling

Logging is handled using the flog.logger framework. This provides a large number of tools for specifying the details of the logging. The EIEngine executes the rules in the withFlogging environment. This means that the default behavior for rules which generate errors is to log the error and move to the next rule. The error message is also added to the event in the event database (markAsError).

The amount of logging done, particularly the amount of detail supplied, is controled by the flog.threshold function. The amount of detail provided at various levels is as follows:

TRACE

• The results of checkCondition are reported.

• The specific rules found from each query are reported.

• A message is logged as each rule is run.

DEBUG

• When an error occurs information about the state, event and rule where the error occurred as well a stack trace are logged.

• Each event is logged as it is processed.

• Rule searches are logged and the number of rules reported.

• A message is logged when each phase starts.

• A message is logged when the context changes.

INFO

Minimal information about which events are being processed is logged.

WARN and above

If an error occurs the error is logged along with context information.

When running the EIEngine as a server, generally the logging should be done as a file. This can be done by running: flog.appender(appender.file(logfile)). When running in interactive mode, it may be useful to have log messages sent to both the console and the log file. The command for this is: flog.appender(appender.file(logfile)).

Concurrency

Note that several applications can share the same database. The app field of the EIEngine is part of the key for all of the collections. Generally, database commands will use the app field as part of the queries, so two engines with different applications ids will have different rule sets, context sets, status sets and event queues.

Because of this separation, it is possible to run multiple EIEngine processes focused on different applications. This gives a limited form of parallel processing.

In theory, processing for each uid could be handled separately. This has not been coded in this version. Future versions may use a language other than R which supports richer tools for multi-threaded computing.

Acknowledgements

Work on the Proc4, EIEvent and EABN packages has been supported by the National Science foundation grants DIP: Game-based Assessment and Support of STEM-related Competencies (#1628937, Val Shute, PI) and Mathematical Learning via Architectual Design and Modeling Using E-Rebuild. (#1720533, Fengfeng Ke, PI).

The EIEvent package developement was led by Russell Almond (Co-PI).

Author(s)

Russell Almond

Maintainer: Russell Almond <[email protected]>

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

There is a “quick start” document which describes how to set up the engine. This is available at https://pluto.coe.fsu.edu/Proc4/EIQuickStart.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

See Also

The package Proc4 provides low level support for the database connectivity, which is handled through the mongo function. Logging is supplied by the flog.logger system.

Examples

## Not run: 

## Initialize the Engine
app <- "ecd://epls.coe.fsu.edu/P4test"
loglevel <- "DEBUG"
cleanFirst <- TRUE
eventFile <- "/home/ralmond/Projects/EvidenceID/c081c3.core.json"

## These configuration steps are generally done in EIini.R
## These are application generic parameters
EIeng.common <- list(host="localhost",username="EI",password="secret",
                     dbname="EIRecords",P4dbname="Proc4",waittime=.25)

appstem <- basename(app)
## These are for application specific parameters
EIeng.params <- list(app=app)

## File for loading configuraitons
config.dir <- "/home/ralmond/ownCloud/Projects/NSFCyberlearning/EvidenceID"

## Location of logfile
logfile <- file.path("/usr/local/share/Proc4/logs",
                     paste("EI_",appstem,"0.log",sep=""))

EI.listenerSpecs <-
  list("InjectionListener"=list(sender=paste("EI",appstem,sep="_"),
            dbname="EARecords",dburi="mongodb://localhost",
            colname="EvidenceSets",messSet="New Observables"))

## Setup logging
flog.appender(appender.file(logfile))
flog.threshold(loglevel)

## Setup Listeners
listeners <- lapply(names(EI.listenerSpecs),
                    function (ll) do.call(ll,EI.listenerSpecs[[ll]]))
names(listeners) <- names(EI.listenerSpecs)
if (interactive()) {
  cl <- new("CaptureListener")
  listeners <- c(listeners,cl=cl)
}

## Load the Rules
ruleList <- lapply(jsonlite::fromJSON("rulefile.json", FALSE), parseRule)
eng$clearAllRules()
eng$loadRules(ruleList)

## Load Contexts
conMat <- read.csv("contextTable.csv")
initCon <- data.frame(CID="*INITIAL*", Name="*INITIAL*", Number=0)
eng$clearContexts()
eng$addContexts(conMat)
eng$addContexts(initCon)

## Setup default status.
eng$clearStatusRecords(TRUE) ## Clears default record
defaultRec <- eng$newUser("*DEFAULT*")
obs(defaultRec,"bankBalance") <- 0
defaultRec <- eng$saveStatus(defaultRec)

## Clean out old records from the database.
if (cleanFirst) {
  eng$eventdb()$remove(buildJQuery(app=app(eng)))
  eng$userRecords$clearAll(FALSE)   #Don't clear default
  eng$listenerSet$messdb()$remove(buildJQuery(app=app(eng)))
  for (lis in eng$listenerSet$listeners) {
    if (is(lis,"UpdateListener") || is(lis,"InjectionListener"))
      lis$messdb()$remove(buildJQuery(app=app(eng)))
  }
}
## Process Event file if supplied
if (!is.null(eventFile)) {
  system2("mongoimport",
          sprintf('-d %s -c Events --jsonArray', eng$dbname),
          stdin=eventFile)
  ## Count the number of unprocessed events
  NN <- eng$eventdb()$count(buildJQuery(app=app(eng),processed=FALSE))
  ## This can be set to a different number to process only a subset of events.
  eng$processN <- NN
}

## Activate engine (if not already activated.)
eng$activate()
mainLoop(eng) ## This will not terminate unless processN was set to a
              ## finite value.

## This shows the details of the last message.  If the test script is
## set up properly, this should be the observables.
if (!is.null(eventFile) && TRUE) {
  details(cl$lastMessage())
}


## End(Not run)

---

Finds context sets to which a given context belongs.

Description

A Context may belong to one or more context sets. A Rule may operate on a specific context, a context set, or all contexts (the special context set "ALL"). The function applicableContexts returns a list of all potential rule context matches. The function belongsTo maintains the implicit contexts.

Usage

applicableContexts(c)
belongsTo(c)
## S4 method for signature 'Context'
belongsTo(c)
## S4 method for signature 'ANY'
belongsTo(c)
belongsTo(c) <- value
## S4 replacement method for signature 'Context'
belongsTo(c) <- value

Arguments

c	An object of class Context
value	A character vector containing the names of the context sets this context belongs to.

Value

The function belongsTo returns a (possibly empty) vector of context set names this context belongs to. (The ANY method always returns the empty list.) The function applicableContexts returns the same vector with the addition of the current context ID and the special context "ALL".

Note

It would seem a natural extension of this system to put the contexts into an acyclic directed graph with the belongsTo function providing the link. This was determined to be more trouble than it was worth for the current application, so the entire hierarchy must be represented within the belongsTo field of each context.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the context system: https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

See Also

Context describes the context object.

Examples

ct <- Context("Level1","First Tutorial",1,
              belongsTo=c("tutorialLevels","easyLevels"),
              doc="First Introductory Level",
              app="ecd://epls.coe.fsu.edu/EITest")
stopifnot(setequal(belongsTo(ct),c("tutorialLevels","easyLevels")))
stopifnot(setequal(applicableContexts(ct),
          c("Level1","tutorialLevels","easyLevels","ALL","ANY")))

belongsTo(ct) <- "tutorialLevels"
stopifnot(setequal(belongsTo(ct),c("tutorialLevels")))

---

More flexible constructor for creating difftime objects.

Description

The function asif.difftime is a constructor for difftime objects from a list with components named “secs”, “mins”, “hours”, “days”, or “weeks”. These get added together.

The function is.difftime is the test function missing from the base package.

Usage

asif.difftime(e2)
is.difftime(x)

Arguments

e2	This should be a list of numeric values with named components with names selected from: c("secs", "mins", "hours", "days", "weeks").
x	An object to be tested for its difftime status.

Value

If the argument to asif.difftime is a list with the appropriate names, then an object of class difftime is returned. Otherwise, the argument is returned.

The function is.difftime returns a logical value indicating whether or not its argument is of class difftime

Author(s)

Russell Almond

See Also

difftime4

Also, asif.difftime is used in variuos predicates.

Examples

dt <- asif.difftime(list(mins=1,secs=5))
stopifnot (is.difftime(dt),
    all.equal(dt,as.difftime(65,units="secs")))

---

These functions build messages for Trigger Rules.

Description

A trigger rule has a special predicate function !send which build a P4Message object to send to listeners of the EIEngine (through the notifyListeners method.

Usage

buildMessages(predicate, state, event)
"!send"(predicate, state, event)
"!send1"(predicate, state, event)
"!send2"(predicate, state, event)

Arguments

predicate	One element of the predicate, or in the case of buildMessages the complete predicate.
state	An object of class Status giving the current state of the system. Note that often context and oldContext will be different.
event	An object of class Event that gives the event that triggered the message.

Details

The function !send builds a P4Message which is then sent to the registered listeners of the EIEngine. The default message is “"Observables Available"”, which is what the Evidence Accumulation process is listening for. The default content is all of the observables.

The rule can override certian defaults of the message by setting appropriate fields of the object that is the value of the !send element in the predicate.

context

The context for the message. Default is oldContext(event). Value can be a direct value of a field name (starting with “event.data.” or “state.”).

mess

The text (subject) of the message. Default is “Observables Available”. Value can be a direct value of a field reference (starting with “event.data.” or “state.”).

data

The fields to be added in the details section of the message. If omitted, all of the observables are sent (with their default names). If supplied this should either be an named character vector or named list. The names are used as the names of the details portion of the message, and the values are field references (starting with “event.data.” or “state.”) for where to find the values.

The function buildMessages runs through multiple elements of the predicate and builds multiple messages. The additional predicates !send1 and !send2 are there to allow for additional messages. This will also work with the name of an arbitrary R function which takes (predicate, state, event) as an argument list and returns a message.

Value

The function !send returns an object of class P4Message with the following fields:

app	The application, value is app(event).
uid	The user ID, value is uid(event).
context	The context (task) for this message, default is the value of oldContext(state).
mess	The message header. The default value is “Observables Available.”
sender	The value is “EIEvent”.
timestamp	The timestamp for the message. The value is timestamp(event).
details	The data that should be sent with the message. The default is all of the observables of the Status argument.

Note

JSON parsers are not happy with multiple fields with the same name, so the !send1 and !send2 operators are provided to get around this problem.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

See Also

Rule describes the rule object and Conditions describes the Conditions, and Predicates the predicates in general.

The functions testRule and testRuleScript can be used to test that rule conditions and predicates function properly together.

Other classes in the EIEvent system: EIEngine, Context, Status, Event, RuleTable.

The P4Message class is from the Proc4-package. The notifyListeners method describes the Proc4 message passing system.

Examples

st9 <- Status(uid="Test0",context="SpiderWeb",
             timestamp=as.POSIXct("2018-09-25 12:13:30 EDT"),
             observables=list(agentsUsed=list("Pendulum"),
                              lastAgent="Pendulum",
                              badge="gold"))


evnt10 <- Event(uid="Test0",
               verb="satisfied",object="game level",
               context="SpiderWeb",
               timestamp=as.POSIXct("2018-09-25 12:13:30 EDT"),
               details= list("badge"="gold"))

mess0 <- buildMessages(list("!send"=list()),st9,evnt10)[[1]]
stopifnot(mess(mess0)=="Observables Available",
          length(details(mess0)) == 3L)
mess1 <- buildMessages(list("!send1"=list()),st9,evnt10)[[1]]
messb <- buildMessages(list("!send"=list(mess="Badge",
                 data=c("badge"="state.observables.badge"))),
         st9,evnt10)[[1]]
stopifnot(mess(messb)=="Badge",length(details(messb))==1L)

---

Checks to see if a condition in a EIEvent Rule is true.

Description

An Rule object contains a list of Conditions. The name of each condition is the name of a field of the Event or Status object. For example, "event.data.trophy"=list("?in"=c("gold","silver")) would test if the trophy field was set to “gold” or “silver”. The function checkCondition returns true if all of the conditions are satisfied and false if any one of them is not satisfied.

Usage

checkCondition(conditions, state, event)

Arguments

conditions	A named list of conditions: see details.
state	An object of class Status to be checked.
event	An object of class Event to be checked.

Details

The condition of a Rule is a list of queries. Each query has the following form:

field=list(?op=arg,...)

Here, field is an identifier of a field in the Status or Event object being tested. This is in the dot notation (see getJS). The query operator, ?op is one of the tests described in Conditions. The arg is a value or field reference that the field will be tested against. In other words, the query is effectively of the form field ?op arg. The ... represents additional ?op–arg pairs to be tested.

The arg can be a literal value (either scalar or vector) or a reference to another field in the Status or Event object using the dot notation.

In general, a rule contains a list of queries. A rule is satisfied only if all of its queries are satisfied: the function checkCondition checks if the rule is satisfied.

Finally, one special query syntax allows for expansion. If the field is replaced with "?where", that is the query has the syntax "?where"=funname, then the named R is called. This should be a function of two arguments, the status and the event, which returns a logical value. The condition is satisfied if the funciton returns true.

See Conditions for more details.

Value

Returns a logical value: TRUE if all conditions are satisfied, false otherwise.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

MongoDB, Inc. (2018). The MongoDB 4.0 Manual. https://docs.mongodb.com/manual/.

See Also

Rule describes the rule object and Conditions describes the conditions. Predicates describes the predicate part of the rule, and executePredicate executes the predicate (when the condition is satisfied).

The functions testQuery and testQueryScript can be used to test that rule conditions function properly.

Other classes in the EIEvent system: EIEngine, Context, Status, Event, RuleTable.

Examples

st <- Status("Phred","Level 1",timerNames=character(),
   flags=list(lastagent="lever",noobj=7,noagents=0),
   observables=list(),
   timestamp=as.POSIXct("2018-12-21 00:01"))


ev <- Event("Phred","test","message",
      timestamp=as.POSIXct("2018-12-21 00:01:01"),
      details=list(agent="lever",newobj=2))


stopifnot(
checkCondition(list(event.data.agent=list("?eq"="lever")),   
               st,ev)==TRUE,             #Agent was lever.
checkCondition(list(event.data.agent="ramp"),               
               st,ev)==FALSE,            #Agent abbreviated form.
checkCondition(list(event.data.agent="state.flags.lastagent"), 
               st,ev)==TRUE,             #Same agent used.
checkCondition(list(state.flags.noobj=list("?lt"=10,"?gte"=5)), 
               st,ev)==TRUE,             #Between 5 and 10 objects.
checkCondition(list(event.data.agent=list("?in"=c("pendulum","springboard"))),
               st,ev)==FALSE,             #Between 5 and 10 objects.
#Abbreviated form (note lack of names)
checkCondition(list(event.data.agent=c("lever","springboard")),
               st,ev)==TRUE,
checkCondition(list(state.flags.lastagent=list("?isna"=TRUE)),
               st,ev)==FALSE,
checkCondition(list(state.flags.noagents=
                    list("?and"=list("?isna"=FALSE,"?gt"=0))),
               st,ev)==FALSE
) 

agplusobj <- function (state,event) {
  return (getJS("state.flags.noobj",state,event) +
          getJS("event.data.newobj",state,event) < 10)
}

stopifnot(checkCondition(list("?where"="agplusobj"),st,ev))

---

Accessor functions for context objects.

Description

These functions access the corresponding fields of the Context class.

Usage

cid(c)
## S4 method for signature 'Context'
cid(c)
number(c)
## S4 method for signature 'Context'
number(c)
number(c) <- value
## S4 replacement method for signature 'Context'
number(c) <- value
## S4 method for signature 'Context'
app(x)

Arguments

c	A Context object.
x	A Context object.
value	An integer giving the new context number.

Value

The function cid returns a unique string identifier for the context. The function number returns a unique integer identifier. The function app returns the application identifier. The cid and number should be unique within the app.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the JSON layout of the Status/State objects. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

Betts, B, and Smith, R. (2018). The Leraning Technology Manager's Guid to xAPI, Second Edition. HT2Labs Research Report: https://www.ht2labs.com/resources/the-learning-technology-managers-guide-to-the-xapi/#gf_26.

HT2Labs (2018). Learning Locker Documentation. https://docs.learninglocker.net/welcome/.

See Also

Context describes the context object. The function applicableContexts describes the context matching logic.

Examples

ct <- Context("Level1","First Tutorial",1,
              belongsTo=c("tutorialLevels","easyLevels"),
              doc="First Introductory Level",
              app="ecd://epls.coe.fsu.edu/EITest")

stopifnot(cid(ct)=="Level1",basename(app(ct))=="EITest",
          number(ct)==1L)

number(ct) <- 0
stopifnot(number(ct)==0L)

---

Conditional query operators for Rules.

Description

These are the conditional operators for a Rule. The check that the target value meets the specified condition (cond). These are called by the function checkCondition which checks that the condition is correct.

Usage

"?eq"(arg, target, state, event)
"?ne"(arg, target, state, event)
"?gt"(arg, target, state, event)
"?gte"(arg, target, state, event)
"?lt"(arg, target, state, event)
"?lte"(arg, target, state, event)
"?in"(arg, target, state, event)
"?nin"(arg, target, state, event)
"?exists"(arg, target, state, event)
"?isnull"(arg, target, state, event)
"?isna"(arg, target, state, event)
"?regexp"(arg, target, state, event)
"?any"(arg, target, state, event)
"?all"(arg, target, state, event)
"?not"(arg, target, state, event)
"?and"(arg, target, state, event)
"?or"(arg, target, state, event)

Arguments

arg	This is the value of the condition clause (the argument) of the query.
target	This is the value of the current state of the referenced field in the query.
state	This is a Status object used to resovle dot notation references.
event	This is a Event object used to resovle dot notation references.

Details

The condition of a Rule is a list of queries. Each query has the following form:

field=list(?op=arg,...)

Here, field is an identifier of a field in the Status or Event object being tested. This is in the dot notation (see getJS). The query operator, ?op is one of the tests described in the section ‘Condition Operators’. The arg is a value or field reference that the field will be tested against. In other words, the query is effectively of the form field ?op arg. The ... represents additional ?op–arg pairs to be tested.

The arg can be a literal value (either scalar or vector) or a reference to another field in the Status or Event object using the dot notation.

In general, a rule contains a list of queries. A rule is satisfied only if all of its queries are satisfied (essentially joining the queries with a logical-and). At the present time, the only way to get a logical-or is to use multiple rules.

Finally, one special query syntax allows for expansion. If the field is replaced with "?where", that is the query has the syntax "?where"=funname, then the named R is called. This should be a function of two arguments, the status and the event, which returns a logical value. The condition is satisfied if the funciton returns true.

Value

The condition operators always return a logical value, TRUE if the query is satisfied, and FALSE if not.

Condition Operators

The syntax for the condition part of the rule resembles the query lanugage used in the Mongo database (MongoDB, 2018). There are two minor differences: first the syntax uses R lists and vectors rather than JSON objects and second the ‘$’ operators are replaced with ‘?’ operators.

In general, each element of the list should have the form field=c(?op=arg). In this expression, field references a field of either the Status or EIEngine (see sQuoteDot Notation section above), ?op is one of the test operators below, and the argument arg is a litteral value (which could be a list) or a character string in dot notation referencing a field of either the Status or Event. If ?op is omitted, it is taken as equals if arg is a scalar and ?in if value is a vector. For more complex queries where arg is a more complex expresison, the c() function is replaced with list().

The following operators (inspired from the operators used in the Mongo database, Mongo DB, 2018, only with ‘?’ instead of ‘$’) are currently supported:

?eq, ?ne

These are the basic operators, which test if the field is (not) equal to the argument.

?gt, ?gte, ?lt, ?lte

These test if the field is greater than (or equal to) or less than (or equal to) the argument. Note that c("?lt"=low,"?gt"=high) can be used to test if the value of the field is between the arguments low and high.

?in, ?nin

These assume that the argument is a vector and are satisfied if the value of the field is (not) in the vector.

?exists, ?isnull, ?isna

These test if the field exists, contains a NULL (often true if the field does not exist) or contains an NA. The arg should be TRUE or FALSE (where false inverts the test.)

?any, ?all

These operators assume that the field contains a vector and check if any (or all) of the elements satisfy the condition given in the argument. Thus, the argument is another expression of the form =c(?op=value). For example field=list("?any"=c("?gt"=3)) will be satisfied if any element of field is greater than 3.

?not

The argument of this query should be another query involving the target field. The not query is satisfied if the inner query is not satistified.

?or, ?and

In both of these cases, the arg should be a list of queries for the applicable field. The ?or query is satisfied if any of the inner queries is satisfied, and the ?and query is satistified if all of the inner queries are satisfied. Like the R || ( &&) operator, the ?or (?and) query runs the subqueries in order and stops at the first true (false) subquery.

?regex

This query uses regular expression matching. The argument should be a regular expression (see regex for a description of R regular expressions). The query is satisfied if the value of the field matches the regular expression.

?where

This query is a trapdoor that allows arbitrary R code to be run to run as the condition. This has a special syntax: "?where"=funname, where the ?where operator takes the place of the field and funname give the name of a function. This should be a function of two arguments, the status and the event, which returns a logical value. The condition is satisfied if the funciton returns true.

Although the ?or operator allows for logical-or expressions for a single field, it does not extend to multiple fields. However, this can be accomplished with separate rules.

Expansion Mechanisms

The special “?where” form obviously allows for expansion. It is particularly designed for queries which involve multiple fields in a complex fashion.

It is also possible to expand the set of ?op functions. The method used for dispatch is to call do.call(op,list(cond, target, state, event)) where cond is everything after ?op=cond in the query expression. (This is the same syntax as the supplied operators).

Condition Testing

The function checkCondition is used internally to check when a set of conditions in a rule are satisfied.

The functions testQuery and testQueryScript can be used to test that rule conditions function properly. The functions testRule and testRuleScript can be used to test that rule conditions and predicates function properly together.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

MongoDB, Inc. (2018). The MongoDB 4.0 Manual. https://docs.mongodb.com/manual/.

See Also

Rule describes the rule object and Predicates describes the predicates. The function checkCondition tests when conditions are satisfied. The functions testQuery and testQueryScript can be used to test that rule conditions function properly.

Other classes in the EIEvent system: EIEngine, Context, Status, Event, RuleTable.

Examples

list(
event.data.agent=list("?eq"="lever"),   #Agent was lever.
event.data.agent="lever",               #Agent abbreviated form.
event.data.agent=list("?ne"="ramp"),   #Agent was not a ramp.
event.data.agent="state.flag.lastagent", #Same agent used.
state.flags.noobj=list("?lt"=10),       #Fewer than 10 objects used.
state.flags.noobj=list("?lt"=10,"?gte"=5), #Between 5 and 10 objects.

#Agent is a lever or a springboard.
event.data.agent=list("?in"=c("lever","springboard")),
#Abbreviated form (note lack of names)
event.data.agent=c("lever","springboard"),
#Agent was not a ramp or a pendulum.
event.data.agent=list("?nin"=c("ramp","pendulum")),   

## Checking for existence of fields and NA values.
state.timers.learningsupport=list("?exists"=TRUE),
event.data.newvalue=("?isnull"=FALSE),
state.flags.lastagent=list("?isna"=TRUE),

## Was the slider a blower (name starts with blower).
event.data.slider=list("?regexp"="^[Bb]lower.*"),

## These assume field is a vector.
state.flags.agentsused=list("?any"=list("?eq"="pendulum")),
state.flags.agentsused=list("?any"="pendulum"), #Abbreviated form.
state.flags.agentsused=list("?all"=list("?eq"="ramp")),
state.flags.agentsused=list("?any"="ramp"), #Abbreviated form.

## ?not
state.flags.agentsused=list("?not"=list("?any"="ramp")), 

## ?and, ?or -- note these stop as soon as falsehood (truth) is proved.
state.flags.noagents=list("?and"=list("?is.na"=FALSE,"?gt"=0)),
state.flags.noagents=list("?or"=list("?is.na"=TRUE,"?eq"=0))

)

## The ?Where operator
agplusobj <- function (state,event) {
  return (getJS("state.flags.noobj",state,event) +
          getJS("event.data.newobj",state,event) < 10)
}

list("?where"="agplusobj")

---

Constructor for the Context object

Description

This is the constructor for the Context objects and context set objects (which are identical). As Context objects are usually read from a database or other input stream, the parseContext function is recreates an event from a JSON list and as.jlist encodes them into a list to be saved as JSON.

Usage

Context(cid, name, number, belongsTo = character(), doc = "", app="default")
parseContext(rec)
## S4 method for signature 'Context,list'
as.jlist(obj, ml, serialize = TRUE)

Arguments

cid	A character identifier for the context. Should be unique within an application (app).
name	A human readable name, used in documentation.
number	A numeric identifier for the context.
belongsTo	A character vector describing context sets this context belongs to.
doc	A character vector providing a description of the context.
app	A character scalar providing a unique identifier for the application.
rec	A named list containing JSON data.
obj	An object of class Context to be encoded.
ml	A list of fields of obj. Usually, this is created by using attributes(obj).
serialize	A logical flag. If true, serializeJSON is used to protect the data field (and other objects which might contain complex R code).

Details

Most of the details about the Context object, and how it works is documented under Context-class. Note that context sets and contexts are represented with the same object.

The function as.jlist converts the obj into a named list. It is usually called from the function as.json.

The parseContext function is the inverse of as.jlist applied to a context object. It is designed to be given as an argument to getOneRec and getManyRecs.

Value

The functions Context and parseContext return objects of class Context. The function as.jlist produces a named list suitable for passing to toJSON.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the context system: https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

Betts, B, and Smith, R. (2018). The Leraning Technology Manager's Guid to xAPI, Second Edition. HT2Labs Research Report: https://www.ht2labs.com/resources/the-learning-technology-managers-guide-to-the-xapi/#gf_26.

HT2Labs (2018). Learning Locker Documentation. https://docs.learninglocker.net/welcome/.

See Also

Context describes the context object, and ContextSet describes the context set object.

buildMessage and as.json describe the JSON conversion system.

The functions getOneRec and getManyRecs use parseStatus to extract events from a database.

Examples

ct <- Context("Level1","First Tutorial",1,
              belongsTo=c("tutorialLevels","easyLevels"),
              doc="First Introductory Level",
              app="ecd://epls.coe.fsu.edu/EITest")

cta <- parseContext(as.jlist(ct,attributes(ct)))
stopifnot(all.equal(ct,cta))

---

Class "Context"

Description

This is a descriptor for a measurement context (e.g., item or game level) in an assessment system. A context plays the role of a task in the four process architecture (Almond, Steinberg, and Mislevy, 2002), but allows for the measurement context to be determined dynamically from an extended task. (Almond, Shute, Tingir, and Rahimi, 2018).

The primary of use of the Context object is determining for which events a rule is applicable. The belongsTo allows designers of an evidence rule system to define context groups for rules which are applicable in multiple contexts.

Objects from the Class

Context object can be created by calls to the Context()) function. Most of the fields in the context are documentation; however, two, the cid and belongsTo fields, play a special role in the rule dispatch logic. The cid field is the identifier of the context as used in the Rule and Status classes.

The belongsTo attribute sets up a hierarchy of classes. In particular, each value of the of this field (if set) should be the cid of a context group to which this context belongs. Context groups are also Context objects and can in turn belong to larger groups. Currently, inheritance is not supported, so that the all parents (direct and indirect) need to be listed).

Context Resolution

When the EIEngine processes an Event, it checks the context of the current Status object. It then searches the rule table for all rules which match on the verb and object fields of the event and the context field of the status. A rule is considered applicable if one of the following conditions is met.

1. The context fields of the Status and Rule class match exactly.

2. The context fields of the Rule class matches the cid of one of the entries in the belongsTo field of the Status.

3. The context fields of the Rule is the keyword “ANY”, i.e., the rule is applicable to all classes.

This is actually accomplished by using the function applicableContexts which creates a list of the current context and all of the context groups that it matches. The EIEngine then grabs from the rule table all rules which match one of the applicable contexts (including “ANY”).

This should seem similar to the way that method dispatch works for S4 classes (see Introduction to the methods package). The Context belongsTo hiearchy is similar to the inheritence hierarchy of a typical class system. There are two key differences. First, inheritense is not currently supported with contexts; all context groups must be explicitly listed in the belongsTo field. Second, while the S4 method dispatch mechanism searches for all methods which are applicable to the current objects, it only executes the most specific method. The table dispatch mechanism executes all the applicable rules and makes no attempt to sort them.

Slots

_id:

Object of class "character" which is the id in the Mongo database; this generally should not be changed.

cid:

Object of class "character" which provides a unique identifier for the context.

name:

Object of class "character" which provides a human readable name for the context.

number:

Object of class "integer" which provides a numeric index for the context.

belongsTo:

Object of class "character" which gives a list of context IDs for context groups to which this context belongs.

doc:

Object of class "character" which provides extended documentation for the context group.

app:

Object of class "character" which a unique identifier for the application in which this context is applicable.

Methods

as.jlist

signature(obj = "Context", ml = "list"): Used in converting the object to JSON for storing in a Mongo database, see as.json.

belongsTo

signature(c = "Context"): Returns the value of the belongsTo field

belongsTo<-

signature(c = "Context"): Sets the value of the belongsTo field.

cid

signature(c = "Context"): Returns the context ID.

doc

signature(x = "Context"): Returns the documentation string.

name

signature(x = "Context"): Returns the name of the context.

number

signature(c = "Context"): Returns the number id of the context.

number<-

signature(c = "Context"): Sets the numeric id of the context.

app

signature(x = "Context"): Returns the app identifier of the context.

show

signature(object = "Context"): Prints the object in a “#<Context >” format.

toString

signature(x = "Context",...): Converts the object to a string in a “#<Context >” format.

Note

It seems natural to create a full inheritance hierarchy for contexts. Probably available in a future version. For now, explicity listing all parents seems easier to implement.

Author(s)

Russell Almond

References

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

See Also

The ContextSet is an object which hold a collection of contexts.

Other classes in the EIEvent system: EIEngine, Event, Status, Rule.

Methods for working with contexts: Context, applicableContexts, parseContext, name, doc, cid, number, app

Examples

showClass("Context")

---

Class "ContextSet"

Description

This is a reference class which is a wrapper for a handle to a database collection of Context objects associated with a particular application. Note that this is a reference class and many of the methods permanently modify the database.

Extends

All reference classes extend and inherit methods from "envRefClass".

Methods

clearContexts

signature(set = "ContextSet"): Removes all of the contexts associated with this applicaiton from the database.

matchContext

signature(id = "character", set = "ContextSet"): searches for a Context with the given context id (cid).

matchContext

signature(id = "integer", set = "ContextSet"): searches for a Context with a given numeric id (number).

updateContext

signature(con = "Context", set = "ContextSet"): Adds or replaces a Context in the database.

ContextSet

signature(app = "character", dbname = "character", dburi = "character", ...): Constructor, creates a new context set for the given application, connecting to the database referenced by dbname and dburi.

Fields

app:

Object of class character giving the identifier for the application. This is part of the database key for the collection.

dbname:

Object of class character giving the name of the database (e.g., “EIRecords”).

dburi:

Object of class character giving the uri for the database, (e.g., “mongodb://localhost”).

db:

Object of class MongoDB which is the handle to the database. As this field is initialized when first requested, it should not be accessed directly, but instead through the contextdb() method.

Class-Based Methods

update(con):

This inserts of replaces a Context object in the database.

initialize(app, dbname, dburi, db, ...):

This sets up the object. Note that the db field is not initialized until contextdb() is first called.

contextdb():

This returns the handle to the mongo collection object. If the connection has not yet been initialized.

named(id):

This searches for the context by context id (cid), not by name.

numbered(num):

This searchers for the context by number.

clearAll():

This removes all contexts associated with this application from the database.

Note

In general, several context sets can share the same database collection. These are distinguished by the application ID (app). The effect primary key for the collection is (app,cid) with a secondary key as (app,number). The collection maintains an index on both of those field pairs.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the context system: https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

See Also

Context describes the context object.

The functions matchContext, updateContext clearContexts, and loadContexts are used to manipulate context sets.

Examples

showClass("ContextSet")
newContextSet(app="default",colname="test",
dbname="test", dburi=character())

---

Meta-data accessors for Rules and Contexts.

Description

To provide both Rule and Context object with adequate documentation, they are given both name and doc properties. The name is used in error reporting and debugging. The doc string is to aid in rule management.

Usage

doc(x)
## S4 method for signature 'Context'
doc(x)
## S4 method for signature 'Rule'
doc(x)
name(x)
## S4 method for signature 'Context'
name(x)
## S4 method for signature 'Rule'
name(x)

Arguments

x	A Context or Rule object whose documentation is being queried.

Value

Both functions return an object of type character.

Author(s)

Russell Almond

See Also

Classes in the EIEvent system: EIEngine, Context, Status, Event, Rule.

Examples

r1 <- Rule(name="Coin Rule",
           doc="Set the value of the badge to the coin the player earned.",
           app="ecd://coe.fsu.edu/PPtest",
           verb="satisfied", object="game level",
           context="ALL",
           ruleType="Observable", priority=5,
           condition=list("event.data.badge"=c("silver","gold")),
           predicate=list("!set"=c("state.observables.badge"=
                                   "event.data.badge")))

stopifnot(name(r1)=="Coin Rule",grepl("badge",doc(r1)))

ct <- Context("Level1","First Tutorial",1,
              belongsTo=c("tutorialLevels","easyLevels"),
              doc="First Introductory Level",
              app="ecd://epls.coe.fsu.edu/EITest")
stopifnot(name(ct)=="First Tutorial",
          grepl("Intro",doc(ct)))

---

Construct an Engine and Load the Rules.

Description

This function reads a configuration file (config.json) and then loads the referenced rules into the database. The assumption is that the 'config.json' will be in git (or other SCCS) system along with the rules expressed in json notation. The referenced rules are then loaded for the specified app.

Usage

doLoad(app, EI.config, EIeng.local, config.dir, override = FALSE)

Arguments

app	A string giving the identifier for the application whose rules are loaded.
EI.config	A list giving details about the EI Event environment. This is usually the content of the 'config.json' file. See details.
EIeng.local	A list giving local (machine specific) configuration information.
config.dir	A string giving the location of the configuation directory, where the rule files are located.
override	A logical flag. If true, then the rules are loaded even if there currently exists a lock file to indicate that they are being used.

Details

The linkS4class{EIEngine} runs against a collection of Rule objects. In the current implementation, the rules are kept in a mongo database ('EIRecords.Rules'). In addition to the rules, there is a table of Contexts which describe collections of tasks (game levels) to which rules should apply. Finally, a default student record with proper initial values needs to be created for the application.

Generally, the rules are coded using a 'json' formatted text file, which is parsed and loaded into the database. These are stored in the directory file.path(config.dir,EI.config$ruledir) (usually the “Rules” directory in the configuration file). The configuration value EI.config$rules is character vector giving the names of the rule files (without the “json” suffix). This is passed to the loadRulesFromList function. The configuration value EI.config$rulesWithTests gives the name of rule files that come with embedded tests. These use the testAndLoad function instead.

In addition to the rule files, the Contexts must be specified. These are loaded from tables (stored in CSV files) where the rows correspond to tasks and the columns correspond to higher level contexts that specific tasks might belong to. This is used to create the collection of contexts in the database. See loadContexts for details. The value EI.config$contextDescriptions gives a character vector containing the name (minus the .csv extension) of the context description files.

Finally, the loader creates a default student record. This allows certain fields (which count across the whole assessment) to be initialized. In Physics playground, the initial back balance is set to zero and the trophy list is initialized to the empty list. The mechanism is that the doLoad function creates a new user record called “*DEFAULT*” and sets its initial observables to the values in the named list EI.config$defaultRecordData. This is then stored in the database. In the future, new student records are created by cloning the “*DEFAULT*” record.

Value

This function is called for its side effects, and the return value should not be used.

Note

The Github project https://github.com/ralmond/PP-EI/ contains and example set of configuration files used for Physics Playground.

This function is meant to be called by the EILoad.R script found in the config directory. (file.path(help(package="EIEvent")$path,"conf","EILoad.R"))

The shell script EILoader found in the same directory will run this script.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

MongoDB, Inc. (2018). The MongoDB 4.0 Manual. https://docs.mongodb.com/manual/.

See Also

EIEngine, doRunrun, loadContexts, loadRulesFromList, testAndLoad

Examples

## This script is available in the conf directory
file.path(help(package="EIEvent")$path,"conf","EILoad.R")

## Not run: 
library(utils)
library(EIEvent)

## It is assumed that EIini.R sets the value of
## config.dir, EIeng.local and Proc4.config
source("/usr/local/share/Proc4/EIini.R")

EI.config <- jsonlite::fromJSON(file.path(config.dir,"config.json"),FALSE)

## appStem is a short nickname for the app
## Proc4.config$apps contains the full names indexed by the nicknames.

appStem <- as.character(EI.config$appStem)

apps <- as.character(Proc4.config$apps[appStem])
if (length(apps)==0L || any(apps=="NULL")) {
  stop("Could not find apps for ",appStem)
}

ruledir <- ifelse(!is.null(EI.config$ruledir),
                  EI.config$ruledir,"Rules")

lockfile <- file.path(config.dir,ruledir,"ruleloader.lock")
file.create(lockfile)
logfile <- (file.path(logpath, sub("<app>","Loader",EI.config$logname)))
if (interactive()) {
  flog.appender(appender.tee(logfile))
} else {
  flog.appender(appender.file(logfile))
}
flog.threshold(EI.config$loglevel)


## Loop over apps
for (app in apps)
  doLoad(app, EI.config, EIeng.local, config.dir)
unlink(lockfile)

## End(Not run)

---

Runs the EIEvent engine using the supplied configuration.

Description

This is a system to run the evidence identifiation engine, taking most of the details from a configuration file. It creates the EIEngine instance then and then runs it in either scoring or rescoring mode. Configuration information in taken from the EI.config and EIeng.local parameters.

Usage

doRunrun(app, EI.config, EIeng.local, config.dir, outdir = config.dir,
override = FALSE, logfile = "", noprep = FALSE)

Arguments

app	A character string giving the global unique identifier for the application being run. This is normally formatted like a URL, and basename(app) is used as a short name.
EI.config	A named list containing the configuration details. See the ‘Configuration’ section below.
EIeng.local	A named list containing additional parameters for the engine constructor. The intention that these are local configuraiton paramete (e.g., database names and passwords) as opposed to more global information. Note this must have an element named “dburi” which gives the URI for the database.
config.dir	The pathname of the directory that contains the the rules subdirectories.
outdir	The pathname of the directory to which output files will be written.
override	A logical flag. If true, the code will ignore locks and restart the run anyway.
logfile	Name for the file in which to do logging.
noprep	A logical flag. If this value is true then the database preparation operations will be skipped. This is intended for continuing a run which was interrupted for some reason.

Details

The goal is to start a run for scoring (evidence identificaiton step) an assessment using the EIEngine class. This function takes care of many of the configuration details and preparatory steps and then calls mainLoop to do the major work. In particular, the steps done by this system are as follows:

1. Configure the listeners.

2. Configure the engine.

3. Clean old scores from the database (optional depending on configuration.)

4. Remove selected events from the collection. Import new evidence sets into the database and mark selected evidence as unprocessed. (This step is skipped if the noprep flag is true.)

5. Launch engine using mainLoop.

6. Build and register the obserables output file.

Note that this will run in either rerun mode, where it will score an selection of existing events and stop, or in server mode where it will continue waiting for new messages until it gets a shut down signal.

Value

This returns the engine invisibly, in case the calling program wants to do something with it.

Configuration

There are a large number of parameters which can be configured. These are passed in through the EI.config argument, which is a list of parameters. The intention is that this can be read in from a JSON file (using fromJSON). The RunEIEvent.R script loads these from a file called config.json. A sample of this file is available on github https://github.com/ralmond/PP-EI. The idea is that an entire configuration directory can be stored in a source code control system to manage the configuration process.

The following fields are available:

ConfigName

An identifier for the configuration. Default value "PP-main". Documentation only, not used by doRunrun.

Branch

The branch name for the git branch for this configuraiton. Default value "PP-main". Documentation only, not used by doRunrun.

Version

A version number for the configuration. Documentation only, not used by doRunrun.

Date

A edit date for the configuration. Documentation only, not used by doRunrun.

appStem

A charcter vector of app stems that will be affected. Sample value ["P4Test"]. This should be the result of applying basename to the longer application IDs.

logLevel

This controls the flog.threshold. Default value "INFO". Note that doRunrun does not set the log value, that should be done in the calling script.

logname

This is the name of the file to which logs should be sent. Example value "EA_<app>0.log". Note that doRunrun does not set the log file, that should be done in the calling script.

sender

The sender field on output messages. Example value "EA_<app>".

ruledir

The directory in which rule files are found. This is only read at load time; see doLoad.

contextDescriptions

A character vector giving the names of context description files (minus the .csv extension) in the ruledir directory. see doLoad and loadContexts.

rules

A character vector giving the names of rule files (minus the .json extension) in the ruledir directory. See doLoad and loadRulesFromList.

rulesWithTests

A character vector giving the names of rule files (minus the .json extension) in the ruledir directory. See doLoad and testAndLoad.

defaultRecordData

A named list giving the initial values of observables in the default (initial) student record.

lscolname

The name of the column to which the listener set should log messages. Example value "Messages".

listeners

This is a list of listener descriptions. See the section ‘Listener Configuration’ below.

listenerReset

Which listeners should be reset before running. This should be a character scalar or vector. The values should be names of listeners. The special value “Self” refers to the ListenerSet object, and the special value “ALL” resets all listeners. See resetListeners. Example value "ALL".

EIEngine

A complex object describing engine parameters. See the section ‘Engine Configuration’ below.

filter

A complex object describing how to prefilter the database. See the section ‘Database Filters’ below.

extensions

This should be a list of paths (relative to config.dir) containing additional R code to load. This is not used by doRunrun, but is supplied for use in scripts that might use doRunrun.

mode

This is a documentation field.

SRreset

A logical flag. If true old student records for the designated application will be cleared before running.

limitNN

An integer: how many events should be processed. Two special string values are also accepted. “ALL” will process all records currently in the database and stop. “Inf” will cause the process to run in server mode until it is shut down.

A number of these values do “<app>” substitution, that is they will substitute the string “<app>” for the short name of the application.

Listener Configruation

The listeners consist of a ListenerSet and a collection of Listener objects. The listener objects are made by using the information from the “listners” element of the EA.config argument. This should be a list of specifications (each specification itself is a list). These are passed to buildListener, which provides some examples.

The listener set is controlled by the EAeng.local$dburi value and the “lscolname” field. If dbuir is a name of a database, then the ListenerSet is logged into the “lscolname” collection. If dburi is null or an empty string, then the listener set will not do logging.

Engine Configruation

The arguments to the appropriate constructor are found between the EIeng.local and EI.config$EIEngine collections. The intent is for the former to include details (e.g., database user names and passwords) which are local to the server on which EIEvent is running, and for EI.config$EIEngine to include more public details which are local to a particular run.

See EIEvent for the expected fields. Note that the “processN” field is taken care of separately after the database operations (next section).

Database Filtering

The EI.config$filter field controls the database filtering process. There are four steps:

Remove

old records from the database.

Import

new records into the database.

Purge

unused records from the database.

Reprocess

Reset the processed flag to ensure records get reprocessed.

These are controlled by the following elements in the EI.config$filter list:

doRemove

Logical, should records be removed before import.

remove

Filter to use for removal. The value {} will remove all records for the given app.

importFile

A list of filenames (in the config.dir) which contain evidence sets to be imported before scoring.

doPurge

Logical, should records be removed after import.

purge

Filter for the purging (after import removal). Leaving this empty will probably not be satisfactory.

doReprocess

Logical, should existing records have the processed flag cleared? Typically TRUE for rerun mode and FALSE for server mode.

reprocess

Filter for the selected records to be marked for reprocessing. The value {} will mark all records (for this app) for reprocessing.

If the doprep argument is false, the database preprocessing will be skipped regardless of the values of the filter field.

Locking

Locking is done via the administrative fields of the database. In particular, if the EIEvent process is marked as running in the database, then the engine will not start. The override switch will force a start anyway.

Data Files

If any of the listeners is a TableListener, then an output file corresponding to the table will be produced when the run finishes. The name of that output file is determined by the field in the lister specificaitons. The datafiles are registered using the ListenerSet$registerOutput method.

Logging

Logging is done through the futile.logger{flog.logger} mechanism. This allows logs to be save to a file.

The “logLevel” and “logname” fields are put in the configuration specification to assist scripts in configuring the logging system.

Both the log file is registered using the ListenerSet$registerOutput method.

Note

This function is meant to be called by the RunEIEvent.R script found in the config directory. (file.path(help(package="EIEvent")$path,"conf","RunEIEvent.R"))

The shell script EIEvent found in the same directory will run this script.

Author(s)

Russell Almond

References

The Bobs (1983) Psychokiller. My I'm Large. Rhino Records. https://www.youtube.com/watch?v=-Gu4PKnCLDg. (Reference is about 2:30 minutes into song.)

See Also

doLoad

EIEngine, mainLoop, ListenerSet

Examples

## This example is in:
file.path(help(package="EIEvent")$path,"conf","RunEIEvent.R")
## Not run: 
library(R.utils)
library(EIEvent)

if (interactive()) {
  ## Edit these for the local application
  appStem <- "P4test"
  loglevel <- ""
  noprep <- FALSE
  override <- FALSE
} else {
  appStem <- cmdArg("app",NULL)
  if (is.null(app) || !grepl("^ecd://",app))
    stop("No app specified, use '--args app=ecd://...'")
  loglevel <- cmdArg("level","")
  noprep <- as.logical(cmdArg("noprep",FALSE))
  override <- as.logical(cmdArg("override",FALSE))
}

source("/usr/local/share/Proc4/EIini.R")

EI.config <- jsonlite::fromJSON(file.path(config.dir,"config.json"),FALSE)


app <- as.character(Proc4.config$apps[appStem])
if (length(app)==0L || any(app=="NULL")) {
  stop("Could not find app for ",appStem)
}
if (!(appStem %in% EI.config$appStem)) {
  stop("Configuration not set for app ",appStem)
}


logfile <- (file.path(logpath, sub("<app>",appStem,EI.config$logname)))
## Let command line override configuration.
if (nchar(loglevel)==OL) loglevel <- EI.config$logLevel

if (interactive()) {
  flog.appender(appender.tee(logfile))
} else {
  flog.appender(appender.file(logfile))
}
flog.threshold(EI.config$loglevel)

## Load extensions.
for (ext in EI.config$extensions) {
  if (is.character(ext) && nchar(ext) > 0L) {
    if (file.exists(file.path(config.dir,ext))) {
      source(file.path(config.dir,ext))
    } else {
      flog.error(paste("Can't find extension file", ext))
    }
  }
}

eng <- doRunrun(app,EI.config,EIeng.local,config.dir,outdir,
                logfile=logfile, override=override, noprep=noprep)


## End(Not run)

---

Creator for the EIEngine class.

Description

The EIEngine is the prime mover for the EIEvent class. This command constructs the engine. The engine stores most of its information in a database, so the constructor mainly consists of the database location and credentials.

Usage

EIEngine(app = "default", dburi = makeDBuri(), listenerSet = NULL,
         dbname = "EIRecords", admindbname = "Proc4", processN = Inf,
         waittime=.25,...)

Arguments

app	The application ID for the engine. See Details.
listenerSet	A ListenerSet which contains the listeners for clients of the engine's messages.
dburi	A character scalar giving the login information for the mongo database. See makeDBuri.
dbname	This the name of the database. The default name is “EIRecords”.
admindbname	Database used for checking if the EIEngine should still be active.
processN	A positive integer. When the mainLoop is started, the engine will process N records before stopping.
waittime	The amout of time (in seconds) to wait before checking again for new evidence sets when the evidence set queue is empty.
...	For future expansions, subclasses.

Details

The EIEngine is an interpreter for the rules, which form a code base for a specific engine. The EIEngine class provides connection to six collections in a database which provide most of the action of the EIEvent system.

Events

Incomming events (Event objects). Accessed through $eventdb() method.

Messages

Outgoing messages (P4Message objects). Accessed through ListenerSet ($listenerSet) object.

Rules

Rule collection (Rule objects). Accessed through RuleTable ($rules) object.

States

Collection of saved statuses (Status objects). Accessed through UserRecordSet ($userRecords) object.

Contexts

Context collection (Context objects). Accessed through ContextSet ($contexts) object.

Tests

Self-test collection (EITest objects). Accessed through TestSet ($tests) object.

Note that the database connections are made in a lazy fashion, connecting to the database when first accessed rather than when the object is created. So the proper accessor methods (e.g., $eventdb() whould be used instead of the raw field names.

All of the collections have the app field as part of their key, so that several applications can share a database. A different EIEngine is needed for each application.

Value

A reference object of class EIEngine. See the class page for information about the methods supported.

Note

This class is not currently threadsafe. In particular, if the handleEvent() function is called simultaneously on two different events with the same uid, the result will not be pretty. However, separate applications can be run in parallel (in different processes or threads).

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

MongoDB, Inc. (2018). The MongoDB 4.0 Manual. https://docs.mongodb.com/manual/.

See Also

The primary classes in the EIEvent system are: EIEngine, Context, Status, Event, Rule.

The EIEngine class is a container for the following classes: UserRecordSet, RuleTable, ContextSet, TestSet and ListenerSet,

flog.logger

mainLoop, handleEvent

Examples

cl <- new("CaptureListener")
app <- "ecd://epls.coe.fsu.edu/EItest"
lset <- ListenerSet(sender=paste(basename(app),"process"),
                    db=mongo::MongoDB("Messages",noMongo=TRUE),
                    listeners=list(cl))
eng <- EIEngine(app=app,listenerSet=lset)

---

Class "EIEngine"

Description

This is the main worker class for the EIEvent process. It handles grapping messages and processing them.

Extends

All reference classes extend and inherit methods from "envRefClass".

Methods

notifyListeners

signature(sender = "EIEngine"): This sends a message to the registered listeners of the EIEngine.

app

signature(sender = "EIEngine"): Returns a string giving the application this Engine is supporting.

Database Connections

The EIEngine is an interprer for a rule-based langauge for processing events. The code, the rules, are stored in a mongo database, as are the events (the input queue) and the status objects. This database is determined by the dburi (note that this is determined with the username, password, host and port of the $initialize() method or EIEngine function) and dbname. Note that this database can be shared by several applications. The app field is used to identify records belonging to this application.

In the default configuration, the database is called “EIRecords” and has the following collections:

Events

This contains the Events to be processed. It can be accessed through the function $eventdb(). Note that each event has a processed field that makes this collection essentially a queue. Other processes can queue events to be processed by inserting event records into this collection.

Messages:

This is used by the ListenerSet to record messages sent by trigger rules. This collection can be accessed throught the $listenerSet field.

Rules:

This contains the collection of Rules. This collection can be accessed through the $ruleTable field.

States:

This contains the collection of Status objects. This collection can be accessed through the $userRecords field.

Context:

This contains the collection of Context objects. This can be accessed through the $contexts field.

Tests:

This contains the collection of RuleTest objects. This can be accessed through the $ruleTests field.

The AuthorizedApps collection in the Proc4 database is used to signal when the EI process should shut down. The initialization argument admindbname is used to set the name of this database, and the admindb() function to access the collection. Note that all database collections should be accessed through the accessor function, which initialize the connection on demand, rather than by accessing the field directly.

The class-based methods, eng$activate() and eng$deactive() turn on an off the flag in the database that indicates that this app is active. The methods eng$isActivated() tests the status of that flag. The methods eng$shouldHalt() and eng$stopWhenFinished() check for signals that the main loop should be terminated immediately or when finished.

The listener objects may also access databases. This is the usual mechanism for the EI process to insert messages into the processing queue of other processes.

Activation and Termination

The mainLoop searches the event database for unprocessed events, and processes them in chronological order. There are three termination conditions. First, if the $processN field (which is decrimented after each event is processed) reaches 0, then the main loop will exit. Second, after each event is processed, it checks the eng$shouldHalt() condition; if this is true, it halts immediately. Third, if the event queue is empty, the engine will check the eng$stopWhenFinished() flag and will stop if it is true. Otherwise, it will wait for the waittime (in seconds) and check again.

The method eng$activate() sets the activated flag for this app to true and eng$deactivate() clears it. These are called from mainLoop. External processes can check this flag in the database by looking at the value of db.AuthorizedApps.find({"app":{"$regex":"P4test"}}, {"appStem":1,"EIactive":1});. The code is Mongo shell javascript, and “P4test” should be replaced with any string uniquely identifying the app (i.e., basename(app)).

The to signal to the process to stop, an external program needs to set the value of the EIsignal field for the appropriate application. The javascript code db.AuthorizedApps.update({app:{"$regex":"P4test"}}, {"$set":{"EIsignal":<value>}});, where “P4test” should be replaced with any string uniquely identifying the app, run in the Mongo shell will work. The <value> should be either “finish” or “halt”. The former triggers the eng$shopWhenFinished() and the latter triggers eng$halt().

Events and the Event Queue

Objects of class Event are stored in a database collection $eventdb(). The processed flag and the timestamp ensure that at any time the system can find the oldest unprocessed event. The function $fetchNextEvent() fetches this event. When the event is processed, the function $setProcessed() can be used to mark it as processed an update the database. The function $setError() is used to add an error message to an event.

A single cycle of the mainLoop consists of the following steps.

1. Fetch the next event with eve <- eng$fetchNextEvent().

2. Process the event with out <- handleEvent(eng,eve).

3. If an error was generated (out is of class try-error), then save the error in the database using eng$setError(eve, out).

4. Mark the event as processed using eng$setProcessed(eve).

Note that $setProcessed() still needs to be called even if $setError() is called.

Rules

Each application has a collection of rules or a RuleTable. This rule table is stored in the $rules field of the EIEngine object. Recall that a rule is applicable to a given event if the verb and object of the event match the verb and object of rule (or the rule has the special object “ALL”) and the context of the rule is one of the applicableContexts of the current context of the state.

In the current implementation, the rules are stored in the Rules database. This solves two problems: rule storage and rule search.

First, the database is where the rules come from when the application starts. The method $loadRules() stores a list of rules in the database and $clearAllRules() removes all rules. Note that trying to add a second rule with the same name will produce an error, unless the second argument (stopOnDups) is set to FALSE. So generally rules need to be cleared before being reloaded. (For finer replacement see the RuleTable object.) Typically, loading the rules is done in three steps:

1. Load in the rules from a JSON file: ruleList <- lapply( fromJSON( filename , FALSE), parseRule). The function parseRule creates a rule object from the output of fromJSON.

2. Clear the old rules using eng$clearAllRules().

3. Load the new rules using eng$loadRules(ruleList).

Second, the search for applicable rules can use standard database queries. The function $findRules() searches the database for rules matching the supplied details. Note that the EIEngine version of this function differs from the RuleTable version in that it accepts a string as input to for the context field and finds a context object to match, while the RuleTable version expects an object of class Context. The phase argument is optional. The first version of the main loop, used the database to separate out rules of different types. However, this was rather inefficient when there were not applicable methods for a rule (the most common case). The current version now searches for rules that would be applicable in any phase of processing and then separates the resulting rule lists.

The $findRules() and rule processing functions do a fair amount of logging, so they can be debugged by setting flog.threshold to a lower value. At the DEBUG level, the number of rules returned for each event is logged. At the TRACE level, the queries used as well as the names of the returned rules are logged.

Context Sets

Rules must be applicable to both the current event and in the current Status. The verb and object fields of the rule can either be a constant, which is matched as an exact string, or the special constant “ALL” which indicates that the rule is applicable to all verbs or objects.

For Contexts, a second intermediate level is introduced: the context set. The context object defines which context sets it belongs to through the belongsTo field. Context objects are stored in the database and are matched to the context field of events using the name field. (The context field of events is trimmed on input to prevent issues with mismatches caused by trailing spaces, see buildMessage.)

The field $contexts is a link to a ContextSet class which contains all of the contexts. The function $getContext() which takes the name of the context as an argument, calls matchContext to find the context object, and returns the context object. Note that if the string is not matched, it will behave like a context which only belongs to the universal context set “ALL”.

Contexts are loaded by parsing a context matrix, a data.frame with the following columns: cid (character, required), number (numeric, required), name (character, optional) and doc (character, required). Additional columns should represent context sets, and should be 1 if the context row is in the set column and 0 otherwise. See loadContexts for an example. A special context with number 0 and cid “*INITIAL*” is required to represent the initial state when the player first logs into the system. The method $addContexts() adds the contexts in the matrix to the system. The method $clearContexts() removes existing contexts.

A typical initialization has the following steps:

1. Load the context set from a datafile: conMat <- read.csv(filename).

2. Create the initial context: initCon <- data.frame(CID="*INITIAL*", Name="*INITIAL*", Number=0).

3. Clear Old contexts: eng$clearContexts().

4. Load context mattrix: eng$addContexts(conMat).

5. Load initial context: eng$addContexts(initCon).

Status and Default Status

The $userReocrds field of the engine points to a UserRecordSet collection. This is a set of Status records for the students. Once again, it is stored in the database to give the status persistence across invocations of the engine.

The method $getStatus() fetches a status from the database for the specified user. If no status is found, then the method $newUser() is called to create the inital status. This will look for a status for a user called “*DEFAULT*”, and will copy that status if available. Otherwise, a blank status (one with the timers, flags and obsservables fields empty.

The method $saveStatus() saves the status back to the database. Note that the return value should be captured, as this will update the _id (see m_id) field of the record. The method $clearStatusRecords(clearDefault) will remove old status records, so that all players start fresh. With the argument set to true, it will also remove the default record.

The following steps can be used to set up the default record.

1. Clear old records including default: eng$clearStatusRecords(TRUE).

2. Create a new blank status: defaultRec <- eng$newUser("*DEFAULT*").

3. Initialize fields on that record:

   • Flag: flag(defaultRec, name ) <- value.

   • Observable: obs(defaultRec, name ) <- value.

   • Timers: timer(defaultRec, name ) <- Timer( name).

4. Save it back to the database: defaultRec <- eng$saveStatus(defaultRec).

Listeners and Messages

The output of the EI process is through the Listener objects. The $listenerSet field holds an object of class ListenerSet which in turn holds the listeners. When a trigger rule fires, the !send predicate creates a message which is then sent to the ListenerSet though the notifyListeners method. This, in turn, calls the receiveMessage method on each of the listeners.

If the dburi field in the ListenerSet is set, then the ListenerSet maintains a connection to the Messages (or other named) collection in the database used by the EIEngine. In this case, all messages are logged to this database. The listeners, in contrast, generally only record messages whose message matches a given criteria. Many of the listeners connect to a database other than the internal one: They could either insert messages in the queue for another process or update a record in a database that tracks player status.

The CaptureListener is particularly useful for testing. It stores all messages in a list and has a specific method for viewing the latest one.

Main Event Loop

The goal of the EIEngine is to act as a server process. It continually scans the event queue, looking for unprocessed events. It processes the oldest unprocessed event, marks it as processed and then looks for the next event. If no unprocessed events are found it sleeps for a bit and then tries again. The field $waittime is the amount of time in seconds that the main loop will wait before checking again.

The function mainLoop implements the main processing loop. Generally, it should be the last call in a script for running the process (see EIEvent.R in the config subdirectory of the package for an example). It terminates under one of three conditions: (1) When the event queue is empty, it checks the $stopWhenFinished() method which checks the flag in the AuthorizedApps collection in the Proc4 database. If the function returns false, the main loop will terminate. (2) The $processN field is decremented with every event processed. If it is given a finite value (the default is infinity) then it will process that many records and then stop. (3) If the $shouldHalt() method returns true, the process stops rather than go onto the next method. (This allows for an emergency shutdown if an error is found in the configuration).

Running mainLoop with $processN set to a positive integer is useful for testing. In one mode a collection of events can be read into the database from a test file, then the $processN field can be set to the number of new events (see EIEvent.R for an example). This will then run the test events and then stop. Alternatively, $processN can be set to a number lower than the total number of events and then the logging threshold can be set higher, or the processing can be run step by step to isolate problematic events.

Logging and Error Handling

Logging is handled using the flog.logger framework. This provides a large number of tools for specifying the details of the logging. The EIEngine executes the rules in the withFlogging environment. This means that the default behavior for rules which generate errors is to log the error and move to the next rule. The error message is also added to the event in the event database (markAsError).

The amount of logging done, particularly the amount of detail supplied, is controled by the flog.threshold function. The amount of detail provided at various levels is as follows:

TRACE

• The results of checkCondition are reported.

• The specific rules found from each query are reported.

• A message is logged as each rule is run.

DEBUG

• When an error occurs information about the state, event and rule where the error occurred as well a stack trace are logged.

• Each event is logged as it is processed.

• Rule searches are logged and the number of rules reported.

• A message is logged when each phase starts.

• A message is logged when the context changes.

INFO

Minimal information about which events are being processed is logged.

WARN and above

If an error occurs the error is logged along with context information.

When running the EIEngine as a server, generally the logging should be done as a file. This can be done by running: flog.appender(appender.file(logfile)). When running in interactive mode, it may be useful to have log messages sent to both the console and the log file. The command for this is: flog.appender(appender.file(logfile)).

Test Sets

This is mostly a placeholder for future capability. Along with the rules, which are after all a program, there should be a test suite. The goal is to provide a collection of tests, so that the test suite can be rerun when rules are modified, creating a unit testing facility for the EI-Event language.

This has not yet been implemented, but the function testRuleScript provides a mechanism for writing test scripts.

Tracing

The EIEngine uses the flog.logger system. In particular, setting the threshold for the “EIEvent” logger, will change the amount of information sent to the log file.

Fields

app:

Object of class character. This is the long url-like name of the application. Note that it is a key to all of the database lookup.s

dburi:

Object of class character giving the URI for the link to the Mongo database

dbname:

Object of class character giving the name of the user database

admindbname:

Name of the database which contains the AuthorizedApps collection used to start and stop the engine.

adminDB:

Object of class MongoDB referencing the AuthorizedApps collection used to start and stop the engine. This field should not be referenced directly, instead, the $admindb() should be used as it will initialize the connection if needed.

waittime:

The number of seconds to wait before rechecking the queue when it is empty.

userRecords:

Object of class UserRecordSet giving the user record set for the applcation.

rules:

Object of class RuleTable giving the rule set for the application

contexts:

Object of class ContextSet giving the set of contexts supported by the application.

events:

Object of class MongoDB a mongo DB connection which allows access to events. Note that this is created when first requested, so should be accessed using the eventsdb() method.

ruleTests:

Object of class TestSet give a collection of tests for the rule system.

listenerSet:

Object of class ListenerSet giving the registered listeners for messages sent by the system.

processN:

An integer counting how many events the mainLoop should process before stopping. If infinite, the main loop will not stop until the authorized app flag is cleared.

Class-Based Methods

initialize(app, listenerSet, dburi, dbname, admindbname, waittime, processN, ...):

This method sets up the object. Note that the listeners object is passed in as an argument, as are the database credentials.

admindb():

Returns a handle to the AuthorizedApps collection in the Proc4 database. Initializes if needed.

isActivated():

Returns true if the active flag is set on this applicaiton (app) in the AuthorizedApps collection.

activate():

Sets the active flag to true for this app in the AuthorizedApps collection.

deactivate():

Clears the active flag to true for this app in the AuthorizedApps collection.

shouldHalt():

This function checks the database to see whether or not the flag is set to cause the process to halt after processing the current record..

stopWhenFinished():

This function checks the database to see whether or not the flag is set to cause the process to stop when the event queue is empty.

show():

Provides a short string identifying the engine.

newUser(uid):

Generates a new student record. (Call to UserRecordSet$newStudent())

getStatus(uid):

Feteches the current status for a student from the database. (Call to UserRecordSet$getStatus())

saveStatus(state):

Saves an updated status to the database. (Call to UserRecordSet$saveStatus())

clearStatusRecords(clearDefault):

Clears the status records, including the default record if the argument is TRUE. (Call to UserRecordSet$clearAll()).

getContext(id):

This is a call to matchContext.

addContexts(conmat):

This is a call to loadContexts.

clearContexts():

This is a call to ContextSet$clearAll().

eventdb():

This returns the event database handle, creating it if it is not yet created. It is recommended to use this method rather than access the slot directly.

setProcessed(mess):

This sets the processed flag on its argument and updates the database.

setError(mess,e):

This sets the processingError flag on mess to e and updates the database..

fetchNextEvent():

This returns the unprocessed event with the oldest timestamp, or NULL if there are no unprocessed events.

findRules(verb, object, context, phase=NULL):

This function finds the potentially applicable for the current verb, object, context and phase. If phase==NULL, the get rules for all phases.

loadRules(rlist, stopOnDups):

This function loads rules into the rule table using loadRulesFromList.

loadAndTest(script, stopOnDups):

This function loads rules into the rule table from test scripts using testAndLoad.

Note

A different EIEninge process is associated with each application (application ID), so different applicaitons can run in parallel. More sophisticated concurancy checking is not currently available.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

There is a “quick start” document which describes how to set up the engine. This is available at https://pluto.coe.fsu.edu/Proc4/EIQuickStart.pdf.

See Also

The function EIEngine tells about the construction and the relationship between the Engine and the database.

The primary classes in the EIEvent system are: EIEngine, Context, Status, Event, Rule.

The EIEngine class is a container for the following classes: UserRecordSet, RuleTable, ContextSet, TestSet and ListenerSet,

flog.logger

Examples

showClass("EIEngine")

---

Class "EITest"

Description

An object describing a test case for a certain evidence identification system. It describes the input state, the event and the target output state so that the interaction of the rules can be checked.

Objects from the Class

An EITest object consists of a test. The test case has an initial Status, a triggering Event, and an expected result Status. If the goal is intead to check output messages, the result can be a P4Message or a list of such messages.

Objects can be created by calls of the form EITest(...), or from JSON through parseEITest.

Slots

_id:

Object of class "character", the internal mongo identifier.

app:

Object of class "character", the unique identifier for the application to which this belongs.

name:

Object of class "character", a human readable name for the test, used in reporting.

doc:

Object of class "character", a documentation string describing the test.

initial:

Object of class Status which describes the initial state of the system before the test.

event:

Object of class Event which describes the incoming event to which the system is reacting.

final:

Object of class Status which describes the expected final state of the system after applying the rules, or an object of class P4Message describing the generated message.

Methods

as.jlist

signature(obj = "EITest", ml = "list"): helper function for converting the test into a JSON object, see as.json.

doc

signature(x = "EITest"): Returns the documenation string.

event

signature(x = "EITest"): Returns the triggering event for the test.

final

signature(x = "EITest"): Returns the expected final state of the system.

initial

signature(x = "EITest"): Returns the initial state of the test.

name

signature(x = "EITest"): Returns the name of the test.

show

signature(object = "EITest"): Displays the rule object.

toString

signature(x = "EITest"): Returns a string describing the rule.

Author(s)

Russell G. Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

See Also

The functions EITest and parseEITest are used to construct the rule.

The functions name, doc, initial, event, and final access the components of the test.

The TestSet maintains a collection of tests for a particular application.

Examples

showClass("EITest")

---

Event object constructor

Description

The Event funciton is the constructor for the Event object. As Event objects are usually read from a database or other input stream, the parseEvent function is recreates an event from a JSON list.

Usage

Event(uid, verb, object = "", timestamp = Sys.time(), details = list(),
      app = "default", context = "", processed = FALSE)
parseEvent(rec)
## S4 method for signature 'Event,list'
as.jlist(obj, ml, serialize = TRUE)

Arguments

uid	A character scalar identifying the examinee or player.
verb	A character scalar identifying the action which triggered the event.
object	A character scalar identifying the direct object of the verb.
timestamp	An object of class POSIXt which provides the time at which the event occurred.
details	A named list of detailed data about the the event. The available fields will depend on the app, verb and object.
app	A character scalar providing a unique identifier for the application (game or assessment). This defines the available vocabulary for verb and object, as well as the set of applicable Rule objects.
context	A character string describing the task, item or game level during which the event occurred. It could be blank if the context needs to be figured out from surrounding events.
processed	A logical flag. Set to true after the event has been processed by the EIEngine.
rec	A named list containing JSON data.
obj	An object of class Event to be encoded.
ml	A list of fields of obj. Usually, this is created by using attributes(obj).
serialize	A logical flag. If true, serializeJSON is used to protect the data field (and other objects which might contain complex R code.

Details

Most of the details about the Event object, and how it works is documented under Event-class.

The function as.jlist converts the obj into a named list. It is usually called from the function as.json.

The parseEvent function is the inverse of as.jlist applied to an event object. It is designed to be given as an argument to getOneRec and getManyRecs.

Value

The functions Event and parseEvent return objects of class event. The function as.jlist produces a named list suitable for passing to toJSON.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the JSON layout of the Event objects. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

Betts, B, and Smith, R. (2018). The Leraning Technology Manager's Guid to xAPI, Second Edition. HT2Labs Research Report: https://www.ht2labs.com/resources/the-learning-technology-managers-guide-to-the-xapi/#gf_26.

HT2Labs (2018). Learning Locker Documentation. https://docs.learninglocker.net/welcome/.

See Also

Event describes the event object.

buildMessage and as.json describe the JSON conversion system. In particular, as Event extends P4Message, the Event method for as.jlist calls the P4Message method.

The functions getOneRec and getManyRecs use parseEvent to extract events from a database.

Examples

ev1 <- Event("Phred","test","message",
      timestamp=as.POSIXct("2018-12-21 00:01:01"),
      details=list("list"=list("one"=1,"two"=1:2),"vector"=(1:3)))
ev2 <- Event("Phred","wash","window",
      timestamp=as.POSIXct("2018-12-21 00:02:01"),
      details=list(condition="streaky"))
ev3 <- Event("Fred","open","can",
      timestamp=as.POSIXct("2018-12-21 00:03:01"),
      details=list(lidOn=FALSE))


ev1a <- parseEvent(ununboxer(as.jlist(ev1,attributes(ev1))))
ev2a <- parseEvent(ununboxer(as.jlist(ev2,attributes(ev2))))
ev3a <- parseEvent(ununboxer(as.jlist(ev3,attributes(ev3))))

stopifnot(all.equal(ev1,ev1a), all.equal(ev2,ev2a), all.equal(ev3,ev3a))

## Not run:  #Requires test DB setup.
testcol <- mongolite::mongo("Messages",
                 url="mongodb://test:[email protected]:27017/test")
## Mongodb is the protocol
## user=test, password =secret
## Host = 127.0.0.1 -- localhost
## Port = 27017 -- Mongo default
## db = test
## collection = Messages
testcol$remove('{}')  ## Clear everything for test.

ev1 <- saveRec(ev1,testcol)
ev2 <- saveRec(ev2,testcol)
ev3 <- saveRec(ev3,testcol)

ev1b <- getOneRec(buildJQuery("_id"=ev1@"_id"),testcol,parseEvent)
ev23 <- getManyRecs(buildJQuery("uid"="Phred"),testcol,parseEvent)
stopifnot(all.equal(ev1,eb1b), length(ev23)==2L)


## End(Not run)

---

Class "Event"

Description

This class represents a generalize event happening in a simulation or a game. All events have common metadata which identify what happened (verb), what was effected (object), the time of the action (timestamp) as well as the application app and user (uid). However, the details field of the event will differ dependiing on the app, verb, and object.

Objects from the Class

Objects can be created by calls the the Event function, although more typically they are passed to the EIEngine by the presentation process.

The event object is a simplifcation of the xAPI events (Betts and Ryan, 2018). In particular, in the xAPI format, both the verb and object fields are complex objects which have a long URL-like identifier to make sure they are unique across applications. In the EIEvent prototocl, only the app field is given a long URL-like name. The application should define the acceptable vocabulary for verbs and objects, which should correpsond to the verb and object fields of the Rule objects.

Slots

verb:

Object of class "character" which provides an identifier for the action which just occurred.

object:

Object of class "character" which provides an identifier for the direct object of the action which just occured.

_id:

Object of class "character" the Mongo database identifier; this should not be modified.

app:

Object of class "character" a unique identifier for the application. This defines which EIEngine is used to handle the event.

uid:

Object of class "character" identifier for the user (student or examinee).

context:

Object of class "character" an identifier for the context, for applications in which the context is determined by the presentation process.

sender:

Object of class "character" the name of the process which generated the event, usually the presentation process.

mess:

Object of class "character" a title for the message, used by the P4Message dispatch system.

timestamp:

Object of class "POSIXt" giving the time at which the event occurred.

data:

Object of class "list" giving the contents of the message. This will be details specific to the verb and object.

Extends

Class "P4Message", directly. All fields except verb and object are inherited from the parent.

Note that the Event is the basic message sent from the presentation process to the evidence identification process in the four-process architecture (Almond, Steinberg and Mislevy, 2002). It has been extended slightly, borrowing (and simplifying) the verb and object header fields from the xAPI format (Almond, Shute, Tingir, and Rahimi, 2018).

Note that Physics Playground uses a slightly different architecture. The presentation uses Learning Locker (HT2Labs, 2018) to log events in the xAPI format into a mongo database. When the game level completes, a message is sent to the EIEngine which extracts the relevant messages from the Learning Locker database and simplifies them into the event format.

Methods

as.jlist

signature(obj = "Event", ml = "list"): ...

object

signature(x = "Event"): Fetches the object component.

verb

signature(x = "Event"): Fetches the verb component.

show

signature(object = "Event"): Prints event object.

toString

signature(x = "Event"): Produces <> representation of object.

all.equal.Event

(target, current, ..., checkTimestamp=FALSE, check_ids=TRUE): (S3 method) Checks for equality. The checkTimestamp flag controls whether or not the timestamp is checked. The check_ids flag controls whether or not the database IDs are checked.

Author(s)

Russell Almond

References

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

Betts, B, and Smith, R. (2018). The Leraning Technology Manager's Guid to xAPI, Second Edition. HT2Labs Research Report: https://www.ht2labs.com/resources/the-learning-technology-managers-guide-to-the-xapi/#gf_26.

HT2Labs (2018). Learning Locker Documentation. https://docs.learninglocker.net/welcome/.

See Also

The event class is a subclass of P4Message.

Other classes in the EIEvent system: EIEngine, Context, Status, Rule.

Methods for working with events: Event, parseEvent

Examples

showClass("Event")

---

Executes the predicate of an EIEvent Rule.

Description

An Rule object contains a list of Predicates. The name of each condition is the name of an operator and its value is a list giving the field of the Status object to be modified as a name and a new value as the value. For example, "!incr"=c("state.flags.agentCount"=1) would increment the agent count flag by one. The value can also be a reference to fields in the Status or Event object. For example, "!set"=c("state.observables.trophy" = "event.data.trophy") would set the value of the trophy observable to the value of the trophy datum in the event.

Usage

executePredicate(predicate, state, event)

Arguments

predicate	A named list of actions: see details.
state	An object of class Status to be modified
event	An object of class Event which will be referenced in setting the values.

Details

The predicate of a Rule is a list of actions to be taken when the rule is satisfied. Each action has the following form:

!op=list(field=arg,...)

Here, field is an identifier of a field in the Status object being modified (the Event fields cannot be modified). This is in the dot notation (see setJS). The setting operator, !op is one of the operators described in Predicates. For the "!set" operator, the arg is the replacement value or field reference that the field will be used as the replacement value. Most of the other operators use arg to modify the value of field and then replace the value of field with the result. For example, the "!incr" operator acts much like the C += operator. The ... represents additional field–arg pairs to be set.

The arg can be a literal value (either scalar or vector) or a reference to another field in the Status or Event object using the dot notation. Note that certain operations on timers use the timestamp field of the Event to update the timer.

In general, a predicate contains a list of actions. These are executed sequentially, but no guarentees are made about the order.

Finally, one special operator allows for expansion. For the "!setCall" operator, arg should be the name of a function, with arguments (name, state, event), where name is the name of the target field, and state and event are the current state and event objects. The value is set to the value returned.

See Predicates for more details.

Value

The function executePredicate returns the modified status object.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

MongoDB, Inc. (2018). The MongoDB 4.0 Manual. https://docs.mongodb.com/manual/.

See Also

Rule describes the rule object and Predicates describes the conditions. Conditions describes the condition part of the rule, and checkCondition checks the conditions.

The functions testPredicate and testPredicateScript can be used to test that rule conditions function properly.

Other classes in the EIEvent system: EIEngine, Context, Status, Event, RuleTable.

Examples

st <- Status("Phred","Level 1",timerNames=c("learningsupports"),
   flags=list(lastagent="lever",agentlist=c("lever"),
              noobj=7,noagents=0),
   observables=list(),
   timestamp=as.POSIXct("2018-12-21 00:01"))

ev <- Event("Phred","test","message",
      timestamp=as.POSIXct("2018-12-21 00:01:01"),
      details=list(agent="lever"))


## Set a flag and an observable.
st1 <- executePredicate(list("!set"=list("state.flags.agent"="ramp",
            "state.observables.trophy"="gold")),st,ev)
stopifnot(getJS("state.flags.agent",st,ev)=="ramp",
          getJS("state.observables.trophy",st,ev)=="gold")

## Set a timer
st1 <- executePredicate(list("!set"=
                             list("state.timers.learningsupports.time"=
                                   as.difftime(0,units="secs"),
                                  "state.timers.learningsupports.run"=TRUE)),
                        st,ev)
stopifnot(
   timerRunning(st1,"learningsupports",as.POSIXct("2018-12-21 00:01:01")),
   timerTime(st1,"learningsupports",as.POSIXct("2018-12-21 00:01:11"))==
   as.difftime(10,units="secs"))


## Delete fields
st1 <- executePredicate(
  list("!unset"=list("state.flags.noobj"="NA", # Set to NA
                     "state.flags.lastagent"="NULL", # Set to NULL
                     "state.flags.noagents"="Delete")), # Delete it. 
   st,ev)
stopifnot(is.na(flag(st1,"noobj")),is.null(flag(st1,"lastagent")),
          is.null(flag(st1,"noagents")))


## Modify fields.
st1 <- executePredicate(
   list("!incr" = list("state.flags.noagents"=1,
                       "state.timers.learningsupports"=
                       as.difftime(1,units="mins"))),
   st,ev)
stopifnot(flag(st1,"noagents")==1,
   timerTime(st1,"learningsupports",as.POSIXct("2018-12-21 00:01:11"))==
   as.difftime(60,units="secs"))

st2 <- executePredicate(
   list("!decr" = list("state.flags.noagents"=1,
                       "state.timers.learningsupports"=
                       as.difftime(30,units="secs"))),
   st1,ev)
stopifnot(flag(st2,"noagents")==0,
   timerTime(st2,"learningsupports",as.POSIXct("2018-12-21 00:01:11"))==
   as.difftime(30,units="secs"))


st1 <- executePredicate(
   list("!mult" = list("state.flags.noobj"=2)),st,ev)
stopifnot(flag(st1,"noobj")==14)
st2 <- executePredicate(
   list("!div"  = list("state.flags.noobj"=2)),st1,ev)
stopifnot(flag(st2,"noobj")==7)

st1 <- executePredicate(
   list("!min"  = list("state.flags.noobj"=5)),st,ev)
stopifnot(flag(st1,"noobj")==5)
st1 <- executePredicate(
   list("!max"  = list("state.flags.noagents"=1)),st,ev)
stopifnot(flag(st1,"noagents")==1)

## Set operators
st1 <- executePredicate(
   list("!addToSet" =list("state.flags.agentlist"="lever")),st,ev)
stopifnot(flag(st1,"agentlist")=="lever")
st2 <- executePredicate(
   list("!addToSet" =list("state.flags.agentlist"="springboard")),st1,ev)
stopifnot(setequal(flag(st2,"agentlist"),c("lever","springboard")))

st3 <- executePredicate(
   list("!pullFromSet" =list("state.flags.agentlist"="lever")),st2,ev)
stopifnot(flag(st3,"agentlist")=="springboard")


st1 <- executePredicate(
   list("!push"=list("state.flags.objects"="Object 1")), st,ev)
stopifnot(flag(st1,"objects")=="Object 1")

st2 <- executePredicate(
   list("!pop"=list("state.flags.objects"="state.flags.lastObject")),st1,ev)
       #Pop first object off the stack and set lastObject to its value.
stopifnot(flag(st2,"lastObject")=="Object 1",
          length(flag(st2,"objects"))==0L)

myOp <- function(name,state,event) {
  return(getJS("state.flags.noobj",state,event)/
         as.double(getJS("state.timers.learningsupports.time",state,event),
         units="mins"))
}

st1 <- executePredicate(
   list("!setCall"=list("state.flags.value"="myOp")),st,ev)
       #Set value to return value of myOp.
stopifnot(!is.finite(flag(st1,"value")))

stx <- Status("Phred","Level 1",timerNames=c("learningsupports"),
   flags=list(lastagent="lever",agentlist=c("lever"),
              noobj=7,noagents=0),
   observables=list(trophyHall=list(),bankBalance=0),
   timestamp=as.POSIXct("2018-12-21 00:01"))

eve <- Event("001c1","initialized","newMoneyEarned",
             timestamp=as.POSIXct("2019-04-29 15:34:35.761500"),
             details=list(gameLevelId="Down Hill",
                          earningType="goldWin",
                          moneyEarned="20",
                          currentMoney="60"
                          ), context="Down Hill")

stx1 <- executePredicate(
          list("!set"=list("state.observables.bankBalance"=
                           "event.data.currentMoney"),
               "!setKeyValue"=list("state.observables.trophyHall"=
                    list(key="event.data.gameLevelId",
                         value="event.data.earningType"))),stx,eve)

---

Accessor functions for context objects.

Description

These are basic accessor functions for the fields of an object of class Status. Note that both the flgs and observables fields of the Status object are named lists. The functions flag(x,name) and obs(x,name) access a single component of those objects.

Usage

flag(x, name)
## S4 method for signature 'Status'
flag(x, name)
flag(x, name) <- value
## S4 replacement method for signature 'Status'
flag(x, name) <- value
obs(x, name)
## S4 method for signature 'Status'
obs(x, name)
obs(x, name) <- value
## S4 replacement method for signature 'Status'
obs(x, name) <- value
## S4 method for signature 'Status'
app(x)
## S4 method for signature 'Status'
timestamp(x)

Arguments

x	A Status object whose fields will be accessed.
name	The name of the component for a flag or obs field.
value	The replacement value for the field.

Value

The functions flag and obs return the named component of the flags or observables field of x respectively. If no component will the given name exists, they return NULL.

The functions app, context, timestamp and oldContext return the value of the corresponding field of x.

The setter methods return the modified Status object.

Author(s)

Russell Almond

See Also

Status describes the state object. See context, context<-, and oldContext for other accessor methods shared by context and other objects.

The functions setJS, getJS and removeJS provide mechanisms for accessing the fields of a status object from Rule Conditions and Predicates.

The following functions access the timer fields of the state object. timer, timerTime, timerRunning, setTimer

Examples

st <- Status("Phred","Level 0",timerNames="watch",
   flags=list("list"=list("one"=1,"two"="too"),"vector"=(1:3)*10),
   observables=list("numeric"=12.5,char="foo",
                 "list"=list("one"="a","two"=2),"vector"=(1:3)*100),
   timestamp=as.POSIXct("2018-12-21 00:01"))

stopifnot(
  uid(st) == "Phred",
  context(st)=="Level 0",
  oldContext(st)=="Level 0",
  all.equal(flag(st,"list"),list("one"=1,"two"="too")),
  all.equal(flag(st,"vector"),(1:3)*10),
  all.equal(obs(st,"numeric"),12.5),
  all.equal(obs(st,"char"),"foo"),
  all.equal(obs(st,"list"),list("one"="a","two"=2)),
  all.equal(obs(st,"vector"),(1:3)*100),
  all.equal(timestamp(st),as.POSIXct("2018-12-21 00:01"))
)

context(st) <- "Level 1"
stopifnot(
  context(st)=="Level 1",
  oldContext(st)=="Level 0")

stopifnot(is.null(flag(st,"numeric")))
flag(st,"numeric") <- 17L
stopifnot(!is.null(flag(st,"numeric")),
          flag(st,"numeric")==17L)

flag(st,"list")$two <- "two"
stopifnot(all.equal(flag(st,"list"),list("one"=1,"two"="two")))

obs(st,"vector")[2] <- 2
stopifnot(all.equal(obs(st,"vector"),c(100,2,300)))

---

Gets a field from an object in Javascript notation.

Description

Fields of a Status can be accessed using JavaScript notation, e.g., state.flags.field, state.observables.field,or state.timers.name. Similarly, fields of an Event can be referenced using event.verb, event.object, event.data.field, or event.timestamp. The function getJS fetches the current value of the referenced field from the state or event object.

Usage

getJS(field, state, event)
getJSfield(obj,fieldlist)

Arguments

field	A character scalar describing the field to be referenced (see details).
state	An object of type Status giving the current status of the user in the system.
event	An object of type Event giving the event being processed.
obj	A collection object to be accessed. The object implmenting state.flags, state.observables or event.details, or one of sub-components.
fieldlist	The successive field names as a vector of characters (split at the ‘.’ and excluding the initial state.flags, state.observables or event.details.

Details

Both the Conditions and Predicates of Rule objects need to reference parts of the current state and event. As these rules are typically written in JSON, it is natural to reference the parts of the Status and Event objects using javascript notation. Javascript, like R, is a weakly typed language, and so javascript objects are similar to R lists: a named collection of values. A period, ‘.’, is used to separate the object from the field name, similar to the way a ‘$’ is used to separate the field name from the object reference when working with R lists. If the object in a certain field is itself an object, a succession of dots. Thus a typical reference looks like: object.field.subfield and so forth as needed.

In EIEvent rules, only two objects can be referenced: state, the current Status, and event, the current Event. Therefore, all dot notation field references must start with either state or event. Furthermore, Status and Event objects have only a certain number of fields so only those fields can be referenced.

The event object has one field which can contain arbitrary collections, event.data. The state object has two state.flags and state.observables. (The state also contains a collection of timer objects, state.timers, which has special rules described below.) Each of these is a named collection (list in R), and components can be refenced by name. The expressions “event.data.name”, “state.flags.name”, and “state.observables.name” reference an object named name in the data field of the event, or the flags or observables field of the state respectively. Note that the available components of these lists fields will depend on the context of the simulation and the verb and object of the event.

The fields of event.data, state.flags and state.observables could also be multipart objects (i.e., R lists). Additional dots can be used to reference the subcomponents. Thus “event.data.position.x” references the x-coordinate of the position object in the event data. These dots can be nested to an aribrary depth.

The fields of event.data, state.flags. and state.observables. can also contain unnamed vectors (either character, numeric, or list). In this case square brackets can be used to index the elements by position. Indexes start at 1, as in R. For example, “state.flags.agentList[3]” references the third value in the agentList flag of the status. Currently only numeric indexes are allowed, variable references are not, nor can sublists be selected.

The function getJSfield is an internal function which is used to access components. It is called recursively to access fields which are themselves lists or vectors.

Value

The value of getJS the the contents of the referenced field. If the referenced field does not exist, or the reference is not well formed, then an error is signaled.

Fields of the Event object.

The following expressions reference the fields of the Event object.

• event.verb The verb associated with the current event.

• event.object The object associated with the current event.

• event.timestamp The time at which the event occurred.

• event.data.field The value of the extra data field named field.

The following additional fields can also be referenced, but are seldom used. In many cases, these fields are handled by the EIEngine before rule processing begins, so their values are irrelevant.

• event.app The application ID associated with the current event..

• event.context The simulator context of the current event as recorded by the presentation process.

• event.uid The user ID of the student or player.

• event.message The message sent from the presentation process. (Often something like “New Event”.)

Fields of the Status object.

The following fields of the Status object can be referenced.

• state.context The current context that the state object is in.

• state.oldContext The the context of the state at the end of the previous event. In particular, this can be compared to the context to check if the context has changed as a result of the event.

• state.observables.field The value of the observable named field.

• state.timers.field The the timer named field. Note that state.timers.field.time or .value refers to the current elapsed time of the timer, and state.timers.field.run or .running is a logical value which refers to whether or not the timer is running.

• state.flags.field The value of the observable named field.

The following additional fields are also recognized, but again they are primarily for use in the EIEngine.

• state.uid The user ID of the student or player. (Should be the same of that of the event.)

• state.timestamp The timestamp of the last event encorporated into the status.

Timers

The state.timers field holds a named list of objects of class Timer. These behave as if they have two settable subfields: running (or run) and time (or value).

The running (or run) virtual field is a logical field: TRUE indicates running and FALSE indicates paused. Setting the value of the field will cause the timer to resume (start) or pause depending on the value.

The time (or value) field gives the elapsed time of the timer. Setting the field to zero will reset the timer to zero, setting it to another value will adjust the time.

Note

It is clear that some kind of indirect reference (i.e., using variables, either integer or character, inside of the square brackects) is needed. This may be implemented in a future version.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

MongoDB, Inc. (2018). The MongoDB 4.0 Manual. https://docs.mongodb.com/manual/.

See Also

The functions setJS for setting fields and removeJS for removing fields (only allowed with state objects). This function is called from the functions checkCondition and executePredicate.

The help files Conditions and Predicates each have detailed descriptions of rule syntax.

Other classes in the EIEvent system: EIEngine, Context, Status, Event, Rule.

Examples

st <- Status("Phred","Level 0",timerNames="watch",
   flags=list("list"=list("one"=1,"two"="too"),"vector"=(1:3)*10),
   observables=list("numeric"=12.5,char="foo",
                 "list"=list("one"="a","two"=2),"vector"=(1:3)*100),
   timestamp=as.POSIXct("2018-12-21 00:01"))
context(st) <- "Level 1"

ev <- Event("Phred","test","message",
      timestamp=as.POSIXct("2018-12-21 00:01:01"),
      details=list("list"=list("one"=1,"two"=1:2),"vector"=(1:3)))

stopifnot(
getJS("state.context",st,ev)=="Level 1",
getJS("state.oldContext",st,ev)=="Level 0",
getJS("state.observables.numeric",st,ev)==12.5,
getJS("state.observables.char",st,ev)=="foo",
all.equal(getJS("state.observables.list",st,ev),list("one"="a","two"=2)),
getJS("state.observables.list.one",st,ev)=="a",
getJS("state.observables.vector[2]",st,ev)==200,
all.equal(getJS("state.flags.list",st,ev),list("one"=1,"two"="too")),
getJS("state.flags.list.two",st,ev)=="too",
getJS("state.flags.vector[3]",st,ev)==30
)

stopifnot(
getJS("state.timers.watch.running",st,ev)==FALSE,
getJS("state.timers.watch.time",st,ev)==as.difftime(0,units="secs")
)

timerTime(st,"watch",as.POSIXct("2018-12-21 00:01")) <-
      as.difftime(1,units="mins")
timerRunning(st,"watch", as.POSIXct("2018-12-21 00:01")) <- TRUE

stopifnot(
getJS("state.timers.watch.run",st,ev)==TRUE,
getJS("state.timers.watch.value",st,ev)==as.difftime(61,units="secs")
)
## Note that value of a running timer references difference between
## internal start time and current time as recorded by the event.


stopifnot(
getJS("event.verb",st,ev)=="test",
getJS("event.object",st,ev)=="message",
getJS("event.timestamp",st,ev)==as.POSIXct("2018-12-21 00:01:01"),
all.equal(getJS("event.data.list",st,ev),list("one"=1,"two"=1:2)),
all.equal(getJS("event.data.list.two",st,ev),1:2),
getJS("event.data.vector[3]",st,ev)==3
)

---

Does the processing for a new event.

Description

This function finds the appropriate rules for handling an event then runs them. It is the core function for the EIEngine.

Usage

handleEvent(eng, event)
processEvent(eng, state, event)

Arguments

eng	An object of class EIEngine which will process the event.
state	An object of class Status that will be updated by the rules processing the event.
event	An object of class Event that will be processed.

Details

The function processEvent first calls eng$findRules to find rules appropriate for the event. If no rules exist, it returns NULL (indicating that the state has not changed and does not need to be saved.). If there are rules, then the following functions are run in sequence:

1. runStatusRules: update internal status variables (flags and timers).

2. runObservableRules: update external status variables (observables).

3. runContextRules: check to see if the context (game level) has changed.

4. runTriggerRules: run rules to general messages for other processes if necessary.

5. runResetRules: resets the state for the beginning of a new context.

It then updates the timestamp and returns the modified status. If an error occurs during any of the steps, then an object of class try-error is returned (see withFlogging).

The function handleEvent first finds the current status for the user from the UserRecordSet linked to the engine. It then runs processEvent, and then if necessary it saves the updated status.

Value

The function processEvent returns NULL if there were no applicable rules, an object of class Status if the rules executed correctly and an object of class try-error if processing the rules generated an error.

The function handleEvent returns the current Status if the handling was successful and an object of class try-error in an error was generated.

Note

This function uses the flog.logger mechanism. The following information is reported at various thresholds:

TRACE

• The results of checkCondition are reported.

• The specific rules found from each query are reported.

• A message is logged as each rule is run.

DEBUG

• When an error occurs information about the state, event and rule where the error occurred as well a stack trace are logged.

• Each event is logged as it is processed.

• Rule searches are logged and the number of rules reported.

• A message is logged when each phase starts.

• A message is logged when the context changes.

INFO and above

If an error occurs the error is logged along with context information.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the rule system. https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

Almond, R. G., Steinberg, L. S., and Mislevy, R.J. (2002). Enhancing the design and delivery of Assessment Systems: A Four-Process Architecture. Journal of Technology, Learning, and Assessment, 1, http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671.

Almond, R. G., Shute, V. J., Tingir, S. and Rahimi, S. (2018). Identifying Observable Outcomes in Game-Based Assessments. Talk given at the 2018 Maryland Assessment Research Conference. Slides: https://education.umd.edu/file/11333/download?token=kmOIVIwi, Video: https://pluto.coe.fsu.edu/Proc4/Almond-Marc18.mp4.

MongoDB, Inc. (2018). The MongoDB 4.0 Manual. https://docs.mongodb.com/manual/.

See Also

Conditions and Predicates each have detailed descriptions. The functions checkCondition and executePredicate run the condition and predicate parts of the rule. The functions runRule and runTRule run the indivual rules, and the functions runTriggerRules, runStatusRules, runObservableRules, runResetRules, and runContextRules run sets of rules.

The functions testQuery and testQueryScript can be used to test that rule conditions function properly. The functions testPredicate and testPredicateScript can be used to test that predicates function properly. The functions testRule and testRuleScript can be used to test that rule conditions and predicates function properly together. The class RuleTest stores a rule test.

Other classes in the EIEvent system: EIEngine, Context, Status, Event, RuleTable.

Examples

## Not run: 
app <- "ecd://epls.coe.fsu.edu/P4test"
loglevel <- "DEBUG"
cleanFirst <- TRUE
eventFile <- "/home/ralmond/Projects/EvidenceID/c081c3.core.json"
## Adjust the path here as necessary
source("/usr/local/share/Proc4/EIini.R")
flog.appender(appender.file(logfile))
flog.threshold(loglevel)

## Setup Listeners
listeners <- lapply(names(EI.listenerSpecs),
                    function (ll) do.call(ll,EI.listenerSpecs[[ll]]))
names(listeners) <- names(EI.listenerSpecs)
if (interactive()) {
  cl <- new("CaptureListener")
  listeners <- c(listeners,cl=cl)
}
EIeng.params <-
  c(EI.config$EIEngine,
    EIeng.local[setdiff(names(EIeng.local),names(EI.config$EIEngine))])

  EIeng.params$listenerSet <-
    ListenerSet(sender= sub("<app>",sapp,EI.config$sender),
                dbname=EIeng.local$dbname, dburi=EIeng.local$dburi,
                listeners=listeners,
                colname=EI.config$lscolname)
  EIeng.params$app <- app

## Make the EIEngine
eng <- do.call(EIEngine,EIeng.params)

## Process Event file if supplied
if (!is.null(eventFile)) {
  system2("mongoimport",
          sprintf('-d 
          stdin=eventFile)
  ## Count the number of unprocessed events
  NN <- eng$eventdb()$count(buildJQuery(app=app(eng),processed=FALSE))
}

for (n in 1L:NN) {
  eve <- eng$fetchNextEvent()
  out <- handleEvent(eng,eve)
  eng$setProcessed(eve)
}

## End(Not run)

---

Loads a set of contexts from a matrix description

Description

The easiest way to load a set of Context objects is from a matrix of cross references. There are columns corresponding to the fixed fields of the context object (cid,number, name and doc). The other columns are logical variables that refer to context sets, which take on a true value when the context represented by the row belongs to the set represented by the column.

Usage

loadContexts(conmat, set, app)

Arguments

conmat	A matrix representing a collection of contexts, see details.
set	Either a ContextSet object, or a list to which the new contexts will be added.
app	A character scalar giving the name of the application; used in constructing the contexts.

Details

A context matrix is a data.frame with the following columns: cid (character, required), number (numeric, required), name (character, optional) and doc (character, required). The cid and number fields should also be unique across all the entries as they serve as indexes for the ContextSet object. The name column if omitted defaults to the cid (the difference is that the name is designed to be more human readable). The doc if omitted produces an empty documentation string. The remaining columns are logical and are indicators for set membership.

The following is an example context matrix:

	cid	number	name	doc	Set1	Subset1.1	Set2
[,1]	Set1	-100	"Set 1"	"A context set"	0	0	0
[,2]	Subset1.1	-110	"Subet 1.1"	"A subset of Set 1"	1	0	0
[,3]	Set2	-200	"Set 2"	"Another context set"	0	0	0
[,4]	Task1	100	"First Task"	"A task beloning to Set 1"	1	0	0
[,5]	Task1a	101	"First Task Variant"	"A task beloning to Subet 1.1"	1	1	0
[,6]	Task2	200	"Second Task"	"A task beloning to Set 2"	0	0	1

A couple things to note about this matrix. First, both actual contexts and context sets are represented (the are both represented internally by Context objects. The names of the extra columns correspond to the names of the context sets. If the database is used to store contexts, it recommended to give sets a negative number and actual contexts positive numbers. If a list is used, then the numbers need to be list indexes, and so should be low numbers.

Also, the belongsTo relationship is not transitive (or at least the transitive implications are not computed), so Task1a must have both set and subset memebership defined.

The code loops through the rows of the table, and creates a new context for each row. This is then added to the set argument. If an existing context of the same cid and number exists, it is replaced, otherwise a new one is added. If the context is replacing an old one, the old one should have both the same name and number as the replacement. The method for updateContext for a ContextSet object (called by loadContexts) checks for this inconsistency; the list method does not.

The matrix is to represent the entire context set, calling clearContexts before loading the matrix should eliminate conflicts.

Value

The modified set argument is returned. This is either a list or a ContextSet.

Note

The ContextSet class is a reference class. Therefore, this function destructively modifieds the database associated with the set.

The list is an ordinary R functional class, and the function behaves in a functional manner when called with a list as a set.

Author(s)

Russell Almond

References

The document “Rules Of Evidence” gives extensive documentation for the context system: https://pluto.coe.fsu.edu/Proc4/RulesOfEvidence.pdf.

See Also

Context describes the context object, and ContextSet describes the context set object.

The functions matchContext, updateContext and clearContexts are used to manipulate context sets.

Examples

## Not run:  # Requires Mongo DB
conmat <- read.csv(file.path(library(help="EIEvent")$path,"testScripts",
                "SampleContextSheet.csv"))

conmat1 <- conmat
conmat1$Number <- 1:nrow(conmat) ## List style needs small integer indexes.
setlist <- loadContexts(conmat1,list(),"ecd://epls.coe.fsu.edu/rga/test")
stopifnot(all.equal(names(setlist),as.character(conmat$CID)))

testSet <- newContextSet(app="ecd://epls.coe.fsu.edu/rga/test",
                      dbname="test",dburi="mongodb://localhost")

clearContexts(testSet)
loadContexts(conmat,testSet,"ecd://epls.coe.fsu.edu/rga/test")
c1 <- matchContext(28L,testSet)
c2 <- matchContext("Stairs",testSet)
stopifnot(all.equal(c1,c2))

## End(Not run)

---