Package 'Peanut'

Parameterized Bayesian Networks, Abstract Classes

Description

This provides support of learning conditional probability tables parameterized using CPTtools. This provides and object oriented layer on top of a CPTtools, to facilitate calculations with Parameterized models for Bayesian networks. Peanut is a collection of abstract classes and generic functions defining a protocol, with the intent that the protocol can be implemented with different Bayes net engines. The companion pacakge PNetica provides an implementation using Netica and RNetica.

Details

The DESCRIPTION file:

Package:	Peanut
Version:	0.9-3
Date:	2023/08/20
Title:	Parameterized Bayesian Networks, Abstract Classes
Author:	Russell Almond
Maintainer:	Russell Almond <[email protected]>
Authors@R:	person(given = "Russell", family = "Almond", role = c("aut", "cre"), email = "[email protected]", comment = c(ORCID = "0000-0002-8876-9337"))
Depends:	R (>= 3.0), CPTtools (>= 0.5), methods
Imports:	utils, futile.logger, jsonlite
Suggests:	PNetica, knitr, rmarkdown, tidyr, htmltools, shiny, shinyjs
VignetteBuilder:	knitr
Description:	This provides support of learning conditional probability tables parameterized using CPTtools. This provides and object oriented layer on top of a CPTtools, to facilitate calculations with Parameterized models for Bayesian networks. Peanut is a collection of abstract classes and generic functions defining a protocol, with the intent that the protocol can be implemented with different Bayes net engines. The companion pacakge PNetica provides an implementation using Netica and RNetica.
License:	Artistic-2.0
URL:	http://pluto.coe.fsu.edu/RNetica
Support:	c( 'Bill & Melinda Gates Foundation grant "Games as Learning/Assessment: Stealth Assessment" (#0PP1035331, Val Shute, PI)', 'National Science Foundation grant "DIP: Game-based Assessment and Support of STEM-related Competencies" (#1628937, Val Shute, PI)', 'National Science Foundation grant "Mathematical Learning via Architectural Design and Modeling Using E-Rebuild." (\#1720533, Fengfeng Ke, PI)', 'Institute of Educational Statistics Grant: "Exploring adaptive cognitive and affective learning support for next-generation STEM learning games." (#R305A170376-20, Val Shute and Russell Almond, PIs')
Repository:	https://ralmond.r-universe.dev
RemoteUrl:	https://github.com/ralmond/Peanut
RemoteRef:	HEAD
RemoteSha:	d6c4b8d0dd5286ab0e6332e286df5bd8f038676d

Peanut (a corruption of Parameterized network or Pnet) is an object oriented layer on top of the tools for constructing conditional probability tables for Bayesian networks in the CPTtools package. In particular, it defines a Pnode (parameterized node) object which stores all of the arguments necessary to use to the calcDPCTable function to build the conditional probability table for the node.

The Pnet object is a Bayesian network containing a number of Pnodes. It supports two key operations, BuildAllTables which sets the values of the conditional probabilities based on current parameters and GEMfit which adjusts the parameters to match a set of cases.

Like the DBI package, this class consists mostly of generic functions which need to be implement for specific Bayes net implementations. The package PNetica provides an implementation of the Peanut generic functions using the RNetica package. All of the Netica-dependent code is isolated in the PNetica package, to make it easier to create other implementations.

Index

Index of help topics:

BuildNetManifest        Builds a network manifest from a list of Pnets
BuildNodeManifest       Builds a table describing a set of Pnodes
BuildTable              Builds the conditional probability table for a
                        Pnode
CompensatoryGadget      Shiny gadget for editinging compensatory pnodes
DPCGadget               Shiny gadget for editinging compensatory pnodes
GEMfit                  Fits the parameters of a parameterized network
                        using the GEM algorithm
OffsetGadget            Shiny gadget for editinging (offset) conjuctive
                        and disjunctive pnodes
Omega2Pnet              Constructs a parameterized network from an
                        Omega matrix.
Peanut-package          Parameterized Bayesian Networks, Abstract
                        Classes
Pnet                    A Parameterized Bayesian network
Pnet-class              Class '"Pnet"'
Pnet2Omega              Constructs an Omega matrix from a parameterized
                        network.
Pnet2Qmat               Makes an augmented Q-matrix from a collection
                        of parameterized nets
PnetAdjoin              Merges (or separates) two Pnets with common
                        variables
PnetCompile             Compiles a Parameterized Bayesian Network
PnetFindNode            Finds nodes in a parameterized network.
PnetHub                 Returns the name of the hub net if this is a
                        spoke net.
PnetMakeStubNodes       Creates (or removes) references to nodes in a
                        network
PnetName                Gets or Sets the name of a Netica network.
PnetPathname            Returns the path associated with a network.
PnetPnodes              Returns a list of Pnodes associated with a Pnet
PnetPriorWeight         Gets the weight to be associated with the prior
                        table during EM learning
PnetSerialize           Writes/restores network from a string.
PnetTitle               Gets the title or comments associated with a
                        parameterized network.
PnetWarehouse-class     Class '"PnetWarehouse"'
Pnode                   A Parameterized Bayesian network node
Pnode-class             Class '"Pnode"'
PnodeBetas              Access the combination function slope
                        parameters for a Pnode
PnodeDefaultAlphas      Reshapes alpha or beta vector based on rule and
                        parents
PnodeEAP                Pnode Marginal Statistics
PnodeEvidence           Accesses the value to which a given node has
                        been instantiated.
PnodeLabels             Lists or changes the labels associated with a
                        parameterize node.
PnodeLink               Accesses the link function associated with a
                        Pnode
PnodeLinkScale          Accesses the link function scale parameter
                        associated with a Pnode
PnodeLnAlphas           Access the combination function slope
                        parameters for a Pnode
PnodeMin                A minimal Pnode class for use in interface
                        construction.
PnodeName               Gets or sets name of a parameterized node.
PnodeParentTvals        Fetches a list of numeric variables
                        corresponding to parent states
PnodeParents            Gets or sets the parents of a parameterized
                        node.
PnodePostWeight         Fetches the posterior weight associated with a
                        node
PnodeProbs              Gets or sets the conditional probability table
                        associated with a Netica node.
PnodeQ                  Accesses a state-wise Q-matrix associated with
                        a Pnode
PnodeRules              Accesses the combination rules for a Pnode
PnodeStateTitles        Accessors for the titles and descriptions
                        associated with states of a parameterized node.
PnodeStateValues        Accesses the numeric values associated with the
                        state of a parameterized node.
PnodeStates             Accessor for states of a parameterized node.
PnodeTitle              Gets the title or Description associated with a
                        parameterized node node.
PnodeWarehouse-class    Class '"PnodeWarehouse"'
Qmat2Pnet               Makes or adjusts parameterized networks based
                        on augmented Q-matrix
RegressionGadget        Shiny gadget for editinging compensatory pnodes
Statistic               Key functions for the Statistics class
Statistic-class         Class '"Statistic"'
Warehouse               A cache for Pnets or Pnodes
WarehouseCopy           Copies and object in the warehouse
WarehouseManifest       Manipulates the manifest for a warehouse
calcExpTables           Calculate expected tables for a parameterized
                        network
calcPnetLLike           Calculates the log likelihood for a set of data
                        under a Pnet model
flattenStats            Flattens a statistic list into a numeric or
                        character vector.
flog.try                Trys to execute an expression with errors
                        logged.
is.legal.name           Checks to see if names are valid for objects in
                        warehouse.
isPnodeContinuous       Functions for handling continuous nodes.
maxAllTableParams       Find optimal parameters of Pnet or Pnode to
                        match expected tables
topsort                 Topologically sorts the rows and columns of an
                        Omega matrix

Author(s)

Russell Almond

Maintainer: Russell Almond <[email protected]>

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

Almond, R. G., Mislevy, R. J., Steinberg, L. S., Yan, D. and Williamson, D. M. (2015) Bayesian Networks in Educational Assessment. Springer. (ISBN 978-1-4939-2124-9).

See Also

PNetica

An implementation of the Peanut object model using RNetica.

CPTtools

A collection of implementation independent Bayes net utilities.

Examples

## Not run: 
library(PNetica) ## Requires implementation
sess <- NeticaSession()
startSession(sess)

## Building CPTs
tNet <- CreateNetwork("TestNet",session=sess)


theta1 <- NewDiscreteNode(tNet,"theta1",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta1) <- effectiveThetas(NodeNumStates(theta1))
NodeProbs(theta1) <- rep(1/NodeNumStates(theta1),NodeNumStates(theta1))
theta2 <- NewDiscreteNode(tNet,"theta2",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta2) <- effectiveThetas(NodeNumStates(theta2))
NodeProbs(theta2) <- rep(1/NodeNumStates(theta2),NodeNumStates(theta2))

partial3 <- NewDiscreteNode(tNet,"partial3",
                            c("FullCredit","PartialCredit","NoCredit"))
PnodeParents(partial3) <- list(theta1,theta2)

partial3 <- Pnode(partial3,Q=TRUE, link="partialCredit")
PnodePriorWeight(partial3) <- 10
BuildTable(partial3)

## Set up so that first skill only needed for first transition, second
## skill for second transition; adjust alphas to match
PnodeQ(partial3) <- matrix(c(TRUE,TRUE,
                             TRUE,FALSE), 2,2, byrow=TRUE)
PnodeLnAlphas(partial3) <- list(FullCredit=c(-.25,.25),
                                PartialCredit=0)
BuildTable(partial3)
partial4 <- NewDiscreteNode(tNet,"partial4",
                            c("Score4","Score3","Score2","Score1"))
NodeParents(partial4) <- list(theta1,theta2)
partial4 <- Pnode(partial4, link="partialCredit")
PnodePriorWeight(partial4) <- 10

## Skill 1 used for first transition, Skill 2 used for second
## transition, both skills used for the 3rd.

PnodeQ(partial4) <- matrix(c(TRUE,TRUE,
                             FALSE,TRUE,
                             TRUE,FALSE), 3,2, byrow=TRUE)
PnodeLnAlphas(partial4) <- list(Score4=c(.25,.25),
                                Score3=0,
                                Score2=-.25)
BuildTable(partial4)

## Fitting Model to data

irt10.base <- ReadNetworks(system.file("testnets", "IRT10.2PL.base.dne",
                                       package="PNetica"),
                           session=sess)
irt10.base <- as.Pnet(irt10.base)  ## Flag as Pnet, fields already set.
irt10.theta <- PnetFindNode(irt10.base,"theta")
irt10.items <- PnetPnodes(irt10.base)
## Flag items as Pnodes
for (i in 1:length(irt10.items)) {
  irt10.items[[i]] <- as.Pnode(irt10.items[[i]])
  ## Add node to list of observed nodes
  PnodeLabels(irt10.items[[1]]) <-
     union(PnodeLabels(irt10.items[[1]]),"onodes")

}


casepath <- system.file("testdat", "IRT10.2PL.200.items.cas", package="PNetica")


BuildAllTables(irt10.base)
PnetCompile(irt10.base) ## Netica requirement

item1 <- irt10.items[[1]]
priB <- PnodeBetas(item1)
priA <- PnodeAlphas(item1)
priCPT <- NodeProbs(item1)

gemout <- GEMfit(irt10.base,casepath)


DeleteNetwork(irt10.base)
DeleteNetwork(tNet)
stopSession(sess)

## End(Not run)

---

Builds a network manifest from a list of Pnets

Description

A network manifest is a table of meta data about a colleciton of networks. Each line corresponds to the specific network. This manifest can be used by a network warehouse (Warehouse) to recreate the network on demand.

Usage

BuildNetManifest(Pnetlist)

Arguments

Pnetlist

A list of Pnet objects which will appear in the network manifest.

Details

A network manifest is a table (data frame) which describes a collection of networks. It contains meta-data about the networks, and not the information about the nodes, contained in the node manifest (BuildNodeManifest) or the relaitonships between the nodes which is contained in the $Q$-matrix (Pnet2Qmat) or the $\Omega$-Matrix (Pnet2Omega). The role of the net manifest is to be used as to create a Net Warehouse which is an argument to the Qmat2Pnet and Omega2Pnet commands, creating networks as they are referenced.

The “Name” column of the table contains the network name and is a key to the table (so it should be unique). It corresponds to PnetName. The “Title” (PnetTitle) and “Description” (PnetDescription) columns contain optional meta-data about the node. The “Pathname” (PnetPathname) column contiains the location of the file to which the network should be written and from which it can be read. The “Hub” (PnetHub) is for spoke models (evidence models) some of whose variables are defined in a hub network. This the network in question is meant to be a spoke, then this field points at the corresponding hub.

Value

An object of type data.frame where the columns have the following values.

Name	A character value giving the name of the network. This should be unique for each row and normally must conform to variable naming conventions. Corresponds to the function PnetName.
Title	An optional character value giving a longer human readable name for the netowrk. Corresponds to the function PnetTitle.
Hub	If this model is incomplete without being joined to another network, then the name of the hub network. Otherwise an empty character vector. Corresponds to the function PnetHub.
Pathname	The location of the file from which the network should be read or to which it should be written. Corresponds to the function PnetPathname.
Description	An optional character value documenting the purpose of the network. Corresponds to the function PnetDescription.

Note that the name column is regarded as a primary key to the table.

Logging

BuildNetManifest uses the flog.logger mechanism to log progress. To see progress messages, use flog.threshold(DEBUG) (or TRACE).

Author(s)

Russell Almond

References

Almond, R. G. (presented 2017, August). Tabular views of Bayesian networks. In John-Mark Agosta and Tomas Singlair (Chair), Bayeisan Modeling Application Workshop 2017. Symposium conducted at the meeting of Association for Uncertainty in Artificial Intelligence, Sydney, Australia. (International) Retrieved from http://bmaw2017.azurewebsites.net/

See Also

Network functions called to find network data: PnetName, PnetTitle, PnetPathname, PnetHub, PnetDescription

Used in the construction of Network Warehouses (see WarehouseManifest).

Similar to the function BuildNodeManifest.

Examples

## This provides an example network manifest.
netman1 <- read.csv(system.file("auxdata", "Mini-PP-Nets.csv", 
                                package="Peanut"),
                    row.names=1, stringsAsFactors=FALSE)

## Not run: 
library(PNetica)  ## Example requires PNetica

sess <- NeticaSession()
startSession(sess)

netpath <- System.file("testnets",package="PNetica")
netnames <- paste(c("miniPP-CM","PPcompEM","PPconjEM","PPtwostepEM",
                  "PPdurAttEM"),"dne",sep=".")


Nets <- ReadNetworks(file.path(netpath,netnames),
                     session=sess)

netman <- BuildNetManifest(Nets)
stopifnot(all.equal(netman,netman1))

## BNWarehouse is the PNetica Net Warehouse.
Nethouse <- BNWarehouse(manifest=netman1,session=sess,key="Name")

stopSession(sess)


## End(Not run)

---

Builds a table describing a set of Pnodes

Description

A node manifest is a table where each line describes one state of a node in a Bayesian network. As a node manifest may contain nodes from more than one network, the key for the table is the first two columns: “Model” and “NodeName”. The primary purpose is that this can be given to a Node Warehouse to create nodes on demand.

Usage

BuildNodeManifest(Pnodelist)

Arguments

Pnodelist

A list of Pnode objects from which the table will be built.

Details

A node manifest is a table (data frame) which describes a collection of nodes. It contains mostly meta-data about the nodes, and not the information about the relaitonships between the nodes which is contained in the $Q$-matrix (Pnet2Qmat) or the $\Omega$-Matrix (Pnet2Omega). The role of the node manifest is to be used as to create a Node Warehouse which is an argument to the Qmat2Pnet and Omega2Pnet commands, creating nodes as they are referenced. Hence it contains the information about the node which is not part of the $Q$ or $\Omega$ matrix.

The $Q$-matrix can span multiple Bayesian networks. The same variable can appear with the same name but slightly different definitions in two different networks. Consequently, the key for this table is the “Model” and “NodeName” columns (usually the first two). The function WarehouseData when applied to a node warehouse should have a key of length 2 (model and node name) and will return multiple lines, one line corresponding to each state of the data frame.

The columns “ModelHub”, “NodeTitle”, “NodeDescription” and “NodeLabels” provide meta-data about the node. They may be missing empty strings, indicating that meta-data is unavailable.

The columns “Nstates” and “StateName” are required. The number of states should be an integer (2 or greater) and there should be as many rows with this model and node name as there are states. Each should have a unique value for “StateName”. The “StateTitle”, “StateDescription” and “StateValue” are optional, although if the variable is to be used as a parent variable, it is strongly recommended to set the state values.

Value

An object of class data.frame with the following columns.

Node-level Key Fields:

Model	A character value giving the name of the Bayesian network to which this node belongs. Corresponds to the value of PnodeNet.
NodeName	A character value giving the name of the node. All rows with the same value in the model and node name columns are assumed to reference the same node. Corresponds to the value of PnodeName.

Node-level Fields:

ModelHub	If this is a spoke model (meant to be attached to a hub) then this is the name of the hub model (i.e., the name of the proficiency model corresponding to an evidence model). Corresponds to the value of PnetHub(PnodeNet(node)).
NodeTitle	A character value containing a slightly longer description of the node, unlike the name this is not generally restricted to variable name formats. Corresponds to the value of PnodeTitle.
NodeDescription	A character value describing the node, meant for human consumption (documentation). Corresponds to the value of PnodeDescription.
NodeLabels	A comma separated list of identifiers of sets which this node belongs to. Used to identify special subsets of nodes (e.g., high-level nodes or observeable nodes). Corresponds to the value of PnodeLabels.

State-level Key Fields:

Continuous	A logical value. If true, the variable will be continuous, with states corresponding to ranges of values. If false, the variable will be discrete, with named states.
Nstates	The number of states. This should be an integer greater than or equal to 2. Corresponds to the value of PnodeNumStates.
StateName	The name of the state. This should be a string value and it should be different for every row within the subset of rows corresponding to a single node. Corresponds to the value of PnodeStates.

State-level Fields:

StateTitle	A longer name not subject to variable naming restrictions. Corresponds to the value of PnodeStateTitles.
StateDescription	A human readable description of the state (documentation). Corresponds to the value of PnodeStateDescriptions.
StateValue	A real numeric value assigned to this state. PnodeStateValues. Note that this has different meaning for discrete and continuous variables. For discrete variables, this associates a numeric value with each level, which is used in calculating the PnodeEAP and PnodeSD functions. In the continuous case, this value is ignored and the midpoint between the “LowerBounds” and “UpperBounds” are used instead.
LowerBound	This servers as the lower bound for each partition of the continuous variagle. -Inf is a legal value for the first or last row.
UpperBound	This is only used for continuous variables, and the value only is needed for one of the states. This servers as the upper bound of range each state. Note the upper bound needs to match the lower bounds of the next state. Inf is a legal value for the first or last row.

Logging

BuildNodeManifest uses the flog.logger mechanism to log progress. To see progress messages, use flog.threshold(DEBUG) (or TRACE).

Continuous Variables

Peanut (following Netica) treats continuous variables as discrete variables whose states correspond to ranges of an underlying continuous variable. Unfortunately, this overlays the meaning of PnodeStateValues, and consequently the “StateValue” column.

Discrete Variables. The states of the discrete variables are defined by the “StateName” fields. If values are supplied in “StateValue”, then these values are used in calculating expected a posteriori statistics, PnodeEAP() and PnodeSD(). The “LowerBound” and “UpperBound” fields are ignored.

Continuous Variables. The states of the continuous variable are defined by breaking the range up into a series of intervals. Right now the intervals must be adjacent (the upper bound of one must match the lower bound of the next) and cannot overlap. This is done by supplying a “LowerBound” and “UpperBound” for each state. If the upper and lower bounds do not match, then an error is signaled.

Author(s)

Russell Almond

References

Almond, R. G. (presented 2017, August). Tabular views of Bayesian networks. In John-Mark Agosta and Tomas Singlair (Chair), Bayeisan Modeling Application Workshop 2017. Symposium conducted at the meeting of Association for Uncertainty in Artificial Intelligence, Sydney, Australia. (International) Retrieved from http://bmaw2017.azurewebsites.net/

See Also

Node functions called to find node meta-data: PnodeName, PnodeTitle, PnodeNet, PnetHub, PnodeDescription, PnodeLabels. PnodeNumStates, PnodeStateTitles, PnodeStateDescriptions, PnodeStateValues.

Used in the construction of Network Warehouses (see WarehouseManifest).

Similar to the function BuildNetManifest.

Examples

## This expression provides an example Node manifest
nodeman1 <- read.csv(system.file("auxdata", "Mini-PP-Nodes.csv",
                                package="Peanut"),
                     row.names=1,stringsAsFactors=FALSE)

## Not run: 
library(PNetica) ## Requires PNetica
sess <- NeticaSession()
startSession(sess)

netpath <- system.file("testnets",package="PNetica")
netnames <- paste(c("miniPP-CM","PPcompEM","PPconjEM","PPtwostepEM",
                  "PPdurAttEM"),"dne",sep=".")

Nets <- ReadNetworks(file.path(netpath,netnames),
                     session=sess)

CM <- Nets[[1]]
EMs <- Nets[-1]

nodeman <- BuildNodeManifest(lapply(NetworkAllNodes(CM),as.Pnode))

for (n in 1:length(EMs)) {
  nodeman <- rbind(nodeman,
                    BuildNodeManifest(lapply(NetworkAllNodes(EMs[[n]]),
                                             as.Pnode)))
}

## Need to ensure that labels are in cannonical order only for the
## purpose of testing
nodeman[,6] <- sapply(strsplit(nodeman[,6],","),
      function(l) paste(sort(l),collapse=","))
nodeman1[,6] <- sapply(strsplit(nodeman1[,6],","),
      function(l) paste(sort(l),collapse=","))

stopifnot(all.equal(nodeman,nodeman1))

## This is the node warehouse for PNetica
Nodehouse <- NNWarehouse(manifest=nodeman1,
                         key=c("Model","NodeName"),
                         session=sess)
phyd <- WarehouseData(Nodehouse,c("miniPP_CM","Physics"))
p3 <- MakePnode.NeticaNode(CM,"Physics",phyd)

attd <- WarehouseData(Nodehouse,c("PPdurAttEM","Attempts"))
att <- MakePnode.NeticaNode(Nets[[5]],"Attempts",attd)

durd <- WarehouseData(Nodehouse,c("PPdurAttEM","Duration"))
dur <- MakePnode.NeticaNode(Nets[[5]],"Duration",durd)


stopSession(sess)


## End(Not run)

---

Builds the conditional probability table for a Pnode

Description

The function BuildTable builds the conditional probability table for a Pnode object, and sets the prior weight for the node using the current values of parameters. It sets these in the Bayesian network object as appropriate for the implementation. The expression BuildAllTables(net) builds tables for all of the nodes in PnetPnodes(net).

Usage

BuildTable(node)
BuildAllTables(net, debug=FALSE)

Arguments

node	A Pnode object whose table is to be built.
net	A Pnet object for whom the tables are needed to be built.
debug	A logical scalar. If true then recover is called after an error, so that the node in question can be inspected.

Details

The fields of the Pnode object correspond to the arguments of the calcDPCTable function. The output conditional probability table is then set in the node object in an implementation dependent way. Similarly, the current value of GetPriorWeight is used to set the weight that the prior table will be given in the EM algorithm.

Value

The node or net argument is returned invisibly. As a side effect the conditional probability table and prior weight of node (or a collection of nodes) is modified.

Logging and Debug Mode

As of version 0.6-2, the meaning of the debug argument is changed. In the new version, the flog.logger mechanism is used for progress reports, and error reporting. In particular, setting flog.threshold(DEBUG) (or TRACE) will cause progress reports to be sent to the logging output.

The debug argument has been repurposed. It now call recover when the error occurs, so that the problem can be debugged.

Note

The function BuildTable is an abstract generic function, and it needs a specific implementation. See the PNetica-package for an example.

Author(s)

Russell Almond

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

See Also

Pnode, PnodeProbs, PnodeQ, PnodePriorWeight, PnodeRules, PnodeLink, PnodeLnAlphas, PnodeAlphas, PnodeBetas, PnodeLinkScale,GetPriorWeight, calcDPCTable

In many implementations, it will be necessary to run PnetCompile after building the tables.

Examples

## Not run: 

## This is the implementation of BuildTable in Netica.  The [<- and
## NodeExperience functions are part of the RNetica implementation.

BuildTable.NeticaNode <- function (node) {
  node[] <- calcDPCFrame(ParentStates(node),PnodeStates(node),
                          PnodeLnAlphas(node), PnodeBetas(node),
                          PnodeRules(node),PnodeLink(node),
                          PnodeLinkScale(node),PnodeQ(node),
                          PnodeParentTvals(node))
  NodeExperience(node) <- GetPriorWeight(node)
  invisible(node)
}

## This is a simplified implementation of BuildAllTables
## (The full implementation adds logging and error handling.)
BuildAllTables <- function (net) {
  lapply(PnetPnodes(net),BuildTable)
  invisible(net)
}



## End(Not run)

---

Calculate expected tables for a parameterized network

Description

The performs the E-step of the GEM algorithm by running the internal EM algorithm of the host Bayes net package on the cases. After this is run, the posterior parameters for each conditional probability distribution should be the expected cell counts, that is the expected value of the sufficient statistic, for each Pnode in the net.

Usage

calcExpTables(net, cases, Estepit = 1, tol = sqrt(.Machine$double.eps))

Arguments

net	A Pnet object
cases	An object representing a set of cases. Note the type of object is implementation dependent. It could be either a data frame providing cases or a filename for a case file.
Estepit	An integer scalar describing the number of steps the Bayes net package should take in the internal EM algorithm.
tol	A numeric scalar giving the stopping tolerance for the Bayes net package internal EM algorithm.

Details

The GEMfit algorithm uses a generalized EM algorithm to fit the parameterized network to the given data. This loops over the following steps:

E-step

Run the internal EM algorithm of the Bayes net package to calculate expected tables for all of the tables being learned. The function calcExpTables carries out this step.

M-step

Find a set of table parameters which maximize the fit to the expected counts by calling mapDPC for each table. The function maxAllTableParams does this step.

Update CPTs

Set all the conditional probability tables in the network to the new parameter values. The function BuildAllTables does this.

Convergence Test

Calculate the log likelihood of the cases under the new parameters and stop if no change. The function calcPnetLLike calculates the log likelihood.

The function calcExpTables performs the E-step. It assumes that the native Bayes net class which net represents has a function which does EM learning with hyper-Dirichlet priors. After this internal EM algorithm is run, then the posterior should contain the expected cell counts that are the expected value of the sufficient statistics, i.e., the output of the E-step. Note that the function maxAllTableParams is responsible for reading these from the network.

The internal EM algorithm should be set to use the current value of the conditional probability tables (as calculated by BuildTable(node) for each node) as a starting point. This starting value is given a prior weight of GetPriorWeight(node). Note that some Bayes net implementations allow a different weight to be given to each row of the table. The prior weight counts as a number of cases, and should be scaled appropriately for the number of cases in cases.

The parameters Estepit and tol are passed to the internal EM algorithm of the Bayes net. Note that the outer EM algorithm assumes that the expected table counts given the current values of the parameters, so the default value of one is sufficient. (It is possible that a higher value will speed up convergence, the parameter is left open for experimentation.) The tolerance is largely irrelevant as the outer EM algorithm does the tolerance test.

Value

The net argument is returned invisibly.

As a side effect, the internal conditional probability tables in the network are updated as are the internal weights given to each row of the conditional probability tables.

Note

The function calcExpTables is an abstract generic functions, and it needs specific implementations. See the PNetica-package for an example.

This function assumes that the host Bayes net implementation (e.g., RNetica-package): (1) net has an EM learning function, (2) the EM learning supports hyper-Dirichlet priors, (3) it is possible to recover the hyper-Dirichlet posteriors after running the internal EM algorithm.

Author(s)

Russell Almond

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

See Also

Pnet, GEMfit, calcPnetLLike, maxAllTableParams

Examples

## Not run: 

library(PNetica) ## Need a specific implementation
sess <- NeticaSession()
startSession(sess)

irt10.base <- ReadNetworks(system.file("testnets","IRT10.2PL.base.dne",
                                       package="Peanut"),
                           session=sess)
irt10.base <- as.Pnet(irt10.base)  ## Flag as Pnet, fields already set.
irt10.theta <- PnetFindNode(irt10.base,"theta")
irt10.items <- PnetPnodes(irt10.base)
## Flag items as Pnodes
for (i in 1:length(irt10.items)) {
  irt10.items[[i]] <- as.Pnode(irt10.items[[i]])
  ## Add node to list of observed nodes
  PnodeLabels(irt10.items[[1]]) <-
     union(PnodeLabels(irt10.items[[1]]),"onodes")
}
PnetCompile(irt10.base) ## Netica requirement

casepath <- system.file("testdat","IRT10.2PL.200.items.cas", package="PNetica")


item1 <- irt10.items[[1]]

priorcounts <- sweep(PnodeProbs(item1),1,GetPriorWeight(item1),"*")

calcExpTables(irt10.base,casepath)

postcounts <- sweep(PnodeProbs(item1),1,PnodePostWeight(item1),"*")

## Posterior row sums should always be larger.
stopifnot(
  all(apply(postcounts,1,sum) >= apply(priorcounts,1,sum))
)

DeleteNetwork(irt10.base)
stopSession(sess)

## End(Not run)

---

Calculates the log likelihood for a set of data under a Pnet model

Description

The function calcPnetLLike calculates the log likelihood for a set of data contained in cases using the current values of the conditional probability table in a Pnet. If it is called after a call to BuildAllTables(net) this will be the current value of the parameters.

Usage

calcPnetLLike(net, cases)

Arguments

net	A Pnet object representing a parameterized network.
cases	An object representing a set of cases. Note the type of object is implementation dependent. It could be either a data frame providing cases or a filename for a case file.

Details

This function provides the convergence test for the GEMfit algorithm. The Pnet represents a model (with parameters set to the value used in the current iteration of the EM algorithm) and cases a set of data. This function gives the log likelihood of the data.

This is a generic function shell. It is assumed that either (a) the native Bayes net implementation provides a way of calculating the log likelihood of a set of cases, or (b) it provides a way of calculating the likelihood of a single case, and the log likelihood of the case set can be calculated though iteration. In either case, the value of cases is implementation dependent. In PNetica-package the cases argument should be a filename of a Netica case file (see write.CaseFile).

Value

A numeric scalar giving the log likelihood of the data in the case file.

Note

The function calcPnetLLike is an abstract generic functions, and it needs specific implementations. See the PNetica-package for an example.

Author(s)

Russell Almond

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

See Also

Pnet, GEMfit, calcExpTables, maxAllTableParams

Examples

## Not run: 

library(PNetica) ## Need a specific implementation
sess <- NeticaSession()
startSession(sess)

irt10.base <- ReadNetworks(system.file("testnets","IRT10.2PL.base.dne",
                                package="PNetica"),
                           session=sess)
irt10.base <- as.Pnet(irt10.base)  ## Flag as Pnet, fields already set.
irt10.theta <- PnetFindNode(irt10.base,"theta")
irt10.items <- PnetPnodes(irt10.base)
## Flag items as Pnodes
for (i in 1:length(irt10.items)) {
  irt10.items[[i]] <- as.Pnode(irt10.items[[i]])
  ## Add node to list of observed nodes
  PnodeLabels(irt10.items[[1]]) <-
     union(PnodeLabels(irt10.items[[1]]),"onodes")
}
PnetCompile(irt10.base) ## Netica requirement

casepath <- system.file("testdat","IRT10.2PL.200.items.cas",
                                package="PNetica")

llike <- calcPnetLLike(irt10.base,casepath)

DeleteNetwork(irt10.base)
stopSession(sess)

## End(Not run)

---

Shiny gadget for editinging compensatory pnodes

Description

These functions open a shiny application (in a browser window or other location) for editing a Pnode object. To reduce the complexity, the display assumes that PnodeLink(pnode) is partialCredit or gradedResponse, and that PnodeLink(pnode) is Compensatory (Conjunctive or Disjunctive are also possibilities, but usually, the OffsetGadget is a better parameterization for these rules).

Usage

MakeCompensatoryGadget(pnode, color = "firebrick")
CompensatoryGadget(pnode, color="firebrick",viewer=shiny::paneViewer())

Arguments

pnode	A Pnode object to be modified.
color	A base color to use for barcharts (see barchart.CPF). Execute colors() for a list of choices.
viewer	This is passed to the viewer argument of shiny::runGadget.

Details

The CompensatoryGadget assumes that:

• The link function is partialCredit or gradedResponse.

• There is a single rule for all states, and PnodeQ(pnode)=TRUE.

• One of the multiple-a rules: Compensatory, Conjunctive or Disjunctive is used, so that there is one alpha for each parent.

• There is one beta for each state except the last, which is a reference state.

It is most useful for compensatory models.

Value

The function MakeCompensatoryGadget returns a list of two functions, ui and server. These are meant to be passed to shiny::runApp to generate the actual app.

The function CompensatoryGadget will return the pnode object or throw a ‘Cancel-Error’.

Note

Although the addition of the 'MakeCompensatoryGadget' was specifically designed to allow the embedding of the gadget in Rmarkdown shiny documents, the default Rmarkdown layout algorithms do not work properly.

Adding the following markdown code:

```{css, echo = FALSE}
.shiny-frame{height: 1000px;}
```  
   

will adjust the space allocated for the gadget to 1000 pixels, allowing for sufficient room for display.

Author(s)

Russell Almond

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

See Also

Pnode, calcDPCFrame, barchart.CPF

OffsetGadget, RegressionGadget, DPCGadget

Examples

## Not run: 
library(PNetica) ## Requires implementation
sess <- NeticaSession()
startSession(sess)

tNet <- CreateNetwork("TestNet",sess)

theta1 <- NewDiscreteNode(tNet,"theta1",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta1) <- effectiveThetas(PnodeNumStates(theta1))
PnodeProbs(theta1) <- rep(1/PnodeNumStates(theta1),PnodeNumStates(theta1))
theta2 <- NewDiscreteNode(tNet,"theta2",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta2) <- effectiveThetas(PnodeNumStates(theta2))
PnodeProbs(theta2) <- rep(1/PnodeNumStates(theta1),PnodeNumStates(theta2))

## CompensatoryGadget

partial3 <- NewDiscreteNode(tNet,"partial3",
                            c("FullCredit","PartialCredit","NoCredit"))
PnodeParents(partial3) <- list(theta1,theta2)

## Usual way to set rules is in constructor
partial3 <- Pnode(partial3,rules="Compensatory", link="partialCredit")
PnodePriorWeight(partial3) <- 10
BuildTable(partial3)

partial3 <- CompensatoryGadget(partial3)

## This expression can be used inside an Rmarkdown document
gadget <- MakeCompensatoryGadget(partial3)
shinyApp(gadget$ui,gadget$server,options(height=2000))

DeleteNetwork(tNet)
stopSession(sess)

## End(Not run)

---

Shiny gadget for editinging compensatory pnodes

Description

These functions open a shiny application (in a browser window or other location) for editing a Pnode object. This is the most complex version taking advantage of all of the complex choices available using the partial credit link (i.e., PnodeLink(pnode) is partialCredit). In particular, it allows specification of different combination rules for each state transition. The rules can be either the multiple-a type (e.g., Compensatory) or muliptle-b type (e.g., OffsetConjunctive or OffsetDisjunctive). It even allows for a subset of parents to be used in each transition by specifying an inner Q-matrix (though PnodeQ). To manage this complexity, each transition is displayed on a separate tab of the interface.

Usage

MakeDPCGadget(pnode, color = "steelblue")
DPCGadget(pnode, color="steelblue",viewer=shiny::paneViewer())

Arguments

pnode	A Pnode object to be modified.
color	A base color to use for barcharts (see barchart.CPF). Execute colors() for a list of choices.
viewer	This is passed to the viewer argument of shiny::runGadget.

Details

The DPCGadget assumes that:

• The link function is partialCredit.

• There is a list of rules, one for each state transition (i.e., one for all states except the last and lowest value. Alternatively, if a single value is given it is used for all transitions.

• The value of PnodeQ(pnode) is a logical matrix with rows corresponding to state transitions and columns corresponding to parent variables. If any cell value is false, that parent variable is not used to calculate the effective theta for that cell transition. As a special case, if PnodeQ(pnode) equals TRUE, then it is considered to be a matrix with all elements true.

Both PnodeAlphas(node) and PnodeBetas(node) should be a list of vectors. The length of each vector should be either one or the number of relevant (after filtering with the inner Q-matrix) parent variables. Which it needs to be depends on whether the rule for that transition is a multiple-a type (alphas should match number of parents) or multiple-b type (betas should match number of parents). In either case, if a single value is given and a longer list is expected, it will be replicated across the parents. The length of the lists should match the state transitions (with the first one corresponding to the transition from the 2nd highest state to the highest, the second the transition to the 2nd highest state and so forth). No entry is needed for the lowest state. Once again, if a single vector is given in place of the list, it will be replicated as needed.

To recap, the outer (list) structure corresponds to state transitions and the inner (vector) structure corresponds to the parent variables. It is recommended to use labeled vectors and lists to annotate the structure.

Value

The function MakeDPCGadget returns a list of two functions, ui and server. These are meant to be passed to shiny::runApp to generate the actual app.

The function DPCGadget will return the pnode object or throw a ‘Cancel-Error’.

Author(s)

Russell Almond

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

See Also

Pnode, calcDPCFrame, barchart.CPF

OffsetGadget, RegressionGadget, CompensatoryGadget

Examples

## Not run: 
library(PNetica) ## Requires implementation
sess <- NeticaSession()
startSession(sess)

tNet <- CreateNetwork("TestNet",sess)

theta1 <- NewDiscreteNode(tNet,"theta1",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta1) <- effectiveThetas(PnodeNumStates(theta1))
PnodeProbs(theta1) <- rep(1/PnodeNumStates(theta1),PnodeNumStates(theta1))
theta2 <- NewDiscreteNode(tNet,"theta2",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta2) <- effectiveThetas(PnodeNumStates(theta2))
PnodeProbs(theta2) <- rep(1/PnodeNumStates(theta1),PnodeNumStates(theta2))

## DPCGadget

partial3 <- NewDiscreteNode(tNet,"partial3",
                            c("FullCredit","PartialCredit","NoCredit"))
PnodeParents(partial3) <- list(theta1,theta2)

## Usual way to set rules is in constructor
partial3 <- Pnode(partial3,rules="Compensatory", link="partialCredit")
PnodePriorWeight(partial3) <- 10
BuildTable(partial3)

partial3 <- DPCGadget(partial3)

## This expression can be used inside an Rmarkdown document
gadget <- MakeDPCGadget(partial3)
shinyApp(gadget$ui,gadget$server,options(height=2000))

## More complex example showing off some of the options:
## Set up so that first skill only needed for first transition, second
## skill for second transition; Adjust alphas to match
PnodeQ(partial3) <- matrix(c(TRUE,TRUE,
                             TRUE,FALSE), 2,2, byrow=TRUE)
PnodeLnAlphas(partial3) <- list(FullCredit=c(theta1=-.25,theta2=.25),
                                PartialCredit=0)

partial3 <- DPCGadget(partial3)


DeleteNetwork(tNet)
stopSession(sess)

## End(Not run)

---

Flattens a statistic list into a numeric or character vector.

Description

Margin statitics are list valued, however, it is often useful to break them down into separate statistics for each state. In particular, this facilitates putting the data into a data.frame format for further analysis.

Usage

flattenStats(statlist)

Arguments

statlist

A named list contatining the statsitics.

Details

Vector valued tatistics produces by PnodeMargin are replaced by multiple statistics with the state name appended (after a dot) to the variable name.

Scalar valued statistics are left as is.

Value

A vector with the statistics. If all statistics are numeric, it will be a numeric vector. If some are character valued (e.g., PnodeMode), it will be a character vector.

Author(s)

Russell Almond

See Also

PnodeMargin, PnodeEAP, PnodeMode,

Examples

slist <- list("Physics_EAP"=.3,
              "Physics_Margin"=c("High"=.5,"Medium"=.3,"Low"=.2),
              "Physics_Mode"="High")
flattenStats(slist)

---

Trys to execute an expression with errors logged.

Description

This is a version of try which logs errors using the flog.logger mechanism.

Usage

flog.try(expr, context = deparse(substitute(expr)), loggername = flog.namespace(), 
         tracelevel = c("WARN", "ERROR", "FATAL"))

Arguments

expr	An R expression to be executed.
context	A character string defining what was operation is being performed for use in the log message.
loggername	A package name defining the logger to be used. See flog.namespace.
tracelevel	A character vector. In response to signals of the listed types, a stack trace will be sent to the log file.

Details

This function behaves like the try function, attempt to execute expr. If successful, the result is returned, if not an object of class try-error is returned, so that the calling function can figure out how to proceed.

It has two important difference from try. The first is the context argument which provides information about what was happening when the error was generated. In a large problem, this can provide vital debugging information, like the issue was with a particular node in a graph.

The second is that the error message and the stack trace are posted to the logging stream using the flog.logger function. This makes the code easier to use in server processes.

Value

Either the result of running expr or an object of class try-error.

Note

I should move this to the RGAutils package as it is generally useful.

Author(s)

Russell Almond

See Also

try, flog.logger

The function maxAllTableParams shows an example of this in use.

Examples

## Not run: 
maxAllTableParams <- function (net, Mstepit=5,
                                    tol=sqrt(.Machine$double.eps),
                                    debug=FALSE) {
  Errs <- list()
  netnm <- PnetName(net)
  lapply(PnetPnodes(net),
         function (nd) {
           ndnm <- PnodeName(nd)
           flog.debug("Updating params for node 
           out <- flog.try(maxCPTParam(nd,Mstepit,tol),
                           context=sprintf("Updating params for node 
                                           ndnm, netnm))
           if (is(out,'try-error')) {
             Errs <- c(Errs,out)
             if (debug) recover()
           }
         })
  if (length(Errs) >0L)
    stop("Errors encountered while updating parameters for ",netnm)
  invisible(net)
}

## End(Not run)

---

Fits the parameters of a parameterized network using the GEM algorithm

Description

A Pnet is a description of a parameterized Bayesian network, with each Pnode giving the parameterization for its conditional probability table. This function uses a generalized EM algorithm to find the values of the parameters for each Pnode which maximize the posterior probability of the data in cases.

Usage

GEMfit(net, cases, tol = sqrt(.Machine$double.eps),
       maxit = 100, Estepit = 1, Mstepit = 30,
       trace=FALSE, debugNo=maxit+1)

Arguments

net	A Pnet object
cases	An object representing a set of cases. Note the type of object is implementation dependent. It could be either a data frame providing cases or a filename for a case file.
tol	A numeric scalar giving the stopping tolerance for the for the EM algorithm.
maxit	An integer scalar giving the maximum number of iterations for the outer EM algorithm.
Estepit	An integer scalar giving the number of steps the Bayes net package should take in the internal EM algorithm during the E-step.
Mstepit	An integer scalar giving the number of steps that should be taken by mapDPC during the M-step.
trace	A logical value which indicates whether or not cycle by cycle information should be sent to to the flog.logger.
debugNo	An integer scalar. When this iteration is reached, then the flog.threshold(DEBUG) will be set, so more debugging information will be provided.

Details

The GEMfit algorithm uses a generalized EM algorithm to fit the parameterized network to the given data. This loops over the following steps:

E-step

Run the internal EM algorithm of the Bayes net package to calculate expected tables for all of the tables being learned. The function calcExpTables carries out this step.

M-step

Find a set of table parameters which maximize the fit to the expected counts by calling mapDPC for each table. The function maxAllTableParams does this step.

Update CPTs

Set all the conditional probability tables in the network to the new parameter values. The function BuildAllTables does this.

Convergence Test

Calculate the log likelihood of the cases under the new parameters and stop if no change. The function calcPnetLLike calculates the log likelihood.

Note that although GEMfit is not a generic function, the four main component functions, calcExpTables, maxAllTableParams, BuildAllTables, and calcPnetLLike, are generic functions. In particular, the cases argument is passed to calcExpTables and calcPnetLLike and must be whatever the host Bayes net package regards as a collection of cases. In PNetica-package the cases argument should be a filename of a Netica case file (see write.CaseFile).

The parameter tol controls the convergence checking. In particular, the algorithm stops when the difference in log-likelihood (as computed by calcPnetLLike) between iterations is less than tol in absolute value. If the number of iterations exceeds maxit the algorithm will stop and report lack of convergence.

The E-step and the M-step are also both iterative; the parameters Estepit and Mstepit control the number of iterations taken in each step respectively. As the goal of the E-step is to calculate the expected tables of counts, the default value of 1 should be fine. Although the algorithm should eventually converge for any value of Mstepit, different values may affect the convergence rate, and analysts may need to experiment with application specific values of this parameter.

The arguments trace and debugNo are used to provide extra debugging information. Setting trace to TRUE means that a message is printed after tables are built but before they are updated. Setting debugNo to a certain integer, will begin node-by-node messages for both BuildAllTables and maxAllTableParams. In particular, setting it to 1 is useful for debugging problems that occur at initialization. If the problem turns up at a later cycle, the trace option can be used to figure out when the error occurs.

Value

A list with three elements:

converged	A logical flag indicating whether or not the algorithm reached convergence.
iter	An integer scalar giving the number of iterations of the outer EM loop taken by the algorithm (plus 1 for the starting point).
llikes	A numeric vector of length iter giving the values of the log-likelihood after each iteration. (The first value is the initial log likelihood.)

As a side effect the PnodeLnAlphas and PnodeBetas fields of all nodes in PnetPnodes(net)) are updated to better fit the expected tables, and the internal conditional probability tables are updated to match the new parameter values.

Logging and Debug Mode

As of version 0.6-2, the meaning of the trace and debugNo has changed. In the new version, the flog.logger mechanism is used for progress reports, and error reporting.

Setting trace to true causes information about the steps of the algoritm (incluing the log likelihood at each step) to be output to the current appender (see flog.appender) The logging is done at the INFO level. As the default appender is the console, and INFO is the default logging level, the meaning of this parameter hasn't changed much.

The meaning of debugNo has changed, howver. Previously, it would turn on extra debug information when the target iteration was reached. That information is now always logged at the DEBUG level. So now if the current iteration reached debugNo, then GEMfit calls flog.threshold(DEBUG) to provide more information. This allows the more detailed DEBUG-level messages to be turned on when the EM algorithm is closer to convergence.

Note

Note that although this is not a generic function, the four main component functions: calcExpTables, maxAllTableParams, BuildAllTables, and calcPnetLLike. All four must have specific implementations for this function to work. See the PNetica-package for an example.

These functions assume that the host Bayes net implementation (e.g., RNetica-package): (1) net has an EM learning function, (2) the EM learning supports hyper-Dirichlet priors, (3) it is possible to recover the hyper-Dirichlet posteriors after running the internal EM algorithm.

Author(s)

Russell Almond

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

See Also

Pnet, calcExpTables, calcPnetLLike, maxAllTableParams, BuildAllTables

Examples

## Not run: 

library(PNetica) ## Need a specific implementation
sess <- NeticaSession()
startSession(sess)

irt10.base <- ReadNetworks(system.file(
                           "testnets","IRT10.2PL.base.dne",
                           package="PNetica"),
                           session=sess)
irt10.base <- as.Pnet(irt10.base)  ## Flag as Pnet, fields already set.
irt10.theta <- PnetFindNode(irt10.base,"theta")
irt10.items <- PnetPnodes(irt10.base)
## Flag items as Pnodes
for (i in 1:length(irt10.items)) {
  irt10.items[[i]] <- as.Pnode(irt10.items[[i]])
  ## Add node to list of observed nodes
  PnodeLabels(irt10.items[[1]]) <-
     union(PnodeLabels(irt10.items[[1]]),"onodes")
}


casepath <- system.file("testdat", "IRT10.2PL.200.items.cas", package="PNetica")


BuildAllTables(irt10.base)
PnetCompile(irt10.base) ## Netica requirement

item1 <- irt10.items[[1]]
priB <- PnodeBetas(item1)
priA <- PnodeAlphas(item1)
priCPT <- PnodeProbs(item1)

gemout <- GEMfit(irt10.base,casepath,trace=TRUE)

postB <- PnodeBetas(item1)
postA <- PnodeAlphas(item1)
postCPT <- PnodeProbs(item1)

## Posterior should be different
stopifnot(
  postB != priB, postA != priA
)


### The network that was used for data generation.
irt10.true <- ReadNetworks(system.file(
                           "testnets", "IRT10.2PL.true.dne",
                           package="PNetica"),
                           session=sess)
irt10.true <- as.Pnet(irt10.true)  ## Flag as Pnet, fields already set.
irt10.ttheta <- PnetFindNode(irt10.true,"theta")
irt10.titems <- PnetPnodes(irt10.true)
## Flag titems as Pnodes
for (i in 1:length(irt10.titems)) {
  irt10.titems[[i]] <- as.Pnode(irt10.titems[[i]])
  ## Add node to list of observed nodes
  PnodeLabels(irt10.titems[[1]]) <-
     union(PnodeLabels(irt10.titems[[1]]),"onodes")
}


BuildAllTables(irt10.true)
PnetCompile(irt10.true) ## Netica requirement

## See how close we are.
for (j in 1:length(irt10.titems)) {
  cat("diff[",j,"] = ",
      sum(abs(PnodeProbs(irt10.items[[j]])-
              PnodeProbs(irt10.titems[[j]])))/
      length(PnodeProbs(irt10.items[[j]])), "\n")
}


DeleteNetwork(irt10.base)
DeleteNetwork(irt10.true)


## End(Not run)

---

Checks to see if names are valid for objects in warehouse.

Description

Objects in a warehouse may have restrictions on the names that are allowed. For example, Netica nodes and nets must have names that follow common variable naming convention (alphanumeric, starts with a letter, no embedded spaces, etc.). The function is.legal.name checks the name type, and as.legal.name munges the name so that it is legal.

Usage

is.legal.name(warehouse, name)
as.legal.name(warehouse, name)

Arguments

warehouse	A warehouse which defines the type of object.
name	A character vector giving names to be tested or munged.

Value

For is.valid.name, a logical value returning the result of each test.

For as.valid.name, a character vector with the modified names.

Note

The BNWarehouse and NNWarehouse have a prefix field which is used to ensure that names always start with a letter.

Author(s)

Russell Almond

Examples

## Not run: 
## Requires PNetica
library(PNetica)

sess <- NeticaSession()
startSession(sess)

## BNWarehouse is the PNetica Net Warehouse.
## This provides an example network manifest.
table.dir <- system.file("auxdata", package="Peanut")
net.dir <- system.file("testnets", package="PNetica")

netman1 <- read.csv(file.path(table.dir,"Mini-PP-Nets.csv"),
                    row.names=1, stringsAsFactors=FALSE)
Nethouse <- BNWarehouse(manifest=netman1,session=sess,key="Name",
                       address=net.dir,prefix="S")

stopifnot(is.legal.name(Nethouse,c("CamelCase","Embedded Space")) ==
          c(TRUE,FALSE),
          as.legal.name(Nethouse,"100c3") == "S100c3")


## This expression provides an example Node manifest
nodeman1 <- read.csv(file.path(table.dir,"Mini-PP-Nodes.csv"),
                     row.names=1,stringsAsFactors=FALSE)
Nodehouse <- NNWarehouse(manifest=nodeman1,
                         key=c("Model","NodeName"),
                         session=sess,prefix="V")

stopifnot(is.legal.name(Nodehouse,c("Neg1","-1")) ==
          c(TRUE,FALSE),
          as.legal.name(Nodehouse,1) == "V1")



## End(Not run)

---

Functions for handling continuous nodes.

Description

Continuous nodes are handled slightly differently from discrete nodes. The function isPnodeContinuous returns a logical value indicating whether or not the node is continuous.

Continuous nodes can behave like discrete nodes (for the purposes of building conditional probability tables, see BuildTable) if states are created from ranges of values. The function PnodeStateBounds accesses those ranges.

Usage

isPnodeContinuous(node)
PnodeStateBounds(node)
PnodeStateBounds(node) <- value

Arguments

node	A Pnode object.
value	A k by 2 numeric matrix giving the upper and lower bound for each state.

Details

Continuous, in this case, covers nodes whose possible states are numeric, either integer or real. The current model supports these nodes in a discrete Bayesian network by discretizing them. In particular, the range is broken up into a number of non-overlapping regions, each region corresponding to a state.

For example, consider a variable which is a count, and the analyst wants to consider the values 0, 1, 2 or 3, and 4 or more. This can be done by setting bounds on these states:

"Zero"	-0.5	0.5
"One"	0.5	1.5
"TwoThree"	1.5	3.5
"FourPlus"	3.5	Inf

This matrix is the NodeStateBounds for the node. Note that the second column is the same as the first (offset by one). Note also that infinite (Inf and -Inf) values are allowed.

Setting the state bounds to a matrix with $k$ rows, will make the variable behave as if it has $k$ states.

Value

The function isPnodeContinuous returns a logical value.

The function PnodeStateBounds returns a $k$ by 2 numeric matrix giving the upper and lower bounds. Note that if bounds have not been set for the node, then it will return a matrix with 0 rows.

Note

This is rather strongly tied to how Netica treats continuous variables. A different mechism might be necessary as Peanut is expanded to cover more implementations.

Right now, the value is the midpoint of the interval. This cause problems when converting to T-values.

The setter function is very strict about the upper and lower bounds matching. Even a mismatch at the least significant digit will cause a problem.

Author(s)

Russell Almond

See Also

Pnode, PnodeStateValues, PnodeParentTvals

Examples

## Not run: 
library(PNetica) ## Requires implementation
sess <- NeticaSession()
startSession(sess)

tNet <- CreateNetwork("TestNet",session=sess)

theta1 <- NewDiscreteNode(tNet,"theta1",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta1) <- effectiveThetas(NodeNumStates(theta1))
stopifnot (!isPnodeContinuous(theta1))

## This gives an error
out <- try(PnodeStateBounds(theta1))
stopifnot (is(out,'try-error'))

theta0 <- NewContinuousNode(tNet,"theta0")
stopifnot(nrow(PnodeStateBounds(theta0)) == 0L)

norm5 <- 
   matrix(c(qnorm(c(.001,.2,.4,.6,.8)),
            qnorm(c(.2,.4,.6,.8,.999))),5,2,
          dimnames=list(c("VH","High","Mid","Low","VL"),
                        c("LowerBound","UpperBound")))
PnodeStateBounds(theta0) <- norm5
PnodeStates(theta0)
PnodeStateBounds(theta0)
PnodeStateValues(theta0)  ## Note these are medians not mean wrt normal!

DeleteNetwork(tNet)
stopSession(sess)


## End(Not run)

---

Find optimal parameters of Pnet or Pnode to match expected tables

Description

These functions assume that an expected count contingency table can be built from the network. They then try to find the set of parameters maximizes the probability of the expected contingency table with repeated calls to mapDPC. The function maxCPTParam maximizes a single Pnode and the function maxAllTableParams maximizes all Pnodes (i.e., the value of PnetPnodes(net) in a Pnet.

Usage

maxAllTableParams(net, Mstepit = 5, tol = sqrt(.Machine$double.eps), debug=FALSE)
maxCPTParam(node, Mstepit = 5, tol = sqrt(.Machine$double.eps))

Arguments

net	A Pnet object giving the parameterized network.
node	A Pnode object giving the parameterized node.
Mstepit	A numeric scalar giving the number of maximization steps to take. Note that the maximization does not need to be run to convergence.
tol	A numeric scalar giving the stopping tolerance for the maximizer.
debug	A logical scalar. If true then recover is called after an error, so that the node in question can be inspected.

Details

The GEMfit algorithm uses a generalized EM algorithm to fit the parameterized network to the given data. This loops over the following steps:

E-step

Run the internal EM algorithm of the Bayes net package to calculate expected tables for all of the tables being learned. The function calcExpTables carries out this step.

M-step

Find a set of table parameters which maximize the fit to the expected counts by calling mapDPC for each table. The function maxAllTableParams does this step.

Update CPTs

Set all the conditional probability tables in the network to the new parameter values. The function BuildAllTables does this.

Convergence Test

Calculate the log likelihood of the cases under the new parameters and stop if no change. The function calcPnetLLike calculates the log likelihood.

The function maxAllTableParams performs the M-step of this operation. Under the global parameter independence assumption, the parameters for the conditional probability tables for different nodes are independent given the sufficient statistics; that is, the expected contingency tables. The default method of maxAllTableParams calls maxCPTParam on each node in PnetPnodes(net).

After the hyper-Dirichlet EM algorithm is run by calcExpTables, a hyper-Dirichlet prior should be available for each conditional probability table. As the parameter of the Dirichlet distribution is a vector of pseudo-counts, the output of this algorithm should be a table of pseudo counts. Often this is stored as the updated conditional probability table and a vector of row weights indicating the strength of information for each row. Using the RNetica-package, this is calculated as: sweep(NodeProbs(item1),1, NodeExperience(item1),"*")

The function maxCPTParm is essentially a wrapper which extracts the table of pseudo-counts from the network and then calls mapDPC to maximize the parameters, updating the parameters of node to the result.

The parameters Mstepit and tol are passed to mapDPC to control the gradient descent algorithm used for maximization. Note that for a generalized EM algorithm, the M-step does not need to be run to convergence, a couple of iterations are sufficient. The value of Mstepit may influence the speed of convergence, so the optimal value may vary by application. The tolerance is largely irrelevant (if Mstepit is small) as the outer EM algorithm does the tolerance test.

Value

The expression maxCPTParam(node) returns node invisibly. The expression maxAllTableParams(net) returns net invisibly.

As a side effect the PnodeLnAlphas and PnodeBetas fields of node (or all nodes in PnetPnodes(net)) are updated to better fit the expected tables.

Logging and Debug Mode

As of version 0.6-2, the meaning of the debug argument is changed. In the new version, the flog.logger mechanism is used for progress reports, and error reporting. In particular, setting flog.threshold(DEBUG) (or TRACE) will cause progress reports to be sent to the logging output.

The debug argument has been repurposed. It now call recover when the error occurs, so that the problem can be debugged.

Note

The function maxCPTParam is an abstract generic function, and it needs specific implementations. See the PNetica-package for an example. A default implementation is provides for maxAllTableParams which loops through calls to maxCPTParam for each node in PnetPnodes(net).

This function assumes that the host Bayes net implementation (e.g., RNetica-package): (1) net has an EM learning function, (2) the EM learning supports hyper-Dirichlet priors, (3) it is possible to recover the hyper-Dirichlet posteriors after running the internal EM algorithm.

Author(s)

Russell Almond

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

See Also

Pnet, Pnode, GEMfit, calcPnetLLike, calcExpTables, mapDPC

Examples

## Not run: 

library(PNetica) ## Need a specific implementation
sess <- NeticaSession()
startSession(sess)

irt10.base <- ReadNetworks(system.file(
                           "testnets", "IRT10.2PL.base.dne",
                                package="PNetica"),
                           session=sess)
irt10.base <- as.Pnet(irt10.base)  ## Flag as Pnet, fields already set.
irt10.theta <- NetworkFindNode(irt10.base,"theta")
irt10.items <- PnetPnodes(irt10.base)
## Flag items as Pnodes
for (i in 1:length(irt10.items)) {
  irt10.items[[i]] <- as.Pnode(irt10.items[[i]])
  ## Add node to list of observed nodes
  PnodeLabels(irt10.items[[1]]) <-
     union(PnodeLabels(irt10.items[[1]]),"onodes")
}

casepath <- system.file("testdat", "IRT10.2PL.200.items.cas", package="PNetica")


BuildAllTables(irt10.base)
PnetCompile(irt10.base) ## Netica requirement

item1 <- irt10.items[[1]]
priB <- PnodeBetas(item1)
priA <- PnodeAlphas(item1)
priCPT <- PnodeProbs(item1)

gemout <- GEMfit(irt10.base,casepath,trace=TRUE)

calcExpTables(irt10.base,casepath)

maxAllTableParams(irt10.base)

postB <- PnodeBetas(item1)
postA <- PnodeAlphas(item1)
BuildTable(item1)
postCPT <- PnodeProbs(item1)

## Posterior should be different
stopifnot(
  postB != priB, postA != priA
)


DeleteNetwork(irt10.base)
stopSession(sess)


## End(Not run)

---

Shiny gadget for editinging (offset) conjuctive and disjunctive pnodes

Description

These functions open a shiny application (in a browser window or other location) for editing a Pnode object. To reduce the complexity, the display assumes that PnodeLink(pnode) is partialCredit or gradedResponse, and that PnodeLink(pnode) is Compensatory (Conjunctive or Disjunctive are also possibilities, but usually, the OffsetGadget is a better parameterization for these rules).

Usage

MakeOffsetGadget(pnode, color = "plum")
OffsetGadget(pnode, color="plum",viewer=shiny::paneViewer())

Arguments

pnode	A Pnode object to be modified.
color	A base color to use for barcharts (see barchart.CPF). Execute colors() for a list of choices.
viewer	This is passed to the viewer argument of shiny::runGadget.

Details

The CompensatoryGadget assumes that:

• The link function is partialCredit

• There is a single rule for all states, and PnodeQ(pnode)=TRUE.

• One of the multiple-b rules: OffsetConjunctive or OffsetDisjunctive is used, so that there is one beta for each parent.

• There is either a single alpha, or one alpha for each state except the last, which is a reference state.

This is the recommended way to model conjuctive and disjunctive models (using a separate difficulty/demand for each parent).

Value

The function MakeOffsetGadget returns a list of two functions, ui and server. These are meant to be passed to shiny::runApp to generate the actual app.

The function OffsetGadget will return the pnode object or throw a ‘Cancel-Error’.

Author(s)

Russell Almond

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

See Also

Pnode, calcDPCFrame, barchart.CPF

CompensatoryGadget, RegressionGadget, DPCGadget

Examples

## Not run: 
library(PNetica) ## Requires implementation
sess <- NeticaSession()
startSession(sess)

tNet <- CreateNetwork("TestNet",sess)

theta1 <- NewDiscreteNode(tNet,"theta1",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta1) <- effectiveThetas(PnodeNumStates(theta1))
PnodeProbs(theta1) <- rep(1/PnodeNumStates(theta1),PnodeNumStates(theta1))
theta2 <- NewDiscreteNode(tNet,"theta2",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta2) <- effectiveThetas(PnodeNumStates(theta2))
PnodeProbs(theta2) <- rep(1/PnodeNumStates(theta1),PnodeNumStates(theta2))

## CompensatoryGadget

partial3 <- NewDiscreteNode(tNet,"partial3",
                            c("FullCredit","PartialCredit","NoCredit"))
PnodeParents(partial3) <- list(theta1,theta2)

## Usual way to set rules is in constructor
partial3 <- Pnode(partial3,rules="OffsetConjuctive", link="partialCredit")
PnodeLnAlphas(partial3) <- 0
PnodeQ(partial3) <- TRUE
PnodeBetas(partial3) <- c(0,1)
BuildTable(partial3)
PnodePriorWeight(partial3) <- 10
BuildTable(partial3)

partial3 <- OffsetGadget(partial3)

## This expression can be used inside an Rmarkdown document
gadget <- MakeOffsetGadget(partial3)
shinyApp(gadget$ui,gadget$server,options(height=2000))

DeleteNetwork(tNet)
stopSession(sess)

## End(Not run)

---

Constructs a parameterized network from an Omega matrix.

Description

An Omega matrix (represented as a data frame) is a structure which describes a Bayesian network as a series of regressions from the parent nodes to the child nodes. It actually contains two matrixes, one giving the structure and the other the regression coefficients. A skeleton matrix can be constructed through the function Pnet2Omega.

Usage

Omega2Pnet(OmegaMat, pn, nodewarehouse, defaultRule = "Compensatory",
defaultLink = "normalLink", defaultAlpha = 1, defaultBeta = 0,
defaultLinkScale = 1, defaultPriorWeight=10, debug = FALSE, override =FALSE,
addTvals = TRUE)

Arguments

OmegaMat	A data frame containing an Omega matrix (see values section of Pnet2Omega).
pn	A (possible empty) Pnet object. This will be modified by the function.
nodewarehouse	A Node Warehouse which contains instructions for building nodes referenced in the Omega matrix but not in the network.
defaultRule	This should be a character scalar giving the name of a CPTtools combination rule (see Compensatory). With the regression model assumed in the algorithm, currently “Compensatory” is the only value that makes sense.
defaultLink	This should be a character scalar giving the name of a CPTtools link function (see normalLink). With the regression model assumed in the algorithm, currently “normalLink” is the only value that makes sense.
defaultAlpha	A numeric scalar giving the default value for slope parameters.
defaultBeta	A numeric scalar giving the default value for difficulty (negative intercept) parameters.
defaultLinkScale	A positive number which gives the default value for the link scale parameter.
defaultPriorWeight	A positive number which gives the default value for the node prior weight hyper-parameter.
debug	A logical scalar. If true then recover is called after an error, so that the node in question can be inspected.
override	A logical value. If false, differences between any exsiting structure in the graph and the Omega matrix will raise an error. If true, the graph will be modified to conform to the matrix.
addTvals	A logical value. If true, nodes which do not have state values set, will have those state values set using the function effectiveThetas.

Details

Whittaker (1990) noted that a normal Bayesian network (one in which all nodes followed a standard normal distribution) could be described using the inverse of the covariance matrix, often denoted Omega. In particular, zeros in the inverse covariance matrix represented variables which were conditionally independent, and therefore reducing the matrix to one with positive and zero values could provide the structure for a graphical model. Almond (2010) proposed using this as the basis for specifying discrete Bayesian networks for the proficiency model in educational assessments (especially as correlation matrixes among latent variables are a possible output of a factor analysis).

The Omega matrix is represented with a data.frame object which contains two square submatrixes and a couple of auxiliary columns. The first column should be named “Node” and contains the names of the nodes. This defines a collection of nodes which are defined in the Omega matrix. Let $J$ be the number of nodes (rows in the data frame). The next $J$ columns should have the names the nodes. Their values give the structural component of the matrix. The following two columns are “Link” and “Rules” these give the name of the combination rule and link function to use for this row. Next follows another series $J$ “A” columns, each should have a name of the form “A.node”. This defines a matrix $A$ containing regression coefficients. Finally, there should be two additional columns, “Intercept” and “PriorWeight”.

Let $Q$ be the logical matrix formed by the $J$ columns after the first and let $A$ be the matrix of coefficients. The matrix $Q$ gives the structure of the graph with $Q[i,j]$ being true when Node $j$ is a parent of node i. By convention, $Q[j,j]=1$. Note that unlike the inverse covariance matrix from which it gets its name, this matrix is not symmetric. It instead reflects the (possibly arbitrary) directions assigned to the edges. Except for the main diagonal, $Q[i,j]$ and $Q[j,i]$ will not both be 1. Note also, that $A[i,j]$ should be positive only when $Q[i,j]=1$. This provides an additional check that structures were correctly entered if the Omega matrix is being used for data entry.

When the link function is set to normalLink and the rules is set of Compensatory the model is described as a series of regressions. Consider Node $j$ which has $K$ parents. Let $\theta_j$ be a real value corresponding to that node and let $\theta_k$ be a real (standard normal) value representing Parent Node $k$ $a_k$ represent the corresponding coefficient from the $A$-table. Let $\sigma_j = a_{j,j}$ that is the diagonal element of the $A$-table corresponding to the variable under consideration. Let $b_j$ be the value of the intercept column for Node $j$. Then the model specifies that $theta_j$ has a normal distribution with mean

$\frac{1}{\sqrt{K}}\sum a_k\theta_k + b_j,$

and standard deviation $\sigma_j$. The regression is discretized to calculate the conditional probability table (see normalLink for details).

Note that the parameters are deliberately chosen to look like a regression model. In particular, $b_j$ is a normal intercept and not a difficulty parameter, so that in general PnodeBetas applied to the corresponding node will have the opposite sign. The $1/\sqrt{K}$ term is a variance stabilization parameter so that the variance of $\theta_j$ will not be affected by number of parents of Node $j$. The multiple R-squared for the regression model is

$\frac{1/K \sum a_k^2}{ 1/K \sum a_k^2 + \sigma_j^2} .$

This is often a more convenient parameter to elicit than $\sigma_j$.

The function Omega2Pnet attempts to make adjustments to its pnet argument, which should be a Pnet, so that it conforms to the information given in the Omega matrix. Nodes are created as necessary using information in the nodewarehouse argument, which should be a Warehouse object whose manifest includes instructions for building the nodes in the network. The warehouse supply function should either return an existing node in pnet or create a new node in pnet. The structure of the graph is adjusted to correspond to the Q-matrix (structural part of the data frame). If the value of the override argument is false, an error is raised if there is existing structure with a different topology. If override is true, then the pnet is destructively altered to conform to the structural information in the Omega matrix.

The “Link” and “Rules” columns are used to set the values of PnodeLink(node) and PnodeRules(node). The off-diagonal elements of the A-matrix are used to set PnodeAlphas(node) and the diagonal elements to set PnodeLinkScale(node). The values in the “Intercept” column are the negatives of the values PnodeBetas(node). Finally, the values in the “PriorWeight” column correspond to the values of PnodePriorWeight(node). In any of these cases, if the value in the Omega matrix is missing, then the default value will be supplied instead.

One challenge is setting up a matrix with the correct structure. If the nodes have been defined, the the Pnet2Omega can be used to create a blank matrix with the proper format which can then be edited.

Value

The network pnet is returned. Note that it is destructively modified by the commands to conform to the Omega matrix.

Omega Matrix Structure

An Omega Matrix should be an object of class data.frame with number of rows equal to the number of nodes. Throughout let node stand for the name of a node.

Node

The name of the node described in this column.

node

One column for each node. The value in this column should be 1 if the node in the column is regarded as a parent of the node referenced in the row.

Link

The name of a link function. Currently, “normalLink” is the only value supported.

Rules

The name of the combination rule to use. Currently, “Compensatory” is recommended.

A.node

One column for each node. This should be a positive value if the corresponding node column has a 1. This gives the regression coefficient. If node corresponds to the current row, this is the residual standard deviation rather than a regression coefficient. See details.

Intercept

A numeric value giving the change in prevalence for the two variables (see details).

PriorWeight

The amount of weight which should be given to the current values when learning conditional probability tables. See PnodePriorWeight.

Logging and Debug Mode

As of version 0.6-2, the meaning of the debug argument is changed. In the new version, the flog.logger mechanism is used for progress reports, and error reporting. In particular, setting flog.threshold(DEBUG) (or TRACE) will cause progress reports to be sent to the logging output.

The debug argument has been repurposed. It now call recover when the error occurs, so that the problem can be debugged.

Side Effects

This function destructively modifies pnet and nodes referenced in the Qmat and supplied by the warehouses.

Note that unlike typical R implementations, this is not necessarily safe. In particular, if the Qmat references 10 node, and an error is raised when trying to modify the 5th node, the first 4 nodes will be modified, the last 5 will not be and the 5th node may be partially modified. This is different from most R functions where changes are not committed unless the function returns successfully.

Note

While the Omega matrix allows the user to specify both link function and combination rule, the description of the Bayesian network as a series of regressions only really makes sense when the link function is normalLink and the combination rule is Compensatory. These are included for future exapnsion.

The representation, using a single row of the data frame for each node in the graph, only works well with the normal link function. In particular, both the partial credit and graded response links require the ability to specify different intercepts for different states of the variable, something which is not supported in the Omega matrix. Furthermore, the OffsetConjunctive rule requires multiple intercepts. Presumable the Conjunctive rule could be used, but the interpretation of the slope parameters is then unclear. If the variables need a model other than the compensatory normal model, it might be better to use a Q-matrix (see Pnet2Qmat to describe the variable.

Author(s)

Russell Almond

References

Whittaker, J. (1990). Graphical Models in Applied Multivariate Statistics. Wiley.

Almond, R. G. (2010). ‘I can name that Bayesian network in two matrixes.’ International Journal of Approximate Reasoning. 51, 167-178.

Almond, R. G. (presented 2017, August). Tabular views of Bayesian networks. In John-Mark Agosta and Tomas Singlair (Chair), Bayeisan Modeling Application Workshop 2017. Symposium conducted at the meeting of Association for Uncertainty in Artificial Intelligence, Sydney, Australia. (International) Retrieved from http://bmaw2017.azurewebsites.net/

See Also

The inverse operation is Pnet2Omega.

See Warehouse for description of the node warehouse argument.

See normalLink and Compensatory for more information about the mathematical model.

The node attributes set from the Omega matrix include: PnodeParents(node), PnodeLink(node), PnodeLinkScale(node), PnodeRules(node), PnodeAlphas(node), PnodeBetas(node), and PnodePriorWeight(node)

Examples

## Sample Omega matrix.
omegamat <- read.csv(system.file("auxdata", "miniPP-omega.csv",
                                package="Peanut"),
                     row.names=1,stringsAsFactors=FALSE)

## Not run: 
library(PNetica) ## Needs PNetica
sess <- NeticaSession()
startSession(sess)

curd <- getwd()

netman1 <- read.csv(system.file("auxdata", "Mini-PP-Nets.csv", 
                                package="Peanut"),
                    row.names=1,stringsAsFactors=FALSE)

nodeman1 <- read.csv(system.file("auxdata", "Mini-PP-Nodes.csv", 
                                package="Peanut"),
                     stringsAsFactors=FALSE)

## Insures we are building nets from scratch
setwd(tempdir())
## Network and node warehouse, to create networks and nodes on demand.
Nethouse <- BNWarehouse(manifest=netman1,session=sess,key="Name")

Nodehouse <- NNWarehouse(manifest=nodeman1,
                         key=c("Model","NodeName"),
                         session=sess)
CM <- WarehouseSupply(Nethouse,"miniPP_CM")
CM1 <- Omega2Pnet(omegamat,CM,Nodehouse,override=TRUE,debug=TRUE)

Om2 <- Pnet2Omega(CM1,NetworkAllNodes(CM1))

DeleteNetwork(CM)
stopSession(sess)
setwd(curd)


## End(Not run)

---

A Parameterized Bayesian network

Description

A parameterized Bayesian network. Note that this an abstract class. If an object implements the Pnet protocol, then is.Pnet(net) should return TRUE.

Usage

is.Pnet(x)
as.Pnet(x)
Pnet(net, priorWeight=10, pnodes=list())
## S4 method for signature 'ANY'
Pnet(net, priorWeight=10, pnodes=list())

Arguments

x	A object to test to see if it a parameterized network, or to coerce into a parameterized network.
net	A network object which will become the core of the Pnet. Note that this should probably already be another kind of network, e.g., a NeticaBN object.
priorWeight	A numeric vector providing the default prior weight for nodes.
pnodes	A list of objects which can be coerced into node objects. Note that the function does not do the coercion.

Details

The Pnet class is basically a protocol which any Bayesian network net object can follow to work with the tools in the Peanut package. This is really an abstract class (in the java programming language, Pnet would be an interface rather than a class). In particular, a Pnet is any object for which is.Pnet returns true. The default method looks for the string "Pnet" in the class list.

A Pnet object has two “fields” (implemented through the accessor methods). The function PnetPnodes returns a list of parameterized nodes or Pnodes associate with the network. The function PnetPriorWeight gets (or sets) the default weight to be used for each node.

The default constructor adds "Pnet" to the class of net and then sets the two fields using the accessor functions. There is no default method for the as.Pnet function.

In addition to the required fields, there are several optional fields. The methods PnetName(), PnetTitle(), PnetDescription(), and PnetPathname() all provide generic setters and getters for mostly self-explanatory properties of the network. For model fragments (such as evidence models) which are meant to be ajoined to other networks, the accessor PnetHub() returns the name of the network to which it is to be adjoined (such as a proficiency model). These optional feilds are referenced by the function BuildNetManifest() which builds a table of meta-data from which to construct a network.

The Pnet supports hub-and-spoke architectures for Bayes nets. The hub is a complete Bayesian network to which spokes, network fragments are attached. For example, in a typical educational testing application, the centeral student proficiency model will be the hub, and the evidence models linking the proficiency variables to the observable outcomes, will be the spokes. Only the spokes corresponding to the tasks on a given test form need to be attached to draw inferences. Spoke models are generally model fragments because they contain “stub” nodes, references to nodes in the corresponding hub model. The function PnetHub() returns or sets the name of the hub model for a spoke. For a hub net, this function returns character(0) or NULL. The function PnetMakeStubNodes() will create stub node objects in the spoke model, and the function PnetRemoveStubNodes() will remove them. These are called before and after creating graph structures in Qmat2Pnet. The functions PnetAdjoin() and PnetDetach() adjoin a hub and spoke node, matching the stub variables with their real counterparts and detach them (reversing the process).

The importance of the Pnet object is that it supports the GEMfit method which adjust the parameters of the Pnode objects to fit a set of case data. In order to be compatible with GEMfit, the Pnet object must support four methods: BuildAllTables, calcPnetLLike, calcExpTables, and maxAllTableParams.

The generic function BuildAllTables builds conditional probability tables from the current values of the parameters in all Pnodes. The default method loops through all of the nodes in PnetPnodes and calls the function BuildTable on each.

The generic function calcPnetLLike calculates the log likelihood of a set of cases given the current values of the parameters. There is no default for this method as it implementation dependent.

The generic function calcExpTables calculates expected cross-tabs for all CPT for the Pnodes given a set of case data. The easiest way to do this is to run the EM algorithm for an unconstrained hyper-Dirichlet model for one or two cycles. There is no default for this as it is implementation dependent.

The generic function maxAllTableParams calculates the parameters that maximize the fit to the expected tables for each Pnode. The default method loops over PnetPnodes(net) and applies the method maxCPTParam to each.

Value

The function is.Pnet returns a logical scalar indicating whether or not the object claims to follow the Pnet protocol.

The function as.Pnet and Pnet convert the argument into a Pnet and return that.

Author(s)

Russell Almond

References

Almond, R. G. (2015) An IRT-based Parameterization for Conditional Probability Tables. Paper presented at the 2015 Bayesian Application Workshop at the Uncertainty in Artificial Intelligence Conference.

See Also

Fields: PnetPriorWeight, PnetPnodes

Generic Functions: BuildAllTables, calcPnetLLike, calcExpTables, maxAllTableParams, PnetName(), PnetTitle(), PnetDescription(), PnetPathname(), PnetAdjoin(), PnetDetach(), PnetMakeStubNodes(), PnetRemoveStubNodes(), PnetFindNode()

Functions: GEMfit, BuildNetManifest, Pnet2Qmat, Pnet2Omega, Qmat2Pnet, Omega2Pnet

Related Classes: Pnode, Warehouse

Examples

## Not run: 

library(PNetica)  ## Implementation of Peanut protocol
sess <- NeticaSession()
startSession(sess)
## Create network structure using RNetica calls
IRT10.2PL <- CreateNetwork("IRT10_2PL",session=sess)

theta <- NewDiscreteNode(IRT10.2PL,"theta",
                         c("VH","High","Mid","Low","VL"))
PnodeStateValues(theta) <- effectiveThetas(PnodeNumStates(theta))
PnodeProbs(theta) <- rep(1/PnodeNumStates(theta),PnodeNumStates(theta))

J <- 10 ## Number of items
items <- NewDiscreteNode(IRT10.2PL,paste("item",1:J,sep=""),
                         c("Correct","Incorrect"))
for (j in 1:J) {
  PnodeParents(items[[j]]) <- list(theta)
  PnodeStateValues(items[[j]]) <- c(1,0)
  PnodeLabels(items[[j]]) <- c("observables")
}

## Convert into a Pnet
IRT10.2PL <- Pnet(IRT10.2PL,priorWeight=10,pnodes=items)

## Draw random parameters
btrue <- rnorm(J)
lnatrue <- rnorm(J)/sqrt(3)
dump(c("btrue","lnatrue"),"IRT10.2PL.params.R")

## Convert nodes to Pnodes
for (j in 1:J) {
  items[[j]] <- Pnode(items[[j]],lnatrue[j],btrue[j])
}
BuildAllTables(IRT10.2PL)
is.Pnet(IRT10.2PL)
WriteNetworks(IRT10.2PL,"IRT10.2PL.true.dne")

DeleteNetwork(IRT10.2PL)
stopSession(sess)

## End(Not run)

---

Class "Pnet"

Description

This is a virtual class. Classes implementing the Pnet protocol should attach themselves using setIs.

Note that NULL is always considered a member so that uninitialized in containers.

Objects from the Class

A virtual Class: No objects may be created from it.

Classes can register as belonging to this abstract class. The trick for doing this is: setIs("NetClass","Pnet")

Currently NeticaBN is an example of an object of this class (but requires the PNetica package to provide all of the required functionality).

Methods

No methods defined with class "Pnet" in the signature; however, the following generic functions are available:

PnetName

signature(net = "Pnet"): Fetches network name.

PnetName<-

signature(net = "Pnet", value="character"): Sets network name.

PnetTitle

signature(net = "Pnet"): Fetches network title.

PnetTitle<-

signature(net = "Pnet", value="character"): Sets network title.

PnetHub

signature(net = "Pnet"): Fetches name of hub (Proficiency model) if this is a spoke network (Evidence model).

PnetHub<-

signature(net = "Pnet", value): Sets name of hub model.

PnetPathname

signature(net = "Pnet"): Fetches name of file in which network is saved.

PnetPathname<-

signature(net = "Pnet", value): Sets name of file in which network is saved.

PnetDescription

signature(net = "Pnet"): Fetches documentation string for network.

PnetDescription<-

signature(net = "Pnet", value="character"): Sets documentation string for network.

PnetFindNode

signature(net = "Pnet", name="character"): Finds a node by name.

PnetMakeStubNodes

signature(net = "Pnet", nodes = "list"): Copies nodes from hub model into spoke model.

PnetRemoveStubNodes

signature(net = "Pnet", nodes = "list"): Removes copied nodes from hub model.

PnetAdjoin

signature(hub = "Pnet", spoke = "Pnet"): Attaches spoke to hub, matching stub nodes in spoke with their counterparts in the hub.

PnetDetach

signature(motif = "Pnet", spoke = "Pnet"): Removes the spoke from the motif (combined hub and spoke).

PnetCompile

signature(net = "Pnet"): Performs topological transformations on the net to make it ready for inference.

PnetSerialize

signature(net = "Pnet"): Saves the net to a string which can be stored in a database.

PnetUnserialize

signature(serial = "character"): Reverses the above procedure.

unserializePnet

signature(factory, data): this is an improved version of unserialize that assumes a store of networks.

Pwned

R really doesn't want me to do this. I'm just having a lot of difficulty creating a class that extends something from a different package.

For now, need to use "ANY" instead of "Pnet" and then do explicit type checking with is.Pnet.

Author(s)

Russell Almond

See Also

Pnet.

The class NeticaBN implements this protocol.

Examples

showClass("Pnet")
## Not run: 
 setIs("NeticaBN","Pnet")

## End(Not run)

---

Constructs an Omega matrix from a parameterized network.

Description

An Omega matrix (represented as a data frame) is a structure which describes a Bayesian network as a series of regressions from the parent nodes to the child nodes. It actually contains two matrixes, one giving the structure and the other the regression coefficients. If the parameters have not yet been added to nodes, then the function will use the supplied default values allowing the parameters to later be defined through the use of the function Pnet2Omega.

Usage

Pnet2Omega(net, prof, defaultRule = "Compensatory", defaultLink = "normalLink",
           defaultAlpha = 1, defaultBeta = 0, defaultLinkScale = 1, debug = FALSE)

Arguments

net	A Pnet object containing the network to be described.
prof	A list of Pnode objects which will become the rows and columns of the matrix.
defaultRule	This should be a character scalar giving the name of a CPTtools combination rule (see Compensatory). With the regression model assumed in the algorithm, currently “Compensatory” is the only value that makes sense.
defaultLink	This should be a character scalar giving the name of a CPTtools link function (see normalLink). With the regression model assumed in the algorithm, currently “normalLink” is the only value that makes sense.
defaultAlpha	A numeric scalar giving the default value for slope parameters.
defaultBeta	A numeric scalar giving the default value for difficulty (negative intercept) parameters.
defaultLinkScale	A positive number which gives the default value for the link scale parameter.
debug	A logical value. If true, extra information will be printed during process of building the Omega matrix.

Details

Whittaker (1990) noted that a normal Bayesian network (one in which all nodes followed a standard normal distribution) could be described using the inverse of the covariance matrix, often denoted Omega. In particular, zeros in the inverse covariance matrix represented variables which were conditionally independent, and therefore reducing the matrix to one with positive and zero values could provide the structure for a graphical model. Almond (2010) proposed using this as the basis for specifying discrete Bayesian networks for the proficiency model in educational assessments (especially as correlation matrixes among latent variables are a possible output of a factor analysis).

The Omega matrix is represented with a data.frame object which contains two square submatrixes and a couple of auxiliary columns. The first column should be named “Node” and contains the names of the nodes. This defines a collection of nodes which are defined in the Omega matrix. Let $J$ be the number of nodes (rows in the data frame). The next $J$ columns should have the names the nodes. Their values give the structural component of the matrix. The following two columns are “Link” and “Rules” these give the name of the combination rule and link function to use for this row. Next follows another series $J$ “A” columns, each should have a name of the form “A.node”. This defines a matrix $A$ containing regression coefficients. Finally, there should be two additional columns, “Intercept” and “PriorWeight”.

Let $Q$ be the logical matrix formed by the $J$ columns after the first and let $A$ be the matrix of coefficients. The matrix $Q$ gives the structure of the graph with $Q[i,j]$ being true when Node $j$ is a parent of node i. By convention, $Q[j,j]=1$. Note that unlike the inverse covariance matrix from which it gets its name, this matrix is not symmetric. It instead reflects the (possibly arbitrary) directions assigned to the edges. Except for the main diagonal, $Q[i,j]$ and $Q[j,i]$ will not both be 1. Note also, that $A[i,j]$ should be positive only when $Q[i,j]=1$. This provides an additional check that structures were correctly entered if the Omega matrix is being used for data entry.

When the link function is set to normalLink and the rules is set of Compensatory the model is described as a series of regressions. Consider Node $j$ which has $K$ parents. Let $\theta_j$ be a real value corresponding to that node and let $\theta_k$ be a real (standard normal) value representing Parent Node $k$ $a_k$ represent the corresponding coefficient from the $A$-table. Let $\sigma_j = a_{j,j}$ that is the diagonal element of the $A$-table corresponding to the variable under consideration. Let $b_j$ be the value of the intercept column for Node $j$. Then the model specifies that $theta_j$ has a normal distribution with mean

$\frac{1}{\sqrt{K}}\sum a_k\theta_k + b_j,$

and standard deviation $\sigma_j$. The regression is discretized to calculate the conditional probability table (see normalLink for details).

Note that the parameters are deliberately chosen to look like a regression model. In particular, $b_j$ is a normal intercept and not a difficulty parameter, so that in general PnodeBetas applied to the corresponding node will have the opposite sign. The $1/\sqrt{K}$ term is a variance stabilization parameter so that the variance of $\theta_j$ will not be affected by number of parents of Node $j$. The multiple R-squared for the regression model is

$\frac{1/K \sum a_k^2}{ 1/K \sum a_k^2 + \sigma_j^2} .$

This is often a more convenient parameter to elicit than $\sigma_j$.

The function Pnet2Omega builds an Omega matrix from an existing Pnet. Only the nodes specified in the prof argument are included in the matrix, each row corresponding to a node. The values in the “Node” column are taken from PnodeName(node). The values in the structural part of the matrix are taken from the graphical structure, specifically PnodeParents(node). The “Link” and “Rules” columns are taken from PnodeLink(node) and PnodeRules(node). The off-diagonal elements of the A-matrix are taken from the values of PnodeAlphas(node) and the diagonal elements from PnodeLinkScale(node). The values in the “Intercept” column are the negatives of the values PnodeBetas(node). Finally, the values in the “PriorWeight” column correspond to the values of PnodePriorWeight(node); note that a value of NA indicates that the prior weight should be taken from the Pnet.

If the nodes do not yet have the various parameters set, then this function will create a blank Omega matrix, with default values (set from various optional arguments) for entries where the parameters have not yet been set. This matrix can then be edited and read back in with Omega2Pnet as a way of setting the parameters of the network.

Value

An object of class (OmegaMat,data.frame) with number of rows equal to the number of nodes. Throughout let node stand for the name of a node.

Node	The name of the node described in this column.
\var{node}	One column for each node. The value in this column should be 1 if the node in the column is regarded as a parent of the node referenced in the row.
Link	The name of a link function. Currently, “normalLink” is the only value supported.
Rules	The name of the combination rule to use. Currently, “Compensatory” is recommended.
A.\var{node}	One column for each node. This should be a positive value if the corresponding node column has a 1. This gives the regression coefficient. If node corresponds to the current row, this is the residual standard deviation rather than a regression coefficient. See details.
Intercept	A numeric value giving the change in prevalence for the two variables (see details).
PriorWeight	The amount of weight which should be given to the current values when learning conditional probability tables. See PnodePriorWeight.

Note

While the Omega matrix allows the user to specify both link function and combination rule, the description of the Bayesian network as a series of regressions only really makes sense when the link function is normalLink and the combination rule is Compensatory. These are included for future exapnsion.

The representation, using a single row of the data frame for each node in the graph, only works well with the normal link function. In particular, both the partial credit and graded response links require the ability to specify different intercepts for different states of the variable, something which is not supported in the Omega matrix. Furthermore, the OffsetConjunctive rule requires multiple intercepts. Presumable the Conjunctive rule could be used, but the interpretation of the slope parameters is then unclear. If the variables need a model other than the compensatory normal model, it might be better to use a Q-matrix (see Pnet2Qmat to describe the variable.

Author(s)

Russell Almond

References

Whittaker, J. (1990). Graphical Models in Applied Multivariate Statistics. Wiley.

Almond, R. G. (2010). ‘I can name that Bayesian network in two matrixes.’ International Journal of Approximate Reasoning. 51, 167-178.

Almond, R. G. (presented 2017, August). Tabular views of Bayesian networks. In John-Mark Agosta and Tomas Singlair (Chair), Bayeisan Modeling Application Workshop 2017. Symposium conducted at the meeting of Association for Uncertainty in Artificial Intelligence, Sydney, Australia. (International) Retrieved from http://bmaw2017.azurewebsites.net/

See Also

The inverse operation is Omega2Pnet.

See normalLink and Compensatory for more information about the mathematical model.

The node functions from which the Omega matrix is populated includes: PnodeParents(node), PnodeLink(node), PnodeLinkScale(node), PnodeRules(node), PnodeAlphas(node), PnodeBetas(node), and PnodePriorWeight(node)

Examples

## Sample Omega matrix.
omegamat <- read.csv(system.file("auxdata", "miniPP-omega.csv",
                                package="Peanut"),
                     row.names=1,stringsAsFactors=FALSE)

## Not run: 
library(PNetica) ## Needs PNetica
sess <- NeticaSession()
startSession(sess)
curd <- getwd()

netman1 <- read.csv(system.file("auxdata", "Mini-PP-Nets.csv", 
                                package="Peanut"),
                    row.names=1,stringsAsFactors=FALSE)

nodeman1 <- read.csv(system.file("auxdata", "Mini-PP-Nodes.csv", 
                                package="Peanut"),
                     row.names=1,stringsAsFactors=FALSE)

## Insures we are building nets from scratch
setwd(tempdir())
## Network and node warehouse, to create networks and nodes on demand.
Nethouse <- BNWarehouse(manifest=netman1,session=sess,key="Name")

Nodehouse <- NNWarehouse(manifest=nodeman1,
                         key=c("Model","NodeName"),
                         session=sess)
CM <- WarehouseSupply(Nethouse,"miniPP_CM")
CM1 <- Omega2Pnet(omegamat,CM,Nodehouse,override=TRUE,debug=TRUE)

Om2 <- Pnet2Omega(CM1,NetworkAllNodes(CM1))

class(omegamat) <- c("OmegMat","data.frame") # To match Pnet2Omega output.
omegamat$PriorWeight <- rep("10",nrow(omegamat))

stopifnot(all.equal(omegamat,Om2))


DeleteNetwork(CM)
stopSession(sess)
setwd(curd)


## End(Not run)

---

Makes an augmented Q-matrix from a collection of parameterized nets

Description

In augmented $Q$-matrix, there is a set of rows for each Pnode which describes the conditional probability table for that node in terms of the model parameters (see BuildTable). As the Pnodes could potentially come from multiple nets, the key for the table is (“Model”, “Node”). As there are multiple rows per node, “State” is the third part of the key.

The function Pnet2 creates an augmented $Q$-matrix out of a collection of Pnodes, possibly spanning multiple Pnets.

Usage

Pnet2Qmat(obs, prof, defaultRule = "Compensatory", defaultLink = "partialCredit", 
          defaultAlpha = 1, defaultBeta = NULL, defaultLinkScale = NULL, debug = TRUE)

Arguments

obs	A list of observable Pnode objects. These could span multiple Pnet objects. Each element of this list will corresponded to one or more rows in the output $Q$-matrix.
prof	A list of proficiency Pnodes. These are the parents of the Pnodes in the obs list. Usually, these are all in a central proficiency or hub model.
defaultRule	This should be a character scalar giving the name of a CPTtools combination rule (see Compensatory).
defaultLink	This should be a character scalar giving the name of a CPTtools link function (see partialCredit).
defaultAlpha	A numeric scalar giving the default value for slope parameters.
defaultBeta	A numeric scalar giving the default value for difficulty (negative intercept) parameters.
defaultLinkScale	A positive number which gives the default value for the link scale parameter.
debug	A logical value. If true, extra information will be printed during process of building the Pnet.

Details

A $Q$-matrix is a 0-1 matrix which describes which proficiency (latent) variables are connected to which observable outcome variables; $q_{jk}=1$ if and only if proficiency variable $k$ is a parent of observable variable $j$. Almond (2010) suggested that augmenting the $Q$-matrix with additional columns representing the combination rules (PnodeRules), link function (PnodeLink), link scale parameter (if needed, PnodeLinkScale) and difficulty parameters (PnodeBetas). The discrimination parameters (PnodeAlphas) could be overloaded with the $Q$-matrix, with non-zero parameters in places where there were 1's in the $Q$-matrix.

This arrangement worked fine with combination rules (e.g., Compensatory) which contained multiple alpha (discrimination) parameters, one for each parent variable, and a single beta (difficulty). The introduction of a new type of offset rule (e.g., OffsetDisjunctive) which uses a multiple difficulty parameters, one for each parent variable, and a single alpha. Almond (2016) suggested a new augmentation which has three matrixes in a single table (a Qmat): the $Q$-matrix, which contains structural information; the $A$-matrix, which contains discrimination parameters; and the $B$-matrix, which contains the difficulty parameters. The names for the columns for these matrixes contain the names of the proficiency variables, prepended with “A.” or “B.” in the case of the $A$-matrix and $B$-matrix. There are two additional columns marked “A” and “B” which are used for the discrimination and difficulty parameter in the multiple-beta and multiple-alpha cases. There is some redundancy between the $Q$, $A$ and $B$ matrixes, but this provides an opportunity for checking the validity of the input.

The introduction of the partial credit link function (partialCredit) added a further complication. With the partial credit model, there could be a separate set of discrimination or difficulty parameters for each transition for a polytomous item. Even the gradedResponse link function requires a separate difficulty parameter for each level of the varaible save the first. The rows of the Qmat data structure are hence augmented to include one row for every state but the lowest-level state. There should be of fewer rows of associated with the node than the value in the “Nstates” column, and the names of the states (values in the “State” column) should correspond to every state of the target variable except the first. It is an error if the number of states does not match the existing node, or if the state names do not match what is already used for the node or is in the manifest for the node Warehouse.

Note that two nodes in different networks may share the same name, and two states in two different nodes may have the same name as well. Thus, the formal key for the Qmat data frame is (“Model”, “Node”, “State”), however, the rows which share the values for (“Model”, “Node”) form a subtable for that particular node. In particular, the rows of the $Q$-matrix subtable for that node form the inner Q-matrix for that node. The inner $Q$-matrix shows which variables are relevant for each state transition in a partial credit model. The column-wise maximum of the inner $Q$-matrix forms the row of the outer $Q$-matrix for that node. This shows which proficiency nodes are the parent of the observable node. This corresponds to PnodeQ(node).

The function Qmat2Pnet creates and sets the parameters of the observable Pnodes referenced in the Qmat argument. As it needs to reference, and possibly create, a number of Pnets and Pnodes, it requires both a network and a node Warehouse. If the override parameter is true, the networks will be modified so that each node has the correct parents, otherwise Qmat2Pnet will signal an error if the existing network structure is inconsistent with the $Q$-matrix.

As there is only one link function for each node, the values of PnodeLink(node) and PnodeLinkScale(node) are set based on the values in the “Link” and “LinkScale” columns and the first row corresponding to node. Note that the choice of link functions determines what is sensible for the other values but this is not checked by the code.

The value of PnodeRules(node) can either be a single value or a list of rule names. The first value in the sub-Qmat must a character value, but if the other values are missing then a single value is used. If not, all of the entries should be non-missing. If this is a single value, then effectively the same combination rule is used for each transition.

The interpretation of the $A$-matrix and the $B$-matrix depends on the value in the “Rules” column. There are two types of rules, multiple-A rules and multiple-B rules (offset rules). The CPTtools funciton isOffsetRule checks to see what kind of a rule it is. The multiple-A rules, of which Compensatory is the canonical example, have one discrimination (or slope) parameter for every parent variable (values of 1 in the $Q$-matrix) and have a single difficulty (negative intercept) parameter which is in the “B” column of the Qmat. The multiple-B or offset rules, of which OffsetConjunctive is the canonical example, have a difficulty (negative intercept) parameter for each parent variable and a single discrimination (slope) parameter which is in the “A” column. The function Qmat2Pnet uses the value of isOffsetRule to determine whether to use the multiple-B (true) or multiple-A (false) paradigm.

A simple example is a binary observable variable which uses the Compensatory rule. This is essentially a regression model (logistic regression with partialCredit or gradedResponse link funcitons, linear regression with normalLink link function) on the parent variables. The linear predictor is:

$\frac{1}{\sqrt{K}} (a_1\theta_1 + \ldots + a_K\theta_K) - b .$

The values $\theta_1, \ldots, \theta_K$ are effective thetas, real values corresponding to the states of the parent variables. The value $a_i$ is stored in the column “A.namei” where namei is the name of the $i$th proficiency variable; the value of PnodeAlphas(node) is the vector $a_1, \ldots, a_k$ with names corresponding to the parent variables. The value of $b$ is stored in the “B” column; the value of PnodeBetas(node) is $b$.

The multiple-B pattern replaces the $A$-matrix with the $B$-matrix and the column “A” with “B”. Consider binary observable variable which uses the OffsetConjunctive rule. The linear predictor is:

$a \min (\theta_1 -b+1, \ldots , \theta_K- b_K) .$

The value $b_i$ is stored in the column “B.namei” where namei is the name of the $i$th proficiency variable; the value of PnodeBetas(node) is the vector $b_1, \ldots, b_k$ with names corresponding to the parent variables. The value of $a$ is stored in the “A” column; the value of PnodeBetas(node) is $a$.

When there are more than two states in the output varible, PnodeRules, PnodeAlphas(node) and PnodeBetas(node) become lists to indicate that a different value should be used for each transition between states. If there is a single value in the “Rules” column, or equivalently the value of PnodeRules is a scalar, then the same rule is repeated for each state transition. The same is true for PnodeAlphas(node) and PnodeBetas(node). If these values are a list, that indicates that a different value is to be used for each transition. If they are a vector that means that different values (of discriminations for multiple-a rules or difficulties for multiple-b rules) are needed for the parent variables, but the same set of values is to be used for each state transition. If different values are to be used then the values are a list of vectors.

The necessary configuration of $a$'s and $b$'s depends on the type of link function. Here are the rules for the currently existing link funcitons:

normal

(normalLink) This link function uses the same linear predictor for each transition, so there should be a single rule, and PnodeAlphas(node) and PnodeBetas(node) should both be vectors (with $b$ of length 1 for a multiple-a rule). This rule also requires a positive value for the PnodeLinkScale(node) in the “"LinkScale"” column. The values in the “A.name” and “B.name” for rows after the first can be left as NA's to indicate that the same values are reused.

graded response

(gradedResponse) This link function models the probability of getting at or above each state and then calculates the differences between them to produce the conditional probability table. In order to avoid negative probabilities, the probability of being in a higher state must always be nonincreasing. The surest way to ensure this is to both use the same combination rules at each state and the same set of discrimination parameters for each state. The difficulty parameters must be nondecreasing. Again, values for rows after the first can be left as NAs to indicate that the same value should be resused.

partial credit

(partialCredit) This link function models the conditional probability from moving from the previous state to the current state. As such, there is no restriction on the rules or parameters. In particular, it can alternate between multiple-a and multiple-b style rules from row to row.

Another restriction that the use of the partial credit rule lifts is the restriction that all parent variable must be used in each transition. Note that there is one row of the $Q$-matrix (the inner $Q$-matrix) for each state transition. Only the parent variables with 1's in the particular state row are considered when building the PnodeAlphas(node) and PnodeBetas(node) for this model. Note that only the partial credit link function can take advantage of the multiple parents, the other two require all parents to be used for every state.

The function Pnet2Qmat takes a collection of nodes (in a series of spoke or evidence models) and builds a Qmat data structure that can reproduce them. It loops through the nodes and fills out the Qmat based on the properties of the Pnodes. Note that if the proprties are not yet set, then the default values are used, thus applying this to a network for which the structure has been established, but the parameters have not yet been set will build a blank Qmat which can be adjusted by experts.

Value

The output augmented $Q$-matrix is a data frame with the columns described below. The number of columns is variable, with items marked prof actually corresponding to a number of columns with names taken from the proficiency variables (the prof argument).

Model	The name of the Pnet in which the node in this row lives.
Node	The name of the Pnode described in this row. Except for the multiple rows corresponding to the same node, the value of this column needs to be unique within “Model”.
Nstates	The number of states for this node. Generally, each node should have one fewer rows than this number.
State	The name of the state for this row. This should be unique within the (“Model”,“Node”) combination.
Link	The name of a link function. This corresponds to PnodeLink(node).
LinkScale	Either a positive number giving the link scale parameter or an NA if the link function does not need scale parameters. This corresponds to PnodeLinkScale(node).
\var{prof}	There is one column for each proficiency variable. This corresponds to the structural part of the $Q$-matrix. There should be 1 in this column if the named proficiency is used in calculating the transition to this state for this particular node, and a 0 otherwise.
Rules	The name of the combination rule to use for this row. This corresponds to PnodeRules(node).
A.\var{prof}	There is one column for each proficiency with the proficiency name appended to “A.”. If a multiple-alpha style combination rule (e.g., Compensatory) this column should contain the appropriate discriminations, otherwise, its value should be NA.
A	If a multiple-beta style combination rule (e.g., OffsetConjunctive) this column should contain the single discrimination, otherwise, its value should be NA.
B.\var{prof}	There is one column for each proficiency with the proficiency name appended to “B.”. If a multiple-bet style combination rule (e.g., OffsetConjunctive) this column should contain the appropriate difficulty (negative intercept), otherwise, its value should be NA.
B	If a multiple-beta style combination rule (e.g., Compensatory) this column should contain the single difficulty (negative intercept), otherwise, its value should be NA.
PriorWeight	The amount of weight which should be given to the current values when learning conditional probability tables. See PnodePriorWeight.

Author(s)

Russell Almond

References

Almond, R. G. (2010). ‘I can name that Bayesian network in two matrixes.’ International Journal of Approximate Reasoning. 51, 167-178.

Almond, R. G. (presented 2017, August). Tabular views of Bayesian networks. In John-Mark Agosta and Tomas Singlair (Chair), Bayeisan Modeling Application Workshop 2017. Symposium conducted at the meeting of Association for Uncertainty in Artificial Intelligence, Sydney, Australia. (International) Retrieved from http://bmaw2017.azurewebsites.net/

See Also

The inverse operation is Qmat2Pnet.

See Warehouse for description of the network and node warehouse arguments

See partialCredit, gradedResponse, and normalLink for currently available link functions. See Conjunctive and OffsetConjunctive for more information about available combination rules.

The node attributes set from the Omega matrix include: PnodeParents(node), PnodeLink(node), PnodeLinkScale(node), PnodeRules(node), PnodeQ(node), PnodeAlphas(node), PnodeBetas(node), and PnodePriorWeight(node)

Examples

## Sample Q matrix
Q1 <- read.csv(system.file("auxdata", "miniPP-Q.csv", package="Peanut"),
                     stringsAsFactors=FALSE)

## Not run: 
library(PNetica) ## Needs PNetica
sess <- NeticaSession()
startSession(sess)
curd <- getwd()

netman1 <- read.csv(system.file("auxdata", "Mini-PP-Nets.csv", 
                                package="Peanut"),
                    row.names=1,stringsAsFactors=FALSE)

nodeman1 <- read.csv(system.file("auxdata", "Mini-PP-Nodes.csv", 
                                package="Peanut"),
                     row.names=1,stringsAsFactors=FALSE)

omegamat <- read.csv(system.file("auxdata", "miniPP-omega.csv",
                                package="Peanut"),
                     row.names=1,stringsAsFactors=FALSE)

## Insures we are building nets from scratch
setwd(tempdir())
## Network and node warehouse, to create networks and nodes on demand.
Nethouse <- BNWarehouse(manifest=netman1,session=sess,key="Name")

Nodehouse <- NNWarehouse(manifest=nodeman1,
                         key=c("Model","NodeName"),
                         session=sess)

## Build the proficiency model first:
CM <- WarehouseSupply(Nethouse,"miniPP_CM")
CM1 <- Omega2Pnet(omegamat,CM,Nodehouse,override=TRUE)

## Build the nets from the Qmat

Qmat2Pnet(Q1, Nethouse,Nodehouse)


## Build the Qmat from the nets
## Generate a list of nodes
obs <-unlist(sapply(list(sess$nets$PPcompEM,sess$nets$PPconjEM,
                  sess$nets$PPtwostepEM,sess$nets$PPdurAttEM),
             NetworkAllNodes))

Q2 <- Pnet2Qmat(obs,NetworkAllNodes(CM))

## adjust Q1 to match Q2
Q1 <- Q1[,-1]  ## Drop unused first column.
class(Q1) <- c("Qmat", "data.frame")
# Force them into the same order
Q1 <- Q1[order(Q1$Model,Q1$Node),]
Q2 <- Q2[order(Q2$Model,Q2$Node),]
row.names(Q1) <- NULL
row.names(Q2) <- NULL


## Force all NA columns into the right type
Q1$LinkScale <- as.numeric(Q1$LinkScale)
Q1$A.Physics <- as.numeric(Q1$A.Physics)
Q1$A.IterativeD <- as.numeric(Q1$A.IterativeD)
Q1$B.Physics <- as.numeric(Q1$B.Physics)
Q1$B.NTL <- as.numeric(Q1$B.NTL)

## Fix fancy quotes added by some spreadsheets
Q1$Rules <- gsub(intToUtf8(c(91,0x201C,0x201D,93)),"\"",Q1$Rules)

## Insert Default Prior Weights
Q1$PriorWeight <- ifelse(is.na(Q1$NStates),"","10")
all.equal(Q1,Q2)


stopSession(sess)
setwd(curd)

## End(Not run)

---

Merges (or separates) two Pnets with common variables

Description

In the hub-and-spoke Bayes net construction method, number of spoke models (evidence models in educational applications) are connected to a central hub model (proficiency models in educational applications). The PnetAdjoin operation combines a hub and spoke model to make a motif, replacing references to hub variables in the spoke model with the actual hub nodes. The PnetDetach operation reverses this.

Usage

PnetAdjoin(hub, spoke)
PnetDetach(motif, spoke)

Arguments

hub	A complete Pnet to which new variables will be added.
spoke	An incomplete Pnet which may contain stub nodes, references to nodes in the hub

.

motif

The combined Pnet which is formed by joining a hub and spoke together.

Details

The hub-and-spoke model for Bayes net construction (Almond and Mislevy, 1999; Almond, 2017) divides a Bayes net into a central hub model and a collection of spoke models. The motivation is that the hub model represents the status of a system—in educational applications, the proficiency of the student—and the spoke models are related to collections of evidence that can be collected about the system state. In the educational application, the spoke models correspond to a collection of observable outcomes from a test item or task. A motif is a hub plus a collection of spoke model corresponding to a single task.

While the hub model is a complete Bayesian network, the spoke models are fragments. In particular, several hub model variables are parents of variables in the spoke model. These variables are not defined in spoke model, but are rather replaced with stub nodes, nodes which reference, but do not define the spoke model.

The PnetAdjoin operation copies the Pnodes from the spoke model into the hub model, and connects the stub nodes to the nodes with the same name in the spoke model. The result is a motif consisting of the hub and the spoke. (If this operation is repeated many times it can be used to build an arbitrarily complex motif.)

The PnetDetach operation reverses the adjoin operation. It removes the nodes associated with the spoke model only, leaving the joint probability distribution of the hub model (along with any evidence absorbed by setting values of observable variables in the spoke) intact.

Value

The function PnetAdjoin returns a list of the newly created nodes corresponding to the spoke model nodes. Note that the names may have changed to avoid duplicate names. The names of the list are the spoke node names, so that any name changes can be discovered.

In both cases, the first argument is destructively modified, for PnetAdjoin the hub model becomes the motif. For PnetDetach the motif becomes the hub again.

Note

Node names must be unique within a Bayes net. If several spokes are attached to a hub and those spokes have common names for observable variables, then the names will need to be modified to make them unique. The function PnetAdjoin always returns the new nodes so that any name changes can be noted by the calling program.

I anticipate that there will be considerable varation in how these functions are implemented depending on the underlying implementation of the Bayes net package. In particular, there is no particular need for the PnetDetach function to do anything. While removing variables corresponding to an unneeded spoke model make the network smaller, they are harmless as far as calculations of the posterior distribution.

Author(s)

Russell Almond

References

Almond, R. G. & Mislevy, R. J. (1999) Graphical models and computerized adaptive testing. Applied Psychological Measurement, 23, 223–238.

Almond, R., Herskovits, E., Mislevy, R. J., & Steinberg, L. S. (1999). Transfer of information between system and evidence models. In Artificial Intelligence and Statistics 99, Proceedings (pp. 181–186). Morgan-Kaufman

Almond, R. G. (presented 2017, August). Tabular views of Bayesian networks. In John-Mark Agosta and Tomas Singlair (Chair), Bayeisan Modeling Application Workshop 2017. Symposium conducted at the meeting of Association for Uncertainty in Artificial Intelligence, Sydney, Australia. (International) Retrieved from http://bmaw2017.azurewebsites.net/

See Also

Pnet, PnetHub, Qmat2Pnet, PnetMakeStubNodes

Examples

## Not run: 
library(PNetica) # Requires PNetica
sess <- NeticaSession()
startSession(sess)

PM <- ReadNetworks(system.file("testnets", "miniPP-CM.dne",
                               package="PNetica"), session=sess)
EM1 <- ReadNetworks(system.file("testnets", "PPcompEM.dne",
                               package="PNetica"), session=sess)

Phys <- PnetFindNode(PM,"Physics")

## Prior probability for high level node
PnetCompile(PM)
bel1 <- PnodeMargin(PM, Phys)

## Adjoin the networks.
EM1.obs <- PnetAdjoin(PM,EM1)
PnetCompile(PM)

## Enter a finding
PnodeEvidence(EM1.obs[[1]]) <- "Right"
## Posterior probability for high level node

bel2 <- PnodeMargin(PM,Phys)

PnetDetach(PM,EM1)
PnetCompile(PM)

## Findings are unchanged
bel2a <- PnodeMargin(PM,Phys)
stopifnot(all.equal(bel2,bel2a,tol=1e-6))

DeleteNetwork(list(PM,EM1))
stopSession(sess)

## End(Not run)

---

Compiles a Parameterized Bayesian Network

Description

This function requests that the Bayes net be compiled—transformed so that inference can be carried out.

Usage

PnetCompile(net)

Arguments

net	A Pnet object to be compiled.

Details

Many Bayesian network algorithm have two phases. The graph is built as an acyclic directed graph. Before inference is carried out, the graph is transformed into a structure called a Junction Tree, Tree of Cliques or Markov Tree (Almond, 1995).

This function requests that implementation specific processing, particularly, building the appropriate Markov Tree, be done for the net, so that it can be placed in inference mode instead of editing mode.

Value

The compile net argument should be returned.

Note

It should be harmless to call this function on a net which is already compiled.

Author(s)

Russell Almond

References

Almond, R. G. (1995). Graphical Belief Models. Chapman and Hall.

See Also

The following functions will likely return errors if the net is not compiled: PnodeEvidence, calcStat, PnodeMargin, PnodeEAP, PnodeSD, PnodeMedian, PnodeMode.

Examples

## Not run: 

library(PNetica) ## Need a specific implementation
sess <- NeticaSession()
startSession(sess)

irt10.base <- ReadNetworks(system.file("testnets", "IRT10.2PL.base.dne",
                           package="PNetica"),session=sess)
irt10.base <- as.Pnet(irt10.base)  ## Flag as Pnet, fields already set.
irt10.theta <- PnetFindNode(irt10.base,"theta")
irt10.items <- PnetPnodes(irt10.base)
## Flag items as Pnodes
for (i in 1:length(irt10.items)) {
  irt10.items[[i]] <- as.Pnode(irt10.items[[i]])
  
}
## Make some statistics
marginTheta <- Statistic("PnodeMargin","theta","Pr(theta)")
meanTheta <- Statistic("PnodeEAP","theta","EAP(theta)")
sdTheta <- Statistic("PnodeSD","theta","SD(theta)")
medianTheta <- Statistic("PnodeMedian","theta","Median(theta)")
modeTheta <- Statistic("PnodeMedian","theta","Mode(theta)")


BuildAllTables(irt10.base)
PnetCompile(irt10.base) ## Netica requirement

calcStat(marginTheta,irt10.base)
calcStat(meanTheta,irt10.base)
calcStat(sdTheta,irt10.base)
calcStat(medianTheta,irt10.base)
calcStat(modeTheta,irt10.base)

PnodeEvidence(irt10.items[[1]]) <- "Correct"

calcStat(marginTheta,irt10.base)
calcStat(meanTheta,irt10.base)
calcStat(sdTheta,irt10.base)
calcStat(medianTheta,irt10.base)
calcStat(modeTheta,irt10.base)


DeleteNetwork(irt10.base)
stopSession(sess)

## End(Not run)

---