Game-based assessments are difficult to score because often the multiple observations that can be made from student game play are dependent. Bayesian networks, which can model the dependence, are thus attractive for providing scores; however, typically, only a small amount of pilot data are available at the time the network is constructed. This presentation will look at the process of creating Bayesian network scoring models, focusing on several practical techniques that have been used in the construction of models for _Physics Playground_ and _Plants vs Zombies_. In particular, it will look at: (1) The use of evidence-centered assessment design to define latent competency variables and observable indicator variables. (2) The use of discrete IRT models to create large portion of the Bayesian networks from a single spreadsheet. (3) The use of correlation matrixes to uncover and validate the conditional independence structure of the Bayes net. (4) Adjusting the Bayes net parameters using both hand tuning and a generalized EM algorithm, creating networks which are a mixture of expert opinion and data. (5) Using expected classification accuracy matrixes to judge assessment validity and reliability. (6) Using evidence balance sheets to identify unusual subjects and observable indicators.

Recent advances in machine learning have shown how it is possible to determine the causal direction, or direction of effect, between two continuous-valued random variables. We show how to use entropy to develop a simple and general framework for determining the causal direction. First, we consider the likelihood ratio under the linear non-Gaussian acyclic model (LiNGAM) and show how it gives rise to a non-Gaussianity measures based on differential entropy. Second, we develop a similar framework for the nonlinear additive noise model. We further discuss how to extend the framework to more than two variables, and how the framework is related to independent component analysis.

Bayesian network structure learning (BNSL) is the problem of finding a Bayesian network structure which best explains a given dataset. Score-based learning, a widely-used technique for solving BNSL, assigns a score to each network structure. The score measures the goodness of fit of that network to the data; commonly-used scores are based on, for example, Bayesian posterior likelihoods or minimum description length principles. Solving BNSL optimally is known to be NP-hard. Nevertheless, in the last decade, a variety of algorithms have been proposed which guarantee to find a network structure that optimizes a given scoring function. The goal of this work is to better understand the empirical behavior of these algorithms. In order to finish within a reasonable amount of time, most of the exact algorithms (safely) prune the search space using a variety of heuristics, such as branch-and-bound search. After briefly introducing several of the exact algorithms, I will discuss problem-dependent characteristics which affect the efficacy of the different heuristics. Empirical results show that, despite the complexity of the algorithms, machine learning techniques based on these characteristics can often be used to accurately predict the algorithms' running times. Within BNSL, the score typically reflects fit to a training dataset; however, it is well-known that a model may fit a training dataset very well but generalize poorly to unseen data. Thus, it is not clear that finding a network which optimizes a scoring function is worthwhile. The second part of this talk will focus on a study which compares exact and local search techniques. In some scenarios, algorithms such as greedy hill climbing or the polynomial-time Chow-Liu algorithm suffice; for more complex datasets, though, the exact algorithms consistently produce better networks.

Research on learning optimal Bayesian network structures from data, once thought impractical, have made strides in the last decade. One promising approach is based on admissible heuristic search. The approach formulates the learning problem as a shortest path problem, and uses search techniques such as A* to solve the problem in finding an optimal network structure. I will discuss how to create admissible heuristic functions for the algorithms. Moreover, I will discuss two techniques for extracting extra information from data to scale up the learning. The first technique can potentially lead to a decomposition of the learning problem to set a set of smaller independent learning problems, and the second technique creates tighter heuristic functions to enable the search to explore a smaller search space in finding the solution.

This talk presents an overview of methods for learning Bayesian networks. It discusses on recent methods and theoretical results to speed up computations and to improve accuracy, leading to an approach which can deal with many thousands of variables. Applications arising in biomedical problems are described, where it is argued that Bayesian networks can provide meaningful and interpretable results. In particular, we discuss on the use of Bayesian networks for data imputation, unsupervised clustering and classification using high-dimensional data sets of lymphoma patients.

We consider the problem of estimating the causal relations of two observed continuous variables in the presence of hidden common causes. We develop a non-Gaussian approach based on a linear non-Gaussian structural equation model known as LiNGAM and a linear mixed model. This approach does not require to explicitly model hidden common causes. We then propose to use an empirical Bayesian approach for estimating the model.