Learning Connectivity Networks from High-Dimensional Point Processes

Learning Connectivity Networks from High-Dimensional Point Processes Ali Shojaie Department of Biostatistics University of Washington faculty.washington.edu/ashojaie Feb 21st 2018

Motivation: Unlocking the Mysteries of the Brain The human brain is composed of 1011 neurons Question: How do neurons work together to perceive the world, make decisions, and perform other higher-level tasks? We will primarily focus on spike train data Sources: Allen Institute for Brain Science (left), Paul De Konnick lab (right) 1

Neuron Spike Train Data Time (s) 2

Neuron Spike Train Data Time (s) Spike Train: times at which a neuron spikes (transmits a signal) 2

Neuron Functional Connectivity Among Neurons Time (s) 3

Learning Functional Connectivity Networks 4

Neuron Challenges in Estimating Functional Connectivity Time (s) May observe thousands of neurons Limited theoretical justification Short duration of stationary period 5

Hawkes Process Introduced by Hawkes (1971) First applied to spike train data by Brillinger et al. 6

A Linear Hawkes Process intensity process point process spontaneous rate transfer function from k to j time when the kth neuron has the ith spike 7

A Linear Hawkes Process intensity process point process spontaneous rate transfer function from k to j time when the kth neuron has the ith spike Functional connectivity: there s an edge from k to j if 7

Intensity of Train 1 A Simple Hawkes Process 8

Intensity of Train 2 A Simple Hawkes Process 8

Penalized Regression for Hawkes Processes Joint work with Shizhe Chen, Eric Shea-Brown, and Daniela Witten The multivariate Hawkes process in high dimensions: Beyond mutual excitation (arxiv:1707.04928); invited revision to Annals of Statistics Nearly assumptionless screening for the mutually-exciting multivariate Hawkes process (2017) Electronic Journal of Statistics 9

Penalized Regression for Hawkes Processes Regress each spike train onto others Neighbourhood selection Estimate incoming edges Joint work with Shizhe Chen, Eric Shea-Brown, and Daniela Witten The multivariate Hawkes process in high dimensions: Beyond mutual excitation (arxiv:1707.04928); invited revision to Annals of Statistics Nearly assumptionless screening for the mutually-exciting multivariate Hawkes process (2017) Electronic Journal of Statistics 9

Parameter Estimation via Penalized Regression Model Finite-dimensional basis expansion Least square loss Regression 10

Parameter Estimation via Penalized Regression Model Finite-dimensional basis expansion Squared error loss Regression 10

Parameter Estimation via Penalized Regression Model Finite-dimensional basis expansion Squared error loss Regression Estimation via block coordinate descent 10

Properties of Penalized Estimation Procedures Existing theory relies on the cluster process representation Assumes non-negative transfer functions Only holds for linear Hawkes processes 11

Gap in Existing Theory: Neurons Excite and Inhibit 12

A New Concentration Inequality for Hawkes Process New theoretical framework that allows inhibition Use the thinning process representation For any j, k, consider Here can be any continuous and integrable function covers a wide range of second-order statistics of the Hawkes process, including the cross-covariance We have 13

An Application of the New Concentration Inequality Neighbourhood selection recovers the graph with high probability where and are true and estimated edges Key assumptions, i.e., we can handle Stationarity Other regularity conditions for lasso-type estimators 14

A Computational Shortcut Penalized regression becomes computationally (and statistically) challenging with many neurons Can we reduce the number of potential edges? 15

A Computational Shortcut Let Vj,k be the cross-covariance between the jth & kth neurons Consider the graph defined by marginal screening This correlation graph is often used by neuroscientists It is computationally (and statistically) efficient 16

Cross-Correlation Graph 17

Properties of Screening Recall Q: How does relate to the functional connectivity network,? 18

Properties of Screening If the process is mutually exciting, 18

Properties of Screening If the process is mutually exciting, These results can be shown using our new theoretical framework Unlike existing approaches, they do not require extra assumption 18

Properties of Screening What if there are negative edges? 19

Properties of Screening What if there are negative edges? Even with negative edges, screening detects connected components of the graph 19

Properties of Screening What if there are negative edges? Even with negative edges, screening detects connected components of the graph Screened Edges Connected Components 19

Neurons in Cat Visual Cortex 20

Addressing Non-Stationarity: Piecewise Stationary VARs Motivation: Analyzing EEG Data 21

Addressing Non-Stationarity: Piecewise Stationary VARs Motivation: Analyzing EEG Data Brain connectivities expected to change after seizure Goal: To locate the seizure and estimate before/after networks 21

Addressing Non-Stationarity: Piecewise Stationary VARs Our proposal: A 3-step procedure based on total variation penalty 22

Addressing Non-Stationarity: Piecewise Stationary VARs Our proposal: A 3-step procedure based on total variation penalty Joint work with Abolfazl Safikhani (Columbia Univ) Joint Structural Break Detection and Parameter Estimation in High-Dimensional Non-Stationary VAR Models (arxiv:1711.07357) 22

Acknowledgment Allen Institute for Brain Sciences Funding NIH: NIGMS & NHLBI NSF: DMS & DMS/NIGMS References Chen, Witten & Shojaie (2017) Nearly assumptionless screening for the mutually-exciting multivariate Hawkes process; Electronic Journal of Statistics, 11(1): 1207-1234. Chen, Shojaie, Shea-Brown & Witten (2018+) The multivariate Hawkes process in high dimensions: Beyond mutual excitation; revision invited to the Annals of Statistics (arxiv:1707.04928). Safikhani & Shojaie (2018+) Joint Structural Break Detection and Parameter Estimation in High-Dimensional Non-Stationary VAR Models (arxiv:1711.07357). 23

Key Dates Modules: July 11-28 Registration now open

Appendix I Theory for Hawkes Process with Inhibitions

Recap: One-Dimensional Linear Hawkes Process intensity process point process spontaneous rate transfer function time of the ith spike 1

Hawkes Process is Temporally Dependent by Definition 2

Hawkes Process is Temporally Dependent by Definition Key to understanding the Hawkes process: quantifying the temporal dependence 2

Temporal Dependence of a Hawkes Process 3

Existing Theory Assumes Non-Negative Transfer Functions 4

Represent Processes by Thinning a Poisson Process s Full Process t 5

Represent Processes by Thinning a Poisson Process Spike s Full Process & Thinned Process t 5

Represent Processes by Thinning a Poisson Process Spike s Full Process & Thinned Process This representation applies to any stationary Hawkes process! t 5

Spike s Bounding the Temporal Dependence Using the Thinning Process Representation t Time u 6

Spike s Bounding the Temporal Dependence Using the Thinning Process Representation t 6

Appendix II Iterative Construction of Thinning Process Representation for Hawkes Process

Recap: One-Dimensional Linear Hawkes Process intensity process point process spontaneous rate transfer function time of the ith spike 1

Thinning Process Representation of the Hawkes Process Spike s n=1 t 1

Thinning Process Representation of the Hawkes Process n=2 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

Thinning Process Representation of the Hawkes Process n=3 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

Thinning Process Representation of the Hawkes Process n=4 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

Thinning Process Representation of the Hawkes Process n=5 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

Thinning Process Representation of the Hawkes Process n=6 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

Thinning Process Representation of the Hawkes Process n=7 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

Thinning Process Representation of the Hawkes Process n=8 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

Thinning Process Representation of the Hawkes Process n=9 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

Thinning Process Representation of the Hawkes Process Spike s n=9 t 2

Appendix III Cluster Process Representation for the Hawkes Process

Cluster Process Representation Proposed by Hawkes and Oakes (1974) Represent a Hawkes process as the summation of processes Consider a one-dimensional Hawkes process Hawkes process

Cluster Process Representation Proposed by Hawkes and Oakes (1974) Represent a Hawkes process as the summation of processes Consider a one-dimensional Hawkes process Hawkes process Ancestral process

Cluster Process Representation Proposed by Hawkes and Oakes (1974) Represent a Hawkes process as the summation of processes Consider a one-dimensional Hawkes process Hawkes process Descendants Ancestral process

The End