Learning Connectivity Networks from High-Dimensional Point Processes

Transcription

1 Learning Connectivity Networks from High-Dimensional Point Processes Ali Shojaie Department of Biostatistics University of Washington faculty.washington.edu/ashojaie Feb 21st 2018

2 Motivation: Unlocking the Mysteries of the Brain The human brain is composed of 1011 neurons Question: How do neurons work together to perceive the world, make decisions, and perform other higher-level tasks? We will primarily focus on spike train data Sources: Allen Institute for Brain Science (left), Paul De Konnick lab (right) 1

3 Neuron Spike Train Data Time (s) 2

4 Neuron Spike Train Data Time (s) 2

5 Neuron Spike Train Data Time (s) 2

6 Neuron Spike Train Data Time (s) Spike Train: times at which a neuron spikes (transmits a signal) 2

7 Neuron Functional Connectivity Among Neurons Time (s) 3

8 Neuron Functional Connectivity Among Neurons Time (s) 3

9 Neuron Functional Connectivity Among Neurons Time (s) 3

10 Neuron Functional Connectivity Among Neurons Time (s) 3

11 Neuron Functional Connectivity Among Neurons Time (s) 3

12 Neuron Functional Connectivity Among Neurons Time (s) 3

13 Learning Functional Connectivity Networks 4

14 Learning Functional Connectivity Networks 4

15 Learning Functional Connectivity Networks 4

16 Learning Functional Connectivity Networks 4

17 Learning Functional Connectivity Networks 4

18 Neuron Challenges in Estimating Functional Connectivity Time (s) May observe thousands of neurons Limited theoretical justification Short duration of stationary period 5

19 Neuron Challenges in Estimating Functional Connectivity Time (s) May observe thousands of neurons Limited theoretical justification Short duration of stationary period 5

20 Neuron Challenges in Estimating Functional Connectivity Time (s) May observe thousands of neurons Limited theoretical justification Short duration of stationary period 5

21 Neuron Challenges in Estimating Functional Connectivity Time (s) May observe thousands of neurons Limited theoretical justification Short duration of stationary period 5

22 Neuron Challenges in Estimating Functional Connectivity Time (s) May observe thousands of neurons Limited theoretical justification Short duration of stationary period 5

23 Neuron Challenges in Estimating Functional Connectivity Time (s) May observe thousands of neurons Limited theoretical justification Short duration of stationary period 5

24 Hawkes Process Introduced by Hawkes (1971) First applied to spike train data by Brillinger et al. 6

25 A Linear Hawkes Process intensity process point process spontaneous rate transfer function from k to j time when the kth neuron has the ith spike 7

26 A Linear Hawkes Process intensity process point process spontaneous rate transfer function from k to j time when the kth neuron has the ith spike Functional connectivity: there s an edge from k to j if 7

27 Intensity of Train 1 A Simple Hawkes Process 8

28 Intensity of Train 1 A Simple Hawkes Process 8

29 Intensity of Train 1 A Simple Hawkes Process 8

30 Intensity of Train 1 A Simple Hawkes Process 8

31 Intensity of Train 1 A Simple Hawkes Process 8

32 Intensity of Train 1 A Simple Hawkes Process 8

33 Intensity of Train 2 A Simple Hawkes Process 8

34 Intensity of Train 2 A Simple Hawkes Process 8

35 Intensity of Train 2 A Simple Hawkes Process 8

36 Intensity of Train 2 A Simple Hawkes Process 8

37 Intensity of Train 2 A Simple Hawkes Process 8

38 Intensity of Train 2 A Simple Hawkes Process 8

39 Penalized Regression for Hawkes Processes Joint work with Shizhe Chen, Eric Shea-Brown, and Daniela Witten The multivariate Hawkes process in high dimensions: Beyond mutual excitation (arxiv: ); invited revision to Annals of Statistics Nearly assumptionless screening for the mutually-exciting multivariate Hawkes process (2017) Electronic Journal of Statistics 9

40 Penalized Regression for Hawkes Processes Regress each spike train onto others Neighbourhood selection Estimate incoming edges Joint work with Shizhe Chen, Eric Shea-Brown, and Daniela Witten The multivariate Hawkes process in high dimensions: Beyond mutual excitation (arxiv: ); invited revision to Annals of Statistics Nearly assumptionless screening for the mutually-exciting multivariate Hawkes process (2017) Electronic Journal of Statistics 9

41 Parameter Estimation via Penalized Regression Model Finite-dimensional basis expansion Least square loss Regression 10

42 Parameter Estimation via Penalized Regression Model Finite-dimensional basis expansion Least square loss Regression 10

43 Parameter Estimation via Penalized Regression Model Finite-dimensional basis expansion Squared error loss Regression 10

44 Parameter Estimation via Penalized Regression Model Finite-dimensional basis expansion Squared error loss Regression 10

45 Parameter Estimation via Penalized Regression Model Finite-dimensional basis expansion Squared error loss Regression 10

46 Parameter Estimation via Penalized Regression Model Finite-dimensional basis expansion Squared error loss Regression Estimation via block coordinate descent 10

47 Properties of Penalized Estimation Procedures Existing theory relies on the cluster process representation Assumes non-negative transfer functions Only holds for linear Hawkes processes 11

48 Gap in Existing Theory: Neurons Excite and Inhibit 12

49 A New Concentration Inequality for Hawkes Process New theoretical framework that allows inhibition Use the thinning process representation For any j, k, consider Here can be any continuous and integrable function covers a wide range of second-order statistics of the Hawkes process, including the cross-covariance We have 13

50 A New Concentration Inequality for Hawkes Process New theoretical framework that allows inhibition Use the thinning process representation For any j, k, consider Here can be any continuous and integrable function covers a wide range of second-order statistics of the Hawkes process, including the cross-covariance 13

51 A New Concentration Inequality for Hawkes Process New theoretical framework that allows inhibition Use the thinning process representation For any j, k, consider Here can be any continuous and integrable function covers a wide range of second-order statistics of the Hawkes process, including the cross-covariance We have 13

52 An Application of the New Concentration Inequality Neighbourhood selection recovers the graph with high probability where and are true and estimated edges Key assumptions, i.e., we can handle Stationarity Other regularity conditions for lasso-type estimators 14

53 A Computational Shortcut Penalized regression becomes computationally (and statistically) challenging with many neurons Can we reduce the number of potential edges? 15

54 A Computational Shortcut Let Vj,k be the cross-covariance between the jth & kth neurons Consider the graph defined by marginal screening This correlation graph is often used by neuroscientists It is computationally (and statistically) efficient 16

55 Cross-Correlation Graph 17

56 Cross-Correlation Graph 17

57 Cross-Correlation Graph 17

58 Cross-Correlation Graph 17

59 Cross-Correlation Graph 17

60 Cross-Correlation Graph 17

61 Cross-Correlation Graph 17

62 Properties of Screening Recall Q: How does relate to the functional connectivity network,? 18

63 Properties of Screening If the process is mutually exciting, 18

64 Properties of Screening If the process is mutually exciting, 18

65 Properties of Screening If the process is mutually exciting, These results can be shown using our new theoretical framework Unlike existing approaches, they do not require extra assumption 18

66 Properties of Screening If the process is mutually exciting, These results can be shown using our new theoretical framework Unlike existing approaches, they do not require extra assumption 18

67 Properties of Screening What if there are negative edges? 19

68 Properties of Screening What if there are negative edges? 19

69 Properties of Screening What if there are negative edges? Even with negative edges, screening detects connected components of the graph 19

70 Properties of Screening What if there are negative edges? Even with negative edges, screening detects connected components of the graph 19

71 Properties of Screening What if there are negative edges? Even with negative edges, screening detects connected components of the graph 19

72 Properties of Screening What if there are negative edges? Even with negative edges, screening detects connected components of the graph Screened Edges Connected Components 19

73 Neurons in Cat Visual Cortex 20

74 Addressing Non-Stationarity: Piecewise Stationary VARs Motivation: Analyzing EEG Data 21

75 Addressing Non-Stationarity: Piecewise Stationary VARs Motivation: Analyzing EEG Data 21

76 Addressing Non-Stationarity: Piecewise Stationary VARs Motivation: Analyzing EEG Data 21

77 Addressing Non-Stationarity: Piecewise Stationary VARs Motivation: Analyzing EEG Data Brain connectivities expected to change after seizure Goal: To locate the seizure and estimate before/after networks 21

78 Addressing Non-Stationarity: Piecewise Stationary VARs Our proposal: A 3-step procedure based on total variation penalty 22

79 Addressing Non-Stationarity: Piecewise Stationary VARs Our proposal: A 3-step procedure based on total variation penalty 22

80 Addressing Non-Stationarity: Piecewise Stationary VARs Our proposal: A 3-step procedure based on total variation penalty 22

81 Addressing Non-Stationarity: Piecewise Stationary VARs Our proposal: A 3-step procedure based on total variation penalty Joint work with Abolfazl Safikhani (Columbia Univ) Joint Structural Break Detection and Parameter Estimation in High-Dimensional Non-Stationary VAR Models (arxiv: ) 22

82 Acknowledgment Allen Institute for Brain Sciences Funding NIH: NIGMS & NHLBI NSF: DMS & DMS/NIGMS References Chen, Witten & Shojaie (2017) Nearly assumptionless screening for the mutually-exciting multivariate Hawkes process; Electronic Journal of Statistics, 11(1): Chen, Shojaie, Shea-Brown & Witten (2018+) The multivariate Hawkes process in high dimensions: Beyond mutual excitation; revision invited to the Annals of Statistics (arxiv: ). Safikhani & Shojaie (2018+) Joint Structural Break Detection and Parameter Estimation in High-Dimensional Non-Stationary VAR Models (arxiv: ). 23

83 Acknowledgment Allen Institute for Brain Sciences Funding NIH: NIGMS & NHLBI NSF: DMS & DMS/NIGMS References Chen, Witten & Shojaie (2017) Nearly assumptionless screening for the mutually-exciting multivariate Hawkes process; Electronic Journal of Statistics, 11(1): Chen, Shojaie, Shea-Brown & Witten (2018+) The multivariate Hawkes process in high dimensions: Beyond mutual excitation; revision invited to the Annals of Statistics (arxiv: ). Safikhani & Shojaie (2018+) Joint Structural Break Detection and Parameter Estimation in High-Dimensional Non-Stationary VAR Models (arxiv: ). Thank You! 23

84 Key Dates Modules: July Registration now open

85 Appendix I Theory for Hawkes Process with Inhibitions

86 Recap: One-Dimensional Linear Hawkes Process intensity process point process spontaneous rate transfer function time of the ith spike 1

87 Hawkes Process is Temporally Dependent by Definition 2

88 Hawkes Process is Temporally Dependent by Definition Key to understanding the Hawkes process: quantifying the temporal dependence 2

89 Temporal Dependence of a Hawkes Process 3

90 Temporal Dependence of a Hawkes Process 3

91 Existing Theory Assumes Non-Negative Transfer Functions 4

92 Existing Theory Assumes Non-Negative Transfer Functions 4

93 Represent Processes by Thinning a Poisson Process s Full Process t 5

94 Represent Processes by Thinning a Poisson Process Spike s Full Process & Thinned Process t 5

95 Represent Processes by Thinning a Poisson Process Spike s Full Process & Thinned Process This representation applies to any stationary Hawkes process! t 5

96 Spike s Bounding the Temporal Dependence Using the Thinning Process Representation t Time u 6

97 Spike s Bounding the Temporal Dependence Using the Thinning Process Representation t 6

98 Appendix II Iterative Construction of Thinning Process Representation for Hawkes Process

99 Recap: One-Dimensional Linear Hawkes Process intensity process point process spontaneous rate transfer function time of the ith spike 1

100 Thinning Process Representation of the Hawkes Process Spike s n=1 t 1

101 Thinning Process Representation of the Hawkes Process n=2 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

102 Thinning Process Representation of the Hawkes Process n=3 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

103 Thinning Process Representation of the Hawkes Process n=4 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

104 Thinning Process Representation of the Hawkes Process n=5 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

105 Thinning Process Representation of the Hawkes Process n=6 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

106 Thinning Process Representation of the Hawkes Process n=7 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

107 Thinning Process Representation of the Hawkes Process n=8 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

108 Thinning Process Representation of the Hawkes Process n=9 Intensity in the previous iteration Spike s Intensity in the current iteration Removed spikes t New spikes 1

109 Thinning Process Representation of the Hawkes Process Spike s n=9 t 2

110 Appendix III Cluster Process Representation for the Hawkes Process

111 Cluster Process Representation Proposed by Hawkes and Oakes (1974) Represent a Hawkes process as the summation of processes Consider a one-dimensional Hawkes process Hawkes process

112 Cluster Process Representation Proposed by Hawkes and Oakes (1974) Represent a Hawkes process as the summation of processes Consider a one-dimensional Hawkes process Hawkes process Ancestral process

113 Cluster Process Representation Proposed by Hawkes and Oakes (1974) Represent a Hawkes process as the summation of processes Consider a one-dimensional Hawkes process Hawkes process Descendants Ancestral process

114 Cluster Process Representation Proposed by Hawkes and Oakes (1974) Represent a Hawkes process as the summation of processes Consider a one-dimensional Hawkes process Hawkes process Descendants Ancestral process

115 Cluster Process Representation Proposed by Hawkes and Oakes (1974) Represent a Hawkes process as the summation of processes Consider a one-dimensional Hawkes process Hawkes process Descendants Ancestral process

116 Cluster Process Representation Proposed by Hawkes and Oakes (1974) Represent a Hawkes process as the summation of processes Consider a one-dimensional Hawkes process Hawkes process Descendants Ancestral process Only holds for linear Hawkes processes with

117 The End