dynamic-systems-DL

KO | EN

gitlite — search

#javascript #python #hacktoberfest #typescript #llm #go #machine-learning #deep-learning #linux #nodejs #cli #kubernetes #mcp #openai

dynamic-systems-DL

(★ 30)

Collection of resources that combine dynamic systems, control with deep learning.

dynamic-systems-DL 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## Dynamic Systems & Deep Learning

> A collection of resources that combine dynamical systems, control with deep learning. **Please feel free to submit a pull request if you want to add good papers. Welcome to contact me for cooperation if there are many overlaps in our research interests。**

## Differential Equations in Deep Learning

### General Architectures

* Recurrent Neural Networks for Multivariate Time Series with Missing Values: [Scientific Reports18](https://arxiv.org/abs/1606.01865)

* Deep Equilibrium Models: [NeurIPS19](https://arxiv.org/abs/1909.01377)

* Fast and Deep Graph Neural Networks: [AAAI20](https://arxiv.org/pdf/1911.08941.pdf)

### Neural ODEs

* Neural Ordinary Differential Equations: [NeurIPS18](https://arxiv.org/pdf/1806.07366.pdf)
* Augmented Neural ODEs: [NeurIPS19](https://arxiv.org/abs/1904.01681)
* Dissecting Neural ODEs: [arXiv20](https://arxiv.org/abs/2002.08071)
* Latent ODEs for Irregularly-Sampled Time Series: [arXiv19](https://arxiv.org/abs/1907.03907)
* Learning unknown ODE models with Gaussian processes: [arXiv18](https://arxiv.org/abs/1803.04303)
* ODE2VAE: Deep generative second order ODEs with Bayesian neural networks: [NeurIPS19](https://arxiv.org/pdf/1905.10994.pdf)
* Stable Neural Flows: [arXiv20](https://arxiv.org/abs/2003.08063)
* On Second Order Behaviour in Augmented Neural ODEs [arXiv20](https://arxiv.org/abs/2006.07220)
* Snode: Spectral discretization of neural odes for system identification [arXiv19](https://arxiv.org/abs/1906.07038)
* Learning Differential Equations that are Easy to Solve [NeurIPS20](https://arxiv.org/abs/2007.04504)    [code](https://github.com/jacobjinkelly/easy-neural-ode)
* An Ode to an ODE [arXiv20](https://arxiv.org/abs/2006.11421)
* ANODEV2: A Coupled Neural ODE Evolution Framework [arXiv19](https://arxiv.org/abs/1906.04596)
* Neural Manifold Ordinary Differential Equations. [NeurIPS20](https://arxiv.org/abs/2006.10254)
* Neural Delay Differential Equations [ICLR21](https://openreview.net/forum?id=Q1jmmQz72M2)
* Neural Jump Ordinary Differential Equations [ICLR21](https://openreview.net/forum?id=JFKR3WqwyXR)
* Learning Neural Event Functions for Ordinary Differential Equations. [ICLR21](https://openreview.net/forum?id=kW_zpEmMLdP)
* Neural ODE Processes. [ICLR21](https://openreview.net/forum?id=27acGyyI1BY)
* ResNet After All: Neural ODEs and Their Numerical Solution. [ICLR21](https://openreview.net/forum?id=HxzSxSxLOJZ)
* Go with the flow: Adaptive control for Neural ODEs. [ICLR21](https://openreview.net/forum?id=giit4HdDNa)
* MALI: A memory efficient and reverse accurate integrator for Neural ODEs. [ICLR21](https://openreview.net/forum?id=blfSjHeFM_e)
* On the Verification of Neural ODEs with Stochastic Guarantees. [AAAI21](https://arxiv.org/abs/2012.08863)
* Forecasting Reservoir Inflow via Recurrent Neural ODEs. [AAAI21]

#### Speed up Training of Neural ODEs

* Accelerating Neural ODEs with Spectral Elements: [arXiv19](https://arxiv.org/abs/1906.07038)
* Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE: [ICML20](https://arxiv.org/abs/2006.02493)
* "Hey, that's not an ODE": Faster ODE Adjoints with 12 Lines of Code [arXiv20]()
* How to Train you Neural ODE: [arXiv20](https://arxiv.org/abs/2002.02798)
* Hypersolvers: Toward Fast Continuous-Depth Models [NeurIPS20](http://proceedings.neurips.cc/paper/2020/hash/f1686b4badcf28d33ed632036c7ab0b8-Abstract.html)
* Interpolation technique to speed up gradients propagation in Neural Ordinary Differential Equations. [NeurIPS20](https://papers.nips.cc/paper/2020/file/c24c65259d90ed4a19ab37b6fd6fe716-Paper.pdf)

### Neural SDEs

* Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise: [arXiv19](https://arxiv.org/abs/1906.02355)
* Neural Jump Stochastic Differential Equations: [arXiv19](https://arxiv.org/abs/1905.10403)
* Towards Robust and Stable Deep Learning Algorithms for Forward Backward Stochastic Differential Equations: [arXiv19](https://arxiv.org/abs/1910.11623)
* Scalable Gradients and Variational Inference for Stochastic Differential Equations [AISTATS20](https://arxiv.org/abs/2001.01328)
* Score-Based Generative Modeling through Stochastic Differential Equations. [ICLR21](https://openreview.net/forum?id=PxTIG12RRHS)
*  Neural SDEs Made Easy: SDEs are Infinite-Dimensional GANs [url](https://openreview.net/forum?id=padYzanQNbg)

### Neural CSDEs

*  Learning Continuous-Time Dynamics by Stochastic Differential Networks. [url](https://openreview.net/forum?id=U850oxFSKmN)

### Neural CDEs

* Neural Controlled Differential Equations for Irregular Time Series: [ArXiv2020](https://arxiv.org/abs/2005.08926)
* Neural CDEs for Long Time Series via the Log-ODE Method: [ArXiv2020](https://arxiv.org/abs/2009.08295)

### Normalizing Flows

* Monge-Ampère Flow for Generative Modeling: [arXiv18](https://arxiv.org/pdf/1809.10188.pdf)
* FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models: [ICLR19](https://arxiv.org/abs/1810.01367)
* Equivariant Flows: sampling configurations for multi-body systems with symmetric energies: [arXiv18](https://arxiv.org/pdf/1910.00753.pdf)
* OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport. [AAAI21]
* Accelerating Continuous Normalizing Flow with Trajectory Polynomial Regularization. [AAAI21]

### Applications

* Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning: [NeurIPS19](https://arxiv.org/abs/1905.11666)
* Graph Neural Ordinary Differential Equations  [arXiv19](https://arxiv.org/abs/1911.07532)
* Continuous graph neural networks [ICML2020](https://arxiv.org/abs/1912.00967)
* Neural Dynamics on Complex Networks [arXiv19](https://arxiv.org/pdf/1908.06491.pdf)

## Energy based models

### Hamilton

* Hamiltonian Neural Networks: [NeurIPS19](https://arxiv.org/abs/1906.01563)  [code](https://github.com/greydanus/hamiltonian-nn)
* Hamiltonian generative networks[ICLR2020](https://arxiv.org/abs/1909.13789)  [code](https://github.com/gaspardbb/HamiltonianGenerativeNetworks)
* Sparse Symplectically Integrated Neural Networks [NeurIPS20](https://arxiv.org/abs/2006.12972)      [code](https://github.com/dandip/ssinn)
* Nonseparable symplectic neural networks   [code](https://openreview.net/forum?id=B5VvQrI49Pa)
* SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems [arxiV20](https://arxiv.org/abs/2001.03750)
* Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control: [arXiv19](https://arxiv.org/abs/1909.12077)
* Symplectic Recurrent Neural Network [arXiv19](https://arxiv.org/abs/1909.13334)
* Conformation-Guided Molecular Representation with Hamiltonian Neural Networks. [ICLR21](https://openreview.net/forum?id=q-cnWaaoUTH)
* Identifying Physical Law of Hamiltonian Systems via Meta-Learning. [ICLR21](https://openreview.net/forum?id=45NZvF1UHam)

#### Applications

* Hamiltonian graph networks with ode integrators [arXiv19](https://arxiv.org/abs/1909.12790)

### Lagrange

* Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning: [ICLR19](https://arxiv.org/abs/1907.04490)
* Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Control [NeurIPS20](https://proceedings.neurips.cc/paper/2020/file/79f56e5e3e0e999b3c139f225838d41f-Supplemental.pdf)
* Lagrangian Neural Networks: [ICLR20 DeepDiffEq](https://arxiv.org/abs/2003.04630)

## Deep Learning Methods for Differential Equations

### Solving Differential Equations

### Learning PDEs

* PDE-Net: Learning PDEs From Data: [ICML18](https://arxiv.org/abs/1710.09668)
* PDE-Net 2.0: Learning PDEs from Data [Journal of Computational Physics](https://arxiv.org/abs/1812.04426)
* Solving parametric PDE problems with artificial neural networks [arXiv](https://arxiv.org/abs/1707.03351)
* Multipole Graph Neural Operator for Parametric Partial Differential Equations. [NeurIPS20](https://arxiv.org/abs/2006.09535)
* Numerically Solving Parametric Families of High-Dimensional Kolmogorov Partial Differential Equations via Deep Learning [NeurIPS20](http://arxiv.org/abs/2011.04602)
* A neural method for symbolically solving partial differential equations [url](https://openreview.net/forum?id=c_YKhwEdUcq)
* Neural Partial Differential Equations with Functional Convolution. [url](https://openreview.net/forum?id=D4A-v0kltaX)
* Fourier Neural Operator for Parametric Partial Differential Equations. [ICLR21](https://openreview.net/forum?id=c8P9NQVtmnO)
* Learning continuous-time PDEs from sparse data with graph neural networks. [ICLR21](https://openreview.net/forum?id=aUX5Plaq7Oy)

### Applications
* PDE-regularized Neural Networks for Image Classification.[url](https://openreview.net/forum?id=DlPnp5_1JMI)
* PDE-Driven Spatiotemporal Disentanglement. [ICLR21](https://openreview.net/forum?id=vLaHRtHvfFp)

### Model Discovery

* Universal Differential Equations for Scientific Machine Learning: [arXiv20](https://arxiv.org/abs/2001.04385)

## Deep Control

### Model-Predictive-Control

* Differentiable MPC for End-to-end Planning and Control: [NeurIPS18](https://arxiv.org/abs/1810.13400)
* Neural lyapunov model predictive control [arXiv20](https://arxiv.org/abs/2002.10451)

## Dynamical System View of Deep Learning

### Recurrent Neural Networks

* A Comprehensive Review of Stability Analysis of Continuous-Time Recurrent Neural Networks: [IEEE Transactions on Neural Networks 2006](https://ieeexplore.ieee.org/abstract/document/6814892)
* AntysimmetricRNN: A Dynamical System View on Recurrent Neural Networks: [ICLR19](https://openreview.net/pdf?id=ryxepo0cFX)
* Recurrent Neural Networks in the Eye of Differential Equations: [arXiv19](https://arxiv.org/pdf/1904.12933.pdf)
* Visualizing memorization in RNNs: [distill19](https://distill.pub/2019/memorization-in-rnns/)
* One step back, two steps forward: interference and learning in recurrent neural networks: [arXiv18](https://arxiv.org/abs/1805.09603)
* Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics: [arXiv19](https://arxiv.org/pdf/1906.10720.pdf)
* System Identification with Time-Aware Neural Sequence Models: [AAAI20](https://arxiv.org/abs/1911.09431)
* Universality and Individuality in recurrent networks: [NeurIPS19](https://arxiv.org/abs/1907.08549)
* The LOB Recreation Model: Predicting the Limit Order Book from TAQ History Using an Ordinary Differential Equation Recurrent Neural Network. [AAAI21]
* Lipschitz Recurrent Neural Networks. [ICLR21](https://arxiv.org/abs/2006.12070)

### Theory and Perspectives

* Deep information propagation [arXiv16](https://arxiv.org/abs/1611.01232)
* A mean field optimal control formulation of deep learning [Research in Mathematical Science](https://link.springer.com/article/10.1007/s40687-018-0172-y)
* A Proposal on Machine Learning via Dynamical Systems: [Communications in Mathematics and Statistics 2017](https://link.springer.com/content/pdf/10.1007/s40304-017-0103-z.pdf)
* Deep learning as optimal control problems:models and numerical methods [arXiv19](https://arxiv.org/abs/1904.05657)
* Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective: [arXiv19](https://arxiv.org/abs/1908.10920)
* Stable Architectures for Deep Neural Networks: [IP17](https://arxiv.org/pdf/1705.03341.pdf)
* Beyond Finite Layer Neural Network: Bridging Deep Architects and Numerical Differential Equations: [ICML18](https://arxiv.org/abs/1710.10121)
* Review: Ordinary Differential Equations For Deep Learning: [arXiv19](https://arxiv.org/abs/1911.00502)
* Universal approximation power of deep residual neural networks via nonlinear control theory. [ICLR21](https://openreview.net/forum?id=-IXhmY16R3M)
* Physics-aware, probabilistic model order reduction with guaranteed stability. [ICLR21](https://openreview.net/forum?id=vyY0jnWG-tK)
* Dynamics of Deep Equilibrium Linear Models. [ICLR21](https://openreview.net/forum?id=p-NZIuwqhI4)
* Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies . [ICLR21](https://openreview.net/forum?id=_XYzwxPIQu6)

### Optimization

* Gradient and Hamiltonian Dynamics Applied to Learning in Neural Networks: [NIPS96](https://papers.nips.cc/paper/1033-gradient-and-hamiltonian-dynamics-applied-to-learning-in-neural-networks.pdf)
* Maximum Principle Based Algorithms for Deep Learning: [JMLR17](https://arxiv.org/abs/1710.09513)
* Hamiltonian Descent Methods: [arXiv18](https://arxiv.org/pdf/1809.05042.pdf)
* Port-Hamiltonian Approach to Neural Network Training: [CDC19](https://arxiv.org/abs/1909.02702), [code](https://github.com/Zymrael/PortHamiltonianNN)
* An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks: [arXiv19](https://arxiv.org/abs/1803.01299)
* Optimizing Millions of Hyperparameters by Implicit Differentiation: [arXiv19](https://arxiv.org/abs/1911.02590)
* Shadowing Properties of Optimization Algorithms: [NeurIPS19](https://papers.nips.cc/paper/9431-shadowing-properties-of-optimization-algorithms)
* NOVAS: Non-convex Optimization via Adaptive Stochastic Search for End-to-end Learning and Control. [ICLR21](https://openreview.net/forum?id=Iw4ZGwenbXf)
* A Hybrid Stochastic Gradient Hamiltonian Monte Carlo Method. [AAAI21]

### Signals and systems

*  Multiplicative Filter Networks. [ICLR21](https://openreview.net/forum?id=OmtmcPkkhT)

### Others

* Training Generative Adversarial Networks by Solving Ordinary Differential Equations. [NeurIPS20](https://arxiv.org/abs/2010.15040)
* Variational Intrinsic Control Revisited. [ICLR21](https://openreview.net/forum?id=P0p33rgyoE)
* ECG ODE-GAN: Learning Ordinary Differential Equations of ECG Dynamics via Generative Adversarial Learning [AAAI21]

## Software and Libraries

### Python

* **torchdyn**: PyTorch library for all things neural differential equations. [repo](https://github.com/diffeqml/torchdyn), [docs](https://torchdyn.readthedocs.io/)
* **torchdiffeq**: Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation: [repo](https://github.com/rtqichen/torchdiffeq)
* **torchsde**: Stochastic differential equation (SDE) solvers with GPU support and efficient sensitivity analysis: [repo](https://github.com/google-research/torchsde)
* **torchSODE**: PyTorch Block-Diagonal ODE solver: [repo](https://github.com/Zymrael/torchSODE)
* **JAX MD**: JAX MD: A Framework for Differentiable Physics. [repo](https://papers.nips.cc/paper/2020/hash/83d3d4b6c9579515e1679aca8cbc8033-Abstract.html) [NeurIPS20](https://arxiv.org/abs/1912.04232)

### Julia

* **DiffEqFlux**: Neural differential equation solvers with O(1) backprop, GPUs, and stiff+non-stiff DE solvers. 
  Supports stiff and non-stiff neural ordinary differential equations (neural ODEs), neural stochastic differential 
  equations (neural SDEs), neural delay differential equations (neural DDEs), neural partial differential 
  equations (neural PDEs), and neural jump stochastic differential equations (neural jump diffusions).
  All of these can be solved with high order methods with adaptive time-stepping and automatic stiffness
  detection to switch between methods. [repo](https://github.com/JuliaDiffEq/DiffEqFlux.jl)

* **NeuralNetDiffEq**: Implementations of ODE, SDE, and PDE solvers via deep neural networks: [repo](https://github.com/JuliaDiffEq/NeuralNetDiffEq.jl)

## Websites and Blogs

* Scientific ML Blog (Chris Rackauckas and SciML): [link](http://www.stochasticlifestyle.com/)

## Slides

* [NeurIPS20 Implicit Layers](https://implicit-layers-tutorial.org/implicit_tutorial.pdf)

* [ODE Talk](https://web.stanford.edu/~yplu/ODETalk.pdf)

dynamic-systems-DL

dynamic-systems-DL 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

dynamic-systems-DL

dynamic-systems-DL 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README