We introduce a doubly stochastic proximal gradient algorithm for optimizing a finite average of smooth convex subfunctions, whose gradients depend on numerically expensive expectations. Indeed, the effectiveness of SGD- like algorithms relies on the assumption that the computation of a subfunction’s gradient is cheap compared to the computation of the total function’s gradient. This is true in the Empirical Risk Minimization (ERM) setting, but can be false when each subfunction depends on a sequence of examples. Our main motivation is acceleration of the training-time of the penalized Cox partial-likelihood, which is the core model used in survival analysis for the study of clinical data, but our algorithm can be used in different settings as well. The proposed algorithm is doubly stochastic in the sense that gradient steps are done using stochastic gradient descent (SGD) with variance reduction, where the inner expectations are approximated by a Monte-Carlo Markov-Chain (MCMC) algorithm. We derive conditions on the MCMC number of iterations under which convergence is guaranteed, and prove that a linear rate of convergence can be achieved under strong convexity. This exhibits a similar behaviour as recent SGD-like algorithms such as Prox-SVRG (which is the basis of our algorithm), SAGA and SDCA. We illustrate numerically the strong improvement given by our algorithm, in comparison with a state-of-the-art solver used for the Cox partial-likelihood
In this paper we propose an overview of the recent academic literature devoted to the applications of Hawkes processes in finance. Hawkes processes constitute a particular class of multivariate point processes that has become very popular in empirical high frequency finance this last decade. After a reminder of the main definitions and properties that characterize Hawkes processes, we review their main empirical applications to address many different problems in high frequency finance. Because of their great flexibility and versatility, we show that they have been successfully involved in issues as diverse as estimating the volatility at the level of transaction data, estimating the market stability, accounting for systemic risk contagion, devising optimal execution strategies or capturing the dynamics of the full order book.
We consider the problem of unveiling the implicit network structure of user interactions in a social network, based only on high-frequency timestamps. Our inference is based on the minimization of the least-squares loss associated with a multivariate Hawkes model, penalized by L1 and trace norms. We provide a first theoretical analysis of the generalization error for this problem, that includes sparsity and low-rank inducing priors. This result involves a new data-driven concentration inequality for matrix martingales in continuous time with observable variance, which is a result of independent interest. A consequence of our analysis is the construction of sharply tuned L1 and trace-norm penalizations, that leads to a data-driven scaling of the variability of information available for each users. Numerical experiments illustrate the strong improvements achieved by the use of such data-driven penalizations.
This paper gives new concentration inequalities for the spectral norm of matrix martingales in continuous time. Both cases of purely discountinuous and continuous martingales are considered. The analysis is based on a new supermartingale property of the trace exponential, based on tools from stochastic calculus. Matrix martingales in continuous time are probabilistic objects that naturally appear for statistical learning of time-dependent sys- tems. We focus here on the the microscopic study of (social) networks, based on self-exciting counting processes, such as the Hawkes process, together with a low-rank prior assumption of the self-exciting component. A consequence of these new concentration inequalities is a push forward of the theoretical analysis of such models.
We present a modified version of the non parametric Hawkes kernel estimation procedure studied in [5] that is adapted to slowly decreasing kernels. We show on numerical simulations involving a reasonable number of events that this method allows us to estimate faithfully a power-law decreasing kernel over at least 6 decades. We then propose a 8-dimensional Hawkes model for all events associated with the first level of some asset order book. Applying our estimation procedure to this model, allows us to uncover the main properties of the coupled dynamics of trade, limit and cancel orders in relationship with the mid-price variations.
In this work we investigate the generic properties of a stochastic linear model in the regime of high-dimensionality. We consider in particular the Vector AutoRegressive model (VAR) and the multivariate Hawkes process. We analyze both deterministic and random versions of these models, showing the existence of a stable and an unstable phase. We find that along the transition region separating the two regimes, the cor- relations of the process decay slowly, and we characterize the conditions under which these slow correlations are expected to become power-laws. We check our findings with numerical simulations showing remarkable agreement with our predictions. We finally argue that real systems with a strong degree of self-interaction are naturally characterized by this type of slow relaxation of the correlations.
Download Pdf version
In this paper, we use a database of around 400,000 metaorders issued by investors and electronically traded on European markets in 2010 in order to study market impact at different scales.
At the intraday scale we confirm a square root temporary impact in the daily participation, and we shed light on a duration factor in 1/T γ with γ ? 0.25. Including this factor in the fits reinforces the square root shape of impact. We observe a power-law for the transient impact with an exponent between 0.5 (for long metaorders) and 0.8 (for shorter ones). Moreover we show that the market does not anticipate the size of the meta-orders. The intraday decay seems to exhibit two regimes (though hard to identify precisely): a “slow” regime right after the execution of the meta-order followed by a faster one. At the daily time scale, we show price moves after a metaorder can be split between realizations of expected returns that have triggered the investing decision and an idiosynchratic impact that slowly decays to zero. Moreover we propose a class of toy models based on Hawkes processes (the Hawkes Impact Models, HIM) to illustrate our reasoning. We show how the Impulsive-HIM model, despite its simplicity, embeds appealing features like transience and decay of impact. The latter is parametrized by a parameter C having a macroscopic interpretation: the ratio of contrarian reaction (i.e. impact decay) and of the "herding" reaction (i.e. impact amplification).
Download Pdf version
We show that the jumps correlation matrix of a multivariate Hawkes process is related to the Hawkes kernel matrix through a system of Wiener-Hopf integral equations. A Wiener-Hopf argument allows one to prove that this system (in which the kernel matrix is the unknown) possesses a unique causal solution and consequently that the second-order properties fully characterize a Hawkes process. The numerical inversion of this system of integral equations allows us to propose a fast and efficient method, which main principles were initially sketched in [Bacry and Muzy, 2013], to perform a non-parametric estimation of the Hawkes kernel matrix. In this paper, we perform a systematic study of this non-parametric estimation procedure in the general framework of marked Hawkes processes. We describe precisely this procedure step by step. We discuss the estimation error and explain how the values for the main parameters should be chosen. Various numerical examples are given in order to illustrate the broad possibilities of this estimation procedure ranging from 1-dimensional (power-law or non positive kernels) up to 3-dimensional (circular dependence) processes. A comparison to other non-parametric estimation procedures is made. Applications to high frequency trading events in financial markets and to earthquakes occurrence dynamics are finally considered.
Download Pdf version
Scattering moments provide non-parametric models of random
processes with stationary increments. They are expected values of
random variables computed with a non-expansive operator, obtained by
iteratively applying wavelet transforms and modulus non-linearities,
which preserves the variance. First and second order scattering
moments are shown to characterize intermittency and self-similarity
properties of multiscale processes. Scattering moments of Poisson
processes, fractional Brownian motions, Levy processes and
multifractal random walks are shown to have characteristic decay.
The Generalized Method of Simulated Moments is applied to scattering
moments to estimate data generating models. Numerical applications
are shown on financial time-series and on energy dissipation of
turbulent flows.
Download Pdf
version
We introduce a multivariate Hawkes process that accounts for the dynamics
of market prices through the impact of market order arrivals at
microstructural level. Our model is a point process mainly characterized
by 4 kernels associated with respectively the trade arrival
self-excitation, the price changes mean reversion the impact of trade
arrivals on price variations and the feedback of price changes on trading
activity. It allows one to account for both stylized facts of market
prices microstructure
(including random time arrival of price moves, discrete price grid, high
frequency mean reversion, correlation functions behavior at various time
scales) and the stylized facts of market impact (mainly the
concave-square-root-like/relaxation characteristic shape of the market
impact of a meta-order). Moreover, it allows one to estimate the entire
market impact profile from anonymous market data. We show that these
kernels can be empirically estimated from the empirical conditional mean
intensities. We provide numerical examples, application to real data and
comparisons to former approaches.
Download Pdf version
In this paper we consider the problem of the existence of some large correlation (integral) scale in random cascade models. We propose a new model that possesses multifractal properties without involving any integral scale. This model relies on a non stationary log-normal process which proper- ties, over any finite time interval, are very close to continuous cascade models. These latter models are notably well known to reproduce faithfully the main stylized fact of financial time series but the integral scale where the cascade is initiated is hard to interpret. Moreover the reported empirical values of this large scale turn out to be closely correlated to the overall length of the sample. As illustrated by the example of Dow-Jones index, this feature is precisely predicted by our model.
We prove a law of large numbers and a functional central limit theorem for multivariate Hawkes processes observed over a time in- terval [0,T] in the limit T ? ?. We further exhibit the asymptotic behaviour of the covariation of the increments of the components of a multivariate Hawkes process, when the observations are imposed by a discrete scheme with mesh \Delta over [0,T] up to some further time shift ?. The behaviour of this functional depends on the relative size of \Delta and ? with respect to T and enables to give a full account of the second-order structure. As an application, we develop our results in the context of financial statistics. We introduced in a micro- scopic stochastic model for the variations of a multivariate financial asset, based on Hawkes processes and that is confined to live on a tick grid. We derive and characterise the exact macroscopic diffusion limit of this model and show in particular its ability to reproduce important empirical stylised fact such as the Epps effect and the lead-lag effect. Moreover, our approach enable to track these effects across scales in rigorous mathematical terms.
We define a numerical method that provides a non-parametric estimation of
the kernel shape in symmetric multivariate Hawkes processes. This method
relies on second order statistical properties of Hawkes processes that
relate the covariance matrix of the process to the kernel matrix. The
square root of the correlation function is computed using a minimal phase
recovering method.
We illustrate our method on some examples and provide an empirical study
of the estimation errors. Within this framework, we analyze high frequency
financial price data modeled as 1D or 2D Hawkes processes. We find slowly
decaying (power-law) kernel shapes suggesting a long memory nature of
self-excitation phenomena at the microstructure level of price dynamics.
We present the construction of a continuous time stochastic process which has moments that satisfy an exact scaling relation, including odd order moments. It is based on a natural extension of the MRW construction. This allows us to propose a continuous time model for the price of a financial asset that reflects most major stylized facts observed on real data, including asymmetry and multifractal scaling.
We introduce a new stochastic model for the variation of asset prices at
the tick-by-tick level in dimension 1 (for a single asset) and 2 (for a
pair of assets). The construction is based on marked point processes and
relies on linear self and mutually exciting stochastic intensities as
introduced by Hawkes. We associate a counting process with the positive
and negative jumps of an asset price. By coupling suitably the stochastic
intensities of upward and downward changes of prices for several assets
simultaneously, we can reproduce microstructure noise (i.e., strong
microscopic mean reversion at the level of seconds to a few minutes) and
the Epps effect ({\it i.e.} the decorrelation of the increments in
microscopic scales) while preserving a standard Brownian diffusion
behaviour on large scales.
More effectively, we obtain analytical closed-form formulae for the mean
signature plot and the correlation of two price increments that enable to
track across scales the effect of the mean-reversion up to the diffusive
limit of the model. We show that the theoretical results are consistent
with empirical fits on futures BUND 10Y and BOBL 5Y in several situations.
By studying all the trades and best bids/asks of ultra high frequency
snapshots recorded from the order books of a basket of 10 futures assets,
we bring qualitative empirical evidence that the impact of a single trade
depends on the intertrade time lags. We find that when the trading rate
becomes faster, the return variance per trade or the impact, as measured
by the price variation in the direction of the trade, strongly increases.
We provide evidence that these properties persist at coarser time scales.
We also show that the spread value is an increasing function of the
activity. This suggests that order books are more likely empty when the
trading rate is high.
Hawkes processes are used for modeling tick-by-tick variations of a
single or of a pair of asset prices. For each asset, two counting
processes (with stochastic intensities) are associated respectively with
the positive and negative jumps of the price. We show that, by coupling
these two intensities, one can reproduce high-frequency mean reversion
structure that is characteristic of the microstructure noise. Moreover, in
the case of two assets, by coupling the stochastic intensities
corresponding to the positive (resp. negative) jumps of each asset, we are
able to reproduce the Epps effect, i.e., the decorrelation of the
increments at microscopic scales.
At large scale our model becomes diffusive and converge towards a standard
Brownian motion. Analytical closed-form formulae for the mean signature
plot, the diffusive correlation matrix and the cross-asset correlation
function at any time-scale are given. Empirical results are shown on
futures Euro-Bund and Euro-Bobl high frequency data.
In this paper, we make a short overview of multifractal models of asset returns. All the proposed models rely upon the notion of random multiplicative cascades. We focus in more details on the simplest of such models namely the log-normal Multifractal Random Walk. This model can be seen as a stochastic volatility model where the (log-) volatility has a peculiar long-range correlated memory. We briefly address calibration issues of such models and their applications to volatility and VaR forecasting.
Multifractal analysis of multiplicative random cascades is revisited
within the framework of mixed asymptotics. In this new framework,
statistics are estimated over a sample which size increases as the
resolution scale (or the sampling period) becomes finer. This allows one
to continuously interpolate between the situation where one studies a
single cascade sample at arbitrary fine scales and where at fixed scale,
the sample length (number of cascades realizations) becomes infinite. We
show that scaling exponents of ”mixed” partitions functions i.e., the
estimator of the cumulant generating function of the cascade generator
distribution, depends on some “mixed asymptotic” exponent ? respectively
above and beyond two critical value. We study the convergence properties
of partition functions in mixed asymtotics regime and establish a central
limit theorem. These results are shown to remain valid within a general
wavelet analysis framework. Their interpretation in terms of Besov
frontier are discussed. Moreover, within the mixed asymptotic framework,
we establish a “box-counting” multifractal formalism that can be seen as a
rigorous formulation of Mandelbrot’s negative dimension theory. Numerical
illustrations of our purpose on specific examples are also
provided.
Log-normal continuous random cascades form a class of multifractal processes that has already been successfully used in various fields. Several statistical issues related to this model are studied. We first make a quick but extensive review of their main properties and show that most of these properties can be analytically studied. We then develop an approximation theory of these processes in the limit of small intermittency, i.e., when the degree of multifractality is small. This allows us to prove that the probability distributions associated with these processes possess some very simple aggregation properties accross time-scales. Such a control of the process properties at different time-scales, allows us to address the problem of parameter estimation. We show that one has to distinguish two different asymptotic regimes: the first one, referred to as the ”low frequency regime”, corresponds to taking a sample whose overall size increases whereas the second one, referred to as the ”high frequency regime”, corresponds to sampling the process at an increasing sampling rate. We show that, the first regime leads to convergent estimators whereas, in the high frequency regime, the situation is much more intricate: only the intermittency coefficient can be estimated using a consistent estimator. However, we show that, in practical situations, one candetect the nature of the asymptotic regime (low frequency versus high frequency) and consequently decide whether the estimations of the other parameters are reliable or not. We finally illustrate how both our results on parameter estimation and on aggregation properties, allow one to successfully use these models for modelization and prediction of financial time series.
In this note, we present results on the behavior for the partition function of multiplicative cascades in the case where the total time of observation is large, compared to the scale of decay for the correlation of the cascade process.
In this paper we revisit an idea originally proposed by Mandelbrot about the possibility to observe “negative dimensions” in random multifractals. For that purpose, we define a new way to study scaling where the observation scale ? and the total sample length L are respectively going to zero and to infinity. This “mixed” asymptotic regime is parametrized by an exponent ? that corresponds to Mandelbrot “supersampling exponent”. In order to study the scaling exponents in the mixed regime, we use a formalism introduced in the context of the physics of disordered systems relying upon traveling wave solutions of some non-linear iteration equation. Within our approach, we show that for random multiplicative cascade models, the parameter ? can be interpreted as a negative dimension and, as anticipated by Mandelbrot, allows one to uncover the “hidden” negative part of the singularity spectrum, corresponding to “latent” singularities. We illustrate our purpose on synthetic cascade models. When applied to turbulence data, this formalism allows us to distinguish two popular phenomenological models of dissipation intermittency: We show that the mixed scaling exponents agree with a log-normal model and not with log-Poisson statistics.
In this paper, we make a short overview of continuous time multifractal
processes recently introduced to model asset return fluctuations. We show
that these models account in a very parcimonious manner for most of
``stylized facts'' of financial time series. We review in more details the
simplest of such models namely the log-normal Multifractal Random Walk. It
can simply be considered as a stochastic volatility model where the (log-)
volatility memory has a peculiar ''logarithmic'' shape. This model
possesses some appealing stability properties as respect to time
aggregation. We describe how one can estimate it using a GMM method and we
present some applications to volatility and VaR forecasting.
Download Pdf version
Removing noise from audio signals requires a non-diagonal processing of time-frequency coefficients to avoid producing ``musical noise''. State of the art algorithms perform a parameterized filtering of spectrogram coefficients with empirically fixed parameters. A block thresholding estimation procedure is introduced, which adjusts all parameters adaptively to signal property by minimizing a Stein estimation of the risk. Numerical experiments demonstrate the performance and robustness of this procedure through objective and subjective evaluations.
We investigate a new audio denoising algorithm. Complex wavelets protect
phase of signals and are thus preferred in audio signal processing to real
wavelets. The block attenuation eliminates the residual noise artifacts in
reconstructed signals and provides a good approximation of the attenuation
with oracle. A connection between the block attenuation and the
decision-directed \textit{a priori} SNR estimator of Ephraim and Malah is
studied. Finally we introduce an adaptive block technique based on the
dyadic CART algorithm. The experiments show that not only the proposed
method does eliminate the residual noise artifacts, but it also preserves
transients of signals better than short-time Fourier based methods do.
Download Pdf version
We discuss a possible scenario explaining in what respect the observed
fat tails of asset returns or volatility fluctuations can be related to
volatility long-range correlations. Our approach is based on recently
introduced multifractal models for asset returns that account for the
volatility correlations through a multiplicative random cascade. Within
the framework of these models, it can be shown that the sample size
required for a correct estimation of the behavior of extreme return
fluctuations is generally huge and outside the range of accessible size of
data. Consequently, in many cases, the extreme tail probability appears as
a power-law, with a rather small (underestimated) tail exponent. We point
out that increasing the amount of data by using smaller and smaller
(intraday) scales, does not contribute to reduce the bias and, as observed
empirically, the tail exponent turns out to be rather stable across
scales.
Download Pdf version
In this paper we discuss the problem of the estimation of extreme event
occurrence probability for data drawn from some multifractal process. We
also study the heavy (power-law) tail behavior of probability density
function associated with such data. We show that because of strong
correlations, standard extreme value approach is not valid and classical
tail exponent estimators should be interpreted cautiously. Extreme
statistics associated with multifractal random processes turn out to be
characterized by non self-averaging properties. Our considerations rely
upon some analogy between random multiplicative cascades and the physics
of disordered systems and also on recent mathematical results about the
so-called multifractal formalism. Applied to financial time series, our
findings allow us to propose an unified framemork that accounts for the
observed multiscaling properties of return fluctuations, the volatility
clustering phenomenon and the observed ``inverse cubic law'' of the return
pdf tails.
Download Pdf version
We define a large class of multifractal random measures and processes
with arbitrary log-infinitely divisible exact or asymptotic scaling law.
These processes generalize within a unified framework both the recently
defined log-normal Multifractal Random Walk processes (MRW) (Bacry etal.)
and the log-Poisson ``product of cynlindrical pulses" (Barral and
Mandelbrot). Their construction involves some ``continuous stochastic
multiplication'' (Schmitt and Marsan) from coarse to fine scales. They are
obtained as limit processes when the finest scale goes to zero. We prove
the existence of these limits and we study their main statistical
properties including non degeneracy, convergence of the moments and
multifractal scaling.
Download PDF version
We introduce a dictionary of elementary waveforms, called harmonic
atoms that extends the Gabor dictionary and fits well the natural
harmonic structures of audio signals. By modifying the ``standard''
matching pursuit, we define a new pursuit along with a fast algorithm,
namely the Fast Harmonic Matching Pursuit, to approximate N-dimensional
audio signals with a linear combination of M harmonic atoms. Our
algorithm has a computational complexity of O(MKN), where K
is the number of partials in a given harmonic atom. The decomposition
method is demonstrated on musical recordings, and we describe a simple
note detection algorithm that shows how one could use a harmonic matching
pursuit to detect notes even in difficult situations, e.g., very different
note durations, lots of reverberation, and overlapping notes.
Download PDF version
We define a large class of continuous time multifractal random measures
and processes with arbitrary log-infinitely divisible exact or asymptotic
scaling law. These processes generalize within a unified framework both
the recently defined log-normal Multifractal Random Walk (MRW) (Bacry
et.al) and the log-Poisson "product of cynlindrical pulses" (Barral and
Mandelbrot). Our construction is based on some "continuous stochastic
multiplication" from coarse to fine scales that can be seen as a
continuous interpolation of discrete multiplicative cascades. We prove the
stochastic convergence of the defined processes and study their main
statistical properties. The question of genericity (universality) of limit
multifractal processes is addressed within this new framework. We finally
provide some methods for numerical simulations and discuss some specific
examples.
Download PDF version