Yuriy Mishchenko: Publication List

The paper explores the os neural generated synthetic audio for training keyword spotters where availability of training data for a keyword is challenge.

This paper presents a formulation of general quantum field theoretical flavor mixing and oscillations using Bogolyubov quantum canonical transformation as the starting point. A brief overview of quantum canonical transformations is also provided.

In this paper we describe meta-data aware model training for more robust wake word detection. Wake word detection is important element of modern voice-enabled intelligent assistants like Alexa or Hey Google required for accurate start of streaming of customer audio to the Cloud for processing while minimizing privacy incursion. In this Because wake word models operate continuously on devices and are exposed to wide variety of audio inputs from different acoustic and background conditions, they commonly affected by OP-condition mismatch problem, where the DNN model's performance may vary widely for different acoustic conditions, leading to requirements of setting different OPs for different such conditions and overall reducing accuracy. In this case, we focus on Alexa detection with and without on-device playback. We show that by incorporating the knowledge of the presence of playback on-device into model training in the form of conditional batch normalization, based on the playback-state metadata reported by devices, it is possible to improve accuracy of wake word spotting model.

In this paper we present fully neural solution for on-device wake word spotting using a single CNN with long "word-level" input context, including the accurate detection of the start and end points of the wakeword in such "word-level" input. Accurate wake word start and end detection is important requirement to on-device wake word spotters for intelligent assistants like Alexa or Hey Google required for accurate start of streaming of customer audio to the Cloud for processing, maximizing streaming requests' accuracy while minimizing privacy incursion. In this paper we develop a novel fully neural solution for such wake word start and end point detection in single-stage CNN "word-level" wake word detectors that provides accuracy on par with gold-standard Acoustic Model+HMM ASR forced alignment while being significantly simpler and cheaper to train and deploy on devices.

In this paper we present data-efficient solutions to address the challenges in WW modeling, such as domain-mismatch, noisy conditions, limited annotation, etc. The proposed system is composed of a multi-condition training pipeline with a stratified data augmentation, which improves the model robustness to a variety of predefined acoustic conditions, together with a semi-supervised learning pipeline to accurately extract the WW and confusable examples from untranscribed speech corpus. Starting from only 10 hours of domain-mismatched WW audio, we are able to enlarge and enrich the training dataset by 20-100 times to capture the acoustic complexity. Our experiments on real user data show that the proposed so- lutions can achieve comparable performance of a production-grade model by saving 97% of the amount of WW-specific data collection and 86% of the bandwidth for annotation.

In this paper, we investigate novel quantization approaches to reduce memory and computational footprint of deep neural network (DNN) based keyword spotters (KWS). We propose a new method for KWS offline and online quantization, which we call dynamic quantization, where we quantize DNN weight matrices column-wise, using each column's exact individual min-max range, and the DNN layers' inputs and outputs are quantized for every input audio frame individually, using the exact min-max range of each input and output vector. We further apply a new quantization-aware training approach that allows us to incorporate quantization errors into KWS model during training. Together, these approaches allow us to significantly improve the performance of KWS in 4-bit and 8-bit quantized precision, achieving the end-to-end accuracy close to that of full precision models while reducing the models' on-device memory footprint by up to 80%.' Full text is here.

Recent advances in technology bring about novel operating environments where the role of human participants is reduced to passive observation. While opening new frontiers in productivity and lifestyle, such environments also create hazards related to the inability of human individuals to maintain focus and concentration during passive control tasks. A passive brain-computer interface for monitoring mental attention states of human individuals (focused, unfocused, and drowsy) by using electroencephalographic (EEG) brain activity imaging and machine learning data analysis methods is developed in this work. An EEG data processing pipeline and a machine learning mental state detection algorithm using the Support Vector Machine (SVM) method were designed and compared with k-Nearest Neighbor and Adaptive Neuro-Fuzzy System methods. To collect 25 h of EEG data from 5 participants, a classic EEG headset was modified. We found that the changes in EEG activity in frontal and parietal lobes occurring...

Recent rapid development in neural activity imaging and analysis in neuroscience had fueled a revolution in our understanding of information processing and representation in the brain. Not only these advances resulted in new fundamental insights into brain?s organization, but they also paved the way for new treatments of earlier unmanageable neurological conditions. The field of Brain-Machine or Brain-Computer Interfaces (BCI) is a relatively new field with the fast advances beginning only in the past 10-15 years. Today, BCI research promises radically novel communication systems and prosthetic devices having potential to significantly improve the quality of life of thousands of people with disabilities or severe injuries. BCI in Turkey in the field of theoretical and practical dimensions has made very little work. In this study, especially basic studies about electroencephalography brain-computer interfaces (EEG BCI) and history are given. In addition, different data processing approaches... Full text is here.

The paper describes development of an electroencephalography (EEG) based noninvasive BCI system that can be used for a robotic manipulator control and detecting user's mental intent based on motor-imagery BCI paradigm. Experiments involving 12 healthy participants and analyzed offline as well as online are described. The EEG BCI system is shown to correctly identify different motor imageries in EEG data with good accuracy. 3 participants are shown to achieve BCI control accuracy in offline experiments of 80-90%. 2 participants did not achieve satisfactory accuracy. Online BCI system is then implemented for control of a virtual 3 degree-of-freedom prosthetic arm and tested with 3 best participants. These participants were able to successfully complete online BCI control tasks demonstrating on average the error rate of 80% and requiring 5-10 seconds to complete manipulator move.' Full text is here.

Operators who use a vehicle have less control load with fast improvements of robotic and autonom systems so that situation causes losing of attention an operator while important control processes. In this paper, a passive brain computer interface for monitoring mental attention state of human individuals by using electroencephalographic (EEG) brain activity imaging is developed using a machine learning data analysis method Support Vector Machine. Also a mental state detection system using EEG data is evolved as well. It has been determined that changes in EEG activity in the frontal and parietal lobes occurring in the 1-5 Hz and 1015 Hz frequency bands are associated with changes in attention state. Such changes were detected with 90% to 95% accuracy in experimental settings. The results of the work done will guide the design of future systems to monitor the status of the operators via EEG brain activity data.

In this work, we release a large set of EEG BCI data collected during the development of a slow cortical potentials-based EEG BCI. The dataset contains 60?h of EEG recordings, 13 participants, 75 recording sessions, 201 individual EEG BCI interaction session-segments, and over 60?000 examples of motor imageries in 4 interaction paradigms. The current dataset presents one of the largest EEG BCI datasets publically available to date. Full text is here.

In this paper, an accessible myoelectric prosthetic hand design is described based on a modification of prior available mechanical prosthesis and all off-the-shelf parts and components. Despite significant advances in myoelectric prosthetics, existing such devices are firmly out of reach for a majority of the patients needing them, due to high cost and complexity of existing designs. This paper offers a simple design for a myoelectric prosthetic built on base of an existing mechanical prosthesis, which can be assembled with minimal expertise and all readily available parts, at approximately 1% the price of the cheapest commercially available such prosthetic. Full text is here.

The paper provides a numerical study of various possible conditions of colliding galaxy clusters and searches for signatures that could be left in such collisions by weak self-interactions of dark matter. Certain such signatures are found and related to some recent observations in galaxy cluster collisions' astrophysics.' Full text is here.

This paper is a tutorial-like resource in Turkish for constructing EEG BCI systems' hardware and performing signal processing in them for detection of BCI mental imageries.' Full text is here.

Various structures in the brain contain many important clues to the brain's development and function. Among these, the organization of neuropil tissue at micron scales is of particular importance since such organization has a direct potential to affect the formation of synaptic connectivity between nearby axons and dendrites, thus, serving as an important factor contributing to the brain's development and disorders. While the organization of the brain at large and intermediate scales had been well studied, the microscopic organization of neuropil tissue remains largely unknown. In particular, presently it is not known what specific structures exist in neuropil at micron scales, what effect such structures have on synaptic connectivity, and what processes shape the neuropil's organization at micron scales. The present work performs an analysis of recent complete electron microscopy reconstructions of blocks of hippocampal CA1 neuropil tissue in rat to produce answers to these questions.'... Full text is here.

This Neuroimaging e-book chapter chapter provides a technical overview of some of the important recent advances in computational connectivity inference from very large-scale population calcium imaging data, emphasizing specifically the opportunities that became open in population calcium imaging thanks to such advances. The chapter discusses the current status of the problem of computational deconvolution of calcium fluorescence signals, inference of neural connectivity from population calcium imaging data, and the recent advances related to the so-called sparse or "shotgun" neural activity imaging. The chapter presents a theoretical framework based on the notion of effective connectivity that can be used to combine recent computational and experimental advances in large-scale population calcium imaging and provide the foundation for rapid advancements in quantitative understanding of the neural circuits in the brain. Chapter's link is http://www.smgebooks.com/neuroimaging/chapters/NI-16-08'... Full text is here.

The paper reports on the development of EEG slow cortical potential-based brain computer interface and the software tools created in the course of that development. Full text is here.

This paper analyses recently proposed "shotgun" neuronal activity imaging solution for impossibility of completely imaging large neuronal circuits in the brain with required accuracy. The paper offers a number of theoretical results establishing correctness of complete neuronal connectivity matrices reconstructions with this type of "sparse" neuronal activity monitoring, as well as establishing necessary conditions that need to be met by imaging protocol designs to insure such correctness. A numerical Expectation Maximization Sequantial Monte-Carlo algorithm is also provided for solving resulting estimation problem in general neuronal activity models. Full text is here.

The paper presents the development of a hidden Markov Model based approach for the analysis of EEG-signals for applications in EEG-based brain computer interfaces. Full text is here.

The paper looks into recent issue of "stochastic variability" in make-up of living cells that are thought to have to be identical (that is same genotype, same environment, same phenotype). It is shown that rather straightforward effect of such cells homeostatically maintaining their internal state would lead very generally to similar phenomenon of "stochastic variability" of the internal make-up of identical organisms. Full text is here.

This paper describes a fast line scan algorithm for calculations of discrete Euclidean distance transform in dimensions three or higher, also available from Matlab Central as function "bwdistsc". Implemented in Matlab as a plain script, described algorithm outperforms standard compiled Matlab's Image Processing Toolbox's function "bwdist" in terms of the calculation time by a factor of two. Described implementation also allows processing data of much larger size and calculations of the Euclidean distance transform with the data having anisotropic pixel dimensions (aspect ratio). Full text is here.

An EEG-BCI system for detecting the attention states of process or machinery operators in primarily passive control task. Suggests and SVM+EEG-BCI system for identification of focused, unfocused and sleepy states of passive control task operators and demonstrates validity on certain virtual reality application. Full text is here.

The paper discusses a novel connectomics approaches based on expressing two random genetic codes in pre- and post-synaptic neuronal cells, and reading out the combinations of the code pairs at actual synapses using a novel time-based fluorescent readout technique.

Economic fluctuations or business cycles are some of the most noted features of market economies also ranked among the most serious of economic problems. Despite a large number of economic cycle theories developed over the past two hundred years, the causes of the cycles remain a mystery and every new recession that hits global economy (such as the Great Recession of 2007) is attributed to a new set of reasons, notably, tied commonly to irrationalities in human economic behavior such as crowd effects, speculations, erroneous policy or strategy decisions, incomplete market information, etc. This paper shows that boom-bust cycles, in fact, are an integral part of open market economies caused by rational competitive dynamics in open markets. The paper does so by using an example of extremely simple and fundamental model of single commodity market with several rational producers. It is shown that already this most fundamental market setting possesses a property otherwise known as "Tragedy of"... Full text is here.

The paper is the first work to perform a large scale imaging of thalamocortical synapsesusing array-tomography imaging technique and detection of synapses from such optical data using co-localization of fluorescent dies associated to pre- and post-synaptic structures (see further Mishchenko Y. (2010) On Optical Detection of Densely Labeled Synapses in Neuropil and Mapping Connectivity with Combinatorially Multiplexed Fluorescent Synaptic Markers. PLoS ONE 5(1): e8853). The paper verifies and quantifies the accuracy of co-localization optical detection of synapses directly against correlative EM data, and then uses imaging of a large segment of volume of layer 5 cortical tissue to reconstruct a large number of thalamocortical synapses and to relate such synaptic distributions to the properties of thalamocortical input onto layer 5 pyramidal neurons. Full text is here.

The paper discusses prospects and economic aspects of application of two types of connectomics approaches - large scale dense electron microscopy imaging and reconstruction of neural tissue and Synaptic Brainbow/BOINC based approaches. Full text is here.

In recent years, the problem of reconstructing the connectivity in large neural circuits ("connectomics") has re-emerged as one of the main objectives of neuroscience. Classically, reconstructions of neural connectivity have been approached anatomically, using electron or light microscopy and histological tracing methods. This paper describes a statistical approach for connectivity reconstruction that relies on relatively easy-to-obtain measurements using fluorescent probes such as synaptic markers, cytoplasmic dyes, transsynaptic tracers, or activity-dependent dyes. We describe the possible design of these experiments and develop a Bayesian framework for extracting synaptic neural connectivity from such data. We show that the statistical reconstruction problem can be formulated naturally as a tractable L_1-regularized quadratic optimization. As a concrete example, we consider a realistic hypothetical connectivity reconstruction experiment in C. elegans, a popular neuroscience... Full text is here.

This paper uses newly developed techniques for semi-automated reconstruction from serial electron microscopy (EM) to obtain the shapes of neurons, the location of synapses, and the resultant synaptic connectivity in a module of the Drosophila melanogaster brain known as lamina cartridge. Relying on these reconstructions, it proposes that wiring length minimization and volume exclusion together can explain the structure of the lamina microcircuit. Full text is here.

Physical organization of the nervous system is a topic of great interest in neuroscience. Although significant amount of knowledge about brain structures had been accumulated in the past, many details of nervous system organization and its role in animals? behavior remain obscure, while the problem of complete connectivity reconstructions has recently re-emerged as one of the major directions in modern neuroscience (i.e. connectomics). In this paper I talk about a novel paradigm for reconstructions of neural connectivity that can yield connectivity maps with high resolution, high speed of imaging, and robust and transparent data analysis. In essence, it is proposed that physical connectivity in a neural circuit can be sampled using anatomical fluorescent synaptic markers localized to different parts of the neural circuit with a technique for randomized genetic targeting such as Cre/Lox. High-resolution connectivity maps then can be extracted from particular datasets thus obtained. I describe... Full text is here.

Monte Carlo approaches have recently been proposed to quantify and estimate connectivity in neuronal networks. The key problem is to sample from the conditional distribution of a single neuronal spike train, given the activity of the other neurons in the network. Dependencies between neurons are usually relatively weak; however, temporal dependencies within the spike train of a single neuron are typically strong. In this paper we develop several Metropolis-Hastings samplers which are specialized to take advantage of this dependency structure. These samplers are based on two ideas: 1) an adaptation of fast forward-backward algorithms from the theory of hidden Markov models to take advantage of the local dependencies inherent in spike trains, and 2) a first-order expansion of the conditional likelihood which allows for efficient exact sampling in the limit of weak coupling between neurons. We also demonstrate that these samplers can effectively incorporate side information, in particular... Full text is here.

In this paper we develop a statistical method for inferring neural connectivity matrix from neural population spontaneous activity recording with fluorescent calcium imaging, and study how well such performance may be done under various experimental conditions. We find that such inference may be performed remarkably well, allowing quite accurate estimation of the functional connectivity matrix under plausible neuro-biological and imaging conditions from observation containing ~1500-3000 spikes per neuron (~5-10 min observation of neural population spontaneously spiking ~5Hz). One interesting, unexpected conclusion is that this observation time requirement does not grow with the size of neural population. Ie, even for very large neural circuits, ~100-1000 neurons, connectivity matrix can be inferred by monitoring their activity for the same duration. That's definitely a very valuable property of this method. The package containing Matlab code used in this work can be downloaded from'... Full text is here.

Complete reconstructions of vertebrate neuronal circuits on the synaptic level require new approaches. Here, serial section transmission electron microscopy was automated to densely reconstruct four volumes, totaling 670 ?m^3, from the rat hippocampus as proving grounds to determine when axo-dendritic proximities predict synapses. First, in contrast with Peters' rule, the density of axons within reach of dendritic spines did not predict synaptic density along dendrites because the fraction of axons making synapses was variable. Second, an axo-dendritic touch did not predict a synapse; nevertheless, the density of synapses along a hippocampal dendrite appeared to be a universal fraction, 0.2, of the density of touches. Finally, the largest touch between an axonal bouton and spine indicated the site of actual synapses with about 80% precision but would miss about half of all synapses. Thus, it will be difficult to predict synaptic connectivity using data sets missing ultrastructural details'... Full text is here.

In this paper I use complete reconstruction of a volume of neuropil from series of electron micrographs, at a few nm resolution, to show that light microscopy techniques such as isotropically diffraction limited light microscopy (ie array tomography or LM on ultra-thin slices) and linear structured illumination microscopy are sufficient to observe individual synapses in dense neuropil with accuracy over 99%. I use this fact to suggest an approach to mapping connectivity in a neural circuit by labeling synapses of different groups of cells using distinct color fluorescent synaptic markers, and then scanning neuropil optically searching for synaptic puncta of given color - thus identifying the location and number and size of all synapses between given populations of neurons. Full text is here.

My PhD dissertation published as a book. The book.

This paper publishes methods for segmentation and tracing of neural processes in the images acquired with serial section transmission electron microscopy of neural tissue. The above two tasks can be seen as part of neural circuits reconstruction problem, using the tools of electron microscopy, which involves identification of synapses, segmentation of axonal and dendritic processes, and tracing them toward cell body in order to establish presence of a connection between two cell in a neural circuit. Such approach is considered to be the main option for large scale, complete, and detailed anatomical reconstructions of neural circuits; although some significant problems with this paradigm had been found and discussed in this paper. [Open access: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2948845/] Full text is here.

In this paper I study the possibility of reconstructing anatomical connectivity matrix of a neural circuit in the brain, from an ensemble of measurements performed with fluorescent synaptic markers genetically introduced into transgenic model to express in random manner in different neurons. This setup can be treated as random sampling of the connections from the connectivity matrix, which therefore allows to infer the connectivity matrix from a sample of such measurements using the analysis methods of compressive sensing. Analysis of potential performance of such approach in one popular neuroscience model, C. elegans, leads to very encouraging results, implying that complete connectivity in that animal can be routinely reconstructed over a span of a few days - a fit yet unheard of in neuroscience. Full text is here.

This works introduces an original "solution" to so called fermion problem of diffusion Monte Carlo. Diffusion Quantum Monte Carlo is a method for solving Schrodinger equation using what is now known particle filtering, ie by interpreting solution wave-function as density of diffusing particles, and then simulating diffusion corresponding to particular potential in Schrodinger equation. Fermion sign problem is a very old issue with this paradigm, applied to calculation of the ground states of fermion, since the true ground state of Schrodinger equation is always bosonic, and fermionic ground state is exp-weak fluctuation on top of it. Solution proposed in this paper was to supplement Schrodinger equation with non-local anti-symmetrization operator, which makes fermion state the true ground state of the problem. This solution was moderately successful, whereas non-local nature of the anti-symmetrization operator led to complications extending this idea to higher dimensional problems. Ultimately... Full text is here.

This work is a pQCD study in Light Front of the pion generalized parton distribution (GPD) computed in an effective theory with exponential wave-function representation of quarks, which was shown earlier by C.-R. Ji to reproduce pion form-factors rather well. The leading perturbative contribution to pion's GPD correspond to 2-body state of that theory only. This particular work calculates the next-to-leading correction containing 3-body state (ie gluon exchange corrections).' Full text is here.

In this work we study translation of high-frequency flavor oscillation effects, found in quantum field theory for flavor mixing and primarily related to virtual anti-particle creation associated with the flavor oscillations, into space-domain. In particular, we predict a small antiparticle anti-beam, traveling in the direction opposing to flavor particles, that should exist in particles oscillating further away from relativistic regime, such as Kaons or eta. Full text is here.

This work studies the phenomenon that certain diagrams in perturbative quantum field theory (QFT), nonvanishing when written covariantly, vanish in Light Front reference frame. The problem is linked, in this paper, to two types of limits, not performed carefully typically when going to Light Front reference frame. Outer arc integration limit is one of them, whereas when reducing covariant integrands by integrating out one of the momentum variables using Cauchy residues, the infinite arc closing the contour is often dismissed, even in certain cases it gives contribution to the integral, as we show. The second type of limits is the case where particular cingular points on the integration axis of the remaining integrand give a finite-constant contribution. Properly taking care of these limits restore the equivalence between LF and covariant frames without need for any ad-hoc regularization prescriptions. Full text is here.

A book chapter on our general formalism for quantum-field-theoretic mixing.

We study here a novel approach to finding ground state of a quantum field theory. Ground state is the quantum state of a field theory that minimizes its energy, E[H]. Since quantum state is, in general, a functional, such optimization, in general, is very difficult. In some cases, E[H] can be decomposed into a sum of fairly simple expectations. An obvious example - E[H]=E[K]+E[V], where K is kinetic energy term, and V is the potential. The essence of this work is to solve variational problem in terms of this expectation values after finding a set of constraints they should satisfy. I.e., instead of solving variational functional problem, solve min A + B, where A and B are simply numbers, A=E[K], B=E[V], satisfying constraints C(A,B)=0 established by analyzing (E[K],E[V]) for all possible quantum states. Here, this approach is applied to studying nonlinear sigma model. Full text is here.

This work re-iterates our earlier finding that the log-mass density of the visible (shining) and the dark (non-shining) matter in spiral galaxies and at least in one galaxy cluster respects amazing linear correlation, i.e. log rho_v = k log rho_d. What's most interesting is that the proportionality constant is the same both in the case of spiral galaxies and the galaxy cluster, k=4. Our earlier interpretation of this fact, re-iterated here, is in terms of thermodynamic Boltzman law, which leads to two conclusions: a) dark and visible components are in near-thermal equilibrium, which means that dark matter in fact is quite strongly interacting, and b) mass of dark matter particles is about 1/4 that of the proton mass, or 250MeV.' Full text is here.

In this paper we study the applications of our formalism for QFT of flavor mixing, predicting in particular high-frequency oscillations component in flavor mixing primarily due to virtual anti-particles creation during oscillations, to phenomenologically interesting cases - neutrinos, kaons and eta. Full text is here.

A book chapter with some more detailed analysis of the distributions of visible and dark mass in the galaxy cluster CL0024. Full text is here.

We present an amazing observation that the log-mass density of the visible (shining) and the dark (non-shining) matter in spiral galaxies and at least in one galaxy cluster respects linear correlation, i.e. log rho_v = k log rho_d, with the same proportionality constant k=4. We interpret this fact in terms of thermodynamic Boltzman law, which leads to two conclusions: a) dark and visible components are in near-thermal equilibrium, which means that dark matter in fact is quite strongly interacting, and b) mass of dark matter particles is about 1/4 that of the proton mass, or 250MeV. Full text is here.

In this paper we develop general quantum-field-theoretic formalism for studying quantum flavor mixing of particles, for arbitrary number of flavors and arbitrary spin. Special cases of spin 0 (mesons), spin 1/2 (neutrinos) and spin 1 (vector-mesons) are explicitly considered. Full text is here.

In this paper we derive high-frequency oscillations contribution in flavor oscillations of mesons. Full text is here.