📄 blind source separation and independent component analysis.htm
字号:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<!-- saved from url=(0048)http://www-sig.enst.fr/~cardoso/guidesepsou.html -->
<HTML><HEAD><TITLE>Blind source separation and Independent component analysis.</TITLE>
<META content="text/html; charset=gb2312" http-equiv=Content-Type>
<META content="MSHTML 5.00.2614.3500" name=GENERATOR></HEAD>
<BODY>
<HR>
<P>
<CENTER>
<H1>BLIND SOURCE SEPARATION <BR>and <BR>INDEPENDENT COMPONENT ANALYSIS </H1>
<P>If this was not permanently <B>under construction</B> <IMG
src="Blind source separation and Independent component analysis.files/hammer.gif"
valign="middle">, it would be useless... </CENTER>
<HR>
<H2>What is this page about?</H2>This page is an attempt to organize some of the
material that I make available regarding source separation and ICA.
<P>This is not very much of tutorial value and you need prior exposition to
source separation ideas to figure out what I am talking about below. An
introduction to the statistical principles at work in ICA/BSS is given in the
tutorial paper listed below.
<H2>Tutorial paper</H2>
<P>I have written a tutorial paper <I>Blind signal separation: statistical
principles</I>, for the Proceedings of IEEE which explains the statistical
principles behind source separation and ICA. Reprints are available, formatted
for <A href="ftp://sig.enst.fr/pub/jfc/Papers/ProcIEEE.a4.ps.gz">A4 paper</A> or
for <A href="ftp://sig.enst.fr/pub/jfc/Papers/ProcIEEE.us.ps.gz">US letter</A>
paper.
<H2>Source separation and equivariance</H2>There is an underlying multiplicative
structure to the source separation problem for the simple reason that the source
separation model is a <EM>transformation model</EM>: the observations are
obtained via <EM>multiplication</EM> of the source signals by the unknown mixing
matrix.
<P>It is very rewarding to explore the consequences of this simple fact. It
leads in particular to the notion of `serial updating' by following the
<EM>relative gradient</EM> by which efficient adaptive algorithms can be
derived. A summary of these ideas can be found in <A
href="ftp://sig.enst.fr/pub/jfc/Papers/nolta95.ps.gz">The invariant approach to
source separation</A> published in the proceedings of NOLTA'95 and in <A
href="ftp://sig.enst.fr/pub/jfc/Papers/iscas96_invar.ps.gz">Performance and
implementation of invariant source separation algorithms</A> published in the
proceedings of ISCAS'96.
<P>Note: the idea of `relative gradient' has been independently introduced by <A
href="http://www.bip.riken.go.jp/irl/amari/amari.html">Pr. Amari</A> who defines
a `natural gradient' (based on the Riemannian structure of the probability
model) which, after some simplification is identical to our relative gradient
(based on the group structure) in the case of ICA. <A
href="http://wol.ra.phy.cam.ac.uk/mackay/">David MacKay</A> also arrived at a
similar idea by what he calls a `covariant' approach. If you are really worried
about the differences between natural gradient and relative gradient, have a
look at this <A href="ftp://sig.enst.fr/pub/jfc/Papers/ssap98.ps">short
paper</A> from the proceedings of SSAP'98.
<P>The notion of `estimating function' is helpful to unify many approaches to
the source separation problem: maximum likelihood, infomax, contrast
optimization, cumulant matching... This recent conference paper about <A
href="ftp://sig.enst.fr/pub/jfc/Papers/icassp97_est_eq.ps">Estimating equations
for source separation</A>, published in the proceedings of ICASSP '97 summarizes
a good part of these ideas. <BR>
<P>
<P>
<H2>An efficient batch algorithm: JADE </H2>
<P>For off-line ICA, we have developed with Antoine Souloumiac an algorithm
based on the (joint) diagonalization of cumulant matrices. `Good' statistical
performance is achieved by involving all the cumulants of order 2 and 4 while a
fast optimization is obtained by the device of joint diagonalization.
<P>JADE has been successfully applied to the processing of real data sets, such
as found in mobile telephony and in airport radar as well as to bio-medical
signals (ECG, EEG, multi-electrode neural recordings). These experiments will be
reported here in the (near?) future.
<P>What we think is the strongest point of JADE for applications of ICA is that
it works off-the-shelf (no parameter tuning). Actually, we advocate using the
code provided below as a plug-in replacement for PCA (whenever one is willing to
investigate if such a replacement is appropriate). The weakest point of the
current implementation is that the number of sources (but not of sensors) is
limited in practice (by the available memory) to something like 40 or 50
depending on your computer.
<UL>
<LI><B>Papers.</B> The Jade algorithm is described or discussed in these
papers.
<P>
<UL>
<LI>The first journal paper describing JADE is <A
href="ftp://sig.enst.fr/pub/jfc/Papers/iee.ps.gz">Blind beamforming for
non-Gaussian signals.</A> This is a reprint from IEE Proc-F., a journal with
a strong focus on array processing. This paper includes some performance
analysis (inclusing the noisy case) and comparisons to more classic
approches to signal separation than ICA. Readers unfamiliar with beamforming
and the processing of complex signals may prefer to read the paper published
in Neural Computation.
<P></P>
<LI>The conference paper: <A
href="ftp://sig.enst.fr/pub/jfc/Papers/eusipco94_perf.ps.gz">On the
performance of orthogonal algorithms</A> (reprint from EUSIPCO'94) also
adresses the issue of asymptotic performance analysis. It elaborates on the
performance of algorithms which, like JADE and many others, are based on
pre-whitening followed by contrast optimization. This is a rather technical
paper.
<P></P>
<LI>The more recent paper <A
href="ftp://sig.enst.fr/pub/jfc/Papers/neuralcomp_revised_2pps.ps">High-order
contrasts for independent component analysis</A>, published in `Neural
Computation' 11(1):157--192, 1999, gives another try at explaining why JADE
and related approaches are statistically and comptutationally attractive.
The paper considers only the noise-free real case, but includes more
explanations about JADE as well as comparisons to gradient-based methods.
</LI></UL>
<P></P>
<LI><B>Code.</B> The JADE algorithm was originally developed to process
complex signals, motivated by applications to digital communications. I now
make available another implementation which is tuned to process more
efficiently <B>real-valued</B> signals.
<P>
<UL>
<LI><B>Implementation for the ICA of real-valued data.</B> <BR>Here is an
implementation of <A href="ftp://sig.enst.fr/pub/jfc/Algo/Jade/jadeR.m">JADE
for real-valued data</A> (latest release: version 1.5. Dec. 1997). You may
want to test the code on this toy <A
href="ftp://sig.enst.fr/pub/jfc/Algo/Jade/demoR.m">demo calling program</A>.
<P></P>
<LI><B>Implementation for complex-valued signals.</B> <BR>You can down-load
the <A href="ftp://sig.enst.fr/pub/jfc/Algo/Jade/jade.m">JADE algorithm</A>
for complex-valued signals as a Matlab function (new, faster, cleaner
version: nov. 97). To run a Matlab demo of it, you can use this <A
href="ftp://sig.enst.fr/pub/jfc/Algo/Jade/demo.m">calling program</A>.
<P></P></LI></UL></LI></UL>
<H2>Adaptive algorithms: relative gradient algorithms </H2>For adaptive source
separation, we have developed with Beate Laheld a class of <EM>equivariant</EM>
algorithms. This means that their performance is independent of the mixing
matrix. They are obtained as <EM>stochastic relative gradient algorithms</EM>.
<P>
<UL>
<LI><B>Papers.</B>
<UL>
<LI><A href="ftp://sig.enst.fr/pub/jfc/Papers/eusipco94_PFS.ps.gz">Adaptive
source separation with uniform performance</A>. This is a brief four-page
description published in the proceedings of EUSIPCO '94.
<LI><A href="ftp://sig.enst.fr/pub/jfc/Papers/easi.ps.gz">Equivariant
adaptive source separation</A>. This comprehensive version appeared in IEEE
Tr. on S.P. (dec. 96) but, due to an editing mistake, the published version
does not include figure 8 showing an application to real digital
communication signals. This is corrected in this on-line version. </LI></UL>
<P></P>
<LI><B>Code.</B> By the way, this algorithm works pretty well... Check it out!
Get a <A href="ftp://sig.enst.fr/pub/jfc/Algo/Easi/easi_demo.m">Matlab
demo</A> of it. Beware: this is a demo with complex-valued signals, of the
type encountered in digital communications. </LI></UL>
<P>
<P>
<P>
<H2>Multi-dimensional independent component analysis. </H2>Performing ICA on ECG
signals with the JADE algorithm, I realized that an interesting extension of the
notion of independent component analysis would be to consider an analysis into
linear components that would be `as independent as possible' as in ICA, but
would be `living' in subspaces of dimension greater than 1. This could be called
`MICA' for Multi-dimensional Independent Component Analysis. This is explained
in an ICASSP paper and is illustrated therein as well as on <A
href="http://sig.enst.fr/~cardoso/RRicassp98.html">this page.</A>
<P>
<P>
<P>
<H2>My travels on the matrix manifold </H2>
<CENTER><IMG
src="Blind source separation and Independent component analysis.files/matman.gif">
</CENTER>
<CENTER>Can you find the straight square ? Can you find it blindly ? </CENTER>
<P>
<HR>
Return to the <A href="http://www-sig.enst.fr/~cardoso/stuff.html">top</A> (I
mean: top page).
<HR>
Note concerning IEEE publications: This material is presented to ensure timely
dissemination of scholarly and technical work. Copyright and all rights therein
are retained by authors or by other copyright holders. All persons copying this
information are expected to adhere to the terms and constraints invoked by each
author's copyright. In most cases, these works may not be reposted without the
explicit permission of the copyright holder. (see <A
href="http://www.ieee.org/about/documentation/copyright/policies.htm">IEEE
copyright policies</A>) </BODY></HTML>
⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -