O UTLINE I NTRODUCTION M ETHOD R ESULT

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION Irina Udaltsova, UC Davis

Samuel Schmidt, Jiming Jiang, UC Davis October 28, 2011

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

O UTLINE I NTRODUCTION M ETHOD R ESULT

O UTLINE

I NTRODUCTION

M ETHOD

R ESULT

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

O UTLINE I NTRODUCTION M ETHOD R ESULT

R EDSHIFT ▶

Redshift: Due to expansion of the Universe, light spectrum from distant galaxies shifts towards longer (redder) wavelengths proportional to each galaxy’s distance

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

P HOTOMETRY & C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

C OLORS

.

.

.

.

.

.

O UTLINE I NTRODUCTION M ETHOD R ESULT

F EW METHODS OF PHOTOMETRIC REDSHIFT ESTIMATION

Author Connolly et.al. (1995) Wang et.al. (2007) Ball et.al. (2006) Collister et al. (2004) Wadadekar (2004) Mobasher et.al. (1999) Benitez (2000), Bender (2001) Feldmann et al. (2006)

Method Empirical training set / Regression Empirical training set / Kernel Regression K-nearest-neighbor (KNN) Neural networks Support vector machines SED template fitting / MLE SED template fitting / Bayesian SED template fitting / MLE/ Bayesian

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

O UTLINE I NTRODUCTION M ETHOD R ESULT

C LASSIC K ERNEL REGRESSION

Observations (Xi , Yi ), i = 1, . . . , n are iid. Estimate regression function m(x) = E(Y|X = x). Kernel regression estimator ∑n K[(Xi − x)/h]Yi ˆ m(x) = ∑i=1 n i=1 K[(Xi − x)/h] where K is kernel, for example, Gaussian kernel.

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

O UTLINE I NTRODUCTION M ETHOD R ESULT

A DAPTIVE K ERNEL M ETHOD

F IGURE : (a) Kernel regression, (b) Adapted-Kernel regression ▶ ▶

Takeda et.al. (2007) Kernel regression for image processing and reconstruction Automated procedure selects importance of local pixels for estimation of neighbors. .

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

O UTLINE I NTRODUCTION M ETHOD R ESULT

A DAPTIVE K ERNEL M ETHOD

KH (Xi − x) =

1

−1

{ } 1 ′ −2 exp − (Xi − x) H (Xi − x) 2

|H| (2π)d/2 Xi = d-dim data in local analysis window, i = 1, ..., N x = center location of interest 1

H = hC− 2 = bandwidth matrix h = bandwidth C = inverse ’covariance’ matrix built from SVD of local gradient matrix

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

O UTLINE I NTRODUCTION M ETHOD R ESULT

S PECIFICATION OF G RADIENT ▶

Gradient matrix ( ) G = z′1 (xj ) z′2 (xj ) . . . z′d (xj ) N×d z′ = first order derivatives along d coordinate directions for xj points in local analysis window

▶

SVD:G = USV ′ , where  s1 0 . . . 0 0 s . . . 0 2     . ( ) .. s   d , V = v1 |v2 | . . . |vd S = 0 0   0 0 ... 0   .. .. .. .. . . . . .

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

O UTLINE I NTRODUCTION M ETHOD R ESULT

S PECIFICATION OF STEERING MATRIX H 1

H = hC− 2

C = γUθ ΛUθ′

where si d Λ = diag{σi }, σi = ∑ = function of eigenvalues of G ( ) sj Uθ = v1 |v2 | . . . |vd = eigenvectors of G for rotation angle θ γ = ˆf 1/d , where ˆf = kernel density estimate

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

D ATA 2-D C OLOR S PACE

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

A DAPTIVE K ERNEL M ETHOD - S AMPLE R ESULT

.

.

.

.

.

.

O UTLINE I NTRODUCTION M ETHOD R ESULT

R ESULTS : SIMULATED REDSHIFT DATA d 2 3 4 5

MSEK 0.0082 (0.0058) 0.0026 (0.0003) 0.0016 (0.0015) 0.0019 (0.0014)

MSEAdaptK 0.0053 (0.0016) 0.0011 (0.0007) 0.0009 (0.0006) 0.0011 (0.0008)

MSEKNN 0.0351 (0.0084) 0.0016 (0.0005) 0.0021 (0.0008) 0.003 (0.0024)

30000 galaxy observations: ugrizy magnitudes based on six

CWWSB templates & spectroscopic redshifts (zs ∼ 0 - 4). Stopping criteria: |yiter − yiter−1 | is mimimum for iter = 1, ..., 10, usually convergence reached in 3-7 iterations.

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

O UTLINE I NTRODUCTION M ETHOD R ESULT

R ESULTS : SDSS DATA

d 2 3 4

MSEK 0.0081 (0.0010) 0.0077 (0.0028) 0.0099 (0.0032)

MSEAdaptK 0.0042 (0.0010) 0.0022 (0.0010) 0.0032 (0.0012)

MSEKNN 0.0065 (0.0010) 0.0090 (0.0024) 0.0039 (0.0015)

16000 galaxy observations: ugriz magnitudes & spectroscopic

redshifts (zs ∼ 0 - 0.5).

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

O UTLINE I NTRODUCTION M ETHOD R ESULT

C OMMENTS

▶

Photometric redshift require excellent photometry and colors (S/N ∼ 30)

▶

Training set must be representative and spanning in order to avoid introducing bias

▶

Bandwidth choice is an issue - no unique optimal solution across various iterations

THANK YOU! .

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

References:

▶ Takeda et.al. (2007) Kernel regression for image processing and reconstruction. IEEE transactions on image processing 16, 349-366. ▶ Vieu (1991) Nonparametric regression: optimal local bandwidth choice. JRSSB 53, 453-464. ▶ Silverman (1986) Density estimation for statistics and data analysis. Chapman & Hall. ▶ Connolly et.al. (1995), Slicing through multicolor space: galaxy redshifts from broadband photometry. Astronomical Journal 110, 2655-2664. ▶ Wang et.al. (2007) Kernel regression for determining photometric redshifts from Sloan broad-band photometry. Monthly notices of Royal Astronomical society 382, 1601-1606. ▶ Ball, Brunner, Myers, et.al. (2006), Robust machine learning applied to astronomical datasets III. Astrophysical Journal 650, 497-506. ▶ Collister et al. (2004) ANNz: estimating photometric redshifts using artificial neural networks. Publications of the Astronomical Society of the Pacific 116, 345-351. ▶ Wadadekar (2004) Estimating photometric redshifts using support vector machines. Publications of the Astronomical Society of the Pacific 117, 79-85. ▶ Mobasher et al. (1999) A self-consistent method for estimating photometric redshifts. ASP Conference Series 191, 37. ▶ Benitez et al. (2000) Bayesian photometric redshift estimation. Astrophysical Journa 536, 571-583. ▶ Feldmann et al. (2006) The Zurich extragalactic bayesian redshift analyzer and its first application: COSMOS. Monthly Notices of the Royal Astronomical Society 372, 565-577.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

O UTLINE I NTRODUCTION M ETHOD R ESULT

I MAGE PROCESSING

▶

Kernel regression for image processing and reconstruction Takeda et.al. (2007)

▶

2-D set of pixel locations 1-D set of pixel grayscale values Goal: estimate grayscale pixel value based on local grayscale values, adapting to local features of image

▶ ▶

.

I RINA U DALTSOVA , UC D AVIS

.

.

.

.

.

P HOTOMETRIC R EDSHIFTS W ITH A DAPTIVE K ERNEL R EGRESSION

F IGURE : Examples of steering weights

.

.

.

.

.

.

Photometric Redshifts With Adaptive Kernel Regression

Oct 28, 2011 - Page 3 ... wavelengths proportional to each galaxy's distance. IRINA UDALTSOVA, UC DAVIS. PHOTOMETRIC REDSHIFTS WITH ADAPTIVE ...
Download PDF
4MB Sizes 3 Downloads 141 Views
Report

Recommend Documents

A Procedure of Adaptive Kernel Combination with ...
tion problems, it is necessary to combine information from ... example, in object classification, color information is not relevant for the car ..... train tvmonitor. Figure 1: Example images of the VOC 2008 data set. An image can contain multiple ob
ADAPTIVE KERNEL SELF-ORGANIZING MAPS ... - Semantic Scholar
4-5 Negative log likelihood of the input versus the kernel bandwidth. . . . . . . . . 34. 5-1 Kernel ...... He received his Bachelor of. Technology degree from ...
ADAPTIVE KERNEL SELF-ORGANIZING MAPS ... - Semantic Scholar
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT ..... The central idea of the information theoretic learning proposed by Principe et al. ...... Technology degree from Department of Electronics and Communication Engineering.
Statistics and Probability Letters Local adaptive smoothing in kernel ...
Let (Xi,Yi),i = 1,2,...,n, be n independent and identically distributed (i.i.d.) bivariate observations with a joint density function f (·,·). Let the marginal density of X be f (·). Let K be a classical second- order kernel function (e.g., see Eu
Adaptive penalized quantile regression for high ...
For example, there is no guarantee that K-fold cross-validation would provide a choice of ln with a proper rate. Third, their statistical properties are still uncharted ...
Regression Discontinuity Design with Measurement ...
Nov 20, 2011 - All errors are my own. †Industrial Relations Section, Princeton University, Firestone Library, Princeton, NJ 08544-2098. E-mail: zpei@princeton.
SLICE INVERSE REGRESSION WITH SCORE ...
Nov 25, 2016 - yi,k := (ai,k/xk)2, i = 1,2,...m, k = 1,2,...,q. Motivating application: dynamic solar imaging from phaseless measurements; image changes are often ...
Regression Discontinuity Design with Measurement ...
“The Devil is in the Tails: Regression Discontinuity Design with .... E[D|X∗ = x∗] and E[Y|X∗ = x∗] are recovered by an application of the Bayes' Theorem. E[D|X.
Interpreting Regression Discontinuity Designs with ...
Gonzalo Vazquez-Bare, University of Michigan. We consider ... normalizing-and-pooling strategy so commonly employed in practice may not fully exploit all the information available .... on Chay, McEwan, and Urquiola (2005), where school im-.
Photometric Stereo and Weather Estimation Using ...
We extend photometric stereo to make it work with in- ternet images, which are typically associated with differ- ent viewpoints and significant noise. For popular tourism sites, thousands of images can be obtained from internet search engines. With t
Kernel-Based Clustering with Automatic Cluster ... - iSEE@SYSU
types of clusters are ill-initialized, and (2) the actual number ..... index array of the 2-nd prototype is δ. 2 = [δ. 2. 1,δ. 2. 2,...,δ. 2 m2. ] = [1, 32, 8, 20, 15] with δ. 2.
Genetic Programming for Kernel-based Learning with ...
Swap node mutation Exchange a primitive with another of the same arity (prob. ... the EKM, the 5 solutions out of 10 runs with best training error are assessed .... the evolution archive, to estimate the probability for an example to be noisy and.
Nyström Method with Kernel K-means++ Samples as Landmarks
1990) guarantees that the optimal solution can be found in the subspace of the ... To find the optimal solution in a problem with n instances, it is ...... Pole-telecom.
Structured Clustering with Automatic Kernel Adaptation
the input data, it does not take the output clustering structure into account. ..... criminative unsupervised training algorithm (denoted XWSS) in [15]; and (3) ..... Conference on Data Mining, pages 106–117, Columbus, Ohio, USA,. April 2010.