A quantile estimation for massive data with generalized Pareto distribution

Authors:
Jongwoo Song;Seongjoo Song
Affiliations:
Department of Statistics, Ewha Womans University, 11-1 Daehyun-dong, Seodaemun-gu, Seoul 120-750, Republic of Korea;Department of Statistics, Korea University, 5-1 Anam-dong, Seongbuk-Gu, Seoul, 136-701, Republic of Korea
Venue:
Computational Statistics & Data Analysis
Year:
2012

Citing 7
Cited 0

Parameter and quantile estimation for the generalized pareto distribution

Technometrics
Modelling extremal events: for insurance and finance

Modelling extremal events: for insurance and finance
Approximate medians and other quantiles in one pass and with limited memory

SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Incremental quantile estimation for massive tracking

Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining
Progress in Selection

SWAT '96 Proceedings of the 5th Scandinavian Workshop on Algorithm Theory
Single-pass low-storage arbitrary quantile estimation for massive datasets

Statistics and Computing
Time bounds for selection

Journal of Computer and System Sciences

Quantified Score

Hi-index	0.03

Visualization

Abstract

This paper proposes a new method of estimating extreme quantiles of heavy-tailed distributions for massive data. The method utilizes the Peak Over Threshold (POT) method with generalized Pareto distribution (GPD) that is commonly used to estimate extreme quantiles and the parameter estimation of GPD using the empirical distribution function (EDF) and nonlinear least squares (NLS). We first estimate the parameters of GPD using EDF and NLS and then, estimate multiple high quantiles for massive data based on observations over a certain threshold value using the conventional POT. The simulation results demonstrate that our parameter estimation method has a smaller Mean square error (MSE) than other common methods when the shape parameter of GPD is at least 0. The estimated quantiles also show the best performance in terms of root MSE (RMSE) and absolute relative bias (ARB) for heavy-tailed distributions.