Skip to content
Snippets Groups Projects
Commit bbf4fdc2 authored by Neil Conway's avatar Neil Conway
Browse files

Prevent corr() from returning the wrong results for negative correlation

values. The previous coding essentially assumed that x = sqrt(x*x), which
does not hold for x < 0.

Thanks to Jie Zhang at Greenplum and Gavin Sherry for reporting this
issue.
parent 4893eadc
No related branches found
No related tags found
No related merge requests found
...@@ -8,7 +8,7 @@ ...@@ -8,7 +8,7 @@
* *
* *
* IDENTIFICATION * IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/utils/adt/float.c,v 1.150 2007/06/05 21:31:06 tgl Exp $ * $PostgreSQL: pgsql/src/backend/utils/adt/float.c,v 1.151 2007/09/19 22:31:48 neilc Exp $
* *
*------------------------------------------------------------------------- *-------------------------------------------------------------------------
*/ */
...@@ -2274,8 +2274,7 @@ float8_corr(PG_FUNCTION_ARGS) ...@@ -2274,8 +2274,7 @@ float8_corr(PG_FUNCTION_ARGS)
if (numeratorX <= 0 || numeratorY <= 0) if (numeratorX <= 0 || numeratorY <= 0)
PG_RETURN_NULL(); PG_RETURN_NULL();
PG_RETURN_FLOAT8(sqrt((numeratorXY * numeratorXY) / PG_RETURN_FLOAT8(numeratorXY / sqrt(numeratorX * numeratorY));
(numeratorX * numeratorY)));
} }
Datum Datum
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment