From aad663a0b4af785d0b245bbded27537f23932839 Mon Sep 17 00:00:00 2001 From: Tom Lane <tgl@sss.pgh.pa.us> Date: Sun, 23 Aug 2015 15:15:47 -0400 Subject: [PATCH] Reduce number of bytes examined by convert_one_string_to_scalar(). Previously, convert_one_string_to_scalar() would examine up to 20 bytes of the input string, producing a scalar conversion with theoretical precision far greater than is of any possible use considering the other limitations on the accuracy of the resulting selectivity estimate. (I think this choice might pre-date the caller-level logic that strips any common prefix of the strings; before that, there could have been value in scanning the strings far enough to use all the precision available in a double.) Aside from wasting cycles to little purpose, this choice meant that the "denom" variable could grow to as much as 256^21 = 3.74e50, which could overflow in some non-IEEE float arithmetics. While we don't really support any machines with non-IEEE arithmetic anymore, this still seems like quite an unnecessary platform dependency. Limit the scan to 12 bytes instead, thus limiting "denom" to 256^13 = 2.03e31, a value more likely to be computable everywhere. Per testing by Greg Stark, which showed overflow failures in our standard regression tests on VAX. --- src/backend/utils/adt/selfuncs.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c index 14b8c2ff545..72bc5025228 100644 --- a/src/backend/utils/adt/selfuncs.c +++ b/src/backend/utils/adt/selfuncs.c @@ -3861,10 +3861,16 @@ convert_one_string_to_scalar(char *value, int rangelo, int rangehi) return 0.0; /* empty string has scalar value 0 */ /* - * Since base is at least 10, need not consider more than about 20 chars + * There seems little point in considering more than a dozen bytes from + * the string. Since base is at least 10, that will give us nominal + * resolution of at least 12 decimal digits, which is surely far more + * precision than this estimation technique has got anyway (especially in + * non-C locales). Also, even with the maximum possible base of 256, this + * ensures denom cannot grow larger than 256^13 = 2.03e31, which will not + * overflow on any known machine. */ - if (slen > 20) - slen = 20; + if (slen > 12) + slen = 12; /* Convert initial characters to fraction */ base = rangehi - rangelo + 1; -- GitLab