When using verbose mode for pg_basebackup, in tar format sent to
stdout, we'd print an unitialized buffer as the filename.

Reported by Pontus Lundkvist

In pg_basebackup.c:reached_end_position(), we're reading from an
internal pipe with our own background process but we're possibly
reading more bytes than will actually fit into our buffer due to
an off-by-one error.  As we're reading from an internal pipe
there's no real risk here, but it's good form to not depend on
such convenient arrangements.

Bug spotted by the Coverity scanner.

Back-patch to 9.2 where this showed up.

These programs don't work against 9.0 or earlier servers, so check that when
the connection is made. That's better than a cryptic error message you got
before.

Also, these programs won't work with a 9.3 server, because the WAL streaming
protocol was changed in a non-backwards-compatible way. As a general rule,
we don't make any guarantee that an old client will work with a new server,
so check that. However, allow a 9.1 client to connect to a 9.2 server, to
avoid breaking environments that currently work; a 9.1 client happens to
work with a 9.2 server, even though we didn't make any great effort to
ensure that.

This patch is for the 9.1 and 9.2 branches, I'll commit a similar patch to
master later. Although this isn't a critical bug fix, it seems safe enough
to back-patch. The error message you got when connecting to a 9.3devel
server without this patch was cryptic enough to warrant backpatching.

It still thought that the --xlog-method option argument could be
empty, as in a previous version of this feature.

The most user-visible part of this is to change the long options
--statusint and --noloop to --status-interval and --no-loop,
respectively, per discussion.

Also, consistently enclose file names in double quotes, per our
conventions; and consistently use the term "transaction log file" to
talk about WAL segments.  (Someday we may need to go over this
terminology and make it consistent across the whole source code.)

Finally, reflow the code to better fit in 80 columns, and have pgindent
fix it up some more.

Before, some places didn't document the short options (-? and -V),
some documented both, some documented nothing, and they were listed in
various orders.  Now this is hopefully more consistent and complete.

To replace it, add -X/--xlog-method that allows the specification
of fetch or stream.

Do this to avoid unnecessary backwards-incompatiblity. Spotted and
suggested by Peter Eisentraut.

commit-fest.

Since the replication protocol deals with TimestampTz, we need to
care for the floating point case as well in the frontend tools.

Fujii Masao, with changes from Magnus Hagander

Avoids the need for an external script in the most common
scenario. Behavior can be overridden using the -n/--noloop
commandline parameter.

When backing up from a standby server, the backup process
will not automatically switch xlog segment. So we must
accept a partially transferred xlog file in this case, but
rename it into position anyway.

In passing, merge the two callbacks for segment end and
stop stream into a single callback, since their implementations
were close to identical, and rename this callback to
reflect that it stops streaming rather than continues it.

Patch by Magnus Hagander, review by Fujii Masao

see also ce8d7bb6

Thomas Ogrisegg and Fujii Masao

Fujii Masao

Per compiler warnings on buildfarm member black_firefly.

gzFile is already a pointer, so code like

gzFile *handle = gzopen(...)

is wrong.

This used to pass silently because gzFile used to be defined as void*,
and you can assign a void* to a void**.  But somewhere between zlib
versions 1.2.3.4 and 1.2.6, the definition of gzFile was changed to
struct gzFile_s *, and with that new definition this usage causes
compiler warnings.

So remove all those extra pointer decorations.

There is a related issue in pg_backup_archiver.h, where

FILE       *FH;             /* General purpose file handle */

is used throughout pg_dump as sometimes a real FILE* and sometimes a
gzFile handle, which also causes warnings now.  This is not yet fixed
here, because it might need more code restructuring.

Jaime Casanova

Remove some dead code, conditionally declare some items or call
some code, and fix one or two declarations.

lines.  Update pg_bsd_indent required version to 1.1 (and update ftp
site).

Problem reported by Magnus.

Add option for parallel streaming of the transaction log while a
base backup is running, to get the logfiles before the server has
removed them.

Also add a tool called pg_receivexlog, which streams the transaction
log into files, creating a log archive without having to wait for
segments to complete, thus decreasing the window of data loss without
having to waste space using archive_timeout. This works best in
combination with archive_command - suggested usage docs etc coming later.

The translation tools are very unhappy about seeing \r in translatable
strings, so move it to a separate fprintf call.

When streaming including WAL, the size estimate will always be incorrect,
since we don't know how much WAL is included. To make sure the output doesn't
look completely unreasonable, this patch increases the total size whenever we
go past the estimate, to make sure we never go above 100%.

Also fix a potential portability bug, because INT64_FORMAT is only
guaranteed to be available with snprintf, not fprintf.

They are identical, but the overwhelming majority of the code uses %d,
so standardize on that.

The short-form -z switch didn't work, for lack of telling getopt_long
about it; and even if specified long-form, it failed to do anything,
because the various tests elsewhere in the file would take
Z_DEFAULT_COMPRESSION (which is -1) as meaning "don't compress".

Per bug #6060 from Shigehiro Honda, though I editorialized on his patch
a bit.

pg_basebackup doesn't need to police the format of port numbers.
libpq already does that.

This code was accidentally part of the patch, it's only
needed for the code that's for 9.2. Not needing the timeline
also removes the need to call IDENTIFY_SYSTEM.

Noted by Peter E.