Before 9.1, PL/Python functions returning composite types could return
a string and it would be parsed using record_in.  The 9.1 changes made
PL/Python only expect dictionaries, tuples, or objects supporting
getattr as output of composite functions, resulting in a regression
and a confusing error message, as the strings were interpreted as
sequences and the code for transforming lists to database tuples was
used.  Fix this by treating strings separately as before, before
checking for the other types.

The reason why it's important to support string to database tuple
conversion is that trigger functions on tables with composite columns
get the composite row passed in as a string (from record_out).
Without supporting converting this back using record_in, this makes it
impossible to implement pass-through behavior for these columns, as
PL/Python no longer accepts strings for composite values.

A better solution would be to fix the code that transforms composite
inputs into Python objects to produce dictionaries that would then be
correctly interpreted by the Python->PostgreSQL counterpart code.  But
that would be too invasive to backpatch to 9.1, and it is too late in
the 9.2 cycle to attempt it.  It should be revisited in the future,
though.

Reported as bug #6559 by Kirill Simonov.

Jan Urbański

Josh Kupershmidt

The header file is needed by any module that wants to use the PL/pgSQL
instrumentation plugin interface. Most notably, the pldebugger plugin needs
this. With this patch, it can be built using pgxs, without having the full
server source tree available.

Clarify that nrows() is the number of rows processed, versus the
number of rows returned, which can be obtained using len.  Also add
tests about that.

The result object methods colnames() etc. would crash when called
after a command that did not produce a result set.  Now they throw an
exception.

discovery and initial patch by Jean-Baptiste Quenot

The parser got confused if a cursor parameter had the same name as
a plpgsql variable.  Reported and diagnosed by Yeb Havinga, though
this isn't exactly his proposed fix.

Also, some mostly-but-not-entirely-cosmetic adjustments to the original
named-cursor-parameter patch, for code readability and better error
diagnostics.

An incorrect and entirely unnecessary "safety check" in exec_stmt_getdiag()
caused the code to treat an assignment to a variable with dno zero as a
no-op.  Unfortunately, that's a perfectly valid dno.  This has been broken
since GET DIAGNOSTICS was invented.  It's not terribly surprising that the
bug went unnoticed for so long, since in most cases you probably wouldn't
use the function's first-created variable (normally its first parameter)
as a GET DIAGNOSTICS target.  Nonetheless, it's broken.  Per bug #6551
from Adam Buraczewski.

Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements.  The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.

In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs.  Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.

Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.

Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn".  There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.

Andres Freund and Tom Lane

Failing to do so causes trigger invocation to fail when they are nested
within a function invocation that changes the current package.

Backpatch to 9.1; previous releases used a different method to obtain
_TD.  Per bug report from Mark Murawski (bug #6511)

Author: Alex Hunsaker

Dave Malcolm of Red Hat is working on a static code analysis tool for
Python-related C code.  It reported a number of problems in plpython,
most of which were failures to check for NULL results from object-creation
functions, so would only be an issue in very-low-memory situations.

Patch in HEAD and 9.1.  We could go further back but it's not clear that
these issues are important enough to justify the work.

Jan Urbański

We forgot to free the per-attribute array element descriptors.

Jan Urbański

This replaces the former global variable PLy_curr_procedure, and provides
a place to stash per-call-level information.  In particular we create a
per-call-level scratch memory context.

For the moment, the scratch context is just used to avoid leaking memory
from datatype output function calls in PLyDict_FromTuple.  There probably
will be more use-cases in future.

Although this is a fix for a pre-existing memory leakage bug, it seems
sufficiently invasive to not want to back-patch; it feels better as part
of the major rearrangement of plpython code that we've already done as
part of 9.2.

Jan Urbański

Datatype I/O functions are allowed to leak memory in CurrentMemoryContext,
since they are generally called in short-lived contexts.  However, plpgsql
calls such functions for purposes of type conversion, and was calling them
in its procedure context.  Therefore, any leaked memory would not be
recovered until the end of the plpgsql function.  If such a conversion
was done within a loop, quite a bit of memory could get consumed.  Fix by
calling such functions in the transient "eval_econtext", and adjust other
logic to match.  Back-patch to all supported versions.

Andres Freund, Jan Urbański, Tom Lane

Don't quote the output of format_procedure(); it's already quoted quite
enough.  Remove the fn_name field, which was now just dead weight.  Fix
remaining expected-output files.

This got broken by commit 4c6cedd1,
which caused PL/pgsql error messages to print the function
signature, not just the name.

Per buildfarm.

This makes it unambiguous which function the message is coming from, if you
have overloaded functions.

Pavel Stehule, reviewed by Abhijit Menon-Sen.

Add result object functions .colnames, .coltypes, .coltypmods to
obtain information about the result column names and types, which was
previously not possible in the PL/Python SPI interface.

reviewed by Abhijit Menon-Sen

Along the way, add a missing dependency in the GNUmakefile.

Alex Hunsaker, with a slight adjustment by me.

Apparently the perl garbage collector was a bit too eager, so here
we control when the new SV is garbage collected.

Certain things like typeglobs or readonly things like $^V cause
perl's SvPVutf8() to die nastily and crash the backend. To avoid
that bug we make a copy of the object, which will subsequently be
garbage collected.

Back patched to 9.1 where we first started using SvPVutf8().

Per -hackers discussion. Original problem reported by David Wheeler.

For easier source reading

This is the standard behavior but was forgotten in some places.

This moves the code around from one huge file into hopefully logical
and more manageable modules.  For the most part, the code itself was
not touched, except: PLy_function_handler and PLy_trigger_handler were
renamed to PLy_exec_function and PLy_exec_trigger, because they were
not actually handlers in the PL handler sense, and it makes the naming
more similar to the way PL/pgSQL is organized.  The initialization of
the procedure caches was separated into a new function
init_procedure_caches to keep the hash tables private to
plpy_procedures.c.

Jan Urbański and Peter Eisentraut

Lots of repetitive code was moved into new functions
PLy_spi_subtransaction_{begin,commit,abort}.

Jan Urbański

Yeb Havinga, reviewed by Kevin Grittner, with small changes by me.

Add a function plpy.cursor that is similar to plpy.execute but uses an
SPI cursor to avoid fetching the entire result set into memory.

Jan Urbański, reviewed by Steve Singer

In the previous coding, callers were faced with an awkward choice:
look up the name, do permissions checks, and then lock the table; or
look up the name, lock the table, and then do permissions checks.
The first choice was wrong because the results of the name lookup
and permissions checks might be out-of-date by the time the table
lock was acquired, while the second allowed a user with no privileges
to interfere with access to a table by users who do have privileges
(e.g. if a malicious backend queues up for an AccessExclusiveLock on
a table on which AccessShareLock is already held, further attempts
to access the table will be blocked until the AccessExclusiveLock
is obtained and the malicious backend's transaction rolls back).

To fix, allow callers of RangeVarGetRelid() to pass a callback which
gets executed after performing the name lookup but before acquiring
the relation lock.  If the name lookup is retried (because
invalidation messages are received), the callback will be re-executed
as well, so we get the best of both worlds.  RangeVarGetRelid() is
renamed to RangeVarGetRelidExtended(); callers not wishing to supply
a callback can continue to invoke it as RangeVarGetRelid(), which is
now a macro.  Since the only one caller that uses nowait = true now
passes a callback anyway, the RangeVarGetRelid() macro defaults nowait
as well.  The callback can also be used for supplemental locking - for
example, REINDEX INDEX needs to acquire the table lock before the index
lock to reduce deadlock possibilities.

There's a lot more work to be done here to fix all the cases where this
can be a problem, but this commit provides the general infrastructure
and fixes the following specific cases: REINDEX INDEX, REINDEX TABLE,
LOCK TABLE, and and DROP TABLE/INDEX/SEQUENCE/VIEW/FOREIGN TABLE.

Per discussion with Noah Misch and Alvaro Herrera.

The old expression sed 's,$(srcdir),python3,' would normally resolve
as sed 's,.,python3,', which is not really what we wanted.  While it
doesn't actually break anything right now, it's still wrong, so put in
a bit more work to make it more robust.

The original coding was
	var->value = (Datum) state;
which is bogus, and then in commit 2f0f7b4b
it was "corrected" to
	var->value = PointerGetDatum(state);
which is a faithful translation but still wrong.

This seems purely cosmetic, though, so no need for a back-patch.

Pavel Stehule

distro version of perl.

David Wheeler and Alex Hunsaker.

Backpatch to 9.1 where it applies cleanly. A simple workaround is available for earlier
branches, and further effort doesn't seem warranted.

Amit Khandekar and Alex Hunsaker.

Backpatched to 9.1 where the problem first occurred.

exception handler. This was a regression in 9.1, when the capability
to catch specific SPI errors was added, so backpatch to 9.1.

Mika Eloranta, with some editing by Jan Urbański.

Fix some bugs in coercion logic and pg_dump; more comment cleanup;
minor cosmetic improvements.

This seems to have been just an oversight in previous foreign-table work.
A quick grep didn't turn up any other places where RELKIND_FOREIGN_TABLE
was obviously omitted.

One change noted by Alexander Soudakov, the other by me.
Back-patch to 9.1.