installations whose pg_config program does not appear first in the PATH.
Per gripe from Eddie Stanley and subsequent discussions with Fabien Coelho
and others.

Patch from Marko Kreen.

Few cleanups and couple of new things:

 - add SHA2 algorithm to older OpenSSL
 - add BIGNUM math to have public-key cryptography work on non-OpenSSL
   build.
 - gen_random_bytes() function

The status of SHA2 algoritms and public-key encryption can now be
changed to 'always available.'

That makes pgcrypto functionally complete and unless there will be new
editions of AES, SHA2 or OpenPGP standards, there is no major changes
planned.

to avoid useless multiple executions of pg_config.

the pubkey functions a bit.  The actual RSA-specific code
there is tiny, most of the patch consists of reorg of the
pubkey code, as lots of it was written as elgamal-only.

---------------------------------------------------------------------------

The SHLIB section was copy-pasted from somewhere and contains
several unnecessary libs.  This cleans it up a bit.

 -lcrypt
   we don't use system crypt()

 -lssl, -lssleay32
   no SSL here

 -lz in win32 section
   already added on previous line

 -ldes
   The chance anybody has it is pretty low.
   And the chance pgcrypto works with it is even lower.

Also trim the win32 section.

---------------------------------------------------------------------------

It is already disabled in Makefile, remove code too.

---------------------------------------------------------------------------

I was bit hasty making the random exponent 'k' a prime.  Further researh
shows that Elgamal encryption has no specific needs in respect to k,
any random number is fine.

It is bit different for signing, there it needs to be 'relatively prime'
to p - 1,  that means GCD(k, p-1) == 1, which is also a lot lighter than
full primality.  As we don't do signing, this can be ignored.

This brings major speedup to Elgamal encryption.

---------------------------------------------------------------------------

o  pgp_mpi_free: Accept NULLs
o  pgp_mpi_cksum: result should be 16bit
o  Remove function name from error messages - to be similar to other
   SQL functions, and it does not match anyway the called function
o  remove couple junk lines

---------------------------------------------------------------------------

o  Support for RSA encryption
o  Big reorg to better separate generic and algorithm-specific code.
o  Regression tests for RSA.

---------------------------------------------------------------------------

o  Tom stuck a CVS id into file.  I doubt the usefulness of it,
   but if it needs to be in the file then rather at the end.
   Also tag it as comment for asciidoc.
o  Mention bytea vs. text difference
o  Couple clarifications

---------------------------------------------------------------------------

There is a choice whether to update it with pgp functions or
remove it.  I decided to remove it, updating is pointless.

I've tried to keep the core of pgcrypto relatively independent
from main PostgreSQL, to make it easy to use externally if needed,
and that is good.  Eg. that made development of PGP functions much
nicer.

But I have no plans to release it as generic library, so keeping such
doc
up-to-date is waste of time.  If anyone is interested in using it in
other products, he can probably bother to read the source too.

Commented source is another thing - I'll try to make another pass
over code to see if there is anything non-obvious that would need
more comments.

---------------------------------------------------------------------------

Marko Kreen

>
> I think this is because we don't have -lz in SHLIB_LINK.
> Following patch fixes it.

Marko Kreen

of password-based encryption from RFC2440 (OpenPGP).

The goal of this code is to be more featureful encryption solution
than current encrypt(), which only functionality is running cipher
over data.

Compared to encrypt(), pgp_encrypt() does following:

* It uses the equvialent of random Inital Vector to get cipher
  into random state before it processes user data
* Stores SHA-1 of the data into result so any modification
  will be detected.
* Remembers if data was text or binary - thus it can decrypt
  to/from text data.  This was a major nuisance for encrypt().
* Stores info about used algorithms with result, so user needs
  not remember them - more user friendly!
* Uses String2Key algorithms (similar to crypt()) with random salt
  to generate full-length binary key to be used for encrypting.
* Uses standard format for data - you can feed it to GnuPG, if needed.

Optional features (off by default):

* Can use separate session key - user data will be encrypted
  with totally random key, which will be encrypted with S2K
  generated key and attached to result.
* Data compression with zlib.
* Can convert between CRLF<->LF line-endings - to get fully
  RFC2440-compliant behaviour.  This is off by default as
  pgcrypto does not know the line-endings of user data.

Interface is simple:


    pgp_encrypt(data text, key text) returns bytea
    pgp_decrypt(data text, key text) returns text
    pgp_encrypt_bytea(data bytea, key text) returns bytea
    pgp_decrypt_bytea(data bytea, key text) returns bytea

To change parameters (cipher, compression, mdc):

    pgp_encrypt(data text, key text, parms text) returns bytea
    pgp_decrypt(data text, key text, parms text) returns text
    pgp_encrypt_bytea(data bytea, key text, parms text) returns bytea
    pgp_decrypt_bytea(data bytea, key text, parms text) returns bytea

Parameter names I lifted from gpg:

   pgp_encrypt('message', 'key', 'compress-algo=1,cipher-algo=aes256')

For text data, pgp_encrypt simply encrypts the PostgreSQL internal data.

This maps to RFC2440 data type 't' - 'extenally specified encoding'.
But this may cause problems if data is dumped and reloaded into database
which as different internal encoding.  My next goal is to implement data
type 'u' - which means data is in UTF-8 encoding by converting internal
encoding to UTF-8 and back.  And there wont be any compatibility
problems with current code, I think its ok to submit this without UTF-8
encoding by converting internal encoding to UTF-8 and back.  And there
wont be any compatibility problems with current code, I think its ok to
submit this without UTF-8 support.


Here is v4 of PGP encrypt.  This depends on previously sent
Fortuna-patch, as it uses the px_add_entropy function.

- New function: pgp_key_id() for finding key id's.
- Add SHA1 of user data and key into RNG pools.  We need to get
  randomness from somewhere, and it is in user best interests
  to contribute.
- Regenerate pgp-armor test for SQL_ASCII database.
- Cleanup the key handling so that the pubkey support is less
  hackish.

Marko Kreen

- Move openssl random provider to openssl.c and builtin provider
  to internal.c
- Make px_random_bytes use Fortuna, instead of giving error.
- Retarget random.c to aquiring system randomness, for initial seeding
  of Fortuna.  There is ATM 2 functions for Windows,
  reader from /dev/urandom and the regular time()/getpid() silliness.

Marko Kreen

New hashes: SHA256, SHA384, SHA512.

Marko Kreen

to make.  We ship the table file in the tarball and so this dependency
just opens file timestamp skew problems without doing anything useful.
(Not that it should hurt, either ... except for cross-compile builds.)

is used in the toplevel configure.  Per Marko Kreen.

* test error handling
* add tests for des, 3des, cast5
* add some tests to blowfish, rijndael
* Makefile: ability to specify different tests for different crypto
  libraries, so we can skip des, 3des and cast5 for builtin.

Marko Kreen

libmcrypt seems to dead, maintainer address bounces,
and cast-128 fails on 2 of the 3 test vectors from RFC2144.

So I see no reason to keep around stuff I don't trust
anymore.

Support for several crypto libraries is probably only
confusing to users, although it was good for initial
developing - it helped to find hidden assumptions and
forced me to create regression tests for all functionality.

Marko Kreen

Claudio Natoli

>
> The patch adds missing the "libpgport.a" file to the installation under
> "install-all-headers". It is needed by some contribs. I install the
> library in "pkglibdir", but I was wondering whether it should be "libdir"?
> I was wondering also whether it would make sense to have a "libpgport.so"?
>
> It fixes various macros which are used by contrib makefiles, especially
> libpq_*dir and LDFLAGS when used under PGXS. It seems to me that they are
> needed to
>
> It adds the ability to test and use PGXS with contribs, with "make
> USE_PGXS=1". Without the macro, this is exactly as before, there should be
> no difference, esp. wrt the vpath feature that seemed broken by previous
> submission. So it should not harm anybody, and it is useful at least to me.
>
> It fixes some inconsistencies in various contrib makefiles
> (useless override, ":=" instead of "=").

Fabien COELHO

$Header: -> $PostgreSQL Changes ...

* minor doc updates

Marko Kreen

Converted pgcrypto one too.

* Changed default randomness source to libc random()
  That way pgcrypto does not have any external dependencies
  and should work everywhere.
* Re-enabled pgcrypto build in contrib/makefile
* contrib/README update - there is more stuff than
  only 'hash functions'
* Noted the libc random fact in README.pgcrypto


Marko Kreen

salt generation code.  He also urged using better random source
and making possible to choose using bcrypt and xdes rounds more
easily.  So, here's patch:

* For all salt generation, use Solar Designer's own code.  This
  is mostly due fact that his code is more fit for get_random_bytes()
  style interface.
* New function: gen_salt(type, rounds).  This lets specify iteration
  count for algorithm.
* random.c: px_get_random_bytes() function.
  Supported randomness soure: /dev/urandom, OpenSSL PRNG, libc random()
  Default: /dev/urandom.
* Draft description of C API for pgcrypto functions.

New files: API, crypt-gensalt.c, random.c

Marko Kreen

under libdir, for a cleaner separation in the installation layout
and compatibility with binary packaging standards.  Point backend's
default search location there.  The contrib modules are also
installed in the said location, giving them the benefit of the
default search path as well.  No changes in user interface
nevertheless.

* remove support for encode() as it is in main tree now
* remove krb5.c
* new 'PX library' architecture
* remove BSD license from my code to let the general
  PostgreSQL one to apply
* md5, sha1: ANSIfy, use const where appropriate
* various other formatting and clarity changes
* hmac()
* UN*X-like crypt() - system or internal crypt
* Internal crypt: DES, Extended DES, MD5, Blowfish
  crypt-des.c, crypt-md5.c from FreeBSD
  crypt-blowfish.c from Solar Designer
* gen_salt() for crypt() -  Blowfish, MD5, DES, Extended DES
* encrypt(), decrypt(), encrypt_iv(), decrypt_iv()
* Cipher support in mhash.c, openssl.c
* internal: Blowfish, Rijndael-128 ciphers
* blf.[ch], rijndael.[ch] from OpenBSD
* there will be generated file rijndael-tbl.inc.

Marko Kreen

Cygwin with the possible exception of mSQL-interface.  Since I don't
have mSQL installed, I skipped this tool.

Except for dealing with a missing getopt.h (oid2name) and HUGE (seg),
the bulk of the patch uses the standard PostgreSQL approach to deal with
Windows DLL issues.

I tested the build aspect of this patch under Cygwin and Linux without
any ill affects.  Note that I did not actually attempt to test the code
for functionality.

The procedure to apply the patch is as follows:

    $ # save the attachment as /tmp/contrib.patch
    $ # change directory to the top of the PostgreSQL source tree
    $ patch -p0 </tmp/contrib.patch

Jason

* reverse the change #include <> -> "" in krb.c.
  It _must not_ include files in "."
* Makefile update.  Inconsistent var usage and SHLIB was
  not set.

Now it should work with all external libs.

arko Kreen

timing, I know :))  At the moment the digest() function returns
hexadecimal coded hash, but I want it to return pure binary.  I
have also included functions encode() and decode() which support
'base64' and 'hex' encodings, so if anyone needs digest() in hex
he can do encode(digest(...), 'hex').

Main reason for it is "to do one thing and do it well" :)

Another reason is if someone needs really lot of digesting, in
the end he wants to store the binary not the hexadecimal result.
It is really silly to convert it to hex then back to binary
again.  As I said if someone needs hex he can get it.

Well, and the real reason that I am doing encrypt()/decrypt()
functions and _they_ return binary.  For testing I like to see
it in hex occasionally, but it is really wrong to let them
return hex.  Only now it caught my eye that hex-coding in
digest() is wrong.  When doing digest() I thought about 'common
case' but hacking with psql is probably _not_ the common case :)

Marko Kreen