Skip to content
Snippets Groups Projects
  1. Jan 20, 2017
    • Robert Haas's avatar
      Avoid useless respawining the autovacuum launcher at high speed. · 5dff230e
      Robert Haas authored
      When (1) autovacuum = off and (2) there's at least one database with
      an XID age greater than autovacuum_freeze_max_age and (3) all tables
      in that database that need vacuuming are already being processed by a
      worker and (4) the autovacuum launcher is started, a kind of infinite
      loop occurs.  The launcher starts a worker and immediately exits.  The
      worker, finding no worker to do, immediately starts the launcher,
      supposedly so that the next database can be processed.  But because
      datfrozenxid for that database hasn't been advanced yet, the new
      worker gets put right back into the same database as the old one,
      where it once again starts the launcher and exits.  High-speed ping
      pong ensues.
      
      There are several possible ways to break the cycle; this seems like
      the safest one.
      
      Amit Khandekar (code) and Robert Haas (comments), reviewed by
      Álvaro Herrera.
      
      Discussion: http://postgr.es/m/CAJ3gD9eWejf72HKquKSzax0r+epS=nAbQKNnykkMA0E8c+rMDg@mail.gmail.com
      5dff230e
  2. May 10, 2016
  3. Jun 20, 2015
  4. Jun 19, 2015
  5. Apr 08, 2015
    • Alvaro Herrera's avatar
      Fix autovacuum launcher shutdown sequence · 37dc228e
      Alvaro Herrera authored
      It was previously possible to have the launcher re-execute its main loop
      before shutting down if some other signal was received or an error
      occurred after getting SIGTERM, as reported by Qingqing Zhou.
      
      While investigating, Tom Lane further noticed that if autovacuum had
      been disabled in the config file, it would misbehave by trying to start
      a new worker instead of bailing out immediately -- it would consider
      itself as invoked in emergency mode.
      
      Fix both problems by checking the shutdown flag in a few more places.
      These problems have existed since autovacuum was introduced, so
      backpatch all the way back.
      37dc228e
  6. Oct 03, 2014
    • Alvaro Herrera's avatar
      Don't balance vacuum cost delay when per-table settings are in effect · 769d6815
      Alvaro Herrera authored
      When there are cost-delay-related storage options set for a table,
      trying to make that table participate in the autovacuum cost-limit
      balancing algorithm produces undesirable results: instead of using the
      configured values, the global values are always used,
      as illustrated by Mark Kirkwood in
      http://www.postgresql.org/message-id/52FACF15.8020507@catalyst.net.nz
      
      Since the mechanism is already complicated, just disable it for those
      cases rather than trying to make it cope.  There are undesirable
      side-effects from this too, namely that the total I/O impact on the
      system will be higher whenever such tables are vacuumed.  However, this
      is seen as less harmful than slowing down vacuum, because that would
      cause bloat to accumulate.  Anyway, in the new system it is possible to
      tweak options to get the precise behavior one wants, whereas with the
      previous system one was simply hosed.
      
      This has been broken forever, so backpatch to all supported branches.
      This might affect systems where cost_limit and cost_delay have been set
      for individual tables.
      769d6815
  7. Jul 30, 2014
    • Tom Lane's avatar
      Avoid wholesale autovacuuming when autovacuum is nominally off. · c83aed34
      Tom Lane authored
      When autovacuum is nominally off, we will still launch autovac workers
      to vacuum tables that are at risk of XID wraparound.  But after we'd done
      that, an autovac worker would proceed to autovacuum every table in the
      targeted database, if they meet the usual thresholds for autovacuuming.
      This is at best pretty unexpected; at worst it delays response to the
      wraparound threat.  Fix it so that if autovacuum is nominally off, we
      *only* do forced vacuums and not any other work.
      
      Per gripe from Andrey Zhidenkov.  This has been like this all along,
      so back-patch to all supported branches.
      c83aed34
  8. May 06, 2014
    • Bruce Momjian's avatar
      Remove tabs after spaces in C comments · 0b44914c
      Bruce Momjian authored
      This was not changed in HEAD, but will be done later as part of a
      pgindent run.  Future pgindent runs will also do this.
      
      Report by Tom Lane
      
      Backpatch through all supported branches, but not HEAD
      0b44914c
  9. Jun 10, 2012
  10. May 14, 2012
    • Heikki Linnakangas's avatar
      Update comments that became out-of-date with the PGXACT struct. · 9e4637bf
      Heikki Linnakangas authored
      When the "hot" members of PGPROC were split off to separate PGXACT structs,
      many PGPROC fields referred to in comments were moved to PGXACT, but the
      comments were neglected in the commit. Mostly this is just a search/replace
      of PGPROC with PGXACT, but the way the dummy PGPROC entries are created for
      prepared transactions changed more, making some of the comments totally
      bogus.
      
      Noah Misch
      9e4637bf
  11. May 10, 2012
    • Tom Lane's avatar
      Make WaitLatch's WL_POSTMASTER_DEATH result trustworthy; simplify callers. · f40022f1
      Tom Lane authored
      Per a suggestion from Peter Geoghegan, make WaitLatch responsible for
      verifying that the WL_POSTMASTER_DEATH bit it returns is truthful (by
      testing PostmasterIsAlive).  Then simplify its callers, who no longer
      need to do that for themselves.  Remove weasel wording about falsely-set
      result bits from WaitLatch's API contract.
      f40022f1
  12. Jan 26, 2012
  13. Jan 19, 2012
    • Magnus Hagander's avatar
      Separate state from query string in pg_stat_activity · 4f42b546
      Magnus Hagander authored
      This separates the state (running/idle/idleintransaction etc) into
      it's own field ("state"), and leaves the query field containing just
      query text.
      
      The query text will now mean "current query" when a query is running
      and "last query" in other states. Accordingly,the field has been
      renamed from current_query to query.
      
      Since backwards compatibility was broken anyway to make that, the procpid
      field has also been renamed to pid - along with the same field in
      pg_stat_replication for consistency.
      
      Scott Mead and Magnus Hagander, review work from Greg Smith
      4f42b546
  14. Jan 02, 2012
  15. Nov 30, 2011
    • Tom Lane's avatar
      Prevent autovacuum transactions from running in serializable mode. · 73d1bfd0
      Tom Lane authored
      Force the transaction isolation level to READ COMMITTED in autovacuum
      worker and launcher processes.  There is no benefit to using a higher
      isolation level, and doing so could result in delaying foreground
      transactions (or maybe even causing unnecessary serialization failures?).
      Noted by Dan Ports.
      
      Also, make sure we disable zero_damaged_pages and statement_timeout in
      the autovac launcher, not only workers.  Now that the launcher can run
      transactions, these settings could affect its behavior, and it seems
      like the same arguments apply to the launcher as the workers.
      73d1bfd0
  16. Sep 11, 2011
    • Peter Eisentraut's avatar
      Remove many -Wcast-qual warnings · 1b81c2fe
      Peter Eisentraut authored
      This addresses only those cases that are easy to fix by adding or
      moving a const qualifier or removing an unnecessary cast.  There are
      many more complicated cases remaining.
      1b81c2fe
  17. Sep 09, 2011
    • Tom Lane's avatar
      Move Timestamp/Interval typedefs and basic macros into datatype/timestamp.h. · a7801b62
      Tom Lane authored
      As per my recent proposal, this refactors things so that these typedefs and
      macros are available in a header that can be included in frontend-ish code.
      I also changed various headers that were undesirably including
      utils/timestamp.h to include datatype/timestamp.h instead.  Unsurprisingly,
      this showed that half the system was getting utils/timestamp.h by way of
      xlog.h.
      
      No actual code changes here, just header refactoring.
      a7801b62
  18. Aug 10, 2011
    • Tom Lane's avatar
      Change the autovacuum launcher to use WaitLatch instead of a poll loop. · 4dab3d5a
      Tom Lane authored
      In pursuit of this (and with the expectation that WaitLatch will be needed
      in more places), convert the latch field that was already added to PGPROC
      for sync rep into a generic latch that is activated for all PGPROC-owning
      processes, and change many of the standard backend signal handlers to set
      that latch when a signal happens.  This will allow WaitLatch callers to be
      wakened properly by these signals.
      
      In passing, fix a whole bunch of signal handlers that had been hacked to do
      things that might change errno, without adding the necessary save/restore
      logic for errno.  Also make some minor fixes in unix_latch.c, and clean
      up bizarre and unsafe scheme for disowning the process's latch.  Much of
      this has to be back-patched into 9.1.
      
      Peter Geoghegan, with additional work by Tom
      4dab3d5a
  19. Aug 04, 2011
    • Robert Haas's avatar
      Create VXID locks "lazily" in the main lock table. · 84e37126
      Robert Haas authored
      Instead of entering them on transaction startup, we materialize them
      only when someone wants to wait, which will occur only during CREATE
      INDEX CONCURRENTLY.  In Hot Standby mode, the startup process must also
      be able to probe for conflicting VXID locks, but the lock need never be
      fully materialized, because the startup process does not use the normal
      lock wait mechanism.  Since most VXID locks never need to touch the
      lock manager partition locks, this can significantly reduce blocking
      contention on read-heavy workloads.
      
      Patch by me.  Review by Jeff Davis.
      84e37126
  20. Jul 08, 2011
    • Heikki Linnakangas's avatar
      Introduce a pipe between postmaster and each backend, which can be used to · 89fd72cb
      Heikki Linnakangas authored
      detect postmaster death. Postmaster keeps the write-end of the pipe open,
      so when it dies, children get EOF in the read-end. That can conveniently
      be waited for in select(), which allows eliminating some of the polling
      loops that check for postmaster death. This patch doesn't yet change all
      the loops to use the new mechanism, expect a follow-on patch to do that.
      
      This changes the interface to WaitLatch, so that it takes as argument a
      bitmask of events that it waits for. Possible events are latch set, timeout,
      postmaster death, and socket becoming readable or writeable.
      
      The pipe method behaves slightly differently from the kill() method
      previously used in PostmasterIsAlive() in the case that postmaster has died,
      but its parent has not yet read its exit code with waitpid(). The pipe
      returns EOF as soon as the process dies, but kill() continues to return
      true until waitpid() has been called (IOW while the process is a zombie).
      Because of that, change PostmasterIsAlive() to use the pipe too, otherwise
      WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while
      PostmasterIsAlive() would claim it's still alive. That could easily lead to
      busy-waiting while postmaster is in zombie state.
      
      Peter Geoghegan with further changes by me, reviewed by Fujii Masao and
      Florian Pflug.
      89fd72cb
  21. Jul 04, 2011
  22. Jun 29, 2011
  23. May 09, 2011
  24. Apr 10, 2011
  25. Apr 05, 2011
  26. Apr 04, 2011
  27. Mar 06, 2011
    • Simon Riggs's avatar
      Efficient transaction-controlled synchronous replication. · a8a8a3e0
      Simon Riggs authored
      If a standby is broadcasting reply messages and we have named
      one or more standbys in synchronous_standby_names then allow
      users who set synchronous_replication to wait for commit, which
      then provides strict data integrity guarantees. Design avoids
      sending and receiving transaction state information so minimises
      bookkeeping overheads. We synchronize with the highest priority
      standby that is connected and ready to synchronize. Other standbys
      can be defined to takeover in case of standby failure.
      
      This version has very strict behaviour; more relaxed options
      may be added at a later date.
      
      Simon Riggs and Fujii Masao, with reviews by Yeb Havinga, Jaime
      Casanova, Heikki Linnakangas and Robert Haas, plus the assistance
      of many other design reviewers.
      a8a8a3e0
  28. Feb 08, 2011
    • Robert Haas's avatar
      Avoid having autovacuum workers wait for relation locks. · 32896c40
      Robert Haas authored
      Waiting for relation locks can lead to starvation - it pins down an
      autovacuum worker for as long as the lock is held.  But if we're doing
      an anti-wraparound vacuum, then we still wait; maintenance can no longer
      be put off.
      
      To assist with troubleshooting, if log_autovacuum_min_duration >= 0,
      we log whenever an autovacuum or autoanalyze is skipped for this reason.
      
      Per a gripe by Josh Berkus, and ensuing discussion.
      32896c40
  29. Jan 01, 2011
  30. Dec 13, 2010
    • Robert Haas's avatar
      Generalize concept of temporary relations to "relation persistence". · 5f7b58fa
      Robert Haas authored
      This commit replaces pg_class.relistemp with pg_class.relpersistence;
      and also modifies the RangeVar node type to carry relpersistence rather
      than istemp.  It also removes removes rd_istemp from RelationData and
      instead performs the correct computation based on relpersistence.
      
      For clarity, we add three new macros: RelationNeedsWAL(),
      RelationUsesLocalBuffers(), and RelationUsesTempNamespace(), so that we
      can clarify the purpose of each check that previous depended on
      rd_istemp.
      
      This is intended as infrastructure for the upcoming unlogged tables
      patch, as well as for future possible work on global temporary tables.
      5f7b58fa
  31. Nov 20, 2010
    • Tom Lane's avatar
      Fix leakage of cost_limit when multiple autovacuum workers are active. · b58c2505
      Tom Lane authored
      When using default autovacuum_vac_cost_limit, autovac_balance_cost relied
      on VacuumCostLimit to contain the correct global value ... but after the
      first time through in a particular worker process, it didn't, because we'd
      trashed it in previous iterations.  Depending on the state of other autovac
      workers, this could result in a steady reduction of the effective
      cost_limit setting as a particular worker processed more and more tables,
      causing it to go slower and slower.  Spotted by Simon Poole (bug #5759).
      Fix by saving and restoring the GUC variables in the loop in do_autovacuum.
      
      In passing, improve a few comments.
      
      Back-patch to 8.3 ... the cost rebalancing code has been buggy since it was
      put in.
      b58c2505
  32. Nov 08, 2010
    • Alvaro Herrera's avatar
      Fix permanent memory leak in autovacuum launcher · 854ae8c3
      Alvaro Herrera authored
      get_database_list was uselessly allocating its output data, along some
      created along the way, in a permanent memory context.  This didn't
      matter when autovacuum was a single, short-lived process, but now that
      the launcher is permanent, it shows up as a permanent leak.
      
      To fix, make get_database list allocate its output data in the caller's
      context, which is in charge of freeing it when appropriate; and the
      memory leaked by heap_beginscan et al is allocated in a throwaway
      transaction context.
      854ae8c3
  33. Sep 20, 2010
  34. Apr 28, 2010
    • Tom Lane's avatar
      Modify ShmemInitStruct and ShmemInitHash to throw errors internally, · 77acab75
      Tom Lane authored
      rather than returning NULL for some-but-not-all failures as they used to.
      Remove now-redundant tests for NULL from call sites.
      
      We had to do something about this because many call sites were failing to
      check for NULL; and changing it like this seems a lot more useful and
      mistake-proof than adding checks to the call sites without them.
      77acab75
  35. Feb 26, 2010
  36. Feb 14, 2010
    • Robert Haas's avatar
      Wrap calls to SearchSysCache and related functions using macros. · e26c539e
      Robert Haas authored
      The purpose of this change is to eliminate the need for every caller
      of SearchSysCache, SearchSysCacheCopy, SearchSysCacheExists,
      GetSysCacheOid, and SearchSysCacheList to know the maximum number
      of allowable keys for a syscache entry (currently 4).  This will
      make it far easier to increase the maximum number of keys in a
      future release should we choose to do so, and it makes the code
      shorter, too.
      
      Design and review by Tom Lane.
      e26c539e
  37. Jan 02, 2010
  38. Dec 30, 2009
    • Tom Lane's avatar
      Revise pgstat's tracking of tuple changes to improve the reliability of · 48c192c1
      Tom Lane authored
      decisions about when to auto-analyze.
      
      The previous code depended on n_live_tuples + n_dead_tuples - last_anl_tuples,
      where all three of these numbers could be bad estimates from ANALYZE itself.
      Even worse, in the presence of a steady flow of HOT updates and matching
      HOT-tuple reclamations, auto-analyze might never trigger at all, even if all
      three numbers are exactly right, because n_dead_tuples could hold steady.
      
      To fix, replace last_anl_tuples with an accurately tracked count of the total
      number of committed tuple inserts + updates + deletes since the last ANALYZE
      on the table.  This can still be compared to the same threshold as before, but
      it's much more trustworthy than the old computation.  Tracking this requires
      one more intra-transaction counter per modified table within backends, but no
      additional memory space in the stats collector.  There probably isn't any
      measurable speed difference; if anything it might be a bit faster than before,
      since I was able to eliminate some per-tuple arithmetic operations in favor of
      adding sums once per (sub)transaction.
      
      Also, simplify the logic around pgstat vacuum and analyze reporting messages
      by not trying to fold VACUUM ANALYZE into a single pgstat message.
      
      The original thought behind this patch was to allow scheduling of analyzes
      on parent tables by artificially inflating their changes_since_analyze count.
      I've left that for a separate patch since this change seems to stand on its
      own merit.
      48c192c1
  39. Nov 16, 2009
  40. Aug 31, 2009
    • Tom Lane's avatar
      Change the autovacuum launcher to read pg_database directly, rather than · 00e6a16d
      Tom Lane authored
      via the "flat files" facility.  This requires making it enough like a backend
      to be able to run transactions; it's no longer an "auxiliary process" but
      more like the autovacuum worker processes.  Also, its signal handling has
      to be brought into line with backends/workers.  In particular, since it
      now has to handle procsignal.c processing, the special autovac-launcher-only
      signal conditions are moved to SIGUSR2.
      
      Alvaro, with some cleanup from Tom
      00e6a16d
Loading