Tuesday, 4 March 2014

Chapter 3 -part 1 - Install structure

Install structure

In this chapter we'll look at the PostgreSQL installation. Whatever installation method you did, the PostgreSQL binaries are the same. The packaged version comes with few extra utilities we'll take a look later as those present some caveats to be aware.
The source installation puts all the binaries into the bin sub directory in the target location specified by the -prefix parameter.
The packaged install puts the binaries into a folder organised per major PostgreSQL version.
e.g. /usr/lib/postgresql/9.3/bin/

The core binaries

We'll look first at the core binaries like postgres or psql.
A separate section is dedicated to wrappers and the contributed modules.


postgres

Is the database itself. It's possible to start it directly or using the pg_ctl utility.
The latter is the best way to control the instance except in case of the XID wraparound failure. In this case the only way to run the instance is executing postgres in single user mode.
For historical reason there's usually also the postmaster symbolic link to postgres.


pg_ctl

As mentioned before this is the utility for managing the PostgreSQL instance.
It can start,stop, reload the postgres process. It's also capable to send kill signals to the running instance.
pg_ctl accepts on the command line various options. The most important are the -D or -pgdata= to specify the database storage area and the -m to specify the shutdown mode. Check the section 4.2.2 for details.
pg_ctl also requires the action to perform on the instance.
The supported actions are

  • init[db] initialises a directory as PostgreSQL data area
  • start starts a PostgreSQL instance
  • stop shutdowns a PostgreSQL instance
  • reload reloads the configuration's files
  • status checks the PostgreSQL instance running status
  • promote promotes a standby server
  • kill sends a custom signal to the running instance


initdb

Is the binary which initialises the PostgreSQL data area. initdb requires an empty directory to initialise. Various options can be specified on the command line, like the character enconding or the collation order.


psql

Is the PostgreSQL command line client. Despite his look very essential is one of the most flexible tools available to interact with the server. As is part of the core distribution is always present.


pg_dump

Is the binary dedicated to the backup. Generates consistent backups in various formats. The default is plain text. Supports the parallel dump implemented using the snapshot exports.
The switch to set is -F followed by a letter to indicate the wanted output format.

  • p saves the sql statements to reconstruct the schema and/or data in plain text with no compression.
  • c is the custom PostgreSQL format. Supports parallel restore, compression and object search.
  • d the dump is saved in a directory. With this format is possible to dump in parallel.
  • t saves the dump in the standard tar format.
Please on't be confused by the pg_dumpall . This does look more like a wrapper for pg_dump rather a dedicated program.
As pg_dumpall doesn't support all the pg_dump features is still very useful to save the cluster wide objects like the users with the switch -globals-only.


pg_restore

As the name suggests this is used to restore the database's dump. It can read the backups generated in all formats by pg_dump. The restore target can be a PostgreSQL connection or a file. If the backup's format is directory or custom then pg_restore can run the data load and index/key creations in multiple jobs.


pg_controldata

The program query the pg_control file where instance's vital informations are stored. The pg_control is one of the most important cluster's files. With a corrupted pg_control the instance cannot start.


pg_resetxlog

If the WAL files or the get corrupted the instance cannot perform a crash recovery. pg_resetxlog can solve the problem and make the instance startable but keep in mind this must the last chance, after trying any other possible solution.
The reset removes the WAL files and creates a new pg_control. The XID are also restarted.
The instance becomes startable at the cost of losing any reference between the transactions and the data files. All the physical data integrity is lost and any attempt to run DML queries results in data corruption.
The on line manual is absolutely clear on this point.
After running pg_resetxlog the database must start without user access, the entire content must be dumped, the data directory must be dropped and recreated from scratch using initdb and then the dump file can be restored using psql or pg_restore

Thursday, 20 February 2014

Chapter 2 - Database installation

This chapter will cover the install procedure, on Debian Gnu linux compiling from source and using the packaged install from the pgdg archive.

Install from source

Installing from source, using the default configuration settings requires the root access as the default install location is in /usr/local/. To simplify things I've created a procedure with minimal need for root access. This of course is still required but only for the os user creation and to install the dependencies.
Before starting with the postgresql part, ask your sysadmin, or do it by yourself, to do the following

  • Create a postgres group and a postgres user
  • Add the postgres user to the postgres group
  • Install the packages
  • build-essential
  • libreadline6-dev
  • zlib1g-dev
When everything is in place login as postgres user and download the source's tarball.
mkdir download cd download wget http://ftp.postgresql.org/pub/source/v9.3.2/postgresql-9.3.2.tar.bz2
Extract the tarbal with
tar xfj postgresql-9.2.3.tar.bz2 cd postgresql-9.2.3
The configure script have the -prefix option to set the install directory in a custom location. Assuming the postgres user have his home directory in /home/postgres, we'll put the install target into the bin directory organised per mayor version. In this way is possible to have multiple versions on the same box whitout hassle.
mkdir -p /home/postgres/bin/9.3 ./configure -prefix=/home/postgres/bin/9.3
The script will check all the dependencies and will generate the makefiles. Any error at configure time is usually explained good.
When everthing looks fine you can start the build process with the make command. The time required to compile depends from the box speed. On a laptop usually this doesn't require more than 30 minutes.
After the build is complete is a good idea to run the regression tests before installing.
To do it simply run make check.
All the output is written in the directory src/test/regress/results.
If everything is fine you can complete the installation with
make install
After this, into the /home/postgres/bin/9.3 directory will appear with 4 subfolders bin include lib and share.
The bin folder is the place of the database binaries, like the client psql or the server postgres.
The include folder is the place for the server's header files.
The lib folder is the location for all shared libraries and functions used by the database installation.
In the share folder you'll find the example files and the extension.

Packaged install

The PostgreSQL Global Group mantains an apt repository to simplify the install on the GNU/Linux based on debian.
The supported Linux versions are listed on the wiki page http://wiki.postgresql.org/wiki/Apt and at moment are

  • Debian 6.0 (squeeze)
  • Debian 7.0 (wheezy)
  • Debian unstable (sid)
  • Ubuntu 10.04 (lucid)
  • Ubuntu 12.04 (precise)
  • Ubuntu 13.10 (saucy)
  • Ubuntu 14.04 (trusty)
The packages are available for amd64 and i386.
The available database versions are
  • PostgreSQL 8.4
  • PostgreSQL 9.0
  • PostgreSQL 9.1
  • PostgreSQL 9.2
  • PostgreSQL 9.3
Before starting you shall import the GPG key to validate the packages.
In a root shell simply run wget -quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
Then add the file pgdg.list into the directory /etc/apt/sources.d/ with the following contents
deb http://apt.postgresql.org/pub/repos/apt/ codename-pgdg main
Change the codename value accordingly with your distribuition. (e.g. wheezy) then you can run
apt-get update
To install the full set of binaries simply run
apt-get install postgreql-9.3 postgreql-contrib-9.3 postgreql-client-9.3
This will set up also a database cluster up and running.