pg chameleon 1.0 alpha1

As mentioned in the presentation I did at the Brighton PostgreSQL Meetup, pg chameleon is a commute project.

In general I work on it when I travel. And thanks to the pgconf and other requirements I travelled a lot recently.

This is the presentation's recording (sorry the audio is suboptimal).

So I was able to build a minimum viable product which seems to work properly.

The alpha1 release is available for download and testing here

https://github.com/the4thdoctor/pg_chameleon/releases/tag/1.0-alpha.1

The system comes with the following limitations.

Installation in virtualenv

For working properly you should use virtualenv for installing the requirements via pip. I'm planning to make the package available via pip once it reaches the RC level.

No daemon yet

The script should be executed in a screen session to keep it running. Currently there's no respawning of the process on failure nor failure detector.

psycopg2 requires python and postgresql dev files

The psycopg2's pip installation requires the python development files and postgresql source code.
Please refer to your distribution for fulfilling those requirements.

DDL replica limitations

DDL and DML mixed in the same transaction are not decoded in the right order. This can result in a replica breakage caused by a wrong jsonb descriptor if the DML change the data on the same table modified by the DDL. I know the issue and I'm working on a solution.
Test please!
Please submit the issues you find.
Bear in mind this is an alpha release. if you use the software in production keep an eye on the process to ensure the data is correctly replicated.

Brighton PostgreSQL Meetup last presentation and recording

This is the recording of the pg_chameleon presentation I gave at the Brighton PostgreSQL Meetup

The slides are available here
 http://www.slideshare.net/FedericoCampoli/pg-chameleon-mysql-to-postgresql-replica

Goodbye to Ubuntu, hello Slackware

When I first read about the systemd I truly believed the linux community would be able to improve this init system. Or at least to stop it if not good enough to be used as replacement for sysvinit.

I was wrong. After a strong and poisonous debating Debian switched to systemd in Jessie. Several contributors left Debian and organised a fork called Devuan which is pushing for the init freedom.

Several distributions switched to this accessory kernel which has lost long time ago the qualification of init. I didn't realise I already had a small part of systemd in my home pc with ubuntu 14.04 until the day of when I upgraded the release to 14.04.4.
The kernel switched to the branch 4 for the old LTS and I wanted to upgrade. After running the usual apt-get update; apt-get dist-upgrade I rebooted the pc and I found that I wasn't able to login anymore in kde.

Opening a console and trying to login manually did not help. The login asked for username again immediately after entering the correct username.

For some reasons systemd.logind got a nervous breakdown after the upgrade.


Searching that particular issue on internet did not return any sort of documents. Only people with similar issues on different distributions which workarounds never worked for me.

After wasting 2 days I decided to reinstall from scratch ubuntu 14.04.4.
Good enough I had all my personal data on a different device and I do regular backups.
But what if this would happen on a production server, where the maintenance windows are well controlled and if you screw up the timeline you get your ass grilled by the users and if you cannot solve quickly this can affect your career.

After this happening I became more sensitive about systemd  taking over the linux ecosystem. The reports all around me were quite scary. Friends with servers unable to mount filesystems after a reboot.  User processes killed on logout. And yes, my bad experience with systemd logind. I needed to find a different distribution.

For my vm I'm using for personal projects I switched to FreeBSD. Simple, clear, robust. It was like rejuvenating 10 years. For the desktop I didn't feel confident enough to switch on this operating system. I started my search but nothing satisfied completely my needs.

Then I remembered a twit from the Debian systemd's mantainer.

https://twitter.com/rfc1036/status/523140918124498944

"If you like obsolete software, I have heard that Slackware is looking for users"

Well, I can tell everything about slackware except it's obsolete.
I'm writing from a shiny Slackware 14.02 with a kernel 4.4.29.
The system comes with kde 4.14.21 and I was able to reinstall it from scratch in few minutes.
The slackware approach is for having super stable software. The slackbuilds repository covers all the missing with the sbo scripts working in a similar way the ports work on FreeBSD.

With slackware I got the same feelings I had with FreeBSD.
Stable, simple to undestand and simple to debug if any issue happens.

And this on Linux, an operating system I'm still in love.

So, goodbye  Ubuntu, hello Slackware. May your days be long and without systemd.

presenting pg chameleon at the brighton postgresql meetup

Back from Estonia after the amazing European PGConf 2016 I'm about to give a talk on the MySQL to PostgreSQL replication project I'm developing.

But first things first. I want to thank all the organisers of the PGConf Europe. The talks were super interesting the location absolutely stunning and the old and new friends I met, remembered me how amazing is the PostgreSQL community. Unfortunately I missed the social events (did somebody said a DBA can have social life? :P).


The meetup will be in Brighton UK Friday 18th of November. There will be  a chat starting at 19.00 with pizza and beer offered by our sponsor.
The talk will start around 19.30 and will be streamed on youtube here.

Here's the talk description.

pg_chameleon is a lightweight replication system written in python. The tool connects to the mysql replication protocol and replicates the data in PostgreSQL.
The author's tool will talk about the history, the logic behind the functions available and will give an interactive usage example.

If you are in Brighton and want to join us please RSVP here.

Slides from the European PGConf 2016

The slides from the talk Life on a rollercoaster are available here 

http://www.slideshare.net/FedericoCampoli/life-on-arollercoaster