I decided to give the project a name which recalled the transformation and I called pg_chameleon.
To be honest I didn't like sqlalchemy. Like any other ORM adds an interface to the data layer with a mental approach to the data itself. I lost the interest to developing a migrator very soon, and after all there are thousands of similar tools thousands of times better than mine (e.g. the awesome pgloader)
However recently I revamped the project after discovering a python library capable to read the mysql replication protocol. In few weeks I cleaned all the sqlalchemy stuff, rebuilt the metadata extraction using the information_schema and finally I had an usable tool to replicate the data across the two systems.
I've also changed the license from GPL to the 2 clause BSD.
The tool requires testing. I'm absolutely sure is full of bugs and issues, but it seems to work quite nice.
Some key aspects:
- Is developed in python 2.7. Bear with me, I'll build a port to python 3.x when/if the project will get to an initial release.
- I use tabs (4 space tabs). Bear with me again. I tried to use spaces and I almost thrown my laptop out of the window
- setup.py is not working. I'll fix this as soon as I'll do a release.
- Yes, the sql part use the "hungarian notation" and the keywords are uppercase with strange indentation on the statements .
- The DDL are not yet replicated. I'm thinking to a clever approach to the problem.
That's it. If you want to test it please do and try to break the tool :)
The tool is on github here: https://github.com/the4thdoctor/pg_chameleon/
"I use tabs (4 space tabs)" - your ide/editor sucks ;)
ReplyDelete1 E101 indentation contains mixed spaces and tabs
10 E128 continuation line under-indented for visual indent
1 E201 whitespace after '['
5 E202 whitespace before ')'
1 E203 whitespace before ','
1 E211 whitespace before '('
1 E221 multiple spaces before operator
1 E222 multiple spaces after operator
321 E225 missing whitespace around operator
56 E231 missing whitespace after ','
4 E251 unexpected spaces around keyword / parameter equals
6 E265 block comment should start with '# '
2 E271 multiple spaces after keyword
4 E302 expected 2 blank lines, found 0
13 E303 too many blank lines (3)
1 E402 module level import not at top of file
63 E501 line too long (80 > 79 characters)
1 E712 comparison to True should be 'if cond is True:' or 'if cond:'
1100 W191 indentation contains tabs
109 W291 trailing whitespace
111 W293 blank line contains whitespace
1 W391 blank line at end of file
Thanks. What did you use for the check?
ReplyDeleteInteresting project.
ReplyDeletePrevious poster probably used http://flake8.pycqa.org/en/latest/ to generate those recommendations.