waste is a schema transformation tool for micro-services , it integrates systems like Chef, Orchestrator & gh-ost to match between artifact, his DB server and the gh-ost utility to make the actual changes (plus some changes done directly...)
waste produces a light workload on the master throughout the migration, decoupled from the existing workload on the migrated table.
It has been designed based on years of experience with existing solutions, taking the worst off each.
does it matter? if you really want to know - just read the code.
Hint: it uses gh-ost for the heavy lifting - small wrapping and connections done by me.
Because we can!
- get's changes off some github repo - approved Pull-requests and such...
- can do stuff
- most likely to fail only 47% of changes
Please refer to the docs for more information. No, really, read the docs.
The cheatsheet has it all. You may be interested in invoking waste in various modes:
- a noop migration (merely testing that the migration is valid and good to go)
- a real migration, utilizing a replica (the migration runs on the master;
wastefigures out identities of servers involved. Required mode if your master uses Statement Based Replication) - a real migration, run directly on the master (but
wasteprefers the former) - a real migration on a replica (master untouched)
- a test migration on a replica, the way for you to build trust with
waste's operation.
Our tips:
- Testing above all, try out
--test-on-replicafirst few times. Better yet, make it continuous. We have multiple replicas where we iterate our entire fleet of production tables, migrating them one by one, checksumming the results, verifying migration is good. - For each master migration, first issue a noop
- Then issue the real thing via
--execute.
More tips:
- Use
--exact-rowcountfor accurate progress indication - Use
--postpone-cut-over-flag-fileto gain control over cut-over timing - Get familiar with the interactive commands
gh-ost requires an account with these privileges: ALTER, CREATE, DELETE, DROP, INDEX, INSERT, LOCK TABLES, SELECT, TRIGGER, UPDATE on the migrated database
Also see:
- requirements and limitations
- common questions
- what if?
- the fine print
- Community questions
- Using
wasteon AWS RDS
A couple of whiskey shots, Vodka, 2 bottles of wine and a six pack.
waste is licensed under the MIT license
waste uses 3rd party libraries, each with their own license. These are found here.
waste is released at a stable state, but with mileage to go. We are open to pull requests. Please first discuss your intentions via Issues.
I developed waste to learn Go-lang. honestly if you find use for this feel free to suggest - I'd be happy to make changes.
Please see Coding waste for a guide to getting started developing with waste.
waste is never GA and never stable.
waste is a Go project; it is built with Go 1.8 (though 1.7 should work as well). To build on your own, use either:
- script/build - this is the same build script used by CI hence the authoritative; artifact is
./bin/wastebinary. - build.sh for building
tar.gzartifacts in/tmp/waste
waste is designed, authored, reviewed and tested by me: