Code Sharing

[lancefogle]

I am opening this issue simply for a place to begin some code sharing back to your project. I ended up creating a brand new project inspired by this one. I have contributed some code here in the past and, in the spirit of the GPL, I would like to open the door for you to gain from new features and code in our project. That starts with knowing it exists to cherry pick whatever you find useful. Being that it is a complete rewrite, it would be hard to put together pull requests because of some different implementations.

OnyxBackupVM

Some notable features I can think of that are currently not implemented in NAUbackup you may want borrow for your project:

• Host backups including rotation like other backups
• VDI backups that are fully configurable to include selecting disks other than just the root (xvda) one
• Fully configurable logging
• Completely external configuration with no changes to script/program to operate
• Works "out of the box" with no configuration for basic backup/rotation with sane defaults
• Configuration sanitized to prevent possible bugs from unknown or incorrect params in config files
• No need for password to run backups
• Slowly migrating to pure XAPI calls for potential remote usability later
• Configuration options for customizing and enabling/disabling SMTP functionality
• Handling of orphaned metadata/VM backup files in case a previous one fails
• Backup size reporting for each VM
• Backup disk space checking prior to each backup to prevent crashes

Please let me know if you have any questions on anything. Thanks for all your hard work on this project as it is a great contribution that shows just in how many people use it.

[thomasmck]

Hi @lancefogle,

As example of a backup program which can run from a remote host using pure API calls see: https://github.com/gaborigloi/xs-cbt-backup

I think the disk space check in partiuclar is a good idea and should be fairly simple to cherrypick.

Tom

[lancefogle]

@thomasmck Thanks! I'll take a look. Should help.

As for the checking the disk space, I think you would be right. There is just a single method/function called get_remaining_space() that is called and then checked to see if it is greater than a user configurable threshold that defaults to 20%. If less than "threshold" backup space remaining, it will log an error and halt further backup.

[NAUbackup]

@lancefogel -- I had to think about all this for a while before responding, and am still confused. Is your project intended to be a means to create code in parallel with the intent of providing potential contributions to this project, as a stand-alone and competitive project, or something in between?

Naturally, any open-source code is subject to reuse and spin-offs and other projects are inevitable and in the interest of pushing innovation. I am just trying to do what I feel is best and consistent for the community with the hope and intention of providing support over a reasonably long period of time to those who make use of a particular project.

Thank you.

[lancefogle]

My project is a separate code-base inspired by the great work you did here. I originally was going to try to implement some of these features for you in my free time but then I needed to have some of these features in the short-term and I had different thoughts on implementation. I wrote my code to meet my needs with my own servers and my clients' use cases but in no way meant it to be "competitive" thus me reaching out here with nothing but goodwill for your project which inspired mine. Being different code, it is not easy to put together a pull request for a specific feature/fix but wanted to circle back and make you aware of my code as it has new features and things that you and I had discussed as well as others have requested here. Simply put, since the GPL is all about the source and any modifications or derivatives being shared as well so everyone can benefit, I am simply seeking to share with you some features you already have requests for in your issues so that you could potentially borrow some code to help you expedite said features in your own project. I know you are always short on time to develop this project from your many statements of such. Just trying to help and also be a good steward of open-source projects and communities...

[lancefogle]

I mean, it would be great if we could create code in parallel in cases where it was needed but isn't a requirement. Just trying to help is all.

…

On Sun, Jun 24, 2018, 12:56 PM Lance Fogle ***@***.***> wrote: My project is a separate code-base inspired by the great work you did here. I originally was going to try to implement some of these features for you in my free time but then I needed to have some of these features in the short-term and I had different thoughts on implementation. I wrote my code to meet my needs with my own servers and my clients' use cases but in no way meant it to be "competitive" thus me reaching out here with nothing but goodwill for your project which inspired mine. Being different code, it is not easy to put together a pull request for a specific feature/fix but wanted to circle back and make you aware of my code as it has new features and things that you and I had discussed as well as others have requested here. Simply put, since the GPL is all about the source and any modifications or derivatives being shared as well so everyone can benefit, I am simply seeking to share with you some features you already have requests for in your issues so that you could potentially borrow some code to help you expedite said features in your own project. I know you are always short on time to develop this project from your many statements of such. Just trying to help and also be a good steward of open-source projects and communities... On Jun 24, 2018 12:30 PM, "NAUbackup" ***@***.***> wrote: @lancefogel -- I had to think about all this for a while before responding, and am still confused. Is your project intended to be a means to create code in parallel with the intent of providing potential contributions to *this* project, as a stand-alone and competitive project, or something in between? Naturally, any open-source code is subject to reuse and spin-offs and other projects are inevitable and in the interest of pushing innovation. I am just trying to do what I feel is best and consistent for the community with the hope and intention of providing support over a reasonably long period of time to those who make use of a particular project. Thank you. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#82 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWoDWxHo2QIadWHTqe-8rwUep4gj7z5rks5t_77AgaJpZM4UszZB> .

[NAUbackup]

Hi, Lance. Thanks for your response. My concern is of course that this will inevitably lead to a split in how the different offerings are used and supported. My preference would be a single development, but I completely see your point of the difficulty of merging code that is very different and in your case, employs some newer coding methodologies that were not incorporated into the original. That and the structure is just very different, making merges all the more complicated. That all said, I'm very impressed how you were able to pull this all together so quickly. You're of course spot on that I have way less time to invest into development these days and have to think about how long-term support should be done, keeping again the community in mind.

We have out own internal version of some of this which we don't release because much of it is not really relevant to the community as a whole. Keeping things generic can be a bit of a challenge.

Maybe the longer-term goal could be some sort of merged project -- just thinking aloud here. I am certainly not going to be around forever as eventual retirement will kick in at some point and consequently, I have to think about the impact of that at some point over the next few years.

Thanks for your clarification and willingness to contribute so freely in the spirit of the benefit to the community at large.

--Tobias

[lancefogle]

You mean you don't plan to work until you're 100? Haha I'm happy to try and help anyway I can. Just let me know! If I can put together some fairly easy pull requests I will try to do so.

[NAUbackup]

I had thought that at one point, but the direction the whole IT industry is turning is discouraging to me and again, time commitments are getting tougher as duties and responsibilities get shifted around. Thanks again!

[lancefogle]

@NAUbackup I submitted pull request #84 to add backup space threshold checks to your program. Let me know what you think.

In OnyxBackupVM I just refactored the entire service layer to make it easier to implement some new features. In doing so, I was able to add automatic detection and running of quiesced snapshots for Windows VMs that have the Xen VSS provider installed.

If we can find a simple way to integrate that into VmBackup it would resolve issue #21. I can try to look into it when I have some time.

Side Note: It being automatic isn't a concern because the user can simply not install the Xen VSS provider (a manual step anyways) on any VMs they don't care to have quiesced snapshots of. That being said, it would be kinda silly to not want the disk consistency of a quiesced vs non-quiesced backup for your Windows VMs.

[NAUbackup]

Hi, Lance -- looks cool. I have been off a few days working on household remodeling/maintenance projects, but will try to catch up, test out, etc. ASAP. Thank you!