Prune functions

[ghost]

Hello,

I'm not sure if this is a bug or not but I'm getting errors. Maybe I'm just doing it wrong but I'm trying to set up a prune cron (that sounds kind of funny). I can't find anything on the wiki about it.

I'm also wondering if there's a way to prune old log files.

I have tried to run both prune and prune_archives but I get errors in both cases. Here is what I've done and what the output is.

Thank you!

nice ./mineos_console.py -d /home/minecraft -s server1 prune_archives 4w
Traceback (most recent call last):
  File "./mineos_console.py", line 67, in <module>
    retval = getattr(instance, args.cmd)(*arguments)
  File "/home/minecraft/mineos.py", line 458, in prune_archives
    self._command_direct(self.command_delete_files(filename), self.env['awd'])
  File "/home/minecraft/mineos.py", line 654, in _command_direct
    preexec_fn=self._demote(self.owner.pw_uid, self.owner.pw_gid))
  File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['rm', '--', '4w']' returned non-zero exit status 1

./mineos_console.py -d /home/minecraft -s server1 prune 4w
Traceback (most recent call last):
  File "./mineos_console.py", line 67, in <module>
    retval = getattr(instance, args.cmd)(*arguments)
  File "/home/minecraft/mineos.py", line 43, in wrapper
    fn(self, *args, **kwargs)
  File "/home/minecraft/mineos.py", line 454, in prune
    self._command_direct(self.command_prune(step), self.env['bwd'])
  File "/home/minecraft/mineos.py", line 654, in _command_direct
    preexec_fn=self._demote(self.owner.pw_uid, self.owner.pw_gid))
  File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['/usr/bin/rdiff-backup', '--force', '--remove-older-than', '4w', '/home/minecraft/backup/server1']' returned non-zero exit status 1

[hexparrot]

I see that you have a non-standard location for your minecraft servers at /home/minecraft, which means that what user you're executing this script at could be a possible problem. What user are you doing it as; you've removed that from your copied command.

Also, you can try this command, which will allow you to see the error you're receiving:

./mineos_console.py -s server1 command_prune 4w

This command won't actually prune, but tell you the command it runs--the command that is returning the non-zero exit status. It should tell us whether it's a permissions issue or something else. Best guess without any additional information is you've made backups as 'root', which could explain why (as user 'minecraft') you're unable to delete or prune the archives/backups.

[ghost]

Aha, thanks for that!

That is how it was setup by default from the OVH distribution of MineOS which I kind of like even when manually setting it up since the /home partition often by default on hosts has much more space than /var but /var/games/minecraft is symlinked to /home/minecraft. And now when I set MineOS up manually on other servers I place it in /home/minecraft or /home/minecraft/mineos when it's far less likely to run out of disk space which can be important on 120GB SSD drives.

Backups are made by minecraft, since they were actually created from using the built-in backup and archive checkboxes in the server.config screens in MineOS.

~$ ./mineos_console.py -s server1 command_prune 4w
'/usr/bin/rdiff-backup --force --remove-older-than 4w /var/games/minecraft/backup/server1'

So it's trying to remove backups from /var/games/minecraft/backup/server1 which obviously doesn't exist other then as a symlink but maybe the rdiff-backup command takes an extra parameter to follow symlinks or something. When I run the command it outputs though I get this:

Fatal Error: Bad time string "4w"

The acceptible time strings are intervals (like "3D64s"), w3-datetime
strings, like "2002-04-26T04:22:01-07:00" (strings like
"2002-04-26T04:22:01" are also acceptable - rdiff-backup will use the
current time zone), or ordinary dates like 2/4/1997 or 2001-04-23
(various combinations are acceptable, but the month always precedes
the day).

So I switched the lowercase w to an uppercase W and the command actually ran fine and deleted over a month of incremental backups.

I checked in man rdiff-backup and found an option --include-symbolic-links but it seems to run anyway without that, and maybe that would be to include symlinks in the backups and not anything to do with pruning, not much info on it in the manual.

So now how can I prune the archives and log files? Do I just have to run a command outside of MineOS or is there a built in way to use so I can easily do it on all the servers?

Thank you for your help! Sent a little donation.

[hexparrot]

Hi Ryan,

Thanks for your donation; it means a lot to me to receive support for my
efforts into Minecraft and MineOS. I hope I can continue to provide a
great product and great support going forward.

That said, as far as your pruning desires:

The web-ui handles pruning of backups and archives quite well--have you
tried that yet? While it is definitely possible to automate it with
scripts as you are (using mineos_console.py), unless you think there's a
pressing need to have automated pruning, it could be easy to just hop on
the web-ui once in a while and see visually what you'd prune.

For example, I just made a dummy server and 4 archives in succession.

On the archives page (and on the backup page) there is a text box which
allows you to type in a date. For ease, it helps autocomplete based on
anything: "Fri", "May" "20:02"... any of these terms will autopopulate a
specific point-in-time for which you can remove all archives older than
that one (excludes one typed in).

Type in a date (let the autocomplete finish it for you) and hit enter, and
the 'prune archives' red confirmation box will show up. You'll see how
much you're reclaiming in space and how many files will be deleted.

Unfortunately, while mineos_console.py does have a prune feature with
rdiff-backup syntax, no such thing exists for Archives. This is why it's
preferred (if possible) to use the web-ui and see it visually.

Unlike backups, however, the file structure is very straightforward:
they're all timestamped and appropriately named tar.gz files. This means
you can use normal bash commands to find all files older than x age and
delete them. Such an example is:
http://www.howtogeek.com/howto/ubuntu/delete-files-older-than-x-days-on-linux/

find /var/games/minecraft/archives/servername -mtime +7 -exec rm {} ;

There's downsides to bash, though; which is if you do something wrong....it
could delete more than you want. If you're feeling confident in your
bash-fu, have at it! Elsewise, I hope the web-ui helps you prune off your
old snapshots and archives just as easily.

[ghost]

Thanks! I have no problem doing the bash prune cron. I was just wondering if there was an easier, nicer, more preferred way to prune the archives and logs on all servers through MineOS in a similar way.

The web interface is quite nice but some people tend to forget about the backups filling up the disk space, especially since in my case the disk space indicated in MineOS is not the correct disk space, possibly due to being installed in /home on a different partition than the root or something.

If they (the people whose servers they are) create new MC servers or forget to prune the backups regularly then bad stuff may happen. But maybe I can just make a script to loop through all the directories inside the servers directory and clean the archives and logs that way.

Thanks again =)

[kaimi]

I'm also wondering if there's a way to prune old log files.

Logrotate is your friend. It supports globbing, so you could let it rotate all files in /<path>/<to>/<servers>/*/server.log. The man page has some examples, or take a look at the files in /etc/logrotate.d.

I checked in man rdiff-backup and found an option --include-symbolic-links

--include-symbolic-links does exactly what it says, it includes any symbolic links in the source directory. It’s on by default. You can use it to include symbolic links even if you did specify --exclude-special-files, for example.

There's downsides to bash, though; which is if you do something wrong....it
could delete more than you want.

You mean something like this? :)