GSoC-GitLab-2021
GSoC-GitLab-2021

I improved backup and restore process for GitLab as a part of Google Summer Of Code 2021 - @imskr.

The backup and restore tools provided by GitLab are relied upon by administrators of GitLab instances as part of their disaster recovery strategy. These procedures rely on standard UNIX tools such as rsync and tar and cover most data excluding GitLab configuration.

There are a number of ways that we can continue to improve the backup and restore processes to make the experience more reliable, efficient, and transparent for system administrators.

Let’s build…

Backup and restore output logs

Log the gitlab:backup:create and gitlab:backup:restore output to .log file.

These logs would come in handy especially for GitLab Support while assisiting customers (and also sys admins as well), especially when debugging for issues with these operations.

Creating a BackupLogger Class from Gitlab::JsonLogger inside backup_logger.rb.

# frozen_string_literal: true
module Gitlab
  class BackupLogger < Gitlab::JsonLogger
    def self.file_name_noext
      'backup_json'
    end
  end
end

Using it inside backup rake which STDOUT the output and saves to .log file also.

Remove backups/tmp directory after restore

Executing bundle exec rake gitlab:backup:restore restores the backup but it also contains tmp dir resulting in disk-full alert.

Created a remove_tmp function inside lib/backup/manager.rb:

def remove_tmp
  # delete tmp inside backups
  progress.print "Deleting backups/tmp ... "

  if FileUtils.rm_rf(File.join(backup_path, "tmp"))
    progress.puts "done".color(:green)
  else
    puts "deleting backups/tmp failed".color(:red)
  end
end

And calling it in backup rake.

Before fix:

backups/tmp directory still present
backups/tmp directory still present

After fix:

Restore deletes backups/tmp
Restore deletes backups/tmp

Done (During GSoC):

  • Add backup/restore logs:
    • Added json logger for backup and restore output - MR !63832.

  • Delete tmp after gitlab:backup:restore:
    • Deletes backups/tmp after restore process - MR !67741.

  • Allow failed gitlab backup to email admin:
    • Ported email admin notification from EE to core - MR !64931.
    • Added error classes for specific backup rake tasks - MR !66260.

To Do (Plan post GSoC):

  • Rescue custom backup errors to notify admin - MR !66790.
  • Add option to disable compression of backup - #17197.

Acknowledgement

  • I would like to thank my mentors Gabriel Mazetto and Aakriti Gupta for helping and guiding me in the project throughout the gsoc journey and making it an awesome experience for me.

  • I am thankful to Google Summer Of Code for providing me an opportunity to work at GitLab with awesome people.