5 Backup Server Recovery Scenarios

Imagine a disaster, in which the virtual infrastructure has to be restored from scratch.

All is lost except the backup files, which are still available on at least one repository, preferably immutable, on-premises or in the cloud.

To restore the environment you have five different options that are a function of how the Veeam Backup & Replication protection and resilience architecture is designed and implemented.

Note 1: Veeam Backup & Replication will hereafter be referred to as. VBR.

Scenario A (Restoration from scratch):

  • You have not made an application-aware backup of the server VBR.
  • You have not performed application aware replication of the server VBR.
  • Export of server DB configuration is not available VBR.
  • You want to restore production workloads immediately.

What to do?

Step 1A: Install Veeam Backup & Replication.

How to: From Veeam’s website(https://www.veeam.com) download the latest version of VBR.

(direct link -> https://www.veeam.com/products/data-platform-trial-download.html?tab=cloud-plugins).

Picture 1

Note 2: VBR can read Backup files created with earlier versions.

The simplicity of installing Veeam Backup & Replication makes it easy, fast, and can be done in unattended mode.

(https://helpcenter.veeam.com/docs/backup/vsphere/silent_mode.html?zoom_highlight=Unattended&ver=120)

In this step, just clicking “next” to complete the operation is often sufficient.

Note 3: It is recommended that you use your license file (which can be downloaded from my.veeam.com) although the Community Edition (unlicensed) is often sufficient for most restores needed at this stage.

Step 2A: Add the production virtual infrastructure where you want to restore workloads protected by VBR.

How: after completing the first step, from the console of VBR add the virtual infrastructure (Menu: “Inventory” -> “Vmware vSphere“-> “Add Server“) (Image 2).

picture 2

The following steps depend on the type of Hypervisor (VMware vSphere, Microsoft Hyper-V, Nutanix AHV, …) but are always very simple.

Step 3A (optional): Add backup proxies.

Even if we are operating at the recovery level, improving the performance by adding backup proxies is always a good idea.

Step 4A: Adding Veeam backup repositories.

The last preparatory step before starting restores is to add the repositories with the backup data.

How: from the console, select “Backup Infrastructure,” “Backup Repository,” and then “Add Repository” (image 3).

Picture 3

Step 5A: Starting the restore

How: from the console of VBR select “Home,” “Backup,” “Disk imported,” the VM you want to restore, and right-click to start the restore process (Image 4).

Picture 4

Note 4: Recovery can be instantaneous. With this mode, VMs are started directly from the backup repository. In this option, the repository serves as the data store (for VMware the DataStore) for the virtual environment.

(Instant VM recovery was invented by Veeam more than a decade ago and has since improved its performance and flexibility.)

Now your production architecture is back up and running!

https://www.veeam.com/blog/restoring-infrastructure-from-scratch-with-veeam.html

Scenario B: The VBR is a virtual server.

  • You have made an application-aware backup of the server VBR.
  • You have not performed application-aware replication of the server VBR.
  • Export of server DB configuration is not available VBR.
  • You want to restore the server immediately VBR.

What to do?

Step 1B: Make a download of the utility “Veeam.Backup.Extractor.exe” from Veeam’s Download site.

(https://www.veeam.com/products/data-platform-trial-download.html?tab=extensions).

Picture 5

Note 5: There is also a command-line Extract option for Windows and Linux platforms.

Step 2B: Start Extract, select the backup of the VBR, and once the files of the VM-VBR are copied them to your preferred VMware datastore.

Now from the vCenter register the VM you just copied.

(Image 6)

Note 6: A command-line extract option is available for Windows and Linux platforms.

Note 7: It is possible to automate and simplify copying to the VMware Datastore by publishing an NFS network share as mentioned in the following article:

https://www.virtualtothecore.com/veeam-extract-utility-quick-restores-without-veeam-server

Step 3B: After completing the recovery of step 2B, start the VBR and carry out the standard operations of use (see Step 5A).

Scenario C: The VBR is a physical server

  • You have made the application aware backup of VBR by creating the recovery media.
  • Export of the configuration file of (VBR).
  • You want to restore the server immediately VBR.

What to do?

Step 1C: Make the recovery media available to the Physical Server VBR (via Network or USB).

Step 2C: Start the Bare Metal Recovery operation by selecting the necessary backup (image 7 and image 8) in the recovery step.

Picture 7

Image 8

(https://helpcenter.veeam.com/docs/agentforwindows/userguide/howto_baremetal_recovery.html?ver=60)

Step 3C: After completing the recovery of step 2C, start the VBR and carry out the standard recovery operations as indicated in Step 5A.

Scenario D: The VBR is a replicated VM.

  • You have not made an application-aware backup of the server VBR.
  • You have performed application-aware replication of the server VBR.
  • Export of server DB configuration is not available VBR.
  • You want to restore the server immediately VBR.

What to do?

Step 1D: Connect to the vCenter and search for the VBR already replicated.

Image 9

Step 2D: Trigger the failover of the VBR.

Image 10

Step 3D: Realize the management operations of VBR as per Step 5A.

Scenario E: The configuration of the VBR.

  • You have not backed up the server VBR.
  • You have not performed Server Replication VBR.
  • Export of the server DB configuration is available VBR.
  • You want to restore the server immediately VBR.

What to do?

Step 1E: Install VBR on the server (physical or virtual, see step 1A).

Step 2D: Perform a configuration reset of the VBR as indicated in the guide.

(https://helpcenter.veeam.com/docs/backup/vsphere/vbr_config_restore.html?zoom_highlight=configuration+restore&ver=120)

Image 11

Step 3D: Realize the management operations of VBR as per Step 5A.

Note 8: It is always a good idea to save the Backup server configuration.

Final Note: The advice is to strive to be able to use all the strategies described in this article so that if one is not available, a second one can be used.

NAS Backup – GFS to TAPE – Part II

In the previous article, we saw how to operate on backup jobs to obtain Fulls that can be used to create a GFS retention policy when the destination of the jobs is a tape.

In this second article, we find out how a similar result can be achieved by copying tapes.

Note1: A second tape library must be present in the DataCenter to pursue this protection process.

Note2: The most common use case for Copy-Tape is to migrate data contained on tapes from an old technology (LT06) to a new one (LTO9), since the new technology would not be able to natively read the data contained on the old tapes.

There are two steps that will enable us to achieve our goal:

  • Step 1: Creation of a tape pool afferent to the second library.
  • Step 2: Tape copy job.

Stage 1

The creation of the Media Pool (image 1), will need to be customized by setting:

    • The use of a new tape for each copy session (image 2).
    • Setting a retention that for that tape group coincides with that required by the GFS policy (image 3).

Picture 1

picture 2

Picture 3

Note3: A 4-week retention was set in Image 3, which addresses the need to keep the full weekly for 1 month.

Note4: Image 4 highlights the possibility of implementing a Vault policy for tape storage.

Picture 4

PHASE 2

From the VBR GUI by selecting the tape to be copied with the right mouse button (image 5), the copy command can be initiated.

Picture 5

The simple next steps shown by images 6,7,8 and 9 show how to complete the copying operation.

Picture 6

Picture 7

Image 8

Image 9

Latest notes:

  • Documentation to refer to in order to know how many resources it is essential to allocate to the various components is available at the following link.
  • Automation of copying can be done through scripts in powershell.
  • Copy to Tape does not consume capacitive licensing but refer to the following link, Capacity Licensing item to know all the details.

NAS backup – GFS to Tape – Part I

Many customers and partners ask whether it is possible to implement a GFS (Grandfather – Father – Son) type of protection policy when the data to be protected pertains to a NAS (Network network-attached storage) and the destination is a tape library.

Such automation with the current version of Veeam Backup & Replication(VBR) 12.1 is not yet available, something that is already possible when the data source is a backup of VMs and Physical Servers.

In this first article, I will help you achieve that goal by taking advantage of VBR ‘s great flexibility in creating backup jobs.

Note1: In the next one I will illustrate how to make GFS copies by exploiting a little-known feature of
VBR
, the Tape Copy.

Flexibility of Backup Jobs:

a. VBR manages tapes using an architecture that is based on:

  • Media Pool(MP) are the logical containers of the tapes and can pertain to one or more Backup jobs (in our scenario we will create one MP per Job).
  • Media Set(MS) identifies the restore points present on the tape (in our scenario we will create one MS per Backup job per single tape).

b. The proposed solution is to create weekly, monthly, and annual backup jobs in full mode. These backups should be created on a specific date and the backups should reside on tape pools created for the purpose.

Let’s see step by step how to proceed:

c. Creation of weekly and monthly Media Pools(MP).

Picture 1

From image 2 it is important to note that a new tape will be used for each backup session.

picture 2

Image 3 shows how to set the retention, which in this scenario is 4 weeks.

Picture 3

For the Monthly MP, the same procedure is used, changing the retention to 12 months (see images 4,5,6).

Picture 4

Picture 5

Image 6 shows that the retention for Full Months is 12 months.

Picture 6

d. Creating Backup Jobs

Picture 7

image 8

Image 9 highlights the scheduling of the Backup job.

The assumption is to make n full backup jobs for each GFS policy.

Our example scenario shows the first week’s job (blue arrow) with weekly retention (green arrow). For the second, third, and subsequent week, we will proceed in a completely similar way, replacing the value first with second, third, etc. under “Run the full backup automatically.”

Image 9

Image 10 highlights (orange arrow) that no incremental backups will be initiated.

image 10

The same steps must be implemented to create monthly type GFS backups, in the example I set the backup job start on the 4th Saturday of the month (image 12 – blue arrow).

Image 11

Image 12

Image 13

Note 2:

  • Licensing counts licenses per individual Backup job (verision 12.1).
  • Conduct tests to make sure the scenario matches your needs. Get help from Veeam support.

In the next article, we will see how to use the Tape Copy feature.

XFS – Resize the immutable file system

In the Veeam Backup & Replication environment, it may be necessary to expand the allocated space of a Linux repository.

In my environment, there is an Ubuntu 22.04 server to which a second disk(dev/sdb) was added, formatted as xfs, and made available as mount point /mnt/backup/ .

The server is used in hardened repository mode (immutability)
(https://helpcenter.veeam.com/docs/backup/vsphere/hardened_repository.html?ver=120).

Let’s look at the simple procedure:

  • The packages to install are cloud-guest-utils and gdisk:
    “sudo apt -y install cloud-guest-utils gdisk”
  • To find out the structure of the file system use the command:
    “sudo lsblk”

      • The result shows the sizing, and mount point of Ubuntu server file system:
        NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
        sda 8:0 0 16G 0 disk
        ├─sda1 8:1 0 1M 0 part
        ├─sda2 8:2 0 1.8G 0 part /boot
        └─sda3 8:3 0 14.2G 0 part
        └─ubuntu–vg-ubuntu–lv 253:0 0 10G 0 lvm /
        sdb 8:16 0 100G 0 disk. └─sdb1 8:17 0 80G 0 part /mnt/backup
        sr0 11:0 1 1024M 0 rom
  • To find out if the file system has additional space to allocate:
    “sudo growpart /dev/sdb 1”

    • The result shows the item changed
      CHANGED: partition=1 start=2048 old: size=167770079 end=167772126 new: size=209713119 end=209715166
  • The final command that widens the file system is: sudo “xfs_growfs /mnt/backup/”
  • Check the result through the command already seen: sudo lsblk”

Veeam + ReFS: How much space you save

ReFS is the advanced file system from Microsoft that improves data availability through technologies that can:

  1. Ensuring greater resilience of data stored on the file system.
  2. Increase the performance in reading and writing.
  3. Improve the scalability (we are talking about millions of TB).

One of the most useful and widely used features in backup is the technology of Block-Cloning which allows Veeam Backup & Replication to create full backups equal in size to an incremental.

The operation logic is simple and consists of 3 phases:

  1. TheBackup copies to the target Repository (ReFS), the incremental data of the VM / Instances / Physical Servers/ Clients To be protected.
  2. The File System ReFS will take care of storing the new blocks and creating the metadatarelated to the newly written data.
  3. The option “create a Syntethic-full” actually triggers anoperation at the level of metadata. ReFS adds to the metadata just created, those related to previous backups, thus creating a new full child of the union of all the necessary metadata. To further simplify, a logical full is created without any block being copied/moved.

Note 1: The result is not only a saving in space but also in the time it takes to make the full.

Well, how is it possible to quantify the disk space saved in the repository (ReFS)?

Timothy DeWin has made a tool (blockstat.exe) perfect for this calculation, to which I refer you for all possible options.

In my case, I solved the client’s need through:

  1. Creation through powershell of a text file (Unicode format) that would search all the Backup files generated by Veeam Backup & Replication within the ReFS repository. (See image 1)
  2. Captured the output of the bloclstat command. (see image 2)

Picture 1

picture 2

Tips VMware – Module MonitorLoop power on failed

During laboratory maintenance operations, suddenly a Virtual Machine was no longer able to start.

The vCENTER console reported an error in initializing the server swap file.

Like any good system engineer, before making any changes to the environment, I tried to back up the aforementioned VM.

The job stopped due to the following error: (” An error occurred while taking a snapshot: Invalid change tracker error code “).

Troubleshooting:

  1. Since the swap file handles memory over-commitment, I tried to change the allocated amount of RAM.
  2. I added space to the Datastore on which the VM resided to make sure VMware had enough space to manage the swap file.
  3. I searched in the configuration file ( vmx ) for differences with respect to the configuration of the other VMs.

All tests and changes made did not solve the problem.

Aware that I would have to change the VM configuration, I implemented a simple strategy to:

  • Backup the VM through the Veeam Agent for Linux (The VAL operates at the Guest-OS level and not at the hypervisor level).
  • Write down all the changes that I would have made to the VMs (editor’s note: I had worn Hop-o’-My-Thumb‘s hat, that is, able to return to the initial configuration in a short time).

The methodical ” change, note, check and turn on” approach allowed me to discover that the problem was related to the CPU configuration of the Virtual Machine.

In fact, by resetting the ” CPU reservation ” values to Zero and ” CPU share” to Normal (see image 1), the problem went away, allowing me to start the VM and back it up.

Sapiens nihil affirmat quod non probet (A wise man says nothing that he cannot prove)

Picture 1