Articoli

Pubblicato il

A flexible file backup Strategy – Part 3

This article’s topic is how to set up and work VBR when it is combined with the NDMP protocol.

This type of configuration is part of the category “file to tape” I treated in my last article “A flexible file backup Strategy – Part 2”.

One more note before starting: VBR requires NDMP version 4 and later.

  • How does it work?

Picture 1

The architecture is quite easy.

Files are gathered from the File Server through the data mover present on the Gateway server. Then they are sent to the Tape Server that performs the write on Tape.

The Datamover installed on Tape Server has the ownership of managing the data traffic to Tape Device.

VBR has the task of enumerating the Volumes and launch the command to Tape Devices.

Point-1: VBR works with a 3 layer backup architecture. It means there is no direct connection between File Servers and Tape Devices. All Data have to pass through the tape Sever.

In this article, I’m not covering the procedure to set up the NAS correctly. Please ask your storage vendor specialist to get all details needed.

How to add the NDMP server to VBR and set up a backup job?

The next video will answer the question.

Video 1

After the backup is completed how to perform a restore?

NDMP backup files are available to perform restores from the FILE menu at the voice Tape and then NDMP as shown in picture 2

Picture 2

 

Please have a look at the next short video (video 2) to see the easy step to perform a restore.

Video 2

Point-2: The backup chain stored on tapes will consist of 10 restore points maximum. On the 11th run, VBR will force an active full.

  • Common scenarios

One of the main characteristics of NDMP backup is that it allows to perform backup and restore of entire volumes.

The most common scenario is the Disaster Recovery of the entire Filer. Imagine the case in which the customer NAS filer is completely out of order and the only way to restart the file sharing service is formatting all the disks and then restoring the volumes (or the worst case is getting a new NAS).

In this case, NDMP volume restores from tape is an excellent low-cost solution.

Main Pro:

NDMP servers backup to tape is available in the Enterprise Plus Edition of VBR.

It means that there is no limit on the amount of data that can be saved. Could be 1 TB or 100 PB it doesn’t matter.

Veeam Universal License (VUL) has the Enterprise Plus Edition available. Is it time to move your old sockets license to the subscription model (VUL)?

Cons:

The granular restore (files and folders) it’s not an available option with VBR and NDMP backup.

Please note that NAS backup is now completely supported by VBR (please refer to article 1 of this series) and it’s the fastest way to perform the granular restores.

Point-3: Even if the NAS device supporting NDMP protocol is already added to VBR, you need to add the NDMP server as a separate procedure. Otherwise, you will not be able to perform file backup to tape.

The next article will cover a recap of the different techniques and show a hidden gem of VBR.

See you next week and take care

Pubblicato il

A flexible file backup Strategy – Part 2

In this second article, we are going to cover the File to Tape strategy.

Why tape devices are still widely present in the IT department?

  • It’s a good way (but not the only one) to manage the offline backup data (read it as improving the Security Strategy of your data).
  •  Media can be easily carried or moved (read it as Portability).
  • Deployment is often very quickly (read it as speeding up the adoption).
  • It has a potentially infinite capacity (Just adding media).
  • The LTO is a neverending technology in a continued evolution.
  • The tape is a well-known device, IT operators have the skills to manage it.
  • The costs for GB is lower than disk technologies.
  • The costs are quite predictable, managers can budget it easily.

VBR needs a Windows Physical Server named Tape Server to control the Drives and Robotic, LTO3 or later Drives, and MS-Windows drivers (supply by the hardware vendor).

The official user guide available on the Veeam site gives all detailed info.

Just a note before starting:

VBR uses Tape Technology in two different ways.

The most used one is back up to tape (Picture 1).

In this case,  the source backup data are the backups already present and created with a backup job or backup copy job.

They are saved to Repository (Repository is a Disk technology).

It means that the scope of backup to tape is to pour out data to tape.

Picture 1

Please have a look at the following video (https://www.youtube.com/watch?v=Il8mH2KB_Uo) to get more details.

The second way is File to Tape and it is the topic of this article (picture 2).

https://lnx.gable.it/wp-content/uploads/2021/01/nas-7.jpgPicture 2

Which type of source files can be saved to tape?

  • Windows & Linux servers (virtual or physical doesn’t matter)
  • NAS file share (SMB (CIFS) and NFS ).
  • NDMP filers (it will be covered in the next article).
  • How does it work?

Picture 3

Picture 3 shows the data streams when a tape process is performed:

  1. The main components are Data Movers. These Services run on the source and on the Tape Server.
  2. VBR triggers the source Data Mover to perform a copy of the files to the target. At the destination, the target Data Mover check if the files have arrived correctly.
  3. The tape Server manages the write operation to the tape also.
  4. VBR stores all info about files saved (media used, retention, etc.)  in a catalog.
  5. In the restore scenario, the step order is four to one.
  • *Note: To perform a backup of Windows and Linux servers, it is requested to add those servers to the managed server as shown in picture 3. Through this process, the Data mover service is properly installed.
  • Network Share: Adding SMB/NFS Share as shown in the previous article (A Flexible file backup strategy – Part 1).
Picture 3
  • Common scenarios

File to Tape backup can be used by any customer. You need just a Tape Server, Tape Devices, Drivers, and VBR.

There are at least two main cases:

  • Customers who want a copy of their data to tape.
  • Customers with a small budget who doesn’t need rapid restore

The next video will show how to set it up.

Main Pro

  • There is not a room limit. It means the license doesn’t count how many GB, TB, PB will be written to Tape.
  • The VBR architecture is as usual flexible. It’s possible to add more tape servers and more than 1 tape library.

Version 11 will add more great features:

  • Tape cloning (https://community.veeam.com/blogs-and-podcasts-57/tape-improvements-in-vbr-v11-277)
  • Tape verification (https://community.veeam.com/blogs-and-podcasts-57/part-ii-tape-improvements-in-vbr-v11-289)

Cons

*This behavior is quite common to all backup software that writes data directly to Tape.

  • For saving a file, VBR needs to trigger a process of discovering the file to the source, gathering and writing it to a media.

If you consider that the common NAS scenario is composed of millions of small files and thousands of folders and that the tape technology has to choose for every file the location in the media (where the file will be copied)  it’s clear that this process, common to all backup servers, stresses the hardware architecture and in particular the drive header.

The backup process has a small speed advantage compared to restoring because writings to media are often sequential and not random.

Image to restore 10k files located in 10k different positions in a single tape.

The drive has to perform a great job. It is going to suffer from an effect called shoe-shining (also known as tape back-hitching)  which occurs when a tape drive cannot transfer data at an acceptable speed.

Shoe shining can contribute to data loss over time, as the repeated back-and-forth motion will wear the tape drive’s read/write heads and negatively affect the readable portion of the tape

  • Loss of Tape Cartridge Capacity
  • Increased Risk of Read/Write Issues
  • Excessively Worn Tape Drive Heads
  • Low Data Transfer Rates
  • Data Loss
  • The Veeam DB needs to be sized correctly and the best practice is to switch from SQL Express to SQL Standard
  • Media management is quite challenging when the amount of tapes is big. Remember to store them in a fireproof and non-magnetic safe.

Do you also prefer the NAS backup feature introduced in v.10? Let me know!

That’s all for now.  

See you next week for talking about NDMP

Pubblicato il

A flexible file backup Strategy – Part 1

As many of you know, one of the biggest innovations introduced in VBR version 10 is the support of NAS backup.

Does this mean that it was not possible to save the unstructured file data before?

Actually not, more than one options were already present.

The scope of the next articles is to show when and how to use those technologies to answer customer needs.

In all the cases the product used is the powerful VBR.

The four main topics are:

  1. NAS Backup
  2. File Backup to Tape
  3. NDMP
  4. File Backup to Disk

For each of the above-mentioned items, the articles will show:

  • How it works
  • Common requests and scenarios.
  • Technology, Pro & Cons.

Let’s start!

1- NAS Backup

  • How does it work?

Veeam mantra is “innovate“!  A clear example is represented in the NAS technology.

The primary idea on which this technology is based is to track the changing of the unstructured files.

Let’s clear it up with a comparison: I’m quite sure all VBR users know the CBT (change block tracking) technology strongly used by VBR to create backups of VMs. It allows saving data blocks that have been changed from the previous backup.

The ingenious idea of Veeam R&D is to use this approach when files and folders must be saved.

I called it  FCT (File change Tracking).

How does it work?

When a NAS backup is performed, VBR calculates on-fly the CRC (Cyclic redundancy check) of any single file that has to be saved. Those metadata are stored in the cache repository (points three and four of Picture 1).

Picture 1

Why is this pre-process so important? Because thx to it it’s possible to:

  • Perform incremental backup forever (only new and changed files are saved). It means a shorter backup window.
  • Speed up the restore phase; image the scenario where a customer has 5 PB of data and luckily just 1 TB of data has been attacked by a Virus or accidentally deleted by a script.
    The IT manager will ask to restore just 1 TB and NOT all PBs.

VBR using the “FCT” can understand which files have been changed /deleted with respect to a specific Restore point and restore just those needed.

This great option is called “rollback to a point in time“.

Just as a reminder, there are two more ways to restore data, Entire file share and single file and folders.

  • Common scenarios

NAS backup can be used by any customer. You need just a repository and a valid license (VUL).

The scenario I like to talk about is where the customer has big fillers in his environment.

Why?

Because it is possible to leverage the storage snapshot to gather files as shown in picture 2.

Version 11 will have improvements in this area too. Stay tuned by signing up to Veeam site (https://go.veeam.com/v11).

Picture 2

Leveraging the storage snapshot a customer can

1. Speed up the backup process.

2. Save files though they are in use (Open files can’t be saved by a backup process while they are processed by users).

This integration allows performing backups to any hour of the working day without any attention to the status of the file.

Main Pro

a -The architecture is very scalable because it leverages the concept of proxy very common to VBR.

Proxies are the data mover that collect data from the source and send them to the Repository. The File Proxy has also the responsibility to calculate the FCT.  You can add more proxies when you need to address the backup of big amounts of data.

b- The files saved to the Repository are written in a customize format. They are managed as an object in a vBLOB Storage and contain the metadata of every single file saved (they contain info about which folder the file belongs to and which are the file rights also).

Pictures 2 and 4 show the new format of the backup file for NAS.

Picture 3

Picture 4

The main advantage is that all file restore tasks are very very fast!

c- It’s possible to copy backup data to the secondary repository setting different retentions. It allows answering the common request to have a copy of backup data in another location (3-2-1 rule).

d- It’s possible to create an archiving backup file policy through the object storage VBR integration. Picture 5 (taken from the VBR user guide) shows the main repositories option available with NAS Backup.

Picture 5

Cons

It doesn’t support the transfer to Tape Devices. Please read the article about NDMP and File to Tape to get an interesting solution.

That’s all for now.

See you in a couple of days for File to Tape Backup

Pubblicato il

NUC upgrade to ESXi 7.0.1 – Part 2

Phase 2: Upgrading ESXi Host

In the previous article, I described how to prepare a customized ISO. If you lost it please click on this link.

Let’s continue!

Requirements: the task I performed before starting the procedure here described has been the vCenter upgrade to 7.0.1-c. I remind you that my starting point was Esxi 7.0.0.b that is fully supported by vCenter 7.0.1-c.

There are two main ways to upgrade the ESXi Hosts.

The first is related to the use of a VMware feature. It’s the vSphere Lifecycle Manager (vLCM) and you can taste part of its potential by watching this video guide.

Instead, I preferred to use an old approach working with a bootable USB pen with the custom ISO prepared in the previous article. The steps are:

1. Shutdown ESXi Hosts
2. Remove the USB-NIC
3. Insert the Pen Drive with custom ISO
4. Upgrade the host following the wizard (The main point is shown in Picture 1)

Picture 1

5. Reboot the host.

6. Now it’s time to install the USB-Nic Driver. It is available from “USB Network Native Driver for ESXi” web site and this article allows me to say thx to the excellent job of  Songtao Zheng and William Lam

Before proceeding please read the instruction carefully and DO NOT insert all the USB cards together. (I have got three USB-NIC cards)

Why? Because during the procedure, I have had more than one purple screen and after a deep analysis I discovered that it depended on the USB-NIC cards.

To get over this issue I created the following procedure.

Picture 2

7. Switch off the ESXi NUC and insert the first USB-NIC card in port-1 and the second in port-4 (please refer to picture 2 to know the nomenclature of the port)

8. Now switch on the NUC and check if it boots correctly.

9. Switch off the NUC and insert the third USB-NUC on port-2.

10. Reboot NUC and check if it works as aspected.

Before ending this article I suggest creating a map between vmks and the physical MAC Address of the USB NIC. The main advantage is that it allows maintaining the same vmkusb address after a reboot also.

Some useful commands:

To Identify the Mac Address:
# esxcli network nic list |grep vusb |awk ‘{print $1, $8}’

To Check persisting binding:
# esxcli system module parameters list -m vmkusb_nic_fling

Pubblicato il

NUC upgrade to ESXi 7.0.1- Part 1

Last weekend I upgraded all my Servers to the last VMware ESXi release (7.0.1 C) and this article is meant to describe all steps I performed.

Just a recommendation before starting. I worked in my lab that it’s not a production environment.

MyLAB before upgrade:

  1. NUC8i7beh2
  2. VMware ESXi 7.0.0 (Build 16324942)
  3. Every NUC has three more network cards added to the embedded standard NIC. They have been obtained through the USB ports leveraging three adapter USB/Ethernet and the flings driver.  Please refer to the FLINGS website to get all info.

The procedure is composed of two main phases and this article will cover the first part.

Phase 1: Creating a customize ISO

Is this step required?

Oh well, it depends if the Standard ESXi VMware ISO has already the driver of your embedded network card inside. The standard ISO, unfortunately, does not contain the NUC8i7BEH network drive (it is named ne1000)

If you upgrade the ESXi through the standard ISO, the process fails with the error shown in picture 1.

Picture 1

How to get over it?

Before upgrading it, it’s necessary to know the driver used by the embedded NIC Card. If you don’t know it, please read the next instructions carefully (they are command launched on host ESXi you are going to upgrade):

1.   lspci -v | grep -A1 -i ethernet
take a note of the string composed of 4:4 values ( xxxx:yyyy)

2.   lspci -n | grep xxx:yyy
take a note of how the nic is named (in my case [vmnic0])

The next step is getting the name of the driver directly from the VMware website (Matrix compatibility).

From that web page, filling up the empty field with the value yyyy and filtering the result by IO Devices, it’s possible to get the device driver name.

For my LAB the result is shown in picture 2 where I highlighted the device driver name in yellow.

Picture 2

The last command to check if the driver is already installed (It should be present) is:

3.   vmkload_mod -l | grep “Device Driver”

In my case: vmkload_mod -l | grep ne1000
                          ne1000          1          352

Optional: if you use the USB ports to add more NIC, please uninstall the fling drivers before proceeding.

4.   esxcli software vib remove  –vibname=vmkusb-nic-fling (before vibname two scores –   –  )

It’s time to create our custom ISO

a- Download the offline bundle from VMware Site, for example:

VMware- ESXi-7.0U1c-17325551-depot.zip

b- Download the NUC ethernet driver for your device (ne1000 in my case).

I found an useful PowerShell script to get it:

#add the software repository
Add-EsxSoftwareDepot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
#define as variable the name of the driver
$vib = Get-EsxSoftwarePackage ne1000
$vib | select Name,Version | ft -AutoSize
$vib[4] | fl
#Get the driver
Invoke-WebRequest $vib[4].SourceUrls[0] -OutFile d:\pkg\ne1000_0.8.4-10vmw.700.1.0.15843807.vib

c- The PowerShell script to create a custom ISO is available on VMware Front Experience Site.

This great script has a lot of options; please refer to the official documentation to see how to create the ISO.

In my case I just launched the following command:

.\ESXi-Customizer-PS-v2.6.0.ps1 -v701 -izip D:\ISO\ESXi-7-0-1\VMware-ESXi-7.0U1c-17325551-depot.zip -pkgDir D:\pkg\ -OutDir D:\ISO\ESXi-7-0-1\ -nsc

d- The last step is creating a bootable USB pen using the just created custom ISO as a source.

I have chosen Rufus to perform this task.

In the next article, we are going to see the final step to upgrade the NUC

Pubblicato il

Ransomware defense part 4: Deep Dive

In this last article about Security and Ransomware, I’m going to add new features and deep dive those you’ve already read in my first article about Veeam Backup & Replication.

The starting point is creating copies of your data (remember the 3-2-1 rule) and from these copies perform automatic tests of availability and security.

Which technology should be deployed to sleep safer?
In this article, I’m going to show you some Veeam technologies that address the threats explaining why they are a must to be used.

Sure-Backup

It is the best way to be certain that a backup is really usable.

What does it mean exactly?

In my public speeches, I often use a pen to explain the Sure-Backup concept correctly.

How can I be sure that the pen is usable? The answer is very easy. If it writes on paper it works and it is usable.

Going back to our scenario,  the only way to know if your backup is usable is to perform a restore in an environment logically separate from the production.

How Sure-Backup works?

It is composed of two parts:
The application group identifies the backup VMs that have to be verified.
DataLab is the way through which VMs are switched-on into a specific network that can’t communicate with the production network.

This great technology allows you to add the number zero meaning no errors (from 3-2-1 to 3-2-1-0) to the above rule.

One more important thing to add here is that you can create scripts to random test your backup.

In the following article by Luca Dell’Oca, you can have an excellent example of how scripts can help you.

How can you test 1000 VMs with Veeam SureBackup?

Sure-Replica:

VBR is a solution that allows customers to perform Backup and Replicas of your VMs.
Replicas are commonly deployed to create a Disaster Recovery Site.

Veeam has “integrated” the Sure-Backup technology for Replica too. It is named Sure-Replica and it gives all advantages shown for Sure-Backup

For managing the automation of Replicas I suggest customers use a new Veeam product, the Veeam Availability Orchestrator (VAO).

On this site, you can find a very detailed guide to use set-up VAO.

https://lnx.gable.it/home-page/2020/08/07/veeam-availability-orchestrator-v-3-0-vao-baseline-1/

Secure Restore

It scans the saved  VM Windows  (VM that has at least one valid restore point), with your antivirus software before restoring the VM to the production environment.

Secure Restore is available for the following restore tasks:

  • Instant VM Recovery
  • Entire VM Restore
  • Virtual Disks Restore

The only requirement is that your antivirus has to be installed on the mount server and supports CLI.

Data Integration API

Veeam Data Integration API is a set of Veeam PowerShell cmdlets that allow you to represent data of backup files as a mounted Windows folder.

This feature allows you to access backed-up data in read-only mode.

It has a lot of possible usages to example data mining and data warehouse.

In the field of security, it can be adopted to check if a virus is already present in your VM checking via backup files the guest OS files through your antivirus software (It scans the files of the VMs and not the backup file). A good example of use is in the following blog article by Niels Engelen

https://www.veeam.com/blog/v10-data-integration-api.html

Storage Integration

VBR has two different storage integration.

The primary storage integration allows customers to perform backup more frequently because this technology allows creating backup without impacting the VMware environment.
The integration allows creating a test environment because it is integrated with Datalab and Sure technologies.

The secondary storage integration is commonly used with deduplication appliances that can allow you another layer of security for ransomware.

When the Veeam Data Mover Service is installed directly on the appliance, it reduces the risk of ransomware encrypting and deleting data.

In addition, if for some reason the first data is compromised, copies can be replicated to a second data center or in the cloud.

Before leaving you my two last cents:
1) The starting point is always to perform copies of your data. Veeam gives you powerful tools to manage them and to perform more check to verify that your data are safe from threats.

2) Some customers still think that the backup asset is just a cost because they are used just to restore. With Veeam you can use backup data to perform different actions relieving the production of not business core tasks (for example think the great use of Data API Integration for data mining, data warehouse etc)

Take care guys and see you at soon