A flexible file backup Strategy – Part 1

As many of you know, one of the biggest innovations introduced in VBR version 10 is the support of NAS backup.

Does this mean that it was not possible to save the unstructured file data before?

Actually not, more than one options were already present.

The scope of the next articles is to show when and how to use those technologies to answer customer needs.

In all the cases the product used is the powerful VBR.

The four main topics are:

  1. NAS Backup
  2. File Backup to Tape
  3. NDMP
  4. File Backup to Disk

For each of the above-mentioned items, the articles will show:

  • How it works
  • Common requests and scenarios.
  • Technology, Pro & Cons.

Let’s start!

1- NAS Backup

  • How does it work?

Veeam mantra is “innovate“!  A clear example is represented in the NAS technology.

The primary idea on which this technology is based is to track the changing of the unstructured files.

Let’s clear it up with a comparison: I’m quite sure all VBR users know the CBT (change block tracking) technology strongly used by VBR to create backups of VMs. It allows saving data blocks that have been changed from the previous backup.

The ingenious idea of Veeam R&D is to use this approach when files and folders must be saved.

I called it  FCT (File change Tracking).

How does it work?

When a NAS backup is performed, VBR calculates on-fly the CRC (Cyclic redundancy check) of any single file that has to be saved. Those metadata are stored in the cache repository (points three and four of Picture 1).

Picture 1

Why is this pre-process so important? Because thx to it it’s possible to:

  • Perform incremental backup forever (only new and changed files are saved). It means a shorter backup window.
  • Speed up the restore phase; image the scenario where a customer has 5 PB of data and luckily just 1 TB of data has been attacked by a Virus or accidentally deleted by a script.
    The IT manager will ask to restore just 1 TB and NOT all PBs.

VBR using the “FCT” can understand which files have been changed /deleted with respect to a specific Restore point and restore just those needed.

This great option is called “rollback to a point in time“.

Just as a reminder, there are two more ways to restore data, Entire file share and single file and folders.

  • Common scenarios

NAS backup can be used by any customer. You need just a repository and a valid license (VUL).

The scenario I like to talk about is where the customer has big fillers in his environment.

Why?

Because it is possible to leverage the storage snapshot to gather files as shown in picture 2.

Version 11 will have improvements in this area too. Stay tuned by signing up to Veeam site (https://go.veeam.com/v11).

Picture 2

Leveraging the storage snapshot a customer can

1. Speed up the backup process.

2. Save files though they are in use (Open files can’t be saved by a backup process while they are processed by users).

This integration allows performing backups to any hour of the working day without any attention to the status of the file.

Main Pro

a -The architecture is very scalable because it leverages the concept of proxy very common to VBR.

Proxies are the data mover that collect data from the source and send them to the Repository. The File Proxy has also the responsibility to calculate the FCT.  You can add more proxies when you need to address the backup of big amounts of data.

b- The files saved to the Repository are written in a customize format. They are managed as an object in a vBLOB Storage and contain the metadata of every single file saved (they contain info about which folder the file belongs to and which are the file rights also).

Pictures 2 and 4 show the new format of the backup file for NAS.

Picture 3

Picture 4

The main advantage is that all file restore tasks are very very fast!

c- It’s possible to copy backup data to the secondary repository setting different retentions. It allows answering the common request to have a copy of backup data in another location (3-2-1 rule).

d- It’s possible to create an archiving backup file policy through the object storage VBR integration. Picture 5 (taken from the VBR user guide) shows the main repositories option available with NAS Backup.

Picture 5

Cons

It doesn’t support the transfer to Tape Devices. Please read the article about NDMP and File to Tape to get an interesting solution.

That’s all for now.

See you in a couple of days for File to Tape Backup

Ransomware defense part 4: Deep Dive

In this last article about Security and Ransomware, I’m going to add new features and deep dive those you’ve already read in my first article about Veeam Backup & Replication.

The starting point is creating copies of your data (remember the 3-2-1 rule) and from these copies perform automatic tests of availability and security.

Which technology should be deployed to sleep safer?
In this article, I’m going to show you some Veeam technologies that address the threats explaining why they are a must to be used.

Sure-Backup

It is the best way to be certain that a backup is really usable.

What does it mean exactly?

In my public speeches, I often use a pen to explain the Sure-Backup concept correctly.

How can I be sure that the pen is usable? The answer is very easy. If it writes on paper it works and it is usable.

Going back to our scenario,  the only way to know if your backup is usable is to perform a restore in an environment logically separate from the production.

How Sure-Backup works?

It is composed of two parts:
The application group identifies the backup VMs that have to be verified.
DataLab is the way through which VMs are switched-on into a specific network that can’t communicate with the production network.

This great technology allows you to add the number zero meaning no errors (from 3-2-1 to 3-2-1-0) to the above rule.

One more important thing to add here is that you can create scripts to random test your backup.

In the following article by Luca Dell’Oca, you can have an excellent example of how scripts can help you.

How can you test 1000 VMs with Veeam SureBackup?

Sure-Replica:

VBR is a solution that allows customers to perform Backup and Replicas of your VMs.
Replicas are commonly deployed to create a Disaster Recovery Site.

Veeam has “integrated” the Sure-Backup technology for Replica too. It is named Sure-Replica and it gives all advantages shown for Sure-Backup

For managing the automation of Replicas I suggest customers use a new Veeam product, the Veeam Availability Orchestrator (VAO).

On this site, you can find a very detailed guide to use set-up VAO.

https://lnx.gable.it/home-page/2020/08/07/veeam-availability-orchestrator-v-3-0-vao-baseline-1/

Secure Restore

It scans the saved  VM Windows  (VM that has at least one valid restore point), with your antivirus software before restoring the VM to the production environment.

Secure Restore is available for the following restore tasks:

  • Instant VM Recovery
  • Entire VM Restore
  • Virtual Disks Restore

The only requirement is that your antivirus has to be installed on the mount server and supports CLI.

Data Integration API

Veeam Data Integration API is a set of Veeam PowerShell cmdlets that allow you to represent data of backup files as a mounted Windows folder.

This feature allows you to access backed-up data in read-only mode.

It has a lot of possible usages to example data mining and data warehouse.

In the field of security, it can be adopted to check if a virus is already present in your VM checking via backup files the guest OS files through your antivirus software (It scans the files of the VMs and not the backup file). A good example of use is in the following blog article by Niels Engelen

https://www.veeam.com/blog/v10-data-integration-api.html

Storage Integration

VBR has two different storage integration.

The primary storage integration allows customers to perform backup more frequently because this technology allows creating backup without impacting the VMware environment.
The integration allows creating a test environment because it is integrated with Datalab and Sure technologies.

The secondary storage integration is commonly used with deduplication appliances that can allow you another layer of security for ransomware.

When the Veeam Data Mover Service is installed directly on the appliance, it reduces the risk of ransomware encrypting and deleting data.

In addition, if for some reason the first data is compromised, copies can be replicated to a second data center or in the cloud.

Before leaving you my two last cents:
1) The starting point is always to perform copies of your data. Veeam gives you powerful tools to manage them and to perform more check to verify that your data are safe from threats.

2) Some customers still think that the backup asset is just a cost because they are used just to restore. With Veeam you can use backup data to perform different actions relieving the production of not business core tasks (for example think the great use of Data API Integration for data mining, data warehouse etc)

Take care guys and see you at soon

Ransomware defense part 3: Monitoring and more

In the previous articles, I described some good ideas to design your architecture to keep it safer as much as possible.

One of the greatest challenges the IT guys have to face is finding the right balance among design, deployment and budget.

It’s very important to have the right tools to measure architectural behaviour. In this way you can easily:

  1. Watch from a privileged point of view the architecture. Let’s image to be on the top of a mountain watching people and goods moving at the bottom of the valley”
  2. Launch the defending actions when an attack is on-going. Referring to my previous example, it’s like blocking some passages to people and goods.
  3. When the attack is over remove any possible threat left (cleaning the passages).
  4. Do a thorough workup understanding of the weak points of your architecture and create a plan to reinforce it.

Monitor tools are your sentinels, but they need to be trained to trigger also the first defense lines. Imagine the new sentinel as a lieutenant warrior with a varied arsenal of weapons.
To be clearer: the required features monitor and respond to actions in function of the severity of the alarm.

But why is measuring so important? The reason is that you can define the KPI (Key performance indicator) for your environment and periodically check if the measures are respected.
In other words, it is possible to measure the service level and understand if the budget and skill invested in the company are enough to address the backup security challenges or if more tunings actions or some great changes are needed.

Let’s see how to use Veeam One to address this common request:

The Possible ransomware activity alarm keeps tracking of the Operating system of the VM.

As shown in picture 1 the monitored counters are by default CPU, Datastore write Rates and networking transmit rate (the case of copy offsite of sensible data for future blackmail).
The value counters can be changed to adapt to your own needs (Tuning phases) and more counters can be added to monitor more objects as shown in picture 2.

Picture 1

Picture 2

Another alarm already present in Veeam One is “Suspicious increment backup size“.

It checks if the restore point size is significantly different from the previously created ones.

The two main reasons I like  Veeam ONE are:

  1. Very easy to use
  2. Customizing the action after an alarm has been triggered

Thx to “customizing action” it’s possible to launch your antivirus/antimalware on the VMs belonging to the backup job that has triggered the suspicious alarm, or disconnect the repository from the network, or what else you wrote on your incident and rescue procedure.

The main point here is that you can manually click on it or automatically execute the action as shown in picture 3

Picture 3

Veeam One has furthermore an exhaustive technology of reporting.

If an alarm is a good way to intercept an error or a misconfiguration because  it works in real-time, through the reporting it is possible to check the status of your protection (KPI, SLA….), understanding the exercise and security cost of your production environment and forecasting the new investment to implement in the next years.

Which are the reports to use?

All of them are important and an all report list is available from the following link: Reports 

Just as an example please check the use of the following

The next article will talk about which are the automatic procedures you can adopt to check your backup infrastructure.

Take care and see you soon

Ransomware defense – part 1: Advanced product features are an mandatory requirement

A lot of new challenges came to people who work in IT-Departments these last months.

The number of ransomware attacks has been growing day by day and their attack strategies are becoming more and more evil and dangerous.

The common questions the Managers ask the IT guys are:

a) Are the company protected against these risks?

A good answer is that a successful approach is when the percentage of certainty is more than the percentage of risk.

b) Which are the best practices to be safer?

The key is defining the right process of protection.

The scope of these articles is showing the correct behavior to keep your architecture as safer as possible or, in case of attack, gain as much time as possible to fend off the assault.

The articles will cover the storage point of view and do not deal with perimetral defenses, antimalware, antiviruses, networking strategies, and so on.

Which are the main strategies to adopt?

  1. Having more copies of your data
  2. Hardening the infrastructure
  3. Monitoring behaviors

Are you ready? Let’s start with the first topic !!!

    1. Having more copies of  your data:

Backup software is the right tool to score the goals of this first part.

It has to be able to:

a) Create application consistency backup.

b) Copy backup data to different locations.

Almost all backup software can do that but some additional features can address better the biggest challenges:

Flexible: Backup software should write backup data to different types of repositories and be able to restore it without any required dependency. To be clearer, the backup data have to be self-consistent. The advantage is being able to fit different architecture scenarios (Let’s call it “Data mobility”).

Data-Offline:  back up data should be put into a “quarantine” area where they cannot be either re-written or read. The classic deployment is a Tape Devices architecture or any scripts that automatically detach the repository devices.

Immutability: The backup data cannot be changed until the immutability period is over. This has a double advantage in comparison to data-offline strategy: It changes the repository status as written & online just for the new backup file. It is offline (as Tape technologies) for re-writing to already present backup data. The speed restore option has to remain unchanged.

Immutability can be reached in two ways:

By WORM  (Write Once, Read Many) devices, where the backup files can be used just to restore once they have been added to repositories. For example, technology can be the optical disk, a technology I have been working on in the past.

At Veeam Software this common customer and partner request has been addressed using the immutability propriety of the Object Storage. The good news is that VBR v. 11 implements this great feature directly in Linux Repositories.

Is this enough? I’m still thinking that the backup solution should at least be able to:

  • Check the backup file and the backup content. The only way to check if a backup file is really reusable is restoring it in a separate area where communication with the production environment is forbidden. At Veeam it is called Sure-Backup.
  • Check with your anti-virus/anti-malware that the backup files have not been already attacked somewhere and sometime. At Veeam the technology used is the Data integration API.
  • Before restoring files or VMs in production, check with your anti-virus/anti-malware if your data has been already attacked. At Veeam it is called Secure Restore
  • Perform Replica Jobs. It helps to create a Disaster Recovery Site useful in performing a quick restart of the service.  At Veeam this feature is included from the beginning and the Sure-Backup can be applied with replica too (it is called Sure-Replica). V.11 has a very powerful feature: CDP.
  • Restore backup data to the public cloud when the primary and replication site is totally out of order. I call it Cold Disaster Recovery and it needs at least one restore point available.

The next article topic is how to hardening your backup architecture

See you soon and take care!

VBO-365 Portal: A nice project just behind the corner – Part 3

This third part will cover the restoration options.

If you need to know how to set the portal up or the option it can provide, please read the previous articles (Part-1 – Part-2)

Let’s start with Exchange Environment:

Clicking on the Exchange voice located on the top part of the web page (in picture 1 it is highlighted with a green arrow )

Picture 1

After selecting the organization you want to restore data from (in my case is myTEST-Environment as shown in picture 2), just choose the restore point you need (Picture 3).

Picture 2

Picture 3

Picture 4 shows the mail-boxes previously saved that you can restore.

Picture 4

What’s happening on VBO-365?  As shown in picture 5 the portal has triggered a  classical restore task.

Picture 5

Which are the restore option available in the portal for the Exchange environment? As shown in picture 6, you can download the mail as a pst file (it requires outlook client installed) or restore to the original or different location as shown in picture 7.

Picture 6

Picture 7

To finish the restore tasks just click on stop restore (Picture 8 and 9).

Picture 8

Picture 9

And what about restore of One-Drive and Sharepoint items?

The procedure is very similar and the restore options are shown in pictures 10 and 11.

Picture 10

https://lnx.gable.it/wp-content/uploads/2020/11/42-martini.jpgPicture 11

VBO-365 v.5 has introduced support for Microsoft-365 Teams. It is not still available in this portal and I hope it will come soon. Stay tuned 🙂

That’s all for the VBO portal for now. Take care

VBR – Proxy linux server UUID

When a Linux VM is added to Veeam console as a Proxy Server,  you can fall out in the error shown in picture 1

Picture 1

The reason for this behavior is that the default VM config does not allow another software to see the UUID of the VM.

What is UUID?

It’s the unique identifier used to uniquely identify partitions in Linux operating systems.

Why is it important to use it?

A backup where the proxy is a Linux VM only works with virtual appliance transport mode. It uses the VMware hot add capability.

Easier: when a job starts, the proxy Linux mounts the disks of the VM that have to be processed and then send a copy of data to the Veeam Repository.

If the backup server knows which are the proxy disks it can process the others easily and without errors.

The result is that it’s mandatory to set it up correctly as shown in the user guide and in Veeam forum

Note 1: the Linux command to show UUID is blkid

To address the issue just switch off the VM and, from vCENTER Console, follow the procedure showed in the next 4 pictures highlighted in yellow.

Picture 2

Picture 3

Picture 4

Picture 5

That’s all folks