Microsoft Azure Storage and Database Part 24 – Azure File Service – Storage Sync Service
Hope you all are doing good !!! 🙂 .
In one of our previous articles we have discussed How To Restore Azure File Share From BackUp Using PowerShell. Today in this article we will discuss One Very Useful Azure Storage Sync Service.
Tool Installation Articles :
- Configure Azure Command Line Interface ( Azure CLI) On Windows
- Configure PowerShell For Microsoft Azure Az Module On Windows
Previous Azure series :
- Learn Basics Of Azure Networking In 60 Hours
- Learn Basic Of Azure Active Directory And Azure Identity And Access Management
- Azure DevOps – Learn at one place
If you have missed our previous articles on Azure Storage and Database Series, please check it in following links.
Storage Sync Service
The Storage Sync Service resource is a peer of the storage account resource, and can similarly be deployed to Azure resource groups. It can create sync groups that contain Azure file shares across multiple storage accounts and multiple registered Windows Servers. Below is the order, one must follow to configure Azure File sync. A sync group ties the Azure file share and the folder on your server together and establishes a sync connection.
Install the Azure File Sync agent on the Window server.
Register the Windows Server with the Storage Sync Service and it will creates a registered server object, which represents a trust relationship between your server or cluster and the Storage Sync Service
Create a sync group in a Storage Sync Service
As per Microsoft documentation, a Sync Group contains one cloud endpoint, or Azure file share, and at least one server endpoint. The server endpoint object contains the settings that configure the cloud tiering capability, which provides the caching capability of Azure File Sync. In order to sync with an Azure file share, the storage account containing the Azure file share must be in the same Azure region as the Storage Sync Service.
Azure File Sync transforms Windows Server into a quick cache of our Azure file share. We can use any protocol that’s available on Windows Server to access our data locally, including SMB, NFS, and FTPS. We can have as many caches as you need across the world.
Many businesses still rely on Server Message Block (SMB) or Network File System (NFS) to work with files across a network. If our organization is in the cloud, it’s time to learn about how to set up Azure Files and Azure File Sync. With Azure File Sync, we can cache remotely-stored files in Azure Files shares locally or on an Azure VM.
Planning An Azure File Sync Deployment
Before deployed the file sync, we should plan it properly to avoid future issue. In our plan we need to consider following points.
Consider the number of storage sync services -As one Server can only be registered to one Storage Sync Service it is often best to only deploy a single Storage Sync Service and register all servers that it. But in some special case we can deploy multiple Storage Sync Service.
Sync with root -Azure File Sync supports syncing the root of a volume to an Azure file share. If we sync the volume root, all subfolders and files will go to the same Azure file share. It is tested with 100 million items (folders and files) but we should try to keep the number below 20 million or 30 million in a single share. We will get following benefits we we keep items limited.
- Initial scan of the cloud content can complete faster, which in turn decreases the wait for the namespace to appear on a server enabled for Azure File Sync.
- Cloud-side restore from an Azure file share snapshot will be faster.
- Disaster recovery of an on-premises server can speed up significantly.
- Changes made directly in an Azure file share (outside of sync) can be detected and synced faster.
Form a structured deployment map -We should create a map between on-premises folders and Azure file shares. We should follow the below best practices.
- A server on which the Azure File Sync agent is installed can sync with up to 30 Azure file shares.
- There are couple of limitation with file shares in standard storage account but those limitations don’t apply to premium storage, where performance is explicitly provisioned and guaranteed for each share.
- There’s a limit of 250 storage accounts per subscription per Azure region.
Operating System Supports
Azure File Sync is supported with the following versions of Windows Server till we write this article. Microsoft recommend keeping all servers that you use with Azure File Sync up to date with the latest updates from Windows Update.
|Version||Supported SKUs||Supported deployment options|
|Windows Server 2019||Datacenter, Standard, and IoT||Full and Core|
|Windows Server 2016||Datacenter, Standard, and Storage Server||Full and Core|
|Windows Server 2012 R2||Datacenter, Standard, and Storage Server||Full and Core|
Azure File Sync requires a server, either physical or virtual, with at least one CPU and a minimum of 2 GiB of memory. If the server is running in a virtual machine with dynamic memory enabled, the VM should be configured with a minimum of 2048 MiB of memory.
Using following cmdlets, we can checks for potential issues with your file system and dataset, such as unsupported characters or an unsupported operating system version. We can evaluate whether it is compatible with our system or not. Followings are few cmdlets to check with different parameters.
To perform both the system and dataset checks -Invoke-AzStorageSyncCompatibilityCheck -Path
To test only your dataset
-Invoke-AzStorageSyncCompatibilityCheck -Path -SkipSystemChecks
To test system requirements only -Invoke-AzStorageSyncCompatibilityCheck -ComputerName -SkipNamespaceChecks
To display the results in CSV–
$validation = Invoke-AzStorageSyncCompatibilityCheck C:\DATA
$validation.Results | Select-Object -Property Type, Path, Level, Description, Result | Export-Csv -Path C:\results.csv -Encoding utf8
Azure File Sync works with our standard AD-based identity without any special setup beyond setting up sync.
The Azure File Sync agent communicates with our Storage Sync Service and Azure file share using the Azure File Sync REST protocol and the FileREST protocol, both of which always use HTTPS over port 443. SMB is never used to upload or download data between our Windows Server and the Azure file share.
Azure File Sync, consider following three different layers of encryptions.
- Windows Server encryption at rest
- Encryption in transit
- Azure file share encryption at rest
Azure File Sync agent update policy
The Azure File Sync agent is updated on a regular basis to add new functionality and to address issues. We recommend you configure Microsoft Update to get updates for the Azure File Sync agent as they’re available.
Azure File Sync scale targets
The following table indicates the boundaries of Microsoft’s testing and also indicates which targets are hard limits.
|Storage Sync Services per region||100 Storage Sync Services||Yes|
|Sync groups per Storage Sync Service||200 sync groups||Yes|
|Registered servers per Storage Sync Service||99 servers||Yes|
|Cloud endpoints per sync group||1 cloud endpoint||Yes|
|Server endpoints per sync group||100 server endpoints||Yes|
|Server endpoints per server||30 server endpoints||Yes|
|File system objects (directories and files) per sync group||100 million objects||No|
|Maximum number of file system objects (directories and files) in a directory (not recursive)||5 million objects||Yes|
|Maximum object (directories and files) security descriptor size||64 KiB||Yes|
|File size||100 GiB||No|
|Minimum file size for a file to be tiered||V9 and newer: Based on file system cluster size (double file system cluster size). For example, if the file system cluster size is 4 KiB, the minimum file size will be 8 KiB.|
V8 and older: 64 KiB
In this article, few section copied from Microsoft document. I hope this is informative to you. In our next article, we will see how to configure Azure File Sync and Storage File Sync. Please let me know if I missed anything important or if my understanding is not up to the mark. Keep reading, share your thoughts, experiences. Feel free to contact us to discuss more.
If you have any suggestion / feedback / doubt, you are most welcome. Stay tuned on Knowledge-Junction, will come up with more such articles.
Thanks for reading 🙂 .