In-Depth
The Secrets of Sysvol
Windows Server 2008 has a new way to migrate Group Policies.
You already know that Windows Server 2008 brings a lot of changes. You've probably heard about the product's Read Only Domain Controllers (RODCs), Server Core and fine-grained password policies. There's another factor that almost no one seems to know about: Server 2008 replaces a vital piece of Active Directory. This change is along the lines of replacing the engine in your car as you're driving 70 miles per hour down the highway: If done badly, it could shut down Group Policies and negate your log-on scripts.
You've worked so hard to get every security setting configured into a Group Policy. Don't let replicating those changes from one domain controller (DC) to another be your downfall. There's a special folder that contains your Group Policy settings, default profiles and log-on/log-off/startup/shutdown scripts. This folder is created and shared when you successfully promote a member server to a DC by running DCPromo: it's called Sysvol.
There should be a Sysvol folder on every DC. When a user logs on to a computer that's a member of a domain, their Group Policy settings, profiles and scripts are downloaded locally from a DC's Sysvol folder. It makes sense that all DCs need to have the exact same Sysvol content. If something changes on one DC (usually the PDC Emulator), you need to copy, or "replicate," those changes to all other DCs.
In the past, we had no choice. The engine that replicated Sysvol was the File Replication Service (FRS). Any network admin who has had the pleasure of troubleshooting morphed files and folders or journal wraps can attest to the fact that FRS's replication of Sysvol leaves a lot to be desired. Server 2008 finally offers a new replication engine option: Distributed File System Replication (DFSR). DFSR was first introduced in Windows Server 2003 R2 but could only replicate Distributed File System Namespaces (DFSNs). Sysvol was still a slave to FRS. Here, I'll explain the process to migrate from FRS to DFSR for your Sysvol replication step-by-step.
[Click on image for larger view.] |
| Figure 1. The Distributed File System Replication lets you carefully create and review your selections when using the Sysvol folder for Group Policy migration. |
Before the Migration
To begin with, my environment is a Windows Server 2003 SP1/R2 domain named Bigfirm.com (migration from FRS to DFSR is performed at the domain level). Bigfirm.com contains two DCs named FRSRIP (PDC Emulator) and DC2. I like to run the migration process from the FSMO role holder of the PDC Emulator.
So, from the PDC Emulator put in the Server 2008 DVD, go to a command prompt, change the drive letter to your Server 2008 DVD, change directories to the \Sources\Adprep folder and run Adprep /forestprep. Next you'll need to raise the domain functional level to Server 2003 if it's not already there. To raise the domain functional level to Server 2003, open Active Directory Domains and Trusts, right-click your domain name and choose Raise Domain Functional Level. You should see an appropriate dialog box after that.
Starting from the top: my domain name is Bigfirm.com. The current domain functional level is Windows 2000 Mixed (yours might be Windows 2000 Native). In the "select an available domain functional level" drop-down list, choose Windows Server 2003 and click the Raise button. Next, from the Server 2008 DVD's Sources\Adprep folder run Adprep /domainprep from a command prompt as I did earlier with the forestprep switch. Then upgrade the PDC and all DCs to Server 2008. Last, you can raise the Domain Functional level to Server 2008 (just like you raised it to Server 2003).
Before you begin your migration, I highly recommend that you test Sysvol replication to ensure it's working properly. FRSDiag is a free tool you can download from Microsoft's download site. This tool lets you perform a propagation test, which will create a brand-new file in Sysvol and track its replication progress so you can see if any DCs are currently having replication issues. To perform a propagation test using FRSDiag, click on the Tools menu and choose "Propagation File Tracer."
If your Sysvol is healthy and replicating throughout your domain, make a backup of Sysvol. However, I prefer the idea of taking a DC that's current with AD and doing Sysvol replication completely offline, physically unplugging it from the network. I like this approach just in case I need to quickly get it back up and running. This doesn't have to be a physical machine; it can be a virtual machine (VM). VMware Inc. makes it a snap to put a VM on a custom subnet so it can't talk to any other machines that are not also on the same subnet. At this point you should place Sysvol in a lockdown state. There should be no changes that occur in Sysvol until the migration is complete. I'll explain why a little later.
Migration
Before we get into the actual migration process, I'd like to give you a 10,000-foot view of what's supposed to happen at each step along the way. I like to call these steps stages, while Microsoft calls them states. First, you should know there are two different types of stages/states: Stable states and Transitional states. The stable states mark the big milestones of the migration process, where the Transitional stages can be viewed as the "working processes" that take you from one stable state to the next. Again, I can explain the states and what happens in each. The tool used for migration is a command-line utility called DFSRMig.exe and can be found on a Server 2008's Windows\System32 folder.
All DCs begin at state 0. State 0 means that all DCs are currently replicating the SYSVOL folder using the FRS. Microsoft recommends running the DFSRMig.exe utility on the PDC Emulator. To kick off the migration process on the PDC Emulator, open a command prompt and type the following:
Dfsrmig /SetGlobalState 1
You should see the following output:
Current DFSR global state: 'Start'
New DFSR global state: 'Prepared'
Migration will proceed to 'Prepared' state. DFSR service will copy the contents of Sysvol to SYSVOL_DFSR folder.
If any DC is unable to start migration, try manual polling, or Run with option /CreateGlobalObjects. Migration can take anywhere from 15 minutes to one hour to start.
Succeeded
The DFSRMig commands are not case-sensitive, so dfsrmig /setglobalstate 1 would have worked as well. State 1 is called the Prepared state, but to get to state 1, Transitional states 4 (T4) and 5 (T5) must be performed. What do they do? T4 creates a new object in Active Directory Users and Computers (ADUC) that will be used for DFSR replication of Sysvol (very similar to the File Replication Service object that exists for FRS). The new object is named DFSR-GlobalSettings and can be found in the System container; you'll need to click the View menu and choose Advanced features to see the System Container and it's contents. T5 creates a new folder in the Windows folder named SYSVOL_DFSR; you can see this in Windows Explorer. Then, robocopy runs the following command:
ROBOCOPY c:\Windows\Sysvol\Domain
c:\WindowsSysvol_DFSR\Domain /Copyall /MIR /B /R:0
/XD "Do_Not_Remove_NtFrs_PreInstall_Directory"
"DfsrPrivate" "NtFrs_Prexisting__See_Eventlog"
"NTFRS_CMD_FILE_MOVE_ROOT" /XF
"DO_NOT_REMOVE_NtFrs_PreInstall_Directory"
"DfsrPrivate" "NtFrs_PreExisting__See_Eventlog"
"NTFRS_CMD_FILE_MOVE_ROOT"
This copies the Sysvol and domain folders from the Sysvol folder to the new SYSVOL_DFSR folder.
The object created in ADUC called DFSR-GlobalSettings contains an attribute named msflags. The object DFSR-GlobalSettings will replicate from one DC to another until all DCs have the object. Then, the msflags attribute is set based on what state the migration process is in at the time. To begin with, the msflags attribute will be set to 16. So if you brought up a new DC in the middle of migration -- though this is not recommended -- the fact that this msflags attribute is set to 16 would tell DFSRMig to begin the migration process. Sometimes you may have one DC that's taking to long to get to state 1, like my DC named DC2. I'll need to move it along a bit by forcing AD replication (to get the DFSR-GlobalSettings object) and then force dfsrdiag to read the AD object. To force replication, you'll use the repadmin command-line utility, which looks like this:
Repadmin /replicate destinationDC
sourceDC domain components
So, my sourceDC that I know contains the DFSR-GlobalSettings is FRSRIP and the DC that's taking too long to begin the migration process is DC2, which will be the destinationDC for the DFSR-GlobalSettings object. Here's an example:
Repadmin /Replicate DC2
FRSRIP dc=bigfirm,dc=com
To tell dfsrdiag to read the AD object, I would type the following from a command prompt:
C:\ dfsrdiag pollad
If this doesn't work, then you'll have to manually create the SYSVOL_DFSR folder in the Windows folder -- the same folder as the existing Sysvol -- and run the robocopy command to populate the SYSVOL_DFSR folder.
The log file that tracks the migration process can be found in the Windows\Debug folder. It's named DfsrMig_###.Log through however many log files there are. On my system at this point I had three: DfsrMig_001.Log.gz, DfsrMig_002.Log.gz and DfsrMig_003.Log.
The extension of .gz shows that once a file is full it's compressed using Gzip. You can check to see if all your DCs have reached the prepared state by typing the following command at the command prompt; I always do it on the PDC Emulator:
Dfsrmig /GetMigrationState
You should get a listing of all DCs and the state they're currently in. In the following output I have two DCs, FRSRIP (PDC Emulator) and DC2:
The following Domain Controllers are not in
sync with Global state ('Prepared'):
Domain Controller (Local Migration State) - DC Type
===========================================
DC2 ('Start') - Writable DC
FRSRIP ('Waiting For Initial Sync') - Primary DC
Migration has not yet reached a consistent state
on all Domain Controllers.
State information might be stale due to AD latency.
If you're tracking things through event viewer you should see the following events in the DFS Replication event log:
Event ID 8000: The DFSR global settings have
been created.
Event ID 8008: SYSVOL migration global state is set
to 'Prepared' the current local state is 'Start'.
Event ID 8010: DFSR will now create the
SYSVOL_DFSR folder, and objects in the local AD.
Event ID 8012: DFSR has successfully created the
SYSVOL_DFSR folder.
Event ID 8008: DFSR has started the transition
to global state 'Prepared'.
Event ID 1210: DFSR successfully set up an RPC listener
for incoming replication requests.
Event ID 6804: DFSR has detected no connections
configured for replication group Domain System
Volume. No data is being replicated for this replication group.
And the Replication Group and
Member ID GUIDs
Event ID 4112: DFSR initialized the replicated folder
at local path C:\WINDOWS\SYSVOL_DFSR\domain.
This member is the designated primary member for
this replicated folder.
- SYSVOL Share
- Replicated Folder ID: GUID
- Domain System Volume
- Replication Group ID: GUID
- Member ID: GUID
The Re-Directed State
When you type DFSRMig /getmigrationstate at a command prompt and receive the following message, migration has reached a consistent state on all DCs:
All Domain Controllers have migrated successfully
to Global state ('Prepared').
You're ready to move onto the Re-Directed state. The purpose of this state is to direct the old Sysvol share to the new SYSVOL_DFSR folder. To do that, Transitional state 6 (T6) runs. T6 edits the registry key HKLM\System\CurrentControlSet\Services\Netlogon\Parameters and sets the SysvolReady key to False, then the Sysvol path is changed to \WINDOWS\SYSVOL_DFSR\sysvol.
Then the SysvolReady key is set to True and you're back in business, but now your Netlogon and Sysvol shares point to the new directory. If you go to a command prompt after this completes and type C:\ Net share, then you should see the following paths for your Netlogon and SYSVOL shares:
Netlogon = Windows\SYSVOL_DFSR\sysvol\DomainName\SCRIPTS
SYSVOL = Windows\SYSVOL_DFSR\sysvol
If you're following the progress in the event logs you should see the following events in the DFS Replication event log:
Event ID 8008: The SYSVOL migration global state is
set to 'Redirected' and the current local
state is 'Prepared'.
Event ID 8015: Starting the process of redirecting
replication of the Sysvol share on DC FRSRIP. NTFRS
will continue to replicate the Sysvol share located at
C:\WINDOWS\SYSVOL until the local state
transitions to 'REDIRECTED'.
Event ID 8017: DC FRSRIP has successfully migrated
to the 'REDIRECTED' state. DFSR is replicating
SYSVOL_DFSR folder located at
C:\WINDOWS\SYSVOL_DFSR.
TO CONTINUE MIGRATION: If you choose to continue
the migration process and proceed to the
'ELIMINATED' state, it will not be possible to revert
the migration process. The Sysvol folder located at
C:\WINDOWS\SYSVOL will be deleted.
At this point the old Sysvol folder is being replicated using FRS -- it isn't shared anymore, but that's no problem for FRS -- and the new SYSVOL_DFSR folder is being replicated with DFSR. Remember that long and ugly robocopy command that ran in T5? Well, if someone were to edit a log-in script, or anything else, that lives in the old Sysvol folder, those changes would never get copied to the new SYSVOL_DFSR folder. This is the reason you want Sysvol to be in lockdown. Of course, when you have many admins, there's always the chance that someone will edit something in the old Sysvol folder and then be very surprised when they can't get the changes to work.
At this stage, if you were to edit a Group Policy by launching GPMC.msc, you'd now be connected to the new SYSVOL_DFSR share. To prevent people from making changes in the wrong folder you can move to the last state -- called the Eliminated state -- but beware, there's no rolling back from this state.
In the Eliminated state Transitional state 7 (T7) runs. T7 actually deletes the old Windows\SYSVOL folder. FRS continues to run, just in case you have some distributed file systems that are being replicated with FRS.
Oops, I Changed My Mind
Microsoft has built in rollback capabilities to the migration process in case you change your mind. For example, if you had migrated to state 1 you could go back to state 0 by typing:
dfsrmig /setglobalstate 0
If you had completed migration to state 2 you could rollback to state 1 by typing:
dfsrmig /setglobalstate 1
But if you migrated to the Eliminated state, the old SYSVOL is gone and there's no rollback procedure for this. You should be fully committed to DFSR replicating SYSVOL before you go to state 3. Once state 3 is complete you'll see the following four events in the DFS Replication event log:
Event ID 8008: The SYSVOL migration global state is
set to 'Eliminated' and the current local state is
'Redirected'.
Event ID 8018: DFSR will now migrate the DC
FRSRIP to the 'ELIMINATED' state. Please note that
there is no going back. DFSR will now proceed to
delete the SYSVOL share located
at C:\WINDOWS\SYSVOL. DFSR will also delete
the local Active Directory objects corresponding to
NTFRS and the NTFRS member object for the
Domain Controller FRSRIP. The NTFRS service will
also no longer depend on the NTDS service.
Additional Information:
Sysvol NTFRS folder: C:\WINDOWS\SYSVOL
Domain Controller: FRSRIP
Event ID 8004: The NTFRS member object for the
Read-only Domain Controller FRSRIP was deleted
successfully.
I'm not exactly sure why I got the 8004 event. This DC was a read/write DC, not a read-only DC, so if you get one as well, know that it's probably OK.
Event ID 8019: DFSR has successfully migrated the DC
FRSRIP to the 'ELIMINATED' state. DFSR migration for
the Domain Controller FRSRIP is now complete.
There are certainly many more questions that need to be answered about SYSVOL and DFSR. These questions include: Now that DFSR is replicating Sysvol, how often does it happen? What kind of control do you have over Sysvol's replication? Can you change the schedule of how often replication occurs? What sort of control do you have over how much network bandwidth Sysvol's replication uses? What monitoring tools are currently available that can give you the entire picture of Sysvol's health in your domain?
These are questions to be answered in future articles. But if you're one of the admins who has been waiting for FRS to be replaced, it's time to rejoice. DFSR is a much more scalable and robust replication engine than FRS ever dreamed of being.