There is a bug in DfsrHelper.dll
If a Powershell script creates a DFSR RG and invokes the CmdLet "Set-DfsrMembership" multiple times to add several member servers then there is a high probability that the CmdLet will fail.
I have a PowerShell script than creates a new DFSR RG, adds the local server and a folder, then calls the above CmdLet.
if the script is run in a loop (deleting the RG between iterations) then the script will fail after 2 - 5 loops.
I've used .NET Reflector to debug the DfsrPowershell, DfsObjectModel and DfsrHelper DLL's and I found that the error is as follows:
Set-DfsrManagement attempts to modify the Replication Group object under the target computer in Active Directory. It first attempts to perform the operation using the currently logged on user's credential and when that fails it retries using using the target computer account. This normally succeeds however, when it does fail the LDAP response is invalidAttributeSyntax
Examining the modifyRequest command I see one of the attributes is:
modification item operation: delete (1)
vals: 1 item
And the AttributeValue is empty - which is invalid!
Now the modifyRequest that was sent using the logged on user's credentials is correct:
vals: 0 items
The difference is the first modifyCommand is built and sent by the DfsObjectModel DLL while the failing command is sent by the DfsHelper DLL which is native code and described in the MS-DSFRH document.
I have debugged the code as far as the interface into DfsrHelper and confirmed that the attributes passed into the DLL are correct i.e. the msDFSR-DfsLinkTarget attribute operation is "delete" and there are no values set. Therefore I am of the opinion that there is a defect in the DfsrHelper DLL implementation that incorrectly appends an empty value when constructing the LDAP command.
I opened a MSDN case with the breakfix group who confirmed it was a bug but were unable (due to internal policy??) to submit their findings to the DFSR dev team.