FIM 2010 UnicodePwd Issue on New AD User Creation

UPDATE 2010.08.17 – Ultimately, 2008 SP1 did not resolve the issue in this case.  We ended up enabling Password Policy Enforcement in FIM 2010 SSPR.  This change alters how FIM connects to AD to make password changes on new user creation, password sync, and password reset operations.  Instead of Kerberos, it uses LDAPS to the domain controller.  Together with a hotfix for the AD MA used, it now enforces the full AD password policy (including password history).

----

Now this was a an odd one. I am working on a pretty basic FIM deployment and got to the point of provisioning users into Active Directory. I set up all of my attribute flows in Outbound Sync Rules and set up my workflows and MPRs accordingly. At this point, I was flowing a static string value to unicodePwd – one that did meet the password requirements of the domain – and it was configured for “Initial Flow Only”. The user gets created in Active Directory, but I was getting a “Cd-Error” in FIM with no real detail, other than the fact that userAccountControl was not being updated to 512 (for an enabled, regular user).  No errors in the event logs on the FIM Sync server or the target domain controller.  Using the account used by my AD MA, I opened ADUC and tried to enable the user.  No luck.  I got the error message about the password not meeting the policy requirements.  I could however manually set the password to the value I was flowing from FIM, and then I could enable the user.  So that told me it wasn’t an AD rights issue, and that for some reason FIM wasn’t flowing the password to the new user at all.

After looking into all of the more common Kerberos issues (time sync, firewalls, DNS name resolution, etc.) I used Wireshark to capture the network traffic on the FIM Sync server during the export to AD.  The capture showed that the KPASSWD command was failing with “KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN”.

The details of the error showed that the DC could not locate the SPN “kadmin/changepw”.

But, running “setspn –Q kadmin/changepw” returned the correct AD account, “krbtgt”, without issue.  A little bit of searching led me to this KB article, which mentions an issue with “kadmin/changepw” when the “krbtgt” account was authoritatively restored in the past.  Though I didn’t catch it right away, it even says that this would cause an issue with ILM setting passwords on newly provisioned accounts.

So, I ran “repadmin.exe /showobjmeta DC1 cn=krbtgt,cn=users,dc=domain,dc=com” and lo and behold, the “krbtgt” account had been authoritatively restored in 2008.  A sample of the output is below.  Note the highlighted attribute versions that have been increased by 100,000.

 

Loc.USN                           Originating DSA  Org.USN  Org.Time/Date        Ver Attribute

=======                           =============== ========= =============        === =========

  14299      aa8e4db6-7d0e-45ae-95e9-82db0e59a327  34567640 2008-08-19 03:02:35 100001 objectClass

  14299               Default-First-Site-Name\DC1     14299 2010-12-04 18:12:51      1 cn

  14299      aa8e4db6-7d0e-45ae-95e9-82db0e59a327  34567640 2008-08-19 03:02:35 100002 description

  14299      aa8e4db6-7d0e-45ae-95e9-82db0e59a327  34567640 2008-08-19 03:02:35 100001 instanceType

  14299      2f2a1f4c-eac1-404a-b305-37fd2e28eddf      1479 2002-03-13 20:27:04      1 whenCreated

So, the hotfix from the KB article was installed on the DCs, and the issue was resolved.  The hotfix is included in Server 2008 R2 SP1, so definitely go that route if you can.  New users provision with the proper password and are properly enabled on creation.  Hopefully you don’t run into this issue, but if you do, I hope this eases your pain.

Office 2010 x64 and SharePoint Datasheet Views

It turns out that you cannot switch a SharePoint list to Datasheet View if you are running Office 2010 x64.  The required components are not part of the Office 2010 x64 install.  Various postings had you verify if Access was installed.  I didn't have it installed, added it, and still couldn't switch to Datasheet View.  To add the required components, simply install the 2007 Office System Driver: Data Connectivity Components.

IE9 RTM and Citrix ICA files

I ran various IE9 betas and RCs without issue when it came to opening and launching Citrix ICA files.  Once I installed the RTM version however, they failed to open.  The file would download, and then just sort of hang.  If I went to the site with Firefox, I had no issues.  Today I learned that to resolve this, you simply need to uninstall "Citrix online plug-in - web", and then reinstall the web client when prompted through your Citrix portal.  Much better now...

Viewing Server 2008 R2 Roles shows "Error"

Two posts in one day.  That might be a new record for me.

I had a 2008 R2 server that wouldn't install Roles properly via PowerShell.  When viewing the Roles with Server Manager, it simply said "Error".  How useful.

Looking in the event log I found this:

Log Name:      Application
Source:        Application Error
Date:          12/30/2010 11:18:47 AM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      server.domain.loc
Description:
Faulting application name: TrustedInstaller.exe, version: 6.1.7600.16385, time stamp: 0x4a5bc4b0
Faulting module name: ntdll.dll, version: 6.1.7600.16559, time stamp: 0x4ba9b802
Exception code: 0xc00000fd
Fault offset: 0x0000000000051ae3
Faulting process id: 0x9fc
Faulting application start time: 0x01cba845913dc3d6
Faulting application path: C:\Windows\servicing\TrustedInstaller.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: dc0dfe78-1438-11e0-8fd9-0050568900bb
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-12-30T17:18:47.000000000Z" />
    <EventRecordID>1935</EventRecordID>
    <Channel>Application</Channel>
    <Computer>server.domain.loc</Computer>
    <Security />
  </System>
  <EventData>
    <Data>TrustedInstaller.exe</Data>
    <Data>6.1.7600.16385</Data>
    <Data>4a5bc4b0</Data>
    <Data>ntdll.dll</Data>
    <Data>6.1.7600.16559</Data>
    <Data>4ba9b802</Data>
    <Data>c00000fd</Data>
    <Data>0000000000051ae3</Data>
    <Data>9fc</Data>
    <Data>01cba845913dc3d6</Data>
    <Data>C:\Windows\servicing\TrustedInstaller.exe</Data>
    <Data>C:\Windows\SYSTEM32\ntdll.dll</Data>
    <Data>dc0dfe78-1438-11e0-8fd9-0050568900bb</Data>
  </EventData>
</Event>

Generally, this seems to happen when the install process is interrupted (hard stopping a VM or a server losing power during an install).  Here are the steps taken to resolve the issue:

1. Installed the System Update Readiness Tool for Windows Server 2008 R2.  This tools runs when installed, so you don't have to actually run it.
2. View the resulting log file (%SYSTEMROOT%\Logs\CBS\CheckSUR.log)
3. Found the offending files:

Summary:
Seconds executed: 131
Found 1 errors
  CBS MUM Corrupt Total count: 1

Unavailable repair files:
    servicing\packages\Package_for_KB2207566_RTM~31bf3856ad364e35~amd64~~6.1.1.0.mum
    servicing\packages\Package_for_KB2207566_RTM~31bf3856ad364e35~amd64~~6.1.1.0.cat

4. Change permissions on the %SYSTEMROOT%\servicing\packages directory to allow administrators Modify rights.
5. Copy the offending files from a different Server 2008 R2 server to the %SYSTEMROOT%\servicing\packages directory on the problem server.
6. Reinstall the offending package (in my case KB2207566).
7. Re-run the SUR Tool (by re-installing it).
8. View the resulting log file (%SYSTEMROOT%\Logs\CBS\CheckSUR.log)
9. Verify the errors have been resolved:

Summary:
Seconds executed: 211
No errors detected

10. View Roles with Server Manager.

Properties Not Returned using Get-ADObject

Server 2008 R2 now includes Active Directory Web Services (ADWS), a new way to access AD information.  The new AD cmdlets available in Server 2008 R2 (and Windows 7 with the Remote Server Administration Tools installed), use these web services when accessing a remote domain controller.

I've been working on a script to automate the build process for remote domain controllers.  The steps include installing the AD DS role, creating the AD site, subnet, and site link, and then running dcpromo with an answer file.  As part of this, I came across an issue where a specific property was not being returned to me, but definitely did exist.  Using Get-ADObject and specifying the remote server, I was trying to get the site that a subnet was linked to, but it wasn't being returned.

Get-ADObject -Identity "CN=172.20.3.0/24,CN=Subnets,CN=Sites
,CN=Configuration,DC=domain,DC=loc" -server 2008r2dc.domain.loc:3268
-properties siteObject | fl

Yielded the following results:

DistinguishedName : CN=172.20.3.0/24,CN=Subnets,CN=Sites,CN=Configuration,DC=domain,DC=loc
Name              : 172.20.3.0/24
ObjectClass       : subnet
ObjectGUID        : 4c189c9f-ae09-4967-be81-ecf1dd293444

Notice that the "siteObject" property I specifically requested in the cmdlet is not there.  As it turns out, in troubleshooting connectivity to the remote server, I had specified the port of 3268 (global catalog) but never took it back out once I resolved my issue (which turned to be something completely unrelated).  Because I was querying the GC, that attribute was not present.  So, if the cmdlet is run without specifying the port, it queries over the default ADWS port of 9389 and the property is returned.

Get-ADObject -Identity "CN=172.20.3.0/24,CN=Subnets,CN=Sites
,CN=Configuration,DC=domain,DC=loc" -server 2008r2dc.domain.loc
-properties siteObject | fl

Yielded the following results:

DistinguishedName : CN=172.20.3.0/24,CN=Subnets,CN=Sites,CN=Configuration,DC=domain,DC=loc
Name              : 172.20.3.0/24
ObjectClass       : subnet
ObjectGUID        : 4c189c9f-ae09-4967-be81-ecf1dd293444
siteObject        : CN=Site001,CN=Sites,CN=Configuration,DC=domain,DC=loc

Hopefully my gaffe will be of help to someone else...