WEM Agent - Failed to get local database parameters / Cannot get database instance id

[Paul Cross]

Hi All.

 

Just updated a customer to WEM 1912. Was just running a refresh cache for the WEM agent and got the following - Cache sync failed with error: SyncFailed

 

 

In the debug I can see this:

 

11:17:24 Warning -> [Sync Client (PID:8888)] ChechCacheSyncMode() : Failed to get local database parameters. Exception: System.ArgumentException: Cannot get database instance id.
   at Citrix.Wem.DBSync.Common.SyncDataExtension.GetDatabaseInstanceId(DbConnection connection)
   at Citrix.Wem.DBSync.Client.Agent.SyncTask.ChechCacheSyncMode()
11:17:24 Event -> [Sync Client (PID:8888)] CloseLocalConnection() : Close local cache database...
11:17:24 Event -> [Sync Client (PID:8888)] DoSync() : Remove existing agent local cache database...
11:17:24 Event -> [Sync Client (PID:8888)] DoSync() : Create agent cache sync task. Sync timeout: 300s
11:17:26 Exception -> [Sync Client (PID:8888)] DoSync() : Dotmim.Sync.SyncException: Server unabled to process sync request. ---> System.ServiceModel.FaultException: Server unabled to process sync request.
   at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
   at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.<>c__DisplayClass7_0`1.<CreateGenericTask>b__0(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Citrix.Wem.DBSync.Client.Agent.AgentSyncProxyProvider.<SendRequestAsync>d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Citrix.Wem.DBSync.Client.Agent.AgentSyncProxyProvider.<GetChangeBatchAsync>d__23.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Dotmim.Sync.SyncAgent.<SynchronizeAsync>d__35.MoveNext()
   --- End of inner exception stack trace ---
   at Dotmim.Sync.SyncAgent.<SynchronizeAsync>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Dotmim.Sync.SyncAgent.<SynchronizeAsync>d__35.MoveNext()
11:17:26 Event -> [Sync Client (PID:8888)] CloseChannel() : Closing remote connection...
11:17:26 Exception -> [Sync Client (PID:8888)] Main() : Citrix.Wem.DBSync.Common.DBSyncException: Error happened while synchronizing agent cache.
   at Citrix.Wem.DBSync.Client.Agent.SyncTask.DoSync(SyncMode mode)
   at Citrix.Wem.DBSync.Client.Agent.SyncTask.Run()
   at Citrix.Wem.DBSync.Client.Agent.SyncExecutor.Run()
   at Citrix.Wem.DBSync.Client.Agent.Program.Main(String[] args)
11:17:26 Exception -> SyncCache() : Cache sync failed with error: SyncFailed

 

 

I have tried deleting the local databases which then get recreated but still get the same error. 

 

Anyone got any suggestions?

 

Thanks, Paul.

 

[Kasper Johansen1709159522]

Hi,

 

Maybe this will help you out:

https://twitter.com/citrixguyblog/status/1210940328859099136

[Paul Cross]

That's sorted it. Thanks Kasper.

[Kasper Johansen1709159522]

No problem Paul. I have seen this issue a few times now, hopefully Citrix will have it sorted out in the next release.

 

Funny thing is that a clean install does not have this issue, it's only upgrades. 

[Damien Baldwin1709160993]

I'm having the same issues as this, yet I have added my WEM service account to the DBSync dir (in fact the whole of "\Norskale Infrastructure services\" dir - full control). I'm still getting a Red Cross for "Synchronization State"? If I revert to 1909, all works fine. I've tried an upgrade of the client (1909 -> 1912) and an uninstall of 1909, install of 1912.

I notice that the client "local databases" dir has .db files rather than .sdf files (localagentcache.db/sdf + localagentdatabase.db/sdf).

 

Any ideas please? Pretty std upgrade/install? Come on Citrix, what the hell?

 

Debuglog:

 

Agent Service Version: 1912.1.0.1
08:53:20 Event -> ConfigurationDataSourcesHelper.CheckBrokerSvcConnection() : Broker Svc Check -> OK
08:53:20 Event -> StatisticsUpdater.() : Detected connection security -> Fallback
08:53:20 Event -> MainServiceHost.ReportLegacyAgentWcfService() : Successfully updated agent callback information.
08:53:20 Event -> MainServiceHost.StartAgentWcfService() : Agent Command Processor service started
08:53:20 Event -> [Sync Client (PID:6992)] OpenOnpremConnection() : Kerberos authentication not successful. Try Ntlm instead...
08:53:20 Event -> [Sync Client (PID:6992)] CloseChannel() : Closing remote connection...
08:53:20 Event -> [Sync Client (PID:6992)] OpenNtlmConnection() : Try open connection with Ntlm authentication...
08:53:21 Event -> VuemAgentServiceConfigurationHelper.TryUpdateAgentStatistic() : Agent statistic update (InstanceGuid: abb25793-c57e-45c5-aef9-929e022b0563) -> Success
08:53:21 Event -> [Sync Client (PID:6992)] OpenNtlmConnection() : Connected to infrastructure server. Authentication method: Ntlm
08:53:21 Event -> [Sync Client (PID:6992)] OpenLocalConnection() : Open local cache database...
08:53:22 Warning -> [Sync Client (PID:6992)] ChechCacheSyncMode() : Failed to get local database parameters. Exception: System.ArgumentException: Cannot get database instance id.
   at Citrix.Wem.DBSync.Common.SyncDataExtension.GetDatabaseInstanceId(DbConnection connection)
   at Citrix.Wem.DBSync.Client.Agent.SyncTask.ChechCacheSyncMode()
08:53:22 Event -> [Sync Client (PID:6992)] CloseLocalConnection() : Close local cache database...
08:53:22 Event -> [Sync Client (PID:6992)] DoSync() : Remove existing agent local cache database...
08:53:22 Event -> [Sync Client (PID:6992)] DoSync() : Create agent cache sync task. Sync timeout: 300s
08:53:24 Event -> RegistryConfigurationMonitor.InternalRun() : Group Policy Watcher Started
08:53:25 Exception -> [Sync Client (PID:6992)] DoSync() : Dotmim.Sync.SyncException: Server unabled to process sync request. ---> System.ServiceModel.FaultException: Server unabled to process sync request.
   at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
   at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.<>c__DisplayClass7_0`1.<CreateGenericTask>b__0(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Citrix.Wem.DBSync.Client.Agent.AgentSyncProxyProvider.<SendRequestAsync>d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Citrix.Wem.DBSync.Client.Agent.AgentSyncProxyProvider.<GetChangeBatchAsync>d__23.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Dotmim.Sync.SyncAgent.<SynchronizeAsync>d__35.MoveNext()
   --- End of inner exception stack trace ---
   at Dotmim.Sync.SyncAgent.<SynchronizeAsync>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Dotmim.Sync.SyncAgent.<SynchronizeAsync>d__35.MoveNext()
08:53:25 Event -> [Sync Client (PID:6992)] CloseChannel() : Closing remote connection...
08:53:25 Exception -> [Sync Client (PID:6992)] Main() : Citrix.Wem.DBSync.Common.DBSyncException: Error happened while synchronizing agent cache.
   at Citrix.Wem.DBSync.Client.Agent.SyncTask.DoSync(SyncMode mode)
   at Citrix.Wem.DBSync.Client.Agent.SyncTask.Run()
   at Citrix.Wem.DBSync.Client.Agent.SyncExecutor.Run()
   at Citrix.Wem.DBSync.Client.Agent.Program.Main(String[] args)
08:53:25 Exception -> SyncCache() : Cache sync failed with error: SyncFailed
 

[Kasper Johansen1709159522]

Hi,

1. Stop the Citrix WEM Agent service or the Norskale Agent Host Service (The Netlogon service stops too)

2. Delete any files in the C:\Program Files (x86)\Norskale\Norskale Agent Host\Local Databases folder or C:\Program Files (x86)\Citrix\Workspace Environment Management Agent\Local Databases (There should only be .db or .sdf files in that folder)

3. Start Citrix WEM Agent service or the Norskale Agent Host Service (Remember to start the Netlogon service as well)

4. Execute (Runas administrator) AgentCacheUtility.exe -refreshcache (Located in either C:\Program Files (x86)\Norskale\Norskale Agent Host or C:\Program Files (x86)\Citrix\Workspace Environment Management Agent)

 

Let me know how it works out.

[Damien Baldwin1709160993]

22 minutes ago, Kasper Johansen1709159522 said:

Hi,

1. Stop the Citrix WEM Agent service or the Norskale Agent Host Service (The Netlogon service stops too)

2. Delete any files in the C:\Program Files (x86)\Norskale\Norskale Agent Host\Local Databases folder or C:\Program Files (x86)\Citrix\Workspace Environment Management Agent\Local Databases (There should only be .db or .sdf files in that folder)

3. Start Citrix WEM Agent service or the Norskale Agent Host Service (Remember to start the Netlogon service as well)

4. Execute (Runas administrator) AgentCacheUtility.exe -refreshcache (Located in either C:\Program Files (x86)\Norskale\Norskale Agent Host or C:\Program Files (x86)\Citrix\Workspace Environment Management Agent)

 

Let me know how it works out.

 

Hi Kasper, thanks for your help.

I had already tried that solution, but reran it anyway and its still failing...

[Damien Baldwin1709160993]

I notice that the Cached Data Sync Port has changed to 8288 (was 8286)... I will investigate this with F/w team

[Paul Cross]

3 minutes ago, Damien Baldwin1709160993 said:

I notice that the Cached Data Sync Port has changed to 8288 (was 8286)... I will investigate this with F/w team

 

Yes this caught me out as well originally. Added an additional LB on port 8288 then applied the permissions change and all was good.

[Damien Baldwin1709160993]

Telnet works on 8288+8285 (the old port - not 8286 as said previously), so its not f/w issue. 

[Daniel Litschewsky]

had the same problem. needed to give the wem service user db owner rights (insted of only datareader data writer)

[Damien Baldwin1709160993]

2 hours ago, Daniel Litschewsky said:

had the same problem. needed to give the wem service user db owner rights (insted of only datareader data writer)

Thanks Daniel. Yes it turned out to be DB permissions. So it looks like previous installs set the service acc to read/write and this upgrade doesnt amend permissions - not only to DB, but also to the DBSync dir. 

Not a great upgrade without prior knowledge.

[Andrei Vlad1709158202]

Hi guys,
I can confirm that adding dbowner and setting full permissions on the DBSync folder fixes this - and also i can confirm this has happened once i upgraded from 1909 to 1912.

However, apparently they have fixed this in the 2003 release?

 

https://docs.citrix.com/en-us/workspace-environment-management/current-release/fixed-issues.html -> [WEM-8008]

[Josh steiner1709160356]

Just had the same issue upgrading from 1811 to 2006. You'll also see this in the Norskale event logs: