Session reliability suspended the connection for user domain\user, session 5. source rpm

[Iliya Iliev]

In the past few days I have been getting kicked off my xendesktop  and xenapp pretty much momentarily as they open/launch. 

Event viewer shows error Session reliability suspended the connection for user domain\user , session 5 and the source is listed as rpm.

 

Has anyone ran into this issue before? Everything seemed to worked fine until this point. 

Also note that this isnt happening with my test user account, but it did happen to another user earlier this morning. 

 

I have two Delivery controllers and two VDA server hosts 2019 server standard with a separate machine for the DB. 

I run ICAClient.x86_64-19.6.0.60-0.rpm package on CentOS 7.6, this is basically the standard for our environment. 

 

Can anyone recommend a good way to approach issues like these such as apps disappearing/closing out of nowhere? I usually try director but It never reports these session closes unfortunately. Currently i am testing this environment and plan on deploy into production soon to about 50 users but with one off problems like these I think i will be creating myself more headaches then making things more efficient. 

 

Thanks,

 

 

[Balaji Muthukrishnan]

In my case, all the affected sessions are showing as disconnected in studio with "Current user" as Blank. if I check the session state by command line using get-brokersession cmdlet...I can see -LogonInProgress parameter is set to "True" for all the affected sessions., whereas for the other disconnected sessions -LogonInProgress parameter is set to "False".

But I am seeing similar event id of 10 and 12 with event source as "RPM". When I login to HSD to check the running process for the user session, i can see only very few process (Runtime process,User mode font drive host, citrix graphics, windows logon Application and Windows Logon User Interface Host).

 

Tried killing the LogonUI process for the user session (Windows Logon User Interface Host), and session gracefully logoff.  From the scenario, so far, when a user tries to login for some reason session is broken, after certain timeout, studio marks the session as disconnected, but controller thinks, session is still in logon phase, probably thats the reason -LogonInProgress parameter is still set to "True".  So far, I am manually killing the process LogonUI to fix the issue.

 

I am correlating to KBs for the issue, which I am yet to try...

 

https://support.citrix.com/article/CTX232490    - After Installing VDA 7.15 on Windows 2016, Unable to Reconnect to Existing Session or Start a New Session

https://support.citrix.com/article/CTX286701   -  Users getting disconnected while working in the published applications

 

 

[Balaji Muthukrishnan]

On 4/2/2021 at 1:28 PM, Balaji Muthukrishnan said:

I have applied the fix suggested in  (https://support.citrix.com/article/CTX232490    - After Installing VDA 7.15 on Windows 2016, Unable to Reconnect to Existing Session or Start a New Session ) CTX KB in the OS layerbut it didnt fix the issue. I am still seeing ghost disconnected sessions

after spending a lot of time in collecting memory dump and cdf montior traces. support came up with few findings on this issue...

 

1. could be due to user trying to do multiple clicks on the receiver for web SF Application/Desktop shortcuts, causing one of the earlier launched session to fail and goes in to disconnect/Ghost session - extend multiple click timeout using https://support.citrix.com/article/CTX236169

 

2. Asusual AV or Security tools causing, revisit on the Exclusions (In Mycase  Trend, Tanium)

 

am still not seeing a way out on this issue...

[Balaji Muthukrishnan]

please refer the post https://discussions.citrix.com/topic/356095-disconnected-ghost-sessions-with-4-processes/page/2/#comment-2081552

and add the AV process to LogoffCheckSysModules and Sysprocs to see if the issue gets resolved. Its working for me and i dont see issues since adding the registry values.

 

Thanks

[Akshay Mane1709160863]

You can disable session reliabilty and enable ica keep alive using Citrix virtual desktop Computer policy --> Goto policies-->User policy.

You can set apply policies to either group of users or group of servers by creating a OU's.

Session Reliabilty:

Session Reliability keeps sessions active when network connectivity is interrupted. Users continue to see the application they are using until network connectivity resumes.

ICA Keep Alives:

Enabling this setting prevents broken connections from being disconnected. If XenApp detects no activity, this setting prevents Remote Desktop Services from disconnecting the session. XenApp sends keep-alive

messages every few seconds to detect if the session is active. If the session is no longer active, XenApp marks the session as disconnected.

*Note:* ICA Keep-Alive does not work if you are using Session Reliability. Configure ICA Keep-Alive only for connections that are not using Session Reliability.

 

 

Regards,

Akshay

[Björn Schläfli]

Hi,

Same issue with Virtual Apps 1912 on server 2016, but only with Igel Thin Clients. Remote connections through Gateway and direct connections are not affected.

Updated from XenApp7.15.

This issue seems to be related to VDA 1912.

Before I’ve updated the site, I’ve updated only VDA to 1912 and used it with the 7.15 site and had the same problem.

Disable Session Reliability doesn‘t help.

Now I‘m testing VDA 1906.2. So far I‘m able to say the rpm events are not logged anymore. I‘m waiting for user‘s feedback.

 

found others with similar issues, but with VDA 1811 and 1906.

https://discussions.citrix.com/topic/400948-xa-1811-server-2012r2-pvs-1811-disconnects/

https://discussions.citrix.com/topic/403711-vda-1906-windows-10-1809-massiv-random-disconnects/
 

any ideas?

 

[Nicolas Crozet]

Hi Xander82,

 

How is it going now with 1906.2? do you still have the same disconnections? 

 

Thank you for your feedback,

[Björn Schläfli]

Hi,

 

1906.2 is working well. I‘ve tested also VDA 1912 CU1 and 1912 CU1 with rpm.dll from 1906.2, both have disconnects.

 

I have no clue what´s the cause.

[Mike Kelly1709153237]

On 6/11/2020 at 7:25 AM, Björn Schläfli said:

Hi,

 

1906.2 is working well. I‘ve tested also VDA 1912 CU1 and 1912 CU1 with rpm.dll from 1906.2, both have disconnects.

 

I have no clue what´s the cause.

We are seeing this on our Server 2016 deployments with the 1912 CU1 VDA.  We are currently rolling back to 7.15.5000 VDA where we did not experience this issue.  

 

Did you ever figure out this issue?  I have opened a support ticket with Citrix on this.

[Björn Schläfli]

no, sadly I didnt. I use VDA 1906.2 for the moment. As I can't reproduce it, I didn't open a support case.

[Mike Kelly1709153237]

7 hours ago, Björn Schläfli said:

no, sadly I didnt. I use VDA 1906.2 for the moment. As I can't reproduce it, I didn't open a support case.

What version is your site database at?  Our TRM thinks it could be related to running disparate versions between the database and the VDA.  We are upgrading from 7.15.5000 to 1912 CU1 on the site database next week... so I'll test it again at that time.

[tylital520]

we are also seeing these errors on a VDA's event viewer: "Session reliability suspended the connection for user xxxx\user1, session 10" Source: rpm, EventID 10

 

When a user tries to launch an application or desktop she/he gets a disconnected notification instead, and a counter which runs for 5 minutes, but the application/desktop never opens.  Director shows that the user has the application open, and if I look at the Connection Center from Workspace App it shows the application name, but not the server name. The application/desktop finally opens when it's clicked 2 or 3 times, but sometimes it also opens right-away.

 

Virtual Apps and Desktops 1912

VDA 1912

4 delivery controllers: 2 x Windows Server 2012 R2 and 2 x Windows Server 2019 (we are in the process of removing the 2012 R2 instances)

2 Storefront-servers: Windows Server 2019

Separate DB server: Windows 2019 + Microsoft SQL Server 2017

 

Application server: Windows Server 2012 R2

Desktops: Windows 10

[Björn Schläfli]

Hi tylital520,

 

try to use VDA 1906.2000 instead and test if the issue is fixed with it. I'm using 1906.2000 with no issues a few months now.

 

Last week one of my co-workers installed Citrix Workspace App 1907 on one of our Igel Thin Clients, while our site was still XenApp 7.15 CU1 with server VDA 1906.2000 and the issue also happened. So it seems if either Workspace App or VDA are on a specific level the disconnects happen. For VDa the level has to be 1909 and up. We reinstalled Citrix Receiver 4.9 on the updated Igel and the disconnects stopped.

 

I cannot find any clues what new feature in VDA and Workspace App could be the culprit.  

[Christoph Sinabell]

Hi,

 

Same issue here using VDA 2006 (running on W2K16) and Workspace App 20.6.0.38. Site DB still on 7.15 CU4

 

Regards

[tylital520]

On 8/24/2020 at 12:38 PM, tylital520 said:

we are also seeing these errors on a VDA's event viewer: "Session reliability suspended the connection for user xxxx\user1, session 10" Source: rpm, EventID 10

 

When a user tries to launch an application or desktop she/he gets a disconnected notification instead, and a counter which runs for 5 minutes, but the application/desktop never opens.  Director shows that the user has the application open, and if I look at the Connection Center from Workspace App it shows the application name, but not the server name. The application/desktop finally opens when it's clicked 2 or 3 times, but sometimes it also opens right-away.

 

Virtual Apps and Desktops 1912

VDA 1912

4 delivery controllers: 2 x Windows Server 2012 R2 and 2 x Windows Server 2019 (we are in the process of removing the 2012 R2 instances)

2 Storefront-servers: Windows Server 2019

Separate DB server: Windows 2019 + Microsoft SQL Server 2017

 

Application server: Windows Server 2012 R2

Desktops: Windows 10

 

I forgot to mention that we are using ADC VPX 12.1 56.22 and all of our connections go through that, so no direct StoreFront access for our end users.

[Björn Schläfli]

We are using direct connections.

[Mike Kelly1709153237]

On 8/24/2020 at 3:38 AM, tylital520 said:

we are also seeing these errors on a VDA's event viewer: "Session reliability suspended the connection for user xxxx\user1, session 10" Source: rpm, EventID 10

 

When a user tries to launch an application or desktop she/he gets a disconnected notification instead, and a counter which runs for 5 minutes, but the application/desktop never opens.  Director shows that the user has the application open, and if I look at the Connection Center from Workspace App it shows the application name, but not the server name. The application/desktop finally opens when it's clicked 2 or 3 times, but sometimes it also opens right-away.

 

Virtual Apps and Desktops 1912

VDA 1912

4 delivery controllers: 2 x Windows Server 2012 R2 and 2 x Windows Server 2019 (we are in the process of removing the 2012 R2 instances)

2 Storefront-servers: Windows Server 2019

Separate DB server: Windows 2019 + Microsoft SQL Server 2017

 

Application server: Windows Server 2012 R2

Desktops: Windows 10

 

What version of Citrix Receiver / Citrix Workspace app are you running on the workstations?

[Mike Kelly1709153237]

On 8/24/2020 at 3:38 AM, tylital520 said:

we are also seeing these errors on a VDA's event viewer: "Session reliability suspended the connection for user xxxx\user1, session 10" Source: rpm, EventID 10

 

When a user tries to launch an application or desktop she/he gets a disconnected notification instead, and a counter which runs for 5 minutes, but the application/desktop never opens.  Director shows that the user has the application open, and if I look at the Connection Center from Workspace App it shows the application name, but not the server name. The application/desktop finally opens when it's clicked 2 or 3 times, but sometimes it also opens right-away.

 

Virtual Apps and Desktops 1912

VDA 1912

4 delivery controllers: 2 x Windows Server 2012 R2 and 2 x Windows Server 2019 (we are in the process of removing the 2012 R2 instances)

2 Storefront-servers: Windows Server 2019

Separate DB server: Windows 2019 + Microsoft SQL Server 2017

 

Application server: Windows Server 2012 R2

Desktops: Windows 10

 

What version of Citrix Receiver / Citrix Workspace app are you running on the workstations?

[Mike Kelly1709153237]

We had this running for several days with no user reports of any issues and I'm leaning towards this as a 'benign' issue.  What I've seen thus far is that the eventid's for rpm with the message "Session reliability suspended the connection for user domain\user , session 5" they only seem to show up when a user DISCONNECTS or LOGSOFF.  They also don't seem to be user impacting as even on a disconnect when we see these events, the user is able to reconnect to the exact same session where rpm recorded that event.    We are still running Receiver 4.9.6001 on the workstations and are targeting a CWA upgrade in Q4.  We've rolled back the VDA to 7.15 CU5 on all of our images except for 1 small deployment so we can use it to troubleshoot with Citrix.  Citrix is requesting CDF control logs from the workstation, VDA, application event logs from the workstation and Delivery controller CDF's all encapsulating the time of the issue.    For the one user reporting they are running VDA 2006 with Server 2016 and Workspace app 2006.1, are your users actually reporting a problem?

[tylital520]

5 hours ago, Mike Kelly1709153237 said:

 

What version of Citrix Receiver / Citrix Workspace app are you running on the workstations?

We have various versions, but I am running Workspace App 2006.1 and I am seeing the problem.

[Mike Kelly1709153237]

11 hours ago, tylital520 said:

We have various versions, but I am running Workspace App 2006.1 and I am seeing the problem.

Your issue seems to be presenting differently than mine.  My users are able to connect and work without issue and the eventid's only present on disconnect / logoff and even then its random, not every user can repro.  Are you seeing this as well or or all of your users seeing the disconnect popup like you described in a previous post?  

[tylital520]

13 hours ago, Mike Kelly1709153237 said:

Your issue seems to be presenting differently than mine.  My users are able to connect and work without issue and the eventid's only present on disconnect / logoff and even then its random, not every user can repro.  Are you seeing this as well or or all of your users seeing the disconnect popup like you described in a previous post?  

Not all of our users are experiencing this, and it does happen randomly. But it occurs when a user launches an application/desktop, not when they logoff/disconnect.

 

I disabled DTLS yesterday from our gateway vServer and I am no longer seeing this issue.

[Mike Kelly1709153237]

7 hours ago, tylital520 said:

Not all of our users are experiencing this, and it does happen randomly. But it occurs when a user launches an application/desktop, not when they logoff/disconnect.

 

I disabled DTLS yesterday from our gateway vServer and I am no longer seeing this issue.

Do all of your users connect in through the Citrix Gateway?  The majority of my users are internal which bypass the gateway and connect directly to Storefront.

 

I have also tried disabling HDX Adaptive Transport but the eventid's still occur.

[tylital520]

On 8/27/2020 at 5:01 PM, Mike Kelly1709153237 said:

Do all of your users connect in through the Citrix Gateway?  The majority of my users are internal which bypass the gateway and connect directly to Storefront.

 

I have also tried disabling HDX Adaptive Transport but the eventid's still occur.

Yes all our user access through gateway. I noticed that we still get them eventid's but the disconnect problem is gone.

[Sebastien Dubrulle]

Hey,

 

Did you tryed this?

 

https://support.citrix.com/article/CTX200841

 

regards

[Rajuan Kimble]

@tylital520 I have a customer with a similar issue. Have you been able to solve this issue? If so, how? Also, when this issue occurs, are users connecting through ADC?