Storefront LoadBalancer ICA Proxy Selfmade DDoS IGEL THIN-Client

[Kai Schoenbach]

Hey Guys,

we are facing an issue with Citrix ADC: ICA-Proxy and Storefront Load-Balancer.

Environment Infos:

Citrix ADC VPX 13 47.22 2-Node HA-Cluster in DMZ (Same Problem with Version 12.x)

Citrix XenServer

Storefront Load-Balancer VServer

DDC Load Balancer

LDAPS Load Balancer

Several ICA-ONLY Gateways (using all LB-Server)

Xen Desktop 7.9 Terminalserverfarm

1000+ IGEL THIN-Client mit Linux Receiver or Workspace-App for Linux

1000+ Windows-Clients with Workspace-App or Receiver

As soon as we configure the IGEL-Thin-Clients for the Storefront-Load-Balancer Server, the active ADC-Node suffers from massive tcp Connections every ~30 Seconds. The requested IP is the VIP from the Storefront -Loadbalancer (172.29.59.40:443). The serious consequence is an cpu-spiking, high adc-internal latency (Ping NS-Node-IP==>VIP) and catastrophic ica-latency and RTT (5000ms+).

The biggest question is why there are so many simultaneous connections every 30 seconds? As soon as we configure the IGEL-Clients for a single-Storefront internal-server everything seems to be fine, no more latency issues.

 

i am attaching  a screenshot from our firewall.

 

Any ideas?

 

Thanks in advance

 

Kai

 

 

[Rhonda Rowland1709152125]

While it might not be related to your issue:

What is your storefront lb vserver, lb method, persistence,  and monitor?

What is your controller/xml lb vserver lb method, persistence, and monitor?

 

As these are internal thin clients, the assumption is that this is an internal non-gateway connection and the igels are talking to the Storefront only.  

How many resources are allocated to your vpx and how many clients are connecting at one time?

do you have any other features enabled such as appfw, appflow or other?

 

Are you seeing any indications of a netscaler core dump or crash or entity outages when this is occurring?

[Kai Schoenbach]

@Rhonda Rowland:

As these are internal thin clients, the assumption is that this is an internal non-gateway connection and the igels are talking to the Storefront only.  ==>YES

How many resources are allocated to your vpx and how many clients are connecting at one time? 8Cores, 6GB RAM,

do you have any other features enabled such as appfw, appflow or other? ==> NO

 

But, in the meantime we have switched all clients to a normal Windows storefront server. The server is and massive load and has not been able to accept any new connections in the meantime. Windows Warning TCPIP 4227
We see thousands of connections during the day. The TCP connections do not seem to be closed in time or stay open for too long with the status "Time Wait".

Our IGEL-Thin Clients still connecting to the storefront server every 30 seconds.

The attachted screenshot shows 5000+ Connections. This morning, when users started there machines there are over 17000 connections.

 

The targeted remote-Hosts are delivery controllers from several attached farms.

 

 

[Rhonda Rowland1709152125]

On 12/20/2019 at 12:49 PM, Rhonda Rowland1709152125 said:

While it might not be related to your issue:

What is your storefront lb vserver, lb method, persistence,  and monitor?

What is your controller/xml lb vserver lb method, persistence, and monitor?

 

I would still double check your load balancing config in case there is something unexpected there.  As the persistence is key here.

Do the igel clients trust the cert of the load balancer, as well?

 

However, if via load balancer IGEL connections behave badly and directly they don't,  you may need to contact support (either igel, citrix, or both)

But you could also try a nstrace to see if anything else weird is going on during hte connection attempt.

And try a non-igel client via the storefront lb to see if it has same behaviors or works, this may help isolate NS-issue vs igel-specific issue.