SAML Relaystate logging - issues with some users

[Richard Cowan1709159918]

I have a setup where they can come in from remote.example.com or myapps.example.com. The remote.example.com gets forwarded by dns to myapps.example.com. This then hits the Netscaler on firmware 12.1 62.25. On the latest release i added the new relaystate rule recommended by citrix on the saml policy (the netscaler is the SP and my1login is the idp)

 

the rule I added was:

AAA.LOGIN.RELAYSTATE.CONTAINS_ANY("samlaccess")

 

added patset saml access

bind patset samlaccess "https://myapps.example.com/"

bind patset samlaccess "https://remote.example.com/"

 

I tested and for me ti worked but the next day i had a select set of users getting the relaystate error message below:

"RelayState in Response does not match with rule in Action. Please contact your administrator"

 

Now i know i can use a chrome extn to see the relaystate but can this be found in the citrix adc logs anywhere? Id like to see what it failed on so I can modify the rule

 

[Rhonda Rowland1709152125]

May not be a factor, but try changing to AAA.LOGIN.RELAYSTATE.set_text_mode(ignorecase).Contains_Any("samlaccess")

As contains is case-sensitive by default and might be why some users aren't matching.

 

I'm not sure if relaystate is or isn't logged with the authentication events. But here are the typical log locations for info like this:

Hopefully, someone else can confirm.

 

If it is logged it would likely be here or here:

Syslog:

shell

cd /var/log

tail -f ns.log | grep -v CMD_EXEC

# use the tail to view events as they occur for a new login or switch to more without the -f parameter to view past events

 

See if the login events show in aaad.debug. This is a named pipe and not a log file; so you can view events as they occur but there is no history to look at.

shell

cd /tmp

cat aaad.debug

 

Separately in /var/log/ there are some auth log files and I don't know if they will or won't have these events.

 

If you still see nothing, some browser and saml viewer tool might be the easiest header trace to use.

 

 

Finally, the only other thing I can think of is that your saml policy (idp or sp though it might be under "more" section") may have a LOG ACTION.  

(NOTE: this is described broadly and you may need more info on; so I would do the other things first)

If you had a policy hit, then the log action can refer to a custom log message that might allow you to "echo" the aaa.login.relaystate value to the log message.

Data logs to syslog by default IF you enable "User Configurable Log Messages" in the syslog logging parameters.

 

The problem is, since you need the log details for the users this is not working, you would need to have the log action invoke for this error case or use 

Getting it to log for the user NOT in the patternset but without breaking other parts of the user transactions would be tricky without maybe making a test vserver for the "affected" users.

So I'm not sure this will really help you get the information; unless there really is no other way to do it---because its going to be messy.

 

 

 

[Richard Cowan1709159918]

Thanks for details on the ignore case - i had thought about that but checked the user was using lower case and then changed it to AAA.LOGIN.RELAYSTATE.EQ("https://myapps.example.com/") || AAA.LOGIN.RELAYSTATE.EQ("https://remote.example.com/") but no luck.

 

Ive set up a test tomorrow for the users that have issues and a couple who havent to add the chrome extension SAML Message Decoder - this gets me the relaystate although when i decrypt mine i get ns_policy=policynamehttps://myapps.example.com/

 

hoping i can find the smoking gun tomorrow. Thanks for the other info.

 

 

[Brooks Carlson]

FQDNs are almost always case insensitive to begin with.  Try testing one at a time to see if you can rule out anything:

 

AAA.LOGIN.RELAYSTATE.EQ("https://myapps.example.com")

 

Then test the other domain.  You might need to install Fiddler on your client machine and capture while attempting to authenticate.  From the capture look for doauth and find the SAML responses, then use a SAML decoder to examine the FQDN.  See if it compares with what you have in your rule.

[Rhonda Rowland1709152125]

Web browsers are case insensitive for hosts, but most policy expressions on the adc do case-sensitive matches unless you tell them not to.  

So, while dns and web browsers don't care if the host name is myapps.example.com vs. MYAPPS.example.com, the ADC policy expression would.

IF the ADC is comparing a request against "myapps.example.com" with contains(), equals(), contains_any() etc... it will do a case-sensitive match unless you tell it not to.  

 

But I would use a nstrace or header viewer utility to compare the client events and the adc events.

 

 

 

[Kevin Smith1709152583]

I am having this problem too.  Our IDP is Microsoft AzureAD and people are getting cursed with this error.  Clearing cache usually fixes the error, but it comes back the next day.  Did you figure this one out?  I will look at the case-sensitive stuff.

[Mark Boesveld1709157761]

We are having the same issues after the 12.1 62.25 firmware update and adding the RelayState Rule.

 

AAA.LOGIN.RELAYSTATE.EQ("https://portal.example.com/")

Some users are getting the "RelayState in Response does not match with rule in Action." error, this only seems to happen with Google Chrome users. 

When they switch to Edge they can login. This could be beacuse Edge has no cache of this site yet.

 

Edit: Can confirm it's also happening to Edge users,  clearing the cache seems to help.

[Kevin Smith1709152583]

It seems we can replicate the problem by letting the Gateway session expire and logoff the user from the portal.  At that point, if they try and relaunch the Gateway they get the error.  It happens in Chrome, IE11, Edge, Firefox. 

 

Setting the cookies for the URL to delete on browser close does "fix" the problem, but that really isn't how we want it to work as many of the computers accessing the Gateway are not managed by us so we have no control over how they manage their browser cookies.

[Rhonda Rowland1709152125]

Does anything in the traffic management policy/logout behavior help you with the cookie termination issue:
https://support.citrix.com/article/CTX133537

and

https://discussions.citrix.com/topic/373350-delete-cookies-and-redirect-back-to-login-page/

(This post also used a client-side code to assist.)

 

It may not be applicable to your scenario, but thought it might be relevant.

 

 

 

[Kevin Smith1709162826]

This suggestion from a support call seems to have fixed our issue so far.  Try this syntax.  Our IDP is AzureAD.  This is for the Citrix ADC Relay State Rule section on the Authentication SAML Server for our AzureAD connection using the GUI.

 

AAA.LOGIN.RELAYSTATE.EQ("https://example.com/") || (AAA.LOGIN.RELAYSTATE.LENGTH.EQ(0) && HTTP.REQ.HOSTNAME.EQ("example.com"))

 

[Gerrit Lansing]

On 7/1/2021 at 5:04 PM, Kevin Smith1709162826 said:

This suggestion from a support call seems to have fixed our issue so far.  Try this syntax.  Our IDP is AzureAD.  This is for the Citrix ADC Relay State Rule section on the Authentication SAML Server for our AzureAD connection using the GUI.

 

AAA.LOGIN.RELAYSTATE.EQ("https://example.com/") || (AAA.LOGIN.RELAYSTATE.LENGTH.EQ(0) && HTTP.REQ.HOSTNAME.EQ("example.com"))

 

We have tried this in our environment, but it did not help. Issue still remains.

We have an Internal and External facing ADC with the same SAML configuration. Both experience the same problem. External with Azure MFA, Internal is Trusted, so no Azure MFA triggered.

 

I also have tried to create a patset with just one url and then the rule "AAA.LOGIN.RELAYSTATE.CONTAINS_ANY("samlaccess") || (AAA.LOGIN.RELAYSTATE.LENGTH.EQ(0) && HTTP.REQ.HOSTNAME.EQ("portal.example.com"))", but that does not resolve it either, unfortunately.

 

[Mark Du Plessis1709152934]

On 7/1/2021 at 1:58 PM, Kevin Smith1709152583 said:

It seems we can replicate the problem by letting the Gateway session expire and logoff the user from the portal.  At that point, if they try and relaunch the Gateway they get the error.  It happens in Chrome, IE11, Edge, Firefox. 

 

Setting the cookies for the URL to delete on browser close does "fix" the problem, but that really isn't how we want it to work as many of the computers accessing the Gateway are not managed by us so we have no control over how they manage their browser cookies.

 

 

I discovered that on the 1st logon attempt the expression worked correct and the Relaystate value sent in the SAML post was correct "ns_policy=AuthServer_nFactornf=nFactor_level_X&https://www.companysite.com/"

 

But, once the Gateway session expires like the statement above, and the user re-authenticates again without clearing cookies/cache the Relaystate is now incorrect and is using the Storefront store name instead of the hostname URL in the post to the IDP and wont match the expression set on the Netscaler.

"ns_policy=AuthServer_nFactornf=nFactor_level_X&/StoreName/"

 

Where to look :

Using Dev tools in Chrome > Network tab - Settings : Preserve log ticked -- Recording On

Logon to your gateway, let the store timeout and log the user off.

Log back onto the gateway with the same auth method.

 

SAML request

In the Dev Tools window, filter for "post" and find the Relaystate: value at the bottom of the Form data ( right hand window)

Copy the value and paste it into the first window (Deflated and Encoded XML)  in https://www.samltool.com/decode.php - Click decode

 

SAML Response

In the Dev Tools window, filter for "samlauth" and find the Relaystate: value at the top of the Form data ( right hand window)

Copy the value and paste it into the first window (Deflated and Encoded XML)  in https://www.samltool.com/decode.php - Click decode

 

Compare the 1st and 2nd Relaystate values.

 

Doesn't fix any problems but shows why its going wrong

 

[Juergen Hopfes]

I tried this rule today : „AAA.LOGIN.RELAYSTATE.REGEX_MATCH(re#^https://www\.companysite\.com/#) || AAA.LOGIN.RELAYSTATE.REGEX_MATCH(re#StoreName#)“ and one MAC User reported, it fixes his problem with Safari, but I’m not sure if this is a valid and secure solution.
 

I had opend a Citrix Case and they found the same reason.
Citrix:“… I have analyzed the trace files. It seems that SAFARI browser is sending Relaystate ‘ns_policy=<Authentication SAML Server>?/Citrix/<StoreName>/’ which is causing the error ‘RelayState in Response does not match with rule in Action’ …”

[Nathan Thoenen]

Upgrading to 12.1-62.27 fixed this issue.

 

[Mark Boesveld1709157761]

Good to know. It's also mentioned in the fixed issues.
https://docs.citrix.com/en-us/citrix-adc/downloads/release-notes-12-1-62-27.html