Consider a user setting a password such as ‘ UNION TABLE spaceships; This may look like an SQL Injection, but it’s actually a very good password.
It is fairly long, it has capital and lowercase letters, it comes with special characters, etc.
And it should not even be a security problem when the developers did the right thing and hashed the passwords that they store in the database. This solves the potential SQL injection problem here since the password above would also be hashed for safe use via SQL. So, what’s not to love here?
CRS begs to differ! CRS does not like this at all. CRS rule 942190 identifies this as “Detects MSSQL code execution and information gathering attempts”.
While we as humans can easily conclude this alert is a false positive in this context, CRS cannot possibly tell the difference, so we need to teach it: we need to tune this false positive away.
In ModSecurity or CRS speak, we need to write a rule exclusion. This sounds more complicated than it is, but you need to understand the concept, or you won’t understand the procedure. So, let’s do the theory first.
CRS is regularly updated. If we start to edit the CRS themselves on the local LoadMaster, we could no longer perform said update, or our changes would be overwritten.
So, we avoid making any change to the Core Rule Set, but we add instructions to the Web Application Firewall (WAF) that tell the engine how to manipulate the ruleset so the false positive disappears. In ModSecurity / CRS speak, we call this a rule exclusion.
Tuning a false positive away means writing a rule exclusion.
As the name implies, we instruct the WAF engine to exclude a certain rule from being applied. There are different ways to write or express a rule exclusion and we’ll try them out one by one.
In many situations, namely the trickier ones, it is very helpful if you can reproduce a request or a ModSecurity / CRS alert at will. For this test, we will use a post request, to trigger the 942190 rule again as outlined below.
There are two ways to reference the Virtual Service running WAF:
Test Method 1: Direct reference to the Virtual Service IP Address
curl -X POST http://<<Virtual Services IP Address>>/submit-login -d "username=Christian&password=' UNION TABLE spaceships;"
Test Method 2: Reference to Virtual Service IP Address via hostname
curl -X POST http://blog-4/submit-login -d "username=Christian&password=' UNION TABLE spaceships;"
I am using an Ubuntu system and have configured /etc/hosts to map the Virtual Service IP Address to blog-4 as shown in Figure 1.
If you are using test method 1, you will see a second rule, 920350 identifying as “Host header is a numeric IP address”, is triggered. For the exercise, we will disable this rule in example 1.
So, we can now trigger the alert, 942190. The next step is to tweak the WAF, so this request no longer triggers an alert (a false positive).
The LoadMaster WAF is sitting before an Apache server. There is no special configuration of the Apache application, it is a default installation. The LoadMaster WAF is running at Paranoia Level set to 1 and Anomaly Scoring Threshold set to 3.
Running a successful test, will pass through the LoadMaster WAF to the Apache server and result in a 404 error as expected.
I will quickly deal with direct Virtual Service IP Address in the curl command to trigger the CRS, test method 1 above. If you have used this, then click Disable Rule under Web Application Firewall – False Positive Analysis for this Virtual Service, as shown in Figure 2.
Then, repeat the test method 1 curl command and you are ready to proceed.
The LoadMaster UI supports disabling a rule completely. With this method, a rule is literally removed from the memory of the engine. Naturally, this is a very effective method to disable rules. It’s also a very coarse approach to the problem since the rule is not only disabled for the password parameter on the login form, but the entire virtual service. I call this the “shotgun approach” to rule exclusions.
Loadmaster allows me to do approach this three ways.First Approach:
Navigate to Manage Rules section in the WAF configuration of the Virtual Service (Virtual Services – View/Modify Services, select the specific Virtual Service under test). There you click on the correct rule group’s name/category’s name to get the full list of rules per group. 942190 is part of the group “application-attack-SQL“. All rules in the 942xxx range are shown, scroll down and uncheck the checkmark in front of the 942191 rule id and click on the “Apply” button, as shown in Figure 3.
Repeat the curl request and it will successfully transit the LoadMaster WAF to the Apache application.
Second Approach:
The second approach to disable a rule is to navigate to Web Application Firewall – False Positive Analysis for this Virtual Service. This opens a view, where you can examine recent rule alerts. There is a counter for the individual rule alerts that include 942190. There is also a list of disabled rules and then a histogram that is organized by the Anomaly Score of the requests as shown in Figure 5.
So, if a request accumulated an anomaly score of 15 points, you would see the three rules that make up these 15 points next to it. Our request triggered 8 points if you called the WAF by its Virtual Service IP address (test method 1) or 5 points if you used a hostname (test method 2). You can disable it by clicking on “Disable Rule” as shown in Figure 5.
The effect will be immediate. Repeat the curl request and it will successfully transit the LoadMaster WAF to the Apache application.
So, these are the two rule exclusion approaches that the LoadMaster UI supports.
Third Approach:We can use a custom rule upload with the ModSecurity directive SecRuleRemoveById. In the background, this is the action that the LoadMaster UI is taking when you de-select the rule (First Approach) or disable the rule (Second Approach).
Let’s write the following into a file named rule_exclusions_method1.conf:
# ModSec Rule Exclusion: 942190 : Detects MSSQL code execution and # information gathering attempts SecRule REQUEST_URI "@beginsWith /submit-login" \ "phase:1,nolog,pass,id:10000,ctl:ruleRemoveById=942190"
Advice: Always accompany rule exclusions with a brief comment on what you are doing and what the rule is all about since there are probably more rule IDs around than you can memorize.
So, insert the command below into a custom rule file, upload, and activate it. The full details of how to upload and enable custom rule files in LoadMaster are outlined in blog #3, ‘Deploying custom rules on LoadMaster’ available here.
The effect will be immediate. Repeat the curl request and it will successfully transit the LoadMaster WAF to the Apache application.
Disabling a rule completely to guarantee a single benign use of a suspicious parameter is no longer being targeted by an overzealous rule, could be a bad practice since it opens a bigger hole in the defense than necessary.
Imagine the password parameter is being used in three separate spots in the application. During the login, but also the registration and the password update process. Disabling 942190 for the password parameter would thus prevent false positives in all three locations. And that is welcome. So how do we do this?
Let’s write the following into a file named rule_exclusions_method2.conf:
# ModSec Rule Exclusion: 942190 : Detects MSSQL code execution and# information gathering attemptsSecRuleUpdateTargetById 942190 "!ARGS:password"
Advice: Always accompany rule exclusions with a brief comment on what you are doing and what the rule is all about since there are probably more rule IDs around than you can memorize.
But what does this directive do exactly? It updates the target list of rule 942190 and removes the password argument. So, the rule is still there, but it is no longer applied to the password arguments (ARGS).
We need to pass this rule exclusion via a custom rule file, upload and activate this custom rule file. The full details of how to upload and enable custom rule files in LoadMaster are outlined in blog #3, ‘Deploying custom rules on LoadMaster’ available here.
The effect will be immediate. Repeat the curl request and it will successfully transit the LoadMaster WAF to the Apache application.
Sometimes, you want to skip a rule only in certain situations. Say the rule is triggering false positives on the login form, but you want to retain it for the search form. This is when run-time rule exclusions come into play.
They are more complex, but also more context-sensitive.
Here is a rule exclusion for 942190 with a URI constraint for the URI with the password submission. Let’s write the following into a file named rule_exclusions_method3.conf:
# ModSec Rule Exclusion: 942190 : Detects MSSQL code execution and # information gathering attempts SecRule REQUEST_URI "@beginsWith /submit-login" "phase:1,nolog,pass,id:10000,\ ctl:ruleRemoveById=942190"
Advice: Always accompany rule exclusions with a brief comment on what you are doing and what the rule is all about since there are probably more rule IDs around than you can memorize.
So, this is a real ModSecurity rule that checks for a request URI beginning with /submit-login. If there is a match, then we tell the rule engine to remove or rather ignore rule 942190 for the remainder of this request. This happens in phase 1 (Request Headers). The principal purpose of this phase is to allow rule writers to assess a request before the costly request body processing is undertaken. Similarly, there is often a need to influence how ModSecurity will process a request body, and this phase is the place to do it.
The full description of the five ModSecurity Phases is available in Chapter 1 of the ModSecurity Handbook, 2nd Ed., available here.
Please be aware that rule IDs need to be unique. So, when you do multiple rule exclusions with this form, you need to raise the rule id for each one of them. The rule space up to 100,000 is available; you do not need to fear any collisions if you stick to this range and keep your rule exclusions organized.
We need to pass this rule exclusion via a custom rule file, upload and activate this custom rule file. The full details of how to upload and enable custom rule files in LoadMaster are outlined in blog #3, ‘Deploying custom rules on LoadMaster’ available here.
The effect will be immediate. Repeat the curl request and it will successfully transit the LoadMaster WAF to the Apache application.
Having done CRS tunings for years, this is the rule exclusion variant I use the least. A use case where this variant is my preferred method is disabling “920350 Host header is a numeric IP address” though. It is very frequent that 920350 triggers on health checks that load balancers or uptime agents employ since they work with IP addresses instead of hostnames. In most other situations, I prefer the other variants.
The following custom rule will address this:
# ModSec Rule Exclusion: 920350 : Host header is a numeric IP addressSecRule REQUEST_URI "@beginsWith /submit-login" "phase:1,nolog,pass,id:10001,\ ctl:ruleRemoveById=920350"
The previous rule exclusion pattern was already quite to the point as was the one about an individual parameter. The one in this section combines the two:
Here is a rule exclusion for 942190 with a URI constraint for the password submission that excludes the password parameter from being examined by rule 942190. Let’s write the following into a file named rule_exclusions_method4.conf:
# ModSec Rule Exclusion: 942190 : Detects MSSQL code execution and # information gathering attempts SecRule REQUEST_URI "@beginsWith /submit-login" "phase:1,nolog,pass,id:10000,\ ctl:ruleRemoveTargetById=942190;ARGS:password"
Advice: Always accompany rule exclusions with a brief comment on what you are doing and what the rule is all about since there are probably more rule IDs around than you can memorize.
So, the condition remains the same, but the control statement has become very complicated. Please be sure you get the format of the control statement correct.
There is a further deep dive tutorial on this rule exclusion method for handling false positives available here.
Methods 3 and 4 were runtime rule exclusions with a URI condition. So, they run during the execution of the request. I wrote the rule exclusions in phase 1, however, LoadMaster executes the custom rules after the standard rules. That means that a phase 1 standard rule is being executed before our rule exclusion rule targeting that rule would be evaluated. That means, for phase 1 rules, our examples 3 and 4 are too late on LoadMaster. Most rules in CRS are phase 2 and so our rule exclusions have precedence.
You can check the phase of a rule by clicking on the rule ID on the false-positive analysis screen. If you find out a rule is happening in phase 1, then you need to write a startup rule exclusion like in method 1 and method 2.
We have now seen the basic four variants to tune a false positive away with the help of a rule exclusion. The first one was very coarse – and supported by the LoadMaster UI – the last one was very granular but rather tedious to do. In practice, you will pick the one variant that suits you best. Over the years, I tend to resort to the first variant more often. It’s true, that number 4 is better from a security standpoint, but it’s also true that business wants to have the thing out of the way and people usually do not care if we have 180 or 179 CRS rules active.
Part 1: Introduction to OWASP CRSPart 2: How do you run OWASP CRS on load masterPart 3: Deploying custom rulesPart 4: False Positive AnalysisPart 5: Reporting