Last Wednesday I was attending the DevOps Connect: DevSecOps at RSAC 2021, whereas a cool project was presented: Crowdsec, a collaborative behavior detection engine, coupled with a global IP reputation network
Crowdsec is an open-source, lightweight software, detecting peers with aggressive behaviors to prevent them from accessing your systems.
and
CrowdSec is a free, modern & collaborative behavior detection engine, coupled with a global IP reputation network. It stacks on fail2ban’s philosophy but is IPV6 compatible and 60x faster (Go vs Python), uses Grok patterns to parse logs and YAML scenario to identify behaviors. collaborative behavior detection engine, coupled with a global IP reputation network
The interesting of the project is the crowd part, whereas the behavior scenarios and the bad ips detected on the base of these, are shared to the community.
Crowdsec-agent is an open-source and lightweight software that allows you to detect peers with malevolent behaviors and block them from accessing your systems at various level (infrastructural, system, applicative)
Bouncers are standalone software pieces in charge of acting upon a decision taken by crowdsec: block an IP, present a captcha, enforce MFA on a given user, etc.
How does cordwsec work?
The following diagram shows, how exactly it works:
The crowdsec-agentconsumes and parses logs from different data sources. This is done by a parser, which is a YAML configuration file that describes how a string is being parsed. The string can be a log line, or a field extracted from a previous parser. Most parsers use the GROK processor but can use additional modules.
The log data may be enriched by so called enrichers, which add extra context.
An example is the geoip-enrich that adds origin country, origin AS and origin IP range to an event.
The agent then passes an Event - i.e. a parsed/normalized log line - to the scenario, which then are matched against the scenarios.
scenarios are YAML files which describe a set of events which characterizing the scenario. Some examples:
scenarios can be of different types (leaky, trigger, counter), and are based on various factors, such as
the speed/frequency of the leaky bucket
the capacity of the leaky bucket
the characteristic(s) of eligible event(s) : “log type XX with field YY set to ZZ”
various filters/directives that can alter the bucket’s behavior, such as groupby, distinct or blackhole
Based on this, the event is qualified using the leaky bucket
The leaky bucket is an algorithm based on an analogy of how a bucket with a constant leak will overflow if either the average rate at which water is poured in exceeds the rate at which the bucket leaks or if more water than the capacity of the bucket is poured in all at once. It can be used to determine whether some sequence of discrete events conforms to defined limits on their average and peak rates or frequencies, e.g. to limit the actions associated to these events to these rates or delay them until they do conform to the rates. It may also be used to check conformance or limit to an average rate alone, i.e. remove any variation from the average.
The [crowdsec-agent] creates one or more buckets, for the events with matching characteristics. When any of these buckets overflows, the scenario has been triggered (an attach was detected)
A [postoverflow] will be applied on overflows (scenario results) before the decision is written to local DB or pushed to API.
This is a parser used for “expensive” enrichment/parsing process that you do not want to perform on all incoming events, but rather on decision that are about to be taken. An example could be slack/mattermost enrichment plugin that requires human confirmation before applying the decision or reverse-dns lookup operations.
After the [crowdsec-agent] generates an alert and eventually one or more associated Decisions:
An alert is the runtime representation of a bucket overflow and servers as an information
A Decision is the representation of the consequence of a bucket overflow: a decision against an IP, a range, an AS, a Country, a User, a Session etc. See also Decision object documentation
Those information (the signal, the associated decisions) are then sent to crowdsec’s local API and stored in the database
The Bouncers will consume the data via the local API and act upon a decision taken by crowdsec: block an IP, present a captcha, enforce MFA on a given user, etc.
Bouncers can be found in the crowdsec-hub and installed via the cscli
install the agent either from repo (Debian and Ubuntu only) or from tarball
install one or multiple bouncers as documented in the [corwdsec-hub]
After that you might install further Collections - these are bundles of parsers, scenarios, postoverflows that form a coherent package and are present in /etc/crowdsec/collections/
With installing the agent you also get the cscli, the cli to manage crowdsec.
We can see what scenarios are there - by default not so much:
So let us install additional [scnearios] by installing valid collections - in our case we would like to have the stuff for nginix as well:
Now this looks better:
At last, to understand the basics, let’s have a look at a scenario - for example /etc/crowdsec/scenarios/ssh-bf.yaml
Without going in the details, but this scenario checks for failed ssh authentications and will ban the IP
The remediation label, if set to true indicate the the originating IP should be ban
So after let the server run trough night, we can see that there were already some ssh bruteforce attacks happening as you can see for example ID=18:
You already see that the IP 221.181.185.19 was banned, but you might check further detaisl
The crowd aspect
As mentioned in the beginning the cool thing about this project is the crowd aspect. For every alert with it’s associated decisions, the meta information about the alert are shared with our central api :
The source ip that triggered the alert
The scenario that was triggered
The timestamp of the attack
After processing the information in the central hub, the information is redistribute relevant blocklists to all the participants.
What else?
There is much more to it, for example there is a dashboard, metrics that can be managed with prometheus and a forensic mode. Also are there a lot of more interesting scenarios, especially also in conjunction with e.g. nginx. But for now that’s it, you have already a pretty good idea how cool that project is.