Monitoring Configuration
In order to detect bot activity, Netacea monitors HTTP request traffic using Akamai DataStream 2.
Prerequisites
To successfully stream web traffic logs to Netacea, please ensure you have:
An active Akamai account with full access to the Akamai Dashboard.
Enabled Akamai DataStream 2.
DataStream 2 is included with content delivery and comes with no additional charge. To enable this feature, get in touch with your Akamai account team.
Provided a list of protection use cases and the pathways that are vulnerable to each.
Provided a list of all known, safe and whitelisted users. This includes a list of trusted bots, IPs and geolocations. Any information on known bad actors is also valuable.
Been given the details (bucket path, name, location, credentials) of an AWS S3 Bucket hosted and provided by Netacea.
DataStream 2 Configuration
Please follow the Akamai documentation found here to create and activate a DataStream 2 stream, based on the following configuration:
Step 1: Configuration
Use the Configuration tab to enter basic stream details and choose any properties that you want to monitor.
Step 2: Data Set
During the Data Set configuration step, use the following minimum data set, defined by Netacea:
Request time
The time which the request was received
Bytes
The Bytes sent as part of the users' request
Client IP
The IP address from which the request was made
HTTP status codes
The HTTP status code returned by the server
Protocol type
The protocol of the response-request cycle.
Request host
The value of the host header in the request
Request method
The HTTP method of the request
Request path
The path of the requested resource
Query string
The query string of the request
User-Agent*
The user agent string sent in the header by the client
Referer*
The web page the user followed a link from
Turn around time
The elapsed time in milliseconds between when the last request header is received and the first byte of the reply is written to the client socket.
Custom field*
This field supports variables and it is required to log custom Netacea-specific fields.
*To monitor these fields you need to enable logging custom data set parameters in the Log Request Details behavior of the associated Akamai property. See Log custom parameters for more details.
Ensure to select JSON as the Log format.
Step 3: Delivery
Select Amazon S3 as the Destination.
In Name, enter a human-readable description for the destination.
In Bucket, enter the name of the relevant S3 Bucket hosted by Netacea.
In Folder path, populate this with "logs/", unless otherwise instructed by Netacea.
In Region, enter the AWS region code where the bucket resides. This should be eu-west-1 (all lowercase) unless otherwise instructed by Netacea.
In Access key ID and Secret access key, enter the values associated with the Amazon S3 bucket provided by Netacea.
Click Validate & Save to validate the connection to the destination, and save the details you provided.
Ensure the Push frequency is set to the lowest possible value.
Step 4: Summary and Activation
On the Summary tab, review all the details of the stream you configured and ensure to follow the above instructions. Once ready, check the Activate stream upon saving box and click Save stream.
Step 5: Enable the DataStream via a Rule
Activating a stream takes up to an hour, but it will start gathering and streaming data only after you add and enable the DataStream behavior via a Rule in your property configuration that hosts the website. Follow these steps to create the Rule to enable your DataStream:
Create a new property version of your existing active property version and edit the newly created version.
Under Property Configuration Settings, select +Rules to create a new Rule
Select the Blank Rule Template and provide a meaningful name for the rule before clicking Insert Rule.
In the Criteria section, add the following matching rule:
If Hostname is one of <hostname>.
If Netacea are monitoring multiple websites you can add multiple hostnames. Only include hostnames that will be monitored by the DataStream that was created earlier.
In the Behaviors section, add a behavior and enable the DataStream that was created earlier.
An existing Rule can be duplicated and re-used.
Additionally, add a new Log Request Details behavior. Set 'Include Custom Log Field' to 'on' and Custom Log Field is set to
{{user.PMUSER_NETACEA_DS2_CUSTOM_FIELD}}
The DataStream and associated Rule are now created.
Activate the Property
When all of the DataStreams and Rules have been created, click Save and use the Activate tab to activate (deploy) your changes to the property's configuration.
Monitoring configuration will now be deployed on the property it was configured against. You can confirm the deployment is correct by:
Requesting Netacea review internal monitoring metrics.
Requesting Netacea validate the data content & format.
Last updated
