LogoLogo
  • Overview
    • Overview
    • Integration Modes
    • Minimum Required Dataset
    • Customer Service Desk
  • Netacea Plugin Information
    • Accessing Your Integration Settings
    • Akamai
      • Akamai Plugin Logic
      • Monitoring Configuration
      • Installation and Configuration
        • Proxy Property Configuration
        • EdgeWorker Installation
        • Property Configuration
          • Optional Integration Configuration
    • Cloudflare
      • Cloudflare Plugin Logic
      • Monitoring Configuration
      • Installation and Configuration
        • Installation via Wrangler (CLI)
        • Installation via Cloudflare UI
    • CloudFront
      • CloudFront Plugin Logic
      • Monitoring Configuration
      • Installation and Configuration
    • Fastly
      • Fastly Plugin Logic
      • Monitoring Configuration
      • Installation and Configuration
        • Advanced Configuration
        • Deployment via Terraform
    • Fastly Magento
      • Fastly/Magento Plugin Logic
      • Monitoring Configuration
      • Installation and Configuration
        • Advanced Configuration
    • F5
      • F5 Plugin Logic
      • Monitoring Configuration
      • Installation and Configuration
    • Vercel
      • Vercel Plugin Logic
      • Monitoring Configuration
      • Installation and Configuration
    • API Direct Integration
      • How to Build a Netacea Plugin
  • Netacea Data Sync
    • Data Sync
    • Recommendations
  • Captcha
    • reCAPTCHA User Journey
    • hCaptcha User Journey
    • Custom reCAPTCHA Page Guide
    • Custom hCaptcha Page Guide
Powered by GitBook

Copyright Netacea 2023

On this page
  • Prerequisites
  • Log Streaming Implementation Steps
  • Shielding consideration
  • Finishing Up
  1. Netacea Plugin Information
  2. Fastly Magento

Monitoring Configuration

The Fastly/Magento Log Shipping module is used to ingest user requests, allowing Netacea to monitor and detect bot activity.

PreviousFastly/Magento Plugin LogicNextInstallation and Configuration

Last updated 11 months ago

Prerequisites

Before Netacea can monitor your platform successfully, please ensure you have:

  1. Provided a list of all known, safe and trusted list users, partners & 3rd party apps. This includes a list of trusted bots & automated tools. Any information on known bad actors is also valuable.

  2. Provided a list of protection use cases and the pathways that are vulnerable to each.

  3. Had the relevant log shipping endpoints & credentials shared via the .

Log Streaming Implementation Steps

  • Log in to the Fastly/Magento web interface.

  • Navigate to Stores → Configuration → Advanced → System → Full Page Cache → Fastly Configuration → Tools → Real Time Log Streaming.

  • Create a new Real-Time Log Streaming job by clicking "Create" on Create Endpoint, selecting Amazon S3 and clicking "Add".

  • Populate the log streaming job with the following information:

Field
Value

Name

Enter a memorable human-readable name for the endpoint, such as Netacea_Logging.

Log Format

Timestamp

Bucket Name

The name provided by Netacea

Access Key

The access key provided by Netacea

Secret Key

The secret key provided by Netacea

Path

The path provided by Netacea

Domain

s3.eu-west-1.amazonaws.com

Log Line Format

Blank

Compression

Gzip

GZIP Level

1

Period

15

If the Shielding is enabled for Fastly Host(s) then please check paragraph below called "Shielding consideration" before you create a logging endpoint.

Log Format

{"@timestamp": "%{%Y-%m-%dT%H:%M:%S%z}t","bc_type": %{if(req.http.netacea_bctype_string, "%u0022" + json.escape(req.http.netacea_bctype_string) + "%u0022", "null")}V,"bytes_sent": "%B","cookie_session_status": %{if(req.http.x-netacea:cookie_session_status, "%u0022" + json.escape(req.http.x-netacea:cookie_session_status) + "%u0022", "null")}V,"client": "%{json.escape(client.ip)}V","domain": %{if(req.http.host, "%u0022" + json.escape(req.http.host) + "%u0022", "null")}V,"integration_mode": %{"%u0022" + json.escape(req.http.x-netacea:integration_mode) + "%u0022"}V,"integration_type": %{if(req.http.integration_type, "%u0022" + json.escape(req.http.integration_type) + "%u0022", "null")}V,"integration_version": %{if(req.http.integration_version, "%u0022" + json.escape(req.http.integration_version) + "%u0022", "null")}V,"method": "%{json.escape(req.method)}V","path": "%{json.escape(req.url.path)}V","mit_svc_latency":%{if(req.http.x-netacea:mit_svc_latency, "%u0022" + json.escape(req.http.x-netacea:mit_svc_latency) + "%u0022", "0")}V,"mit_status":%{if(req.http.x-netacea:mit_status, "%u0022" + json.escape(req.http.x-netacea:mit_status) + "%u0022", "0")}V,"protocol": "%{json.escape(req.proto)}V","query": "%{json.escape(req.url.qs)}V","referrer": %{if(req.http.referer, "%u0022" + json.escape(req.http.referer) + "%u0022", "null")}V,"request_time": %{time.elapsed}V,"status": "%{json.escape(resp.status)}V","user_agent": %{if(req.http.user-agent, "%u0022" + json.escape(req.http.user-agent) + "%u0022", "null")}V,"user_id": %{if(req.http.x-netacea-userid, "%u0022" + json.escape(req.http.x-netacea-userid) + "%u0022", "null")}V,"client_ja3_md5": %{if(tls.client.ja3_md5, "%u0022" + json.escape(tls.client.ja3_md5) + "%u0022", "null")}V,"x_forwarded_for": %{if(req.http.X-Forwarded-For, "%u0022" + json.escape(req.http.X-Forwarded-For) + "%u0022", "null")}V}

Timestamp

%Y-%m-%dT%H:%M:%S.000

Shielding consideration

  • Click on a gear icon next to the logging endpoint

  • In the popup window please find "Condition" row and click on "Attach a condition"

  • After that you can hit "Create a new response condition"

  • A popup window will open where you need to put these details:

Field
Value

Name

e.g. Netacea_Log_Visit

Apply if

(req.url !~ "/media/" && req.url !~ "/static/") && fastly.ff.visits_this_service == 0

Priority

10 (default)

  • Click on "Create". You should now see that the Condition has been attached to the Logging stream

  • Now you can click on "Create" again and this will update Logging appropriately

Finishing Up

Check you have completed the following steps:

  • Ensure no error warnings are appearing

You are now ready to deploy by clicking "Save Config"

A Cache flush can sometimes be required. To do this go to System -> Cache Management and click "Flush Magento Cache".

Monitoring will now be deployed on the Fastly/Magento environment. You can verify the deployment is active by:

  • Reviewing the active version for the new log shipping job.

  • Requesting Netacea review internal ingest metrics.

  • Requesting Netacea validate the data content & format.

Fastly allows users to enable Host's feature called "" that has some benefits like reducing origin load, improving cache hit ratio, etc. Enabling this feature in a Fastly service that uses our Integration results in a duplication of logs that are being send by it through Netacea logging. We do not expect this. In order to solve this issue we need to attach a Condition to the Netacea Logging.

Configured to the S3 bucket provided

Shielding
log streaming
Log Format
Timestamp
Netacea portal