# Monitoring Configuration

### Prerequisites

Before Netacea can monitor your platform successfully, please ensure you have:

1. Provided a list of all known, safe and trusted list users, partners & 3rd party apps. This includes a list of trusted bots & automated tools. Any information on known bad actors is also valuable.
2. Provided a list of protection use cases and the pathways that are vulnerable to each.
3. Had the relevant log shipping endpoints & credentials shared via the [Netacea portal](https://docs.netacea.com/netacea-plugin-information/accessing-your-integration-settings).

### Log Streaming Implementation Steps

* Log in to the Fastly/Magento web interface.
* Navigate to Stores → Configuration → Advanced → System → Full Page Cache → Fastly Configuration → Tools → Real Time Log Streaming.
* Create a new Real-Time Log Streaming job by clicking "Create" on Create Endpoint, selecting Amazon S3 and clicking "Add".

<figure><img src="https://3359534748-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8KQH1bDl0sVMvZgHUjkC%2Fuploads%2Fgit-blob-07cb34c7b98389bb72ab1eef2b0f542c5e28e380%2Fimage%20(6)%20(2).png?alt=media" alt=""><figcaption></figcaption></figure>

* Populate the log streaming job with the following information:

<table><thead><tr><th width="216">Field</th><th>Value</th></tr></thead><tbody><tr><td>Name</td><td>Enter a memorable human-readable name for the endpoint, such as Netacea_Logging.</td></tr><tr><td>Log Format</td><td><a href="#log-format">Log Format</a></td></tr><tr><td>Timestamp</td><td><a href="#undefined">Timestamp</a></td></tr><tr><td>Bucket Name</td><td>The name provided by Netacea</td></tr><tr><td>Access Key</td><td>The access key provided by Netacea</td></tr><tr><td>Secret Key</td><td>The secret key provided by Netacea</td></tr><tr><td>Path</td><td>The path provided by Netacea</td></tr><tr><td>Domain</td><td>s3.eu-west-1.amazonaws.com</td></tr><tr><td>Log Line Format</td><td>Blank</td></tr><tr><td>Compression</td><td>Gzip</td></tr><tr><td>GZIP Level</td><td>1</td></tr><tr><td>Period</td><td>15</td></tr></tbody></table>

{% hint style="warning" %}
If the Shielding is enabled for Fastly Host(s) then please check paragraph below called "Shielding consideration" before you create a logging endpoint.
{% endhint %}

<figure><img src="https://3359534748-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8KQH1bDl0sVMvZgHUjkC%2Fuploads%2Fgit-blob-367810b88d1d79f38320d144aca452ee527996a5%2Fimage%20(22).png?alt=media" alt=""><figcaption></figcaption></figure>

#### **Log Format**

{% code overflow="wrap" %}

```
{"@timestamp": "%{%Y-%m-%dT%H:%M:%S%z}t","bc_type": %{if(req.http.netacea_bctype_string, "%u0022" + json.escape(req.http.netacea_bctype_string) + "%u0022", "null")}V,"bytes_sent": "%B","cookie_session_status": %{if(req.http.x-netacea:cookie_session_status, "%u0022" + json.escape(req.http.x-netacea:cookie_session_status) + "%u0022", "null")}V,"client": "%{json.escape(client.ip)}V","domain": %{if(req.http.host, "%u0022" + json.escape(req.http.host) + "%u0022", "null")}V,"integration_mode": %{"%u0022" + json.escape(req.http.x-netacea:integration_mode) + "%u0022"}V,"integration_type": %{if(req.http.integration_type, "%u0022" + json.escape(req.http.integration_type) + "%u0022", "null")}V,"integration_version": %{if(req.http.integration_version, "%u0022" + json.escape(req.http.integration_version) + "%u0022", "null")}V,"method": "%{json.escape(req.method)}V","path": "%{json.escape(req.url.path)}V","mit_svc_latency":%{if(req.http.x-netacea:mit_svc_latency, "%u0022" + json.escape(req.http.x-netacea:mit_svc_latency) + "%u0022", "0")}V,"mit_status":%{if(req.http.x-netacea:mit_status, "%u0022" + json.escape(req.http.x-netacea:mit_status) + "%u0022", "0")}V,"protocol": "%{json.escape(req.proto)}V","query": "%{json.escape(req.url.qs)}V","referrer": %{if(req.http.referer, "%u0022" + json.escape(req.http.referer) + "%u0022", "null")}V,"request_time": %{time.elapsed}V,"status": "%{json.escape(resp.status)}V","user_agent": %{if(req.http.user-agent, "%u0022" + json.escape(req.http.user-agent) + "%u0022", "null")}V,"user_id": %{if(req.http.x-netacea-userid, "%u0022" + json.escape(req.http.x-netacea-userid) + "%u0022", "null")}V,"client_ja3_md5": %{if(tls.client.ja3_md5, "%u0022" + json.escape(tls.client.ja3_md5) + "%u0022", "null")}V,"x_forwarded_for": %{if(req.http.X-Forwarded-For, "%u0022" + json.escape(req.http.X-Forwarded-For) + "%u0022", "null")}V}
```

{% endcode %}

#### Timestamp

```
%Y-%m-%dT%H:%M:%S.000
```

### Shielding consideration

Fastly allows users to enable Host's feature called "[Shielding](https://developer.fastly.com/learning/concepts/shielding/)" that has some benefits like reducing origin load, improving cache hit ratio, etc. Enabling this feature in a Fastly service that uses our Integration results in a duplication of logs that are being send by it through Netacea logging. We do not expect this.\
\
In order to solve this issue we need to attach a Condition to the Netacea Logging.

* Click on a gear icon next to the logging endpoint

<figure><img src="https://3359534748-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8KQH1bDl0sVMvZgHUjkC%2Fuploads%2Fgit-blob-626a51d81973b5103f49a8e33246896b0271301a%2Fimage%20(94).png?alt=media" alt=""><figcaption></figcaption></figure>

* In the popup window please find "Condition" row and click on "Attach a condition"

<figure><img src="https://3359534748-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8KQH1bDl0sVMvZgHUjkC%2Fuploads%2Fgit-blob-1ced47a8c87470a821c3597ebd892061e06057a4%2Fimage%20(95).png?alt=media" alt=""><figcaption></figcaption></figure>

* After that you can hit "Create a new response condition"

<figure><img src="https://3359534748-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8KQH1bDl0sVMvZgHUjkC%2Fuploads%2Fgit-blob-c55046df2fc6c26dbd3016fc909858c3466b550b%2Fimage%20(96).png?alt=media" alt=""><figcaption></figcaption></figure>

* A popup window will open where you need to put these details:

<table><thead><tr><th width="186">Field</th><th>Value</th></tr></thead><tbody><tr><td>Name</td><td>e.g. Netacea_Log_Visit</td></tr><tr><td>Apply if</td><td><code>(req.url !~ "/media/" &#x26;&#x26; req.url !~ "/static/") &#x26;&#x26; fastly.ff.visits_this_service == 0</code></td></tr><tr><td>Priority</td><td>10 (default)</td></tr></tbody></table>

* Click on "Create". You should now see that the Condition has been attached to the Logging stream

<figure><img src="https://3359534748-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8KQH1bDl0sVMvZgHUjkC%2Fuploads%2Fgit-blob-0bb08c3597bbc64934aad380df1765c689af14a9%2Fimage%20(97).png?alt=media" alt=""><figcaption></figcaption></figure>

* Now you can click on "Create" again and this will update Logging appropriately

### Finishing Up

Check you have completed the following steps:

* Configured[ log streaming](#log-streaming) to the S3 bucket provided
* Ensure no error warnings are appearing

You are now ready to deploy by clicking "Save Config"

{% hint style="info" %}
A Cache flush can sometimes be required. To do this go to System -> Cache Management and click "Flush Magento Cache".
{% endhint %}

Monitoring will now be deployed on the Fastly/Magento environment. You can verify the deployment is active by:

* Reviewing the active version for the new log shipping job.
* Requesting Netacea review internal ingest metrics.
* Requesting Netacea validate the data content & format.