# Akamai DataStream 2

### **Akamai DataStream 2 log delivery to S3**

To send access logs from Akamai to an Amazon S3 bucket, you can use **Akamai DataStream 2**, which is Akamai’s standard and recommended method for real-time log delivery. Below are the documented steps focused on setting up **DataStream 2**, with configuration guidance specific to delivery into a Netacea-hosted S3 bucket.

### Prerequisites

To successfully stream web traffic logs to Netacea, please ensure the following:

* You have an active Akamai account with full access to the Akamai Dashboard.
* Akamai DataStream 2 is enabled.

{% hint style="info" %}
DataStream 2 is included with content delivery and comes with no additional charge. To enable this feature, get in touch with your Akamai account team.
{% endhint %}

* You have received the necessary AWS S3 bucket details (bucket path, name, region, and credentials) from Netacea. These can be found [here](https://netacea.gitbook.io/portal-docs/user-guides-and-videos/portal-setup-and-log-shipping-guide).
* You are aware of the [minimum required dataset](#step-2-data-set) fields needed by Netacea for log analysis.

## DataStream 2 Configuration

Please follow the Akamai documentation found [here](https://techdocs.akamai.com/datastream2/docs/quick-start) to create and activate a DataStream 2 stream, based on the following configuration:

#### Step 1: Configuration

Use the **Configuration** tab to enter basic stream details and choose any properties that you want to monitor.

#### Step 2: Data Set

During the Data Set configuration step, use the following minimum data set, defined by Netacea:

| <h4 id="required-field">Required Field</h4>    | <h4 id="description">Description</h4>                                                                                                              |
| ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| Request time                                   | The time which the request was received                                                                                                            |
| Bytes                                          | The Bytes sent as part of the users' request                                                                                                       |
| Client IP                                      | The IP address from which the request was made                                                                                                     |
| HTTP status codes                              | The HTTP status code returned by the server                                                                                                        |
| Protocol type                                  | The protocol of the response-request cycle.                                                                                                        |
| Request host                                   | The value of the host header in the request                                                                                                        |
| Request method                                 | The HTTP method of the request                                                                                                                     |
| Request path                                   | The path of the requested resource                                                                                                                 |
| Query string                                   | The query string of the request                                                                                                                    |
| User-Agent<mark style="color:red;">\*</mark>   | The user agent string sent in the header by the client                                                                                             |
| Referer<mark style="color:red;">\*</mark>      | The web page the user followed a link from                                                                                                         |
| Turn around time                               | The elapsed time in milliseconds between when the last request header is received and the first byte of the reply is written to the client socket. |
| Custom field<mark style="color:red;">\*</mark> | This field supports variables and it is required to log custom Netacea-specific fields.                                                            |
| True-Client-IP\*\*                             | The trusted source of the original client IP                                                                                                       |
| X-Forwarded-For\*\*                            | Connecting IP address \[array] of a client request, if proxied                                                                                     |

\*To monitor these fields you need to enable logging custom data set parameters in the **Log Request Details** behavior of the associated Akamai property. See [Log custom parameters](https://techdocs.akamai.com/datastream2/docs/log-custom-parameters) for more details.

\*\* Useful for when proxies are in the line of traffic from client > origin

{% hint style="warning" %}
Ensure to select **JSON** as the Log format.
{% endhint %}

#### Step 3: Delivery

1. Select **Amazon S3** as the Destination.
2. In **Name**, enter a human-readable description for the destination.
3. In **Bucket**, enter the name of the relevant S3 Bucket hosted by Netacea.
4. In **Folder path**, populate this with "**`logs/"`** unless otherwise instructed by Netacea.
5. In **Region**, enter the AWS region code where the bucket resides. This should be `eu-west-1` (all lowercase) unless otherwise advised.
6. In **Access key ID** and **Secret access key,** enter the credentials provided by Netacea.
7. Click **Validate & Save** to validate the connection to the destination and save the configuration.
8. Ensure **Push frequency** is set to the lowest possible value.

#### Step 4: Summary and Activation

1. On the **Summary** tab, review all configured details carefully.
2. Once confirmed, check the **Activate stream upon saving** box.
3. Click **Save stream** to complete the setup.

#### Step 5: Enable the DataStream via a Rule

Activating the stream may take up to an hour. However, data will only be collected and streamed **after** you add and enable the [DataStream behavior](https://techdocs.akamai.com/datastream2/docs/enable-datastream-behavior) via a rule in your site’s property configuration.

To do this:

1. Create a new property version of your currently active property.
2. Edit the new version.
3. Under **Property Configuration Settings**, click **+Rules** to add a new rule.
4. Select the **Blank Rule Template**, provide a meaningful name, and click **Insert Rule**.

<figure><img src="/files/jceKdX5ugqObY5ndT9fV" alt=""><figcaption></figcaption></figure>

**Configure the rule**

* **Criteria Section**\
  Add the following matching condition:
  * **If Hostname is one of `<hostname>`**

<figure><img src="/files/dXXO0EM9fHF9zHAdWiEZ" alt=""><figcaption><p>Rule Criteria</p></figcaption></figure>

{% hint style="info" %}
If Netacea is monitoring multiple websites, you can include multiple hostnames. Only include those that are intended to be monitored by the DataStream created earlier.
{% endhint %}

**Behaviors Section**

* Add a behavior to **enable the DataStream** created earlier.

  > You can also duplicate and reuse an existing rule if appropriate.

<figure><img src="/files/XBxpjsjMHfDzslxBBFD8" alt=""><figcaption><p>Rule Behaviors</p></figcaption></figure>

* Add another behavior: **Log Request Details**
  * Set **Include Custom Log Field** to `on`
  * Set **Custom Log Field** to `{{user.PMUSER_NETACEA_DS2_CUSTOM_FIELD}}`

<figure><img src="/files/cCk0Fm9Jf4E2ocrDRsZK" alt=""><figcaption><p>Log Request Details</p></figcaption></figure>

The DataStream and associated Rule are now created.

{% hint style="info" %}

#### Repeat the above procedure for each separate website (or group of websites) that Netacea will be monitoring.

{% endhint %}

### Activate the Property

When all of the DataStreams and Rules have been created, click **Save** and use the **Activate** tab to activate (deploy) your changes to the property's configuration.

{% hint style="success" %}
Monitoring configuration will now be deployed on the property it was configured against. You can confirm the deployment is correct by:

* Requesting Netacea review internal monitoring metrics.
* Requesting Netacea validate the data content & format.
  {% endhint %}

### Final Steps

Once log delivery is active, **notify your Netacea Solutions Engineer**. This allows us to:

* Confirm receipt of data
* Validate the dataset format and completeness
* Begin analysis once a sufficient volume of data has been collected

If you require support during setup, the Netacea SE team is here to help — don’t hesitate to reach out.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.netacea.com/netacea-or-offline-pov-guide/akamai-datastream-2.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.