Universal Log Transfer
Guide for securely transferring log files to Netacea via AWS Command Line Interface
Netacea Universal Offline POV Guide
To send log data to Netacea for analysis when real-time streaming or platform integrations are not viable, you can use the AWS Command Line Interface (CLI) to complete a one-off upload to a secure, Netacea-hosted Amazon S3 bucket. Below are the documented steps required to install and use the AWS CLI to perform a secure transfer.
Pre-requisites
To successfully complete an offline log transfer to Netacea, please ensure the following:
The AWS CLI is installed and confirmed to be working on your machine.
You have received the necessary S3 bucket details (bucket name and ARN) from Netacea. These can be found here.
You have created a test file that can be used to validate the connection before transferring actual data.
You are using a machine that:
Allows command-line access
Has permission to install and run CLI tools
Has access to the log file(s) intended for upload
Step 1: Install the AWS CLI
Download the AWS CLI from the official source based on your operating system:
Windows (64-bit)
macOS (PKG Installer)
Linux (Installer)
Amazon Linux (AMI)
After downloading, run the installer and follow the on-screen prompts.
To confirm successful installation, run the following command in your terminal or command prompt:
aws --version
Step 2: Configure the AWS CLI
Use the provided credentials to set up your environment:
aws configure
You will then be asked to input some additional details. The table below explains what information needs to be provided and where this can be located from:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION
eu-west-1
Step 3: Test Access with a Sample File
Ensure that you have created a small test file (this can be empty) to verify access:
aws s3 cp localtestfile.txt s3://name-of-bucket
Replace
/full/path/to/testfile.txt
with the actual path to your test fileReplace
<bucket-name>
with the S3 bucket name provided by Netacea
Once uploaded, contact the Netacea Solutions Engineering team to confirm that the file was received successfully.
Step 4: Upload the Log Data
After confirmation from Netacea, you may proceed with uploading your full log file using the same command:
aws s3 cp localtestfile.txt s3://name-of-bucket
Make sure to:
Replace
/full/path/to/logfile.log
with the actual path to your log fileUse the same bucket name as provided
Ensure the upload completes without errors.
Netacea Minimum Dataset
To perform a successful Proof of Value (PoV), the log file(s) you upload must include the following fields:
Timestamp
The time at which the request was received
IP Address
The IP address from which the request was made
User Agent
The user agent string sent in the header by the client
Method
The HTTP method of the request
Path
The path of the requested resource
Query
The query string of the request
Status
The HTTP status code returned by the server
Referrer
The web page the user followed a link from
Bytes Sent
The Bytes sent as part of the servers response
Client JA3
Clients JA3 fingerprint (if available)
X-Forwarded-For*
Connecting IP address [array] of a client request, if proxied
True Client IP*
The trusted source of the original client IP
Host
The destination host of the request
Protocol
The protocol of the request
Request Time
The complete amount of time it took to process the request
* Useful for when proxies are in the line of traffic from client > origin.
Final Steps
Once log delivery is active, notify your Netacea Solutions Engineer. This allows us to:
Confirm receipt of data
Validate the dataset format and completeness
Begin analysis once a sufficient volume of data has been collected
If you require support during setup, the Netacea SE team is here to help — don’t hesitate to reach out.
Last updated