Help parsing AWS Application Load Balancers logs


Author
Message
cv
cv
New Member (5 reputation)New Member (5 reputation)New Member (5 reputation)New Member (5 reputation)New Member (5 reputation)New Member (5 reputation)New Member (5 reputation)New Member (5 reputation)New Member (5 reputation)
Group: Forum Members
Posts: 1, Visits: 4
Hello,

I'm trying to parse AWS Application Load Balancers logs. I looked into the DSV parser option, but it still requires that I configure every argument. Am I missing an easier option?

Thank you,
cv

Sample line:
https 2018-07-02T22:23:00.186641Z app/my-loadbalancer/50dc6c495c0c9188
192.168.131.39:2817 10.0.0.1:80 0.086 0.048 0.037 200 200 0 57
"GET https://www.example.com:443/ HTTP/1.1" "curl/7.46.0" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2
arn:aws:elasticloadbalancing:us-east-2:123456789012:targetgroup/my-targets/73e2d6bc24d8a067
"Root=1-58337281-1d84f3d73c47ec4e58577259" "www.example.com" "arn:aws:acm:us-east-2:123456789012:certificate/12345678-1234-1234-1234-123456789012"
1 2018-07-02T22:22:48.364000Z "authenticate,forward" "-" "-" 10.0.0.1:80 200 "-" "-"

source: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html#access-log-entry-format

LogViewPlus Support
LogViewPlus Support
Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Hi,

That is a bit of a complicated format.  I suspect the problem with auto-detection here is that the log file does not contain a header and 'fields' definitions.  Otherwise, it should really have been automatically detected as a W3C log file format.

Fortunately, as you indicated, the format is well defined.

You can parse this file using the PatternParser with the pattern:

%s{type} %d{yyyy-MM-ddTHH:mm:ss.ffffffZ} %s{elb} %s{client:port} %s{target:port} %s{request_processing_time} %s{target_processing_time} %s{response_processing_time} %s{elb_status_code} %s{target_status_code} %s{received_bytes} %s{sent_bytes} "%S{request}" "%S{user_agent}" %s{ssl_cipher} %s{ssl_protocol} %s{target_group_arn} "%S{trace_id}" "%S{domain_name}" "%S{chosen_cert_arn}" %s{matched_rule_priority} %s{request_creation_time} "%S{actions_executed}" "%S{redirect_url}" "%S{error_reason}" %s{target:port_list} %S{target_status_code_list} "%S{classification}" "%S{classification_reason}"%n




If you are not interested in all of the fields provided, I would recommend removing some of the column names.  This will indicate to LogViewPlus that the data should be parsed, but not presented.

Hope that helps,

Toby


Edited 4 Years Ago by LogViewPlus Support
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Similar Topics

Login

Explore
Messages
Mentions
Search