In this post I've a little lab system (still running StorageGRID 11.7) and Kiwi Free Syslog Server installed. I am interested in seeing the Syslog outputs for certain events.
My grid has just 5 systems (admin node, gateway node, 3 storage nodes) with grid IPs 192.168.0.80, 88, 82, 84, 86.
My Initial Syslog Configuration
Configuration > Audit and syslog server
- Audit levels (currently default)
- System: Normal
- Storage: Error
- Management: Normal
- Client reads: Normal
- Client writes: Normal
- ILM: Normal
- Audit protocol headers - not configured!
- Use external syslog server: Edit external syslog server
- 1. Enter syslog info
- Host: 192.168.0.5
- Port: 514
- Protocol: UDP
- 2. Manage syslog content
- TICK: Send audit logs
- Severity: Passthrough
- Facility: Passthrough
- TICK: Send security events
- Severity: Passthrough
- Facility: Passthrough
- UNTICKED: Send application logs
- Select: External syslog server*
- Note: For the outputs below, to display the output more easily, I have split them into 5 lines:
- Date
- Time
- Priority
- Hostname
- Message
1) Failed Login via Grid Manager
I try to login with the non-existent user DUMMYUSER.
I get two entries, one from the primary admin node (DC1-ADM1) and one from the second storage node (DC1-S2).
- 06-21-2025
- 13:03:51
- Auth.Info
- 192.168.0.80
- Jun 21 13:03:56 DC1-ADM1 NMS: {"MGAU":{"sourceIp":"192.168.0.5","destinationIp":"dc1-adm1.demo.acme.com","domainName":"dc1-adm1.demo.acme.com","requestMethod":"POST","requestBody":"{\"cookie\":true,\"csrfToken\":true,\"username\":\"DUMMYUSER\",\"password\":\"********\"}","requestPath":"/api/v3/authorize","queryParameters":"","responseCode":401,"userURN":"","responseBody":"","success":false,"forceCreate":true}}
- 06-21-2025
- 13:03:51
- Local7.Info
- 192.168.0.84
- Jun 21 13:03:56 DC1-S2 Audit: 2025-06-21T13:03:56.925584 [AUDT:[MRMD(CSTR):"POST"][MPAT(CSTR):"/api/v3/authorize"][MPQP(CSTR):""][MDNA(CSTR):"dc1-adm1.demo.acme.com"][MSIP(CSTR):"192.168.0.5"][MDIP(CSTR):"dc1-adm1.demo.acme.com"][MUUN(CSTR):""][MRSC(UI32):401][RSLT(FC32):FAIL][MRSP(CSTR):""][MRBD(CSTR):"{\"cookie\":true,\"csrfToken\":true,\"username\":\"DUMMYUSER\",\"password\":\"********\"}"][AVER(UI32):10][ATIM(UI64):1750511036925584][ATYP(FC32):MGAU][ANID(UI32):12512245][AMID(FC32):GMGT][ATID(UI64):10847661126524505624]]
2) Successful Account Creation (of a local user) in Grid Manager
I create the local user DUMMYUSER.
I get four entries, two from the primary admin node (DC1-ADM1) and two from the second storage node (DC1-S2).
- 06-21-2025
- 13:13:53
- Auth.Info
- 192.168.0.80
- Jun 21 13:13:58 DC1-ADM1 NMS: {"MGAU":{"sourceIp":"192.168.0.5","destinationIp":"dc1-adm1.demo.acme.com","domainName":"dc1-adm1.demo.acme.com","requestMethod":"POST","requestBody":"","requestPath":"/api/v3/grid/users/e21c5043-a7d1-4152-b7da-6f2a1d90bc57/change-password","queryParameters":"","responseCode":204,"userURN":"urn:sgws:identity::0:root","responseBody":""}}
- 06-21-2025
- 13:13:53
- Local7.Info
- 192.168.0.84
- Jun 21 13:13:58 DC1-S2 Audit: 2025-06-21T13:13:58.969441 [AUDT:[MRMD(CSTR):"POST"][MPAT(CSTR):"/api/v3/grid/users/e21c5043-a7d1-4152-b7da-6f2a1d90bc57/change-password"][MPQP(CSTR):""][MDNA(CSTR):"dc1-adm1.demo.acme.com"][MSIP(CSTR):"192.168.0.5"][MDIP(CSTR):"dc1-adm1.demo.acme.com"][MUUN(CSTR):"urn:sgws:identity::0:root"][MRSC(UI32):204][RSLT(FC32):SUCS][MRSP(CSTR):""][MRBD(CSTR):""][AVER(UI32):10][ATIM(UI64):1750511638969441][ATYP(FC32):MGAU][ANID(UI32):12512245][AMID(FC32):GMGT][ATID(UI64):5158737502540410748]]
- 06-21-2025
- 13:13:53
- Auth.Info
- 192.168.0.80
- Jun 21 13:13:58 DC1-ADM1 NMS: {"MGAU":{"sourceIp":"192.168.0.5","destinationIp":"dc1-adm1.demo.acme.com","domainName":"dc1-adm1.demo.acme.com","requestMethod":"POST","requestBody":"","requestPath":"/api/v3/grid/users","queryParameters":"","responseCode":201,"userURN":"urn:sgws:identity::0:root","responseBody":"{\"id\":\"e21c5043-a7d1-4152-b7da-6f2a1d90bc57\",\"accountId\":\"0\",\"fullName\":\"DUMMYUSER\",\"uniqueName\":\"user/DUMMYUSER\",\"userURN\":\"urn:sgws:identity::0:user/DUMMYUSER\",\"federated\":false,\"memberOf\":null,\"disable\":false}"}}
- 06-21-2025
- 13:13:53
- Local7.Info
- 192.168.0.84
- Jun 21 13:13:58 DC1-S2 Audit: 2025-06-21T13:13:58.666454 [AUDT:[MRMD(CSTR):"POST"][MPAT(CSTR):"/api/v3/grid/users"][MPQP(CSTR):""][MDNA(CSTR):"dc1-adm1.demo.acme.com"][MSIP(CSTR):"192.168.0.5"][MDIP(CSTR):"dc1-adm1.demo.acme.com"][MUUN(CSTR):"urn:sgws:identity::0:root"][MRSC(UI32):201][RSLT(FC32):SUCS][MRSP(CSTR):"{\"id\":\"e21c5043-a7d1-4152-b7da-6f2a1d90bc57\",\"accountId\":\"0\",\"fullName\":\"DUMMYUSER\",\"uniqueName\":\"user/DUMMYUSER\",\"userURN\":\"urn:sgws:identity::0:user/DUMMYUSER\",\"federated\":false,\"memberOf\":null,\"disable\":false}"][MRBD(CSTR):""][AVER(UI32):10][ATIM(UI64):1750511638666454][ATYP(FC32):MGAU][ANID(UI32):12512245][AMID(FC32):GMGT][ATID(UI64):3296999087362237378]]
3.1) Successful Account Modification: Modifying a Local User
Here we edit our local user DUMMYUSER with the additional group membership to TESTGRP01.
- 06-21-2025
- 13:59:21
- Auth.Info
- 192.168.0.80
- Jun 21 13:59:27 DC1-ADM1 NMS: {"MGAU":{"sourceIp":"192.168.0.5","destinationIp":"dc1-adm1.demo.acme.com","domainName":"dc1-adm1.demo.acme.com","requestMethod":"PUT","requestBody":"","requestPath":"/api/v3/grid/users/e21c5043-a7d1-4152-b7da-6f2a1d90bc57","queryParameters":"","responseCode":200,"userURN":"urn:sgws:identity::0:root","responseBody":"{\"id\":\"e21c5043-a7d1-4152-b7da-6f2a1d90bc57\",\"accountId\":\"0\",\"fullName\":\"DUMMYUSER\",\"uniqueName\":\"user/DUMMYUSER\",\"userURN\":\"urn:sgws:identity::0:user/DUMMYUSER\",\"federated\":false,\"memberOf\":[\"d28b6988-efc5-48d1-a4ef-7d8563aa8809\"],\"disable\":false}"}}
- 06-21-2025
- 13:59:21
- Local7.Info
- 192.168.0.84
- Jun 21 13:59:27 DC1-S2 Audit: 2025-06-21T13:59:27.083555 [AUDT:[MRMD(CSTR):"PUT"][MPAT(CSTR):"/api/v3/grid/users/e21c5043-a7d1-4152-b7da-6f2a1d90bc57"][MPQP(CSTR):""][MDNA(CSTR):"dc1-adm1.demo.acme.com"][MSIP(CSTR):"192.168.0.5"][MDIP(CSTR):"dc1-adm1.demo.acme.com"][MUUN(CSTR):"urn:sgws:identity::0:root"][MRSC(UI32):200][RSLT(FC32):SUCS][MRSP(CSTR):"{\"id\":\"e21c5043-a7d1-4152-b7da-6f2a1d90bc57\",\"accountId\":\"0\",\"fullName\":\"DUMMYUSER\",\"uniqueName\":\"user/DUMMYUSER\",\"userURN\":\"urn:sgws:identity::0:user/DUMMYUSER\",\"federated\":false,\"memberOf\":[\"d28b6988-efc5-48d1-a4ef-7d8563aa8809\"],\"disable\":false}"][MRBD(CSTR):""][AVER(UI32):10][ATIM(UI64):1750514367083555][ATYP(FC32):MGAU][ANID(UI32):12512245][AMID(FC32):GMGT][ATID(UI64):9834277365223746066]]
3.2) Successful Account Deletion: Delete a Local User
DUMMYUSER deleted.
- 06-21-2025
- 14:22:52
- Auth.Info
- 192.168.0.80
- Jun 21 14:22:57 DC1-ADM1 NMS: {"MGAU":{"sourceIp":"192.168.0.5","destinationIp":"dc1-adm1.demo.acme.com","domainName":"dc1-adm1.demo.acme.com","requestMethod":"DELETE","requestBody":"","requestPath":"/api/v3/grid/users/e21c5043-a7d1-4152-b7da-6f2a1d90bc57","queryParameters":"","responseCode":204,"userURN":"urn:sgws:identity::0:root","responseBody":""}}
- 06-21-2025
- 14:22:52
- Local7.Info
- 192.168.0.84
- Jun 21 14:22:57 DC1-S2 Audit: 2025-06-21T14:22:57.631643 [AUDT:[MRMD(CSTR):"DELETE"][MPAT(CSTR):"/api/v3/grid/users/e21c5043-a7d1-4152-b7da-6f2a1d90bc57"][MPQP(CSTR):""][MDNA(CSTR):"dc1-adm1.demo.acme.com"][MSIP(CSTR):"192.168.0.5"][MDIP(CSTR):"dc1-adm1.demo.acme.com"][MUUN(CSTR):"urn:sgws:identity::0:root"][MRSC(UI32):204][RSLT(FC32):SUCS][MRSP(CSTR):""][MRBD(CSTR):""][AVER(UI32):10][ATIM(UI64):1750515777631643][ATYP(FC32):MGAU][ANID(UI32):12512245][AMID(FC32):GMGT][ATID(UI64):3988843311435878603]]
4.1) Audit Policy Change: Changing Audit Level
Here we change Management from Normal to Off.
- 06-21-2025
- 14:34:20
- Auth.Info
- 192.168.0.80
- Jun 21 14:34:26 DC1-ADM1 NMS: {"MGAU":{"sourceIp":"192.168.0.5","destinationIp":"dc1-adm1.demo.acme.com","domainName":"dc1-adm1.demo.acme.com","requestMethod":"PUT","requestBody":"","requestPath":"/api/v3/grid/audit","queryParameters":"","responseCode":200,"userURN":"urn:sgws:identity::0:root","responseBody":"{\"levels\":{\"system\":\"normal\",\"storage\":\"error\",\"management\":\"off\",\"clientReads\":\"normal\",\"clientWrites\":\"normal\",\"ilm\":\"normal\"},\"loggedHeaders\":[]}","forceCreate":true}}
- 06-21-2025
- 14:34:20
- Local7.Info
- 192.168.0.84
- Jun 21 14:34:26 DC1-S2 Audit: 2025-06-21T14:34:26.536621 [AUDT:[MRMD(CSTR):"PUT"][MPAT(CSTR):"/api/v3/grid/audit"][MPQP(CSTR):""][MDNA(CSTR):"dc1-adm1.demo.acme.com"][MSIP(CSTR):"192.168.0.5"][MDIP(CSTR):"dc1-adm1.demo.acme.com"][MUUN(CSTR):"urn:sgws:identity::0:root"][MRSC(UI32):200][RSLT(FC32):SUCS][MRSP(CSTR):"{\"levels\":{\"system\":\"normal\",\"storage\":\"error\",\"management\":\"off\",\"clientReads\":\"normal\",\"clientWrites\":\"normal\",\"ilm\":\"normal\"},\"loggedHeaders\":[]}"][MRBD(CSTR):""][AVER(UI32):10][ATIM(UI64):1750516466536621][ATYP(FC32):MGAU][ANID(UI32):12512245][AMID(FC32):GMGT][ATID(UI64):14117089778871083853]]
And we put it back to normal after the test.
4.2) Audit Policy Change: Removing "Send Security Events"
- 06-21-2025
- 14:38:59
- Auth.Info
- 192.168.0.80
- Jun 21 14:39:05 DC1-ADM1 NMS: {"MGAU":{"sourceIp":"192.168.0.5","destinationIp":"dc1-adm1.demo.acme.com","domainName":"dc1-adm1.demo.acme.com","requestMethod":"PUT","requestBody":"","requestPath":"/api/v3/private/audit-destinations","queryParameters":"","responseCode":200,"userURN":"urn:sgws:identity::0:root","responseBody":"{\"defaults\":{\"adminNodes\":{\"enabled\":false},\"remoteSyslogServerA\":{\"enabled\":true,\"protocol\":\"udp\",\"serverCaCert\":null,\"insecureTLS\":false,\"clientCert\":null,\"clientKey\":null,\"clientKeyPassphrase\":null,\"tlsConfigurationParameters\":null,\"hostname\":\"192.168.0.5\",\"port\":514,\"authEventsSend\":false,\"authEventsFacility\":-1,\"authEventsSeverity\":-1,\"auditLogsSend\":true,\"auditLogsFacility\":-1,\"auditLogsSeverity\":-1,\"applicationLogsSend\":false,\"applicationLogsFacility\":-1,\"applicationLogsSeverity\":-1},\"remoteSyslogServerATest\":{\"enabled\":false,\"protocol\":\"udp\",\"serverCaCert\":null,\"insecureTLS\":false,\"clientCert\":null,\"clientKey\":null,\"clientKeyPassphrase\":null,\"tlsConfigurationParameters\":null,\"hostname\":\"192.168.0.5\",\"port\":514,\"authEventsSend\":false,\"authEventsFacility\":-1,\"authEventsSeverity\":-1,\"auditLogsSend\":true,\"auditLogsFacility\":-1,\"auditLogsSeverity\":-1,\"applicationLogsSend\":false,\"applicationLogsFacility\":-1,\"applicationLogsSeverity\":-1}},\"nodes\":{}}"}}
- 06-21-2025
- 14:38:59
- Local7.Info
- 192.168.0.84
- Jun 21 14:39:05 DC1-S2 Audit: 2025-06-21T14:39:05.032198 [AUDT:[MRMD(CSTR):"PUT"][MPAT(CSTR):"/api/v3/private/audit-destinations"][MPQP(CSTR):""][MDNA(CSTR):"dc1-adm1.demo.acme.com"][MSIP(CSTR):"192.168.0.5"][MDIP(CSTR):"dc1-adm1.demo.acme.com"][MUUN(CSTR):"urn:sgws:identity::0:root"][MRSC(UI32):200][RSLT(FC32):SUCS][MRSP(CSTR):"{\"defaults\":{\"adminNodes\":{\"enabled\":false},\"remoteSyslogServerA\":{\"enabled\":true,\"protocol\":\"udp\",\"serverCaCert\":null,\"insecureTLS\":false,\"clientCert\":null,\"clientKey\":null,\"clientKeyPassphrase\":null,\"tlsConfigurationParameters\":null,\"hostname\":\"192.168.0.5\",\"port\":514,\"authEventsSend\":false,\"authEventsFacility\":-1,\"authEventsSeverity\":-1,\"auditLogsSend\":true,\"auditLogsFacility\":-1,\"auditLogsSeverity\":-1,\"applicationLogsSend\":false,\"applicationLogsFacility\":-1,\"applicationLogsSeverity\":-1},\"remoteSyslogServerATest\":{\"enabled\":false,\"protocol\":\"udp\",\"serverCaCert\":null,\"insecureTLS\":false,\"clientCert\":null,\"clientKey\":null,\"clientKeyPassphrase\":null,\"tlsConfigurationParameters\":null,\"hostname\":\"192.168.0.5\",\"port\":514,\"authEventsSend\":false,\"authEventsFacility\":-1,\"authEventsSeverity\":-1,\"auditLogsSend\":true,\"auditLogsFacility\":-1,\"auditLogsSeverity\":-1,\"applicationLogsSend\":false,\"applicationLogsFacility\":-1,\"applicationLogsSeverity\":-1}},\"nodes\":{}}"][MRBD(CSTR):""][AVER(UI32):10][ATIM(UI64):1750516745032198][ATYP(FC32):MGAU][ANID(UI32):12512245][AMID(FC32):GMGT][ATID(UI64):10456084872167946066]]
And we restore "Send Security Events" after the test.
4.3) Audit Policy Change: Returning to default (no External syslog server)
Changing log locations to: Default (Admin Nodes/local nodes)
- 06-21-2025
- 14:50:11
- Auth.Info
- 192.168.0.80
- Jun 21 14:50:17 DC1-ADM1 NMS: {"MGAU":{"sourceIp":"192.168.0.5","destinationIp":"dc1-adm1.demo.acme.com","domainName":"dc1-adm1.demo.acme.com","requestMethod":"PUT","requestBody":"","requestPath":"/api/v3/grid/audit","queryParameters":"","responseCode":200,"userURN":"urn:sgws:identity::0:root","responseBody":"{\"levels\":{\"system\":\"normal\",\"storage\":\"error\",\"management\":\"normal\",\"clientReads\":\"normal\",\"clientWrites\":\"normal\",\"ilm\":\"normal\"},\"loggedHeaders\":[]}","forceCreate":true}}
- 06-21-2025
- 14:50:09
- Auth.Info
- 192.168.0.80
- Jun 21 14:50:15 DC1-ADM1 NMS: {"MGAU":{"sourceIp":"192.168.0.5","destinationIp":"dc1-adm1.demo.acme.com","domainName":"dc1-adm1.demo.acme.com","requestMethod":"PUT","requestBody":"","requestPath":"/api/v3/private/audit-destinations","queryParameters":"","responseCode":200,"userURN":"urn:sgws:identity::0:root","responseBody":"{\"defaults\":{\"adminNodes\":{\"enabled\":true},\"remoteSyslogServerA\":{\"enabled\":false,\"protocol\":\"udp\",\"serverCaCert\":null,\"insecureTLS\":false,\"clientCert\":null,\"clientKey\":null,\"clientKeyPassphrase\":null,\"tlsConfigurationParameters\":null,\"hostname\":\"192.168.0.5\",\"port\":514,\"authEventsSend\":true,\"authEventsFacility\":-1,\"authEventsSeverity\":-1,\"auditLogsSend\":true,\"auditLogsFacility\":-1,\"auditLogsSeverity\":-1,\"applicationLogsSend\":false,\"applicationLogsFacility\":-1,\"applicationLogsSeverity\":-1},\"remoteSyslogServerATest\":{\"enabled\":false,\"protocol\":\"udp\",\"serverCaCert\":null,\"insecureTLS\":false,\"clientCert\":null,\"clientKey\":null,\"clientKeyPassphrase\":null,\"tlsConfigurationParameters\":null,\"hostname\":\"192.168.0.5\",\"port\":514,\"authEventsSend\":true,\"authEventsFacility\":-1,\"authEventsSeverity\":-1,\"auditLogsSend\":true,\"auditLogsFacility\":-1,\"auditLogsSeverity\":-1,\"applicationLogsSend\":false,\"applicationLogsFacility\":-1,\"applicationLogsSeverity\":-1}},\"nodes\":{}}"}}
And we return to 'External syslog server' after the test.
5.1) Timezone Change
I don't think you can change the timezone.
5.2) NTP Settings Change
We add an additional (bogus) NTP server.
Unfortunately my lab was broken at this point (a Maintenance task - decomission - failed) so I was unable to test.
And remove it after the test.
THE END
Note: I wasn't able to do as many tests as I would have liked to due to technical and time issues. The examples are a good start for understanding what the syslog output looks like.
Further Reading: StorageGRID Configuration for syslog
- Considerations for using an external syslog server
- Configure audit messages and external syslog server
- System audit messages
- Object storage audit messages
- Management audit message
- Client read audit messages
- Client write audit messages
- ILM audit messages
- CGRR: Cross-Grid Replication Request
- Enhanced observability with StorageGRID 11.9
- StorageGRID and Elasticsearches | Acting Technologist
- StorageGRID log analytics using ELK stack (has downloadable Logstash samples)
- With videos:
- StorageGRID log analysis using ELK stack (video)
- Endpoint access log forwarding in StorageGRID 11.9 (video)
- And additional resources:
- Syslog 101: Everything You Need to Know to Get Started - Coralogix
- Elastic Stack: (ELK) Elasticsearch, Kibana & Logstash | Elastic
- logstash-patterns/files/grok-patterns.json at master · hpcugent/logstash-patterns · GitHub
- A Beginner’s Guide to Logstash Grok | Logz.io
- A Practical Guide to Logstash: Syslog Deep Dive - Coralogix
- Customize the Discover view | Elastic Docs
- More resources:
Comments
Post a Comment