Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Latest Transform - Wiz Vulnerabilities #10895

Merged
merged 28 commits into from
Sep 12, 2024
Merged

Add Latest Transform - Wiz Vulnerabilities #10895

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
ffedcde
update data stream for CDR
CohenIdo Aug 27, 2024
3392d37
uptate changelog
CohenIdo Aug 27, 2024
0b37343
uptate mappings
CohenIdo Aug 27, 2024
62c4649
WIP
CohenIdo Aug 27, 2024
138e558
Update test generated files and update readme
kcreddy Aug 27, 2024
cf1325c
Merge branch 'update-wiz-vul-ds' into add-latest-vul-transform-wiz
CohenIdo Aug 27, 2024
b152e20
update index name
CohenIdo Aug 27, 2024
368e3f3
Merge remote-tracking branch 'upstream/main' into add-latest-vul-tran…
CohenIdo Aug 28, 2024
9cca808
wip
CohenIdo Aug 28, 2024
c09d118
update readme
CohenIdo Aug 28, 2024
263dfcc
add cloud account name
CohenIdo Aug 28, 2024
307c49a
update tests files
CohenIdo Aug 28, 2024
0d55419
update readme
CohenIdo Aug 28, 2024
a8c8dec
update test
CohenIdo Aug 28, 2024
3c0e0b9
update readme
CohenIdo Aug 29, 2024
f3ef6f8
code review comments
CohenIdo Sep 2, 2024
34621ec
Merge remote-tracking branch 'upstream/main' into add-latest-vul-tran…
CohenIdo Sep 2, 2024
e7662ee
code review comments
CohenIdo Sep 2, 2024
4dc1d3b
update version
CohenIdo Sep 2, 2024
2828c70
add ecs fields
CohenIdo Sep 3, 2024
28d6aae
update version
CohenIdo Sep 4, 2024
49acef6
revert undesired changes
CohenIdo Sep 5, 2024
9f77a15
Merge remote-tracking branch 'upstream/main' into add-latest-vul-tran…
CohenIdo Sep 8, 2024
4c3f8f9
build fixes
CohenIdo Sep 9, 2024
cdaa24a
Merge upstream/main, resolving conflicts by taking upstream changes
CohenIdo Sep 10, 2024
4ec1ae7
update version
CohenIdo Sep 10, 2024
4a7732a
update stack version
CohenIdo Sep 10, 2024
b75bcfd
update changelog
CohenIdo Sep 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add ecs fields
  • Loading branch information
@CohenIdo
CohenIdo committed Sep 3, 2024
commit 2828c7076de7fdda8dfed323691b0b29a302b52d
16 changes: 8 additions & 8 deletions packages/wiz/data_stream/vulnerability/sample_event.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{
"@timestamp": "2023-08-16T18:40:57.000Z",
"agent": {
"ephemeral_id": "c8365310-2cd0-4065-812e-cd321482d237",
"id": "2c073a38-d1ee-46ef-976d-3ee12b71713e",
"name": "elastic-agent-95204",
"ephemeral_id": "7f3e71cf-b88c-4d67-a03a-b0cd154d4d5b",
"id": "5996d015-5920-4e19-9385-62bff448eef4",
"name": "elastic-agent-52480",
"type": "filebeat",
"version": "8.13.0"
"version": "8.14.3"
},
"cloud": {
"account": {
Expand All @@ -16,7 +16,7 @@
},
"data_stream": {
"dataset": "wiz.vulnerability",
"namespace": "43521",
"namespace": "50057",
"type": "logs"
},
"device": {
Expand All @@ -26,17 +26,17 @@
"version": "8.11.0"
},
"elastic_agent": {
"id": "2c073a38-d1ee-46ef-976d-3ee12b71713e",
"id": "5996d015-5920-4e19-9385-62bff448eef4",
"snapshot": false,
"version": "8.13.0"
"version": "8.14.3"
},
"event": {
"agent_id_status": "verified",
"category": [
"vulnerability"
],
"dataset": "wiz.vulnerability",
"ingested": "2024-08-28T20:59:41Z",
"ingested": "2024-09-03T13:28:28Z",
"kind": "alert",
"original": "{\"CVEDescription\":\"In LibTIFF, there is a memory malloc failure in tif_pixarlog.c. A crafted TIFF document can lead to an abort, resulting in a remote denial of service attack.\",\"CVSSSeverity\":\"MEDIUM\",\"dataSourceName\":\"data Source\",\"description\":\"Thepackage`libtiff`version`4.0.3-35.amzn2`wasdetectedin`YUMpackagemanager`onamachinerunning`Amazon2(Karoo)`isvulnerableto`CVE-2020-35522`,whichexistsinversions`\\u003c4.0.3-35.amzn2.0.1`.\\n\\nThevulnerabilitywasfoundinthe[OfficialAmazonLinuxSecurityAdvisories](https://alas.aws.amazon.com/AL2/ALAS-2022-1780.html)withvendorseverity:`Medium`([NVD](https://nvd.nist.gov/vuln/detail/CVE-2020-35522)severity:`Medium`).\\n\\nThevulnerabilitycanberemediatedbyupdatingthepackagetoversion`4.0.3-35.amzn2.0.1`orhigher,using`yumupdatelibtiff`.\",\"detailedName\":\"libtiff\",\"detectionMethod\":\"PACKAGE\",\"epssPercentile\":46.2,\"epssProbability\":0.1,\"epssSeverity\":\"LOW\",\"exploitabilityScore\":1.8,\"firstDetectedAt\":\"2022-05-01T11:36:10.063767Z\",\"fixedVersion\":\"4.0.3-35.amzn2.0.1\",\"hasCisaKevExploit\":false,\"hasExploit\":false,\"id\":\"5e95ff50-5490-514e-87f7-11e56f3230ff\",\"ignoreRules\":{\"enabled\":true,\"expiredAt\":\"2023-08-16T18:40:57Z\",\"id\":\"aj3jqtvnaf\",\"name\":\"abc\"},\"impactScore\":3.6,\"lastDetectedAt\":\"2023-08-16T18:40:57Z\",\"layerMetadata\":{\"details\":\"xxxx\",\"id\":\"5e95ff50-5490-514e-87f7-11e56f3230ff\",\"isBaseLayer\":true},\"link\":\"https://alas.aws.amazon.com/AL2/ALAS-2022-1780.html\",\"locationPath\":\"package/library/file\",\"name\":\"CVE-2020-3333\",\"portalUrl\":\"https://app.wiz.io/explorer/vulnerability-findings#~(entity~(~'xxx-xxx*2cSECURITY_TOOL_FINDING))\",\"projects\":[{\"businessUnit\":\"\",\"id\":\"83b76efe-a7b6-5762-8a53-8e8f59e68bd8\",\"name\":\"Project2\",\"riskProfile\":{\"businessImpact\":\"MBI\"},\"slug\":\"project-2\"},{\"businessUnit\":\"Dev\",\"id\":\"af52828c-4eb1-5c4e-847c-ebc3a5ead531\",\"name\":\"project4\",\"riskProfile\":{\"businessImpact\":\"MBI\"},\"slug\":\"project-4\"},{\"businessUnit\":\"Dev\",\"id\":\"d6ac50bb-aec0-52fc-80ab-bacd7b02f178\",\"name\":\"Project1\",\"riskProfile\":{\"businessImpact\":\"MBI\"},\"slug\":\"project1\"}],\"remediation\":\"yumupdatelibtiff\",\"resolutionReason\":\"resolutionReason\",\"resolvedAt\":\"2023-08-16T18:40:57Z\",\"score\":5.5,\"status\":\"OPEN\",\"validatedInRuntime\":true,\"vendorSeverity\":\"MEDIUM\",\"version\":\"4.0.3-35.amzn2\",\"vulnerableAsset\":{\"cloudPlatform\":\"AWS\",\"cloudProviderURL\":\"https://us-east-1.console.aws.amazon.com/ec2/v2/home?region=us-east-1#InstanceDetails:instanceId=i-0a0f7e1451da5f4a3\",\"hasLimitedInternetExposure\":true,\"hasWideInternetExposure\":true,\"id\":\"c828de0d-4c42-5b1c-946b-2edee094d0b3\",\"ipAddresses\":[\"89.160.20.112\",\"89.160.20.128\"],\"isAccessibleFromOtherSubscriptions\":false,\"isAccessibleFromOtherVnets\":false,\"isAccessibleFromVPN\":false,\"name\":\"test-4\",\"operatingSystem\":\"Linux\",\"providerUniqueId\":\"arn:aws:ec2:us-east-1:998231069301:instance/i-0a0f7e1451da5f4a3\",\"region\":\"us-east-1\",\"status\":\"Active\",\"subscriptionExternalId\":\"998231069301\",\"subscriptionId\":\"94e76baa-85fd-5928-b829-1669a2ca9660\",\"subscriptionName\":\"wiz-integrations\",\"tags\":{\"Name\":\"test-4\"},\"type\":\"VIRTUAL_MACHINE\"}}",
"type": [
Expand Down
16 changes: 8 additions & 8 deletions packages/wiz/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -630,11 +630,11 @@ An example event for `vulnerability` looks as following:
{
"@timestamp": "2023-08-16T18:40:57.000Z",
"agent": {
"ephemeral_id": "c8365310-2cd0-4065-812e-cd321482d237",
"id": "2c073a38-d1ee-46ef-976d-3ee12b71713e",
"name": "elastic-agent-95204",
"ephemeral_id": "7f3e71cf-b88c-4d67-a03a-b0cd154d4d5b",
"id": "5996d015-5920-4e19-9385-62bff448eef4",
"name": "elastic-agent-52480",
"type": "filebeat",
"version": "8.13.0"
"version": "8.14.3"
},
"cloud": {
"account": {
Expand All @@ -645,7 +645,7 @@ An example event for `vulnerability` looks as following:
},
"data_stream": {
"dataset": "wiz.vulnerability",
"namespace": "43521",
"namespace": "50057",
"type": "logs"
},
"device": {
Expand All @@ -655,17 +655,17 @@ An example event for `vulnerability` looks as following:
"version": "8.11.0"
},
"elastic_agent": {
"id": "2c073a38-d1ee-46ef-976d-3ee12b71713e",
"id": "5996d015-5920-4e19-9385-62bff448eef4",
"snapshot": false,
"version": "8.13.0"
"version": "8.14.3"
},
"event": {
"agent_id_status": "verified",
"category": [
"vulnerability"
],
"dataset": "wiz.vulnerability",
"ingested": "2024-08-28T20:59:41Z",
"ingested": "2024-09-03T13:28:28Z",
"kind": "alert",
"original": "{\"CVEDescription\":\"In LibTIFF, there is a memory malloc failure in tif_pixarlog.c. A crafted TIFF document can lead to an abort, resulting in a remote denial of service attack.\",\"CVSSSeverity\":\"MEDIUM\",\"dataSourceName\":\"data Source\",\"description\":\"Thepackage`libtiff`version`4.0.3-35.amzn2`wasdetectedin`YUMpackagemanager`onamachinerunning`Amazon2(Karoo)`isvulnerableto`CVE-2020-35522`,whichexistsinversions`\\u003c4.0.3-35.amzn2.0.1`.\\n\\nThevulnerabilitywasfoundinthe[OfficialAmazonLinuxSecurityAdvisories](https://alas.aws.amazon.com/AL2/ALAS-2022-1780.html)withvendorseverity:`Medium`([NVD](https://nvd.nist.gov/vuln/detail/CVE-2020-35522)severity:`Medium`).\\n\\nThevulnerabilitycanberemediatedbyupdatingthepackagetoversion`4.0.3-35.amzn2.0.1`orhigher,using`yumupdatelibtiff`.\",\"detailedName\":\"libtiff\",\"detectionMethod\":\"PACKAGE\",\"epssPercentile\":46.2,\"epssProbability\":0.1,\"epssSeverity\":\"LOW\",\"exploitabilityScore\":1.8,\"firstDetectedAt\":\"2022-05-01T11:36:10.063767Z\",\"fixedVersion\":\"4.0.3-35.amzn2.0.1\",\"hasCisaKevExploit\":false,\"hasExploit\":false,\"id\":\"5e95ff50-5490-514e-87f7-11e56f3230ff\",\"ignoreRules\":{\"enabled\":true,\"expiredAt\":\"2023-08-16T18:40:57Z\",\"id\":\"aj3jqtvnaf\",\"name\":\"abc\"},\"impactScore\":3.6,\"lastDetectedAt\":\"2023-08-16T18:40:57Z\",\"layerMetadata\":{\"details\":\"xxxx\",\"id\":\"5e95ff50-5490-514e-87f7-11e56f3230ff\",\"isBaseLayer\":true},\"link\":\"https://alas.aws.amazon.com/AL2/ALAS-2022-1780.html\",\"locationPath\":\"package/library/file\",\"name\":\"CVE-2020-3333\",\"portalUrl\":\"https://app.wiz.io/explorer/vulnerability-findings#~(entity~(~'xxx-xxx*2cSECURITY_TOOL_FINDING))\",\"projects\":[{\"businessUnit\":\"\",\"id\":\"83b76efe-a7b6-5762-8a53-8e8f59e68bd8\",\"name\":\"Project2\",\"riskProfile\":{\"businessImpact\":\"MBI\"},\"slug\":\"project-2\"},{\"businessUnit\":\"Dev\",\"id\":\"af52828c-4eb1-5c4e-847c-ebc3a5ead531\",\"name\":\"project4\",\"riskProfile\":{\"businessImpact\":\"MBI\"},\"slug\":\"project-4\"},{\"businessUnit\":\"Dev\",\"id\":\"d6ac50bb-aec0-52fc-80ab-bacd7b02f178\",\"name\":\"Project1\",\"riskProfile\":{\"businessImpact\":\"MBI\"},\"slug\":\"project1\"}],\"remediation\":\"yumupdatelibtiff\",\"resolutionReason\":\"resolutionReason\",\"resolvedAt\":\"2023-08-16T18:40:57Z\",\"score\":5.5,\"status\":\"OPEN\",\"validatedInRuntime\":true,\"vendorSeverity\":\"MEDIUM\",\"version\":\"4.0.3-35.amzn2\",\"vulnerableAsset\":{\"cloudPlatform\":\"AWS\",\"cloudProviderURL\":\"https://us-east-1.console.aws.amazon.com/ec2/v2/home?region=us-east-1#InstanceDetails:instanceId=i-0a0f7e1451da5f4a3\",\"hasLimitedInternetExposure\":true,\"hasWideInternetExposure\":true,\"id\":\"c828de0d-4c42-5b1c-946b-2edee094d0b3\",\"ipAddresses\":[\"89.160.20.112\",\"89.160.20.128\"],\"isAccessibleFromOtherSubscriptions\":false,\"isAccessibleFromOtherVnets\":false,\"isAccessibleFromVPN\":false,\"name\":\"test-4\",\"operatingSystem\":\"Linux\",\"providerUniqueId\":\"arn:aws:ec2:us-east-1:998231069301:instance/i-0a0f7e1451da5f4a3\",\"region\":\"us-east-1\",\"status\":\"Active\",\"subscriptionExternalId\":\"998231069301\",\"subscriptionId\":\"94e76baa-85fd-5928-b829-1669a2ca9660\",\"subscriptionName\":\"wiz-integrations\",\"tags\":{\"Name\":\"test-4\"},\"type\":\"VIRTUAL_MACHINE\"}}",
"type": [
Expand Down
20 changes: 20 additions & 0 deletions packages/wiz/elasticsearch/transform/latest_cdr_vulnerabilities/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
- name: cloud.account.id
external: ecs
- name: cloud.region
external: ecs
- name: package.name
external: ecs
- name: package.version
external: ecs
- name: vulnerability.description
external: ecs
- name: vulnerability.id
external: ecs
- name: vulnerability.score.base
external: ecs
- name: vulnerability.score.version
external: ecs
- name: vulnerability.severity
external: ecs
Comment on lines +1 to +18
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these need to be added?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the transform destination index is not predefined in Elasticsearch (unlike the logs-* and metrics-* indices), so the ECS fields are not inherited automatically and must be explicitly defined.

- name: wiz
type: group
fields:
Expand Down Expand Up @@ -166,6 +184,8 @@
type: keyword
- name: fixed_version
type: keyword
- name: name
type: keyword
- name: resource
type: group
fields:
Expand Down