String Function "substringBetween(String open, String close)" is returning unexpected values

[Sam Sexson]

I'm trying to parse a webhook response to remove some brackets by utilizing the "substringBetween()" function like this:

{{webhookResponse.body.schedules.schedule.onCallUser.username.substringBetween("[","]")}}

Without the substringBetween function, the value returned is "[user]"

When I check the audit log when using the function, it appears that it's _only_ capturing the brackets as shown here:

Am I using this function incorrectly?

[Simmo · Accepted solution]

Hi @Sam Sexson ,

Could you provide an example of the web hook payload for me (remove anything thats PII or company related). Would make it a bit easier to see what is going on.

Cheers,

Simmo

[Sam Sexson]

Hi @Simmo , thank you for your time!  Of course!


{
"team": {
"name": "XXXXXXXXX",
"slug": "XXXXXXXXX"
},
"schedules": [
{
"policy": {
"name": "XXXXXXXXX",
"slug": "XXXXXXXXX"
},
"schedule": [
{
"onCallUser": {
"username": "XXXXXXXXX"
},
"onCallType": "XXXXXXXXX",
"rotationName": "XXXXXXXXX",
"shiftName": "XXXXXXXXX",
"shiftRoll": "XXXXXXXXX",
"rolls": [
{
"start": "XXXXXXXXX",
"end": "XXXXXXXXX",
"onCallUser": {
"username": "XXXXXXXXX"
},
"isRoll": true
},
{
"start": "XXXXXXXXX",
"end": "XXXXXXXXX",
"onCallUser": {
"username": "XXXXXXXXX"
},
"isRoll": true
},
{
"start": "XXXXXXXXX",
"end": "XXXXXXXXX",
"onCallUser": {
"username": "XXXXXXXXX"
},
"isRoll": true
}
]
}
],
"overrides": []
}
]
}

[Simmo]

Hey @Sam Sexson ,

Sorry about the delay in getting back to you. So, what I think is going on here is that schedule is an array/list. 

So, webhookResponse.body.schedules and .schedule are both referring to a list. When that happens, we apply whatever comes next to all the list elements. In your case that is .onCallUser.username.substringBetween("[","]"). And then, because its a list we render it in list format which adds those square brackets.

If you try the following:

{{webhookResponse.body.schedules.first.schedule.first.onCallUser.username}}

 That might hopefully resolve your issue.

Cheers,

Simeon.

[Sam Sexson]

@Simmo !  You are amazing!

Yes, that worked perfectly...  thank you so much for your time in getting me on track!  I appreciate the thorough explanation and recommended change!  Works perfectly!

[Radzhiv Apasov]

@Simmo Hello, I hope this message finds you well. I'm writing to inquire about how to effectively use the substringBetween function in the given situation. I'm dealing with the following JSON structure:

"{
"Type": "Notification",
"MessageId": "96d4c7c2-999e-57ab-aade",
"TopicArn": "arn:aws:sns:us-west-2:test",
"Message": {
"version": "0",
"id": "3ee38987-e0ce--91a1",
"detail-type": "EC2 Instance State-change Notification",
"source": "aws.ec2",
"account": "abc",
"time": "2017-09-11T10:49:41Z",
"region": "us-west-2",
"resources": ["arn:aws:ec2:us-west-2:asdf:instance/i-abc"],
"detail": {
"actionName": "custom-action-name",
"actionDescription": "description of the action",
"findings": [
{
"AwsAccountId": "abc",
"Compliance": { "Status": "PASSED" },
"Confidence": 42,
"CreatedAt": "2017-03-22T13:22:13.933Z",
"Criticality": 99,
"Description": "The version of openssl found on instance i-abcd1234 is known to contain a vulnerability.",
"FirstObservedAt": "2017-03-22T13:22:13.933Z",
"GeneratorId": "acme-vuln-9ab348",
"Id": "us-west-2/111111111111/98aebb2207407c87f51e89943f12b1ef",
"LastObservedAt": "2017-03-23T13:22:13.933Z",
"Malware": [
{
"Name": "Stringler",
"Type": "COIN_MINER",
"Path": "/usr/sbin/stringler",
"State": "OBSERVED"
}
],
"Network": {
"Direction": "IN",
"Protocol": "TCP",
"SourceIpV4": "1.2.3.4",
"SourceIpV6": "FE80:CD00:0000:0CDE:1257:0000:211E:729C",
"SourcePort": "42",
"SourceDomain": "here.com",
"SourceMac": "00:0d:83:b1:c0:8e",
"DestinationIpV4": "2.3.4.5",
"DestinationIpV6": "FE80:CD00:0000:0CDE:1257:0000:211E:729C",
"DestinationPort": "80",
"DestinationDomain": "there.com"
},
"Note": {
"Text": "Don't forget to check under the mat.",
"UpdatedBy": "jsmith",
"UpdatedAt": "2018-08-31T00:15:09Z"
},
"Process": {
"Name": "syslogd",
"Path": "/usr/sbin/syslogd",
"Pid": 12345,
"ParentPid": 56789,
"LaunchedAt": "2018-09-27T22:37:31Z",
"TerminatedAt": "2018-09-27T23:37:31Z"
},
"ProductArn": "arn:aws:securityhub:us-east-1:111111111111:product/111111111111/default",
"ProductFields": {
"generico/secure-pro/Count": "6",
"Service_Name": "cloudtrail.amazonaws.com",
"aws/inspector/AssessmentTemplateName": "My daily CVE assessment",
"aws/inspector/AssessmentTargetName": "My prod env",
"aws/inspector/RulesPackageName": "Common Vulnerabilities and Exposures"
},
"RecordState": "ACTIVE",
"RelatedFindings": [
{ "ProductArn": "arn:aws:securityhub:us-west-2::product/aws/guardduty", "Id": "123e4567-e89b-12d3-a456-426655440000" },
{ "ProductArn": "arn:aws:securityhub:us-west-2::product/aws/guardduty", "Id": "AcmeNerfHerder--x189dx7824" }
],
"Remediation": {
"Recommendation": {
"Text": "Run sudo yum update and cross your fingers and toes.",
"Url": "http://myfp.com/recommendations/dangerous_things_and_how_to_fix_them.html"
}
},
"Resources": [
{
"Type": "AwsEc2Instance",
"Id": "i-cafebabe",
"Partition": "aws",
"Region": "us-west-2",
"Tags": { "billingCode": "Lotus-1-2-3", "needsPatching": "true" },
"Details": {
"AwsEc2Instance": {
"Type": "i3.xlarge",
"ImageId": "ami-abcd1234",
"IpV4Addresses": ["54.194.252.215", "192.168.1.88"],
"IpV6Addresses": ["2001:db8:1234:1a2b::123"],
"KeyName": "my_keypair",
"IamInstanceProfileArn": "arn:aws:iam:::instance-profile/AdminRole",
"VpcId": "vpc-11112222",
"SubnetId": "subnet-56f5f633",
"LaunchedAt": "2018-05-08T16:46:19.000Z"
}
}
}
],
"SchemaVersion": "2018-10-08",
"Severity": { "Product": 8.3, "Normalized": 25 },
"SourceUrl": "string",
"ThreatIntelIndicators": [
{
"Type": "IPV4_ADDRESS",
"Value": "8.8.8.8",
"Category": "BACKDOOR",
"LastObservedAt": "2018-09-27T23:37:31Z",
"Source": "Threat Intel Weekly",
"SourceUrl": "http://threatintelweekly.org/backdoors/8888"
}
],
"Title": "title",
"Types": ["Software and Configuration Checks/Vulnerabilities/CVE"],
"UpdatedAt": "123578964332",
"UserDefinedFields": { "reviewedByCio": "true", "comeBackToLater": "Check this again on Monday" },
"VerificationState": "string",
"WorkflowState": "NEW"
}
]
}
},
"Timestamp": "2017-09-11T10:49:42.630Z",
"SignatureVersion": "1",
"Signature": "sign",
"SigningCertURL": "https://sns.us-west-2.amazonaws.com/SimpleNotification.pem",
"UnsubscribeURL": "https://sns.us-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-west-2:"
}
"

My goal is to extract information about findings' resources from the "Message" field and present it in a human-readable format. Specifically, I want to retrieve details from the "findings" array, specifically the "Resources" object within it.

I've attempted to use the substringBetween function in the following manner:

{{ Message.substringBetween("Resources:",",Id") }}

However, this approach didn't yield the desired outcome. I also tried using the regular expression extraction method:

{{ Message.extract(/"Resources":\s*\[(.*?)\]/) }}

I'm seeking guidance on the correct approach to achieve my goal of making the alert more human-readable and extracting information from the "Message" field's "findings" section. Any assistance or insights you can provide would be greatly appreciated.

Thank you in advance for your help and expertise.