[BUG] BD000701/BD000704/BD000300/BD000804 miss strings reconstructed at runtime (base64/XOR/concatenation) #540

Case Study

Malicious literals written directly (detected):

A = "auth_token=abc123"
B = "~/.ssh/id_rsa"
C = "https://tcp.ngrok.io:4443/upload"
D = "0123456789abcdef0123456789abcdef01234567"
print(A, B, C, D)

Detected as 4 findings:

• BD000300 Security.Backdoor.ReverseShell (Moderate) for "tcp.ngrok.io"
• BD000701 Security.Backdoor.DataExFiltration.Token (Important) twice ("auth_token", ".ssh")
• BD000804 Security.Backdoor.Obfuscation.LongStrings (Moderate) for the 44-char token

Semantically identical program where the same strings are rebuilt at runtime (not detected):

def _sk_dec(_x):
    _k = b'M3z!\x9cX.f'
    return bytes(_c ^ _k[_i % len(_k)] for _i, _c in enumerate(_x)).decode('utf-8')
A = _sk_dec([44, 70, 14, 73, 195, 44, 65, 13, 40, 93, 71, 64, 254, 59, 31, 84, 126])
B = _sk_dec([51, 28, 84, 82, 239, 48, 1, 15, 41, 108, 8, 82, 253])
C = _sk_dec([37, 71, 14, 81, 239, 98, 1, 73, 57, 80, 10, 15, 242, 63, 92, 9, 38, 29, 19, 78, 166, 108, 26, 82, 126, 28, 15, 81, 240, 55, 79, 2])
D = _sk_dec([125, 2, 72, 18, 168, 109, 24, 81, 117, 10, 27, 67, 255, 60, 75, ...])
print(A, B, C, D)

Result: "0 matches found."

adv.py is generated from ori.py with the SKILLCLOAK structural-obfuscation operator "encryption" (XOR-cipher every string literal); the same class of evasion is produced by "encoding" (base64 literals) and "string-sampling" (split literals such as "auth_" "token").

Current rules

File: Resources/BackdoorRules/data_exfiltration.json

BD000701  regex: (npm owner|password|htpasswd|auth_?token|secret_?key|private_?key|authorized_keys?|npmrc|\.ssh|usersecrets?|api_?keys|nuget\.config|\.identityservice)
          scopes: code, comment; confidence: high

File: Resources/BackdoorRules/reverse_shell.json

BD000300  regex: (shell\.now\.sh|tcp\.ngrok\.io|reverse shell|reverse-shell\.sh)
          scopes: code, comment

File: Resources/BackdoorRules/obfuscation.json

BD000804  regex: ["'][a-z0-9]{40,}["']
          scope: code

The same limitation applies to other literal-content rules, e.g. BD000704 (string "hostname"/".hostname") and BD000500 (regex pastebin|zerobin|ghostbin|hastebin|klgrth.io).

All of these are pure text matches performed by ApplicationInspector on the raw source bytes. base64, XOR integer lists, adjacent-literal concatenation and f-string constant joining only reconstruct the original value at runtime, so the sequence the rule keys on never appears in the scanned file. ApplicationInspector does not fold constants or evaluate decoding expressions before matching.

Suggested Fix

Add a normalization pass before rule matching that folds constant string expressions and decodes common literal encodings into a scan buffer:

• concatenate adjacent string literals and joined f-string constants;
• decode base64/hex string literals and XOR integer lists (bytes([...]), chr() sequences) when the decoder is local and constant;
• run the existing rules on the folded value as well as on the raw text.
  Additionally, treat decoder primitives themselves (base64.b64decode, bytes(list), chr-join, XOR loops) as obfuscation indicators, so a file that hides every literal is still flagged. Please add regression tests for base64, XOR-array and split-literal variants.

Reproduction results

Environment: OSSGadget 0.2.38 (ApplicationInspector rules engine, .NET 8), Windows 11 (10.0.26200).

Commands (from this directory):

ossgadget detect-backdoor -f text ./run/ori
ossgadget detect-backdoor -f text ./run/adv

Results:

ori: 4 matches (BD000300, BD000701 x2, BD000804).
adv: 0 matches.

Corpus evidence: on a 453-file malicious corpus, applying the same operator reduced BD000701 196 -> 95 (encryption) and 196 -> 142 (string-sampling), BD000704 283 -> 161 / 162, BD000804 21 -> 0, BD000300 43 -> 36.

Attachments: ori.py, adv.py

adv.py
ori.py