Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid repeated string join()s in analyze_macros() #750

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Blacksyke
Copy link

@Blacksyke Blacksyke commented Mar 1, 2022

Change addressing #749
When an input file has on the order of ~100,000s of strings, analyze_macros() becomes very slow. Let's change this behavior to instead store extracted strings in their own structure, and add them to the module source afterwards instead of incrementally.

Example with input file 4a87ee5ecd46a3fab735656b77d0e4fea8d3d72f3a6e0fb791999a2dfe8d59d2 (available on VirusTotal), using Python 3.9.10 on macOS:
Time performance without this patch: python3 ./oletools/olevba.py > /dev/null 929.50s user 1896.56s system 58% cpu 1:20:32.49 total
Time performance with this patch: python3 ./oletools/olevba.py > /dev/null 5.81s user 2.43s system 95% cpu 8.605 total

When an input file has on the order of ~100,000s of strings, `analyze_macros()` becomes very slow. Let's change this behavior to instead store extracted strings in their own structure, and add them to the module source afterwards instead of incrementally.

Example with input file 4a87ee5ecd46a3fab735656b77d0e4fea8d3d72f3a6e0fb791999a2dfe8d59d2 (available on VirusTotal), using Python 3.9.10 on macOS:
Time performance without this patch: `python3 ./oletools/olevba.py  > /dev/null  929.50s user 1896.56s system 58% cpu 1:20:32.49 total`
Time performance with this patch: `python3 ./oletools/olevba.py  > /dev/null  5.81s user 2.43s system 95% cpu 8.605 total`
@Blacksyke Blacksyke changed the title Change addressing #749 Avoid repeated string join()s in analyze_macros() Mar 1, 2022
@decalage2 decalage2 self-requested a review March 3, 2022 08:02
@decalage2 decalage2 self-assigned this Mar 3, 2022
@decalage2 decalage2 added this to the oletools 0.60 milestone Mar 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants