Speed up import ~9x by using one shared Lark parser#11
Conversation
Previously, make_syntax_validator() created a new Lark parser for each rule, which was expensive (~660ms total import time). This change: - Defines ALL_START_RULES containing all grammar rules needed by validators - Creates a single shared syntax_parser with all start rules upfront - Refactors make_syntax_validator() to use functools.partial with is_valid_syntax() instead of creating new parsers Import time reduced from ~660ms to ~72ms (9x speedup). Testing protocol: - Created timing script that clears module cache and measures import time - Ran 5 iterations for each version to account for variance - Used git stash to preserve dirty state while testing HEAD - Results: HEAD averaged ~662ms, this change averages ~72ms - All existing tests pass (2/2)
There was a problem hiding this comment.
Pull request overview
This PR optimizes import time by ~9x (from ~660ms to ~72ms) by creating a single shared Lark parser instead of instantiating a new parser for each validation rule.
Key Changes:
- Introduced
ALL_START_RULEScontaining all 38 grammar rules needed by validators - Modified
syntax_parserto be instantiated once with all start rules upfront - Refactored
make_syntax_validator()to usefunctools.partialwith the shared parser instead of creating individual parsers
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
agentydragon
changed the title
|
@willynilly hi! do you think we could merge this? or would you like me to do any changes / testing? |
|
Shouldn't these changes be merged? Are there any maintainers active here? |
see #15 |
|
Hi! Since this repository is no longer actively maintained, I created and now maintain a fork of this package long-term:
Would you mind retargeting (or reopening) this pull request against the maintained fork so your fix/feature can be reviewed and merged there? Thank you! |
|
this has been solved in maintained fork of the library since version 1.3.0 |
Previously, make_syntax_validator() created a new Lark parser for each rule, which was expensive (~660ms total import time). This change:
Import time reduced from ~660ms to ~72ms (9x speedup).
Also fixed a forgotten comma that previously caused implicit string literal concatenation: