Update HolisticTraceAnalysis dependency to the traceinsight package#217
Update HolisticTraceAnalysis dependency to the traceinsight package#217tejadhith wants to merge 1 commit into
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
jinsun-yoo
left a comment
jinsun-yoo
left a comment
There was a problem hiding this comment.
Thank you for the work. As a sanity check, coud you run the MLA conversion once more using the old version dependencies (networkx before 3.2.0, etc.), so that the generated Chakra graph stays the same before & after this change?
| git submodule update --init | ||
| pip install -r requirements.txt | ||
| pip install -e . | ||
| pip install traceinsight |
There was a problem hiding this comment.
Since this is now fixed to be a one-liner and is now represented in the requirements list in pyproject, perhaps we can remove this 'run item' altogether
| - name: Install HTA | ||
| run: | | ||
| git clone https://github.com/facebookresearch/HolisticTraceAnalysis.git | ||
| cd HolisticTraceAnalysis | ||
| git checkout d731cc2e2249976c97129d409a83bd53d93051f6 | ||
| git submodule update --init | ||
| pip install -r requirements.txt | ||
| pip install -e . | ||
| pip install traceinsight |
|
|
||
| ### Step 4: Install Holistic Trace Analysis | ||
| Installing Holistic Trace Analysis is necessary for Trace link. | ||
| Installing Holistic Trace Analysis (published on PyPI as `traceinsight`) is necessary for Trace link. |
There was a problem hiding this comment.
Let's clarify: "published on PyPI as traceinsight - the old PyPi name HolisticTraceAnalysis is no longer valid"
| @@ -36,15 +36,10 @@ $ pip install . | |||
| ``` | |||
|
|
|||
| ### Step 4: Install Holistic Trace Analysis | |||
There was a problem hiding this comment.
Actually, similar to above comments, just remove this?
| "networkx", | ||
| "pydot", | ||
| "HolisticTraceAnalysis @ git+https://github.com/facebookresearch/HolisticTraceAnalysis.git@d731cc2e2249976c97129d409a83bd53d93051f6" | ||
| "traceinsight" |
There was a problem hiding this comment.
Let's add a comment here clarifying this is HTA
| @@ -19,7 +19,7 @@ dependencies = [ | |||
| "graphviz", | |||
| "networkx", | |||
2 participants
Summary
Chakra pins HolisticTraceAnalysis (HTA) to git commit
d731cc2einpyproject.toml, in both CI workflows (python_tests,end_to_end_tests), and inUSER_GUIDE.md. That mid-2024 checkpoint predates HTA's pandas-2.x compatibility fix (HTA PR #333), so runningchakra_trace_linkon a real trace logsCritical path analysis encountered an invalid graph structure: Cannot specify both 'axis' and 'index'/'columns'and silently skips critical-path analysis.HTA now publishes its releases to PyPI under the renamed distribution
traceinsight(0.6.0 / 0.6.1); the importable module remainshta, so no source changes are needed. This PR replaces the pinned git checkout with thetraceinsightpackage acrosspyproject.toml, both CI workflows, andUSER_GUIDE.md— which also removes the manual clone/submodule/editable-install step from CI.Test Plan
Environment: Python 3.13, torch 2.9.0+cu128, 2× V100;
traceinsight 0.6.1, PARAMet_replay,networkx 3.6.1.python -m pytest tests→ 117 passed (no regression).chakra_trace_link→Cannot specify both 'axis' and 'index'/'columns', critical path skipped.calculating critical path took 0.0027s,Linking process successful;chakra_converter→ valid.et(115COMP_NODE)..et= 154COMP_NODE+ 1METADATA_NODE+ 2COMM_COLL_NODE(ncclDevKernel_AllReduce_…); critical path runs. No regression on comm-heavy traces.import htastill resolves (module name unchanged in traceinsight 0.6.x).Additional Notes
HolisticTraceAnalysis→traceinsightat 0.6.x; the importable module stayshta.traceinsightonly exists at 0.6.0+, so a bare requirement cannot resolve below the fix.networkxchange: theDiGraph.__new__crash was specific to networkx 3.6.0 (networkx #8367) and is fixed in 3.6.1; nothing forces 3.6.0.mlc-chakraPyPI release is needed to propagate this topip install mlc-chakrausers.