Conversation
monrog2
left a comment
monrog2
left a comment
There was a problem hiding this comment.
The new GX validation has correctness and check-registration issues that can produce false PASS/FAIL results or attempt SSH in API-only mode. Please address the inline findings before merge.
| try: | ||
| cmd = r"vsh -c 'show sprom cpu-info' | grep 'HW Changes Bits'" | ||
| c.cmd(cmd) | ||
| if "HW Changes Bits : 0x3" in c.output: |
There was a problem hiding this comment.
Blocking: This exact substring check treats every other output as safe, including empty/malformed output and a command timeout (Connection.cmd() can return timeout without raising). That can report PASS without ever reading a valid HW Changes Bits value. Please parse and validate the returned value, report ERROR when it is missing/unparseable or the command did not return prompt, and add empty, malformed, and timeout tests.
| if ( | ||
| cversion.older_than("4.2(3a)") or cversion.newer_than("4.2(5a)") | ||
| ) and tversion.newer_than("4.2(5l)"): | ||
| result = NA |
There was a problem hiding this comment.
Blocking: Setting result = NA here does not stop the check. The function continues into SSH collection, so an out-of-scope upgrade can be changed to FAIL_UF when 0x3 is found or ERROR when SSH fails. Please return the NA Result immediately when the version combination is not applicable, and cover the boundaries with affected GX fixtures (the current NA tests use the non-GX fixture and therefore cannot catch this).
| standby_sup_sync_check, | ||
| isis_database_byte_check, | ||
| configpush_shard_check, | ||
| gx_hw_changes_bit_check, |
There was a problem hiding this comment.
High: This check opens SSH connections but is registered in api_checks. With --api-only, query_common_data() intentionally leaves username/password as None, yet this check will still attempt SSH to every GX switch. Please move it to ssh_checks and add a CheckManager API-only registration test.
| @@ -222,6 +223,7 @@ Items | Defect | This Script | |||
| [d26]: #stale-pconsra-object | |||
There was a problem hiding this comment.
Medium: This fragment does not match the generated Markdown anchor for ### GX HW Changes Bit, which is #gx-hw-changes-bit; the link will not navigate to the new section. Please correct this reference and the matching runtime doc_url in gx_hw_changes_bit_check (which currently uses the same underscore fragment).
| result = PASS | ||
| headers = ["Switch ID", "Switch Name", "HW Changes Bits value"] | ||
| data = [] | ||
| recommended_action = 'Review The Bug RNE and apply the workaround before Upgrade' |
There was a problem hiding this comment.
Medium: Review The Bug RNE is internal terminology and does not give users an actionable, customer-safe remediation. Please point users to the authoritative CSCvv04251 guidance or instruct them to contact Cisco TAC before proceeding, rather than referring to an RNE they may not be able to access.
2 participants
issue #299 fixed
Lab Output (logic modified):
[Check 84/90] GX HW_Changes_Bit... FAIL - UPGRADE FAILURE!!
Switch ID Switch Name HW Changes Bits value
101 b-leaf1 0x3 Found!
103 b-leaf3 0x3 Found!
201 b-spine1 0x3 Found!
Pytest results:
((ACI-PRE) ) jeestrad@JEESTRAD-M-L6VV ACI-Pre-Upgrade-Validation-Script % python3.12 -m pytest tests/checks/gx_hw_changes_bit_check
=================================================================================================== test session starts ===================================================================================================
platform darwin -- Python 3.12.12, pytest-7.4.0, pluggy-1.2.0
rootdir: /Users/jeestrad/Documents/Python/pre-upgrade/ACI-Pre-Upgrade-Validation-Script
configfile: pytest.ini
collected 7 items
tests/checks/gx_hw_changes_bit_check/test_gx_hw_changes_bit_check.py::test_logic[icurl_outputs0-fabric_nodes0-4.2(3b)-4.2(4p)-False-conn_cmds0-N/A-expected_data0]
----------------------------------------------------------------------------------------------------- live log setup ------------------------------------------------------------------------------------------------------
[12:16:28.680 INFO init_system:6107(MainThread)] Cleaning up previous run files in preupgrade_validator_logs/
[12:16:28.681 INFO init_system:6109(MainThread)] Creating directories preupgrade_validator_logs/ and preupgrade_validator_logs/json_results/
------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------
[12:16:28.682 INFO init_result:1182(MainThread)] Initialized in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
[12:16:28.682 INFO _start_thread:1105(MainThread)] (gx_hw_changes_bit_check) Starting thread.
[12:16:28.682 INFO wrapper:1349(gx_hw_changes_bit_check)] Start gx_hw_changes_bit_check
[12:16:28.683 INFO update_result:1191(gx_hw_changes_bit_check)] Finalized result in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
PASSED [ 14%]
tests/checks/gx_hw_changes_bit_check/test_gx_hw_changes_bit_check.py::test_logic[icurl_outputs1-fabric_nodes1-4.2(5l)-6.0(1h)-False-conn_cmds1-N/A-expected_data1]
------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------
[12:16:28.696 INFO init_result:1182(MainThread)] Initialized in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
[12:16:28.697 INFO _start_thread:1105(MainThread)] (gx_hw_changes_bit_check) Starting thread.
[12:16:28.697 INFO wrapper:1349(gx_hw_changes_bit_check)] Start gx_hw_changes_bit_check
[12:16:28.697 INFO update_result:1191(gx_hw_changes_bit_check)] Finalized result in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
PASSED [ 28%]
tests/checks/gx_hw_changes_bit_check/test_gx_hw_changes_bit_check.py::test_logic[icurl_outputs2-fabric_nodes2-4.2(1f)-5.2(7f)-False-conn_cmds2-N/A-expected_data2]
------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------
[12:16:28.711 INFO init_result:1182(MainThread)] Initialized in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
[12:16:28.711 INFO _start_thread:1105(MainThread)] (gx_hw_changes_bit_check) Starting thread.
[12:16:28.711 INFO wrapper:1349(gx_hw_changes_bit_check)] Start gx_hw_changes_bit_check--------------------| 0/1 checks completed
[12:16:28.712 INFO update_result:1191(gx_hw_changes_bit_check)] Finalized result in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
PASSED [ 42%]
tests/checks/gx_hw_changes_bit_check/test_gx_hw_changes_bit_check.py::test_logic[icurl_outputs3-fabric_nodes3-4.2(3b)-4.2(4b)-True-conn_cmds3-ERROR !!-expected_data3]
------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------
[12:16:28.723 INFO init_result:1182(MainThread)] Initialized in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
[12:16:28.723 INFO _start_thread:1105(MainThread)] (gx_hw_changes_bit_check) Starting thread.
[12:16:28.723 INFO wrapper:1349(gx_hw_changes_bit_check)] Start gx_hw_changes_bit_check
[12:16:28.723 INFO update_result:1191(gx_hw_changes_bit_check)] Finalized result in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
PASSED [ 57%]
tests/checks/gx_hw_changes_bit_check/test_gx_hw_changes_bit_check.py::test_logic[icurl_outputs4-fabric_nodes4-4.2(3b)-4.2(4p)-False-conn_cmds4-ERROR !!-expected_data4]
------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------
[12:16:28.737 INFO init_result:1182(MainThread)] Initialized in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
[12:16:28.737 INFO _start_thread:1105(MainThread)] (gx_hw_changes_bit_check) Starting thread.
[12:16:28.738 INFO wrapper:1349(gx_hw_changes_bit_check)] Start gx_hw_changes_bit_check--------------------| 0/1 checks completed
[12:16:28.738 INFO update_result:1191(gx_hw_changes_bit_check)] Finalized result in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
PASSED [ 71%]
tests/checks/gx_hw_changes_bit_check/test_gx_hw_changes_bit_check.py::test_logic[icurl_outputs5-fabric_nodes5-4.2(3b)-4.2(4c)-False-conn_cmds5-PASS-expected_data5]
------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------
[12:16:28.751 INFO init_result:1182(MainThread)] Initialized in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
[12:16:28.752 INFO _start_thread:1105(MainThread)] (gx_hw_changes_bit_check) Starting thread.--------------| 0/1 checks completed
[12:16:28.752 INFO wrapper:1349(gx_hw_changes_bit_check)] Start gx_hw_changes_bit_check
[12:16:28.752 INFO update_result:1191(gx_hw_changes_bit_check)] Finalized result in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
PASSED [ 85%]
tests/checks/gx_hw_changes_bit_check/test_gx_hw_changes_bit_check.py::test_logic[icurl_outputs6-fabric_nodes6-4.2(3b)-4.2(4p)-False-conn_cmds6-FAIL - UPGRADE FAILURE!!-expected_data6]
------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------
[12:16:28.766 INFO init_result:1182(MainThread)] Initialized in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
[12:16:28.766 INFO _start_thread:1105(MainThread)] (gx_hw_changes_bit_check) Starting thread.--------------| 0/1 checks completed
[12:16:28.766 INFO wrapper:1349(gx_hw_changes_bit_check)] Start gx_hw_changes_bit_check
[12:16:28.767 INFO update_result:1191(gx_hw_changes_bit_check)] Finalized result in preupgrade_validator_logs/json_results/gx_hw_changes_bit_check.json
PASSED [100%]
==================================================================================================== 7 passed in 0.10s ====================================================================================================