fix(extensions): force UTF-8 worker stdio to prevent deadlock on non-UTF-8 Windows locales#216
Closed
Lorchie wants to merge 1 commit into
Closed
fix(extensions): force UTF-8 worker stdio to prevent deadlock on non-UTF-8 Windows locales#216Lorchie wants to merge 1 commit into
Lorchie wants to merge 1 commit into
Conversation
…UTF-8 Windows locales The extension worker was launched with text=True but no explicit encoding, so pipes were decoded with the system locale (cp932 on Japanese Windows). The worker emits UTF-8, so a UnicodeDecodeError killed the stderr reader thread, the stderr buffer filled up, and the worker blocked inside tqdm - a silent, permanent hang around 80% progress with 0% CPU. - _build_env(): set PYTHONUTF8=1 so the worker emits UTF-8 regardless of locale - Popen/subprocess.run: use encoding="utf-8", errors="replace" so reader threads can never die on undecodable bytes Affects all non-UTF-8 Windows locales (Japan, China, Korea, ...). Fixes #214
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contexte
Corrige #214 — la génération 3D se bloque indéfiniment sur Windows en locale non-UTF-8 (japonais/cp932, chinois, coréen). La barre de progression s'arrête vers 80 %, le worker consomme 0 % CPU et ne se termine jamais.
Cause racine
Dans
api/services/extension_process.py, le worker était lancé avecsubprocess.Popen(..., text=True)sansencoding=explicite. Les pipes étaient donc décodés avec l'encodage système (cp932 au Japon). Le worker émet de l'UTF-8 →UnicodeDecodeErrordès qu'un octet non-cp932 arrive → le thread lecteur_stderr_loopmeurt → le buffer stderr se remplit → le worker se bloque danstqdm. Blocage silencieux et permanent.Correction
_build_env(): ajout dePYTHONUTF8=1pour que le worker émette de l'UTF-8 indépendamment de la locale.Popen(démarrage worker) etsubprocess.run(install de module) :encoding="utf-8", errors="replace"en remplacement detext=True.errors="replace"garantit que les threads lecteurs (_read_loop/_stderr_loop) ne peuvent plus mourir sur un octet indécodable.Impact
Tous les utilisateurs Windows en locale non-UTF-8. Aucun changement de comportement sur les locales UTF-8. Suite de tests verte (pre-push).