Skip to content

Git history indexing fails with UnicodeDecodeError on non-UTF-8 Windows locales (e.g. Korean cp949) #140

Description

@cangma

What happened?

Git history indexing fails with a UnicodeDecodeError on Windows machines using a non-UTF-8 system locale (e.g. Korean, cp949). Code chunk indexing itself completes fine — only the git-history sub-feature fails.

Root cause: in context_engine/indexer/git_indexer.py, both subprocess.run calls in index_commits use text=True without an explicit encoding=:

meta_result = await asyncio.to_thread(
    subprocess.run,
    ["git", "log", range_arg, "--format=%H%n%an%n%ai%n%s%n%b%x00"],
    cwd=project_dir, capture_output=True, text=True, check=False,
)

Without encoding=, Python decodes subprocess output using locale.getpreferredencoding(False). On Windows this is the system ANSI codepage — on a Korean system that's cp949 (confirmed via python -c "import locale; print(locale.getpreferredencoding(False))").

git log output is UTF-8 by default. Any non-ASCII commit message/author in UTF-8 that isn't a valid cp949 byte sequence raises UnicodeDecodeError, aborting index_commits — which surfaces upstream as 'NoneType' object has no attribute 'split'.

What did you expect?

Git history indexing should succeed regardless of the host OS's default locale, since git's own log output encoding doesn't depend on it.

Suggested fix: pass encoding="utf-8", errors="replace" explicitly on both subprocess.run calls in index_commits, matching git's actual output encoding regardless of the host OS locale.

Steps to reproduce

  1. On Windows with a non-UTF-8 default system locale (Korean/Japanese/Chinese/etc. DBCS codepage)
  2. cce init in a git repo that has at least one non-ASCII (UTF-8) commit message or author name
  3. Git history indexing fails; code indexing completes but the history sub-feature errors out

Relevant logs or error output

Git history indexing failed: 'NoneType' object has no attribute 'split'
UnicodeDecodeError: 'cp949' codec can't decode byte ...

Python version

3.13

OS

Windows 11, Korean system locale (cp949)

CCE version

0.4.25

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions