feat: Add image and video download support to rn-wechat-extract - #2
Open
runningZ1 wants to merge 6 commits into
Open
feat: Add image and video download support to rn-wechat-extract#2runningZ1 wants to merge 6 commits into
runningZ1 wants to merge 6 commits into
Conversation
Added fetch_wechat_with_media.py - downloads both images and videos.
Features:
- Downloads images from WeChat articles (same as with_images)
- Downloads videos from mpvideo.qpic.cn CDN
- Saves videos to {title}_videos/ directory
- Handles authentication parameters for video URLs
- Updates Markdown with video references
Tested with article containing 6 images and 4 videos.
Successfully downloaded 6/6 images and 4/8 videos (4 require auth).
Technical details:
- Videos hosted on mpvideo.qpic.cn
- URLs extracted from JavaScript with escaped characters
- Some videos require authentication tokens (403 on expired tokens)
Enhanced fetch_wechat_with_media.py with video channel detection: - Detects wxv_ video channel IDs - Reports video channel IDs in output - Downloads videos from video channels (when auth tokens are valid) Tested with article containing: - 17 images (all downloaded) - 8 videos (downloaded) - 2 video channel IDs detected: - wxv_4602687508227964933 - wxv_4602682391613407239 Technical details: - Video channels use wxv_ prefix for IDs - Videos hosted on mpvideo.qpic.cn - Auth tokens required (some may return 403)
Enhanced fetch_wechat_with_media.py with finder video detection: - Detects finder usernames (username@finder) - Detects finder video URLs (findermp.video.qq.com) - Reports expired URLs (finder url expire!) Tested with article: https://mp.weixin.qq.com/s/ELTM4oSlQkfqbJfbpQLHkQ Results: - 7/7 images downloaded - 2 video channel usernames detected - 2 finder video URLs detected (expired) Technical details: - Video channels use findermp.video.qq.com domain - URLs have expiration tokens - Expired URLs return 400 Bad Request with 'finder url expire!'
Added fetch_wechat_with_channels.py - enhanced version with video channel support. Features: - Detects video channel share links (weixin.qq.com/sph/xxx) - Integrates with online parsing service: https://sph.litao.workers.dev/ - Detects finder usernames and export IDs - Reports expired finder URLs Based on ltaoo/wx_channels_download project: - GitHub: https://github.com/ltaoo/wx_channels_download - Online service: https://sph.litao.workers.dev/ Usage: - Visit https://sph.litao.workers.dev/ - Paste video channel share link - Get direct video download URL Note: Direct download from HTML requires real-time parsing as video URLs have expiration tokens.
WeChat Channels videos embedded in an article have no directly playable URL, only an export_id/object_nonce_id pair in the <mp-common-videosnap> tag. Resolve them through TikHub (fetch_video_detail -> real object_id, fetch_video_share_url -> share link) and a community worker (sph.litao.workers.dev) that returns a CDN url carrying an X-snsvideoflag param — WeChat serves that url unencrypted (X-encflag: 0), so no ISAAC64 decrypt step is needed. Also fixes a prior bug where the script treated the embed's data-nonceid as the TikHub object_id; it is actually object_nonce_id and does not resolve on its own. Verified end-to-end against four live articles (720p/1094s landscape, two 576x1024 shorts, one 720x1280 clip); all downloaded as valid, playable h264+aac mp4 confirmed with ffprobe. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Explicitly call out that the 视频号 download pipeline relies on two services this project does not build or operate: TikHub (paid API, requires the user's own key) and sph.litao.workers.dev (a free, already-deployed public instance of the open-source project ltaoo/wx_channels_download, run by a third party). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds enhanced versions of
fetch_wechat.pythat download images, native videos, and WeChat Channels (视频号) videos embedded in WeChat public account articles.New Scripts
fetch_wechat_with_images.py
{title}_images/directoryfetch_wechat_with_media.py
mpvideo.qpic.cn) videosfetch_wechat_with_channels.py 🆕
Video Channel (视频号) Video Download
A 视频号 video embedded in an article only exposes an
export_id(data-id="export/...") andobject_nonce_id(data-nonceid) pair inside a<mp-common-videosnap>tag — neither is a usable download URL on its own, andobject_nonce_idis not the TikHubobject_id(an earlier version of this PR mixed these up).Resolution pipeline (no manual copy-paste needed anymore):
export_id+object_nonce_idfrom the article HTML (or use a directweixin.qq.com/sph/...share link if the article already has one).fetch_video_detail(export_id + object_nonce_id) → resolves the real numericobject_id.fetch_video_share_url(object_id) →weixin.qq.com/sph/xxxshare link.sph.litao.workers.devfetch_video_profile(share_url) → an h264/h265 CDN url carrying anX-snsvideoflagquery param.Step 4's
X-snsvideoflagparam turns out to make WeChat's CDN return the plaintext, pre-transcoded stream (X-encflag: 0in the response headers) instead of the ISAAC64-encrypted original that TikHub's ownmedia.full_urlpoints to — so this pipeline needs no video decryption step at all, despite 视频号 videos normally requiring WeChat's proprietary WASM-based decryptor to play.Requires a TikHub API key (
--tikhub-keyorTIKHUB_API_KEYenv var). Steps 2–3 cost $0.01 each ($0.02/video total); step 4 is free but is a third-party community service outside this project's control. Without a key, 视频号 videos are still detected and reported (export_id/object_nonce_id printed to stderr) but not downloaded — everything else still works.Usage
Testing Results
Verified end-to-end (real network calls, real TikHub key) against 4 live articles, downloaded files checked with
ffprobefor valid duration/codec/no corruption:4/4 视频号 videos resolved and downloaded successfully; all played back correctly and matched their expected duration.
Changes
scripts/fetch_wechat_with_images.pyscripts/fetch_wechat_with_media.pyscripts/fetch_wechat_with_channels.py: replaced the old speculativewxv_/bare-finder/bare-nonceiddetection with<mp-common-videosnap>-basedexport_id+object_nonce_idextraction, and a working TikHub + community-worker resolution and download pipelineSKILL.mdwith full documentation for all three enhanced versions, including the 视频号 resolution pipeline and its TikHub costTechnical Details
Native videos (
mpvideo.qpic.cn):视频号 videos:
data-nonceidin the article HTML isobject_nonce_id, notobject_id— do not pass it directly to TikHub'sfetch_video_share_urlsph.litao.workers.devworker sit behind Cloudflare and reject Python's defaulturllibUser-Agent (403); the script sends a browser UALimitations
sph.litao.workers.devis a third-party community deployment, not an official API — availability isn't guaranteed