4e36622a47
Introduces Nth-frame full face detection combined with KCF bounding
box tracking and Lucas-Kanade (LK) optical flow for keypoint (KPS)
tracking on intermediate frames. This is primarily for single-face
webcam mode to improve performance while maintaining per-frame swaps.
Key Changes:
- Modified `face_swapper.py` (`process_frame`):
- Full `insightface.FaceAnalysis` runs every N frames (default 5)
or if tracking is lost.
- KCF tracker updates bounding box on intermediate frames.
- Optical flow (`cv2.calcOpticalFlowPyrLK`) tracks the 5 keypoints
from the previous frame to the current intermediate frame.
- A `Face` object is constructed with tracked bbox and KPS for
swapping on intermediate frames (detailed landmarks like
`landmark_2d_106` are None for these).
- Experimental similar logic added to `_process_live_target_v2`
for `map_faces=True` live mode (non-many_faces path).
- Robustness:
- Mouth masking and face mask creation functions in `face_swapper.py`
now handle cases where `landmark_2d_106` is `None` (e.g., by
skipping mouth mask or using bbox for face mask).
- Added division-by-zero check in `apply_color_transfer`.
- State Management:
- Introduced `reset_tracker_state()` in `face_swapper.py` to clear
all tracking-related global variables.
- `ui.py` now calls `reset_tracker_state()` at appropriate points
(webcam start, mode changes, new source image selection) to ensure
clean tracking for new sessions.
- `DETECTION_INTERVAL` in `face_swapper.py` increased to 5.
This aims to provide you with a smoother face swap experience with better FPS
by reducing the frequency of expensive full face analysis, while the
actual swap operation continues on every frame using tracked data.
|
||
|---|---|---|
| .. | ||
| processors | ||
| __init__.py | ||
| capturer.py | ||
| cluster_analysis.py | ||
| core.py | ||
| face_analyser.py | ||
| gettext.py | ||
| globals.py | ||
| hair_segmenter.py | ||
| metadata.py | ||
| predicter.py | ||
| typing.py | ||
| ui.json | ||
| ui.py | ||
| utilities.py | ||
| video_capture.py | ||