Merge 57e6020efa into dc8563372d

Update README.md to add an additional model file for the face swapper function

README.md

@@ -1,3 +1,4 @@
+
 ![demo-gif](demo.gif)
 
 
@@ -14,6 +15,7 @@ Users of this software are expected to use this software responsibly while abidi
 ### Basic: It is more likely to work on your computer but it will also be very slow. You can follow instructions for the basic install (This usually runs via **CPU**)
 #### 1.Setup your platform
 -   python (3.10 recommended)
+-   python-tk
 -   pip
 -   git
 -   [ffmpeg](https://www.youtube.com/watch?v=OlNWCpFdVMA) 
@@ -24,7 +26,7 @@ Users of this software are expected to use this software responsibly while abidi
 #### 3. Download Models
 
  1. [GFPGANv1.4](https://huggingface.co/hacksider/deep-live-cam/resolve/main/GFPGANv1.4.pth)
- 2. [inswapper_128_fp16.onnx](https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128_fp16.onnx)
+ 2. [inswapper_128_fp16.onnx](https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128_fp16.onnx) *(Note: Use this [replacement version](https://github.com/facefusion/facefusion-assets/releases/download/models/inswapper_128_fp16.onnx) if an issue occurs on your computer)*
 
 Then put those 2 files on the "**models**" folder
 
@@ -37,27 +39,25 @@ For MAC OS, You have to install or upgrade python-tk package:
 ```
 brew install python-tk@3.10
 ```
-##### DONE!!! If you dont have any GPU, You should be able to run roop using `python run.py` command. Keep in mind that while running the program for first time, it will download some models which can take time depending on your network connection.
+##### DONE!!! If you don't have any GPU, You should be able to run roop using `python run.py` command. Keep in mind that while running the program for first time, it will download some models which can take time depending on your network connection.
+
+#### 5. Proceed if you want to use GPU acceleration (optional)
+
+<details>
+<summary>Click to see the details</summary>
 
-### *Proceed if you want to use GPU Acceleration
 ### CUDA Execution Provider (Nvidia)*
 
 1.  Install  [CUDA Toolkit 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive)
-    
 2.  Install dependencies:
-    
-
 ```
 pip uninstall onnxruntime onnxruntime-gpu
 pip install onnxruntime-gpu==1.16.3
-
 ```
 
 3.  Usage in case the provider is available:
-
 ```
 python run.py --execution-provider cuda
-
 ```
 
 ### [](https://github.com/s0md3v/roop/wiki/2.-Acceleration#coreml-execution-provider-apple-silicon)CoreML Execution Provider (Apple Silicon)
@@ -67,65 +67,52 @@ python run.py --execution-provider cuda
 ```
 pip uninstall onnxruntime onnxruntime-silicon
 pip install onnxruntime-silicon==1.13.1
-
 ```
 
 2.  Usage in case the provider is available:
-
 ```
 python run.py --execution-provider coreml
-
 ```
 
 ### [](https://github.com/s0md3v/roop/wiki/2.-Acceleration#coreml-execution-provider-apple-legacy)CoreML Execution Provider (Apple Legacy)
 
 1.  Install dependencies:
-
 ```
 pip uninstall onnxruntime onnxruntime-coreml
 pip install onnxruntime-coreml==1.13.1
-
 ```
 
 2.  Usage in case the provider is available:
-
 ```
 python run.py --execution-provider coreml
-
 ```
 
 ### [](https://github.com/s0md3v/roop/wiki/2.-Acceleration#directml-execution-provider-windows)DirectML Execution Provider (Windows)
 
 1.  Install dependencies:
-
 ```
 pip uninstall onnxruntime onnxruntime-directml
 pip install onnxruntime-directml==1.15.1
-
 ```
 
 2.  Usage in case the provider is available:
-
 ```
 python run.py --execution-provider directml
-
 ```
 
 ### [](https://github.com/s0md3v/roop/wiki/2.-Acceleration#openvino-execution-provider-intel)OpenVINO™ Execution Provider (Intel)
 
 1.  Install dependencies:
-
 ```
 pip uninstall onnxruntime onnxruntime-openvino
 pip install onnxruntime-openvino==1.15.0
-
 ```
 
 2.  Usage in case the provider is available:
-
 ```
 python run.py --execution-provider openvino
 ```
+</details>
 
 ## How do I use it?
 > Note: When you run this program for the first time, it will download some models ~300MB in size.
@@ -173,6 +160,152 @@ options:
 
 Looking for a CLI mode? Using the -s/--source argument will make the run program in cli mode.
 
+### Webcam mode on Windows 11 using WSL2 Ubuntu (optional)
+
+<details>
+<summary>Click to see the details</summary>
+
+If you want to use WSL2 on Windows 11 you will notice, that Ubuntu WSL2 doesn't come with USB-Webcam support in the Kernel. You need to do two things: Compile the Kernel with the right modules integrated and forward your USB Webcam from Windows to Ubuntu with the usbipd app. Here are detailed Steps:
+
+This tutorial will guide you through the process of setting up WSL2 Ubuntu with USB webcam support, rebuilding the kernel, and preparing the environment for the Deep-Live-Cam project.  
+  
+#### 1. Install WSL2 Ubuntu  
+  
+Install WSL2 Ubuntu from the Microsoft Store or using PowerShell:  
+  
+#### 2. Enable USB Support in WSL2  
+  
+1. Install the USB/IP tool for Windows:  
+[https://learn.microsoft.com/en-us/windows/wsl/connect-usb](https://learn.microsoft.com/en-us/windows/wsl/connect-usb)  
+  
+2. In Windows PowerShell (as Administrator), connect your webcam to WSL:  
+  
+```powershell  
+usbipd list  
+usbipd bind --busid x-x # Replace x-x with your webcam's bus ID  
+usbipd attach --wsl --busid x-x # Replace x-x with your webcam's bus ID  
+```  
+ You need to redo the above every time you reboot wsl or re-connect your webcam/usb device.
+ 
+#### 3. Rebuild WSL2 Ubuntu Kernel with USB and Webcam Modules  
+  
+Follow these steps to rebuild the kernel:  
+  
+1. Start with this guide: [https://github.com/PINTO0309/wsl2_linux_kernel_usbcam_enable_conf](https://github.com/PINTO0309/wsl2_linux_kernel_usbcam_enable_conf)  
+  
+2. When you reach the `sudo wget [github.com](http://github.com/)...PINTO0309` step, which won't work for newer kernel versions, follow this video instead or alternatively follow the video tutorial from the beginning: 
+[https://www.youtube.com/watch?v=t_YnACEPmrM](https://www.youtube.com/watch?v=t_YnACEPmrM)  
+  
+Additional info: [https://askubuntu.com/questions/1413377/camera-not-working-in-cheese-in-wsl2](https://askubuntu.com/questions/1413377/camera-not-working-in-cheese-in-wsl2)  
+  
+3. After rebuilding, restart WSL with the new kernel. 
+  
+#### 4. Set Up Deep-Live-Cam Project  
+ Within Ubuntu:
+1. Clone the repository:  
+  
+```bash  
+git clone [https://github.com/hacksider/Deep-Live-Cam](https://github.com/hacksider/Deep-Live-Cam)  
+```  
+  
+2. Follow the installation instructions in the repository, including cuda toolkit 11.8, make 100% sure it's not cuda toolkit 12.x.  
+   
+#### 5. Verify and Load Kernel Modules  
+  
+1. Check if USB and webcam modules are built into the kernel:  
+  
+```bash  
+zcat /proc/config.gz | grep -i "CONFIG_USB_VIDEO_CLASS"  
+```  
+  
+2. If modules are loadable (m), not built-in (y), check if the file exists:  
+  
+```bash  
+ls /lib/modules/$(uname -r)/kernel/drivers/media/usb/uvc/  
+```  
+  
+3. Load the module and check for errors (optional if built-in):  
+  
+```bash  
+sudo modprobe uvcvideo  
+dmesg | tail  
+```  
+  
+4. Verify video devices:  
+  
+```bash  
+sudo ls -al /dev/video*  
+```  
+  
+#### 6. Set Up Permissions  
+  
+1. Add user to video group and set permissions:  
+  
+```bash  
+sudo usermod -a -G video $USER  
+sudo chgrp video /dev/video0 /dev/video1  
+sudo chmod 660 /dev/video0 /dev/video1  
+```  
+  
+2. Create a udev rule for permanent permissions:  
+  
+```bash  
+sudo nano /etc/udev/rules.d/81-webcam.rules  
+```  
+  
+Add this content:  
+  
+```  
+KERNEL=="video[0-9]*", GROUP="video", MODE="0660"  
+```  
+  
+3. Reload udev rules:  
+  
+```bash  
+sudo udevadm control --reload-rules && sudo udevadm trigger  
+```  
+  
+4. Log out and log back into your WSL session.  
+  
+5. Start Deep-Live-Cam with `python run.py --execution-provider cuda --max-memory 8` where 8 can be changed to the number of GB VRAM of your GPU has, minus 1-2GB. If you have a RTX3080 with 10GB I suggest adding 8GB. Leave some left for Windows.
+
+#### Final Notes  
+  
+- Steps 6 and 7 may be optional if the modules are built into the kernel and permissions are already set correctly.  
+- Always ensure you're using compatible versions of CUDA, ONNX, and other dependencies.  
+- If issues persist, consider checking the Deep-Live-Cam project's specific requirements and troubleshooting steps.  
+  
+By following these steps, you should have a WSL2 Ubuntu environment with USB webcam support ready for the Deep-Live-Cam project. If you encounter any issues, refer back to the specific error messages and troubleshooting steps provided.
+
+#### Troubleshooting CUDA Issues  
+
+If you encounter this error:  
+  
+```  
+[ONNXRuntimeError] : 1 : FAIL : Failed to load library [libonnxruntime_providers_cuda.so](http://libonnxruntime_providers_cuda.so/) with error: libcufft.so.10: cannot open shared object file: No such file or directory  
+```  
+  
+Follow these steps:  
+  
+1. Install CUDA Toolkit 11.8 (ONNX 1.16.3 requires CUDA 11.x, not 12.x):  
+[https://developer.nvidia.com/cuda-11-8-0-download-archive](https://developer.nvidia.com/cuda-11-8-0-download-archive)  
+  select: Linux, x86_64, WSL-Ubuntu, 2.0, deb (local)
+2. Check CUDA version:  
+  
+```bash  
+/usr/local/cuda/bin/nvcc --version  
+```  
+  
+3. If the wrong version is installed, remove it completely:  
+[https://askubuntu.com/questions/530043/removing-nvidia-cuda-toolkit-and-installing-new-one](https://askubuntu.com/questions/530043/removing-nvidia-cuda-toolkit-and-installing-new-one)  
+  
+4. Install CUDA Toolkit 11.8 again [https://developer.nvidia.com/cuda-11-8-0-download-archive](https://developer.nvidia.com/cuda-11-8-0-download-archive), select: Linux, x86_64, WSL-Ubuntu, 2.0, deb (local)
+  
+```bash  
+sudo apt-get -y install cuda-toolkit-11-8  
+```
+</details>
+
 ## Want the Next Update Now?
 If you want the latest and greatest build, or want to see some new great features, go to our [experimental branch](https://github.com/hacksider/Deep-Live-Cam/tree/experimental) and experience what the contributors have given.
 

modules/capturer.py

@@ -1,16 +1,28 @@
 from typing import Any
 import cv2
+import modules.globals  # Import the globals to check the color correction toggle
 
 
 def get_video_frame(video_path: str, frame_number: int = 0) -> Any:
     capture = cv2.VideoCapture(video_path)
+
+    # Set MJPEG format to ensure correct color space handling
+    capture.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*'MJPG'))
+    
+    # Only force RGB conversion if color correction is enabled
+    if modules.globals.color_correction:
+        capture.set(cv2.CAP_PROP_CONVERT_RGB, 1)
+    
     frame_total = capture.get(cv2.CAP_PROP_FRAME_COUNT)
     capture.set(cv2.CAP_PROP_POS_FRAMES, min(frame_total, frame_number - 1))
     has_frame, frame = capture.read()
+
+    if has_frame and modules.globals.color_correction:
+        # Convert the frame color if necessary
+        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+
     capture.release()
-    if has_frame:
-        return frame
-    return None
+    return frame if has_frame else None
 
 
 def get_video_frame_total(video_path: str) -> int:

modules/globals.py

@@ -17,6 +17,7 @@ keep_fps = None
 keep_audio = None
 keep_frames = None
 many_faces = None
+color_correction = None  # New global variable for color correction toggle
 nsfw_filter = None
 video_encoder = None
 video_quality = None

modules/predicter.py

@@ -1,6 +1,8 @@
 import numpy
 import opennsfw2
 from PIL import Image
+import cv2  # Add OpenCV import
+import modules.globals  # Import globals to access the color correction toggle
 
 from modules.typing import Frame
 
@@ -10,10 +12,16 @@ MAX_PROBABILITY = 0.85
 model = None
 
 def predict_frame(target_frame: Frame) -> bool:
+    # Convert the frame to RGB before processing if color correction is enabled
+    if modules.globals.color_correction:
+        target_frame = cv2.cvtColor(target_frame, cv2.COLOR_BGR2RGB)
+        
     image = Image.fromarray(target_frame)
     image = opennsfw2.preprocess_image(image, opennsfw2.Preprocessing.YAHOO)
     global model
-    if model is None: model = opennsfw2.make_open_nsfw_model()
+    if model is None: 
+        model = opennsfw2.make_open_nsfw_model()
+        
     views = numpy.expand_dims(image, axis=0)
     _, probability = model.predict(views)[0]
     return probability > MAX_PROBABILITY

modules/processors/frame/face_swapper.py

@@ -49,6 +49,10 @@ def swap_face(source_face: Face, target_face: Face, temp_frame: Frame) -> Frame:
 
 
 def process_frame(source_face: Face, temp_frame: Frame) -> Frame:
+    # Ensure the frame is in RGB format if color correction is enabled
+    if modules.globals.color_correction:
+        temp_frame = cv2.cvtColor(temp_frame, cv2.COLOR_BGR2RGB)
+        
     if modules.globals.many_faces:
         many_faces = get_many_faces(temp_frame)
         if many_faces:

modules/ui.py

@@ -93,6 +93,11 @@ def create_root(start: Callable[[], None], destroy: Callable[[], None]) -> ctk.C
     many_faces_switch = ctk.CTkSwitch(root, text='Many faces', variable=many_faces_value, cursor='hand2', command=lambda: setattr(modules.globals, 'many_faces', many_faces_value.get()))
     many_faces_switch.place(relx=0.6, rely=0.65)
 
+    # Add color correction toggle button
+    color_correction_value = ctk.BooleanVar(value=modules.globals.color_correction)
+    color_correction_switch = ctk.CTkSwitch(root, text='Fix Blueish Cam\n(force cv2 to use RGB instead of BGR)', variable=color_correction_value, cursor='hand2', command=lambda: setattr(modules.globals, 'color_correction', color_correction_value.get()))
+    color_correction_switch.place(relx=0.6, rely=0.70)
+
 #    nsfw_value = ctk.BooleanVar(value=modules.globals.nsfw_filter)
 #    nsfw_switch = ctk.CTkSwitch(root, text='NSFW filter', variable=nsfw_value, cursor='hand2', command=lambda: setattr(modules.globals, 'nsfw_filter', nsfw_value.get()))
 #    nsfw_switch.place(relx=0.6, rely=0.7)