unofficial implementation of DCT-Net: Domain-Calibrated Translation for Portrait

Overview

DCT-NET.Pytorch

unofficial implementation of DCT-Net: Domain-Calibrated Translation for Portrait Stylization.
you can find official version here

show

img video

environment

you can build your environment follow this
pip install tensorboardX for show

how to run

train

download pretrain weights

cd utils
bash download_weight.sh

follow rosinality/stylegan2-pytorch and put 550000.pt in pretrain_models

CCN

  1. prepare the style pictures and align them
    the image path is like this
    style-photos/
    |-- 000000.png
    |-- 000006.png
    |-- 000010.png
    |-- 000011.png
    |-- 000015.png
    |-- 000028.png
    |-- 000039.png

  2. change your own path in ccn_config

  3. train ccn

    # single gpu
    python  train.py \
    --model ccn \
    --batch_size 16 \
    --checkpoint_path checkpoint \
    --lr 0.002 \
    --print_interval 100 \
    --save_interval 100 --dist 
     # multi gpu
    python -m torch.distributed.launch train.py \
    --model ccn \
    --batch_size 16 \
    --checkpoint_path checkpoint \
    --lr 0.002 \
    --print_interval 100 \
    --save_interval 100 

almost 1000 steps, you can stop

TTN

  1. prepare expression information
    you can follow LVT to estimate facial landmark
    cd utils
    python get_face_expression.py \
    --img_base '' # your real image path base,like ffhq \
    --pool_num 2 # multiprocess number \
    --LVT '' # the LVT path you put \
    --train  # train data or val data
  2. prepare your generator image
    cd utils
    python get_tcc_input.py \
    --model_path '' # ccn model path \
    --output_path '' # save path
    select almost 5k~1w good image manually
  3. change your own path in ttn_config
    # like
    self.train_src_root = '/StyleTransform/DATA/ffhq-2w/img'
    self.train_tgt_root = '/StyleTransform/DATA/select-style-gan'
    self.val_src_root = '/StyleTransform/DATA/dmloghq-1k/img'
    self.val_tgt_root = '/StyleTransform/DATA/select-style-gan'
  4. train tnn
    # like ccn single and multi gpus
    python  train.py \
    --model ttn \
    --batch_size 64 \
    --checkpoint_path checkpoint \
    --lr 2e-4 \
    --print_interval 100 \
    --save_interval 100 \
    --dist

inference

you can follow inference.py to put your own ttn model path and image path
python inference.py

Credits

SEAN model and implementation:
https://github.com/ZPdesu/SEAN Copyright © 2020, ZPdesu.
License https://github.com/ZPdesu/SEAN/blob/master/LICENSE.md

stylegan2-pytorch model and implementation:
https://github.com/rosinality/stylegan2-pytorch Copyright © 2019, rosinality.
License https://github.com/rosinality/stylegan2-pytorch/blob/master/LICENSE

White-box-Cartoonization model and implementation:
https://github.com/SystemErrorWang/White-box-Cartoonization Copyright © 2020, SystemErrorWang.

White-box-Cartoonization model pytorch model and implementation:
https://github.com/vinesmsuic/White-box-Cartoonization-PyTorch Copyright © 2022, vinesmsuic.
License https://github.com/vinesmsuic/White-box-Cartoonization-PyTorch/blob/main/LICENSE

arcface pytorch model pytorch model and implementation:
https://github.com/ronghuaiyang/arcface-pytorch Copyright © 2018, ronghuaiyang.

You might also like...

This is an unofficial implementation of BOAT: Bilateral Local Attention Vision Transformer

BOAT: Bilateral Local Attention Vision Transformer This is an unofficial implementation of the paper BOAT: Bilateral Local Attention Vision Transforme

Sep 9, 2022

This is an unofficial PyTorch implementation of Meta Pseudo Labels.

Meta Pseudo Labels This is an unofficial PyTorch implementation of Meta Pseudo Labels. The official Tensorflow implementation is here. Results CIFAR-1

Mar 31, 2022

Unofficial PyTorch Implementation of OpenAI's GPT-3

gpt3-pytorch Unofficial PyTorch Implementation of OpenAI's GPT-3 (discount edition) Foreword I'm definitely aware I'm late to the party but here's my

Nov 11, 2022

Unofficial PyTorch Implementation for pNLP-Mixer: an Efficient all-MLP Architecture for Language (https://arxiv.org/abs/2202.04350)

Unofficial PyTorch Implementation for pNLP-Mixer: an Efficient all-MLP Architecture for Language (https://arxiv.org/abs/2202.04350)

pNLP-Mixer - Unofficial PyTorch Implementation pNLP-Mixer: an Efficient all-MLP Architecture for Language Implementation of pNLP-Mixer in PyTorch and

Nov 3, 2022

This is an unofficial PyTorch implementation of EdgeViT in "EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers", arXiv 2022.

This is an unofficial PyTorch implementation of EdgeViT in "EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers", arXiv 2

Oct 28, 2022

Unofficial implementation of CURL: Neural Curve Layers for Global Image Enhancement paper.

CURL Unofficial pytorch implementation of CURL: Neural Curve Layers for Global Image Enhancement paper. I used many parts of the official repository t

Aug 27, 2022

Unofficial implementation of "KeepAugment: A Simple Information-Preserving Data Augmentation Approach". CVPR2021

 Unofficial implementation of

KeepAugment_Pytorch Unofficial implementation of "KeepAugment: A Simple Information-Preserving Data Augmentation Approach". CVPR2021 Results CIFAR 10

Nov 18, 2022

Unofficial pytorch implementation of BigVGAN: A Universal Neural Vocoder with Large-Scale Training

Unofficial pytorch implementation of BigVGAN: A Universal Neural Vocoder with Large-Scale Training

BigVGAN: A Universal Neural Vocoder with Large-Scale Training In this repository, I try to implement BigVGAN (specifically BigVGAN-base model) [Paper]

Nov 11, 2022

Unofficial implementation of QaNER: Prompting Question Answering Models for Few-shot Named Entity Recognition.

QaNER Unofficial implementation of QaNER: Prompting Question Answering Models for Few-shot Named Entity Recognition. You can adopt this pipeline for a

Nov 14, 2022
Comments
  • Inference does not work correctly

    Inference does not work correctly

    Hi @LeslieZhoa, thanks for your implementation. I tried running the inference.py script and I faced some problems.

    I fixed the 'netG' key in the line: https://github.com/LeslieZhoa/DCT-NET.Pytorch/blob/ebdc7bd5c7cff9a2dedcf2dd8f3630aa0ec303ce/inference.py#L25 to 'G' because final.pth does not contain 'netG'.

    Also in the postprocess method you make the slice [..., ::-1], and you also make it after calling the run method: https://github.com/LeslieZhoa/DCT-NET.Pytorch/blob/ebdc7bd5c7cff9a2dedcf2dd8f3630aa0ec303ce/inference.py#L55

    I removed the extra slice. But I still get an unrealistic image (ran at 1024 and 256 resolutions). I'm attaching an example (first line is 1024, second line is 256) and waiting for a response/comment. dct_net

    opened by darkasevgen 4
  • 关于ccn的blend和eometry expansion module

    关于ccn的blend和eometry expansion module

    十分感谢你的贡献,但我有几个问题想要咨询一下。 我发现你在训练ccn的时候没有做论文中提到的blend操作,Geometry expansion module我也没有找到,这是有什么原因吗? 另外论文作者提到他用相同的模型对头部和背景进行了两次推理,只是对头部模型进行了额外的优化,但在你复现中似乎只有一个模型并且只做了一次推理,是有什么改动吗? 另外show中展示的效果似乎不是很好,有更好的效果图可以分享一下吗?谢谢

    opened by Cococyh 0
  • Training error

    Training error

    Hi @LeslieZhoa ! How can you compare the dictionary and the float? https://github.com/LeslieZhoa/DCT-NET.Pytorch/blob/b303f416c2caf05114fd38f10b0570df903b4462/trainer/ModelTrainer.py#L124

    This method is called here: https://github.com/LeslieZhoa/DCT-NET.Pytorch/blob/b303f416c2caf05114fd38f10b0570df903b4462/trainer/ModelTrainer.py#L96

    But the val_loss obtained in CCNTrainer.evalution is an empty dictionary: https://github.com/LeslieZhoa/DCT-NET.Pytorch/blob/b303f416c2caf05114fd38f10b0570df903b4462/trainer/CCNTrainer.py#L196-L206

    I want to understand how to fix this and what metrics to count? And as I understand it, you trained the network without early_stopping. How many epochs/iterations did it take?

    opened by darkasevgen 1
  • Training data size

    Training data size

    What is the image size used for training CCN, Why can't the matrix be multiplied when I input 1024 in training, and The problem of ' 'Nan or Inf found in input tensor '' when inputting 256 during training

    opened by shiminxuan003 3
Owner
LeslieZhao
a DL lover
LeslieZhao
Official Pytorch implementation of "Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis", ECCV 2022

Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis Project page | Paper "Injecting 3D Perception of

Jeong-gi Kwak 84 Nov 26, 2022
The official repo for the paper "Rethinking Portrait Matting with Privacy Preserving". For further questions, please contact Sihan Ma at [email protected] or Jizhizi Li at [email protected]

Rethinking Portrait Matting with Privacy Preserving This is the official repository of the paper Rethinking Portrait Matting with Privacy Preserving.

null 176 Nov 12, 2022
Is it a perfect portrait!

Propportrait pip install -r requirements.txt git clone https://github.com/vijishmadhavan/Proppotrait python lightdetect.py --image "test/1233.jpg" pyt

Vijish Madhavan 4 Sep 9, 2022
Repository for the paper : ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning

ME-D2N_for_CDFSL Repository for the paper : ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning (to appear in ACM M

Fu Yuqian 15 Nov 18, 2022
Code and reuslts accompanying the NeurIPS 2022 paper with the title SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG

SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG This repository contains code and data accompanyi

null 4 Nov 17, 2022
Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch

Memorizing Transformers - Pytorch Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memori

Phil Wang 345 Nov 21, 2022
Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

?? Flamingo - Pytorch Implementation of Flamingo, state-of-the-art few-shot visual question answering attention net, in Pytorch. It will include the p

Phil Wang 570 Nov 21, 2022
A pytorch implementation of PRS-Net

PRS-Net Introduction A pytorch implementation of PRS-Net, a research work on TVCG by Lin Gao et al. Official link: http://geometrylearning.com/prs-net

null 1 Jun 30, 2022
Implementation of Transframer, Deepmind's U-net + Transformer architecture for up to 30 seconds video generation, in Pytorch

Transframer - Pytorch (wip) Implementation of Transframer, Deepmind's U-net + Transformer architecture for up to 30 seconds video generation, in Pytor

Phil Wang 50 Nov 22, 2022
Official PyTorch implementation of TriHorn-Net

TriHorn-Net This repository contains the PyTorch implementation of TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimation. It contains

Mohammad Rezaei 46 Nov 13, 2022