Some useful scripts for my low-level projects

Overview

Low-Level-Tools

LICENSE Python

There are some useful tools for my low-level vision projects, thanks for OpenUnity.

Dependencies

  • Python 3 (Recommend to use Anaconda)
  • PyTorch
  • numpy: conda install numpy
  • matplotlib: conda install matplotlib
  • opencv: conda install opencv
  • pandas: conda install pandas

Easy to use

Here are some simple demos. If you want to learn more about the usage of these tools, you can refer to the source code for more optional parameters.

1. Calculating Metrics ( PSNR, SSIM, etc.)

1.1 PSNR/SSIM
  • Following the demo for batch operation:
# Images
from utils.image_metric_utils import batch_calc_image_PSNR_SSIM
root_list = [
    {
        'output': '/path/to/output images 1',
        'gt': '/path/to/gt images 1'
    },
    {
        'output': '/path/to/output images 2',
        'gt': '/path/to/gt images 2'
    },
]
batch_calc_image_PSNR_SSIM(root_list)

# Videos
from utils.video_metric_utils import batch_calc_video_PSNR_SSIM
root_list = [
    {
        'output': '/path/to/output videos 1',
        'gt': '/path/to/gt videos 1'
    },
    {
        'output': '/path/to/output videos 2',
        'gt': '/path/to/gt videos 2'
    },
]
batch_calc_video_PSNR_SSIM(root_list)
1.2 LPIPS
  • Following the demo for batch operation:
# Images
from utils.image_metric_utils import batch_calc_image_LPIPS
root_list = [
    {
        'output': '/path/to/output images 1',
        'gt': '/path/to/gt images 1'
    },
    {
        'output': '/path/to/output images 2',
        'gt': '/path/to/gt images 2'
    },
]
batch_calc_image_LPIPS(root_list)

# Videos
from utils.video_metric_utils import batch_calc_video_LPIPS
root_list = [
    {
        'output': '/path/to/output videos 1',
        'gt': '/path/to/gt videos 1'
    },
    {
        'output': '/path/to/output videos 2',
        'gt': '/path/to/gt videos 2'
    },
]
batch_calc_video_LPIPS(root_list)
1.3 NIQE
  • Following the demo for batch operation:
# Images
from utils.image_metric_utils import batch_calc_image_NIQE
root_list = [
    {
        'output': '/path/to/output images 1',
        'gt': '/path/to/gt images 1'
    },
    {
        'output': '/path/to/output images 2',
        'gt': '/path/to/gt images 2'
    },
]
batch_calc_image_NIQE(root_list)

# Videos
from utils.video_metric_utils import batch_calc_video_NIQE
root_list = [
    {
        'output': '/path/to/output videos 1',
        'gt': '/path/to/gt videos 1'
    },
    {
        'output': '/path/to/output videos 2',
        'gt': '/path/to/gt videos 2'
    },
]
batch_calc_video_NIQE(root_list)
1.4 Kernel Gradient Similarity
  • Following the demo for batch operation:
# Images: video_type=False
# Videos: video_type=True
from utils.kernel_metric_utils import batch_calc_kernel_metric
root_list = [
    {
        'output': '/path/to/output kernel 1',
        'gt': '/path/to/gt kernel 1'
    },
    {
        'output': '/path/to/output kernel 2',
        'gt': '/path/to/gt kernel 2'
    },
]
batch_calc_kernel_metric(root_list, video_type=False)

2. Image / Video Processing ( resize, crop, shift, etc.)

2.1 Image/Video Resize
  • We use fatheral's python implementation of matLab imresize() function fatheral/matlab_imresize.
  • Following the demo for batch operation:
# Images
from utils.image_utils import matlab_resize_images

ori_root = '/path/to/ori images'
dest_root = '/path/to/dest images'
matlab_resize_images(ori_root, dest_root, scale=2.0)

# Videos
from utils.video_utils import matlab_resize_videos

ori_root = '/path/to/ori videos'
dest_root = '/path/to/dest videos'
matlab_resize_videos(ori_root, dest_root, scale=2.0)
  • We also apply opencv for resizing.
  • Following the demo for batch operation:
from utils.image_utils import cv2_resize_images

ori_root = '/path/to/ori images'
dest_root = '/path/to/dest images'
cv2_resize_images(ori_root, dest_root, scale=2.0)

# Videos
from utils.video_utils import cv2_resize_videos

ori_root = '/path/to/ori videos'
dest_root = '/path/to/dest videos'
cv2_resize_videos(ori_root, dest_root, scale=2.0)
2.2 Crop and combine images
  • When you need to infer large image, you can crop image to many patches with padding by following the demo:
# Notice: 
#   filenames should not contain the character "-"
#   the crop flag "x-x-x-x" will be at the end of filename when cropping
#   the combine operation will use the crop flag "x-x-x-x"
from utils.image_crop_combine_utils import *
ori_root = '/path/to/ori images'
dest_root = '/path/to/dest images'
batch_crop_img_with_padding(ori_root, dest_root, min_size=(800, 800), padding=100)
  • When you finish inferring large image with cropped patches, you can combine patches to image by following the demo:
# Notice: 
#   filenames should not contain the character "-" except for the crop flag
#   the crop flag "x-x-x-x" should be at the end of filename when combining
from utils.image_crop_combine_utils import *
ori_root = '/path/to/ori images'
dest_root = '/path/to/dest images'
batch_combine_img(ori_root, dest_root, padding=100)
  • You can traversal crop image to many patches with same interval by following the demo:
from utils.image_crop_combine_utils import *
ori_root = '/path/to/ori images'
dest_root = '/path/to/dest images'
batch_traverse_crop_img(ori_root, dest_root, dsize=(800, 800), interval=400)
  • You can select valid patch that are not too smooth by following the demo:
from utils.image_crop_combine_utils import *
ori_root = '/path/to/ori images'
dest_root = '/path/to/dest images'
batch_select_valid_patch(ori_root, dest_root)
2.3 Image/Video Shift
  • We use "Bilinear" interpolation method to shift images/videos for sub-pixels.
  • Following the demo for batch operation:
# Images
from utils.image_utils import shift_images

ori_root = '/path/to/ori images'
dest_root = '/path/to/dest images'
shift_images(ori_root, dest_root, offset_x=0.5, offset_y=0.5)

# Videos
from utils.video_utils import shift_videos

ori_root = '/path/to/ori videos'
dest_root = '/path/to/dest videos'
shift_videos(ori_root, dest_root, offset_x=0.5, offset_y=0.5)

3. Deep Model Properties ( Params, Flops, etc. )

  • You can only calculate model Params by following the demo:
from utils.dnn_utils import cal_parmeters
network = None  # Please define the model
cal_parmeters(network)
  • You can also calculate more properties of model: Params, Memory, MAdd, Flops, etc.
  • We use Swall0w's tools Swall0w/torchstat.
  • [Swall0w/torchstat] can not use cuda, so we modified it for using cuda. If you want to calculate Flops on cuda, please using the following command to install torchstat.
cd ./utils/torchstat
python3 setup.py install
  • If you do not want to using cuda, please using the following command to install torchstat.
pip install torchstat  # pytorch >= 1.0.0
pip install torchstat==0.0.6  # pytorch = 0.4.1
  • And then you can calculate properties by following the demo:
from torchstat import stat
network = None  # Please define the model
input_size = (3, 80, 80)  # the size of input (channel, height, width)
stat(network, input_size)

4. File Processing ( csv, etc. )

4.1 csv file
  • You can read a csv file by following the demo:
from base import file_io_base
data, col_names, row_names = file_io_base.read_csv('filename.csv', col_name_ind=0, row_name_ind=0)
  • You can write a numpy.array into a csv file by following the demo:
from base import file_io_base
import numpy as np
row_names = ['r1', 'r2', 'r3']
col_names = ['c1', 'c2', 'c3', 'c4']
data_array = np.array([[1, 2, 3, 4],
                       [2, 3, 4, 5],
                       [3, 4, 5, 6]])
file_io_base.write_csv('filename.csv', data_array, col_names, row_names)

5. Visualize Tools ( plot, optical-flow, etc. )

5.1 Plot multiple curves in one figure
  • You plot multiple curves in one figure by following the demo:
import numpy as np
from utils import plot_utils
array_list = [
    np.array([1, 4, 5, 3, 6]),
    np.array([2, 3, 7, 4, 5]),
]
label_list = ['curve-1', 'curve-2']
plot_utils.plot_multi_curve(array_list, label_list)
5.2 Visualize optical flow
  • You can visualize optical flow by following the demo:
from utils import flow_utils
flow = None  # this is flow, shape=(h, w, 2)
rgb_image = flow_utils.visual_flow(flow)
You might also like...

Python programming projects and scripts for beginners.

Tutorial Python programming projects and scripts for beginners. Bank Project : Simple bank project. You need to enter username and password . If the

May 11, 2022

Random Python projects comprising of Games, Graphics, Data Science, OpenCV and random scripts

Python Random Python projects comprising of Data Science, Games, Graphics, OpenCV and random scripts Repository Tree view: ├───Data Science │ └───Mo

Apr 28, 2022

DataAnalysis: Some data analysis projects in charles_pikachu

DataAnalysis: Some data analysis projects in charles_pikachu

DataAnalysis DataAnalysis: Some data analysis projects in charles_pikachu You can star this repository to keep track of the project if it's helpful fo

Aug 12, 2022

Some of the projects I've created on my Python journey

Python Bootcamp Projects! This Repository contains some of the projects I created while taking an online Python Bootcamp course. The format of the cou

Apr 20, 2022

These are some fun and small projects one can make while using the Python Language.

Beginners-Projects These are some fun and small projects one can make while using the Python Language. A Random Number Guesser game Rock Paper Scissor

Jul 17, 2022

🏆This repository contains some simple mini python projects.

30DaysOfPython 🏆 This repository contains some simple mini python projects. Projects The following projects are found in this repository: Dictionary

Jul 24, 2022

A very low word tester ,but it will be more powerful when I growing up!

CaptainWordTester About: A very low word tester ,but it will be more powerful when I growing up! Something to say: In fact,there may be some problems

Aug 6, 2022

Official code of the paper "Expanding Low-Density Latent Regions for Open-Set Object Detection" (CVPR 2022)

Official code of the paper

OpenDet Expanding Low-Density Latent Regions for Open-Set Object Detection (CVPR2022) Jiaming Han, Yuqiang Ren, Jian Ding, Xingjia Pan, Ke Yan, Gui-So

Sep 14, 2022

Low Bandwidth Distributed Denial-of-service attack ToolKit.

Low Bandwidth Distributed Denial-of-service attack ToolKit.

Warlof Low Bandwidth Distributed Denial-of-service attack ToolKit. Warlof is a Low Bandwidth Distributed Denial-of-service attack (DDoS) Toolkit. You

May 3, 2022
Owner
House_Lee
Ph.D. Candidate of IMAG, NJUST. Image Super-Resolution / Denoising
House_Lee
This is a full ddos scripts set by finix dev a.k.a ZxC some scripts are stolen from stressing websites so you'll need this ❤📖

Many Scripts Will Be Added Every Day ?? Better use a vps to attack ?? finixv1 - a simple script to down TCP network brrr - a script to down home netwo

null 1 Sep 11, 2022
An easy-handle pipeline project for low level vision tasks (Image2Image) by pytorch

LowLevelVision-Pipeline-pytorch This is an easy-handle pipeline project for common AI tasks by pytorch. Currently, this project is mainly about low le

darcy 3 May 19, 2022
A "library" of useful python scripts, to be executed by a HTTP request to the raw URL

Library Revision for 0.2.0 List of contents: Brief Introduction Instructions Legal (MIT license) Brief Introduction A library of useful python scripts

Ivan 1 Apr 12, 2022
Telegram Group Calls Streaming bot with some useful features, written in Python with Pyrogram and Py-Tgcalls.

Telegram Group Calls Streaming bot with some useful features, written in Python with Pyrogram and Py-Tgcalls. Supporting platforms like Youtube, Spotify, Resso, AppleMusic, Soundcloud and M3u8 Links.

Cute Inspire 11 Aug 10, 2022
Some useful Spider. EverPhoto check in, ZhiHu Article crawling.

Spider “zhihu.com” Article scraping --- 知乎盐选文章爬取 EverPhoto Check in --- 时光相册签到脚本 Hobby A collection about hobby. I'm crawl this from bilibili - LKS in

Shem 1 Aug 17, 2022
Some useful codes for real time EEG streaming from the Muse (Interaxon)

brainflow-useful-codes Some useful codes for real time EEG streaming from the Muse (Interaxon) Files Basics Real-time streaming of all channels Real-t

Sruthi Kuriakose 1 Sep 13, 2022
Storage for some useful connector for R-Vision SOAR

Сборник коннекторов для R-Vision SOAR nad_connector.py Коннектор для обогащения инцидентов, порождаемых системой класса NTA - PT Network Attack Discov

Serega 1 Sep 15, 2022
A project opening CLI tool for god-level programmers. Yes, using it upgrades you to god-level programmer status.

I hate this routine: open terminal cd to project location activate virtual env ( if included) enter command to open in IDE Run server/vms I hate it be

bichwaa 2 Sep 24, 2022
These projects are created for learning purpose ,its open source but copying and pasting projects is not learning. So,learn and create your own.

Python-Projects-for-learning These projects are created for learning purpose ,its open source but copying and pasting projects is not learning. So,lea

null 1 May 22, 2022
Italia Opensource is a list of open source projects created by Italian companies or developers. The repository intends to give visibility to open source projects and stimulate the community to contribute to growing the ecosystem.

Italia Opensource Italia Opensource is a list of open source projects created by Italian companies or developers. The repository intends to give visib

Italia Opensource 136 Sep 28, 2022